Skip to content

Grafana plugin

Grafana is a dashboarding and graphing application, initially built for time series data, but has evolved into a tool that can pull data from multiple sources and visualize it in a single dashboard for logs and traces too. It is a popular tool for monitoring and observability.

You DO NOT need grafana for visualizing logs and metrics data from OpenObserve by maintaining one more app. OpenObserve has a very advanced GUI for that, which you can use for logs, metrics, traces, front end monitoring, dashboards and alerting. You could however use grafana if you are already using it for some of your needs e.g. monitoring metrics using prometheus and are comfortable using it and don't want to change. In this case you can use OpenObserve grafana plugin to visualize logs and metrics data from OpenObserve in your grafana dashboard. You can also use grafana explore tool to query adhoc logs and metrics data from OpenObserve. You could also use OpenObserve as a prometheus source in grafana and use grafana to visualize metrics data from OpenObserve.

Below are the steps to install and configure the plugin. We will provide guide on how to set up grafana in a kubernetes environment. You can adapt these steps to set up grafana in a non-kubernetes environment.

If you are already familiar with grafana plugin installation, you can download the plugin from here and get started or follow the below steps.

Install Grafana

Grafana requires a persistent store to store its data and configuration. While configuration can be stored in a configmap or secret, data needs to be stored in a database. Grafana supports sqlite, mysql and postgres. Most installations I have seen in the wild use a single node grafana installation using sqlite. I have also seen that many of these use a kubernetes deployment .

If you are using a single node grafana installation using sqlite then you should use statefulset instead of deployment so you do not lose your data when the pod restarts. If you are using mysql/postgres then you can use deployment as the data is stored in the database.

You would also need a grafana.ini config file to configure grafana. You can use the below minimalistic working grafana.ini file to start. You can add more configuration as needed.

Configuration

grafana.ini
1
2
3
4
5
6
7
8
9
[date_formats]
default_timezone = UTC
[server]
root_url = https://grafana.yourdomain.com

[plugins]
enable_alpha = true
app_tls_skip_verify_insecure = false
allow_loading_unsigned_plugins = zinclabs_openobserve

Line 4 should be updated with the root url of your grafana installation. This is the url that you will use to access grafana. e.g. https://grafana.yourdomain.com

Line 9 is the one that is important where we specify that grafana should use the unsigned plugin zinclabs_openobserve. This is the plugin that we will install using the init container in the statefulset.

Once you have created the file, you can create a kubernetes secret using the below command.

1
kubectl create secret generic grafana-config --from-file=grafana.ini

Deployment

Now let's install grafana.

grafana_statefulset.yaml
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: grafana
spec:
  serviceName: "grafana"
  replicas: 1
  selector:
    matchLabels:
      app: grafana
  template:
    metadata:
      labels:
        app: grafana
    spec:
      securityContext:
        fsGroup: 2000
        runAsUser: 10000
        runAsGroup: 3000
      initContainers:
        - name: openobserve-plugin-loader
          image: wbitt/network-multitool
          imagePullPolicy: IfNotPresent
          command:
            [
              "sh",
              "-c",
              "curl -o /var/lib/grafana/plugins/zo_gp.tar.gz https://zincsearch-releases.s3.us-west-2.amazonaws.com/zo_gp/zo_gp.tar.gz && cd /var/lib/grafana/plugins &&  tar -zxvf zo_gp.tar.gz",
            ]
          volumeMounts:
            - name: grafana-base
              mountPath: /var/lib/grafana
            - name: grafana-plugins
              mountPath: /var/lib/grafana/plugins
      containers:
        - name: grafana
          image: grafana/grafana:latest
          ports:
            - containerPort: 3000
              name: grafana
          volumeMounts:
            - name: grafana-base
              mountPath: /var/lib/grafana
            - name: grafana-plugins
              mountPath: /var/lib/grafana/plugins
            - name: grafana-config
              mountPath: /etc/grafana
      volumes:
        - name: grafana-base
          persistentVolumeClaim:
            claimName: grafana-base
        - name: grafana-plugins
          persistentVolumeClaim:
            claimName: grafana-plugins
        - name: grafana-config
          secret:
            defaultMode: 420
            secretName: grafana-config
  volumeClaimTemplates:
    - metadata:
        name: grafana-base
      spec:
        accessModes: ["ReadWriteOnce"]
        resources:
          requests:
            storage: 1Gi
    - metadata:
        name: grafana-plugins
      spec:
        accessModes: ["ReadWriteOnce"]
        resources:
          requests:
            storage: 1Gi
---
apiVersion: v1
kind: Service
metadata:
  name: grafana
spec:
  ports:
    - port: 3000
      targetPort: 3000
  selector:
    app: grafana

We are using an init container in this case to download and configure openobserve plugin for the grafana container.

Now let's install the statefulset using the below command.

1
kubectl apply -f grafana_statefulset.yaml

Using Grafana plugin

Set up data source

Once grafana starts you could go to the data sources section and search for OpenObserve

Next let's add OpenObserve data source server details.

You need to do following:

  1. Update URL
  2. Enable Basic Auth
  3. Provide user id and password for basic auth details.

Once you have updated the above, click on Save and Test button. If everything is correct you should see a success message.

Explore logs

Now let's explore some logs. Click on Explore menu item on the left and select OpenObserve as the data source.

Select appropriate: 1. organization 1. stream 1. time range

and click on Run Query button. You should see the logs from the stream.

You should now be able to see the results.

If you want to explore metrics from OpenObserve in OpenObserve you can setup grafana as a standard prometheus data source and explore metrics. You do not need the plugin for that as grafana supports prometheus data source out of the box.