Integrate Prometheus with the Dragonfly Operator

This guide provides step-by-step instructions to set up Prometheus with the Dragonfly Operator. In this guide, you’ll use the Promethues Operator and PodMonitor to manage Prometheus resources.

Prerequisites

Install Prometheus Operator

First make sure you have all the Prometheus CRDs installed in your cluster. Run the below commands to install the Prometheus Operator.

  1. LATEST=$(curl -s https://api.github.com/repos/prometheus-operator/prometheus-operator/releases/latest | jq -cr .tag_name)
  2. curl -sL https://github.com/prometheus-operator/prometheus-operator/releases/download/${LATEST}/bundle.yaml | kubectl create -f -

Create the PodMonitor Resource

PodMonitors allow Prometheus to monitor specific pods that has the target labels. It is the easiest way to monitor Dragonfly pods. You can either create your own PodMonitor config file or use our sample podMonitor.yaml file to create a PodMonitor object.

  1. kubectl apply -f https://github.com/dragonflydb/dragonfly-operator/blob/main/monitoring/podMonitor.yaml

Note you must use app: <dragonfly-name> as a selector label in the PodMonitor to target the correct Dragonfly instances. The label value is the name of your Dragonfly resource (in this case, dragonfly-sample).

The Dragonfly pod exposes a port named admin which you can use as the endpoint in PodMonitor.

Create Promethues Resources

Now you have installed the operator and PodMonitor, it is time to create the prometheus resources. If you have RBAC enabled, create the necessary serviceaccount, clusterrole and clusterrolebinding resources first.

  1. $ kubectl apply -f https://github.com/dragonflydb/dragonfly-operator/blob/main/monitoring/promServiceAccount.yaml
  2. $ kubectl apply -f https://github.com/dragonflydb/dragonfly-operator/blob/main/monitoring/promClusterRole.yaml
  3. $ kubectl apply -f https://github.com/dragonflydb/dragonfly-operator/blob/main/monitoring/promClusterBinding.yaml

This will allow Prometheus to scrape data from Dragonfly resources. Run the below command to create the Prometheus object. It will create a pod named prometheus-prometheus-0.

  1. $ kubectl apply -f https://github.com/dragonflydb/dragonfly-operator/blob/main/monitoring/prometheus-config.yaml

You can also create a service that points to the Prometheus pod.

  1. $ kubectl apply -f https://github.com/dragonflydb/dragonfly-operator/blob/main/monitoring/prometheus-service.yaml

Run kubectl get all to check if all the resources have successfully been created.

  1. $ kubectl get all
  2. NAME READY STATUS RESTARTS AGE
  3. pod/dragonfly-sample-0 1/1 Running 4 (3h50m ago) 12d
  4. pod/dragonfly-sample-1 1/1 Running 4 (3h50m ago) 12d
  5. pod/prometheus-operator-744c6bb8f9-vnxw4 1/1 Running 16 (3h49m ago) 19d
  6. pod/prometheus-prometheus-0 2/2 Running 6 (3h50m ago) 11d
  7. NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
  8. service/dragonfly-sample ClusterIP 10.96.2.149 <none> 6379/TCP 12d
  9. service/kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 45d
  10. service/prometheus-operated ClusterIP None <none> 9090/TCP 11d
  11. service/prometheus-operator ClusterIP None <none> 8080/TCP 19d
  12. service/prometheus-svc ClusterIP 10.96.8.22 <none> 9090/TCP 11d
  13. NAME READY UP-TO-DATE AVAILABLE AGE
  14. deployment.apps/prometheus-operator 1/1 1 1 19d
  15. NAME DESIRED CURRENT READY AGE
  16. replicaset.apps/prometheus-operator-744c6bb8f9 1 1 1 19d
  17. NAME READY AGE
  18. statefulset.apps/dragonfly-sample 2/2 12d
  19. statefulset.apps/prometheus-prometheus 1/1 11d

Access Prometheus UI

Prometheus has a beautiful UI that you can use to query certain metrics. Use the port-forward command to either directly expose the Prometheus pod (port 9090) or expose the Prometheus service.

  1. kubectl port-forward prometheus-prometheus-0 9090:9090

Now go to localhost:9090. Congratulations! You just integrated Prometheus with Dragonfly!