Metrics
Argo CD exposes different sets of Prometheus metrics per server.
Application Controller Metrics
Metrics about applications. Scraped at the argocd-metrics:8082/metrics
endpoint.
Metric | Type | Description |
---|---|---|
argocd_app_info | gauge | Information about Applications. It contains labels such as sync_status and health_status that reflect the application state in ArgoCD. |
argocd_app_k8s_request_total | counter | Number of kubernetes requests executed during application reconciliation |
argocd_app_labels | gauge | Argo Application labels converted to Prometheus labels. Disabled by default. See section below about how to enable it. |
argocd_app_reconcile | histogram | Application reconciliation performance. |
argocd_app_sync_total | counter | Counter for application sync history |
argocd_cluster_api_resource_objects | gauge | Number of k8s resource objects in the cache. |
argocd_cluster_api_resources | gauge | Number of monitored kubernetes API resources. |
argocd_cluster_cache_age_seconds | gauge | Cluster cache age in seconds. |
argocd_cluster_connection_status | gauge | The k8s cluster current connection status. |
argocd_cluster_events_total | counter | Number of processes k8s resource events. |
argocd_cluster_info | gauge | Information about cluster. |
argocd_kubectl_exec_pending | gauge | Number of pending kubectl executions |
argocd_kubectl_exec_total | counter | Number of kubectl executions |
argocd_redis_request_duration | histogram | Redis requests duration. |
argocd_redis_request_total | counter | Number of redis requests executed during application reconciliation |
If you use ArgoCD with many application and project creation and deletion, the metrics page will keep in cache your application and project’s history. If you are having issues because of a large number of metrics cardinality due to deleted resources, you can schedule a metrics reset to clean the history with an application controller flag. Example: --metrics-cache-expiration="24h0m0s"
.
Exposing Application labels as Prometheus metrics
There are use-cases where ArgoCD Applications contain labels that are desired to be exposed as Prometheus metrics. Some examples are:
- Having the team name as a label to allow routing alerts to specific receivers
- Creating dashboards broken down by business units
As the Application labels are specific to each company, this feature is disabled by default. To enable it, add the --metrics-application-labels
flag to the ArgoCD application controller.
The example below will expose the ArgoCD Application labels team-name
and business-unit
to Prometheus:
containers:
- command:
- argocd-application-controller
- --metrics-application-labels
- team-name
- --metrics-application-labels
- business-unit
In this case, the metric would look like:
# TYPE argocd_app_labels gauge
argocd_app_labels{label_business_unit="bu-id-1",label_team_name="my-team",name="my-app-1",namespace="argocd",project="important-project"} 1
argocd_app_labels{label_business_unit="bu-id-1",label_team_name="my-team",name="my-app-2",namespace="argocd",project="important-project"} 1
argocd_app_labels{label_business_unit="bu-id-2",label_team_name="another-team",name="my-app-3",namespace="argocd",project="important-project"} 1
API Server Metrics
Metrics about API Server API request and response activity (request totals, response codes, etc…). Scraped at the argocd-server-metrics:8083/metrics
endpoint.
Metric | Type | Description |
---|---|---|
argocd_redis_request_duration | histogram | Redis requests duration. |
argocd_redis_request_total | counter | Number of kubernetes requests executed during application |
reconciliation. | ||
grpc_server_handled_total | counter | Total number of RPCs completed on the server, regardless of success or failure. |
grpc_server_msg_sent_total | counter | Total number of gRPC stream messages sent by the server. |
## Repo Server Metrics | ||
Metrics about the Repo Server. | ||
Scraped at the argocd-repo-server:8084/metrics endpoint. |
Metric | Type | Description |
---|---|---|
argocd_git_request_duration_seconds | histogram | Git requests duration seconds. |
argocd_git_request_total | counter | Number of git requests performed by repo server |
argocd_redis_request_duration_seconds | histogram | Redis requests duration seconds. |
argocd_redis_request_total | counter | Number of kubernetes requests executed during application reconciliation. |
argocd_repo_pending_request_total | gauge | Number of pending requests requiring repository lock |
Prometheus Operator
If using Prometheus Operator, the following ServiceMonitor example manifests can be used. Change metadata.labels.release
to the name of label selected by your Prometheus.
apiVersion: monitoring.coreos.com/v1
kind: ServiceMonitor
metadata:
name: argocd-metrics
labels:
release: prometheus-operator
spec:
selector:
matchLabels:
app.kubernetes.io/name: argocd-metrics
endpoints:
- port: metrics
apiVersion: monitoring.coreos.com/v1
kind: ServiceMonitor
metadata:
name: argocd-server-metrics
labels:
release: prometheus-operator
spec:
selector:
matchLabels:
app.kubernetes.io/name: argocd-server-metrics
endpoints:
- port: metrics
apiVersion: monitoring.coreos.com/v1
kind: ServiceMonitor
metadata:
name: argocd-repo-server-metrics
labels:
release: prometheus-operator
spec:
selector:
matchLabels:
app.kubernetes.io/name: argocd-repo-server
endpoints:
- port: metrics
apiVersion: monitoring.coreos.com/v1
kind: ServiceMonitor
metadata:
name: argocd-applicationset-controller-metrics
labels:
release: prometheus-operator
spec:
selector:
matchLabels:
app.kubernetes.io/name: argocd-applicationset-controller
endpoints:
- port: metrics
Dashboards
You can find an example Grafana dashboard here or check demo instance dashboard.