Defining recording rules

Configuring rules

Prometheus supports two types of rules which may be configured and thenevaluated at regular intervals: recording rules and alertingrules. To include rules in Prometheus, create a filecontaining the necessary rule statements and have Prometheus load the file viathe rule_files field in the Prometheus configuration.Rule files use YAML.

The rule files can be reloaded at runtime by sending SIGHUP to the Prometheusprocess. The changes are only applied if all rule files are well-formatted.

Syntax-checking rules

To quickly check whether a rule file is syntactically correct without startinga Prometheus server, install and run Prometheus's promtool command-lineutility tool:

  1. go get github.com/prometheus/prometheus/cmd/promtool
  2. promtool check rules /path/to/example.rules.yml

When the file is syntactically valid, the checker prints a textualrepresentation of the parsed rules to standard output and then exits witha 0 return status.

If there are any syntax errors or invalid input arguments, it prints an error message to standard error and exits with a 1 return status.

Recording rules

Recording rules allow you to precompute frequently needed or computationallyexpensive expressions and save their result as a new set of time series.Querying the precomputed result will then often be much faster than executingthe original expression every time it is needed. This is especially useful fordashboards, which need to query the same expression repeatedly every time theyrefresh.

Recording and alerting rules exist in a rule group. Rules within a group arerun sequentially at a regular interval.

The syntax of a rule file is:

  1. groups:
  2. [ - <rule_group> ]

A simple example rules file would be:

  1. groups:
  2. - name: example
  3. rules:
  4. - record: job:http_inprogress_requests:sum
  5. expr: sum(http_inprogress_requests) by (job)

<rule_group>

  1. # The name of the group. Must be unique within a file.
  2. name: <string>
  3. # How often rules in the group are evaluated.
  4. [ interval: <duration> | default = global.evaluation_interval ]
  5. rules:
  6. [ - <rule> ... ]

<rule>

The syntax for recording rules is:

  1. # The name of the time series to output to. Must be a valid metric name.
  2. record: <string>
  3. # The PromQL expression to evaluate. Every evaluation cycle this is
  4. # evaluated at the current time, and the result recorded as a new set of
  5. # time series with the metric name as given by 'record'.
  6. expr: <string>
  7. # Labels to add or overwrite before storing the result.
  8. labels:
  9. [ <labelname>: <labelvalue> ]

The syntax for alerting rules is:

  1. # The name of the alert. Must be a valid metric name.
  2. alert: <string>
  3. # The PromQL expression to evaluate. Every evaluation cycle this is
  4. # evaluated at the current time, and all resultant time series become
  5. # pending/firing alerts.
  6. expr: <string>
  7. # Alerts are considered firing once they have been returned for this long.
  8. # Alerts which have not yet fired for long enough are considered pending.
  9. [ for: <duration> | default = 0s ]
  10. # Labels to add or overwrite for each alert.
  11. labels:
  12. [ <labelname>: <tmpl_string> ]
  13. # Annotations to add to each alert.
  14. annotations:
  15. [ <labelname>: <tmpl_string> ]