Prometheus endpoints support in InfluxDB

Prometheus remote read and write API support

Note: The Prometheus API Stability Guarantees states that remote read and remote write endpoints are features listed as experimental or subject to change, and thus considered unstable for 2.x. Any breaking changes will be included in the InfluxDB release notes.

InfluxDB support for the Prometheus remote read and write API adds the following HTTP endpoints to InfluxDB:

• /api/v1/prom/read
• /api/v1/prom/write

Additionally, there is a /metrics endpoint configured to produce default Go metrics in Prometheus metrics format.

Create a target database

Create a database in your InfluxDB instance to house data sent from Prometheus. In the examples provided below, prometheus is used as the database name, but you’re welcome to use the whatever database name you like.

CREATE DATABASE "prometheus"

Configuration

To enable the use of the Prometheus remote read and write APIs with InfluxDB, add URL values to the following settings in the Prometheus configuration file:

• remote_write
• remote_read

The URLs must be resolvable from your running Prometheus server and use the port on which InfluxDB is running (8086 by default). Also include the database name using the db= query parameter.

Example: Endpoints in Prometheus configuration file

remote_write:
  - url: "http://localhost:8086/api/v1/prom/write?db=prometheus"
remote_read:
  - url: "http://localhost:8086/api/v1/prom/read?db=prometheus"

Read and write URLs with authentication

If authentication is enabled on InfluxDB, pass the username and password of an InfluxDB user with read and write privileges using the u= and p= query parameters respectively.

Examples of endpoints with authentication enabled**_

remote_write:
  - url: "http://localhost:8086/api/v1/prom/write?db=prometheus&u=username&p=password"
remote_read:
  - url: "http://localhost:8086/api/v1/prom/read?db=prometheus&u=username&p=password"

Including plain text passwords in your Prometheus configuration file is not ideal. Unfortunately, environment variables and secrets are not supported in Prometheus configuration files. See this Prometheus issue for more information:

Support for environment variable substitution in configuration file

How Prometheus metrics are parsed in InfluxDB

As Prometheus data is brought into InfluxDB, the following transformations are made to match the InfluxDB data structure:

• The Prometheus metric name becomes the InfluxDB measurement name.
• The Prometheus sample (value) becomes an InfluxDB field using the value field key. It is always a float.
• Prometheus labels become InfluxDB tags.
• All # HELP and # TYPE lines are ignored.
• [v1.8.6 and later] Prometheus remote write endpoint drops unsupported Prometheus values (NaN,-Inf, and +Inf) rather than reject the entire batch.
  • If write trace logging is enabled ([http] write-tracing = true), then summaries of dropped values are logged.
  • If a batch of values contains values that are subsequently dropped, HTTP status code 204 is returned.

Example: Parse Prometheus to InfluxDB

# Prometheus metric
example_metric{queue="0:http://example:8086/api/v1/prom/write?db=prometheus",le="0.005"} 308
# Same metric parsed into InfluxDB
measurement
  example_metric
tags
  queue = "0:http://example:8086/api/v1/prom/write?db=prometheus"
  le = "0.005"
  job = "prometheus"
  instance = "localhost:9090"
  __name__ = "example_metric"
fields
  value = 308

In InfluxDB v1.5 and earlier, all Prometheus data goes into a single measurement named _ and the Prometheus measurement name is stored in the __name__ label. In InfluxDB v1.6 or later, every Prometheus measurement gets its own InfluxDB measurement.

This format is different than the format used by the Telegraf Prometheus input plugin.