InfluxDB

The influxdb output plugin, allows to flush your records into a InfluxDB time series database. The following instructions assumes that you have a fully operational InfluxDB service running in your system.

Configuration Parameters

Key Description default
Host IP address or hostname of the target InfluxDB service 127.0.0.1
Port TCP port of the target InfluxDB service 8086
Database InfluxDB database name where records will be inserted fluentbit
Bucket InfluxDB bucket name where records will be inserted - if specified, database is ignored and v2 of API is used
Org InfluxDB organization name where the bucket is (v2 only) fluent
Sequence_Tag The name of the tag whose value is incremented for the consecutive simultaneous events. _seq
HTTP_User Optional username for HTTP Basic Authentication
HTTP_Passwd Password for user defined in HTTP_User
HTTP_Token Authentication token used with InfluDB v2 - if specified, both HTTP_User and HTTP_Passwd are ignored
Tag_Keys Space separated list of keys that needs to be tagged
Auto_Tags Automatically tag keys where value is string. This option takes a boolean value: True/False, On/Off. Off

TLS / SSL

InfluxDB output plugin supports TTL/SSL, for more details about the properties available and general configuration, please refer to the TLS/SSL section.

Getting Started

In order to start inserting records into an InfluxDB service, you can run the plugin from the command line or through the configuration file:

Command Line

The influxdb plugin, can read the parameters from the command line in two ways, through the -p argument (property) or setting them directly through the service URI. The URI format is the following:

  1. influxdb://host:port

Using the format specified, you could start Fluent Bit through:

  1. $ fluent-bit -i cpu -t cpu -o influxdb://127.0.0.1:8086 -m '*'

Configuration File

In your main configuration file append the following Input & Output sections:

  1. [INPUT]
  2. Name cpu
  3. Tag cpu
  4. [OUTPUT]
  5. Name influxdb
  6. Match *
  7. Host 127.0.0.1
  8. Port 8086
  9. Database fluentbit
  10. Sequence_Tag _seq

Tagging

Basic example of Tag_Keys usage:

  1. [INPUT]
  2. Name tail
  3. Tag apache.access
  4. parser apache2
  5. path /var/log/apache2/access.log
  6. [OUTPUT]
  7. Name influxdb
  8. Match *
  9. Host 127.0.0.1
  10. Port 8086
  11. Database fluentbit
  12. Sequence_Tag _seq
  13. # make tags from method and path fields
  14. Tag_Keys method path

With Auto_Tags=On in this example cause error, because every parsed field value type is string. Best usage of this option in metrics like record where one ore more field value is not string typed.

Testing

Before to start Fluent Bit, make sure the target database exists on InfluxDB, using the above example, we will insert the data into a fluentbit database.

1. Create database

Log into InfluxDB console:

  1. $ influx
  2. Visit https://enterprise.influxdata.com to register for updates, InfluxDB server management, and monitoring.
  3. Connected to http://localhost:8086 version 1.1.0
  4. InfluxDB shell version: 1.1.0
  5. >

Create the database:

  1. > create database fluentbit
  2. >

Check the database exists:

  1. > show databases
  2. name: databases
  3. name
  4. ----
  5. _internal
  6. fluentbit
  7. >

2. Run Fluent Bit

The following command will gather CPU metrics from the system and send the data to InfluxDB database every five seconds:

  1. $ bin/fluent-bit -i cpu -t cpu -o influxdb -m '*'

Note that all records coming from the cpu input plugin, have a tag cpu, this tag is used to generate the measurement in InfluxDB

3. Query the data

From InfluxDB console, choose your database:

  1. > use fluentbit
  2. Using database fluentbit

Now query some specific fields:

  1. > SELECT cpu_p, system_p, user_p FROM cpu
  2. name: cpu
  3. time cpu_p system_p user_p
  4. ---- ----- -------- ------
  5. 1481132860000000000 2.75 0.5 2.25
  6. 1481132861000000000 2 0.5 1.5
  7. 1481132862000000000 4.75 1.5 3.25
  8. 1481132863000000000 6.75 1.25 5.5
  9. 1481132864000000000 11.25 3.75 7.5

The CPU input plugin gather more metrics per CPU core, in the above example we just selected three specific metrics. The following query will give a full result:

  1. > SELECT * FROM cpu

4. View tags

Query tagged keys:

  1. > SHOW TAG KEYS ON fluentbit FROM "apache.access"
  2. name: apache.access
  3. tagKey
  4. ------
  5. _seq
  6. method
  7. path

And now query method key values:

  1. > SHOW TAG VALUES ON fluentbit FROM "apache.access" WITH KEY = "method"
  2. name: apache.access
  3. key value
  4. --- -----
  5. method "MATCH"
  6. method "POST"