json stage

The json stage is a parsing stage that reads the log line as JSON and acceptsJMESPath expressions to extract data.

Schema

  1. json:
  2.   # Set of key/value pairs of JMESPath expressions. The key will be
  3.   # the key in the extracted data while the expression will the value,
  4.   # evaluated as a JMESPath from the source data.
  5.   expressions:
  6.     [ <string>: <string> ... ]
  8.   # Name from extracted data to parse. If empty, uses the log message.
  9.   [source: <string>]

This stage uses the Go JSON unmarshaler, which means non-string types likenumbers or booleans will be unmarshaled into those types. The extracted datacan hold non-string values and this stage does not do any type conversions;downstream stages will need to perform correct type conversion of these valuesas necessary. Please refer to the the template stage for howto do this.

If the value extracted is a complex type, such as an array or a JSON object, itwill be converted back into a JSON string before being inserted into theextracted data.

Examples

Using log line

For the given pipeline:

  1. - json:
  2.     expressions:
  3.       output: log
  4.       stream: stream
  5.       timestamp: time

Given the following log line:

  1. {"log":"log message\n","stream":"stderr","time":"2019-04-30T02:12:41.8443515Z"}

The following key-value pairs would be created in the set of extracted data:

  • output: log message\n
  • stream: stderr
  • timestamp: 2019-04-30T02:12:41.8443515

Using extracted data

For the given pipeline:

  1. - json:
  2.     expressions:
  3.       output:    log
  4.       stream:    stream
  5.       timestamp: time
  6.       extra:
  7. - json:
  8.     expressions:
  9.       user:
  10.     source: extra

And the given log line:

  1. {"log":"log message\n","stream":"stderr","time":"2019-04-30T02:12:41.8443515Z","extra":"{\"user\":\"marco\"}"}

The first stage would create the following key-value pairs in the set ofextracted data:

  • output: log message\n
  • stream: stderr
  • timestamp: 2019-04-30T02:12:41.8443515
  • extra: {"user": "marco"}

The second stage will parse the value of extra from the extracted data as JSONand append the following key-value pairs to the set of extracted data:

  • user: marco