KV processor

This processor helps automatically parse messages (or specific event fields) which are of the foo=bar variety.

For example, if you have a log message which contains ip=1.2.3.4 error=REFUSED, you can parse those fields automatically by configuring:

  1. {
  2. "kv": {
  3. "field": "message",
  4. "field_split": " ",
  5. "value_split": "="
  6. }
  7. }

Using the KV Processor can result in field names that you cannot control. Consider using the Flattened data type instead, which maps an entire object as a single field and allows for simple searches over its contents.

Table 26. KV Options

NameRequiredDefaultDescription

field

yes

-

The field to be parsed

field_split

yes

-

Regex pattern to use for splitting key-value pairs

value_split

yes

-

Regex pattern to use for splitting the key from the value within a key-value pair

target_field

no

null

The field to insert the extracted keys into. Defaults to the root of the document

include_keys

no

null

List of keys to filter and insert into document. Defaults to including all keys

exclude_keys

no

null

List of keys to exclude from document

ignore_missing

no

false

If true and field does not exist or is null, the processor quietly exits without modifying the document

prefix

no

null

Prefix to be added to extracted keys

trim_key

no

null

String of characters to trim from extracted keys

trim_value

no

null

String of characters to trim from extracted values

strip_brackets

no

false

If true strip brackets (), <>, [] as well as quotes and from extracted values

if

no

-

Conditionally execute this processor.

on_failure

no

-

Handle failures for this processor. See Handling Failures in Pipelines.

ignore_failure

no

false

Ignore failures for this processor. See Handling Failures in Pipelines.

tag

no

-

An identifier for this processor. Useful for debugging and metrics.