process_field

A processor that extracts the value of a field dot path within payloads according to a specified codec, applies a list of processors to the extracted value and finally sets the field within the original payloads to the processed result.

# Common config fields, showing default values
process_field:
codec: json
path: ""
result_type: string
processors: []

The result can be marshalled into a specific data type with the field result_type.

It's therefore possible to use this codec without any child processors as a way of casting string values into other types. For example, with an input JSON document {"foo":"10"} it's possible to cast the value of the field foo to an integer type with:

process_field:
path: foo
result_type: int

Codecs

json

Parses the payload as a JSON document, extracts and sets the field using a dot notation path.

metadata

Extracts and sets a metadata value identified by the path field.

Fields

codec

A codec to use in order to extract (and set) the target field.

Type: string
Default: "json"
Options: json, metadata.

path

A dot path pointing to the target field.

Type: string
Default: ""

result_type

The final data type to marshal the processing result into. The discard type is a special case that discards the result of the processing steps entirely.

Type: string
Default: "string"
Options: string, int, float, bool, object, discard.

processors

A list of child processors to execute on the extracted value.

Type: array
Default: []

parts

An optional array of message indexes of a batch that the processor should apply to. If left empty all messages are processed. This field is only applicable when batching messages at the input level.

Indexes can be negative, and if so the part will be selected from the end counting backwards starting from -1.

Type: array
Default: []

Examples

For example, with an input JSON document {"foo":"hello world"} it's possible to uppercase the value of the field 'foo' by using the JSON codec and a text child processor:

process_field:
codec: json
path: foo
processors:
- text:
operator: to_upper

If the number of messages resulting from the processing steps does not match the original count then this processor fails and the messages continue unchanged. Therefore, you should avoid using batch and filter type processors in this list.