cache

Stores each message in a cache.

# Config fields, showing default values
output:
cache:
target: ""
key: ${!count:items}-${!timestamp_unix_nano}
max_in_flight: 1

Caches are configured within the resources section and can target any of the following types:

The target field must point to a configured cache like follows:

output:
cache:
target: foo
key: ${!json_field:document.id}
resources:
caches:
foo:
memcached:
addresses:
- localhost:11211
ttl: 60

In order to create a unique key value per item you should use function interpolations described here. When sending batched messages the interpolations are performed per message part.

Performance

This output benefits from sending multiple messages in flight in parallel for improved performance. You can tune the max number of in flight messages with the field max_in_flight.

Fields

target

The target cache to store messages in.

Type: string
Default: ""

key

The key to store messages by, function interpolation should be used in order to derive a unique key for each message. This field supports interpolation functions.

Type: string
Default: "${!count:items}-${!timestamp_unix_nano}"

# Examples
key: ${!count:items}-${!timestamp_unix_nano}
key: ${!json_field:doc.id}
key: ${!metadata:kafka_key}

max_in_flight

The maximum number of messages to have in flight at a given time. Increase this to improve throughput.

Type: number
Default: 1