redis_streams

Pushes messages to a Redis (v5.0+) Stream (which is created if it doesn't already exist) using the XADD command.

# Config fields, showing default values
output:
redis_streams:
url: tcp://localhost:6379
stream: benthos_stream
body_key: body
max_length: 0
max_in_flight: 1

It's possible to specify a maximum length of the target stream by setting it to a value greater than 0, in which case this cap is applied only when Redis is able to remove a whole macro node, for efficiency.

Redis stream entries are key/value pairs, as such it is necessary to specify the key to be set to the body of the message. All metadata fields of the message will also be set as key/value pairs, if there is a key collision between a metadata item and the body then the body takes precedence.

Performance

This output benefits from sending multiple messages in flight in parallel for improved performance. You can tune the max number of in flight messages with the field max_in_flight.

Fields

url

The URL of a Redis server to connect to.

Type: string
Default: "tcp://localhost:6379"

# Examples
url: tcp://localhost:6379

stream

The stream to add messages to.

Type: string
Default: "benthos_stream"

body_key

A key to set the raw body of the message to.

Type: string
Default: "body"

max_length

When greater than zero enforces a rough cap on the length of the target stream.

Type: number
Default: 0

max_in_flight

The maximum number of messages to have in flight at a given time. Increase this to improve throughput.

Type: number
Default: 1