subprocess

Executes a command as a subprocess and, for each message, will pipe its contents to the stdin stream of the process followed by a newline.

# Common config fields, showing default values
subprocess:
name: cat
args: []

The subprocess must then either return a line over stdout or stderr. If a response is returned over stdout then its contents will replace the message. If a response is instead returned from stderr it will be logged and the message will continue unchanged and will be marked as failed.

The execution environment of the subprocess is the same as the Benthos instance, including environment variables and the current working directory.

The field max_buffer defines the maximum response size able to be read from the subprocess. This value should be set significantly above the real expected maximum response size.

Subprocess requirements

It is required that subprocesses flush their stdout and stderr pipes for each line. Benthos will attempt to keep the process alive for as long as the pipeline is running. If the process exits early it will be restarted.

Messages containing line breaks

If a message contains line breaks each line of the message is piped to the subprocess and flushed, and a response is expected from the subprocess before another line is fed in.

Fields

name

The command to execute as a subprocess.

Type: string
Default: "cat"

# Examples
name: cat
name: sed
name: awk

args

A list of arguments to provide the command.

Type: array
Default: []

max_buffer

The maximum expected response size.

Type: number
Default: 65536

parts

An optional array of message indexes of a batch that the processor should apply to. If left empty all messages are processed. This field is only applicable when batching messages at the input level.

Indexes can be negative, and if so the part will be selected from the end counting backwards starting from -1.

Type: array
Default: []