http_client

Connects to a server and continuously performs requests for a single message.

# Common config fields, showing default values
input:
http_client:
url: http://localhost:4195/get
verb: GET
headers:
Content-Type: application/octet-stream
rate_limit: ""
timeout: 5s
payload: ""
stream:
enabled: false
reconnect: true

The URL and header values of this type can be dynamically set using function interpolations described here.

Streaming

If you enable streaming then Benthos will consume the body of the response as a line delimited feed of message parts. Each part is read as an individual message unless multipart is set to true, in which case an empty line indicates the end of a message.

Fields

url

The URL to connect to. This field supports interpolation functions.

Type: string
Default: "http://localhost:4195/get"

verb

A verb to connect with

Type: string
Default: "GET"

# Examples
verb: POST
verb: GET
verb: DELETE

headers

A map of headers to add to the request. This field supports interpolation functions.

Type: object
Default: {"Content-Type":"application/octet-stream"}

# Examples
headers:
Content-Type: application/octet-stream

oauth

Allows you to specify open authentication.

Type: object
Default: {"access_token":"","access_token_secret":"","consumer_key":"","consumer_secret":"","enabled":false,"request_url":""}

# Examples
oauth:
access_token: baz
access_token_secret: bev
consumer_key: foo
consumer_secret: bar
enabled: true
request_url: http://thisisjustanexample.com/dontactuallyusethis

basic_auth

Allows you to specify basic authentication.

Type: object
Default: {"enabled":false,"password":"","username":""}

# Examples
basic_auth:
enabled: true
password: bar
username: foo

tls

Custom TLS settings can be used to override system defaults.

Type: object
Default: {"client_certs":[],"enabled":false,"root_cas_file":"","skip_cert_verify":false}

tls.enabled

Whether custom TLS settings are enabled.

Type: bool
Default: false

tls.skip_cert_verify

Whether to skip server side certificate verification.

Type: bool
Default: false

tls.root_cas_file

The path of a root certificate authority file to use.

Type: string
Default: ""

tls.client_certs

A list of client certificates to use.

Type: array
Default: []

# Examples
client_certs:
- cert: foo
key: bar
client_certs:
- cert_file: ./example.pem
key_file: ./example.key

copy_response_headers

Sets whether to copy the headers from the response to the resulting payload.

Type: bool
Default: false

rate_limit

An optional rate limit to throttle requests by.

Type: string
Default: ""

timeout

A static timeout to apply to requests.

Type: string
Default: "5s"

retry_period

The base period to wait between failed requests.

Type: string
Default: "1s"

max_retry_backoff

The maximum period to wait between failed requests.

Type: string
Default: "300s"

retries

The maximum number of retry attempts to make.

Type: number
Default: 3

backoff_on

A list of status codes whereby retries should be attempted but the period between them should be increased gradually.

Type: array
Default: [429]

drop_on

A list of status codes whereby the attempt should be considered failed but retries should not be attempted.

Type: array
Default: []

successful_on

A list of status codes whereby the attempt should be considered successful (allows you to configure non-2XX codes).

Type: array
Default: []

payload

An optional payload to deliver for each request.

Type: string
Default: ""

stream

Allows you to set streaming mode, where requests are kept open and messages are processed line-by-line.

Type: object
Default: {"delimiter":"","enabled":false,"max_buffer":1000000,"multipart":false,"reconnect":true}

stream.enabled

Enables streaming mode.

Type: bool
Default: false

stream.reconnect

Sets whether to re-establish the connection once it is lost.

Type: bool
Default: true

stream.multipart

When running in stream mode sets whether an empty line indicates the end of a message batch, and only then is the batch flushed downstream.

Type: bool
Default: false

stream.max_buffer

Must be larger than the largest line of the stream.

Type: number
Default: 1000000

stream.delimiter

A string that indicates the end of a message within the stream. If left empty then line feed (\n) is used.

Type: string
Default: ""