HTTP Sink - Kurrentdb

Overview

The HTTP sink allows for integration between KurrentDB and external APIs over HTTP or HTTPS. This connector consumes events from a KurrentDB stream and converts each event's data into JSON format before sending it in the request body to a specified Url. Events are sent individually as they are consumed from the stream, without batching. The event data is transmitted as the request body, and metadata can be included as HTTP headers. The connector also supports Basic Authentication and Bearer Token Authentication. See Authentication.

Prerequisites

Before using the HTTP sink connector, ensure you have:

A data protection token configured in your KurrentDB instance (required to encrypt sensitive fields like passwords and bearer tokens)
A target HTTP/HTTPS endpoint that can receive POST requests
Appropriate authentication credentials if the endpoint requires authentication

::: tip See the Data Protection documentation for instructions on configuring the encryption token. :::

Quickstart

You can create the HTTP Sink connector as follows. Replace {id} with your desired connector ID:

http

POST /connectors/{id}
Host: localhost:2113
Content-Type: application/json

{
  "settings": {
    "instanceTypeName": "http-sink",
    "url": "https://api.example.com/",
    "subscription:filter:scope": "stream",
    "subscription:filter:filterType": "streamId",
    "subscription:filter:expression": "example-stream"
  }
}

After creating and starting the HTTP sink connector, every time an event is appended to the example-stream, the HTTP sink connector will send the record to the specified URL. You can find a list of available management API endpoints in the API Reference.

Settings

Adjust these settings to specify the behavior and interaction of your HTTP sink connector with KurrentDB, ensuring it operates according to your requirements and preferences.

::: tip The HTTP sink inherits a set of common settings that are used to configure the connector. The settings can be found in the Sink Options page. :::

Name	Details
`url`	required

Description: The URL or endpoint to which the request or message will be sent. See Template Parameters for advanced settings.

Default: "" | | method | Description: The method or operation to use for the request or message.

Default: "POST" | | defaultHeaders | Description: Custom headers to include in the request. See Specifying Custom Headers.

Default: "Accept-Encoding:*" | | pooledConnectionLifetime | Description: Maximum time a connection can stay in the pool before it is no longer reusable.

Default: "00:05:00" (5 minutes) |

Authentication

The HTTP sink connector supports both Basic and Bearer authentication methods.

Name	Details
`authentication:method`	Description:
The authentication method to use.

Default: "None"

Accepted Values: "None","Basic", or "Bearer" |

Basic Authentication

Name	Details
`authentication:basic:username`	Description:
The username for basic authentication.

Default: "" | | authentication:basic:password | Description: The password for basic authentication.

Default: "" |

Bearer Authentication

Name	Details
`authentication:bearer:token`	Description:
The token for bearer authentication.

Default: "" |

Resilience

Resilience options can be found in the Resilience Configuration section.

Headers

In addition to the default headers that are included with every request, the HTTP sink connector also includes the following headers in the HTTP request:

Header	Description
`esdb-request-id`	A unique identifier for the request
`esdb-request-date`	The timestamp when the request was processed
`content-type`	The media type of the resource
`accept-encoding`	The encoding types the client can handle. Default value is `*`

Specifying Custom Headers

The HTTP sink connector lets you include custom headers in the HTTP requests it sends to your specified URL. To add custom headers, use the defaultHeaders setting in your connector configuration. Each custom header should be specified with the prefix defaultHeaders: followed by the header name.

Example Request:

http

PUT /connectors/{id}
Host: localhost:2113
Content-Type: application/json

{
  "defaultHeaders:X-API-Key": "your-api-key-here",
  "defaultHeaders:X-Tenant-ID": "production-tenant",
  "defaultHeaders:X-Source-System": "KurrentDB"
}

With this configuration, every HTTP request sent by the connector will include the specified custom headers along with their respective values, in addition to the default headers that are automatically added by the connector's plugin.

Template parameters

The HTTP sink supports the use of template parameters in the URL, allowing for dynamic construction of the request URL based on event data. This feature enables you to customize the destination URL for each event, making it easier to integrate with APIs that require specific URL structures.

The following template parameters are available for use in the URL:

Parameter	Description
`{schema-subject}`	The event's schema subject, converted to lowercase with hyphens
`{event-type}`	Alias for `{schema-subject}`
`{stream}`	The KurrentDB stream ID

Usage

To use template parameters, include them in the Url option of your HTTP sink configuration. The parameters will be replaced with their corresponding values for each event.

Example:

https://api.example.com/{schema-subject}

For an event with schema subject "TestEvent", this would result in the URL:

https://api.example.com/test-event

Tutorial

Learn how to configure and use a connector using the HTTP Sink in KurrentDB through a tutorial.