addons/otel/README.md
The OTel filter enables HAProxy to emit telemetry data -- traces, metrics and logs -- to any OpenTelemetry-compatible backend via the OpenTelemetry protocol (OTLP).
It is the successor to the OpenTracing (OT) filter, built on the OpenTelemetry standard which unifies distributed tracing, metrics and logging into a single observability framework.
The filter requires the OpenTelemetry C Wrapper library, which wraps the OpenTelemetry C++ SDK.
The OTel filter is compiled together with HAProxy by adding USE_OTEL=1 to the
make command.
PKG_CONFIG_PATH=/opt/lib/pkgconfig make -j8 USE_OTEL=1 TARGET=linux-glibc
make -j8 USE_OTEL=1 OTEL_INC=/opt/include OTEL_LIB=/opt/lib TARGET=linux-glibc
| Variable | Description |
|---|---|
USE_OTEL | Enable the OpenTelemetry filter |
OTEL_DEBUG | Compile in debug mode |
OTEL_INC | Force path to opentelemetry-c-wrapper include files |
OTEL_LIB | Force path to opentelemetry-c-wrapper library |
OTEL_RUNPATH | Add opentelemetry-c-wrapper RUNPATH to executable |
OTEL_USE_VARS | Enable context propagation via HAProxy variables |
PKG_CONFIG_PATH=/opt/lib/pkgconfig make -j8 USE_OTEL=1 OTEL_DEBUG=1 TARGET=linux-glibc
PKG_CONFIG_PATH=/opt/lib/pkgconfig make -j8 USE_OTEL=1 OTEL_USE_VARS=1 TARGET=linux-glibc
./haproxy -vv | grep -i opentelemetry
If the filter is built in, the output contains:
Built with OpenTelemetry support (C++ version 1.26.0, C Wrapper version 1.0.0-842).
[OTEL] opentelemetry
When pkg-config is not used, the executable may not find the library at startup.
Use LD_LIBRARY_PATH or build with OTEL_RUNPATH=1:
LD_LIBRARY_PATH=/opt/lib ./haproxy ...
make -j8 USE_OTEL=1 OTEL_RUNPATH=1 OTEL_INC=/opt/include OTEL_LIB=/opt/lib TARGET=linux-glibc
The filter uses a two-file configuration model:
.cfg) -- defines the telemetry model:
instrumentation settings, scopes and groups..yml) -- defines the OpenTelemetry SDK
pipeline: exporters, samplers, processors, providers and signal routing.Add the filter to a HAProxy proxy section (frontend/listen/backend):
frontend my-frontend
...
filter opentelemetry [id <id>] config <file>
...
If no filter id is specified, otel-filter is used as default.
The OTel configuration file contains three section types:
otel-instrumentation -- mandatory; references the YAML file, sets rate
limits, error modes, logging and declares groups and scopes.otel-scope -- defines actions (spans, attributes, metrics, logs) triggered
by stream events or from groups.otel-group -- a named collection of scopes triggered from HAProxy TCP/HTTP
rules.exporters:
my_exporter:
type: otlp_http
endpoint: "http://localhost:4318/v1/traces"
samplers:
my_sampler:
type: always_on
processors:
my_processor:
type: batch
providers:
my_provider:
resources:
- service.name: "haproxy"
signals:
traces:
scope_name: "HAProxy OTel"
exporters: my_exporter
samplers: my_sampler
processors: my_processor
providers: my_provider
| Type | Description |
|---|---|
otlp_grpc | OTLP over gRPC |
otlp_http | OTLP over HTTP (JSON or Protobuf) |
otlp_file | Local files in OTLP format |
zipkin | Zipkin-compatible backends |
elasticsearch | Elasticsearch |
ostream | Text output to a file (for debugging) |
memory | In-memory buffer (for testing) |
| Keyword | Description |
|---|---|
span | Create or reference a span |
attribute | Set key-value span attributes |
event | Add timestamped span events |
baggage | Set context propagation data |
status | Set span status (ok/error/ignore/unset) |
link | Add span links to related spans |
inject | Inject context into headers or variables |
extract | Extract context from headers or variables |
finish | Close spans (supports wildcards: *, *req*, *res*) |
instrument | Create or update metric instruments |
log-record | Emit a log record with severity |
otel-event | Bind scope to a filter event with optional ACL |
idle-timeout | Set periodic event interval for idle streams |
Available via the HAProxy CLI socket (prefix: flt-otel):
| Command | Description |
|---|---|
flt-otel status | Show filter status |
flt-otel enable | Enable the filter |
flt-otel disable | Disable the filter |
flt-otel hard-errors | Enable hard-errors mode |
flt-otel soft-errors | Disable hard-errors mode |
flt-otel logging [state] | Set logging state |
flt-otel rate [value] | Set or show the rate limit |
flt-otel debug [level] | Set debug level (debug build only) |
When invoked without arguments, rate, logging and debug display the
current value.
Benchmark results from the standalone (sa) configuration, which exercises all
events (worst-case scenario):
| Rate limit | Req/s | Avg latency | Overhead |
|---|---|---|---|
| 100.0% | 38,202 | 213.08 us | 21.6% |
| 50.0% | 42,777 | 190.49 us | 12.2% |
| 25.0% | 45,302 | 180.46 us | 7.0% |
| 10.0% | 46,879 | 174.69 us | 3.7% |
| 2.5% | 47,993 | 170.58 us | 1.4% |
| disabled | 48,788 | 167.74 us | ~0 |
| off | 48,697 | 168.00 us | baseline |
With a rate limit of 10% or less, the performance impact is negligible.
Detailed methodology and additional results are in the test/ directory.
The test/ directory contains ready-to-run example configurations:
Start a Jaeger all-in-one container:
docker run -d --name jaeger -p 4317:4317 -p 4318:4318 -p 16686:16686 jaegertracing/all-in-one:latest
Run one of the test configurations:
./test/run-sa.sh
Open the Jaeger UI at http://localhost:16686 to view traces.
Detailed documentation is available in the following files:
Copyright 2026 HAProxy Technologies
Miroslav Zagorac [email protected]