OpenTelemetry Integration Guide

This guide explains how to configure and integrate OpenTelemetry (OTEL) using environment variables. It assumes you are familiar with OTEL concepts and provides details on environment variable configuration. The provided docker-compose.yml includes an OTEL Collector for local development, but you can use any collector of your choice.

Install required dependencies

Install dependencies listed in requirements/requirements-opentelemetry.txt, or install [opentelemetry] extra installing mindsdb:

pip install mindsdb[opentelemetry]

Configuring OpenTelemetry with Environment Variables

OpenTelemetry behavior is controlled via environment variables. Below is a breakdown of the key variables and their purposes:

General Variables

• OTEL_EXPORTER_TYPE: Defines the type of exporter (console or otlp).

  • Default: console.

• OTEL_EXPORTER_PROTOCOL: Defines the protocol for the exporter (grpc or http).

  • Default: grpc.

• OTEL_SDK_DISABLED: Disables the Open Telemetry SDK.

  • Default: Disabled for local environments unless OTEL_SDK_FORCE_RUN is true.

• OTEL_SDK_FORCE_RUN: Forces Open Telemetry SDK to run even in disabled environments.

  • Example: Set to true for testing in local environments.

OTLP Exporter Endpoints

• OTEL_OTLP_ENDPOINT: Sets the base endpoint for all OTLP exports.

  • Default: http://localhost:4317.

• OTEL_OTLP_LOGGING_ENDPOINT: Endpoint for logging exports.

  • Default: Same as OTEL_OTLP_ENDPOINT.

• OTEL_OTLP_TRACING_ENDPOINT: Endpoint for tracing exports.

  • Default: Same as OTEL_OTLP_ENDPOINT.

• OTEL_OTLP_METRICS_ENDPOINT: Endpoint for metrics exports.

  • Default: Same as OTEL_OTLP_ENDPOINT.

Service Information

• OTEL_SERVICE_NAME: Specifies the service name.

  • Default: mindsdb_new_test.

• OTEL_SERVICE_INSTANCE_ID: Unique ID for the service instance.

  • Default: mindsdb.

• OTEL_SERVICE_ENVIRONMENT: Specifies the deployment environment (e.g., local, dev, prod).

  • Default: local.

• OTEL_SERVICE_RELEASE: Version or release identifier of the service.

  • Default: local.

• OTEL_TRACE_SAMPLE_RATE: Determines the sampling rate for tracing.

  • Default: 1.0.

Attributes

• OTEL_EXTRA_ATTRIBUTES: Allows passing additional attributes as a comma-separated list of key-value pairs.
  • Example: service.version=0.0.1,foo=bar.

Logging, Metrics, and Tracing Toggles

• OTEL_LOGGING_DISABLED: Disables logging integration.

  • Default: Follows OTEL_SDK_DISABLED.

• OTEL_TRACING_DISABLED: Disables tracing integration.

  • Default: Follows OTEL_SDK_DISABLED or true.

• OTEL_METRICS_DISABLED: Disables metrics integration.

  • Default: Follows OTEL_SDK_DISABLED or true.

Using with Docker Compose

The docker-compose.yml provides an example configuration for local development. Replace the OTEL Collector with your preferred collector in production.

MindsDB Service Configuration

environment:
  OTEL_EXPORTER_TYPE: "otlp"
  OTEL_OTLP_ENDPOINT: "http://otel-collector:4317"
  OTEL_SERVICE_NAME: "mindsdb"
  OTEL_SERVICE_INSTANCE_ID: "mindsdb-instance"
  OTEL_SERVICE_ENVIRONMENT: "local"
  OTEL_LOGGING_DISABLED: "true"
  OTEL_TRACING_DISABLED: "true"
  OTEL_METRICS_DISABLED: "false"
  OTEL_EXTRA_ATTRIBUTES: "service.version=0.0.1,foo=bar"
  OTEL_SDK_FORCE_RUN: "true"

OTEL Collector for Local Development

otel-collector:
  image: otel/opentelemetry-collector-contrib:0.116.1
  volumes:
    - ./otel-collector-config.yaml:/etc/otelcol-contrib/config.yaml
  ports:
    - 4317:4317 # OTLP gRPC receiver