Configure the Zipkin data source

This document explains how to configure the Zipkin data source in Grafana, including connection settings, authentication, trace-to-logs and trace-to-metrics integrations, and provisioning with YAML or Terraform.

Before you begin

Before configuring the data source, ensure you have:

• Grafana permissions: Organization administrator role
• Zipkin instance: A running Zipkin instance accessible from your Grafana server

Add the data source

To add the Zipkin data source:

1. Click Connections in the left-side menu.
2. Click Add new connection.
3. Type Zipkin in the search bar.
4. Select Zipkin.
5. Click Add new data source.

Configure settings

The following table describes the basic configuration settings for the Zipkin data source.

Authentication

The Zipkin data source supports the following authentication methods:

• Basic authentication: Provide a username and password to authenticate with the Zipkin instance.
• TLS client authentication: Configure client certificates for mutual TLS.
• Forward OAuth identity: Forward the user's OAuth token to the Zipkin instance.
• With credentials: Send credentials (cookies, TLS client certificates) with cross-site requests.

You can also configure custom headers and TLS settings in the Advanced HTTP settings section, which is located inside Additional settings.

Trace to logs

{{< figure src="/media/docs/tempo/tempo-trace-to-logs-9-4.png" max-width="800px" class="docs-image--no-shadow" caption="Trace to logs settings" >}}

{{< admonition type="note" >}} If you use Grafana Cloud, open a support ticket in the Cloud Portal to access this feature. {{< /admonition >}}

The Trace to logs setting configures the trace to logs feature that is available when you integrate Grafana with Zipkin. This feature lets you navigate from a span in a trace view directly to the relevant logs.

There are two ways to configure the trace to logs feature:

• Use a simplified configuration with a default query, or
• Configure a custom query where you can use a template language to interpolate variables from the trace or span.

Use a simple configuration

1. Select the target data source from the drop-down list.

   You can also click Open advanced data source picker to see more options, including adding a data source.

2. Set start and end time shift. Since the log timestamps may not exactly match the timestamps of the spans in the trace, you may need to widen or shift the time range to find the desired logs.

3. Select which tags to use in the logs query. The tags you configure must be present in the span attributes or resources for a trace to logs span link to appear. You can optionally configure a new name for the tag. This is useful if the tag has dots in the name and the target data source doesn't allow dots in labels. For example, you can remap http.status to http_status.

4. Optionally, switch on the Filter by trace ID and/or Filter by span ID setting to further filter the logs if your logs consistently contain trace or span IDs.

Configure a custom query

1. Select the target data source from the drop-down list.

   You can also click Open advanced data source picker to see more options, including adding a data source.

2. Set start and end time shift. Since the log timestamps may not exactly match the timestamps of the spans in the trace, you may need to widen or shift the time range to find the desired logs.

3. Optionally, select tags to map. These tags can be used in the custom query with the ${__tags} variable. This variable interpolates the mapped tags as a list in an appropriate syntax for the data source and only includes tags that are present in the span. You can optionally configure a new name for the tag. This is useful when the tag has dots in the name and the target data source doesn't allow dots in labels. For example, you can remap http.status to http_status. If you don't map any tags here, you can still use any tag in the query like this: method="${__span.tags.method}".

4. Skip Filter by trace ID and Filter by span ID settings as these can't be used with a custom query.

5. Switch on Use custom query.

6. Specify a custom query to be used to query the logs. You can use various variables to make the query relevant for the current span. The link only appears if all the variables are interpolated with non-empty values to prevent creating an invalid query.

Variables for custom queries

To use a variable, wrap it in ${}. For example: ${__span.name}.

Trace to logs settings

The following table describes the trace to logs configuration options.

Trace to metrics

The Trace to metrics setting lets you navigate from a span in a trace view to a metrics query in a configured metrics data source.

To configure trace to metrics:

1. Select the target data source from the drop-down list.

   You can also click Open advanced data source picker to see more options, including adding a data source.

2. Create any desired linked queries.

Each linked query consists of:

• Link Label: (Optional) Descriptive label for the linked query.
• Query: The query run when navigating from a trace to the metrics data source. Interpolate tags using the $__tags keyword. For example, when you configure the query requests_total{$__tags} with the tags k8s.pod=pod and cluster, the result looks like requests_total{pod="nginx-554b9", cluster="us-east-1"}.

Additional settings

The Additional settings section is collapsible and contains optional settings for the node graph, span bar, advanced HTTP configuration, and secure SOCKS proxy (when enabled in your Grafana configuration).

Node graph

The Enable node graph toggle enables the Node graph visualization, which is disabled by default.

After you enable it, Grafana displays the node graph above the trace view.

Span bar

The Span bar setting lets you display additional information in the span bar row.

The Label drop-down has three options:

Verify the connection

Click Save & test to verify the connection. A successful connection displays the message Data source is working.

If you encounter errors, refer to Troubleshoot Zipkin data source issues.

Provision the data source

You can define and configure the data source in YAML files as part of the Grafana provisioning system. For more information about provisioning and available configuration options, refer to Provisioning Grafana.

Provisioning example

apiVersion: 1

datasources:
  - name: Zipkin
    type: zipkin
    uid: zipkin-ds
    url: http://localhost:9411
    access: proxy
    basicAuth: true
    basicAuthUser: my_user
    readOnly: false
    isDefault: false
    jsonData:
      tracesToLogsV2:
        datasourceUid: 'loki'
        spanStartTimeShift: '1h'
        spanEndTimeShift: '-1h'
        tags: ['job', 'instance', 'pod', 'namespace']
        filterByTraceID: false
        filterBySpanID: false
        customQuery: true
        query: 'method="$${__span.tags.method}"'
      tracesToMetrics:
        datasourceUid: 'prom'
        spanStartTimeShift: '1h'
        spanEndTimeShift: '-1h'
        tags: [{ key: 'service.name', value: 'service' }, { key: 'job' }]
        queries:
          - name: 'Sample query'
            query: 'sum(rate(traces_spanmetrics_latency_bucket{$$__tags}[5m]))'
      nodeGraph:
        enabled: true
      spanBar:
        type: 'None'
    secureJsonData:
      basicAuthPassword: my_password

Configure with Terraform

You can configure the Zipkin data source using Terraform with the Grafana Terraform provider.

For more information about provisioning resources with Terraform, refer to the Grafana as code using Terraform documentation.

Basic Terraform example

The following example creates a basic Zipkin data source:

resource "grafana_data_source" "zipkin" {
  name = "Zipkin"
  type = "zipkin"
  url  = "http://localhost:9411"
}

Terraform example with trace to logs

The following example includes a trace to logs configuration that links traces to a Loki data source:

resource "grafana_data_source" "zipkin" {
  name = "Zipkin"
  type = "zipkin"
  url  = "http://localhost:9411"

  json_data_encoded = jsonencode({
    tracesToLogsV2 = {
      datasourceUid    = grafana_data_source.loki.uid
      spanStartTimeShift = "1h"
      spanEndTimeShift   = "-1h"
      filterByTraceID  = true
      filterBySpanID   = false
      tags = [
        { key = "service.name", value = "service" },
        { key = "namespace" }
      ]
    }
    nodeGraph = {
      enabled = true
    }
  })
}

Terraform example with basic authentication

The following example includes basic authentication:

resource "grafana_data_source" "zipkin" {
  name = "Zipkin"
  type = "zipkin"
  url  = "http://localhost:9411"

  basic_auth_enabled  = true
  basic_auth_username = "zipkin_user"

  secure_json_data_encoded = jsonencode({
    basicAuthPassword = var.zipkin_password
  })

  json_data_encoded = jsonencode({
    nodeGraph = {
      enabled = true
    }
  })
}

For all available configuration options, refer to the Grafana provider data source resource documentation.