ContextQMD
Back to Pyroscope

Span Profiles with Grafana Tempo and Pyroscope

examples/tracing/ruby/README.md

2.0.13.1 KB
Original Source

Span Profiles with Grafana Tempo and Pyroscope

The docker compose consists of:

  • Three Ruby Rideshare App instances (us-east, eu-north, ap-south regions)
  • Tempo for trace collection
  • Pyroscope for continuous profiling
  • Grafana for visualization
  • Load Generator for simulating traffic

For a detailed guide about Ruby span profiles configuration, refer to the docs Pyroscope documentation.

The rideshare app generates traces and profiling data that should be available in Grafana. Pyroscope and Tempo datasources are provisioned automatically.

Build and run

The project can be run locally with the following commands:

shell
# Pull latest pyroscope and grafana images:
docker pull grafana/pyroscope:latest
docker pull grafana/grafana:latest

bundle install

docker compose up

The load generator will automatically start sending requests to all regional instances.

Viewing Traces and Profiles

Navigate to the Explore page, select a trace and click on a span that has a linked profile:

By default, only the root span gets labeled (the first span created locally): such spans are marked with the link icon and have pyroscope.profile.id attribute set to the corresponding span ID. Please note that presence of the attribute does not necessarily indicate that the span has a profile: stack trace samples might not be collected, if the utilized CPU time is less than the sample interval (10ms).

Grafana Tempo configuration

In order to correlate trace spans with profiling data, the Tempo datasource should have the following configured:

  • The profiling data source
  • Tags to use when making profiling queries

While tags are optional, configuring them is highly recommended for optimizing query performance. In our example, we configured the service.name tag for use in Pyroscope queries as the service_name label. This configuration restricts the data set for lookup, ensuring that queries remain consistently fast. Note that the tags you configure must be present in the span attributes or resources for a trace to profiles span link to appear.

Please refer to our documentation for more details.