Back to Grafana

Query and transform data

docs/sources/visualizations/panels-visualizations/query-transform-data/_index.md

13.1.021.3 KB
Original Source

Query and transform data

Grafana supports many types of data sources. Data source {{< term "query" >}}queries{{< /term >}} return data that Grafana can {{< term "transformation" >}}transform{{< /term >}} and visualize. Each data source uses its own query language, and data source plugins each implement a query-building user interface called a query editor.

About queries

Grafana panels communicate with data sources using queries, which retrieve data for the visualization. A query is a question written in the query language used by the data source.

You can configure query frequency and data collection limits in the panel's data source options. Grafana supports up to 26 queries per panel.

{{< admonition type="note" >}} You must be familiar with a data source's query language. For more information, refer to Data sources. {{< /admonition >}}

Query editors

{{< admonition type="note" >}} The screenshots of the panel query editor on this page show the new query editing experience, currently in public preview. For information about the classic, generally available query editor, refer to the Grafana v13.0 documentation.

Grafana Labs offers limited support for public preview features, and breaking changes might occur prior to the feature being made generally available. To use this feature, enable the queryEditorNext feature toggle in your Grafana configuration file or contact Support. {{< /admonition >}}

Each data source's query editor provides a customized user interface that helps you write queries that take advantage of its unique capabilities.

Because of the differences between query languages, each data source query editor looks and functions differently. Depending on your data source, the query editor might provide auto-completion features, metric names, variable suggestions, or a visual query-building interface.

{{< figure src="/media/docs/grafana/panels-visualizations/screenshot-query-editor-v13.1.png" max-width="750px" alt="The Prometheus query editor" >}}

For details on a specific data source's unique query editor features, refer to its documentation:

  • For data sources included with Grafana, refer to Built-in core data sources, which links to each core data source's documentation.
  • For data sources installed as plugins, refer to the documentation for the plugin.
    • Data source plugins in the Grafana plugin catalog link to or include their documentation in their catalog listings. For details about the plugin catalog, refer to Plugin management.
    • For links to Grafana Enterprise data source plugin documentation, refer to the Enterprise plugins index.

Query syntax

Each data source uses a different query languages to request data. For details on a specific data source's unique query language, refer to its documentation.

PostgreSQL example:

SELECT hostname FROM host WHERE region IN($region)

PromQL example:

query_result(max_over_time(<metric>[${__range_s}s]) != <state>)

Special data sources

Grafana also includes three special data sources: Grafana, Mixed, and Dashboard. For details, refer to Data sources

Saved queries

{{< admonition type="note" >}} Saved queries is currently in public preview. Grafana Labs offers limited support, and breaking changes might occur prior to the feature being made generally available.

This feature is only available on Grafana Enterprise and Grafana Cloud. It will gradually roll out to all Grafana Cloud users with no action required. To try out this feature on Grafana Enterprise, enable the queryLibrary feature toggle. {{< /admonition >}}

You can save queries that you've created so they can be reused by you and others in your organization. This helps users across your organization create dashboards or find insights in Explore without having to create their own queries or know a query language. It also helps you avoid having several users build the same queries for the same data sources multiple times.

Saved queries are available in:

Learn more about saved queries:

Saved queries dialog box

The Saved queries dialog box gives you access to all the saved queries in your organization:

{{< figure src="/media/docs/grafana/dashboards/screenshot-saved-queries-v13.0.png" max-width="750px" alt="List of saved queries" caption="The Saved queries dialog box accessed from Dashboards" >}}

From here, you can:

  • Search for queries by data source name, query content, title, or description.

  • Sort queries alphabetically or by creation date.

  • Filter by data source name, author name, and tags. The tag filter uses the OR operator, while the others use the AND operator.

    {{< admonition type="tip">}} Use the Remember filters switch to persist your filter selections across sessions in your local storage. {{< /admonition >}}

  • Star queries so that they appear in the Starred queries filter view.

  • Duplicate, or delete a saved query.

  • Edit a query title, description, or tags.

  • When you access the Saved queries dialog box from Explore, you can use the Edit in Explore option to edit the body of a query.

You can apply all the same search, filter, and sort options in the Starred queries filter view.

To access your saved queries, click the blue plus sign in the sidebar and select Add saved query or click Replace in the editor pane:

{{< figure src="/media/docs/grafana/panels-visualizations/screenshot-add-save-reuse-query-v13.1.png" max-width="750px" alt="Access saved queries" >}}

Clicking Add saved query adds an additional query, while clicking Replace updates your configured query.

{{< admonition type="tip">}} When you select a query with a Loki, Mimir, Tempo, or Pyroscope data source, the Saved queries dialog box displays a Drilldown button. Click the button to open the associated Drilldown app, while maintaining the context of the query. Learn more about these apps in the Drilldown documentation. {{< /admonition >}}

Roles, permissions, and RBAC

Saved queries support role-based access controls. By default, saved queries have two RBAC roles:

  • Writer: Create, update, and delete all saved queries.
  • Reader: Reuse saved queries.

If you used saved queries prior to the addition of RBAC support in Grafana v12.4, Grafana user roles are mapped as follows:

  • Admin > Writer
  • Editor > Writer
  • Viewer > Reader

Save a query

To save a query you've created:

  1. From the query editor, open the Saved queries drop-down menu and click the Save query option:

    {{< figure src="/media/docs/grafana/panels-visualizations/screenshot-save-query-v13.1.png" max-width="750px" alt="Save a query" >}}

  2. In the Saved queries dialog box, enter a title for the query that makes it easier to find later.

  3. (Optional) Enter a description and relevant tags.

  4. Click Save.

Variables in saved queries

If a saved query includes variables, you can substitute the variables in the query without modifying it. This is useful in environments where variable names or available values differ between dashboards.

You can map the original variables to either:

  • A variable in your dashboard
  • A custom value that you enter

{{< figure src="/media/docs/grafana/panels-visualizations/screenshot-saved-query-variable-v13.0.png" max-width="450px" alt="A saved query with substituted variables" >}}

Grafana applies your selections to the query before inserting it into the dashboard. However, the substitutions only apply to the query when it's reused, and the original saved query remains unchanged.

{{< admonition type="note">}} In Explore, you can map variables to custom values. {{< /admonition >}}

Known limitations

  • No validation is performed when you save a query, so it's possible to save an invalid query. You should confirm the query is working properly before you save it.
  • Saved queries are currently accessible from the query editors in Dashboards and Explore.
  • You can save a maximum of 1000 queries.
  • If you have multiple queries open in Explore and you edit one of them by way of the Edit in Explore function in the Saved queries dialog box, the edited query replaces your open queries in Explore.

{{< admonition type="note" >}} The screenshots of the panel query editor on this page reflect the new query editing experience, currently in public preview. For information about the classic, generally available query editor, refer to the Grafana v13.0 documentation.

Grafana Labs offers limited support for public preview features, and breaking changes might occur prior to the feature being made generally available. To use this feature, enable the queryEditorNext feature toggle in your Grafana configuration file or contact Support. {{< /admonition >}}

A panel's query editor has the following elements. Hover your cursor over a number to display information about the query editor element.

{{< image-map key="query-editor" >}}

Grafana organizes queries, expressions, and transformations into rows, each with its own card and editor pane. Queries and expressions are identified with a letter (A, B, C, and so on) by default.

You can also rearrange the order of elements by dragging and dropping the cards in the list.

When you hover your cursor over the cards, a blue plus sign appears beside the card. You can use that to add queries, expressions, or transformations in that position. Hovering your cursor over a card also displays duplicate, hide, and delete actions for a specific data element:

{{< figure src="/media/docs/grafana/panels-visualizations/screenshot-query-editor-card-actions-v13.1.png" max-width="250px" alt="Query card on hover showing available actions." >}}

For pipelines with many queries, expressions, or transformations, you can expand the sidebar to full height to see everything at once:

{{< figure src="/media/docs/grafana/panels-visualizations/screenshot-query-editor-expanded-v13.1.png" max-width="750px" alt="Query editor with expanded sidebar" >}}

You can also use the stacked view to see all of the data elements at once in the editor pane. In this view, when you click a card, it takes you to the corresponding element:

{{< figure src="/media/docs/grafana/panels-visualizations/screenshot-query-editor-stacked-view-v13.1.png" max-width="750px" alt="Query editor in stacked view" >}}

Add a query

A query returns data that Grafana visualizes in dashboard panels. When you create a panel, Grafana automatically selects the default data source.

To add a query to a panel, follow these steps:

  1. Hover the cursor over any part of the panel to which you're adding a query and click Configure visualization.

    The panel opens in edit mode with a first query card added and selected in the query editor pane.

  2. In the editor pane, click the Data source drop-down menu and select a data source.

    If you're creating a new dashboard, you'll be prompted to select a data source when you add the first panel.

  3. Click Query options to configure the maximum number of data points you need.

    For more information about query options, refer to Query options.

  4. To create a query, do one of the following:

    • Write or construct a query in the query language of your data source.
    • Click Replace to reuse a saved query.

    {{< admonition type="note" >}} Saved queries is currently in public preview. Grafana Labs offers limited support, and breaking changes might occur prior to the feature being made generally available.

    This feature is only available on Grafana Enterprise and Grafana Cloud. {{< /admonition >}}

  5. (Optional) To save the query for reuse, click the Save in the editor pane.

  6. (Optional) In the sidebar, click the blue plus sign next to Queries & Expressions and select Add query or Add saved query to add more queries as needed.

  7. Click Run queries in the editor pane.

Grafana queries the data source and visualizes the data.

Manage queries

The following table describes actions you can take for each query:

<!-- prettier-ignore-start -->
IconDescription
{{< figure src="/media/docs/grafana/panels-visualizations/replace-query-icon-v13.1.png" max-width="30px" max-height="30px" alt="Replace query icon" >}}Replace the configured query with a saved query. For more information, refer to Saved queries (Enterprise and Cloud only).
{{< figure src="/media/docs/grafana/panels-visualizations/save-to-query-icon.png" max-width="30px" max-height="30px" alt="Save query icon" >}}Save query. Saves the query so it can be reused. Access saved queries by clicking Replace. For more information, refer to Saved queries (Enterprise and Cloud only).
{{< figure src="/media/docs/grafana/panels-visualizations/create-recorded-query-icon.png" max-width="30px" max-height="30px" alt="Create recorded query icon" >}}Create recorded queries so you can see trends over time by taking a snapshot of a data point on a set interval (Enterprise and Cloud only).
{{< figure src="/static/img/docs/queries/hide-query-icon-7-0.png" max-width="30px" max-height="30px" alt="Hide icon" >}}Hides a query. Grafana does not send hidden queries to the data source.
{{< figure src="/static/img/docs/queries/remove-query-icon-7-0.png" max-width="30px" max-height="30px" alt="Remove icon">}}Removes a query. Removing a query permanently deletes it, but sometimes you can recover deleted queries by reverting to previously saved versions of the panel.
{{< figure src="/static/img/docs/queries/duplicate-query-icon-7-0.png" max-width="30px" max-height="30px" alt="Duplicate icon" >}}Copies a query. Duplicating queries is useful when working with multiple complex queries that are similar and you want to either experiment with different variants or do minor alterations. Click the More query actions menu to access this option.
{{< figure src="/static/img/docs/queries/query-editor-help-7-4.png" max-width="30px" max-height="30px" alt="Help icon" >}}Toggles query editor help. If supported by the data source, click this icon to display information on how to use the query editor or provide quick access to common queries. Click the More query actions menu to access this option.
<!-- prettier-ignore-end -->

Query options

Click Query options in the footer of the editor pane to see settings for the selected data source. Changes you make here affect only queries made in this panel.

{{< figure src="/media/docs/grafana/panels-visualizations/screenshot-query-options-v13.1.png" max-width="750px" alt="Data source query options" >}}

Grafana sets defaults that are shown in dark gray text. Changes are displayed in white text. To return a field to the default setting, delete the white text from the field.

Max data points

If the data source supports it, this sets the maximum number of data points for each series returned. If the query returns more data points than the max data points setting, then the data source reduces the number of points returned by aggregating them together by average, max, or another function.

You can limit the number of points to improve query performance or smooth the visualized line. The default value is the width (or number of pixels) of the graph, because you can only visualize as many data points as the graph panel has room to display.

Because the default depends on the panel’s pixel width, the same dashboard can show a different visual representation of the data depending on the panel’s size on screen. When you open a panel by clicking View in the panel menu, or by pressing v, the panel expands to fill the window. This doesn’t change the dashboard’s grid layout, but it gives the panel more pixels to render with. With more pixels, Grafana can request more data points, which increases the resolution and reduces the step size between points. As a result, the same query can return a more detailed series in view mode than in the dashboard grid.

To get a consistent visual representation in both views, set Max data points to a fixed value. For example, if you set it to 100, Grafana requests no more than 100 points, regardless of the panel’s pixel width. This means the query resolution remains more consistent whether the panel is in the dashboard grid or in view mode.

With streaming data, Grafana uses the max data points value for the rolling buffer. Streaming is a continuous flow of data, and buffering divides the stream into chunks. For example, Loki streams data in its live tailing mode.

Min interval

Sets a minimum limit for the automatically calculated interval, which is typically the minimum scrape interval. If a data point is saved every 15 seconds, you don't benefit from having an interval lower than that. You can also set this to a higher minimum than the scrape interval to retrieve queries that are more coarse-grained and well-functioning.

{{< admonition type="note" >}} The Min interval corresponds to the min step in Prometheus. Changing the Prometheus interval can change the start and end of the query range because Prometheus aligns the range to the interval. Refer to Min step for more details. {{< /admonition >}}

Interval

Sets a time span that you can use when aggregating or grouping data points by time.

Grafana automatically calculates an appropriate interval that you can use as a variable in templated queries. The variable is measured in either seconds ($__interval) or milliseconds ($__interval_ms).

Intervals are typically used in aggregation functions like sum or average. For example, this is a Prometheus query that uses the interval variable: rate(http_requests_total[$__interval]).

This automatic interval is calculated based on the width of the graph. As the user zooms out on a visualization, the interval grows, resulting in a more coarse-grained aggregation. Likewise, if the user zooms in, the interval decreases, resulting in a more fine-grained aggregation.

For more information, refer to Global variables.

Relative time

Overrides the relative time range for individual panels, which causes them to be different than what is selected in the dashboard time picker in the top-right corner of the dashboard. You can use this to show metrics from different time periods or days on the same dashboard.

{{< admonition type="note">}} Panel time overrides have no effect when the dashboard's time range is absolute. {{< /admonition >}}

ExampleRelative time field
Last 5 minutesnow-5m
The day so farnow/d
Last 5 daysnow-5d/d
This week so farnow/w
Last 2 yearsnow-2y/y

{{< docs/play title="Time range override" url="https://play.grafana.org/d/000000041/" >}}

Time shift

Overrides the time range for individual panels by shifting its start and end relative to the time picker. For example, you can shift the time range for the panel to be two hours earlier than the dashboard time picker.

{{< admonition type="note">}} Panel time overrides have no effect when the dashboard's time range is absolute. {{< /admonition >}}

ExampleTime shift field
Last entire week1w/w
Two entire weeks ago2w/w
Last entire month1M/M
This entire year1d/y
Last entire year1y/y

Cache timeout

This option is visible only if it's available in the data source. It overrides the default cache timeout if your time series store has a query cache. Specify this value as a numeric value in seconds.