docs/en/10-third-party/03-visual/01-grafana.md
import Tabs from "@theme/Tabs"; import TabItem from "@theme/TabItem";
This document describes how to integrate the TDengine data source with the open-source data visualization system Grafana to achieve data visualization and build a monitoring and alert system. With the TDengine plugin, you can easily display data from TDengine tables on Grafana dashboards without the need for complex development work.
To add the TDengine data source to Grafana normally, the following preparations are needed.
Record the following information:
http://tdengine.local:6041.For users using Grafana version 8.x or later or configuring with Grafana Provisioning, you can use the installation script on the Grafana server to automatically install the plugin and add the data source Provisioning configuration file.
bash -c "$(curl -fsSL \
https://raw.githubusercontent.com/taosdata/grafanaplugin/master/install.sh)" -- \
-a http://localhost:6041 \
-u root \
-p taosdata
After installation, you need to restart the Grafana service for it to take effect.
Save the script and execute ./install.sh --help to view detailed help documentation.
Use the grafana-cli command line tool to install the plugin. After installation, Grafana needs to be restarted.
On Linux or macOS, run the following command in your terminal:
grafana-cli --pluginUrl \
https://www.tdengine.com/assets-download/grafana-plugin/tdengine-datasource.zip \
plugins install tdengine-datasource
# with sudo
sudo -u grafana grafana-cli --pluginUrl \
https://www.tdengine.com/assets-download/grafana-plugin/tdengine-datasource.zip \
plugins install tdengine-datasource
On Windows, first ensure that the plugin installation directory exists (by default, it is located in the data/plugins subdirectory of your Grafana installation directory). Then, run the following command in the bin directory of the Grafana installation path using an administrator account:
./grafana-cli.exe --pluginUrl https://www.tdengine.com/assets-download/grafana-plugin/tdengine-datasource.zip plugins install tdengine-datasource
Afterward, users can directly access the Grafana server at http://localhost:3000 (username/password: admin/admin), and add a data source through Configuration -> Data Sources,
Click Add data source to enter the new data source page, type TDengine in the search box, then click select to choose and you will enter the data source configuration page, modify the configuration according to the default prompts:
http://localhost:6041Click Save & Test to test, if successful, it will prompt: TDengine Data source is working
Download tdengine-datasource.zip to your local machine and unzip it into the Grafana plugins directory. Example command line download is as follows:
wget https://www.tdengine.com/assets-download/grafana-plugin/tdengine-datasource.zip
For CentOS 7.2 operating system, unzip the plugin package into the /var/lib/grafana/plugins directory and restart Grafana.
sudo unzip tdengine-datasource.zip -d /var/lib/grafana/plugins/
Afterward, users can directly access the Grafana server at http://localhost:3000 (username/password: admin/admin), and add a data source through Configuration -> Data Sources,
Click Add data source to enter the new data source page, type TDengine in the search box, then click select to choose and you will enter the data source configuration page, modify the configuration according to the default prompts:
http://localhost:6041Click Save & Test to test, if successful, it will prompt: TDengine Data source is working
Refer to Grafana containerized installation instructions. Use the following command to start a container and automatically install the TDengine plugin:
docker run -d \
-p 3000:3000 \
--name=grafana \
-e "GF_INSTALL_PLUGINS=https://www.tdengine.com/assets-download/grafana-plugin/tdengine-datasource.zip;tdengine-datasource" \
grafana/grafana
Using docker-compose, configure Grafana Provisioning for automated setup, and experience a zero-configuration start with TDengine + Grafana:
Save this file as tdengine.yml.
apiVersion: 1
datasources:
- name: TDengine
type: tdengine-datasource
orgId: 1
url: "$TDENGINE_API"
isDefault: true
secureJsonData:
url: "$TDENGINE_API"
basicAuth: "$TDENGINE_BASIC_AUTH"
version: 1
editable: true
Save the file as docker-compose.yml.
version: "3.7"
services:
tdengine:
image: tdengine/tdengine:latest
container_name: tdengine
hostname: tdengine
environment:
TAOS_FQDN: tdengine
MONITOR_FQDN: tdengine
EXPLORER_CLUSTER: http://tdengine:6041
TAOS_KEEPER_TDENGINE_HOST: tdengine
volumes:
- tdengine-data:/var/lib/taos/
ports:
- 6060:6060
grafana:
image: grafana/grafana:latest
volumes:
- ./tdengine.yml:/etc/grafana/provisioning/datasources/tdengine.yml
- grafana-data:/var/lib/grafana
environment:
# install tdengine plugin at start
GF_INSTALL_PLUGINS: "https://www.tdengine.com/assets-download/grafana-plugin/tdengine-datasource.zip;tdengine-datasource"
TDENGINE_API: "http://tdengine:6041"
#printf "$TDENGINE_USER:$TDENGINE_PASSWORD" | base64
TDENGINE_BASIC_AUTH: "cm9vdDp0YW9zZGF0YQ=="
ports:
- 3000:3000
volumes:
grafana-data:
tdengine-data:
Use the docker-compose command to start TDengine + Grafana: docker-compose up -d.
Open Grafana http://localhost:3000, and now you can add Dashboards.
</TabItem> </Tabs>:::info
In the following text, we use Grafana v11.0.0 as an example. Other versions may have different features, please refer to Grafana's official website.
:::
This section is organized as follows:
The Variable feature in Grafana is very powerful and can be used in Dashboard queries, panel titles, tags, etc., to create more dynamic and interactive Dashboards, enhancing user experience and efficiency.
The main functions and features of variables include:
Dynamic data querying: Variables can be used in query statements, allowing users to dynamically change query conditions by selecting different variable values, thus viewing different data views. This is very useful for scenarios that require dynamically displaying data based on user input.
Improved reusability: By defining variables, the same configuration or query logic can be reused in multiple places without having to rewrite the same code. This makes maintaining and updating Dashboards simpler and more efficient.
Flexible configuration options: Variables offer a variety of configuration options, such as predefined static value lists, dynamically querying values from data sources, regular expression filtering, etc., making the application of variables more flexible and powerful.
Grafana provides both built-in and custom variables, which can be referenced when writing SQL as $variableName, where variableName is the name of the variable. For other referencing methods, please refer to Referencing Methods.
Grafana has built-in variables such as from, to, and interval, all derived from the Grafana plugin panel. Their meanings are as follows:
from is the start time of the query rangeto is the end time of the query rangeinterval is the window split intervalFor each query, it is recommended to set the start and end time of the query range, which can effectively reduce the amount of data scanned by the TDengine server during query execution. interval is the size of the window split, and in Grafana version 11, it is calculated based on the time interval and the number of returned points.
In addition to the three common variables mentioned above, Grafana also provides variables such as __timezone, __org, __user, etc. For more details, please refer to Built-in Variables.
We can add custom variables in the Dashboard. The usage of custom variables is no different from built-in variables; they are referenced in SQL with $variableName.
Custom variables support multiple types, including common types such as Query (query), Constant (constant), Interval (interval), Data source (data source), etc.
Custom variables can reference other custom variables, for example, one variable represents a region, and another variable can reference the value of the region to query devices in that region.
In the Dashboard configuration, select Variables, then click New variable:
selected_groups.select distinct(groupid) from power.meters where groupid < 3 and ts > $from and ts < $to;After completing the above steps, we have successfully added a new custom variable $selected_groups in the Dashboard. We can later reference this variable in the Dashboard's queries through $selected_groups.
We can also add another custom variable to reference this selected_groups variable, such as adding a query variable named tbname_max_current, with its SQL as select tbname from power.meters where groupid = $selected_groups and ts > $from and ts < $to;
We can customize the time window interval to better fit business needs.
interval.1s,2s,5s,10s,15s,30s,1m.After completing the above steps, we have successfully added a new custom variable $interval in the Dashboard. We can later reference this variable in the Dashboard's queries through $interval.
:::note
When custom variables and Grafana built-in variables have the same name, custom variables are referenced preferentially.
:::
On top of supporting standard SQL, TDengine also offers a series of special query syntaxes that meet the needs of time-series business scenarios, greatly facilitating the development of applications for time series scenarios.
partition by clause can split data by certain dimensions, then perform a series of calculations within the split data space, often replacing group by.interval clause is used to generate windows of equal time periods.fill statement specifies the filling mode for missing data in a window interval.Timestamp pseudocolumns If you need to output the time window information corresponding to the aggregation results in the results, you need to use timestamp-related pseudo-columns in the SELECT clause: window start time (_wstart), window end time (_wend), etc.Detailed introduction to the above features can be found at Distinguished Queries.
With the foundational knowledge from earlier, we can configure a time-series data display Dashboard based on the TDengine data source.
Create a Dashboard on the Grafana main interface, click on Add Query to enter the panel query page:
As shown in the image above, select the TDengine data source in "Query", and enter the corresponding SQL in the query box below. Continuing with the example of smart meters, to display a beautiful curve, virtual data is used here.
Suppose we want to query the average current size over a period of time, with the time window divided by $interval, and fill with null if data is missing in any time window.
select _wstart as ts, avg(current) as current from power.meters where groupid in ($selected_groups) and ts > $from and ts < $to interval($interval) fill(null), where from, to, and interval are Grafana built-in variables, and selected_groups is a custom variable.In the custom variables at the top, if the value of selected_groups is set to 1, then querying the average value changes of all devices' current in the meters supertable with groupid 1 is shown in the following image:
:::note
Since the REST interface is stateless, you cannot use the use db statement to switch databases. In the Grafana plugin, SQL statements can specify the database using <db_name>.<table_name>.
:::
Suppose we want to query the average current size over a period of time and display it grouped by groupid, we can modify the previous SQL to select _wstart as ts, groupid, avg(current) as current from power.meters where ts > $from and ts < $to partition by groupid interval($interval) fill(null)
group by or partition by column names in half-width commas. If it is a group by or partition by query statement, set the "Group by" column to display multidimensional data. Here, set the "Group by" column name as groupid to display data grouped by groupid.Group by or Partition by scenarios. For example, in the above INPUT SQL, set the "Group By Format" to groupid-{{groupid}}, and the displayed legend name will be the formatted group name.After completing the settings, the display grouped by groupid is shown in the following image:
For information on how to use Grafana to create corresponding monitoring interfaces and more about using Grafana, please refer to the official documentation of Grafana.
select column_name from db.table where ts > $from and ts < $to;CACHEMODEL set to last_row or both), a common SQL syntax is select last(column_name) from db.table where ts > $from and ts < $to;On the data source configuration page, you can import the TDinsight panel for this data source, serving as a monitoring visualization tool for the TDengine cluster. If the TDengine server is version 3.0, please select TDinsight for 3.x for import. Note that TDinsight for 3.x requires running and configuring taoskeeper.
The Dashboard compatible with TDengine 2.* has been released on Grafana: Dashboard 15167 - TDinsight).
Other panels using TDengine as a data source can be searched here. Below is a non-exhaustive list:
The TDengine Grafana plugin supports alerts. To configure alerts, follow these steps:
In Grafana 11, the alert interface has 6 tabs, namely "Alert rules", "Contact points", "Notification policies", "Silences", "Groups", and "Settings".
This section uses email and Lark as examples to configure contact points.
Add the SMTP/Emailing and Alerting modules to the configuration file of the Grafana service. (For Linux systems, the configuration file is usually located at /etc/grafana/grafana.ini)
Add the following content to the configuration file:
#################################### SMTP / Emailing ##########################
[smtp]
enabled = true
host = smtp.qq.com:465 #Email used
user = [email protected]
password = *********** #Use mail authorization code
skip_verify = true
from_address = [email protected]
Then restart the Grafana service (for Linux systems, execute systemctl restart grafana-server.service) to complete the addition.
On the Grafana page, go to "Home" -> "Alerting" -> "Contact points" and create a new contact point
"Name": Email Contact Point
"Integration": Select the contact type, here choose Email, fill in the email receiving address, and save the contact point after completion
After configuring the contact points, you can see there is a Default Policy
Click on the right side "..." -> "Edit", then edit the default notification policy, a configuration window pops up:
Then configure the following parameters:
Taking the configuration of smart meter alerts as an example, the configuration of alert rules mainly includes alert name, query and alert trigger conditions, rule evaluation strategy, tags, alert channels, and notification copy.
In the panel where you need to configure the alert, select "Edit" -> "Alert" -> "New alert rule".
"Enter alert rule name" (input alert rule name): Here, enter power meters alert as an example
In "Define query and alert condition" configure the alert rule.
Choose data source: TDengine Datasource
Query statement:
select _wstart as ts, groupid, avg(current) as current from power.meters where ts > $from and ts < $to partition by groupid interval($interval) fill(null)
Set "Expression" (expression): Threshold is above 100
Click [Set as alert condition]
"Preview": View the results of the set rules
After setting, you can see the image displayed below:
Grafana's "Expression" (expression) supports various operations and calculations on data, which are divided into:
As shown in the screenshot above, here we set the maximum value to trigger an alarm when it exceeds 100.
Complete the following configurations:
Complete the following configurations:
Set "Summary" and "Description", and if an alert is triggered, you will receive a notification.