BanyanDB self observability dashboard

BanyanDB, as an observability database, aims to ingest, analyze and store Metrics, Tracing, and Logging data. It's designed to handle observability data generated by Apache SkyWalking，it also provides a dashboard to visualize the self-observability metrics.

Data flow

1. BanyanDB collects metrics data internally and exposes a Prometheus http endpoint to retrieve the metrics.
2. OpenTelemetry Collector fetches metrics from BanyanDB and pushes metrics to SkyWalking OAP Server via OpenTelemetry gRPC exporter.
3. The SkyWalking OAP Server parses the expression with MAL to filter/calculate/aggregate and store the results.

Set up

1. Start BanyanDB,supporting both Standalone Mode and Cluster Mode.
2. Set up OpenTelemetry Collector . For details on Prometheus Receiver in OpenTelemetry Collector, refer to here.
3. Config SkyWalking OpenTelemetry receiver.

BanyanDB monitoring

Self observability monitoring provides monitoring of the status and resources of the BanyanDB server itself. banyandb-server is a Service in BanyanDB, and land on the Layer: BANYANDB.

Self observability metrics

Unit	Metric Name	Description	Data Source
o/s	meter_banyandb_write_rate	Write Rate (Operations per Second)	BanyanDB
GiB	meter_banyandb_total_memory	Total Memory	BanyanDB
GiB	meter_banyandb_disk_usage	Disk Usage	BanyanDB
r/s	meter_banyandb_query_rate	Query Rate (Requests per Second)	BanyanDB
Count	meter_banyandb_total_cpu	Total CPU Cores	BanyanDB
c/m	meter_banyandb_write_and_query_errors_rate	Write and Query Errors Rate（Counts per Minute）	BanyanDB
c/s	meter_banyandb_etcd_operation_rate	Etcd Operation Rate（Counts per Second）	BanyanDB
Count	meter_banyandb_active_instance	Active Instances	BanyanDB
%	meter_banyandb_cpu_usage	CPU Usage Percentage	BanyanDB
%	meter_banyandb_rss_memory_usage	RSS Memory Usage Percentage	BanyanDB
%	meter_banyandb_disk_usage_all	Disk Usage Percentage	BanyanDB
KiB/s	meter_banyandb_network_usage_recv	Network Receive Rate	BanyanDB
KiB/s	meter_banyandb_network_usage_sent	Network Send Rate	BanyanDB
o/s	meter_banyandb_storage_write_rate	Storage Write Rate (Operations per Second)	BanyanDB
s	meter_banyandb_query_latency	Query Latency (s)	BanyanDB
Count	meter_banyandb_total_data	Total Data Elements	BanyanDB
r/m	meter_banyandb_merge_file_data	Merge File Data Rate(Revolutions per Minute)	BanyanDB
s	meter_banyandb_merge_file_latency	Merge File Latency(s)	BanyanDB
Count	meter_banyandb_merge_file_partitions	Merge File Partitions	BanyanDB
o/s	meter_banyandb_series_write_rate	Series Write Rate (Operations per Second)	BanyanDB
o/s	meter_banyandb_series_term_search_rate	Series Term Search Rate (Operations per Second)	BanyanDB
Count	meter_banyandb_total_series	Total Series Count	BanyanDB
ops	meter_banyandb_stream_write_rate	Stream Write Rate (Operations per Second)	BanyanDB
ops	meter_banyandb_term_search_rate	Term Search Rate (Operations per Second)	BanyanDB
Count	meter_banyandb_total_document	Total Document Count	BanyanDB

Customizations

You can customize your own metrics/expression/dashboard panel.The metrics definition and expression rules are found in /config/otel-rules/banyandb.The BanyanDB dashboard panel configurations are found in /config/ui-initialized-templates/banyandb.