Back to Redis

Prometheus Metrics V2

content/operate/rs/7.8/references/metrics/prometheus-metrics-v2.md

latest28.3 KB
Original Source

{{<banner-article>}} While the metrics stream engine is in preview, this document provides only a partial list of v2 metrics. More metrics will be added. {{</banner-article>}}

You can [integrate Redis Enterprise Software with Prometheus and Grafana]({{<relref "/operate/rs/7.8/monitoring/prometheus_and_grafana">}}) to create dashboards for important metrics.

The v2 metrics in the following tables are available as of Redis Enterprise Software version 7.8.0. For help transitioning from v1 metrics to v2 PromQL, see [Prometheus v1 metrics and equivalent v2 PromQL]({{<relref "/operate/rs/7.8/references/metrics/prometheus-metrics-v1-to-v2">}}).

The v2 scraping endpoint also exposes metrics for node_exporter version 1.8.1. For more information, see the Prometheus node_exporter GitHub repository.

Database metrics

MetricTypeDescription
<span class="break-all">endpoint_client_connections</span>counterNumber of client connection establishment events
<span class="break-all">endpoint_client_disconnections</span>counterNumber of client disconnections initiated by the client
<span class="break-all">endpoint_client_connection_expired</span>counterTotal number of client connections with expired TTL (Time To Live)
<span class="break-all">endpoint_client_establishment_failures</span>counterNumber of client connections that failed to establish properly
<span class="break-all">endpoint_client_expiration_refresh</span>counterNumber of expiration time changes of clients
<span class="break-all">endpoint_client_tracking_off_requests</span>counterTotal number of CLIENT TRACKING OFF requests
<span class="break-all">endpoint_client_tracking_on_requests</span>counterTotal number of CLIENT TRACKING ON requests
<span class="break-all">endpoint_disconnected_cba_client</span>counterNumber of certificate-based clients disconnected
<span class="break-all">endpoint_disconnected_ldap_client</span>counterNumber of LDAP clients disconnected
<span class="break-all">endpoint_disconnected_user_password_client</span>counterNumber of user&password clients disconnected
<span class="break-all">endpoint_disposed_commands_after_client_caching</span>counterTotal number of client caching commands that were disposed due to misuse
<span class="break-all">endpoint_egress</span>counterNumber of egress bytes
<span class="break-all">endpoint_egress_pending</span>counterNumber of send-pending bytes
<span class="break-all">endpoint_egress_pending_discarded</span>counterNumber of send-pending bytes that were discarded due to disconnection
<span class="break-all">endpoint_failed_cba_authentication</span>counterNumber of clients that failed certificate-based authentication
<span class="break-all">endpoint_failed_ldap_authentication</span>counterNumber of clients that failed LDAP authentication
<span class="break-all">endpoint_failed_user_password_authentication</span>counterNumber of clients that failed user password authentication
<span class="break-all">endpoint_ingress</span>counterNumber of ingress bytes
<span class="break-all">endpoint_longest_pipeline_histogram</span>counterTracks the distribution of longest observed pipeline lengths, where a pipeline is a sequence of client commands sent without waiting for responses.
<span class="break-all">endpoint_other_requests</span>counterNumber of other requests
<span class="break-all">endpoint_other_requests_latency_histogram</span>histogramLatency (in µs) histogram of other commands
<span class="break-all">endpoint_other_requests_latency_histogram_bucket</span>histogramLatency histograms for commands other than read or write commands. Can be used to represent different latency percentiles.
p99.9 example:
<span class="break-all">histogram_quantile(0.999, sum(rate(endpoint_other_requests_latency_histogram_bucket{cluster="$cluster", db="$db"}[$__rate_interval]) ) by (le, db))</span>
<span class="break-all">endpoint_other_responses</span>counterNumber of other responses
<span class="break-all">endpoint_proxy_disconnections</span>counterNumber of client disconnections initiated by the proxy
<span class="break-all">endpoint_read_requests</span>counterNumber of read requests
<span class="break-all">endpoint_read_requests_latency_histogram</span>histogramLatency (in µs) histogram of read commands
<span class="break-all">endpoint_read_requests_latency_histogram_bucket</span>histogramLatency histograms for read commands. Can be used to represent different latency percentiles.
p99.9 example:
<span class="break-all">histogram_quantile(0.999, sum(rate(endpoint_read_requests_latency_histogram_bucket{cluster="$cluster", db="$db"}[$__rate_interval]) ) by (le, db))</span>
<span class="break-all">endpoint_read_responses</span>counterNumber of read responses
<span class="break-all">endpoint_successful_cba_authentication</span>counterNumber of clients that successfully authenticated with certificate-based authentication
<span class="break-all">endpoint_successful_ldap_authentication</span>counterNumber of clients that successfully authenticated with LDAP
<span class="break-all">endpoint_successful_user_password_authentication</span>counterNumber of clients that successfully authenticated with user&password
<span class="break-all">endpoint_write_requests</span>counterNumber of write requests
<span class="break-all">endpoint_write_requests_latency_histogram</span>histogramLatency (in µs) histogram of write commands
<span class="break-all">endpoint_write_requests_latency_histogram_bucket</span>histogramLatency histograms for write commands. Can be used to represent different latency percentiles.
p99.9 example:
<span class="break-all">histogram_quantile(0.999, sum(rate(endpoint_write_requests_latency_histogram_bucket{cluster="$cluster", db="$db"}[$__rate_interval]) ) by (le, db))</span>
<span class="break-all">endpoint_write_responses</span>counterNumber of write responses

Node metrics

MetricTypeDescription
<span class="break-all">node_available_flash_bytes</span>gaugeAvailable flash in the node (bytes)
<span class="break-all">node_available_flash_no_overbooking_bytes</span>gaugeAvailable flash in the node (bytes), without taking into account overbooking
<span class="break-all">node_available_memory_bytes</span>gaugeAmount of free memory in the node (bytes) that is available for database provisioning
<span class="break-all">node_available_memory_no_overbooking_bytes</span>gaugeAvailable RAM in the node (bytes) without taking into account overbooking
<span class="break-all">node_bigstore_free_bytes</span>gaugeSum of free space of back-end flash (used by flash database's [BigRedis]) on all cluster nodes (bytes); returned only when BigRedis is enabled
<span class="break-all">node_cert_expires_in_seconds</span>gaugeCertificate expiration (in seconds) per given node; read more about [certificates in Redis Enterprise]({{< relref "/operate/rs/7.8/security/certificates" >}}) and [monitoring certificates]({{< relref "/operate/rs/7.8/security/certificates/monitor-certificates" >}})
<span class="break-all">node_ephemeral_storage_avail_bytes</span>gaugeDisk space available to RLEC processes on configured ephemeral disk (bytes)
<span class="break-all">node_ephemeral_storage_free_bytes</span>gaugeFree disk space on configured ephemeral disk (bytes)
<span class="break-all">node_memory_MemFree_bytes</span>gaugeFree memory in the node (bytes)
<span class="break-all">node_persistent_storage_avail_bytes</span>gaugeDisk space available to RLEC processes on configured persistent disk (bytes)
<span class="break-all">node_persistent_storage_free_bytes</span>gaugeFree disk space on configured persistent disk (bytes)
<span class="break-all">node_provisional_flash_bytes</span>gaugeAmount of flash available for new shards on this node, taking into account overbooking, max Redis servers, reserved flash, and provision and migration thresholds (bytes)
<span class="break-all">node_provisional_flash_no_overbooking_bytes</span>gaugeAmount of flash available for new shards on this node, without taking into account overbooking, max Redis servers, reserved flash, and provision and migration thresholds (bytes)
<span class="break-all">node_provisional_memory_bytes</span>gaugeAmount of RAM that is available for provisioning to databases out of the total RAM allocated for databases
<span class="break-all">node_provisional_memory_no_overbooking_bytes</span>gaugeAmount of RAM that is available for provisioning to databases out of the total RAM allocated for databases, without taking into account overbooking
<span class="break-all">node_metrics_up</span>gaugeNode is part of the cluster and is connected

Cluster metrics

MetricTypeDescription
<span class="break-all">generation{cluster_wd=<node_uid>}</span>gaugeGeneration number of the specific cluster_wd
<span class="break-all">has_quorum{cluster_wd=<node_uid>, has_witness_disk=BOOL}</span>gaugeHas_quorum = 1
No quorum = 0
<span class="break-all">is_primary{cluster_wd=<node_uid>}</span>gaugeprimary = 1
secondary = 0
<span class="break-all">license_shards_limit</span>gaugeTotal shard limit by the license by shard type (ram / flash)
<span class="break-all">total_live_nodes_count{cluster_wd=<node_uid>}</span>gaugeNumber of live nodes
<span class="break-all">total_node_count{cluster_wd=<node_uid>}</span>gaugeNumber of nodes
<span class="break-all">total_primary_selection_ended{cluster_wd=<node_uid>}</span>counterMonotonic counter for each selection process that ended
<span class="break-all">total_primary_selections{cluster_wd=<node_uid>}</span>counterMonotonic counter for each selection process that started

Replication metrics

MetricTypeDescription
<span class="break-all">database_syncer_config</span>gaugeUsed as a placeholder for configuration labels
<span class="break-all">database_syncer_current_status</span>gaugeSyncer status for traffic; 0 = in-sync, 2 = out of sync
<span class="break-all">database_syncer_dst_connectivity_state</span>gaugeDestination connectivity state
<span class="break-all">database_syncer_dst_connectivity_state_ms</span>gaugeDestination connectivity state duration
<span class="break-all">database_syncer_dst_lag</span>gaugeLag in milliseconds between the syncer and the destination
<span class="break-all">database_syncer_dst_repl_offset</span>gaugeOffset of the last command acknowledged
<span class="break-all">database_syncer_flush_counter</span>gaugeNumber of destination flushes
<span class="break-all">database_syncer_ingress_bytes</span>gaugeNumber of bytes read from source shard
<span class="break-all">database_syncer_ingress_bytes_decompressed</span>gaugeNumber of bytes read from source shard
<span class="break-all">database_syncer_internal_state</span>gaugeInternal state of the syncer
<span class="break-all">database_syncer_lag_ms</span>gaugeLag time between the source and the destination for traffic in milliseconds
<span class="break-all">database_syncer_rdb_size</span>gaugeThe source's RDB size in bytes to be transferred during the syncing phase
<span class="break-all">database_syncer_rdb_transferred</span>gaugeNumber of bytes transferred from the source's RDB during the syncing phase
<span class="break-all">database_syncer_src_connectivity_state</span>gaugeSource connectivity state
<span class="break-all">database_syncer_src_connectivity_state_ms</span>gaugeSource connectivity state duration
<span class="break-all">database_syncer_src_repl_offset</span>gaugeLast known source offset
<span class="break-all">database_syncer_state</span>gaugeInternal state of the shard syncer
<span class="break-all">database_syncer_syncer_repl_offset</span>gaugeOffset of the last command handled by the syncer
<span class="break-all">database_syncer_total_requests</span>gaugeNumber of destination writes
<span class="break-all">database_syncer_total_responses</span>gaugeNumber of destination writes acknowledged

Shard metrics

MetricDescription
<span class="break-all">redis_server_active_defrag_running</span>Automatic memory defragmentation current aggressiveness (% cpu)
<span class="break-all">redis_server_allocator_active</span>Total used memory, including external fragmentation
<span class="break-all">redis_server_allocator_allocated</span>Total allocated memory
<span class="break-all">redis_server_allocator_resident</span>Total resident memory (RSS)
<span class="break-all">redis_server_aof_last_cow_size</span>Last AOFR, CopyOnWrite memory
<span class="break-all">redis_server_aof_rewrite_in_progress</span>The number of simultaneous AOF rewrites that are in progress
<span class="break-all">redis_server_aof_rewrites</span>Number of AOF rewrites this process executed
<span class="break-all">redis_server_aof_delayed_fsync</span>Number of times an AOF fsync caused delays in the main Redis thread (inducing latency); this can indicate that the disk is slow or overloaded
<span class="break-all">redis_server_blocked_clients</span>Count the clients waiting on a blocking call
<span class="break-all">redis_server_connected_clients</span>Number of client connections to the specific shard
<span class="break-all">redis_server_connected_slaves</span>Number of connected replicas
<span class="break-all">redis_server_db0_avg_ttl</span>Average TTL of all volatile keys
<span class="break-all">redis_server_db0_keys</span>Total key count
<span class="break-all">redis_server_evicted_keys</span>Keys evicted so far (since restart)
<span class="break-all">redis_server_expire_cycle_cpu_milliseconds</span>The cumulative amount of time spent on active expiry cycles
<span class="break-all">redis_server_expired_keys</span>Keys expired so far since restart
<span class="break-all">redis_server_forwarding_state</span>Shard forwarding state (on or off)
<span class="break-all">redis_server_keys_trimmed</span>The number of keys that were trimmed in the current or last resharding process
<span class="break-all">redis_server_keyspace_read_hits</span>Number of read operations accessing an existing keyspace
<span class="break-all">redis_server_keyspace_read_misses</span>Number of read operations accessing a non-existing keyspace
<span class="break-all">redis_server_keyspace_write_hits</span>Number of write operations accessing an existing keyspace
<span class="break-all">redis_server_keyspace_write_misses</span>Number of write operations accessing a non-existing keyspace
<span class="break-all">redis_server_master_link_status</span>Indicates if the replica is connected to its master
<span class="break-all">redis_server_master_repl_offset</span>Number of bytes sent to replicas by the shard; calculate the throughput for a time period by comparing the value at different times
<span class="break-all">redis_server_master_sync_in_progress</span>The primary shard is synchronizing (1 true; 0 false)
<span class="break-all">redis_server_max_process_mem</span>Current memory limit configured by redis_mgr according to node free memory
<span class="break-all">redis_server_maxmemory</span>Current memory limit configured by redis_mgr according to database memory limits
<span class="break-all">redis_server_mem_aof_buffer</span>Current size of AOF buffer
<span class="break-all">redis_server_mem_clients_normal</span>Current memory used for input and output buffers of non-replica clients
<span class="break-all">redis_server_mem_clients_slaves</span>Current memory used for input and output buffers of replica clients
<span class="break-all">redis_server_mem_fragmentation_ratio</span>Memory fragmentation ratio (1.3 means 30% overhead)
<span class="break-all">redis_server_mem_not_counted_for_evict</span>Portion of used_memory (in bytes) that's not counted for eviction and OOM error
<span class="break-all">redis_server_mem_replication_backlog</span>Size of replication backlog
<span class="break-all">redis_server_module_fork_in_progress</span>A binary value that indicates if there is an active fork spawned by a module (1) or not (0)
<span class="break-all">namedprocess_namegroup_cpu_seconds_total</span>Shard process CPU usage in seconds
<span class="break-all">namedprocess_namegroup_thread_cpu_seconds_total</span>Shard main thread CPU time spent in seconds
<span class="break-all">namedprocess_namegroup_open_filedesc</span>Shard number of open file descriptors
<span class="break-all">namedprocess_namegroup_memory_bytes</span>Shard memory size in bytes
<span class="break-all">namedprocess_namegroup_oldest_start_time_seconds</span>Shard start time of the process since unix epoch in seconds
<span class="break-all">redis_server_rdb_bgsave_in_progress</span>Indication if bgsave is currently in progress
<span class="break-all">redis_server_rdb_last_cow_size</span>Last bgsave (or SYNC fork) used CopyOnWrite memory
<span class="break-all">redis_server_rdb_saves</span>Total count of bgsaves since the process was restarted (including replica fullsync and persistence)
<span class="break-all">redis_server_repl_touch_bytes</span>Number of bytes sent to replicas as TOUCH commands by the shard as a result of a READ command that was processed; calculate the throughput for a time period by comparing the value at different times
<span class="break-all">redis_server_total_commands_processed</span>Number of commands processed by the shard; calculate the number of commands for a time period by comparing the value at different times
<span class="break-all">redis_server_total_connections_received</span>Number of connections received by the shard; calculate the number of connections for a time period by comparing the value at different times
<span class="break-all">redis_server_total_net_input_bytes</span>Number of bytes received by the shard; calculate the throughput for a time period by comparing the value at different times
<span class="break-all">redis_server_total_net_output_bytes</span>Number of bytes sent by the shard; calculate the throughput for a time period by comparing the value at different times
<span class="break-all">redis_server_up</span>Shard is up and running
<span class="break-all">redis_server_used_memory</span>Memory used by shard (in BigRedis this includes flash) (bytes)
<span class="break-all">redis_server_search_number_of_indexes</span>Total number of indexes in the shard <sup>1</sup>
<span class="break-all">redis_server_search_number_of_active_indexes</span>The total number of indexes running a background indexing and/or background query processing operation. Background indexing refers to vector ingestion process, or in-progress background indexer. <sup>1</sup>
<span class="break-all">redis_server_search_number_of_active_indexes_running_queries</span>Total count of indexes currently running a background query process. <sup>1</sup>
<span class="break-all">redis_server_search_number_of_active_indexes_indexing</span>Total count of indexes currently undergoing a background indexing process. Background indexing refers to vector ingestion process, or in-progress background indexer. This metric is limited by the number of WORKER threads allocated for writing operations + the number of indexes. <sup>1</sup>
<span class="break-all">redis_server_search_total_active_write_threads</span>Total count of background write (indexing) processes currently running in the shard. Background indexing refers to vector ingestion process, or in-progress background indexer. This metric is limited by the number of threads allocated for writing operations. <sup>1</sup>
<span class="break-all">redis_server_search_fields_text_Text</span>The total number of TEXT fields across all indexes in the shard. <sup>1</sup>
<span class="break-all">redis_server_search_fields_text_Sortable</span>The total number of SORTABLE TEXT fields across all indexes in the shard. This field appears only if its value is larger than 0. <sup>1</sup>
<span class="break-all">redis_server_search_fields_text_NoIndex</span>The total number of NOINDEX TEXT fields across all indexes in the shard; i.e., used for sorting only but not indexed. This field appears only if its value is larger than 0. <sup>1</sup>
<span class="break-all">redis_server_search_fields_numeric_Numeric</span>The total number of NUMERIC fields across all indexes in the shard. <sup>1</sup>
<span class="break-all">redis_server_search_fields_numeric_Sortable</span>The total number of SORTABLE NUMERIC fields across all indexes in the shard. This field appears only if its value is larger than 0. <sup>1</sup>
<span class="break-all">redis_server_search_fields_numeric_NoIndex</span>The total number of NOINDEX NUMERIC fields across all indexes in the shard, which are used for sorting only but not indexed. This field appears only if its value is larger than 0. <sup>1</sup>
<span class="break-all">redis_server_search_fields_tag_Tag</span>The total number of TAG fields across all indexes in the shard. <sup>1</sup>
<span class="break-all">redis_server_search_fields_tag_Sortable</span>The total number of SORTABLE TAG fields across all indexes in the shard. This field appears only if its value is larger than 0. <sup>1</sup>
<span class="break-all">redis_server_search_fields_tag_NoIndex</span>The total number of NOINDEX TAG fields across all indexes in the shard; i.e., used for sorting only but not indexed. This field appears only if its value is larger than 0. <sup>1</sup>
<span class="break-all">redis_server_search_fields_tag_CaseSensitive</span>The total number of CASESENSITIVE TAG fields across all indexes in the shard. This field appears only if its value is larger than 0. <sup>1</sup>
<span class="break-all">redis_server_search_fields_geo_Geo</span>The total number of GEO fields across all indexes in the shard. <sup>1</sup>
<span class="break-all">redis_server_search_fields_geo_Sortable</span>The total number of SORTABLE GEO fields across all indexes in the shard. This field appears only if its value is larger than 0. <sup>1</sup>
<span class="break-all">redis_server_search_fields_geo_NoIndex</span>The total number of NOINDEX GEO fields across all indexes in the shard; i.e., used for sorting only but not indexed. This field appears only if its value is larger than 0. <sup>1</sup>
<span class="break-all">redis_server_search_fields_vector_Vector</span>The total number of VECTOR fields across all indexes in the shard. <sup>1</sup>
<span class="break-all">redis_server_search_fields_vector_Flat</span>The total number of FLAT VECTOR fields across all indexes in the shard. <sup>1</sup>
<span class="break-all">redis_server_search_fields_vector_HNSW</span>The total number of HNSW VECTOR fields across all indexes in the shard. <sup>1</sup>
<span class="break-all">redis_server_search_fields_geoshape_Geoshape</span>The total number of GEOSHAPE fields across all indexes in the shard. <sup>2</sup>
<span class="break-all">redis_server_search_fields_geoshape_Sortable</span>The total number of SORTABLE GEOSHAPE fields across all indexes in the shard. This field appears only if its value is larger than 0. <sup>2</sup>
<span class="break-all">redis_server_search_fields_geoshape_NoIndex</span>The total number of NOINDEX GEOSHAPE fields across all indexes in the shard; i.e., used for sorting only but not indexed. This field appears only if its value is larger than 0. <sup>2</sup>
<span class="break-all">redis_server_search_fields_<field>_IndexErrors</span>The total number of indexing failures caused by attempts to index a document containing <field> field. <sup>1</sup>
<span class="break-all">redis_server_search_used_memory_indexes</span>The total memory allocated by all indexes in the shard in bytes. <sup>1</sup>
<span class="break-all">redis_server_search_smallest_memory_index</span>The memory usage of the index with the smallest memory usage in the shard in bytes. <sup>1</sup>
<span class="break-all">redis_server_search_largest_memory_index</span>The memory usage of the index with the largest memory usage in the shard in bytes. <sup>1</sup>
<span class="break-all">redis_server_search_total_indexing_time</span>The total time spent on indexing operations, excluding the background indexing of vectors in the HNSW graph. <sup>1</sup>
<span class="break-all">redis_server_search_used_memory_vector_index</span>The total memory usage of all vector indexes in the shard. <sup>1</sup>
<span class="break-all">redis_server_search_global_idle</span>The total number of user and internal cursors currently holding pending results in the shard. <sup>1</sup>
<span class="break-all">redis_server_search_global_total</span>The total number of user and internal cursors in the shard, either holding pending results or actively executing FT.CURSOR READ. <sup>1</sup>
<span class="break-all">redis_server_search_bytes_collected</span>The total amount of memory freed by the garbage collectors from indexes in the shard memory in bytes. <sup>1</sup>
<span class="break-all">redis_server_search_total_cycles</span>The total number of garbage collection cycles executed <sup>1</sup>
<span class="break-all">redis_server_search_total_ms_run</span>The total duration of all garbage collection cycles in the shard, measured in milliseconds. <sup>1</sup>
<span class="break-all">redis_server_search_total_docs_not_collected_by_gc</span>The number of documents marked as deleted whose memory has not yet been freed by the garbage collector. <sup>1</sup>
<span class="break-all">redis_server_search_marked_deleted_vectors</span>The number of vectors marked as deleted in the vector indexes that have not yet been cleaned. <sup>1</sup>
<span class="break-all">redis_server_search_total_queries_processed</span>The total number of successful query executions (When using cursors, not counting reading from existing cursors) in the shard. <sup>1</sup>
<span class="break-all">redis_server_search_total_query_commands</span>The total number of successful query command executions (including FT.SEARCH, FT.AGGREGATE, and FT.CURSOR READ). <sup>1</sup>
<span class="break-all">redis_server_search_total_query_execution_time_ms</span>The cumulative execution time of all query commands, including FT.SEARCH, FT.AGGREGATE, and FT.CURSOR READ, measured in ms. <sup>1</sup>
<span class="break-all">redis_server_search_total_active_queries</span>The total number of background queries currently being executed in the shard, excluding FT.CURSOR READ. <sup>1</sup>
<span class="break-all">redis_server_search_errors_indexing_failures</span>The total number of indexing failures recorded across all indexes in the shard. <sup>1</sup>
<span class="break-all">redis_server_search_errors_for_index_with_max_failures</span>The number of indexing failures in the index with the highest count of failures. <sup>1</sup>
  1. <a name="tnote-1"></a> Available since RediSearch 2.6.
  2. <a name="tnote-2"></a> Available since RediSearch 2.8.