docs/en/administration/management/monitoring/metric_details/q-z.md
:::note
Metrics for materialized views and shared-data clusters are detailed in the corresponding sections:
For more information on how to build a monitoring service for your StarRocks cluster, see Monitor and Alert.
:::
query_cache_capacityquery_cache_hit_countquery_cache_hit_ratioquery_cache_lookup_countquery_cache_usagequery_cache_usage_ratioquery_mem_bytesquery_scan_bytesquery_scan_bytes_per_secondquery_scan_rowsreadable_blocks_total (Deprecated)resource_group_bigquery_countresource_group_concurrency_overflow_countresource_group_connector_scan_use_ratio (Deprecated)resource_group_cpu_limit_ratioresource_group_cpu_use_ratio (Deprecated)resource_group_inuse_cpu_coresresource_group_mem_inuse_bytesresource_group_mem_limit_bytesresource_group_running_queriesresource_group_scan_use_ratio (Deprecated)resource_group_total_queriesresult_block_queue_countresult_buffer_block_countroutine_load_task_countrowset_count_generated_and_in_userowset_metadata_mem_bytesrunning_base_compaction_task_numrunning_cumulative_compaction_task_numrunning_update_compaction_task_numschema_change_mem_bytessegment_flush_queue_countsegment_metadata_mem_bytessegment_readsegment_replicate_queue_countsegment_zonemap_mem_bytesshort_key_index_mem_bytessmall_file_cache_countsnmp/proc/net/snmp.starrocks_be_clone_task_copy_bytesstarrocks_be_clone_task_copy_duration_msstarrocks_be_exec_state_report_active_threadsstarrocks_be_exec_state_report_queue_countstarrocks_be_exec_state_report_running_threadsstarrocks_be_exec_state_report_threadpool_sizestarrocks_be_files_scan_num_bytes_readfile_format, scan_type.starrocks_be_files_scan_num_files_readfile_format, scan_type.starrocks_be_files_scan_num_raw_rows_readfile_format, scan_type.starrocks_be_files_scan_num_rows_returnfile_format, scan_type.starrocks_be_files_scan_num_valid_rows_readfile_format, scan_type.starrocks_be_mem_pool_mem_limit_bytesstarrocks_be_mem_pool_mem_usage_bytesstarrocks_be_mem_pool_mem_usage_ratiostarrocks_be_mem_pool_workgroup_countstarrocks_be_pipe_prepare_pool_queue_lenstarrocks_be_priority_exec_state_report_active_threadsstarrocks_be_priority_exec_state_report_queue_countstarrocks_be_priority_exec_state_report_running_threadsstarrocks_be_priority_exec_state_report_threadpool_sizestarrocks_be_resource_group_cpu_limit_ratiostarrocks_be_resource_group_cpu_use_ratiostarrocks_be_resource_group_mem_inuse_bytesstarrocks_be_resource_group_mem_limit_bytesstarrocks_be_segment_file_not_found_totalstarrocks_be_staros_shard_info_fallback_totalg_starlet->get_shard_info()) that the BE's StarOSWorker had to issue because the requested shard info was not in the local cache (i.e. the FE had not pushed the shard to this BE before a query/compaction/lake operation referenced it). Only counted when the starlet readiness check passes and the RPC is actually dispatched; starlet-not-ready timeouts are not included. Should normally be near zero. A sustained or rising rate is a strong signal that FE-side task or node selection is scheduling work on a BE that does not yet have the shard, or that shard push propagation from FE is lagging. Recommended alert: high per-BE rate over a 5-minute window.starrocks_be_staros_shard_info_fallback_failed_totalstarrocks_be_staros_shard_info_fallback_total where the starmgr RPC returned a non-OK status. Use the ratio failed_total / fallback_total to alert on transient starmgr errors separately from routine successful fallbacks.starrocks_be_staros_shard_countStarOSWorker::add_shard and StarOSWorker::remove_shard (push-on-mutation), so the value reflects the last shard table mutation rather than being recomputed at scrape time. The gauge is not reset on BE shutdown and will retain its last value until the next mutation. Use it to observe shard distribution balance across BEs and to detect drift from the FE-side placement.starrocks_fe_clone_task_copy_bytesstarrocks_fe_clone_task_copy_duration_msstarrocks_fe_clone_task_successstarrocks_fe_clone_task_totalstarrocks_fe_last_finished_job_timestampstarrocks_fe_memory_usagestarrocks_fe_meta_log_count100000 is considered reasonable..starrocks_fe_publish_version_daemon_loop_totalpublish-version-daemon loop runs on this FE node.The following metrics are summary-type metrics that provide latency distributions for different phases of a transaction. These metrics are reported exclusively by the Leader FE node.
Each metric includes the following outputs:
quantile label, which can have values of 0.75, 0.95, 0.98, 0.99, and 0.999.<metric_name>_sum: The total cumulative time spent in this phase, for example, starrocks_fe_txn_total_latency_ms_sum.<metric_name>_count: The total number of transactions recorded for this phase, for example, starrocks_fe_txn_total_latency_ms_count.All transaction metrics share the following labels:
type: Categorizes transactions by their load job source type (for example, all, stream_load, routine_load). This allows for monitoring both overall transaction performance and the performance of specific load types. The reported groups can be configured via the FE parameter txn_latency_metric_report_groups.is_leader: Indicates whether the reporting FE node is the Leader. Only the Leader FE (is_leader="true") reports actual metric values. Followers will have is_leader="false" and report no data.starrocks_fe_query_resource_groupstarrocks_fe_query_resource_groupstarrocks_fe_query_resource_group_errstarrocks_fe_query_resource_group_errstarrocks_fe_query_resource_group_latencystarrocks_fe_query_resource_group_latencystarrocks_fe_routine_load_error_rowsstarrocks_fe_routine_load_jobsUnit: Count
Description: The total number of Routine Load jobs in different states. For example:
starrocks_fe_routine_load_jobs{state="NEED_SCHEDULE"} 0
starrocks_fe_routine_load_jobs{state="RUNNING"} 1
starrocks_fe_routine_load_jobs{state="PAUSED"} 0
starrocks_fe_routine_load_jobs{state="STOPPED"} 0
starrocks_fe_routine_load_jobs{state="CANCELLED"} 1
starrocks_fe_routine_load_jobs{state="UNSTABLE"} 0
starrocks_fe_routine_load_max_lag_of_partitionenable_routine_load_lag_metrics is set to true and the offset lag is greater than or equal to the FE configuration min_routine_load_lag_for_metrics. By default, enable_routine_load_lag_metrics is false, and min_routine_load_lag_for_metrics is 10000.starrocks_fe_routine_load_max_lag_time_of_partitionenable_routine_load_lag_time_metrics is set to true. By default, enable_routine_load_lag_time_metrics is false.starrocks_fe_routine_load_pausedstarrocks_fe_routine_load_receive_bytesstarrocks_fe_routine_load_rowsstarrocks_fe_safe_mode0 (disabled) and 1 (enabled). When Safe Mode is enabled, the cluster no longer accepts any loading requests.starrocks_fe_scheduled_pending_tablet_numstarrocks_fe_scheduled_running_tablet_numstarrocks_fe_slow_lock_held_time_msslow_lock_threshold_ms configuration parameter. It tracks the maximum lock held time among all lock owners when a slow lock event is detected. Each metric includes quantile values (0.75, 0.95, 0.98, 0.99, 0.999), _sum, and _count outputs. Note: This metric may not accurately reflect the exact lock held time under high contention, because the metric is updated once the wait time exceeds the threshold, but the held time may continue to increase until the owner completes its operation and releases the lock. However, this metric can still be updated even when deadlock occurs.starrocks_fe_slow_lock_wait_time_msslow_lock_threshold_ms configuration parameter. It accurately tracks how long threads wait to acquire locks during lock contention scenarios. Each metric includes quantile values (0.75, 0.95, 0.98, 0.99, 0.999), _sum, and _count outputs. This metric provides precise wait time measurements. Note: This metric cannot be updated when deadlock occurs, hence it cannot be used to detect deadlock situations.starrocks_fe_sql_block_hit_countstarrocks_fe_tablet_max_compaction_scorestarrocks_fe_tablet_numstarrocks_fe_txn_publish_ack_latency_msready-to-finish time to the final finish time when the transaction is marked as VISIBLE. This metric includes final acknowledgment steps after the transaction is ready to finish.starrocks_fe_txn_publish_can_finish_latency_mspublish task completion to the moment canTxnFinish() first returns true, measured from publish version finish time to ready-to-finish time.starrocks_fe_txn_publish_execute_latency_mspublish task, from when the task is picked up to when it finishes. This metric represents the actual time being spent to make the transaction's changes visible.starrocks_fe_txn_publish_latency_mspublish phase, from commit time to finish time. This is the duration it takes for a committed transaction to become visible to queries. It is the sum of the schedule, execute, can_finish, and ack sub-phases.starrocks_fe_txn_publish_schedule_latency_mscommit time to when the publish task is picked up. This metric reflects scheduling delays or queueing time in the publish pipeline.starrocks_fe_txn_total_latency_msprepare time to the finish time. This metric represents the full end-to-end duration of a transaction.starrocks_fe_txn_write_latency_mswrite phase of a transaction, from prepare time to commit time. This metric isolates the performance of the data writing and preparation stage before the transaction is ready to be published.starrocks_fe_unfinished_backup_job0.starrocks_fe_unfinished_querystarrocks_fe_unfinished_restore_job0.storage_page_cache_mem_bytesstream_loadstream_load_pipe_countstreaming_load_bytesstreaming_load_current_processingstreaming_load_duration_msstreaming_load_requests_totaltablet_base_max_compaction_scoretablet_cumulative_max_compaction_scoretablet_metadata_mem_bytestablet_schema_mem_bytestablet_update_max_compaction_scorethrift_connections_totalthrift_current_connections (Deprecated)thrift_opened_clientsthrift_used_clientstotal_column_pool_bytes (Deprecated)transaction_streaming_load_bytestransaction_streaming_load_current_processingtransaction_streaming_load_duration_mstransaction_streaming_load_requests_totaltxn_requestuint8_column_pool_bytesunused_rowsets_countupdate_apply_queue_countupdate_compaction_duration_usupdate_compaction_outputs_bytes_totalupdate_compaction_outputs_totalupdate_compaction_task_byte_per_secondupdate_compaction_task_cost_time_nsupdate_del_vector_bytes_totalupdate_del_vector_deletes_newupdate_del_vector_deletes_total (Deprecated)update_del_vector_dels_num (Deprecated)update_del_vector_numupdate_mem_bytesupdate_primary_index_bytes_totalupdate_primary_index_numupdate_rowset_commit_apply_duration_usupdate_rowset_commit_apply_totalupdate_rowset_commit_request_failedupdate_rowset_commit_request_totalwait_base_compaction_task_numwait_cumulative_compaction_task_numwritable_blocks_total (Deprecated)