docs/en/administration/management/FE_parameters/shared_lake_other.md
import FEConfigMethod from '../../../_assets/commonMarkdown/FE_config_method.mdx'
import AdminSetFrontendNote from '../../../_assets/commonMarkdown/FE_config_note.mdx'
import StaticFEConfigNote from '../../../_assets/commonMarkdown/StaticFE_config_note.mdx'
<FEConfigMethod />After your FE is started, you can run the ADMIN SHOW FRONTEND CONFIG command on your MySQL client to check the parameter configurations. If you want to query the configuration of a specific parameter, run the following command:
ADMIN SHOW FRONTEND CONFIG [LIKE "pattern"];
For detailed description of the returned fields, see ADMIN SHOW CONFIG.
:::note You must have administrator privileges to run cluster administration-related commands. :::
You can configure or modify the settings of FE dynamic parameters using ADMIN SET FRONTEND CONFIG.
ADMIN SET FRONTEND CONFIG ("key" = "value");
This topic introduces the following types of FE configurations:
aws_s3_access_keyaws_s3_endpointhttps://s3.us-west-2.amazonaws.com.aws_s3_external_idaws_s3_iam_role_arnaws_s3_pathtestbucket/subpath.aws_s3_regionus-west-2.aws_s3_secret_keyaws_s3_use_aws_sdk_default_behavioraws_s3_use_instance_profilefalse, and specify aws_s3_access_key and aws_s3_secret_key.true.true, and specify aws_s3_iam_role_arn.aws_s3_external_id.azure_adls2_endpointhttps://test.dfs.core.windows.net.azure_adls2_oauth2_client_idazure_adls2_oauth2_tenant_idazure_adls2_oauth2_use_managed_identityazure_adls2_pathtestfilesystem/starrocks.azure_adls2_sas_tokenazure_adls2_shared_keyazure_blob_endpointhttps://test.blob.core.windows.net.azure_blob_pathtestcontainer/subpath.azure_blob_sas_tokenazure_blob_shared_keyazure_use_native_sdkfalse, only authentication with Shared Key and SAS Token is allowed.cloud_native_hdfs_urlhdfs://127.0.0.1:9000/user/xxx/starrocks/.cloud_native_meta_portcloud_native_storage_typeS3 (Default), HDFS, AZBLOB, ADLS2, and GS. If you specify this parameter as S3, you must add the parameters prefixed by aws_s3. If you specify this parameter as AZBLOB, you must add the parameters prefixed by azure_blob. If you specify this parameter as ADLS2, you must add the parameters prefixed by azure_adls2. If you specify this parameter as GS, you must add the parameters prefixed by gcp_gcs. If you specify this parameter as HDFS, you only need to specify cloud_native_hdfs_url.enable_load_volume_from_conftrue to false from v3.4.1 onwards.gcp_gcs_impersonation_service_accountgcp_gcs_pathtestbucket/subpath.gcp_gcs_service_account_email[email protected].gcp_gcs_service_account_private_key-----BEGIN PRIVATE KEY----xxxx-----END PRIVATE KEY-----\n.gcp_gcs_service_account_private_key_idgcp_gcs_use_compute_engine_service_accounthdfs_file_system_expire_secondsHdfsFs.getDFSFileSystem, getUserName, getConfiguration) update the last-access timestamp, so expiry is based on inactivity. Lower values reduce idle resource holding but increase reopen overhead; higher values keep handles longer and may consume more resources.lake_autovacuum_grace_period_minutes5 to 30 since v3.3.0, v3.2.5, and v3.1.10.lake_autovacuum_parallel_partitionslake_autovacuum_partition_naptime_secondslake_autovacuum_stale_partition_thresholdlake_compaction_allow_partial_successtrue, the system will consider the Compaction operation in a shared-data cluster as successful when one of the sub-tasks succeeds.lake_compaction_disable_idstableId1;partitionId2, seperated by semicolon, for example, 12345;98765.lake_compaction_history_sizeSHOW PROC '/compactions' command. Note that the Compaction history is stored in the FE process memory, and it will be lost if the FE process is restarted.lake_compaction_max_parallel_defaultlake_compaction_max_parallel is not specified in table properties. 0 means disable parallel compaction. This config is used as the default value for the table property lake_compaction_max_parallel.lake_compaction_max_tasks-1 indicates to calculate the concurrent task number in an adaptive manner. Setting this value to 0 will disable compaction.lake_compaction_score_selector_min_scorelake_compaction_score_upper_bound0 indicates no upper limit. This item only takes effect when lake_enable_ingest_slowdown is set to true. When the Compaction Score of a partition reaches or exceeds this upper limit, incoming loading tasks will be rejected. From v3.3.6 onwards, the default value is changed from 0 to 2000.lake_compaction_interval_ms_on_successlake_min_compaction_interval_ms_on_success.lake_enable_balance_tablets_between_workerstrue indicates to balance the tablets among Compute Nodes, and false indicates to disabling this feature.lake_enable_ingest_slowdownlake_ingest_slowdown_threshold, loading tasks on that partition will be throttled down. This configuration only takes effect when run_mode is set to shared_data. From v3.3.6 onwards, the default value is chenged from false to true.lake_ingest_slowdown_thresholdlake_enable_ingest_slowdown is set to true.lake_publish_version_max_threadsmeta_sync_force_delete_shard_metatrue only when there is an excessive number of shards to be cleaned, which leads to extreme memory pressure on the FE JVM. Note that the data files belonging to the shards or tablets cannot be automatically cleaned after this feature is enabled.run_modeDefault: shared_nothing
Type: String
Unit: -
Is mutable: No
Description: The running mode of the StarRocks cluster. Valid values: shared_data and shared_nothing (Default).
shared_data indicates running StarRocks in shared-data mode.shared_nothing indicates running StarRocks in shared-nothing mode.CAUTION
- You cannot adopt the
shared_dataandshared_nothingmodes simultaneously for a StarRocks cluster. Mixed deployment is not supported.- DO NOT change
run_modeafter the cluster is deployed. Otherwise, the cluster fails to restart. The transformation from a shared-nothing cluster to a shared-data cluster or vice versa is not supported.
Introduced in: -
shard_group_clean_threshold_secstar_mgr_meta_sync_interval_secstarmgr_grpc_server_max_worker_threadsstarmgr_grpc_timeout_secondsfiles_enable_insert_push_down_schemafiles() table function for INSERT ... FROM files() operations. This only applies when the source is a FileTableFunctionRelation, the target is a native table, and the SELECT list contains corresponding slot-ref columns (or *). The analyzer will match select columns to target columns (counts must match), lock the target table briefly, and replace file-column types with deep-copied target column types for non-complex types (complex types such as Parquet JSON -> array<varchar> are skipped). Column names from the original files table are preserved. This reduces type-mismatch and looseness from file-based type inference during ingestion.hdfs_read_buffer_size_kb<< 10) and uses it to initialize HDFS read buffers in HdfsFsManager and to populate the thrift field hdfs_read_buffer_size_kb sent to BE tasks (e.g., TBrokerScanRangeParams, TDownloadReq) when broker access is not used. Increasing hdfs_read_buffer_size_kb can improve sequential read throughput and reduce syscall overhead at the cost of higher per-stream memory usage; decreasing it reduces memory footprint but may lower IO efficiency. Consider workload (many small streams vs. few large sequential reads) when tuning.hdfs_write_buffer_size_kb<< 10) and initializes the local write buffer in HdfsFsManager, and it is propagated in Thrift requests (e.g., TUploadReq, TExportSink, sink options) so backends/agents use the same buffer size. Increasing this value can improve throughput for large sequential writes at the cost of more memory per writer; decreasing it reduces per-stream memory usage and may lower latency for small writes. Tune alongside hdfs_read_buffer_size_kb and consider available memory and concurrent writers.lake_batch_publish_max_version_numlake_batch_publish_min_version_num to determine the candidate range size for a TransactionStateBatch. Larger values can increase publish throughput by batching more commits, but increase the scope of an atomic publish (longer visibility latency and larger rollback surface) and may be limited at runtime when versions are not consecutive. Tune according to workload and visibility/latency requirements.lake_batch_publish_min_version_numlake_batch_publish_max_version_num to select dependent transactions. A value of 1 allows single-transaction publishes (no batching). Values >1 require at least that many consecutively-versioned, single-table, non-replication transactions to be available; batching is aborted if versions are non-consecutive, a replication transaction appears, or a schema change consumes a version. Increasing this value can improve publish throughput by grouping commits but may delay publishing while waiting for enough consecutive transactions.lake_enable_batch_publish_versionlake_publish_version_max_threads.lake_enable_tablet_creation_optimizationfile_bundling (the latter reuses the same optimization logic). Note: schema-change and rollup jobs explicitly disable the optimization for tables using file_bundling to avoid overwriting files with identical names. Enable cautiously — it changes the granularity of created tablet metadata and can affect how replica creation and file naming behave.lake_use_combined_txn_logtrue, the system allows Lake tables to use the combined transaction log path for relevant transactions. Available for shared-data clusters only.lake_repair_metadata_fetch_max_version_batch_sizelake_enable_drop_tablet_cacheenable_iceberg_commit_queueiceberg_commit_queue_timeout_secondsenable_iceberg_commit_queue=true), each commit operation must complete within this timeout. If a commit takes longer than this timeout, it will be cancelled and an error will be raised. Factors that affect commit time include: number of data files being committed, metadata size of the table, performance of the underlying storage (e.g., S3, HDFS).iceberg_commit_queue_max_sizeenable_iceberg_commit_queue=true), this limits the number of commit operations that can be queued for a single table. When the limit is reached, additional commit operations will execute in the caller thread (blocking until capacity available). This configuration is read at FE startup and applies to newly created table executors. Requires FE restart to take effect. Increase this value if you expect many concurrent commits to the same table. If this value is too low, commits may block in the caller thread during high concurrency.f = (MAX(tablets) - MIN(tablets)) / AVERAGE(tablets). If the factor is greater than lake_balance_tablets_threshold, a tablet balance will be triggered. This item takes effect only when lake_enable_balance_tablets_between_workers is set to true.agent_task_resend_wait_time_msallow_system_reserved_names__op and __row. To enable this feature, set this parameter to TRUE. Please note that these name formats are reserved for special purposes in StarRocks and creating such columns may result in undefined behavior. Therefore this feature is disabled by default.auth_tokenauthentication_ldap_simple_bind_base_dnauthentication_ldap_simple_bind_root_dnauthentication_ldap_simple_bind_root_pwdauthentication_ldap_simple_server_hostauthentication_ldap_simple_server_portauthentication_ldap_simple_user_search_attrbackup_job_default_timeout_msenable_collect_tablet_num_in_show_proc_backend_disk_pathSHOW PROC /BACKENDS/{id} commandenable_colocate_restoretrue indicates enabling Backup and Restore for Colocate Tables and false indicates disabling it.enable_external_catalog_information_schema_tables_access_full_metadatainformation_schema.tables is allowed to access external metadata services when resolving tables in external catalogs (such as Hive, Iceberg, JDBC). When set to false (default), columns like TABLE_COMMENT may be empty for external tables but the query is fast and avoids remote calls. When set to true, the FE contacts the corresponding external metadata service and can populate fields like TABLE_COMMENT at the cost of additional latency and remote calls per table.enable_materialized_view_concurrent_prepareenable_metric_calculatorTRUE and FALSE. TRUE specifies to enable this feature, and FALSE specifies to disable this feature.enable_table_metrics_collectenable_mv_post_image_reload_cacheenable_mv_query_context_cacheenable_mv_refresh_collect_profileenable_mv_refresh_extra_prefix_loggingenable_mv_refresh_query_rewriteenable_trace_historical_nodetrue, you can enable the Cache Sharing feature and allow the system to choose the right cache nodes during elastic scaling.es_state_sync_interval_secondhive_meta_cache_refresh_interval_shive_meta_store_timeout_sjdbc_connection_idle_timeout_msjdbc_connection_timeout_msjdbc_query_timeout_msjdbc_network_timeout_msjdbc_connection_max_lifetime_msjdbc_connection_keepalive_time_msjdbc_connection_max_lifetime_ms. Invalid enabled values are silently disabled (reset to 0).jdbc_connection_leak_detection_threshold_msjdbc_connection_pool_sizejdbc_meta_default_cache_enablejdbc_meta_default_cache_expire_secjdbc_meta_default_cache_enable is set to true, newly created JDBC Catalogs will default to setting the expiration time of the metadata cache.jdbc_minimum_idle_connectionsjwt_jwks_urlfe/conf directory.jwt_principal_fieldsub) in the JWT. The default value is sub. The value of this field must be identical with the username for logging in to StarRocks.jwt_required_audienceaud) in the JWT. The JWT is considered valid only if one of the values in the list match the JWT audience.jwt_required_issueriss) in the JWT. The JWT is considered valid only if one of the values in the list match the JWT issuer.zh_CN.UTF-8max_agent_task_threads_nummax_download_task_per_bemax_mv_check_base_table_change_retry_timesmax_mv_refresh_failure_retry_timesmax_mv_refresh_try_lock_failure_retry_timesmax_small_file_numbermax_small_file_size_bytesmax_upload_task_per_bemv_create_partition_batch_interval_msmv_plan_cache_max_sizemv_plan_cache_thread_pool_sizemv_refresh_default_planner_optimize_timeoutmv_refresh_fail_on_filter_datamv_refresh_try_lock_timeout_msoauth2_auth_server_urloauth2_client_idoauth2_client_secretoauth2_jwks_urlconf directory.oauth2_principal_fieldsub) in the JWT. The default value is sub. The value of this field must be identical with the username for logging in to StarRocks.oauth2_redirect_urlhttp://<starrocks_fe_url>:<fe_http_port>/api/oauth2.oauth2_required_audienceaud) in the JWT. The JWT is considered valid only if one of the values in the list match the JWT audience.oauth2_required_issueriss) in the JWT. The JWT is considered valid only if one of the values in the list match the JWT issuer.oauth2_token_server_urlplugin_dirSystem.getenv("STARROCKS_HOME") + "/plugins"plugin_enableproc_profile_jstack_depthproc_profile_mem_enabletrue, the system generates an HTML profile named mem-profile-<timestamp>.html under sys_log_dir/proc_profile, sleeps for proc_profile_collect_time_s seconds while sampling, and uses proc_profile_jstack_depth for Java stack depth. Generated files are compressed and purged according to proc_profile_file_retained_days and proc_profile_file_retained_size_bytes. The native extraction path uses STARROCKS_HOME_DIR to avoid /tmp noexec issues. This item is intended for troubleshooting memory-allocation hotspots. Enabling it increases CPU, I/O and disk usage and may produce large files.query_detail_explain_levelreplication_interval_msreplication_max_parallel_data_size_mbreplication_max_parallel_replica_countreplication_max_parallel_table_countreplication_transaction_timeout_secskip_whole_phase_lock_mv_limitsmall_file_dirStarRocksFE.STARROCKS_HOME_DIR + "/small_files"task_runs_max_history_numberenable_task_history_archive is false, this value bounds in-memory history: Force GC trims older entries so only the newest task_runs_max_history_number remain. When archive history is queried (and no explicit LIMIT is provided), TaskRunHistoryTable.lookup uses "ORDER BY create_time DESC LIMIT <value>" if this value is greater than 0. Note: setting this to 0 disables the query-side LIMIT (no cap) but will cause in-memory history to be truncated to zero (unless archiving is enabled).tmp_dirStarRocksFE.STARROCKS_HOME_DIR + "/temp_dir"transform_type_prefer_string_for_varchar