Back to Rustfs

Global State Inventory

docs/architecture/global-state-inventory.md

1.0.0-rc.121.8 KB
Original Source

Global State Inventory

This inventory records the issue #730 baseline for global runtime state after the AppContext foundation and owner-local runtime-source boundaries were added. It is intentionally documentation-only: it classifies migration targets without changing startup, readiness, object IO, lifecycle, replication, or notification behavior.

Counting Baseline

The audit uses the current workspace Rust sources and keeps broad static caches separate from runtime migration targets.

ScopeCountCommand
Rust source files1,252rg --files -g '*.rs'
OnceLock references221 linesrg -n --glob '*.rs' 'OnceLock'
GLOBAL_* references273 linesrg -n --glob '*.rs' '\bGLOBAL_[A-Za-z0-9_]*\b'
static NAME: definitions621 linesrg -n --glob '*.rs' '^\s*(pub(\([^)]*\))?\s+)?static(\s+mut)?\s+[A-Za-z_][A-Za-z0-9_]*\s*:'
lazy_static! static ref definitions58 linesrg -n --glob '*.rs' '^\s*(pub\s+)?static\s+ref\s+[A-Za-z_][A-Za-z0-9_]*\s*:'
static mut definitions0 linesrg -n --glob '*.rs' '^\s*(pub(\([^)]*\))?\s+)?static\s+mut\s+'

Global State Classification

CategoryRuleRepresentative owners
Process-globalProcess identity, metrics registries, lock manager, audit guard, TLS material, or other state that is intentionally one per process.crates/credentials, crates/common, crates/io-metrics, crates/lock, crates/obs, crates/tls-runtime
Runtime migration targetMutable runtime state that describes the active object store, endpoints, local disks, lifecycle, replication, notification, config, or background controllers.crates/ecstore/src/runtime/global.rs, crates/ecstore/src/runtime/sources.rs, rustfs/src/app/context/*
Owner-local compatibilityExisting compatibility adapters that are allowed to read globals while callers migrate to AppContext-first or owner-local runtime-source APIs.rustfs/src/*/runtime_sources.rs, rustfs/src/*/storage_api.rs, crates/*/storage_api.rs
Test or fixture stateStatic setup used by tests to amortize expensive ECStore setup or isolate compatibility harness state.rustfs/src/app/*_test.rs, crates/scanner/tests/*, crates/ecstore/src/**/tests
Cache or constantRegexes, metrics descriptors, defaults, KVS registrations, headers, path constants, and small process caches that are not runtime ownership handles.crates/config, crates/obs/src/metrics, crates/utils, rustfs/src/server/readiness.rs
Legacy naming or review-neededOld MinIO-port naming, stale comments, or names that need owner confirmation before code movement.GLOBAL_OBJECT_API

Runtime Migration Inventory

These are the issue #730 targets that should remain visible until an owner migration PR removes or replaces each item.

StateCurrent boundaryCategoryMigration stance
APP_CONTEXT_SINGLETONrustfs/src/app/context/global.rsOwner-local compatibilityKeep as the context-first facade while no-context startup and embedded callers still exist.
GLOBAL_OBJECT_API, GLOBAL_OBJECT_STORE_RESOLVERcrates/ecstore/src/runtime/global.rs, rustfs/src/app/context/global.rs, and storage compatibility APIsRuntime migration targetDo not migrate first; it is tied to storage startup, IAM-after-storage AppContext publication, and data-plane resolver compatibility. The object-store resolver is now published from the AppContext owner path, no longer re-exported from the RustFS storage root, and RustFS AppContext tests no longer use the old new_object_layer_fn fallback chain. RustFS storage root no longer re-exports ECStore runtime/global facade symbols; callers must use storage/app/admin facades.
GLOBAL_ENDPOINTS, GLOBAL_IS_ERASURE, GLOBAL_IS_DIST_ERASURE, GLOBAL_IS_ERASURE_SD, GLOBAL_ROOT_DISK_THRESHOLDcrates/ecstore/src/runtime/global.rs and crates/ecstore/src/runtime/sources.rsRuntime migration targetEndpoint and setup-type reads now flow through ECStore api::runtime helpers at the RustFS storage facade boundary; root-disk-threshold access stays behind ECStore runtime helpers. Move endpoint ownership only after readiness and quorum behavior have explicit coverage.
GLOBAL_LOCAL_DISK_MAP, GLOBAL_LOCAL_DISK_ID_MAP, GLOBAL_LOCAL_DISK_SET_DRIVEScrates/ecstore/src/runtime/global.rs and crates/ecstore/src/runtime/sources.rsRuntime migration targetLocal disk map, disk-id cache, and set-drive access now stay behind ECStore runtime-source helpers instead of direct global access; preserve disk lookup, remote/local classification, and test reset hooks in later ownership changes.
GLOBAL_EXPIRY_STATE, GLOBAL_TRANSITION_STATE, GLOBAL_LIFECYCLE_SYScrates/ecstore/src/bucket/lifecycle/*, crates/ecstore/src/runtime/global.rs, and crates/ecstore/src/runtime/sources.rsRuntime migration targetLifecycle state globals now stay behind ECStore lifecycle owner helpers and ECStore runtime-source helpers; RustFS AppContext has expiry/transition state interfaces and resolver coverage, and daily tier stats derive from the transition-state handle instead of a separate context boundary; scanner expiry-state access still uses the ECStore runtime expiry_state_handle boundary until scanner gets an injected provider.
GLOBAL_REPLICATION_POOL, GLOBAL_REPLICATION_STATS, GLOBAL_BUCKET_MONITORcrates/ecstore/src/bucket/replication/*, crates/ecstore/src/runtime/global.rsRuntime migration targetReplication pool/stat access now stays behind replication owner and ECStore runtime-source helpers; bucket-monitor reads now flow through ECStore api::runtime at the RustFS storage facade boundary while AppContext/runtime-source resolvers remain the caller boundary.
GLOBAL_TIER_CONFIG_MGR, GLOBAL_STORAGE_CLASS, GLOBAL_CONFIG_SYS, GLOBAL_SERVER_CONFIGcrates/ecstore/src/config, crates/config, rustfs/src/app/context/runtime_sources.rsRuntime migration targetTier config manager reads and reloads now use the ECStore runtime-source helper; move remaining config state through config/runtime-source owners only, without combining storage-class behavior or persistence changes.
GLOBAL_EVENT_NOTIFIER, GLOBAL_NOTIFICATION_SYScrates/ecstore/src/runtime/global.rs, crates/ecstore/src/runtime/sources.rs, and crates/ecstore/src/services/*Runtime migration targetGLOBAL_EVENT_NOTIFIER and GLOBAL_NOTIFICATION_SYS access now stay behind ECStore runtime-source and notification owner helpers; move remaining notification ownership only through notify/runtime-source boundaries.
EVENT_DISPATCH_HOOKcrates/ecstore/src/services/event_notification.rs, RustFS server event bridge, and storage compatibility APIsRuntime migration target / owner helperDirect hook storage stays inside the ECStore event-notification owner; RustFS registers the bridge through the storage compatibility facade until event dispatch ownership moves behind an injected notification sink.
GLOBAL_BUCKET_METADATA_SYScrates/ecstore/src/bucket/metadata_sys.rs, crates/ecstore/src/runtime/sources.rs, and RustFS storage compatibility APIsRuntime migration targetBucket metadata system direct access now stays inside the ECStore metadata owner; callers use metadata owner helpers or storage/runtime-source compatibility functions until metadata ownership moves behind an injected runtime context.
GLOBAL_BOOT_TIME, GLOBAL_BACKGROUND_SERVICES_CANCEL_TOKEN, GLOBAL_DEPLOYMENT_ID, GLOBAL_REGION, GLOBAL_RUSTFS_PORT, GLOBAL_LOCAL_NODE_NAME_FALLBACK, GLOBAL_LOCAL_NODE_NAME_HEX_FALLBACKcrates/ecstore/src/runtime/global.rs, crates/ecstore/src/runtime/sources.rsRuntime migration targetBoot time, background service cancellation token reads, ECStore local-node-name fallback reads, and deployment ID/region/port reads now stay behind the ECStore runtime-source API; scalar writes remain behind bootstrap owner helpers until ownership handles replace them.
WORKLOAD_ADMISSION_SNAPSHOT_PROVIDERcrates/ecstore/src/runtime/sources.rs, RustFS startup background setup, and storage compatibility APIsRuntime migration target / owner helperStartup publishes the workload provider through the storage compatibility facade, and ECStore data movement reads it only through the runtime-source helper until workload admission ownership moves into an explicit runtime context.
GLOBAL_LOCAL_LOCK_CLIENT, GLOBAL_LOCK_CLIENTS, GLOBAL_LOCK_MANAGERcrates/ecstore/src/runtime/global.rs, crates/lockRuntime migration target / process-global splitECStore lock client reads now flow through ECStore api::runtime helpers at the RustFS storage facade boundary; preserve lock quorum and lock client selection while keeping the process-level lock manager separate from endpoint-specific clients.
GLOBAL_CONN_MAP, GLOBAL_LOCAL_NODE_NAME, GLOBAL_RUSTFS_HOST, GLOBAL_RUSTFS_ADDR, GLOBAL_ROOT_CERT, GLOBAL_MTLS_IDENTITY, GLOBAL_OUTBOUND_TLS_GENERATIONcrates/common, crates/tls-runtime, crates/ecstore/src/runtime/sources.rsRuntime migration target / process-global splitInternode connection cache, common local node name, RustFS host/address reads, and outbound TLS material reads are now owned behind rustfs_common helpers; migrate the remaining transport and TLS state only after internode transport and outbound TLS ownership are explicit, without changing cached channel reuse or TLS reload semantics.
GLOBAL_RUSTFS_RPC_SECRETcrates/credentials, crates/ecstore/src/runtime/sources.rsRuntime migration target / process-global splitRPC auth token writes now stay behind the rustfs_credentials helper boundary; migrate only if runtime secret ownership changes, preserving lazy environment and credential-derived token semantics.
GLOBAL_HEAL_MANAGER, GLOBAL_HEAL_CHANNEL_PROCESSOR, GLOBAL_AHM_SERVICES_CANCEL_TOKENcrates/heal/src/lib.rsRuntime migration target / process-global splitDirect access now stays inside the heal owner; callers use heal helper functions until heal runtime ownership moves behind explicit owner handles.
AUDIT_SYSTEMcrates/audit/src/global.rsRuntime migration target / process-global splitDirect global access now stays inside the audit owner; callers use audit helper functions until audit lifecycle ownership moves behind AppContext or a runtime-source boundary.
GLOBAL_PROCESSORScrates/ecstore/src/services/batch_processor.rs, crates/ecstore/src/runtime/sources.rsRuntime migration target / owner helperDirect static access now stays inside the ECStore batch processor owner; callers use get_global_processors or the ECStore runtime-source helper until processor ownership moves into an injected runtime context.
INTERNODE_DATA_TRANSPORTcrates/ecstore/src/cluster/rpc/internode_data_transport.rsRuntime migration target / owner helperDirect static access now stays inside the ECStore internode transport owner; callers use build_internode_data_transport_from_env until backend selection moves into an injected runtime context.
GLOBAL_KMS_SERVICE_MANAGERcrates/kms/src/service_manager.rs, RustFS KMS runtime sourcesRuntime migration target / owner helperDirect static access now stays inside the rustfs_kms service manager owner; RustFS callers use KMS helpers or AppContext/runtime-source handles until KMS ownership fully moves into runtime context.
GLOBAL_CAPACITY_MANAGERcrates/object-capacity/src/capacity_manager.rs, RustFS capacity serviceRuntime migration target / owner helperDirect static access now stays inside the object-capacity owner; callers use get_capacity_manager or isolated manager factories until capacity ownership moves into an injected runtime context.
GLOBAL_BUCKET_TARGET_SYScrates/ecstore/src/bucket/bucket_target_sys.rs, admin/app/scanner/replication target pathsRuntime migration target / owner helperDirect static access now stays inside the ECStore bucket target owner; callers still use BucketTargetSys::get() until bucket target ownership moves behind a runtime-source or replication target boundary.
USAGE_MEMORY_CACHE, USAGE_CACHE_UPDATINGcrates/ecstore/src/data_usage/mod.rsRuntime migration target / owner-local cacheData-usage memory overlay and singleflight state stay private to the ECStore data-usage owner; callers use data-usage functions until scanner/data-usage ownership moves behind an injected runtime context.

Owner-Local Cache Inventory

These owner-local caches and static guards are part of the broad issue #730 OnceLock audit, but they are not runtime ownership handles. They stay private to the defining owner module; callers must use the existing owner APIs instead of reaching across module boundaries.

StateOwner boundaryCategoryMigration stance
READ_REPAIR_HEAL_CACHEcrates/ecstore/src/set_disk/read.rsCache or constant / owner-local cacheRead-repair heal suppression stays local to set-disk read handling.
DISK_COMPRESSION_CONFIGcrates/ecstore/src/io_support/compress.rsCache or constant / owner-local cacheDisk compression environment parsing stays local to IO support compression helpers.
CACHED_MAX_INFLIGHT_BYTES, CACHED_BATCH_BLOCKS, CACHED_BYTESMUT_INGESTcrates/ecstore/src/erasure/coding/encode.rsCache or constant / owner-local cacheErasure encode tuning caches stay local to the coding owner.
CACHED_PUT_LARGE_BATCH_MIN_SIZE_BYTES, CACHED_MULTIPART_PUT_LARGE_BATCH_MIN_SIZE_BYTES, OBJECT_LOCK_DIAG_ENABLEDcrates/ecstore/src/set_disk/mod.rsCache or constant / owner-local cacheSet-disk batching and diagnostics caches stay local to the set-disk owner.
DRIVE_TIMEOUT_PROFILE_CACHE, DRIVE_TIMEOUT_HEALTH_POLICY_CACHEcrates/ecstore/src/disk/disk_store.rsCache or constant / owner-local cacheDrive timeout environment caches stay local to the disk-store owner.
TIER_FREE_VERSION_RECOVERY_STARTED, TIER_DELETE_JOURNAL_RECOVERY_STARTEDcrates/ecstore/src/bucket/lifecycle/bucket_lifecycle_ops.rsCache or constant / owner-local static guardLifecycle recovery single-run guards stay local to lifecycle operations.
REMOTE_DELETE_INFLIGHT, REMOTE_DELETE_LIMITER, REMOTE_DELETE_BREAKER, REMOTE_TIER_DELETE_TEST_HOOKcrates/ecstore/src/bucket/lifecycle/tier_sweeper.rsCache or constant / owner-local static guardRemote tier delete concurrency, breaker, and test hook state stay local to the tier sweeper owner.
ACTIVE_REGISTRY, BackendCapacitycrates/kms/src/policy.rsProcess-global owner-local admission capacity registryKMS policy generations share only active semaphore capacity by backend identity; each generation owns fresh bounded queues and circuit breakers. Callers access this state only through RetryPolicy.

RustFS Owner-Local Static Inventory

These RustFS-side lazy, atomic, and OnceLock statics are also part of the issue #730 process-static audit. They are private implementation details for their owner modules, not shared runtime ownership handles. This section excludes allocator statics, public contract/error references, route handler constants, and APP_CONTEXT_SINGLETON, which is classified in the runtime migration inventory. Generic function-local names such as CACHE, LOCK, INIT, and ENABLED are documented by owner row instead of name-regex guarded.

StateOwner boundaryCategoryMigration stance
KEYSTONE_AUTH, KEYSTONE_MAPPER, KEYSTONE_CONFIGrustfs/src/auth_keystone.rsProcess-global owner-local stateKeystone authentication provider, identity mapper, and config stay private to the Keystone auth owner.
LICENSE_STATE, LICENSE_VERIFIERrustfs/src/license.rsProcess-global owner-local stateLicense state and verifier selection stay private to the license owner; callers use license helper functions.
CPU_CONT_GUARD, PROFILING_CANCEL_TOKENrustfs/src/profiling.rsProcess-global owner-local guardCPU profiling guard and cancellation state stay private to the profiling owner.
MEMORY_SYSTEMrustfs/src/memory_observability.rsProcess-global owner-local cacheMemory sampling keeps the sysinfo::System cache private to the memory observability owner.
DISPLAY_CONFIG_SNAPSHOT, GLOBAL_CONFIG_SNAPSHOTrustfs/src/config/snapshot.rsProcess-global owner-local stateConfig snapshots stay private to the config snapshot owner.
BUFFER_CONFIG_SINGLETON, BUFFER_PROFILE_ENABLEDrustfs/src/config/workload_profiles.rsProcess-global owner-local stateWorkload buffer profile configuration stays private to workload profile helpers.
LEGACY_CREDENTIAL_WARNED_KEYSrustfs/src/config/config_struct.rsProcess-global owner-local cacheLegacy credential warning de-duplication stays private to config parsing.
CONSOLE_CONFIGrustfs/src/admin/console.rsProcess-global owner-local stateConsole bootstrap config stays private to the admin console owner.
ACTIVE_HTTP_REQUESTSrustfs/src/server/http.rsProcess-global owner-local counterHTTP request inflight accounting stays private to the HTTP server owner.
Function-local CACHE and LOCK staticsrustfs/src/server/readiness.rsCache or constant / owner-local cacheReadiness and cluster-health caches stay function-local to readiness probes.
USE_STARSHARD_CACHE, BUCKET_CACHE_SMALL, BUCKET_CACHE_LARGErustfs/src/storage/ecfs_extend.rsCache or constant / owner-local cacheBucket validation cache backend selection and cache storage stay private to the ECFS extension owner.
GLOBAL_SSE_DEK_PROVIDER, SSE_TEST_LOCKrustfs/src/storage/sse.rsOwner-local cache / test stateSSE DEK provider cache and test serialization lock stay private to the SSE owner.
AUTH_FSrustfs/src/storage/access.rsCache or constant / owner-local cacheAuthorization tag-condition lookup keeps its filesystem helper private to the access owner.
LOCK_STATSrustfs/src/storage/lock_optimizer.rsProcess-global owner-local metricsLock optimization statistics stay private behind lock optimizer helper APIs.
DEADLOCK_DETECTORrustfs/src/storage/deadlock_detector.rsProcess-global owner-local stateDeadlock detector lifecycle state stays private to the storage deadlock detector owner.
CONCURRENCY_MANAGER, ACTIVE_GET_REQUESTS, ACTIVE_PUT_REQUESTS, IO_PRIORITY_METRICSrustfs/src/storage/concurrency/*Process-global owner-local scheduler stateStorage concurrency manager, counters, and metrics remain inside the storage concurrency owner boundary.
GET_OBJECT_BUFFER_THRESHOLD_WARNED, GET_READER_STREAM_BUFFER_SIZE_OVERRIDE, function-local ENABLED, OBJECT_SEEK_SUPPORT_THRESHOLD, OBJECT_SEEK_SUPPORT_CONCURRENCY_THRESHOLDSrustfs/src/app/object_usecase.rsCache or constant / owner-local cacheObject GET/seek tuning caches and warning guards stay private to object usecase helpers.
SUPPORTED_HEADERSrustfs/src/storage/options.rsCache or constant / owner-local constantSupported-header lookup state stays private to storage option parsing.
AUDIT_TARGET_SPECS, NOTIFICATION_TARGET_SPECSrustfs/src/admin/handlers/audit.rs, rustfs/src/admin/handlers/event.rs, rustfs/src/admin/handlers/plugins_instances.rsCache or constant / owner-local constantAdmin target descriptor tables stay private to their handler owners.
SITE_REPLICATION_PEER_CLIENT, SITE_REPLICATION_STATE_LOCKrustfs/src/admin/handlers/site_replication.rsProcess-global owner-local cache / guardSite-replication peer client cache and state lock stay private to site-replication handlers.
AUDIT_MODULE_ENABLED, NOTIFY_MODULE_ENABLED, PERSISTED_NOTIFY_MODULE_ENABLED, PERSISTED_AUDIT_MODULE_ENABLED, PERSISTED_MODULE_SWITCH_CONFIGUREDrustfs/src/server/audit.rs, rustfs/src/server/event.rs, rustfs/src/server/module_switch.rsProcess-global owner-local togglesAudit/notify module snapshots stay private to the server module switch owners.
DELETE_TAIL_TOTAL, DELETE_CLEANUP_TOTAL, DELETE_REPLICATION_TOTAL, DELETE_NOTIFY_TOTALrustfs/src/delete_tail_activity.rsProcess-global owner-local countersDelete-tail activity counters stay private behind delete-tail activity helpers.
EMBEDDED_SERVER_STARTEDrustfs/src/startup_lifecycle.rsProcess-global owner-local guardEmbedded startup single-start protection stays private to startup lifecycle.
TEST_OUTBOUND_TLS_GENERATIONrustfs/src/admin/runtime_sources.rsTest or fixture stateOutbound TLS generation test hook state stays private to admin runtime-source tests.
TEST_REMAINING_FAILURESrustfs/src/startup_iam.rsTest or fixture stateIAM startup retry injection state stays private to debug/test startup code.
CAPACITY_DIRTY_SCOPE_ENV, CAPACITY_DIRTY_SCOPE_INIT, GLOBAL_ENV, function-local INITrustfs/src/app/*_test.rsTest or fixture stateApp integration test fixture state stays private to the owning test modules.

First Code-Bearing Candidate

GLOBAL_EXPIRY_STATE is the safest first runtime migration candidate:

  • AppContext already exposes ExpiryStateInterface and resolver coverage in rustfs/src/app/context.rs.
  • ECStore access is already concentrated in crates/ecstore/src/runtime/sources.rs.
  • The main external readers can be moved through storage/observability facades before changing lifecycle queue ownership.

Do not migrate GLOBAL_OBJECT_API first. It is coupled to storage startup, object-store resolver publication, IAM-after-storage AppContext initialization, and broad data-plane compatibility.

Verification

Inventory and guardrail PRs should run:

  • bash -n scripts/check_architecture_migration_rules.sh
  • ./scripts/check_architecture_migration_rules.sh
  • cargo fmt --all --check
  • git diff --check

Code-bearing migration PRs must add focused tests for the owner being moved before running broader gates.