docs-internal/engine/depot.md
How the Depot SQLite backend reads, writes, compacts, and fences branchable database storage. Read this before changing anything in engine/packages/depot/.
For VFS-side parity rules, see sqlite-vfs.md. For exact key formats, see sqlite/storage-structure.md.
Depot stores SQLite pages in UDB/FDB. OSS Depot does not include object-backed cold storage.
| Row family | Holds | Owner |
|---|---|---|
DBPTR / BUCKET_PTR | Current database and bucket branch pointers | Conveyer branch APIs |
BUCKET_CATALOG | Database membership facts in bucket branches | Conveyer branch APIs |
BRANCHES / BUCKET_BRANCH | Branch records, refcounts, pin floors, lifecycle generations | Conveyer, GC, workflow checks |
BR/{branch}/META/head | Current database head | Commit path |
BR/{branch}/COMMITS and BR/{branch}/VTX | Commit metadata and versionstamp-to-txid lookup | Commit path |
BR/{branch}/PIDX and BR/{branch}/DELTA | Recent page-owner index and LTX delta chunks | Commit path |
BR/{branch}/SHARD | Reader-visible hot shard versions | Workflow manager and reclaimer |
BR/{branch}/CMP/* | Workflow root and staged hot output | Workflow manager and companions |
BR/{branch}/PITR_INTERVAL | Automatic PITR interval coverage rows | Workflow hot install and reclaim |
RESTORE_POINT and DB_PIN | User retained restore points and exact history pins | Restore point APIs and workflow proof |
The main invariant is simple: commits write deltas directly to UDB; workflow compaction is the only publish/delete authority for compaction output.
Reads resolve the database pointer to a database branch, build a branch-aware read plan, and fetch each page through FDB-backed coverage:
Missing required DELTA/SHARD coverage below EOF is a storage error. The in-process PIDX and branch ancestry caches are perf caches only; correctness comes from UDB rows and workflow revalidation.
SQLite commits call Depot through the conveyer path:
The commit path does not publish SHARD rows or delete old history. It records new committed history and wakes workflow compaction.
Each active database branch has one DB manager workflow plus hot and reclaimer companions, all unique by database branch id.
CMP/stage/{job_id}/hot_shard; the manager validates the active job, copies output to reader-visible SHARD, advances CMP/root, writes selected PITR_INTERVAL rows, and compare-clears matching PIDX.CMP/root watermarks are scheduling summaries, not deletion proof by themselves. CompactionRoot retains legacy cold watermark fields for persisted compatibility, but OSS Depot does not update or act on them.
Automatic timestamp restore coverage is stored as PITR_INTERVAL rows selected during hot compaction from commit wall-clock timestamps and the effective bucket/database PITR policy. Expired interval rows are soft pins until reclaim compare-clears them.
Restore points are retained user tokens. Creating a restore point resolves a SnapshotSelector to exact branch, txid, versionstamp, and wall-clock metadata, then writes a RestorePointRecord and DB_PIN(kind=RestorePoint). Deleting it removes that hard pin and recomputes branch pin floors.
Fork and restore use the same primitive: resolve a snapshot selector, derive a branch at that exact point, and let the caller decide whether to keep a fork or move the database pointer.