docs/references/file/file-manager-architecture.md
SoT scope — this document owns: FileEntry / FileRef data models, physical storage layout, version detection & concurrency control (OCC), atomic writes, recycle bin, reference cleanup, DirectoryWatcher internals, on-demand orphan sweep, DanglingCache state machine, and AI SDK integration design. Module-level concerns (type system, IPC / DataApi contracts, layered architecture, business-service integration, lifecycle assignment) live in
architecture.md. In case of conflict, the layer ownership above decides: positioning / contract → the module-level doc, implementation → this document.When a section describes a behavior (dispatch, OCC, atomic writes, orphan sweep, etc.), read it as the specification the implementation must satisfy. Sections explicitly tagged "(deferred)" describe planned architecture that is not yet implemented.
FileManager provides file management capabilities for two origins; callers choose based on their own needs:
internal: Cherry owns the file content, stored at {userData}/Data/Files/{id}.{ext}. The caller hands the source content to FileManager, which copies it and takes over the lifecycle.external: Cherry only records an absolute path reference on the user's side; does not copy content. File availability and content changes are determined by the user side.The caller decides the origin; FileManager makes no assumptions about the business layer.
Best-effort semantics for external: an external entry is a persistent record that "the caller expressed intent to reference this path at some point in time"—no guarantee the file remains stable, no guarantee content matches the reference-time content. Cherry does no bidirectional DB-FS sync, doesn't track external rename/move; external changes naturally surface as "reading new content next time" or "dangling".
Data categories that do not enter FileManager (auto-derived data, logs, Agent workspace, OCR intermediates, MCP config, files self-managed by FS-first modules, etc.) are detailed in architecture.md §1.3.
Each FileEntry corresponds to a file the user uploaded/saved. FileEntry is a flat individual record—no directory tree, no parent-child relationship, no mount concept.
FileEntry
├── id: UUID (v7 for v2-native entries; v4 preserved from v1 Dexie migration)
├── origin: 'internal' | 'external'
├── name: filename (without extension)
├── ext: extension (without leading dot), nullable
├── size: bytes
├── contentHash: '{algo}:{hex}' content-dedup detection hash; internal-only, null while unknown/in-flight/awaiting repair
├── externalPath: absolute path, non-null only when origin='external'
├── deletedAt: ms epoch | null
├── createdAt / updatedAt
The origin field of each FileEntry defines content ownership:
| origin | Physical location | Ownership | Mutability |
|---|---|---|---|
internal | {userData}/Data/Files/{id}.{ext} | Fully owned by Cherry | Read-write |
external | Absolute path pointed to by externalPath | Owned by user, referenced by Cherry | Changeable by explicit user action (write / rename / permanentDelete apply, delegated to the FS primitives); Cherry does no automatic/watcher-driven modifications; does not track external rename/move—external changes cause the entry to naturally go dangling |
Path uniqueness: at most one entry can exist whose externalPath agrees with another under case folding. Implemented via SQLite functional unique index:
CREATE UNIQUE INDEX fe_external_path_lower_unique_idx
ON file_entry (lower(external_path));
fe_external_path_idx (plain index on the raw external_path) backs byte-exact lookups (findByExternalPath, rename re-finds, path-resolution call sites). The functional index simultaneously serves the case-insensitive lookup path (WHERE lower(externalPath) = lower(?)) used by findCaseInsensitivePeers and enforces the uniqueness invariant — ensureExternalEntry MUST resolve case-collisions at the application layer before INSERT (see "Duplicate-entry detection on insert" below) because a DB-level rejection would otherwise surface as an opaque SQLITE_CONSTRAINT. Internal rows (externalPath = NULL) are exempt — SQLite treats multiple NULLs as distinct in a UNIQUE index.
Stored form of externalPath — byte-faithful: externalPath is persisted exactly as the OS handed it to us, with only lexical cleanup (null-byte reject, .//..//repeated-separator collapse, trailing-separator trim). It is not Unicode-normalized, so the stored string always reaches the real file on every filesystem — including normalization-sensitive ones (Linux ext4/btrfs) where an NFC-rewritten path would not exist on disk (see Rejected: Unicode (NFC) normalization below). Uniqueness is therefore byte-exact plus case-fold only: the lower() functional index catches identical and case-different paths; it deliberately does not fold Unicode forms.
Compile-time enforcement via the AbsoluteFilePath brand: AbsoluteFilePathSchema.parse() returns a branded AbsoluteFilePath (Zod brand, zero runtime cost; see src/shared/types/file/common.ts), and the external-entry lookup/write surfaces (findByExternalPath, setExternalPathAndName) accept the narrower CanonicalFilePath produced by the canonicalizeFilePath() factory. Every DB read/write surface that filters by externalPath MUST accept the branded type, not a plain string, so a caller cannot pass an unvalidated raw path and silently miss all matches. (AbsoluteFilePathSchema only validates the absolute-path shape — it does not mutate the value; see AbsoluteFilePath vs CanonicalFilePath.) "Narrower" is a proper subset, not a spelling normalization: canonicalizeFilePath() is partial over AbsoluteFilePath and throws on input it cannot reduce to a key (today: UNC — see UNC paths). Holding an AbsoluteFilePath does not entitle a call site to assume it can obtain the CanonicalFilePath these surfaces demand.
| Source | Natively canonical | Relies on normalization to disambiguate |
|---|---|---|
Electron showOpenDialog | ✅ (OS returns the on-disk true case) | None |
| Drag-drop from Finder/Explorer | ✅ (OS drag source) | None |
User-typed @/path/... / clipboard paste | ❌ | Risk of case / NFD/NFC |
| External URL scheme / shell integration | ❌ | Same as above |
| v1 migration (inherits Dexie stored values) | ❌ (inherits legacy value quality) | Canonicalize once during migration |
Normalization scope (synchronous, no FS IO) — lexical cleanup only:
Null-byte rejection — raw.includes('\0') → throw, so poisoned paths never reach DB persistence (reject at the earliest boundary, not at use-time inside resolvePhysicalPath)
path.resolve(raw) → absolutize + eliminate ./ ../
Trailing separator trimming
Windows: drive-letter case folding (c:\ → C:\) and separator normalization to \ (so C:/a/b and C:\a\b share one stored form)
UNC rejection — \\server\share\… throws. \\server\share is an indivisible root that ../ must not be able to escape, and neither the POSIX nor the drive-letter branch models that; producing a silently wrong key is worse than refusing. Note the asymmetry this creates with the brand, and that it is deliberate: see UNC paths below.
These steps are purely lexical (no FS IO). Unicode (NFC) normalization is deliberately NOT a step here — an NFC-rewritten NFD path would not exist on disk on normalization-sensitive filesystems, so byte-faithful storage is what keeps the stored path reachable (see Rejected below). The .//../ collapse, by contrast, is not guaranteed to preserve the on-disk target across symlinks/junctions (/a/link/../b ≠ /a/b if link resolves elsewhere) — this cleanup is a lexical dedup-key normalizer, not a reachability/security primitive; use fs.realpath at the main-process boundary when true target equivalence matters.
Intentionally omitted (deferred until concrete user feedback warrants the cost):
fs.realpath as a step inside canonicalizeFilePath itself (would require async FS IO at every call site and a file-existence precondition). fs.realpath IS used on the ensureExternalEntry collision path described below — that is a per-collision probe, not a per-canonicalize step.See the JSDoc for canonicalizeFilePath in src/shared/utils/file/canonicalize.ts for the detailed contract.
AbsoluteFilePathSchema accepts UNC (\\server\share\…); canonicalizeFilePath rejects it. The split is the point, not an oversight — the two gates answer different questions:
| Gate | Question | UNC |
|---|---|---|
AbsoluteFilePathSchema | Is this shape safe to hand to fs? | ✅ Node reads UNC natively on Windows |
canonicalizeFilePath | Is this safe to persist as a byte-compare dedup key? | ❌ no defined \\server\share root handling |
Consequence: a UNC path can be read, previewed, and copied into an internal entry (createInternalEntry copies bytes; it never canonicalizes). It cannot become an external entry, because that requires a canonical externalPath — ensureExternalEntry and findByExternalPath both throw a descriptive UNC error rather than storing a corrupted key.
Why the brand accepts it. UNC is an absolute filesystem path, and the schema was already inconsistent about it: fileUrlToPath decodes a UNC file:// URL to the forward-slash form //server/share/…, which has always passed the POSIX branch. Rejecting only the backslash spelling of the same path meant a network-share file:// snapshot (the shape v1 migration produces) silently became an unreadable, dropped attachment. Accepting both spellings makes the gate describe a property of the path instead of a property of how it happened to be spelled.
Known gap (pre-existing, unchanged): the forward-slash form //server/share/x still passes the POSIX branch of canonicalizeFilePath and is silently reduced to /server/share/x. Only the backslash form is caught. Fixing it means deciding whether // is UNC or a POSIX double-slash path, which is platform-dependent — tracked separately, not addressed here.
URL encoding puts the server in the authority. toFileUrl emits file://server/share/… for a UNC path, not file:////server/share/…. The leading // produced by separator normalization already is the URL authority marker, so appending it to file:// would leave the authority empty and demote the server to path text — a URL Node rejects with ERR_INVALID_FILE_URL_PATH. This also makes toFileUrl the exact inverse of fileUrlToPath, which decodes file://host/… to //host/….
Containment checks degrade, they do not throw. isPathWithinAccessiblePath / getAccessiblePathRelativePath (src/renderer/components/composer/variants/agent/accessiblePath.ts) canonicalize before comparing. A path that cannot be canonicalized is not provably inside anything, so they return false / the input unchanged. Containment is a predicate; it must stay total.
externalPathAn earlier design NFC-normalized externalPath (inside canonicalizeFilePath) before persistence, so a file named with a decomposable character (é = e + combining acute vs the precomposed é) would dedup to one row on macOS. This is deliberately rejected. externalPath is stored byte-faithful (see "Stored form" above) and is never Unicode-normalized.
ensureExternalEntry's fs.stat, the copy source, and resolvePhysicalPath all ENOENT — the file becomes unimportable and unreachable. Breaking Linux functionality to dedup on macOS is not an acceptable trade.showOpenDialog, drag-drop, fs.readdir) ever surface the same file in divergent normalization forms.fs.realpath collision probe below does not catch it (NFD vs NFC are not lower() case-peers), so NFC-in-storage was the sole mechanism folding it — bought at the cost of item 1.If real NFD/NFC duplicates are ever observed, the correct fix is an FS-aware fs.realpath fold at the collision probe (run only when the file exists), never re-introducing Unicode normalization into canonicalizeFilePath or the stored form.
Removing the NFC step removed a reachability hazard (an NFC-rewritten path does not exist on disk). It did not remove the desync hazard, which is a separate problem with a separate cause: the canonical form is application-layer logic whose output is used as a byte-compare key. Any change to canonicalizeFilePath that changes its output desynchronizes historical rows (written under the old rule) from new queries (running under the new rule) — regardless of whether the step consults the filesystem. The failure is silent: byte-compare misses, the user sees "my file is in the library but the app says it isn't", and ensureExternalEntry inserts a duplicate.
Every remaining step is output-sensitive. The .//../ collapse is the likeliest one to be revisited, precisely because "Normalization scope" above already records that it is not target-preserving across symlinks; changing it would invalidate every stored row containing a collapsed segment.
Rule: modifying canonicalizeFilePath ≡ ship a paired Drizzle migration that re-canonicalizes every existing file_entry row with origin='external' in the same PR. No exceptions — even if the new rule is claimed "strictly more permissive", the byte-compare will still miss.
Current exception (expires at the first shipped release). v2 has never shipped, so there are no historical rows to desync —
src/main/data/db/schemas/andmigrations/sqlite-drizzle/are pre-release throwaway per CLAUDE.md. That is why removing the NFC step needed no paired migration. This exemption is temporal, not a property of the rule: it stops applying the moment a release writes real user rows, and it never covered the rule itself.
When a rule change additionally collapses previously-distinct strings to the same canonical form (e.g. folding / and \ on Windows merges C:/a and C:\a), the migration MUST also merge the colliding rows. The rules below are prescriptive; follow them exactly rather than improvising per-migration.
Winner selection when merging rows:
createdAt wins (preserves user-visible history — a 3-year-old entry's creation timestamp is more valuable than a 3-day-old one's).id by lexicographic order (deterministic, no FS-state dependency).Losers' dependents (executed in the same Drizzle transaction as the merge):
fileEntryId = loser.id → deduplicate, then update to winner.id. chat_message_file_ref and painting_file_ref are unique on (fileEntryId, sourceId, role), not on (sourceId, role) — one message legitimately references many files under the same role. So a source that referenced both a loser and the winner (or two losers) yields duplicate rows the moment they are rewritten to winner.id. Delete every loser row whose (sourceId, role) the winner already covers, then update the survivors; that is the semantically correct merge, since the two refs now point at one file. Any conflict remaining after this step is a genuine invariant violation — fail loudly, never ON CONFLICT DO NOTHING. provider_logo_file_ref / mini_app_logo_file_ref are unique on (sourceId) alone, so a source holds at most one row and a plain update cannot collide.file_entry.id = loser.id → delete.loser.id (future file_upload.fileEntryId, business-service caches keyed by entryId) MUST be enumerated and updated in the same migration. If you add a new table that references file_entry.id, the canonicalization migration procedure expands — document the expansion alongside the table's schema.Atomicity: the entire re-canonicalize + merge operation runs in one Drizzle migration transaction. On failure the DB rolls back to the pre-migration state and the next startup re-attempts; partial progress is not possible.
Renderer-side cache invalidation: after the migration runs, some React Query caches keyed by the loser's id may be stale. Because migrations execute before the renderer boots, this is self-healing on the first query — no special coordination required.
Case-insensitive uniqueness on externalPath is enforced at both layers: the functional UNIQUE index fe_external_path_lower_unique_idx (DB) and ensureExternalEntry's pre-INSERT collision check (application). The two-layer scheme keeps the DB-level guarantee unbreakable while letting the application disambiguate the FS-correct interpretation case-by-case.
// Inside ensureExternalEntry, AFTER canonicalize, AFTER findByExternalPath miss,
// AFTER fs.stat verifies the new path exists, BEFORE INSERT:
const peers = await fileEntryService.findCaseInsensitivePeers(canonicalPath)
if (peers.length > 0) {
// `fs.realpath` is the platform-correct probe for "are these the same FS
// entity": on case-insensitive volumes (macOS APFS default, Windows NTFS
// default) the FS folds case to its on-disk canonical form, so two case-
// different inputs resolve to the same string. On case-sensitive volumes
// (Linux ext4, case-sensitive APFS) they resolve to distinct strings.
const reusable = await resolveCaseCollisionPeer(canonicalPath, peers)
if (reusable) return reusable // same FS entity → reuse existing entry
// No peer is the same FS entity. The DB unique constraint will reject the
// INSERT, but we throw early with a descriptive error and full peer detail
// so the caller can decide (rename one of the colliding paths, or surface
// the conflict to the user) instead of seeing an opaque SQLITE_CONSTRAINT.
throw new Error(`ensureExternal: case-collision with existing entries…`)
}
// No peers → safe to INSERT; DB unique constraint is now a redundant safety net.
Behavioral matrix (/foo/A.txt already an entry; user invokes ensureExternalEntry('/foo/a.txt')):
| Filesystem class | fs.realpath('/foo/A.txt') | fs.realpath('/foo/a.txt') | Outcome |
|---|---|---|---|
| Case-insensitive (macOS APFS default, NTFS default) | /foo/A.txt | /foo/A.txt (FS folds) | Same string → reuse existing entry |
| Case-sensitive (Linux ext4, case-sensitive APFS) | /foo/A.txt | /foo/a.txt | Distinct strings → throw case-collision |
Dangling peer (/foo/A.txt missing on disk) | ENOENT | /foo/a.txt | Cannot disambiguate → throw case-collision (caller must permanentDelete the dangling row first) |
Scope:
findCaseInsensitivePeers now runs unconditionally regardless of table size.fs.realpath call resolves symlinks too, which is the right semantic for "same logical file"; symlink targets are intentionally NOT canonicalized at storage time (see "Intentionally omitted" above), so two symlinks pointing at the same target each get their own entry, but a case-different reference to one of those entries reuses it.This subsumes the fs.realpath upgrade that earlier revisions of this section described as "deferred until user feedback" — the same probe is applied at exactly the moment it matters (collision resolution) without paying the FS IO cost on every canonicalize call.
Invariants:
| Field | origin='internal' | origin='external' |
|---|---|---|
name | SoT (user can rename actively) | Pure projection of externalPath (basename) |
ext | SoT | Pure projection of externalPath (extname) |
size | SoT (non-null, ≥ 0) | Always null — no DB snapshot; live value via getMetadata |
externalPath | NULL | Absolute path (the authoritative identity of external) |
contentHash | Detection substrate {algo}:{hex} (for example xxh3-64:…); maintained on every write; null means unknown, in-flight, or awaiting repair | Always null — content lives outside Cherry (fe_contenthash_external_null CHECK) |
For external entries the row stores only identity + stable projections. name / ext do not drift because externalPath is fixed for the lifetime of the entry (external rename by the user surfaces as a dangling entry, not an in-place rewrite of name). size / mtime are served live by File IPC getMetadata(id) on demand — see §3 External Entry Liveness Model.
contentHash)contentHash is a detection substrate for content-level deduplication, not an identity or key. It is orthogonal to external-path deduplication: external paths enforce identity and upsert behavior, while internal hashes only surface possible content matches for a consumer-defined policy.
{algo}:{lowercase hex}; the current producer is XXH3-64 via native @node-rs/xxhash. Keeping the algorithm in the value allows old and new algorithms to coexist during a future incremental migration.createInternalEntry, write, writeIfUnchanged, and createWriteStream persist the current hash and size. Renderer/callers cannot provide an authoritative hash. Bytes, base64, copy, download, and stream sources calculate both values incrementally from the same byte stream that materializes the durable tmp file; no post-write reread or caller-trusted shortcut is used. External rows never carry either snapshot.contentHash = NULL, and records the foreground commitAt in updatedAt; after rename it writes the prepared size / contentHash while preserving the row's current updatedAt. Rename failure restores the old metadata (and restores the old timestamp only when no concurrent metadata update advanced it). Rename success followed by DB-finalize failure keeps the hash null and throws ContentCommittedMetadataPendingError; callers refresh the file state and must not retry the same write automatically.fe_content_hash_idx is a plain index. Multiple logical entries may contain identical bytes, and a collision must never raise a constraint violation. FileEntryService.findInternalByContentHash, exposed to renderers through DataApi GET /files/entries/by-content-hash, returns active candidates oldest-first without selecting one; FileManager retains the same query for Main-side consumers.createInternalEntry never looks up or reuses candidates. A consumer may hash source bytes, query candidates, apply its own secondary check or user decision, and then either reuse or create; the created entry's persisted hash is still derived independently by Main from the bytes it writes.app_state['fileManager:contentMetadataGeneration']. A missing, malformed, or non-v2 marker causes one synchronous transaction to null every non-null internal hash (explicitly preserving updatedAt) and write { version: 2 }. This safely invalidates hashes created by older releases without rewriting an executed migration. After all services are ready, the singleton file.contenthash-backfill job keyset-pages through null rows by id, including trashed rows, and rebuilds size and contentHash from one read. It shares FileManager's per-entry write lock and persists with contentHash IS NULL, so it cannot overwrite a foreground write. Missing or unreadable blobs remain null for the next startup attempt. During reconciliation, dedup queries may temporarily miss candidates but never trust an incorrect non-null hash.Business objects associate with FileEntry through source-owned ref tables plus a shared FileRef projection:
chat_message_file_ref / painting_file_ref / ...
├── fileEntryId → FileEntry (FK, CASCADE delete)
├── sourceId → owning source row (FK, CASCADE delete)
├── role: business-semantic reference role (defined by the source module)
└── UNIQUE(fileEntryId, sourceId, role)
FileRefService aggregates these source-owned tables into the shared FileRef discriminated union for DataApi reads, ref counts, and sweep reporting. It does not own persistent ref writes.
When a persistent business object is deleted, SQLite FK cascade removes its association rows. Relationship replacement (for example, replacing a painting's complete file set) is handled directly by the owning business service.
architecture.md §2FileHandle (polymorphic reference crossing IPC), FileInfo (path-indexed data shape), and the full reference-vs-data-shape symmetry are defined at the module-level architecture document, not here. This document concerns FileManager's internal implementation only.
FileHandle (tagged union / factories / dispatch): architecture.md §2.2FileEntry vs FileInfo (semantic comparison / field invariants / projection rules): architecture.md §2.3architecture.md §2.4Method applicability inside FileManager:
| Category | Methods |
|---|---|
Accept FileHandle (entry + path branches via IPC dispatch) | read / getMetadata / getVersion / getContentHash / write / writeIfUnchanged / rename / permanentDelete / copy / open / showInFolder |
Accept FileEntryId only (entry-identity operations; no path-handle counterpart) | trash / restore / createInternalEntry / ensureExternalEntry / withTempCopy |
AI SDK SharedV4ProviderReference integration and the file_upload table are deferred until the Vercel AI SDK Files API stabilises. The module-level DataApi surface (ensureUploaded / buildProviderReference / invalidate) is outlined in architecture.md §3.5; the detailed schema and FileUploadService API are retained here in §9 AI SDK Integration for the eventual landing PR.
FileManager is the sole public entry point of the FileEntry management system but is not a 30-method God class. The implementation uses a facade + private pure-function modules pattern. (Note: "sole public entry" is scoped to FileEntry — the file module also hosts DirectoryTreeBuilder as a parallel top-level primitive for in-memory tree mirrors; see architecture.md §1.2 and directory-tree.md.)
A method-by-method audit of FileManager's public API for "does it depend on class instance state" concludes: the vast majority of methods do not depend on instance state.
| State | Users | Ownership |
|---|---|---|
versionCache (LRU) | write / writeIfUnchanged / getVersion | class private field (held by FileManager instance) |
fileEntryService / fileRefService | All DB operations | container singleton (application.get(...)) |
danglingCache | External-related methods | file-module singleton (module import) |
@main/utils/file/* | All FS operations | pure functions, stateless |
| IPC handler registration handles | lifecycle | managed by onInit / onStop |
Only versionCache and lifecycle artifacts are truly bound to the FileManager instance; business methods themselves are stateless.
src/main/services/file/
├── index.ts ← barrel: exports FileManager + public entry-facing types/helpers
├── FileManager.ts ← facade class; lifecycle + legacy IPC + versionCache + inline getMetadata
├── internal/ ← private implementation (not re-exported by index.ts; external imports forbidden)
│ ├── deps.ts — FileManagerDeps type
│ ├── dispatch.ts — FileHandle.kind dispatch helper (entry vs path adapter)
│ ├── entry/
│ │ ├── create.ts — createInternal / ensureExternal
│ │ ├── lifecycle.ts — trash / restore / permanentDelete + batches
│ │ ├── rename.ts
│ │ └── copy.ts
│ ├── content/
│ │ ├── read.ts — entry-aware read / createReadStream
│ │ ├── write.ts — write / writeIfUnchanged / createWriteStream
│ │ └── hash.ts — getContentHash / getVersion
│ ├── system/
│ │ ├── shell.ts — open / showInFolder
│ │ └── tempCopy.ts — withTempCopy
│ └── orphanSweep.ts — FS-level orphan sweep
├── utils/
│ ├── content.ts — consistent path read + path conditional write
│ ├── metadata.ts — path-arm metadata projection
│ └── pathResolver.ts — FileEntry path resolution + external canonicalization
└── versionCache.ts ← LRU type definition
getMetadata is the one entry-level read that does NOT live under
internal/entry/ — it is implemented inline on the FileManager class
because it is a thin wrapper around fs.stat with no entry-flow logic
of its own. Adding a future entry-flow concern (e.g. presence event
emission on success) would justify extracting it, but until then the
inline definition keeps the facade's stat path single-hop.
Each internal/* pure function explicitly receives FileManagerDeps:
// internal/deps.ts (illustrative — see src/main/services/file/internal/deps.ts for the authoritative definition)
export interface FileManagerDeps {
readonly fileEntryService: FileEntryService
readonly fileRefService: FileRefService
readonly danglingCache: DanglingCache
readonly versionCache: VersionCache
}
// internal/entry/create.ts — two APIs, corresponding to two public methods on the FileManager facade
// Note: CreateInternalEntryParams is a source-discriminated union
// (source: 'path' | 'url' | 'base64' | 'bytes'); each branch only exposes content
// that name/ext cannot be derived from. Full matrix in `src/shared/file/types/ipc.ts`.
export async function createInternalEntry(
deps: FileManagerDeps,
params: CreateInternalEntryParams
): Promise<FileEntry> {
// Extract { name, ext, bytes } by source branch → write physical file → DB insert; always produces a new entry
}
export async function ensureExternalEntry(
deps: FileManagerDeps,
params: EnsureExternalEntryParams
): Promise<FileEntry> {
// Upsert by externalPath: reuse the existing row or insert a new one
// (external entries cannot be trashed, so there is no restore branch)
}
// FileManager.ts (illustrative — see src/main/services/file/FileManager.ts for the authoritative wiring)
@Injectable('FileManager')
@ServicePhase(Phase.WhenReady)
export class FileManager extends BaseService implements IFileManager {
private readonly _versionCache = createVersionCacheImpl(2000)
private readonly deps: FileManagerDeps = {
fileEntryService,
fileRefService,
danglingCache,
versionCache: this._versionCache
}
// Public API: thin delegates. Internal modules export entry-flavoured
// functions directly (no `*ByEntry` suffix — see §1.6.5); `*ByPath`
// siblings exist only on the path branch and are NOT exposed here.
createInternalEntry(params) { return createInternal(this.deps, params) }
ensureExternalEntry(params) { return ensureExternal(this.deps, params) }
read(id, opts?) { return read(this.deps, id, opts) }
trash(id) { return trash(this.deps, id) }
// ... one line per method
protected async onInit() {
await this.deps.danglingCache.initFromDb()
this.registerIpcHandlers()
// Auto-policy reaper: a non-awaited startup pass + an idle-gated interval
// (file-entry-cleanup.md §5.5). The tick also fires the FS orphan sweep
// concurrently, behind its own weekly floor (`fileSweepTick`). The `runSweep`
// umbrella is separate — on demand via IPC only.
void this.runEntryCleanup()
this.registerInterval(() => this.entryCleanupTick(), CLEANUP_INTERVAL_MS)
}
}
Dispatch location: FileHandle.kind dispatch stays at the IPC handler registration site. Rationale:
FileHandle is the input shape at the IPC serialization layer—the renderer sends a { kind, ... } tagged union, and post-deserialization kind-dispatch is a "request interpretation" concern—it is the proper responsibility of the IPC adapter layerFileEntryId); main-side business service calls are intuitive without needing a createFileEntryHandle(id) wrapperFilePathHandle branch only needs the IPC handler; main-side business services hold FileEntries—they have no arbitrary-path scenarioImplementation convention: entry-aware actions stay under internal/* and
receive FileManagerDeps; renderer-facing path-arm actions have no entry state
to coordinate, live under utils/*, and are re-exported by the module barrel:
// internal/content/read.ts
export async function read(deps, entryId, opts): Promise<ReadResult<T>>
// utils/content.ts
export async function readByPath(path, opts): Promise<ReadResult<T>>
export async function writeIfUnchangedByPath(path, data, version): Promise<FileVersion>
Naming convention (per the shipped exports): entry-flavoured variants
use the bare verb (read, createInternal, ensureExternal, trash,
copy, rename, …); path-flavoured siblings carry the *ByPath suffix.
The bare entry variant is what FileManager's public method delegates to.
Renderer-facing *ByPath variants do not flow through FileManager's public
methods — they serve the path-handle branch of the IPC handler and live in
utils/* so internal/* remains private.
Unified style for dispatch helper: generic FileHandle routes use the file
module's dispatchHandle helper at the renderer transport boundary. Operations
whose contract is intentionally path-only call their path helper directly:
// src/main/ipc/handlers/file.ts
export const fileHandlers = {
'file.read': async ({ handle, options }) =>
options.mode === 'range'
? dispatchHandle(
handle,
id => fileManager.readChunk(id, options.offset, options.length),
path => readChunkByPath(path, options.offset, options.length)
)
: dispatchHandle(
handle,
id => fileManager.read(id, { encoding: options.encoding }),
path => readByPath(path, { encoding: options.encoding })
),
'file.write_if_unchanged': async ({ path, data, expectedVersion }) =>
writeIfUnchangedByPath(path, data, expectedVersion)
}
Impact of adding a new handle kind (e.g., virtual pointing into archive members, remote pointing to an S3 URI):
src/shared/data/types/file.ts — add variant to handle unioninternal/*/*.ts or utils/*.ts — add the entry-aware or path-like operationsrc/main/ipc/handlers/file.ts — extend dispatchHandle; each IPC handler explicitly handles that kind (or throws "unsupported")The renderer extension surface is concentrated in the File IPC adapter—it is immediately obvious which kinds each IPC method supports, which aids auditing.
| Location | May import | Forbidden to import |
|---|---|---|
| Main-side business service (KnowledgeService, MessageService, etc.) | @main/services/file (FileManager + selected path helpers) / @main/utils/file/{fs,path,metadata,search,shell} / @main/services/file/watcher | Deep imports into @main/services/file/internal/** or @main/services/file/utils/** |
Inside the file module itself (internal/*, watcher/*) | May reference each other as needed; may also import @main/utils/file/* primitives | Except FileManager, must not import internal/* |
| External Node/renderer | N/A (file-module is main-side) | — |
Boundary enforcement: the src/main/services/file/index.ts barrel does not
re-export general internal/ implementation. dispatchHandle is the temporary
documented exception while legacy handlers remain; path-arm operations are
implemented in utils/* and re-exported through the barrel. ESLint's
barrel/closed rule rejects deep imports.
| Option | Adopted? | Rationale |
|---|---|---|
| Split business methods into 5 lifecycle services | ❌ | Overkill—lifecycle registration, dependency ordering, and test mocking costs all 5×, in exchange only for "methods split across files" |
FileManager as facade + internal/* pure functions | ✅ | Only 1 lifecycle node; pure functions can be unit-tested with stub deps directly; external API surface remains stable |
FileAccessor as a standalone class handling FileHandle dispatch | ❌ | Dispatch belongs to the IPC adapter layer; the adapter reuses dispatchHandle from internal/dispatch.ts via the file-module barrel, so another class would add pure complexity |
| FileManager public API switched to handle-native | ❌ | IPC and Main-side call contracts need not share shape; main-side business services using entry-native directly is more intuitive, without needing a createFileEntryHandle wrapper |
| Extract versionCache as a module singleton | ❌ | As a FileManager private field, it naturally supports test isolation (new instance = fresh cache) |
FileManager exposes three typed Event<T> on its instance surface and forwards each to every live renderer window. The public contract and queryKey invalidation table live in architecture.md §3.6; this section pins down the emission mechanics that the FileManager implementation must satisfy.
class FileManager extends BaseService {
readonly onEntryRowChanged: Event<EntryRowChangedEvent>
readonly onEntryContentChanged: Event<EntryContentChangedEvent>
readonly onDanglingStateChanged: Event<DanglingStateChangedEvent>
private readonly _entryRow = new Emitter<EntryRowChangedEvent>()
private readonly _entryContent = new Emitter<EntryContentChangedEvent>()
// onDanglingStateChanged is re-exposed from DanglingCache — see §11.7
constructor(private readonly windowManager: WindowManager) { super() }
protected override onInit(): void {
this.registerIpcHandlers()
danglingCache.initFromDb()
// Wire internal events → renderer broadcast. Each disposable auto-cleans on stop.
this.registerDisposable(this.onEntryRowChanged((e) =>
this.windowManager.broadcast('file-manager-event', { type: 'entry-row', ...e })))
this.registerDisposable(this.onEntryContentChanged((e) =>
this.windowManager.broadcast('file-manager-event', { type: 'entry-content', ...e })))
this.registerDisposable(this.onDanglingStateChanged((e) =>
this.windowManager.broadcast('file-manager-event', { type: 'dangling-state', ...e })))
}
}
Emission point per mutation (all emits happen after the DB transaction commits, never inside):
| Method | Emit |
|---|---|
createInternalEntry / batch | onEntryRowChanged { kind: 'created' } per newly-inserted row |
ensureExternalEntry / batch | Insert branch → onEntryRowChanged { kind: 'created' }. Reuse branch → no emit (no state change). |
write / writeIfUnchanged | After atomic rename + (internal-only) DB size update commits → onEntryContentChanged { id, version }. StaleVersionError → no emit. |
createWriteStream | On stream 'finish' → emit. On 'abort' / 'error' / .destroy() → no emit. |
rename | After DB commit (and FS rename for external) → onEntryRowChanged { kind: 'updated' }. |
trash / restore / batch | After DB update commits → onEntryRowChanged { kind: 'updated' } per affected id. |
permanentDelete / batch | After DB delete commits (internal: FS unlink runs first; external: FS untouched per §1.2) → onEntryRowChanged { kind: 'deleted' }. CASCADE-dropped association rows emit no extra events — the renderer invalidates ['fileManager', 'entries'] and refetches. |
copy | Creates a new internal entry → emit onEntryRowChanged { kind: 'created' } for the new id only (source is untouched). |
Atomicity & crash semantics: emits are plain Emitter.fire() calls, not part of the DB transaction. A process crash between commit and fire loses the event. This is acceptable because:
staleTime contract (see architecture.md §4.1.1) refreshes the query on its natural cadenceEvents are accelerators, not authoritative notifications. This is an explicit design choice documented in architecture.md §3.6.
Multi-window fan-out: windowManager.broadcast iterates every live webContents in the window pool and sends the IPC payload. Backgrounded / minimized windows still receive the IPC (Electron delivers to paused webContents); they process it on resume via React Query's standard refresh-on-focus behavior, and any brief overlap with the renderer's first query after resume is resolved idempotently by queryClient.invalidateQueries. Windows whose renderer bundle has not finished loading miss the event entirely, but their first query hits a fresh state with no ghost data.
Testing contract: unit tests of mutation methods must not depend on broadcast — windowManager.broadcast is mocked in test fixtures. The emission-point table above is the contract tests should assert against (e.g. "ensureExternalEntry reuse branch does not emit").
Physical paths are not persisted; resolved at runtime based on origin:
function resolvePhysicalPath(entry: FileEntry): string {
if (entry.origin === 'internal') {
return application.getPath('feature.files.data', `${entry.id}${entry.ext ? '.' + entry.ext : ''}`)
}
return entry.externalPath!
}
internal physical paths are always flat: {userData}/Data/Files/{uuid}.{ext}, and do not change with the FileEntry's name. UUID naming makes internal files invisible and not manually organizable by the user—this is an intentional design choice.
external physical paths are entirely determined by the user; Cherry does not touch them.
{userData}/Data/Files/
├── {uuid-1}.pdf
├── {uuid-2}.png
├── ...
└── {uuid-n}.tmp-{uuid} ← Temporary files for atomic writes (abnormal residues cleaned by `runSweep`)
Cherry creates no subdirectories under {userData}/Data/Files/. All internal files are stored flat.
Transient processing files (OCR intermediates, PDF pagination, archive extraction, etc.) do not create FileEntry and use @main/utils/file/fs primitives directly under {userData}/temp/ (or process-level os.tmpdir()). After processing, the business side cleans up or relies on OS mechanisms.
The external file can be modified or moved by the user at any time. Rather than carrying a DB snapshot that silently drifts (and then chasing it with "refresh" paths), file_module stores only the fields that cannot drift while the entry exists:
Field on file_entry (external) | Source of truth | Drift possible? |
|---|---|---|
id, origin, createdAt, updatedAt | DB row | No |
externalPath | User intent at registration time | No (user-explicit changes go through ensureExternalEntry(newPath)) |
name / ext | Pure projection of externalPath (path.basename / path.extname) | No (stable as long as externalPath is stable) |
size | Not stored — always null (enforced by fe_size_internal_only CHECK) | N/A |
Live size / mtime for an external entry are obtained via File IPC getMetadata(id) (fs.stat on demand). This makes the freshness cost explicit at the call site rather than hiding a stale snapshot behind the FileEntry.size field.
The classic "DB snapshot + refresh paths" design produces two symmetric defect classes:
FileEntry.size assuming freshness, missing the part of the doc that says "snapshot may be stale".Making size unavailable on the row eliminates both: the renderer cannot read a stale value (there is nothing to read), and the main-side code has no snapshot to maintain. The cost — one extra fs.stat per external row when size is actually needed — is localized and observable.
Paths that would otherwise need to refresh a snapshot: read / getVersion / getContentHash on external still run fs.stat as part of their own work (and update DanglingCache as a side effect), but they do not write to the DB row — no size column exists to refresh.
Cherry does not track external rename: after a user mv/rename outside of Cherry, the corresponding entry goes dangling. The user must re-@ inside Cherry to establish a new reference at the new path via ensureExternalEntry(newPath).
When an external file does not exist on disk (or is inaccessible), the corresponding entry is called dangling. Dangling state is maintained by DanglingCache (a file_module singleton); see §11 for details.
Three states:
| State | Meaning |
|---|---|
'present' | The file was recently observed to exist (watcher event / stat success / ops operation observation) |
'missing' | The file was recently observed to be absent (watcher unlink / stat ENOENT) |
'unknown' | No watcher coverage, no prior stat (or cache was actively cleared) |
Detection timing:
getDanglingState / batchGetDanglingStates query → danglingCache.check(entry) (synchronous on cache hit, single fs.stat on cold miss). DataApi never reads this cache.createDirectoryWatcher(), the factory auto-wires add/unlink events into DanglingCacheUI semantics: dangling entries show a failed style in the UI (grayscale, icon marker), but dangling state itself never triggers cleanup—the ref association chain is preserved. manual entries require the user to explicitly "Remove from library" or attempt to re-point; delete_when_unreferenced entries are instead reclaimed by the ref-count-driven cleanup pass (file-entry-cleanup.md) regardless of presence.
interface FileVersion {
mtime: number // ms epoch
size: number
}
Used as a fast signal for detecting external changes. Two tiers of usage:
statVersion(path) (microsecond-level, covers 99% of cases)contentHash(path) → tagged XXH3-64 (millisecond-to-second level, used when mtime/size match but further confirmation is needed). This uses the same algorithm as persisted dedup detection, but the OCC value is computed on demand and is not persisted by the comparison itself.Rationale for mtime + size as a signature:
interface ReadResult<T> {
content: T
mime: string
version: FileVersion
}
read(id, opts?: { encoding?: 'text' }): Promise<ReadResult<string>>
read(id, opts: { encoding: 'base64' }): Promise<ReadResult<string>>
read(id, opts: { encoding: 'binary' }): Promise<ReadResult<Uint8Array>>
read uniformly returns { content, mime, version }. Returning version comes at near-zero cost—stat is already required on the read path.
write(id, data): Promise<FileVersion>
writeIfUnchanged(id, data, expectedVersion: FileVersion): Promise<FileVersion>
Two independent methods that force the caller to explicitly choose whether conflict detection is needed:
| Caller | Which to use | Reason |
|---|---|---|
| First-time write, overwrite, migration, preprocessing | write | No concurrency semantics |
| Editor save (Notes, Markdown, and other potential future consumers) | writeIfUnchanged | Must detect external changes |
On conflict, writeIfUnchanged throws StaleVersionError, and the caller decides on UX after catching (dialog, three-way merge, keep both versions, etc.). Both entry and raw-path arms fully prepare the tmp payload first, then re-stat immediately before rename; an edit that lands while a large payload is being materialized is therefore detected before replacement.
Behavior on external: write / writeIfUnchanged / createWriteStream / rename / permanentDelete all apply—Cherry supports user-explicitly-triggered external file modifications (editor save, UI rename, user-confirmed delete), delegated to the FS primitives at @main/utils/file/fs (atomic write / rename / remove). Cherry does not perform automatic / watcher-driven external file modifications.
FileManager maintains Map<FileEntryId, CachedVersion> internally (LRU, ~2000 entries):
| Trigger | Action | Phase |
|---|---|---|
write / writeIfUnchanged completes | set(id, new version) | Phase 1 (shipped) — see internal/content/write.ts:42, 68, 83 |
| Critical path detects external change | set(id, new version) | Deferred — paired with the change-detector that would observe "external change" outside the watcher path |
| Startup reconcile completes | clear() | Deferred — no startup reconcile pass exists yet |
Trust boundary: the cache only accelerates getVersion queries and is not used for critical decisions. writeIfUnchanged's concurrency comparison must re-stat; it does not trust the cache.
All writes (entry/internal to userData, entry/external to externalPath, path-handle to any path) use PreparedAtomicWrite and follow the POSIX atomic flow:
1. Create {target}.tmp-{uuid} in the same directory
2. Write data while incrementally deriving size + contentHash
3. fsync(tmp fd) ← data flushed to disk; state = prepared
4. rename(tmp, target) ← atomic replacement (POSIX guarantee)
5. fsync(dir fd) ← rename metadata flushed to disk; state = committed
Key rules:
{target}.tmp-{uuidv7}—UUID avoids concurrent-write conflicts^.+\.tmp-<uuidv7>$commit() and abort() are terminal and idempotent; a prepared write can enter only one terminal stateFor an internal entry, steps 3–5 are wrapped by the recoverable metadata protocol described in §1.2: mark hash unknown before rename, finalize exact prepared metadata afterward. A DB failure before rename leaves the target and old metadata untouched; a DB failure after rename leaves a durable null recovery marker instead of a stale non-null hash.
createWriteStream(id): Promise<AtomicWriteStream>
Stream writes also follow the same prepared commit. The returned AtomicWriteStream extends Writable; _final() does not complete until rename, DB finalization, and version-cache update all succeed, so finish is an end-to-end success signal. .abort() / stream errors before commit unlink the tmp and do not replace the target. During shutdown, FileManager aborts streams that have not entered commit and waits for streams that have.
The atomicWriteFile / atomicWriteIfUnchanged / createAtomicWriteStream primitives exported by @main/utils/file/fs are open to modules outside the file module. BootConfig, MCP oauth storage, and any other main-process service that needs a safe atomic write imports them directly; scattered ad-hoc tmp+rename implementations are not introduced.
Renderer-provided raw paths are a separate trust boundary. Before any renderer-reachable path write/move/rename/delete/mkdir/import, Main rejects lexical or realpath overlap with application.getPath('feature.files.data'), including ancestors and symlinks into the managed tree. Internal FileManager operations use entry ids and are not routed through that raw-path guard.
All soft deletes are implemented via the deletedAt timestamp, without physically moving files:
| Operation | Physical impact (internal) | Physical impact (external) |
|---|---|---|
trash(id) | None | N/A (fe_external_no_delete CHECK rejects; external rows cannot be trashed) |
restore(id) | None | N/A (no trashed external rows to restore) |
permanentDelete(id) | DB delete + best-effort remove(physicalPath) (@main/utils/file/fs) | DB delete only — user's file is never modified (matches architecture.md §3.4) |
trash / restore are internal-only. External entries cannot be trashed by definition (fe_external_no_delete CHECK enforces this); the trash semantics make sense only for files Cherry owns.
permanentDelete on internal: DB row is removed first, then the physical file at {userData}/Data/Files/{id}.{ext} is best-effort unlinked. Unlink failures (ENOENT, insufficient permissions, etc.) are logged but do not block — the DB-row-gone outcome is what callers observe; any orphaned blob is later cleaned by the next user-triggered orphan sweep (§10).
permanentDelete on external: DB row is removed; the user's file at externalPath is never modified — Cherry only owns the reference, not the content. This is the only safe contract: silently deleting user files from inside the app would violate the "best-effort external reference" semantics (§1.0.2 in architecture.md). Users who actually want the underlying file gone do so through their OS file manager.
Status: design only. Phase 1 ships no expiry timer service, no Preferences key, and no
WHERE deletedAt < now() - retentionMsquery. Trashed entries persist until the user runs an explicitpermanentDelete(or the next user-triggered orphan sweep collects an already-deleted entry's residual blob). The 30-day window below is the proposed retention; the actual default and configurability land with the timer service.
By default trashed entries are cleaned up after 30 days (lifecycle service timer); the user may configure the days or disable it in Preferences.
Query: WHERE deletedAt < now() - retentionMs → batch permanentDelete.
| Scenario | Handling |
|---|---|
| unlink fails on permanentDelete internal (file already missing, permission issue) | Log warn; the DB row is already gone, so the failure surfaces only as an orphan blob that the next user-triggered orphan sweep will reclaim |
| permanentDelete on external | DB-only by design; the user's file at externalPath is never touched — Cherry owns only the reference |
ensureExternalEntry(path) when an entry for the same path already exists | Entry point first calls AbsoluteFilePathSchema.parse(raw); upsert returns the existing row. External entries cannot be trashed, so there is no "restore" branch. |
| Two entries for the same file | Case differences (macOS APFS, Windows NTFS): rejected at INSERT by the DB lower() functional unique index plus the fs.realpath-based reuse-or-throw probe in ensureExternalEntry (see §1.2 "Duplicate-entry detection on insert"). Unicode (NFD ↔ NFC) differences: deliberately not deduped — externalPath is stored byte-faithful, and the rare, cosmetic macOS case is accepted rather than break reachability on Linux (see §1.2 "Rejected: Unicode (NFC) normalization of externalPath"). |
| External file at original path externally replaced with a different file | Cherry does not check content consistency (best-effort). name / ext on the row are derived from externalPath and do not change; size is always served live by getMetadata. DanglingCache flips to 'present' on the next stat, so the UI just renders the new file under the existing reference. |
| A trashed entry is permanently externally deleted and then restored | Appears dangling (DanglingCache returns missing on next check), UI shows failed style |
| External write with permission error / disk full on target path | Throw without polluting DB; caller decides retry or user notification |
Three layers of protection, with each layer as a fallback for the next:
+-------------------------------------------------------+
| Layer 1: fileEntryId CASCADE |
| FileEntry deleted -> ref rows auto-cascaded |
| file_upload auto-cascaded |
| (DB FK constraint, zero app code) |
+-------------------------------------------------------+
| Layer 2: source FK cascade / relationship replacement |
| business entity deleted -> source-FK cascade |
| relationship replaced -> explicit cleanup+insert |
+-------------------------------------------------------+
| Layer 3: on-demand DB orphan sweep |
| report zero-ref manual entries |
| reclaim zero-ref delete_when_unreferenced entries |
| via the cleanup pass |
+-------------------------------------------------------+
Layer 3 is not a generic persistent-source reconciler. Persistent association rows are FK-constrained and should disappear through Layer 1 / Layer 2 cascades; the sweep reports manual zero-ref entries, and reclaims delete_when_unreferenced zero-ref entries via the cleanup pass described in file-entry-cleanup.md.
The default stance — FileEntry is preserved even when no business refs point at it — is chosen so the user never loses a file they (or Cherry) bothered to track merely because the original consumer got deleted. A UI surface may show an "unreferenced" marker for user-triggered cleanup.
Automatic deletion applies only to entries whose cleanup_policy = 'delete_when_unreferenced' (see file-entry-cleanup.md); manual entries have no automatic deletion exceptions. Even an external manual entry that is currently missing and has zero refs is still a user-visible library record: it may represent a temporarily unmounted drive, a file the user wants to re-link later, or simply a stale record the user should remove explicitly. The file module may report these rows, but it must not delete them without an explicit user/caller action.
Policy matrix by (origin, dangling state, refs): the rows below describe manual-policy behavior. A delete_when_unreferenced entry is instead reclaimed once it clears the grace window with zero refs — see file-entry-cleanup.md §5.
| origin | dangling state | refs | Policy |
|---|---|---|---|
internal | n/a (always 'present') | any | Preserve — user may re-link via UI; only user-initiated cleanup |
external | 'present' | any | Preserve — file still exists, fully re-attachable |
external | 'unknown' | any | Preserve — not yet observed; treated as still-live until proven otherwise |
external | 'missing' | >0 | Preserve — business objects still reference this entry. Automatic deletion would CASCADE-drop association rows and silently mutate user data (messages' attachment count drops, UI state shifts). The business service owning those refs is the right layer to decide replacement / removal policy, not the file module. Reference-oriented UI surfaces (§3.4 UI convention) show these as "file missing" so the user can act. |
external | 'missing' | 0 | Preserve + report — no refs remain, but the row is still user-visible library state. FilesPage / cleanup UI may show "missing" and offer "Remove from library"; no time-based auto-delete. |
This section applies to manual-policy entries. Dangling external manual entries are never deleted automatically by a scheduler, startup task, or runSweep() policy pass — deletion of delete_when_unreferenced entries is handled by the separate cleanup pass in file-entry-cleanup.md, and is driven by ref count and grace window, not dangling state. Cleanup for manual entries is explicit:
runDbSweep may report unreferenced entries by origin so a UI can surface candidates, but it does not delete FileEntry rows based on dangling state or ref count.Rationale:
'missing' state is still not sufficient authority to delete a user-visible library record.refs = 0.Consequences:
('external', 'missing', 0) automatic deletion branch keyed on dangling state. Layer 3's reporting sub-path remains zero-ref reporting for manual entries; delete_when_unreferenced entries are instead reclaimed by the policy-driven cleanup pass (file-entry-cleanup.md), independent of dangling state.DirectoryWatcher is a non-lifecycle general FS primitive (not a service), available for business modules to new themselves. It is merely a chokidar wrapper and binds no business semantics.
Placed in src/main/services/file/watcher/, as a dedicated submodule of the file module distinct from the pure FS primitives at @main/utils/file/*. Rationale for the split:
| Aspect | @main/utils/file/* primitives | watcher/ |
|---|---|---|
| Paradigm | Pure functions (stateless) | Stateful class |
| Lifecycle | None (completes upon call) | Has one (start → running → dispose) |
| Resource holding | None | FSWatcher instance + pending queues + timers |
| Consumption contract | const x = await read(path) | const w = new DirectoryWatcher(...); ... w.dispose() |
Grouping a stateful class with pure-function primitives would break the primitives' stateless contract. This mirrors the layering between Node.js official fs.readFile (function) and fs.watch returning an FSWatcher instance (class): functionally related but fundamentally different consumption shapes, so they live in separate submodules.
Shipped surface mirrors src/main/services/file/watcher/index.ts —
a single onEvent(listener) subscriber over a normalized event union.
Earlier drafts proposed seven separate event channels (onAdd /
onAddDir / onUnlink / onUnlinkDir / onRename / onReady /
onError) with file vs directory split and built-in rename detection;
the watcher module ships a flat union instead because no current
consumer needs the dir-event split, and rename detection is deferred
to the same change that lands the first onRename consumer (see §8.3).
export type WatcherEvent =
| { readonly kind: 'add'; readonly path: AbsoluteFilePath }
| { readonly kind: 'addDir'; readonly path: AbsoluteFilePath }
| { readonly kind: 'unlink'; readonly path: AbsoluteFilePath }
| { readonly kind: 'unlinkDir'; readonly path: AbsoluteFilePath }
| { readonly kind: 'change'; readonly path: AbsoluteFilePath }
| { readonly kind: 'ready' }
| { readonly kind: 'error'; readonly error: Error }
export type WatcherListener = (event: WatcherEvent) => void
export interface DirectoryWatcher {
/** Subscribe to normalized FS events. Returns an unsubscribe function. */
onEvent(listener: WatcherListener): () => void
/** Stop watching and release OS-level resources. Idempotent. */
close(): Promise<void>
}
export interface CreateDirectoryWatcherOptions {
/** Recurse into subdirectories. Default: true. */
readonly recursive?: boolean
/** Custom ignore predicate. Built-in OS-junk ignores always apply. */
readonly ignore?: (path: AbsoluteFilePath) => boolean
/** Stability window for `awaitWriteFinish` (ms). Default: 200. Set to 0 to disable. */
readonly stabilityThresholdMs?: number
}
export function createDirectoryWatcher(
path: AbsoluteFilePath,
opts?: CreateDirectoryWatcherOptions
): Promise<DirectoryWatcher>
Adding new event kinds: the flat union is additive — extending
WatcherEvent with a new kind is non-breaking to existing
subscribers (they observe and skip), so the dir-split / rename
channels can be reintroduced without a watcher-rewrite if a real
consumer surfaces.
When enabled, unlink/add events are delayed windowMs to attempt pairing as a rename:
onRename is emitted (the matched unlink/add are suppressed)Key guarantee: when enabled, onUnlink/onAdd and onRename do not fire simultaneously, so consumer semantics are clear.
Platform precision:
Only file rename is handled. Directory renames are not specially recognized; consumers combine sub-file events themselves.
OS garbage files are ignored by default (not disable-able):
{ basename: '.DS_Store' }{ basename: '.localized' }{ basename: 'Thumbs.db' }{ basename: 'desktop.ini' }Consumers may append ignored; merged after the default rules.
Business modules new + dispose themselves as needed:
// Illustrative (non-file_module implementation)
const watcher = new DirectoryWatcher({
path: source.basePath,
renameDetection: { enabled: true }
})
watcher.onAdd(...)
watcher.onRename(...)
await watcher.start()
// ...
watcher.dispose()
file_module starts no watcher instances. Whether to monitor external directories is the business module's decision.
⚠️ This section is a design record; the corresponding implementation is deferred. Vercel AI SDK's Files Upload API (
FilesV4,SharedV4ProviderReference) is still pre-release, and the corresponding dependency is unstable. FileUploadService, thefile_uploadtable, and related IPC methods are all deferred to a separate PR after the SDK reaches stable. This section preserves the design intent for direct landing in the future.
Cherry needs to integrate with the Vercel AI SDK's file upload API. The SDK's SharedV4ProviderReference models "the same logical file may be uploaded to N providers, each with its own fileId".
When it lands, a dedicated file_upload table tracks these uploads, decoupled from fileEntry.
CREATE TABLE file_upload (
id TEXT PRIMARY KEY,
file_entry_id TEXT NOT NULL REFERENCES file_entry(id) ON DELETE CASCADE,
provider TEXT NOT NULL,
remote_id TEXT NOT NULL,
content_version TEXT NOT NULL, -- XXH3-64 at upload time
uploaded_at INTEGER NOT NULL,
expires_at INTEGER,
status TEXT NOT NULL, -- 'active' | 'expired' | 'failed'
metadata TEXT, -- JSON, provider-specific
UNIQUE(file_entry_id, provider)
);
interface IFileUploadService {
ensureUploaded(fileEntryId: FileEntryId, provider: string): Promise<FileUpload>
buildProviderReference(fileEntryId: FileEntryId): Promise<SharedV4ProviderReference>
invalidate(fileEntryId: FileEntryId): Promise<void>
}
ensureUploaded logic:
file_uploadprovider.files().uploadFile() → upsert file_uploadbuildProviderReference assembles all active uploads into Record<provider, remoteId>.
invalidate + re-uploadThe FS-level pass (§10) runs unattended from FileManager.fileSweepTick — the same idle-gated tick as the entry cleanup, concurrently with it, behind a 7-day floor. That is what reclaims orphan blobs in production. The DB-level report pass (§7 Layer 3) has no scheduled trigger and runs only inside the runSweep umbrella, which is reachable solely via the File_RunSweep IPC channel — a channel with no renderer caller today. FileManager exposes a single runSweep() maintenance method: it first runs the entry-cleanup pass (auto-run separately on init/interval — see file-entry-cleanup.md §5), then runs the FS-level pass and the DB-level report pass concurrently, folding the cleanup pass's own summary into counts.entryCleanup, and returns a single OrphanReport once all three settle. The FS and DB passes each begin with a hasPendingRestore() guard (src/main/data/db/restore/restoreJournal.ts): while a staged backup restore awaits promotion, the sweep stands aside with outcome: 'aborted', abortReason: 'pending-restore' — a staged restore's blobs are on disk but not yet referenced by the live DB, which is exactly what the sweep would otherwise reclaim. (No user-facing UI calls runSweep — the entry cleanup it wraps is silent, and the cleanup mechanism has no user surface; see file-entry-cleanup.md's Decision note.)
protected override async onInit(): Promise<void> {
// DanglingCache reverse index built from DB before any IPC accepts
// a dangling query, so a renderer cannot race the first call.
await this.deps.danglingCache.initFromDb()
// IPC handlers, including `File_RunSweep`, are registered here.
this.registerIpcHandlers()
// Entry-cleanup pass auto-runs here (previous-session backlog) and on a
// 30min idle-gated interval — independently of `runSweep`. See
// file-entry-cleanup.md §5.5.
void this.runEntryCleanup()
this.registerInterval(() => this.entryCleanupTick(), FileManager.CLEANUP_INTERVAL_MS)
}
async runSweep(): Promise<OrphanReport> {
// Three passes, cleanup first:
// 1. Entry-cleanup pass (file-entry-cleanup.md §5): reclaims zero-ref
// `delete_when_unreferenced` entries. No volume abort (spec §5.3).
// Runs first so the DB report below doesn't re-report entries it
// just reclaimed.
// 2. FS-level file sweep (§10): scan {userData}/Data/Files/* for
// orphans not present in the file_entry snapshot.
// 3. DB-level entry report (§7 Layer 3): report unreferenced
// `manual` entries.
// Passes 2/3 settle independently with their own error capture. A DB
// failure dominates as `failed`; FS-side partial/aborted/failed outcomes
// degrade the umbrella report to `partial` via `fsSweepIssue`. The
// cleanup pass's own outcome rides in `counts.entryCleanup` and never
// changes the umbrella `outcome`. Exception: while a staged backup restore
// is pending promotion, BOTH passes stand aside up front and the umbrella
// returns `aborted` (abortReason: 'pending-restore') verbatim — expected
// behavior, never disguised as a degraded `partial` run.
}
Rationale for the split (scheduled FS pass, on-demand report):
unlinkFailures, plus crash residue between row-delete and unlink). Reclamation cannot wait on a caller that does not exist, so it rides the idle tick — but behind a coarse floor, because an orphan blob costs disk, never correctness.runSweep and gets both halves plus the entry-cleanup summary in one OrphanReport.lastDbSweepReport / lastDbSweepRanAt. UIs that want "last scan" timing should hold the previously-returned OrphanReport.lastRunAt themselves.A note on initVersionCache: an earlier draft of this section bundled a synchronous initVersionCache() call into onInit. It didn't survive implementation — version cache is per-FileManager-instance and constructs at field-init time (no boot step), so there is no separate init call to make. registerIpcHandlers() did survive and is the convention used across lifecycle services for the same reason it surfaces in lifecycle-migration-guide.md: keeps onInit a narrow init→register sequence and gives a single spot for Phase 2 channels to land.
The sweep uses a single snapshot of file_entry.id taken at sweep start, not per-file DB lookups:
SELECT id FROM file_entry -- one query, held in an in-memory Set for the duration of the sweep
It then diffs readdir({userData}/Data/Files/) against the snapshot in memory. Chosen properties:
mtime > 5min filter in §10.3 is the only thing that protects them — the snapshot strategy makes this reliance explicit rather than hiding it behind DB isolation levels.Implementations MUST NOT silently switch to per-file DB lookups — the snapshot strategy, the mtime > 5min filter (§10.3), and the safety threshold (§10.4) are co-designed; any change to one requires re-evaluating the other two.
The mtime > 5min filter is an engineering heuristic, not a formal guarantee. It is sufficient under the following assumptions:
createInternalEntry's "FS write UUID file → DB insert" window is << 5min in practice (microseconds to seconds)atomicWriteFile's "open tmp → write → fsync → rename" window is << 5min, with streaming writes refreshing mtime on each write(2)userData resides on a local POSIX-compliant filesystem with sub-minute mtime resolution (APFS / ext4 / NTFS local)Known breakage modes (documented, accepted as residual risk):
| Scenario | Consequence |
|---|---|
| Very slow write (huge file + slow disk/fsync) exceeds 5min between FS write and DB insert | Newly-written internal file may be unlinked on the next sweep |
| Process frozen / suspended > 5min mid-write; then a subsequent sweep runs | Same as above |
| System clock jumps forward > 5min after file creation | Recent residue gets mis-aged; usually harmless — those files were orphans anyway |
| System clock jumps backward | Filter becomes permissive (now < mtime disqualifies the file); cleanup delayed to the next sweep run (safe) |
userData on FAT32 / exFAT / SMB / NFS (second-precision or offset-prone mtime) | Filter still works at coarse granularity; extreme clock skew between client and server can mis-age files |
userData on tmpfs / CoW FS with unusual mtime semantics | Out of contract; user responsibility |
The sweep does not defend against deliberately hostile mtime manipulation. These are known limits, not bugs.
To bound the blast radius of application-layer regressions (e.g. a faulty migration that truncates file_entry, a regression in the sweep predicate, a developer running the app against a stale DB), the sweep MUST evaluate its plan before executing and abort on suspicious scale:
interface SweepPlan {
orphanFiles: { path: string; bytes: number }[]
tmpFiles: { path: string; bytes: number }[]
}
function shouldAbort(
plan: SweepPlan,
totalFilesOnDisk: number,
totalBytesOnDisk: number
): boolean {
const toDelete = plan.orphanFiles.length + plan.tmpFiles.length
const toDeleteBytes = sumBytes(plan.orphanFiles) + sumBytes(plan.tmpFiles)
// Normal crash residue: always proceed.
if (toDelete < 20 && toDeleteBytes < 10 * 1024 * 1024) return false
// Otherwise check proportion. If the sweep would erase a large fraction of
// the on-disk UUID population, something upstream is wrong — refuse and
// warn; the next sweep run re-evaluates after the bug is fixed.
const countFraction = toDelete / Math.max(1, totalFilesOnDisk)
const byteFraction = toDeleteBytes / Math.max(1, totalBytesOnDisk)
return countFraction > 0.5 || byteFraction > 0.5
}
Contract:
warn-level structured log (see §10.5) so developers / on-call can diagnose.Error is thrown into the .catch() handler.Scope note: the threshold defends against internal bugs, not user-side manipulation of {userData}/Data/Files/. Users are not expected or encouraged to edit the storage directory (all file operations should go through the in-app entry system). The threshold's job is to ensure "nothing Cherry itself does, internally, silently deletes the bulk of a user's library".
Every sweep run emits one structured log record through loggerService — info on normal completion, warn on partial / aborted outcomes, error on unexpected failure caught by .catch():
{
event: 'orphan-file-sweep', // disambiguates from the DB-side 'orphan-sweep' (§7 Layer 3)
outcome: 'completed' | 'partial' | 'aborted' | 'failed',
entriesInDb: number,
direntsScanned: number, // total readdir entries (informational)
filesOnDisk: number, // UUID files + tmp residue candidates only
bytesOnDisk: number, // bytes of candidates (drives the abort fraction math)
plannedDeleteCount: number,
plannedDeleteBytes: number,
actualDeleteCount: number, // 0 on aborted / failed
actualDeleteBytes: number,
oldestDeletedMtime?: number, // ms epoch of the oldest file unlinked this run
statFailedCount: number, // non-ENOENT stat errors during planning
scanDurationMs: number,
// outcome-specific fields (discriminated union):
// 'partial': failedDeleteCount: number, failedSamples: readonly string[] (capped at 5)
// 'aborted': abortReason: 'count-fraction' | 'byte-fraction' | 'pending-restore'
// 'failed': errorMessage: string
}
The DB-side sweep emits a parallel record under event: 'orphan-sweep'. Its current outcomes are completed, aborted (abortReason: 'pending-restore' — the same stand-aside as the FS pass), or failed: it reports manual entries with zero refs. The shared partial wire branch remains for compatibility, but there is no generic per-source checker pass.
The entry-cleanup pass (§7.1, file-entry-cleanup.md §5.6) emits a third, independent record under event: 'file-entry-cleanup' — info on completed and on 'skipped' (the pending-staged-restore stand-aside), error on failed (it has no aborted outcome; the volume abort was removed, spec §5.3) — covering candidate/deleted/gonePinned/failed counts and skip/unlink-failure breakdowns for the delete_when_unreferenced reclaim path. It fires on its own triggers (init, idle-gated interval) in addition to running as the first of runSweep's three passes (§10.1).
These three records are the single source of truth for post-hoc diagnosis. No separate metrics pipeline is needed — at most three records per user-triggered sweep run is a trivial volume for log aggregation.
The reverse index of DanglingCache (Map<path, Set<entryId>>) is built via a single synchronous DB query:
SELECT id, externalPath FROM file_entry
WHERE origin = 'external' AND deletedAt IS NULL
No stat performed—the state field (Map<entryId, DanglingState>) is initially empty; lazy stat on query (see §11).
The old version batch-stat'd all external entries at startup to build the dangling set. The new version cut this step:
getDanglingState / batchGetDanglingStates); most query scenarios don't need it, so it's never computed eagerly@-ed external paths — the main source of external entries — are almost never covered by any watcher. For those paths, the TTL-based cache (§11.6) and ops observations are the dominant freshness mechanisms, not watchers. Do not read this bullet as "watchers cover most paths"; they don't.| Concurrency scenario | Result |
|---|---|
| createInternalEntry creates a new internal file during sweep | The mtime > 5min filter (§10.3) prevents the new file from being mistakenly deleted; the snapshot strategy (§10.2) makes this reliance explicit |
| FileManager.read/write on existing entries during sweep | No mutual exclusion; read/write follow different code paths and are unaffected |
| Upstream bug causes bulk deletion plan | Safety threshold (§10.4) aborts the sweep without unlinking |
| app exits during sweep | No persistent side effect; the next scheduled fileSweepTick re-derives everything |
file_module's crash window is very narrow:
| Operation | Order | Crash mid-operation | Recovery |
|---|---|---|---|
| createInternalEntry | FS write UUID file → DB insert | Orphan file | Orphan sweep |
| write (internal) | prepare tmp → mark hash null → atomic rename → finalize size/hash | Crash before rename preserves old bytes; crash after rename leaves new bytes + null hash | Startup reconciliation rebuilds null metadata |
| trash / restore / rename | DB only | None | None |
| permanentDelete (internal) | DB delete → best-effort FS unlink | Crash after DB delete leaves an orphan blob | Orphan sweep |
| copy (internal) | FS copy → DB insert | Orphan file | Orphan sweep |
| ensureExternalEntry | DB insert / reuse (doesn't touch user file) | None | None |
| permanentDelete (external) | DB delete only | None — user's file at externalPath is never modified | None needed |
No WAL / pending_fs_ops table needed. Orphan sweep covers the internal crash residue; the external side naturally doesn't need it (delete failure just leaves it on disk).
DanglingCache is a singleton in file_module (not a lifecycle service) that maintains the "latest known on-disk state" for external entries.
// src/main/services/file/danglingCache.ts
export const danglingCache = new DanglingCache()
Role:
getDanglingState / batchGetDanglingStates. Cache hit within TTL (§11.2, 30 min) returns synchronously; TTL-expired or cold miss runs a single fs.stat and refreshes the cache. DataApi never reads this cache — DataApi is pure SQL.@-ed external paths — which are the bulk of external entries — are typically NOT watcher-covered. Do not assume event-driven updates as the primary freshness mechanism; the TTL (§11.6) is.DanglingCache exposes lazy, query-driven presence checks only. There is no cleanup-only recheck path because dangling state never drives deletion — manual entries are not auto-deleted (§7.2), and the cleanup pass for delete_when_unreferenced entries keys on ref count and grace window, not presence (file-entry-cleanup.md). If a future explicit user workflow needs a strict re-stat escape hatch, add it with that concrete caller and document the user-visible action.
type DanglingState = 'present' | 'missing' | 'unknown'
interface CachedState {
state: 'present' | 'missing'
/** ms epoch of last observation — drives TTL expiry in `check` */
observedAt: number
/** Where this observation came from (for diagnostics / log context). */
source: 'watcher' | 'ops' | 'stat'
}
class DanglingCache {
/** TTL for cached observations. 30min — external file path moves are rare,
* and freshness is bounded by TTL for any entry the user actually queries. */
private static readonly TTL_MS = 30 * 60 * 1000
private byEntryId: Map<FileEntryId, CachedState>
private pathToEntryIds: Map<string, Set<FileEntryId>> // reverse index
/** Public event: fires on every genuine state transition (see §11.7). */
readonly onDanglingStateChanged: Event<DanglingStateChangedEvent>
// Query (TTL-aware; re-stats when cache entry is stale)
async check(entry: FileEntry): Promise<DanglingState>
// Event entry (for watcher factory + FileManager ops) — resets observedAt
onFsEvent(path: string, state: 'present' | 'missing'): void
// Index maintenance (for FileManager entry CRUD)
addEntry(entryId: FileEntryId, externalPath: string): void
removeEntry(entryId: FileEntryId, externalPath: string): void
// Startup init
initFromDb(): void
}
Query strategy for check — lazy TTL expiration:
async check(entry: FileEntry): Promise<DanglingState> {
if (entry.origin === 'internal') return 'present'
// L1: cache hit AND still within TTL → return cached
const cached = this.byEntryId.get(entry.id)
if (cached && Date.now() - cached.observedAt < DanglingCache.TTL_MS) {
return cached.state
}
// L2: no cache OR TTL expired → re-stat and update
return this.doStatAndUpdate(entry, 'stat')
}
private async doStatAndUpdate(
entry: FileEntry,
source: CachedState['source']
): Promise<DanglingState> {
const state = await statToState(entry.externalPath!)
const prev = this.byEntryId.get(entry.id)
this.byEntryId.set(entry.id, { state, observedAt: Date.now(), source })
if (!prev || prev.state !== state) {
this._onDanglingStateChanged.fire({ id: entry.id, state })
}
return state
}
Key design points:
observedAt to Date.now() — a path with active watcher coverage stays fresh indefinitely and never triggers TTL-driven re-stat.staleTime ≤ 5min means most renderer refetches hit cache (desired: the cache adds value).Business modules need not be directly aware of DanglingCache. All watchers must be created via the createDirectoryWatcher() factory; the factory subscribes to its own event stream and mirrors presence transitions into DanglingCache before re-emitting the raw event to external subscribers:
add → cache marks presentunlink → cache marks missingchange → cache untouched (file is still present; mtime drift is not tracked here)Because externalPath is stored byte-faithful (see §1.2 "Stored form"), the reverse index is keyed by that exact stored form and the watcher matches chokidar events by raw byte equality — no NFC normalization on either leg. (The NFC step that used to sit in the watcher existed only to bridge to the old NFC-canonical keys; it is removed together with them — see §1.2 "Rejected: Unicode (NFC) normalization".) The path forwarded to subscribers is likewise the raw OS form chokidar saw.
check() is the correctness baseline: it stats entry.externalPath (byte-faithful) directly, so an entry's dangling state is always eventually correct regardless of whether any watcher event matched — the watcher leg is an eager-update latency optimization on top of it, never a correctness dependency. On Linux the raw event byte-matches the stored key by construction; on macOS/Windows it matches when the path source and chokidar agree on Unicode form. If they ever diverge (unverified — same open question as §1.2 "Rejected"), the only effect is that a watcher event misses and the badge stays stale until the next check() — benign and self-healing. The platform-conditional fold that could bridge such a divergence, if ever observed, belongs on the reverse-index comparison key (process.platform-aware: NFC on darwin/win32, identity on linux) — never on the stored form.
Note: watcher rename events do not auto-update an external entry's externalPath—Cherry does not track external rename. After a rename, the original entry goes dangling; the user must re-@ to establish a new reference.
Timing for changes to pathToEntryIds (fully self-governed inside file_module, no DB-FS sync):
| Event | Action |
|---|---|
Startup initFromDb() | SELECT id, externalPath FROM file_entry WHERE origin='external' AND deletedAt IS NULL → batch add |
ensureExternalEntry creates new | addEntry(id, path) |
ensureExternalEntry reuses (upsert hit) | No change (path already indexed) |
permanentDelete(external) | removeEntry(id, path) |
rename(external) (explicit user action) | removeEntry(id, oldPath) + addEntry(id, newPath) |
External entries cannot be trashed (fe_external_no_delete CHECK enforces this
at the schema level; trash / restore throw at the entry layer before
reaching the reverse-index update). Earlier drafts listed restore(external)
and trash(external) rows here — they were dead branches and have been
removed.
The File IPC batchGetDanglingStates handler fans out over the requested ids in parallel:
async function batchGetDanglingStates(ids: FileEntryId[]): Promise<Record<FileEntryId, DanglingState>> {
const entries = await fileEntryService.batchGetById(ids)
const pairs = await Promise.all(
entries.map(async (e) => [e.id, await danglingCache.check(e)] as const)
)
return Object.fromEntries(pairs)
}
TTL-based lazy expiration. A cached entry is considered fresh while now - observedAt < TTL_MS (30 min); once stale, the next check() call re-stats and updates the cache. observedAt is refreshed by any of:
'present' commit through onFsEvent(..., 'ops')). Read / hash / getMetadata / getVersion do not flip the cache to 'present' on success — they only commit 'missing' on ENOENT through the observeExternalAccess chokepoint. The watcher-led design deliberately keeps presence learning out of the passive-read path; see internal/observe.ts for the contract.fs.stat from check() / getMetadata / getDanglingStateFreshness guarantee: for any path the caller queries, cached state is never older than the TTL. Paths that are never queried may stay stale indefinitely — but by construction, no consumer is looking at them, so the staleness has no user-visible impact.
Why no background sweep: a periodic background re-validation across all cached entries was considered and rejected. See §12 Key Design Decisions. The short version: FS IO cost would scale with total entry count instead of query frequency, and deletion is never driven by dangling state (manual entries require explicit action; auto-policy cleanup keys on ref count), so stale presence state should be corrected at use/query time rather than by a hidden global scanner.
Known residual case — stale 'present' with refs > 0: if an external file is deleted outside Cherry, without any watcher or ops observation to signal it, and no UI ever queries getDanglingState for that entry, the cache stays 'present' past TTL boundaries (first query after TTL will re-stat and fix). Business services that depend on referenced files MUST re-validate at use time (read will surface ENOENT anyway); DanglingCache is a UI/presence helper, not a correctness boundary.
DanglingCache exposes onDanglingStateChanged: Event<DanglingStateChangedEvent> fired on every genuine state transition (watcher event, cold-path fs.stat observation after a cache miss, explicit ops observation):
export interface DanglingStateChangedEvent {
id: FileEntryId
state: 'present' | 'missing'
// Note: 'unknown' is never broadcast — transitions FROM unknown TO concrete
// fire; 'unknown' itself is the default pre-observation state, not a signal.
}
FileManager subscribes in onInit (see §1.6.8) and fans the event out to all renderer windows via the shared file-manager-event IPC channel, closing the "main-side FS observation → renderer React Query invalidation" loop. The public queryKey invalidation target is ['fileManager', 'dangling', id]; see architecture.md §3.6 for the full dispatch table.
Emission rules:
onFsEvent(path, 'present') arrives and the cached state is already 'present', no emit. Prevents broadcast floods on watcher-chatty filesystems.externalPath (allowed by schema — one row per path per non-trashed state, but historical / edge cases may produce more), §11.4's reverse index yields all affected ids. Emit one event per id. Renderer invalidates ['fileManager', 'dangling', id] per id.'present' by construction (§3.3); DanglingCache never tracks them.Staleness backstop retained: the staleTime ≤ 5min contract from architecture.md §4.1.1 still applies. Events accelerate refresh; a lost event is bounded by React Query's natural refetch cadence. DanglingCache's emission is therefore an optimization over pure pull, not a replacement for the pull path.
DanglingCache emits a structured info-level log record at a fixed cadence (every 10 minutes, driven by a simple timer in onInit) summarizing its recent activity. This mirrors the orphan sweep observability contract (§10.5) — one periodic record plus opportunistic warn / error on anomalies, no separate metrics pipeline.
{
event: 'dangling-cache-snapshot',
cachedEntries: number, // total entries currently in byEntryId
pathIndexSize: number, // total keys in pathToEntryIds
// counters since the last snapshot record (reset each emit)
checkCalls: number, // total check() invocations
checkCacheHits: number, // returned cached within TTL
checkTtlExpiredReStats: number, // re-stat triggered by TTL
checkColdStats: number, // re-stat triggered by cache miss
watcherEvents: number, // onFsEvent calls from watcher factory
opsObservations: number, // onFsEvent calls from FileManager ops
transitionsFired: number, // onDanglingStateChanged fires
statErrors: number, // fs.stat threw (permission, I/O error, etc.)
windowMs: number, // interval this snapshot covers (≈ 600_000)
}
Emission cadence: every 10 minutes while the service is active. Upon onStop, one final snapshot flushes any outstanding counters. Snapshot volume at steady state is 6 records/hour × 24 = 144 records/day through loggerService — trivial for log aggregation even on long-running installs.
Anomaly triggers (emitted out-of-band at warn level, independent of the snapshot cadence):
statErrors / checkCalls > 0.1 sustained across two consecutive snapshot windows → likely a systemic FS issue (unmounted drive, permission regression)cachedEntries > 50_000 → memory-budget anomaly; suggests either a runaway caller or a bug in removeEntry cleanuptransitionsFired > 1000 within one 10-minute window → likely a watcher feedback loop or mass unmount eventThese thresholds are heuristic starting points — tune based on real-world telemetry once available. IPC latency is not instrumented here; loggerService is a log pipeline, not a metrics system, and Cherry has no telemetry backend. Per-IPC latency concerns should be diagnosed ad-hoc via performance.now() in the affected handler during investigation, not baked into a permanent counter.
| Decision | Conclusion | Core rationale |
|---|---|---|
| Tree vs flat | Flat | FileEntry manages "user-submitted independent files"; directory organization is not a file_module responsibility |
| Mount abstraction | Removed | All internal files live flat under {userData}/Data/Files/ (via the feature.files.data path key); external is reached directly via externalPath; no mount needed |
| Origin two-state | internal/external | Express "Cherry-owned" and "user-owned, Cherry-referenced" respectively; clear semantics |
| External read/write permissions | Explicit user ops may change; Cherry doesn't auto-change | VS Code-style behavior model—change when told to; don't modify behind the scenes |
| External operation symmetry | write/rename/permanentDelete all delegate to the FS primitives and take effect; trash/restore touch DB only | Soft delete preserves reversibility (doesn't touch FS); hard delete is the terminal action (really deletes FS) |
| External identity | externalPath unique(where not trashed) | At most one active entry at a time for the same path; ensureExternalEntry upserts by path |
| Cherry tracks external rename | Not tracked | Best-effort semantics; external rename → dangling → user re-@ |
| Snapshot vs realtime stat | External row stores only identity + stable projections (name / ext from externalPath); live size / mtime via getMetadata on demand | Eliminates stale-snapshot bug class at the type level; cost of the extra fs.stat is explicit at the call site instead of hidden behind a DB field |
| Dangling state carrier | In-memory singleton DanglingCache | Not in DB (avoids bidirectional DB-FS sync); three states present/missing/unknown; TTL-based lazy expiration (§11.6, 30 min); refreshed on query / FS observation / watcher; no periodic background sweep — IO cost scales with query frequency, not entry count |
| Dangling exposure method | File IPC getDanglingState / batchGetDanglingStates (never DataApi) | DataApi is pure SQL; FS probe lives in IPC where side effects are expected; zero cost by default; parallel stat on demand |
| Watcher → DanglingCache wiring | Factory auto-wires | Business modules unaware of DanglingCache; a single watcher instance serves business events + dangling tracking |
| Content hash algorithm | XXH3-64 via native @node-rs/xxhash, stored as {algo}:{hex} | One implementation serves persisted dedup detection and on-demand OCC comparison. The native package avoids the previous WASM throughput ceiling and provides incremental XXH3-64 hashing; its XXH3-128 API is one-shot only, which would require buffering streamed files. The 64-bit value is a candidate signal, never an identity: collisions only produce a candidate that the consumer must verify, and the algorithm tag preserves an incremental upgrade path. |
| Does write carry version | Split into write / writeIfUnchanged | Force the caller to explicitly choose; avoid silent degradation to blind write when version is forgotten |
| Atomic write fsync | On by default | Correctness guarantee takes precedence over performance; Cherry is not a high-throughput scenario |
| Trash model | deletedAt timestamp | parentId unchanged; naturally supports expiry; no system_trash entries |
| pending_fs_ops | Removed | After extreme simplification, orphan sweep suffices to cover crashes |
| Startup dangling probe | Removed | Changed to lazy + Promise.all; stat only when an IPC caller explicitly requests dangling state |
| Is Watcher a lifecycle service | No | DirectoryWatcher is a primitive; business modules new it via the factory; file_module doesn't actively watch |
| Directory import / bidirectional sync | Moved out of file_module | Business modules (Knowledge, etc.) implement this with DirectoryWatcher + their own mapping tables |
| AI SDK upload cache | Standalone file_upload table (deferred) | Decoupled from mount / remote; naturally aligns with SharedV4ProviderReference |
| Notes | File tree is an independent domain, not mirrored to FileEntry | If other modules need to reference Notes files, they use the origin of their choice via the corresponding path |
| CacheService integration for DanglingCache / versionCache | Not integrated; both stay bespoke | CacheService (src/main/data/CacheService.ts) is a general TTL KV + cross-window sync primitive. DanglingCache needs a path → Set<entryId> reverse index (§11.4), transition-aware event emission (§11.7 — fire only on genuine state change by comparing old vs new), and observedAt-based "TTL expired → re-stat then update" semantics (§11.6 — CacheService's TTL is "expired → deleted", which would destroy the prev-state comparison needed for transition detection). versionCache needs size-bounded LRU (§4.4), not TTL — a fundamentally different eviction policy; and lives as a per-FileManager-instance field for test isolation, not as a BeforeReady singleton. Wrapping either in CacheService would flatten the value schema, bolt on the secondary structures separately, and bypass the TTL layer — no logic shed, only domain expression lost. CacheService remains the right tool for future scenarios that genuinely match "simple per-id TTL cache" or "cross-window cache" shape (e.g. a short-lived batchGetMetadata result cache, a future FileUploadService provider-upload cache). |
origin Variant — Developer ChecklistThe origin field ('internal' | 'external' today) is the single most cross-cutting axis in the file module: its value implicitly drives storage layout, DB constraints, mutability policy, dangling semantics, UX language, and cleanup scope. Adding a new variant (e.g. 'archived', 'shared', 'synced') requires coordinated changes across the layers below. Missing any row silently breaks an invariant — TypeScript catches discriminated-union exhaustiveness but not CHECK constraints, not ad-hoc policy branches, and not documentation tables.
This checklist is the canonical addition procedure. A PR introducing a new origin MUST tick every row it touches (or explicitly justify a skip).
| Location | Change required |
|---|---|
src/shared/data/types/file/fileEntry.ts → FileEntryOriginSchema | Add the new enum value |
Same file → new XxxEntrySchema | Define the row shape for the new variant (which columns are nullable / required / branded) |
Same file → FileEntrySchema discriminated union | Add the new schema as a union member |
Same file → any type guard helpers (isInternalEntry, etc.) | Add isXxxEntry helper if code needs to narrow |
| Location | Change required |
|---|---|
src/main/data/db/schemas/file.ts | Review every CHECK constraint naming origin — fe_origin_consistency, fe_size_internal_only, fe_external_no_delete, fe_external_path_unique, etc. — and decide whether the new variant honors / violates / is exempt from each |
| Drizzle migration | Ship the constraint updates in the same migration as the enum expansion. Partial unique indexes on externalPath may need a new branch |
| Existing rows | No migration should run for existing rows unless the new variant has a natural subset mapping (unlikely) |
| Location | Change required |
|---|---|
src/main/services/file/utils/pathResolver.ts → resolvePhysicalPath | Add the new entry.origin branch; decide storage layout |
src/shared/utils/file/canonicalize.ts → canonicalizeFilePath() | If the new variant is path-based and distinct from 'external', decide whether it shares the canonical form or needs its own normalization + brand |
Every ad-hoc if (entry.origin === 'internal') / === 'external' in the codebase is a policy decision that must be re-evaluated. Grep for both and review:
| Policy | Location |
|---|---|
| Trash-ability (who can soft-delete) | trash / restore in FileManager; DB CHECK fe_external_no_delete |
| Size snapshot storage | write / writeIfUnchanged internal-DB-update branch; toFileInfo projection |
| Name / ext as SoT vs projection | rename mutation; toFileInfo projection; FileEntrySchema field docs |
| DanglingCache participation | DanglingCache.check returns 'present' for internal; consider where the new variant falls on the present/missing/unknown axis |
permanentDelete semantics | Does it touch physical files? Just DB? Refer to §6 and architecture.md §3.4 |
| Orphan sweep scope | §10 scans origin='internal' UUID files; does the new variant have a sweepable disk presence? |
| Explicit cleanup semantics | §7.2 forbids dangling-state-driven automatic deletion of manual entries (auto-policy entries are reclaimed by the ref-count-driven cleanup pass — file-entry-cleanup.md); decide whether the new origin is preserved, reported, or removable only through an explicit user/caller action |
| IPC dispatch applicability | architecture.md §3.3 tables per method — does each method make sense for the new variant? |
| Location | Change required |
|---|---|
| architecture.md §3.4 UX labeling convention table | Add a row for the new origin's permanentDelete user-facing label + confirmation copy |
| Product-side component copy | Concrete button labels / menu items that branch on origin |
| Location | Change required |
|---|---|
| architecture.md §3.6 event payloads | onEntryRowChanged.origin field value domain expands — TS catches via discriminated-union narrowing in the renderer binding |
| Observability logs | dangling-cache-snapshot and orphan-sweep records may need per-origin breakdowns if the new variant is material to diagnostics |
| Location | Change required |
|---|---|
architecture.md §1.0.1 Semantics of Origin | Add a paragraph describing the new variant |
file-manager-architecture.md §1.2 Origin table | Add the row — physical location, ownership, mutability |
file-manager-architecture.md §1.2 Invariants table | Add the column — name / ext / size / externalPath behavior per-origin |
file-manager-architecture.md §12 Key Design Decisions | If the addition surfaces a notable new trade-off, record it |
PR description MUST list each ticked row with a one-line justification, and each explicitly-skipped row with a reason. A reviewer MUST spot-check §13.2 (DB CHECK constraints) and §13.4 (ad-hoc policy grep) before approval — these are the two layers where silent bugs most often hide.