packages/store/SPEC.md
This document states the rules that @tldraw/store implements. It is written to drive testing: each rule has a stable ID (e.g. S4, MG2), each rule is independently observable through the public API (or the documented internal API where noted), and the unit tests should be an expression of these rules. When a test and this document disagree, one of them is wrong — figure out which and fix it.
Sections marked internal describe supporting machinery (ImmutableMap, IncrementalSetConstructor, set utilities) that has its own contract worth testing directly, even though users rarely touch it.
id and a typeName.document (persisted and synced), session (per store instance, not synced), or presence (per instance, synced but not persisted).@tldraw/state unless explicitly "without capture".@tldraw/state transaction and produce one round of side-effect callbacks.added, updated ([from, to] pairs), and removed, each keyed by id.'user' (local changes) or 'remote' (changes applied inside mergeRemoteChanges).before/after × create/change/delete, plus operationComplete) registered per type name.isRecord(v) is true exactly for non-null objects with both an id and a typeName property; false for null, undefined, primitives, and objects missing either property.typeName:uniquePart.createRecordType(typeName, config) makes a RecordType with the given scope and an optional validator; without a validator, records pass through validation unchanged. RecordType's own constructor defaults scope to 'document'.create(props) returns { ...defaults, ...props, id, typeName }: default properties from createDefaultProperties(), overridden by the given props. Properties whose value is undefined do not override defaults.create uses the given id if the props contain a defined one, else generates typeName:<unique> via createId(); an explicitly undefined id is treated as absent (matching RT2). createId(customUniquePart) uses the custom part when given.clone(record) deep-clones the record and assigns a fresh id.parseId(id) returns the part after the colon and throws if the id does not belong to this type. isId(id) is true exactly for strings starting with typeName:; false for undefined and other types' ids. isInstance(record) checks record?.typeName === typeName.assertIdType(id, type) throws for undefined, empty, or wrong-type ids, and passes for valid ones.withDefaultProperties(fn) returns a new RecordType with the same type name, validator, scope, and ephemeral keys, whose create no longer requires the defaulted properties.validate(record, recordBefore?) calls validator.validateUsingKnownGoodVersion(recordBefore, record) when both are available, else validator.validate(record).ephemeralKeys marks properties; ephemeralKeySet contains exactly the keys mapped to true.createEmptyRecordsDiff() returns { added: {}, updated: {}, removed: {} }; isRecordsDiffEmpty is true exactly when all three collections are empty.reverseRecordsDiff swaps added and removed and reverses each [from, to] pair.squashRecordDiffs(diffs) combines sequential diffs into one. Per record id: add then update → add with the final value; add then remove → nothing; update then update → one update from the original from to the final to; update then remove → remove of the original from; remove then add → update from the removed value to the added value, unless the added value is reference-identical to the removed one, in which case nothing.squashRecordDiffs does not mutate its inputs unless mutateFirstDiff: true, in which case the first diff is the (mutated) result. squashRecordDiffsMutable(target, diffs) applies diffs onto target in place with the same semantics.initialData is given; initial data is validated with phase 'initialize'. Records read from the store are deep-frozen in dev/test builds (DF1).put([record]) creates records whose ids are not present (validated with phase 'createRecord') and updates those that are (phase 'updateRecord', with the previous record as the known-good version). get returns the stored record; has reports presence.put skips a record entirely — no store change, no history entry — when the validated result is reference-equal to the currently stored record. (A validateUsingKnownGoodVersion implementation that returns the known-good object for equal values therefore makes equal re-puts complete no-ops; likewise a beforeChange handler returning prev blocks an update, provided the validator is reference-preserving.) If every record in a put is skipped, no history entry is produced.update(id, updater) is put([updater(current)]). For a missing id it logs an error and changes nothing (it does not throw).remove(ids) deletes the records with those ids; missing ids are ignored. If nothing is actually deleted, no history entry is produced. clear() removes all records.get/has are reactive (capture as parents of the running computed/effect); unsafeGetWithoutCapture reads without capturing. allRecords() returns all records as an array.serialize(scope?) returns the records whose type's scope matches; the default scope is 'document'; 'all' includes everything. store.scopedTypes maps each scope to the set of type names in it.getStoreSnapshot(scope?) is { store: serialize(scope), schema: schema.serialize() }.loadStoreSnapshot(snapshot) migrates the snapshot, replaces all current records with the result, and runs the integrity checker — all with side effects disabled (restoring the previous enabled state afterwards). It throws if migration fails, leaving the store unchanged.migrateSnapshot(snapshot) returns the migrated snapshot stamped with the current serialized schema, and throws if migration fails.store.atomic() (directly or implicitly via put/remove/etc.), which wraps a @tldraw/state transaction: reactive effects observe all changes of the operation at once, when the outermost atomic completes.atomic calls join the outer operation: their after-events fold into the outer flush, and mergeRemoteChanges inside an atomic op throws.atomic(fn, runCallbacks = true): passing false to a top-level call disables side effects for the whole operation — before* handlers are bypassed while fn runs, and the flush skips the after* and operationComplete handlers. A nested atomic(fn, false) inside an enabled operation suppresses only the before* handlers for its duration: its after-events flush with the outer operation, where side effects are enabled again. A nested atomic can switch callbacks off but cannot switch them back on if an enclosing operation turned them off.before and the final after. A record created and then deleted in the same operation produces no callback at all; created then updated produces a single afterCreate with the final value.afterChange fires only when the before and after records differ by deep equality. (Putting a structurally-equal copy still records a history entry per S3/H1 — only the callback is suppressed.)Maximum store update depth exceeded.operationComplete handlers fire after the after-events of an operation settle; if an operationComplete handler makes further changes, the flush (including operationComplete) runs again until quiescent.mergeRemoteChanges(fn) runs fn atomically with source 'remote'; nested calls inside another mergeRemoteChanges just run fn. After the merge the integrity checker runs. Changes that side-effect handlers make in response to remote changes are attributed to 'user'.register*Handler call returns a remover. register({ type: { beforeCreate, ... } }) registers many at once and returns one cleanup that removes them all.beforeCreate runs before validation on create; its return value is what gets validated and stored. Multiple handlers chain, each receiving the previous one's output.beforeChange runs before validation on update, receiving (prev, next, source); its return value is stored. Returning prev blocks the update (with a reference-preserving validator this makes the put a complete no-op per S3).beforeDelete may return false to prevent that record's deletion; other records in the same remove call are still deleted.afterCreate/afterChange/afterDelete run per AO4–AO6 and observe the final state of the operation; all handlers receive the source ('user' or 'remote').setIsEnabled(false), a top-level atomic(fn, false), applyDiff(diff, { runCallbacks: false }), snapshot loads), before* handlers pass values through unchanged, beforeDelete cannot block, and after-events flushed while disabled are dropped — after* and operationComplete handlers do not run. (A nested atomic(fn, false) is weaker: per AO3 its after-events fire with the outer, enabled flush.) An operation may switch side effects off, but never on while they are disabled: setIsEnabled(false) keeps handlers off across subsequent operations until setIsEnabled(true).store.history is an atom that increments by exactly one for each committed change-set, carrying the RecordsDiff as its history diff (historyLength 1000).listen(fn, filters?) registers a listener and returns a remover. Listener notification is asynchronous: accumulated entries are flushed on the next frame, not synchronously with the change.[user, user, remote, user] receives three entries (user, remote, user), each with the squashed (D3) diff and its source.filters.source ('user' | 'remote' | 'all') drops entries from other sources. filters.scope ('document' | 'session' | 'presence' | 'all') filters each entry's diff down to records of that scope; if nothing remains, the listener is not called for that entry.listen flushes pending history first, so a new listener never sees changes made before it subscribed.extractingChanges(fn) returns the squashed diff of exactly the changes made during fn (listeners still see those changes normally).addHistoryInterceptor(fn) calls fn(entry, source) synchronously for every change-set as it happens and returns a remover.applyDiff(diff) puts the added and updated records and removes the removed ids. runCallbacks: false disables side effects for the application (AO3). Applying a diff and then its reverseRecordsDiff (D2) restores the prior state.applyDiff with ignoreEphemeralKeys: true ignores changes to keys in the type's ephemeralKeySet when applying updates to existing records: non-ephemeral changed keys are merged onto the stored record, and an update touching only ephemeral keys is dropped. Updates for records that don't exist are applied in full, as are records in added.'initialize', 'createRecord', 'updateRecord'). A record whose typeName has no RecordType in the schema fails validation with Missing definition for record type <name>.validateUsingKnownGoodVersion when the validator implements it (RT8).put.onValidationFailure, the handler receives { error, store, record, phase, recordBefore } and its return value is stored instead of throwing — on creation and update alike. More generally, the validator's return value is what gets stored, so validators may substitute a transformed record.store.validate(phase) re-validates every record currently in the store.store.createComputedCache(name, derive) returns a cache whose get(id) returns derive(record) for an existing record and undefined for a missing one.derive runs at most once per record change, and re-runs (with a fresh result) when the record changes.areRecordsEqual controls which record changes invalidate the cache: when the old and new record are "equal", derive does not re-run. areResultsEqual controls change propagation: an "equal" result keeps the previous value object.createComputedCache(name, derive) works with any StoreObject (a store or { store }), keeping a separate cache per context object, and passes the context to derive.store.createCache(create) is the low-level form: create(id, recordSignal) returns the signal to cache; get(id) on a missing record returns undefined without calling create.store.query.filterHistory(typeName) returns a computed epoch whose history diffs contain only records of that type; it is cached per type name.from).store.query.index(typeName, property) returns a computed Map from each distinct property value to the Set of ids of records with that value. Records whose value is undefined are not indexed.(typeName, property) pair.RSIndexDiff (map of value → CollectionDiff of ids) in the computed's history.property containing backslashes indexes a nested path: 'metadata\\sessionId' indexes record.metadata.sessionId. Missing intermediate objects yield undefined (not indexed).store.query.ids(typeName, queryCreator?) returns a computed Set of the ids of matching records; with no query it contains all ids of that type. Its history diffs are CollectionDiffs.queryCreator may read signals, and when the expression it returns changes (by deep equality), the query rebuilds and emits a correct diff. An expression that is deep-equal to the previous one causes no rebuild.Set object in place.records(typeName, queryCreator?) returns the matching records as an array, with shallow-array equality (same members → no change). record(typeName, queryCreator?) returns one matching record or undefined.exec(typeName, query) runs one non-reactive query and returns matching records; with no matches it returns the shared empty array.{ eq: v } matches strict equality; { neq: v } matches records whose value is defined and differs from v — records missing the property do not match, mirroring the indexes, which only track defined values; { gt: n } matches only numbers strictly greater than n (non-numeric values never match gt).{} matches every record of the type: objectMatchesQuery({}, r) is true and ids(type) contains all ids. (The raw executeQuery helper instead returns an empty set for an empty expression; StoreQueries.ids special-cases it before calling executeQuery.)executeQuery (index-based) and objectMatchesQuery (predicate) agree: the set of ids returned equals the set of records matching the predicate, including for nested paths and across record types.StoreSchema.create(types, options?) builds a schema. It throws at creation time for duplicate migration sequenceIds, invalid migration sequences (M3), and migrations whose dependsOn references a migration that does not exist.serialize() returns { schemaVersion: 2, sequences } mapping each sequence id to the version of its last migration (0 for an empty sequence).serializeEarliestVersion() maps every sequence to version 0.getType(typeName) returns the RecordType and throws for unknown type names.upgradeSchema converts a v1 serialized schema to v2: storeVersion becomes com.tldraw.store, each record version becomes com.tldraw.<typeName>, and each subtype version becomes com.tldraw.<typeName>.<subType>. v2 schemas pass through unchanged; schema versions other than 1 or 2 produce an error result.createMigrationSequence({ sequenceId, sequence, retroactive? }) validates and returns the sequence; retroactive defaults to true.{ dependsOn } entry in a sequence is squashed into the next migration in the sequence, prepending to that migration's own dependsOn. A standalone entry with no following migration is dropped.validateMigrations throws when: the sequence id is empty or contains /; a migration id does not have the form <sequenceId>/<integer>; the first migration's version is not 1; or versions do not increase in increments of exactly 1.createMigrationIds(sequenceId, { name: version }) returns { name: 'sequenceId/version' } for each entry. parseMigrationId('seq/3') returns { sequenceId: 'seq', version: 3 }.createRecordMigrationSequence produces record-scope migrations filtered to the given recordType; a per-migration filter and a sequence-level filter compose (all must pass).sortMigrations orders migrations within a sequence by version (foo/1 before foo/2), regardless of input order.dependsOn sorts after all of its dependencies, across sequences.getMigrationsSince(persistedSchema) decides which migrations a snapshot still needs.
retroactive, and skipped entirely if not.Incompatible schema?).migratePersistedRecord(record, persistedSchema, 'up') applies the needed record-scope migrations in order and returns { type: 'success', value }. The input record is not mutated.filter skips non-matching records (the migration applies to others).target-version-too-new going up, target-version-too-old going down.'down' requires every needed migration to have a down migrator (else target-version-too-old) and applies them in reverse order.migration-error result rather than propagating.migrateStoreSnapshot(snapshot) migrates every record and returns the new store object; the input snapshot is not modified unless mutateInputStore: true, in which case snapshot.store is updated in place (including deletions) and returned. (In dev builds the input snapshot's records are deep-frozen as a side effect of migration, even without mutateInputStore — their contents are unchanged, but they become immutable.)SynchronousStorage (get/set/delete/keys/values/entries) and may use it to read and write records directly.'document' are removed from the result (legacy cleanup). A snapshot needing no migrations keeps such records.migration-error result.migrateStorage(storage) applies the same process to external storage, writing the current serialized schema via setSchema and updating only records that actually changed (deep equality).ensureStoreIsUsable() creates the schema's integrity checker on first use and runs it; it runs automatically after mergeRemoteChanges and during loadStoreSnapshot.markAsPossiblyCorrupted() sets a flag readable via isPossiblyCorrupted(); new stores start unflagged.AtomMap is the store's reactive record container: a drop-in Map whose reads are reactive.
get/has are reactive per key: an effect reading a present key re-runs when that key's value changes or the key is deleted, but not when other keys are set, updated, or deleted.set adds or updates and returns the map. update(key, fn) replaces an existing value and throws for a missing key.delete returns whether the key existed. deleteMany(keys) deletes in one transaction (one reaction for the whole batch), returns the [key, value] pairs actually deleted, and ignores missing keys.clear() empties the map.entries, keys, values, forEach, [Symbol.iterator], and size see exactly the live entries and are reactive. forEach honors thisArg.@tldraw/state transaction are restored: additions disappear, updates revert, deletions reappear.Object.prototype.toString.call(map) is [object AtomMap].AtomSet behaves as a reactive set: add (returns the set), has, delete (returns whether present), clear, size, forEach, keys/values/iteration yield the elements, and entries yields [value, value] pairs.has(x) re-runs when x is added or removed.devFreeze(object) recursively freezes the object and its nested objects and returns it; mutation afterwards throws in strict mode.cannot include non-js data in a record) for objects with non-plain prototypes (class instances, Maps, etc.); arrays, plain objects, null-prototype objects, and structured-clone objects are allowed.intersectSets(sets) returns the elements present in every set; [] yields an empty set; a single set yields a copy.diffSets(prev, next) returns { added?, removed? } with only the populated keys, or undefined when the sets have the same members. Membership is by reference for objects.get() returns undefined when there are no net changes: nothing done, adding already-present items, removing absent items, or add/remove round trips that cancel out.get() returns { value, diff }, where value is the new set and diff is the CollectionDiff relative to the original set; the original set is not mutated.diff.removed); removing an item added earlier cancels the add.ImmutableMap is the persistent (HAMT) map under AtomMap.
set and delete return a new map and leave the original unchanged.get(k) returns the value or undefined; get(k, notSetValue) returns notSetValue for missing keys.withMutations(fn) batches many changes into one new map; if fn changes nothing, the same instance is returned.deleteAll(keys) removes all the given keys.entries/keys/values/iteration yield every entry exactly once, consistent with size.