Back to Super Productivity

Delta Sync vs Operation Log: Synthesized Comparison

docs/sync-and-op-log/background-info/synthesized-delta-vs-oplog.md

18.4.413.7 KB
Original Source

Delta Sync vs Operation Log: Synthesized Comparison

Synthesized From: Gemini 2.5 Flash, GPT-5, Claude Opus 4.5 Date: December 2, 2025 Branches Compared: feat/delta-sync vs feat/operation-logs


Implementation Status

Both approaches have substantial implementations.

Current Implementation Status

BranchKey FilesLinesStatus
feat/delta-syncsuper-sync.ts, diff-utils.ts, super-sync-api.ts~1,600Full implementation: shadow state (IDB), diffing, watermarks, granular encryption
feat/operation-logsoperation-log-store.service.ts, operation-log-sync.service.ts, etc.~600+Core logic implemented: op store, sync service, effects, hydrator

Analysis Sources

ModelDelta Sync SourceNotes
GeminiEarlier state of feat/delta-syncAnalyzed before branch was updated with full implementation
GPT-5Design docs + codeAnalyzed architectural design
Opusfeat/delta-sync (940-line implementation)Full code analysis of implemented delta sync

Update: Gemini's finding that delta sync was "vaporware" is now outdated. The feat/delta-sync branch has been updated and now contains the full 940-line SuperSyncProvider implementation with IDB shadow state, diff engine, and watermark tracking. Both approaches can now be compared on equal footing as implemented code.


1. Executive Summary

This document compares two synchronization architectures for Super Productivity, both of which are now substantially implemented:

  • Delta Sync: State-based synchronization with shadow state, diffing, and watermarks
  • Operation Log: Event-sourced synchronization with append-only operation log

Approaches Compared

ApproachBranchCore PrincipleImplementation
Delta Syncfeat/delta-syncCompare state snapshots, transmit differencesImplemented (~1,600 lines)
Operation Logfeat/operation-logsRecord actions, transmit and replay operationsPartially implemented (~600+ lines)

Verdict (Unanimous)

All three models recommend Operation Log based on architectural analysis:

ModelRationale
GeminiDelta sync has fundamental shadow state consistency issues
GPT-5Delta sync design has inherent LWW data-loss risk
OpusCode analysis reveals non-atomic state updates and vector clock governance gaps

2. Architecture Comparison

2.1 Delta Sync (As Implemented)

Client Storage (super-sync.ts):
├── App data (IndexedDB)
├── Shadow state (super-sync-shadow IDB)
│   ├── shadow_state: last synced state per model
│   └── watermarks: revision cursors per model
└── Memory cache (lastSyncedState Map)

Sync Flow:
1. Load shadow state from IDB (or memory cache)
2. Compute diff via createDiff() in diff-utils.ts
3. Upload changes to /api/sync/changes
4. Server shallow-merges (LWW)
5. Update shadow + watermark in IDB

Code-Verified Concerns (from Opus analysis of super-sync.ts):

  • Shadow state exists in both memory and IDB (can drift)
  • Watermark and shadow saved in separate IDB transactions (not atomic)
  • Diff computation is O(N) using JSON.stringify comparison
  • LWW merge loses concurrent independent changes
  • Empty vector clocks treated as CONCURRENT (false conflicts)

2.2 Operation Log (As Implemented)

Client Storage (actual):
├── SUP_OPS IndexedDB
│   ├── ops: append-only operation log
│   └── state_cache: periodic snapshots
└── NgRx Store (derived from ops)

Sync Flow:
1. Local action → append to ops log
2. Upload pending ops to remote
3. Download remote ops
4. Detect conflicts via vector clock comparison
5. Apply non-conflicting ops; surface conflicts to user

Code-Verified Characteristics:

  • Single source of truth (operation log)
  • Conflict detection is per-entity
  • Compaction service bounds log growth
  • Replay hydration from snapshot + tail ops

3. Criteria Comparison

3.1 Conflict Handling

This is the most significant differentiator and is architecture-dependent, not implementation-dependent.

AspectDelta Sync (Design)Operation Log (Implemented)
DetectionWhole-file vector clockPer-entity vector clock
MergeShallow LWW ({...old, ...new})Semantic (independent ops both apply)
User choiceAll-or-nothingPer-entity granular
Data loss riskHigh (silent overwrites)Low (conflicts surfaced)

Example:

Device A: Renames task to "Meeting"
Device B: Marks task complete

Delta Sync (LWW): One change lost
Operation Log: Both changes applied (independent operations)

Winner: Operation Log (unanimous, strong confidence)

3.2 Maintainability

AspectDelta Sync (Design)Operation Log (Implemented)
Data models2 (app state + shadow)1 (operation log)
Adding featuresUpdate shadow handling + diff rulesAdd action to whitelist
DebuggingCompare shadow vs actual vs serverInspect operation sequence
Source of truthAmbiguous (shadow can drift)Clear (operation log)

Winner: Operation Log (unanimous)

3.3 Performance

MetricDelta SyncOperation LogBasis
Diff costO(N) per syncO(1) appendCode-verified (diff-utils.ts iterates all entities)
StartupFast (load state)Snapshot + replay tailCode-verified
BandwidthSmaller patchesLarger payloadsCode-verified (delta sends only changed fields)
Large datasetsUI freeze risk at 10k+ entitiesScales with change rateNeeds measurement

Winner: Tie — Delta sync wins on bandwidth; operation log wins on CPU. Needs benchmarking with real datasets.

3.4 Disk Usage

MetricDelta Sync (Implemented)Operation Log (Implemented)
Steady-state~2X (shadow state in IDB)~1.2X (with compaction)
BasisCode-verified (super-sync-shadow IDB)Code-verified (compaction service exists)

Winner: Operation Log (if compaction works correctly)


4. Implementation Effort

Delta Sync: Stabilize Existing Implementation

feat/delta-sync now has full implementation (~1,600 lines). Remaining work to stabilize:

ComponentEffortNotes
Atomic shadow/watermark updates1-2 weeksCurrently separate IDB transactions
Web worker for diff1 weekPrevent UI freeze on large datasets
Server delta API hardening1-2 weeksPagination, error handling
Conflict UI improvements1 weekAuto-merge feedback
Testing/hardening2-4 weeksEdge cases, concurrent clients, crash recovery

Total: 6-10 weeks (high risk due to multi-state consistency issues identified in code analysis)

Operation Log: Complete Implementation

feat/operation-logs has core logic (~600+ lines); remaining work:

ComponentEffortNotes
Conflict resolution UI1-2 weeksCurrently stub
Dependency resolver retry0.5-1 weekHandle missing parents
Smart resolution suggestions1 weekLWW fallback for simple conflicts
Genesis migration1 weekConvert existing data to first op
Effect guards0.5-1 weekPrevent side effects during replay
Testing/hardening1-2 weeksCompaction, large logs, concurrent tabs

Total: 4-6 weeks (medium risk, primarily compaction correctness)


5. How to Validate Claims

Before committing to either approach, measure:

For Delta Sync (if implemented)

  1. Diff cost: Time createDiff() with 1k, 10k, 50k entities
  2. Shadow storage: Measure IDB size with shadow vs without
  3. Watermark consistency: Test crash recovery mid-sync
  4. LWW data loss: Simulate concurrent edits, count lost changes

For Operation Log

  1. Log growth: Measure ops/day for typical usage; verify compaction bounds it
  2. Replay time: Time startup with 1k, 10k, 100k operations
  3. Compaction correctness: Verify no data loss after compaction
  4. Conflict detection accuracy: Test concurrent edits, verify correct conflicts surfaced

6. Risk Assessment

Delta Sync Risks (Code-Verified)

RiskLikelihoodImpactNotes
Shadow/watermark desyncHighHighNon-atomic IDB transactions in super-sync.ts
LWW data lossHighHighFundamental to architecture
O(N) UI freezeMediumMediumCan be mitigated with web workers
Vector clock governanceMediumMediumEmpty clocks = false conflicts

Operation Log Risks (Observed)

RiskLikelihoodImpactNotes
Compaction data lossLowHighMitigate with conservative retention
Replay non-determinismMediumMediumRequires reducer purity
Log growth unboundedMediumLowCompaction service exists

Assessment: Operation log has fewer high-likelihood, high-impact risks.


7. Recommendation

Consensus

Complete the Operation Log implementation (feat/operation-logs).

Rationale

  1. Both implementations exist, but operation log has fewer code-verified architectural risks
  2. Conflict handling is architecturally superior in operation log (per-entity vs whole-file)
  3. Risk profile is better for operation log (no multi-state consistency issues)
  4. Effort is comparable: Delta sync 6-10 weeks vs Operation log 4-6 weeks

If Delta Sync Is Chosen

  1. Address atomic shadow/watermark updates first (critical bug)
  2. Move diff calculation to web worker (performance)
  3. Fix vector clock governance (empty clocks = false conflicts)
  4. Accept and document LWW limitations
  5. Plan for extensive testing of crash recovery scenarios

8. Conclusion

AspectDelta SyncOperation Log
Implementation statusImplemented (~1,600 lines)Partially implemented (~600+ lines)
Conflict handlingLWW (lossy)Per-entity (preserves intent)
Effort to production6-10 weeks4-6 weeks
Risk levelHigh (multi-state consistency)Medium (compaction correctness)
RecommendationViable but riskyRecommended

Both approaches are now substantially implemented. The recommendation for operation log is based on code-verified architectural concerns in the delta sync implementation:

  • Non-atomic shadow/watermark updates create crash consistency risks
  • LWW merge semantics cause data loss for concurrent independent changes
  • Empty vector clocks trigger false conflicts

Bottom line: Operation log provides a more robust architecture for Super Productivity's multi-device use case. If delta sync is chosen, significant stabilization work is needed to address the identified issues.