Back to Super Productivity

File-Based Sync Flow — Mermaid Chart

docs/sync-and-op-log/file-based-sync-flowchart.md

18.4.49.6 KB
Original Source

File-Based Sync Flow — Mermaid Chart

Visual overview of the sync decision tree for file-based providers (Dropbox, WebDAV, LocalFile). For the SuperSync equivalent see supersync-scenarios-flowchart.md.

Both share the same op-log infrastructure (OperationLogSyncService, RemoteOpsProcessingService, conflict detection) but differ in the transport/adapter layer.

mermaid
flowchart TD
    START([Sync Triggered]) --> DL[Download sync-data.json
gap detection handled internally]

    %% ── SERVER MIGRATION (file-based specific) ──────────────────
    DL --> MIG_CHK{Server migration?
gap + empty server}
    MIG_CHK -->|Yes| MIGRATION[Server migration:
handleServerMigration
creates SYNC_IMPORT]
    MIG_CHK -->|No| DECRYPT

    %% ── DECRYPTION (shared with SuperSync) ────────────────────────
    DECRYPT{Encrypted?}
    DECRYPT -->|Yes| DECRYPT_OK{Decryption succeeds?}
    DECRYPT -->|No| SNAPSHOT_CHK
    DECRYPT_OK -->|Yes| SNAPSHOT_CHK
    DECRYPT_OK -->|No password configured| NO_PWD_DLG[Enter Password dialog:
Save & Sync / Force Overwrite / Cancel]
    DECRYPT_OK -->|Wrong password| WRONG_PWD_DLG[Decrypt Error dialog:
Save & Sync / Use Local / Cancel]
    NO_PWD_DLG -->|Save & Sync| START
    NO_PWD_DLG -->|Force Overwrite| FORCE_UP[Force upload local state
SYNC_IMPORT]
    WRONG_PWD_DLG -->|Save & Sync| START
    WRONG_PWD_DLG -->|Use Local| FORCE_UP

    %% ── SNAPSHOT vs INCREMENTAL BRANCH (file-based specific) ────
    SNAPSHOT_CHK{snapshotState
received?
seq 0 download only}
    SNAPSHOT_CHK -->|Yes| SS_OPS
    SNAPSHOT_CHK -->|No| HAS_OPS

    %% ── SNAPSHOT PATH (file-based specific) ─────────────────────
    SS_OPS{Has meaningful
unsynced ops?}
    SS_OPS -->|Yes| SS_CONFLICT[SyncConflictDialog:
USE_LOCAL / USE_REMOTE / CANCEL]
    SS_OPS -->|No| SS_FRESH{Fresh client?}
    SS_FRESH -->|Yes + meaningful
store data| SS_CONFLICT
    SS_FRESH -->|Yes + no local data| SS_CONFIRM[Confirm dialog:
Download remote?]
    SS_FRESH -->|Not fresh| HYDRATE
    SS_CONFLICT -->|Use Local| FORCE_UP
    SS_CONFLICT -->|Use Remote| FORCE_DL[Force download
from seq 0]
    SS_CONFLICT -->|Cancel| CANCELLED([Sync Cancelled])
    SS_CONFIRM -->|OK| HYDRATE[Hydrate from
snapshotState]
    SS_CONFIRM -->|Cancel| CANCELLED
    HYDRATE --> UPLOAD

    %% ── INCREMENTAL OPS PATH (shared logic) ────────────────────
    HAS_OPS{Remote ops found?}

    %% No remote ops
    HAS_OPS -->|No| EMPTY_SVR{Empty server
+ fresh client
+ has local data?}
    EMPTY_SVR -->|Yes| SILENT_MIG[Silent server migration
creates SYNC_IMPORT]
    EMPTY_SVR -->|No| UPLOAD
    SILENT_MIG --> UPLOAD

    %% Has remote ops
    HAS_OPS -->|Yes| IS_FRESH{Fresh client?}
    IS_FRESH -->|No| IS_IMPORT
    IS_FRESH -->|Yes + has local data| CONFLICT_DLG[SyncConflictDialog:
USE_LOCAL / USE_REMOTE / CANCEL]
    IS_FRESH -->|Yes + no local data| CONFIRM[Confirm dialog:
Download remote?]
    CONFIRM -->|OK| APPLY
    CONFIRM -->|Cancel| CANCELLED
    CONFLICT_DLG -->|Use Local| FORCE_UP
    CONFLICT_DLG -->|Use Remote| FORCE_DL
    CONFLICT_DLG -->|Cancel| CANCELLED

    %% SYNC_IMPORT handling (shared logic)
    IS_IMPORT{Contains SYNC_IMPORT?}
    IS_IMPORT -->|No| CONFLICT_CHK
    IS_IMPORT -->|Yes| ENC_ONLY{Encryption-only change
+ no pending ops?}
    ENC_ONLY -->|Yes| APPLY
    ENC_ONLY -->|No| IMPORT_HAS{Has pending ops
or meaningful
local data?}
    IMPORT_HAS -->|Yes| IMPORT_DLG[ImportConflictDialog:
import reason shown,
Use Server Data recommended]
    IMPORT_HAS -->|No| APPLY_IMPORT[Apply full state replacement]
    IMPORT_DLG -->|Use Server| FORCE_DL
    IMPORT_DLG -->|Use Local| FORCE_UP
    IMPORT_DLG -->|Cancel| CANCELLED

    %% Conflict detection (shared logic)
    CONFLICT_CHK{Vector clock conflict?} -->|CONCURRENT| LWW[Auto-resolve LWW
later timestamp wins
ties → remote wins
archive ops always win]
    CONFLICT_CHK -->|No conflict| APPLY
    LWW --> APPLY[Apply ops to NgRx store]
    APPLY_IMPORT --> UPLOAD

    %% ── UPLOAD PHASE (file-based specific) ─────────────────────
    APPLY --> UPLOAD[Upload: merge state
into sync-data.json]
    UPLOAD --> REV{Rev match
on upload?}
    REV -->|OK| IN_SYNC([IN_SYNC ✓])
    REV -->|Mismatch| RETRY[Exponential backoff:
re-download, rebuild,
re-upload]
    RETRY --> RETRY_CHK{Max retries?}
    RETRY_CHK -->|Not exceeded| REV
    RETRY_CHK -->|Exceeded| ERROR

    FORCE_UP --> IN_SYNC
    FORCE_DL --> IN_SYNC
    MIGRATION --> IN_SYNC
    ERROR([ERROR])

    %% Styling
    classDef success fill:#2d6,stroke:#1a4,color:#fff
    classDef error fill:#d33,stroke:#a11,color:#fff
    classDef cancel fill:#888,stroke:#555,color:#fff
    classDef dialog fill:#48f,stroke:#26d,color:#fff
    classDef action fill:#e90,stroke:#b60,color:#fff,stroke-width:3px

    class IN_SYNC success
    class ERROR error
    class CANCELLED cancel
    class SS_CONFIRM,CONFIRM,SS_CONFLICT,CONFLICT_DLG,IMPORT_DLG,NO_PWD_DLG,WRONG_PWD_DLG dialog
    class APPLY,APPLY_IMPORT,FORCE_UP,FORCE_DL,MIGRATION,UPLOAD,SILENT_MIG,HYDRATE,RETRY action

Error Handling (SyncWrapperService)

Errors thrown during sync are caught by SyncWrapperService._sync(). File-based providers surface additional error types not seen with SuperSync:

mermaid
flowchart LR
    ERR([Error thrown
during sync]) --> TYPE{Error type?}

    TYPE -->|DecryptNoPasswordError| PWD_DLG[Enter Password dialog]
    TYPE -->|DecryptError| DEC_DLG[Decrypt Error dialog]
    TYPE -->|LocalDataConflictError| CONF_DLG[SyncConflictDialog]
    TYPE -->|RevMismatchForModelError
NoRemoteModelFile| INC_DLG["Incomplete sync" dialog:
Force Upload / Force Download]
    TYPE -->|SyncInvalidTimeValuesError| TIME_DLG["Incoherent timestamps" dialog:
Force Upload / Force Download]
    TYPE -->|LockPresentError| LOCK[Snackbar + Force Overwrite action]
    TYPE -->|PotentialCorsError| CORS[CORS error snackbar]
    TYPE -->|AuthFail / MissingCredentials| AUTH[Auth error snackbar
+ Configure action]
    TYPE -->|WebCryptoNotAvailable| CRYPTO[WebCrypto snackbar]
    TYPE -->|Timeout| TIMEOUT[Timeout error snackbar]
    TYPE -->|Permission error| PERM[Permission error snackbar]
    TYPE -->|Other| GENERIC[Generic error snackbar]

    classDef dialog fill:#48f,stroke:#26d,color:#fff
    classDef snack fill:#f90,stroke:#b60,color:#fff

    class PWD_DLG,DEC_DLG,CONF_DLG,INC_DLG,TIME_DLG dialog
    class LOCK,CORS,AUTH,CRYPTO,TIMEOUT,PERM,GENERIC snack

Legend:

  • 🟢 Green = success states
  • 🔴 Red = error states
  • 🔵 Blue = user-facing dialogs
  • 🟠 Orange = key actions (state changes, uploads, downloads)
  • ⚫ Gray = cancelled/disabled

Key Differences from SuperSync

AspectFile-Based (Dropbox, WebDAV, LocalFile)SuperSync
TransportDownloads/uploads a single sync-data.json filePaginated API (server-side op log)
Snapshot pathFull snapshotState on seq 0 download, with its own conflict-checking flowNo snapshot concept — all ops are incremental
Gap detectionAdapter detects syncVersion reset / snapshot replacement / partial trimming → re-download from seq 0Server handles gap detection internally
Server migrationGap on empty server → needsFullStateUploadhandleServerMigration()Same concept but detected via different mechanism
Upload retryRev matching (ETag) + exponential backoff with jitterServer rejection codes (CONFLICT_CONCURRENT)
PiggybackingNot applicable — no server to piggyback. Concurrent changes are discovered on re-download during retry.Server returns piggybacked ops in upload response
Post-sync encryption promptNot applicablePrompts user to set password or disable sync
File-based error typesRevMismatchForModelError, NoRemoteModelFile, SyncInvalidTimeValuesError, LockPresentError, PotentialCorsErrorNot applicable

Notes

  • The Enter Password and Decrypt Error dialogs correspond to DecryptNoPasswordError and DecryptError respectively — they are shared with SuperSync and are distinct components with different options.
  • Encryption-only change bypass: when an incoming SYNC_IMPORT has syncImportReason === 'PASSWORD_CHANGED' and there are no meaningful pending ops, the dialog is skipped (data is identical, only encryption changed).
  • LWW tie-breaking: on equal timestamps, remote wins (server-authoritative). moveToArchive operations always win regardless of timestamp.
  • Gap detection triggers: (1) syncVersion reset — another client uploaded a snapshot resetting the counter; (2) snapshot replacement — recentOps is empty but state exists and clientId differs; (3) partial trimming — oldestOpSyncVersion > sinceSeq and buffer is full.
  • Upload retry uses exponential backoff: base × 2^(attempt-1) + random(0..50%) with max retries defined by FILE_BASED_SYNC_CONSTANTS.MAX_UPLOAD_RETRIES.

Key Source Files

FileRole
src/app/imex/sync/sync-wrapper.service.tsTop-level orchestration + error handling
src/app/op-log/sync/operation-log-sync.service.tsDownload/upload orchestration, fresh client checks, SYNC_IMPORT handling
src/app/op-log/sync/operation-log-download.service.tsDownload + internal gap detection
src/app/op-log/sync-providers/file-based/file-based-sync-adapter.service.tsFile adapter (rev matching, gap detection, snapshot upload)