Back to Cherry Studio

ChatMigrator

src/main/data/migration/v2/migrators/README-ChatMigrator.md

2.0.09.3 KB
Original Source

ChatMigrator

The ChatMigrator handles the largest data migration task: topics and messages from Dexie/IndexedDB to SQLite.

Data Sources

DataSourceFile/Path
Topics with messagesDexie topics tabletopics.json
Topic metadata (name, pinned, etc.)Redux assistants[].topics[] and defaultAssistant.topics[]ReduxStateReader.getCategory('assistants')
Message blocksDexie message_blocks tablemessage_blocks.json
Assistants (for meta)Redux assistants slice (incl. defaultAssistant)ReduxStateReader.getCategory('assistants')

Topic Data Split (Important!)

The old system stores topic data in two separate locations:

  1. Dexie topics table: Contains only id and messages[] array (NO assistantId!)
  2. Redux assistants[].topics[] and defaultAssistant.topics[]: Contains metadata (name, pinned, prompt, isNameManuallyEdited) and implicitly the assistantId (from parent assistant)

Redux deliberately clears messages[] to reduce storage size. The migrator merges these sources:

  • Messages come from Dexie
  • Metadata (name, pinned, etc.) comes from Redux
  • assistantId comes from Redux structure (each assistant owns its topics)

Note: state.defaultAssistant is a sibling slot of state.assistants[], not a member of it. Topics living under defaultAssistant.topics[] were silently dropped before the migration walked this slot — their post-migration rows would otherwise have shown up as "Unnamed Topic" with no timestamp source.

Key Transformations

  1. Linear → Tree Structure

    • Old: Messages stored as linear array in topic.messages[]
    • New: Tree via parentId + siblingsGroupId
  2. Multi-model Responses

    • Old: askId links responses to user message, foldSelected marks active
    • New: Shared parentId + non-zero siblingsGroupId groups siblings
  3. Block Inlining

    • Old: message.blocks: string[] (IDs) + separate message_blocks table
    • New: message.data.blocks: MessageDataBlock[] (inline JSON)
  4. Citation Migration

    • Old: Separate CitationMessageBlock with response, knowledge, memories
    • New: Merged into MainTextBlock.references as ContentReference[]
  5. Mention Migration

    • Old: message.mentions: Model[]
    • New: MentionReference[] in MainTextBlock.references

Data Quality Handling

The migrator handles potential data inconsistencies from the old system:

IssueDetectionHandling
Duplicate message IDSame ID appears within or across topicsAssign globally unique IDs before tree construction, remap same-topic references to the nearest earlier occurrence, and log a warning
Forward askIdA response points to a message appearing later in the source arrayTreat it as unresolved and use the chronological fallback; parent edges may only target previously processed messages
Disconnected/cyclic message graphA migrated message is not reachable from a virtual topic rootFail migration validation before marking the migration complete
TopicId mismatchmessage.topicId ≠ parent topic.idUse correct parent topic.id (silent fix)
Missing blocksBlock ID not found in message_blocksSkip missing block (silent)
Invalid topicTopic missing required id fieldSkip entire topic
Empty source topictopic.messages missing or [] AND no user-intent signal (pinned / isNameManuallyEdited / non-blank prompt from Redux meta)Skip topic — v1 surfaced empty topics on first launch and on every abandoned "new topic" click; they have no timestamp source and would just clutter the post-migration list. Logged at info level. Empty topics that the user pinned, renamed, or wrote a topic-level prompt for are kept (intentional state).
Missing topic metadataTopic not found in Redux assistants[].topics[] / defaultAssistant.topics[]Use Dexie values, fallback name if empty
Legacy 'default' assistantIdtopic.assistantId === 'default' (or topic lived under state.defaultAssistant.topics[])Rewrite via sharedData.legacyAssistantIdRemap ('default' → UUID produced by AssistantMigrator). Resolves under the migrated user assistant — v2 has no 'default' sentinel row.
Missing assistantIdTopic not in any assistant.topics[], or empty/null assistantId after remapSet assistantId = NULL. v2's topic.assistantId is nullable (FK ON DELETE SET NULL); the renderer composes a runtime default from Preference.defaultModelId when no specific assistant is attached. orphanedAssistantTopics counter increments.
Orphan assistantIdtopic.assistantId (post-remap) not in validAssistantIdsSame NULL fallback as above; orphanedAssistantTopics counter increments and a warning is logged.
Empty topic nameBoth Dexie and Redux have empty name (ancient bug)Use fallback "Unnamed Topic"
Missing topic timestampsBoth Dexie and Redux miss createdAt / updatedAtDerive from messages: createdAt = min(message.createdAt), updatedAt = max(message.createdAt). If no message has a parseable createdAt, falls through to parseTimestamp()'s Date.now() fallback (logged as a warning).
Message with no blocksblocks array is empty after resolutionKeep legacy type: 'clear' messages as context boundaries, store them as hidden data-clear parts, and skip/re-link other empty messages
Topic where all messages are skippedAll messages dropped (no blocks)Keep topic, set activeNodeId to null. Distinct from the "empty source topic" case above (which is dropped).

Field Mappings

Topic Mapping

Topic data is merged from Dexie + Redux before transformation:

SourceTarget (topicTable)Notes
Dexie: ididDirect copy
Redux: namenameMerged from Redux assistants[].topics[]
Redux: isNameManuallyEditedisNameManuallyEditedMerged from Redux
Redux: (parent assistant.id)assistantIdFrom topicAssistantLookup mapping
(from Assistant)assistantMetaGenerated from assistant entity
Redux: promptpromptMerged from Redux
(computed)activeNodeIdSmart selection: original active → foldSelected → last migrated
(none)sortOrder0 (new field)
Redux: pinnedisPinnedMerged from Redux, renamed
(none)pinnedOrder0 (new field)
createdAtcreatedAtISO string → timestamp; if missing on both Dexie and Redux, derived from min(message.createdAt)
updatedAtupdatedAtISO string → timestamp; if missing on both Dexie and Redux, derived from max(message.createdAt)

Dropped fields: type ('chat' | 'session')

Message Mapping

Source (OldMessage)Target (messageTable)Notes
ididDirect copy (new UUID if duplicate)
(computed)parentIdFrom tree building algorithm
(from parent topic)topicIdUses parent topic.id for consistency
roleroleDirect copy
blocks + mentions + citations + clear typedataComplex transformation; converts legacy type: 'clear' to a hidden data-clear part
(extracted)searchableTextExtracted from text blocks
statusstatusNormalized to success/error/paused
(computed)siblingsGroupIdFrom multi-model detection
assistantIdassistantIdDirect copy
modelIdmodelIdDirect copy
(from Message.model)modelMetaGenerated from model entity
usage + metricsstatsMerged into single stats object
createdAtcreatedAtISO string → timestamp
updatedAtupdatedAtISO string → timestamp

Dropped fields: type after converting clear to data-clear, useful, enabledMCPs, agentSessionId, traceId (span detail files are not part of the v1 chat migration source set), providerMetadata, multiModelMessageStyle, askId (replaced by parentId), foldSelected (replaced by siblingsGroupId)

Block Type Mapping

Old TypeNew TypeNotes
main_textMainTextBlockDirect, references added from citations/mentions
thinkingThinkingBlockthinking_millsecthinkingMs
translationTranslationBlockDirect copy
codeCodeBlockDirect copy
imageImageBlockfile.idfileId
fileFileBlockfile.idfileId
videoVideoBlockDirect copy
toolToolBlockDirect copy
citation(removed)Converted to MainTextBlock.references
errorErrorBlockDirect copy
compactCompactBlockDirect copy
unknown(skipped)Placeholder blocks are dropped

Implementation Files

  • ChatMigrator.ts - Main migrator class with prepare/execute/validate phases
  • mappings/ChatMappings.ts - Pure transformation functions and type definitions

Code Quality

All implementation code includes detailed comments:

  • File-level comments: Describe purpose, data flow, and overview
  • Function-level comments: Purpose, parameters, return values, side effects
  • Logic block comments: Step-by-step explanations for complex logic
  • Data transformation comments: Old field → new field mapping relationships