Migration System Improvements Summary

Overview

Comprehensive improvements to the WeKan migration system to ensure migrations only run when needed and show real progress, not simulated progress.

Problem Statement

The previous migration system had several issues:

1. Simulated Progress: Many migrations were showing simulated progress instead of tracking actual database changes
2. False Positives: Fresh WeKan installations were running migrations unnecessarily (no old data to migrate)
3. Missing Checks: Some migration types didn't have explicit "needs migration" checks

Solutions Implemented

1. Fixed isMigrationNeeded() Default Case

File: server/cronMigrationManager.js (lines 402-490)

Change: Modified the default case in isMigrationNeeded() switch statement:

// BEFORE: default: return true;  // This caused all unknown migrations to run
// AFTER:  default: return false; // Only run migrations we explicitly check for

Impact:

• Prevents spurious migrations on fresh installs
• Only migrations with explicit checks are considered "needed"

2. Added Explicit Checks for All 13 Migration Types

All migrations now have explicit checks in isMigrationNeeded():

3. All Migrations Now Use REAL Progress Tracking

Each migration implementation uses actual database queries and counts:

Example - Board Color Migration (executeBoardColorMigration):

// Real check - finds boards that actually need migration
const boardsNeedingMigration = Boards.find({
  $or: [
    { color: { $exists: true, $ne: null } },
    { color: { $regex: /^(?!css-)/ } }
  ]
}, { fields: { _id: 1 } }).fetch();

// Real progress tracking
for (const board of boardsNeedingMigration) {
  Boards.update(board._id, { $set: { colorClass: `css-${board.color}` } });
  updated++;
  
  const progress = Math.round((updated / total) * 100);
  cronJobStorage.saveJobStep(jobId, stepIndex, {
    progress,
    currentAction: `Migrating board colors: ${updated}/${total}`
  });
}

4. Implementation Methods Added/Updated

New Methods:

• executeAvatarMigration() (line 1344): Checks for legacy avatars, returns immediately for fresh installs
• executeBoardColorMigration() (line 1375): Converts old color format to CSS classes with real progress
• executeChecklistItemsMigration() (line 1432): Initializes checklist items array with real progress

Updated Methods (all with REAL implementations):

• executeLowercasePermission() - Converts board permissions to lowercase
• executeAttachmentTypeStandardization() - Updates attachment types with counts
• executeCardCoversMigration() - Migrates card cover data with progress tracking
• executeMemberActivityMigration() - Adds isActive field to board members
• executeAddSwimlanesIdMigration() - Adds swimlaneId to cards
• executeAddCardTypesMigration() - Adds type field to cards
• executeAttachmentMigration() - Migrates attachments from CollectionFS
• executeDenormalizeStarCount() - Counts and denormalizes starred board data
• executeEnsureValidSwimlaneIds() - Validates swimlane references
• executeComprehensiveBoardMigration() - Handles per-swimlane migration

5. Removed Simulated Execution Fallback

File: server/cronMigrationManager.js (lines 556-567)

Change: Removed the simulated progress fallback and replaced with a warning:

// BEFORE: Simulated 10-step progress for unknown migrations
// AFTER: 
console.warn(`Unknown migration step: ${stepId} - no handler found.`);
cronJobStorage.saveJobStep(jobId, stepIndex, {
  progress: 100,
  currentAction: `Migration skipped: No handler for ${stepId}`
});

Impact:

• No more simulated work for unknown migrations
• Clear logging if a migration type is not recognized
• All migrations show real progress or properly report as not needed

6. Added Missing Import

File: server/cronMigrationManager.js (line 17)

Added import for Checklists model:

import Checklists from '/models/checklists';

Migration Behavior on Fresh Install

When WeKan is freshly installed:

1. Each migration's isMigrationNeeded() is called
2. Checks run for actual old data structures
3. No old structures found → isMigrationNeeded() returns false
4. Migrations are skipped efficiently without unnecessary database work
5. Example log: "All checklists properly configured. No migration needed."

Migration Behavior on Old Database

When WeKan starts with an existing database containing old structures:

1. Each migration's isMigrationNeeded() is called
2. Checks find old data structures present
3. isMigrationNeeded() returns true
4. Migration handler executes with real progress tracking
5. Actual database records are updated with real counts
6. Progress shown: "Migrating X records (50/100)"

Benefits

✅ No Unnecessary Work: Fresh installs skip all migrations immediately
✅ Real Progress: All shown progress is based on actual database operations
✅ Clear Logging: Each step logs what's happening
✅ Error Tracking: Failed records are logged with context
✅ Transparent: No simulated execution hiding what's actually happening
✅ Safe: All 13 migration types have explicit handlers

Testing Checklist

• Fresh WeKan install shows all migrations as "not needed"
• No migrations execute on fresh database
• Old database with legacy data triggers migrations
• Migration progress shows real record counts
• All migrations complete successfully
• Migration errors are properly logged with context
• Admin panel shows accurate migration status

Files Modified

• server/cronMigrationManager.js - Core migration system with all improvements
• client/components/swimlanes/swimlanes.js - Drag-to-empty-swimlane feature (previous work)

Migration Types Summary

The WeKan migration system now properly manages 13 migration types:

All migrations now have real implementations with actual progress tracking!