Back to Ruflo

Memory & Neural System Migration Guide

v3/implementation/v3-migration/MEMORY-NEURAL-MIGRATION.md

3.6.3012.7 KB
Original Source

Memory & Neural System Migration Guide

Migrating from V2 Memory/Neural to V3 Enhanced Systems

Overview

V3 significantly enhances both memory and neural systems:

  • Memory: +6 new features (20 total vs 14)
  • Neural: +11 new features (14 total vs 3)

Memory System Comparison

V2 Memory Architecture

v2/src/memory/
├── backends/
│   ├── base.ts          # IMemoryBackend interface
│   ├── sqlite.ts        # SQLite backend
│   └── markdown.ts      # Markdown backend
├── manager.ts           # Memory Manager
├── advanced-memory-manager.ts
├── indexer.ts           # Search indexing
├── cache.ts             # LRU cache
├── swarm-memory.ts      # Multi-agent memory
└── distributed-memory.ts # CRDT sync

V3 Memory Architecture

v3/@claude-flow/memory/
├── src/
│   ├── types.ts              # Type definitions
│   ├── index.ts              # UnifiedMemoryService
│   ├── sqlite-backend.ts     # SQLite (secured)
│   ├── sqljs-backend.ts      # WASM SQLite (NEW)
│   ├── agentdb-backend.ts    # AgentDB (NEW)
│   ├── hybrid-backend.ts     # SQLite+AgentDB (NEW)
│   ├── database-provider.ts  # Cross-platform (NEW)
│   ├── hnsw-index.ts         # HNSW vector index (NEW)
│   ├── cache-manager.ts      # Enhanced cache
│   ├── query-builder.ts      # Fluent queries (NEW)
│   ├── migration.ts          # Data migration (NEW)
│   ├── agentdb-adapter.ts    # AgentDB adapter
│   └── domain/               # DDD structure (NEW)
│       ├── entities/
│       ├── repositories/
│       └── services/

Memory Feature Migration

Implemented in V3 ✅

V2 FeatureV3 EquivalentEnhancement
SQLite Backendsqlite-backend.tsSQL injection protection
Memory ManagerUnifiedMemoryServiceDDD architecture
Indexerhnsw-index.ts150x-12,500x faster
Cachecache-manager.tsTTL, importance-based eviction
Swarm MemoryshareWith()/getSharedWith()Simplified API

New in V3 ✅

FeatureFileDescription
HNSW Indexhnsw-index.tsApproximate nearest neighbor search
AgentDB Backendagentdb-backend.tsNative vector database
Hybrid Backendhybrid-backend.tsSQLite + AgentDB combo
SQL.js Backendsqljs-backend.tsWASM for cross-platform
Database Providerdatabase-provider.tsAuto-selects best backend
Query Builderquery-builder.tsFluent API for queries
Quantizationhnsw-index.ts4-32x memory reduction
DDD Domaindomain/Clean architecture

Missing in V3 ❌

V2 FeaturePriorityMigration Path
Markdown BackendLOWOptional human-readable export
Distributed MemoryMEDIUMAdd CRDT sync for multi-node

Memory Migration Code

Backend Migration

typescript
// V2: Initialize memory
import { MemoryManager } from 'claude-flow/memory';
const memory = new MemoryManager({
  backend: 'sqlite',
  path: './.claude-flow/memory.db'
});

// V3: Initialize memory
import { UnifiedMemoryService } from '@claude-flow/memory';
const memory = new UnifiedMemoryService({
  backend: 'hybrid',  // SQLite + AgentDB
  sqlite: {
    path: './.claude-flow/memory.db'
  },
  agentdb: {
    enableHNSW: true,
    dimensions: 384
  }
});
await memory.initialize();

Store Migration

typescript
// V2: Store entry
await memory.store({
  namespace: 'default',
  sessionId: 'session-1',
  agentId: 'agent-1',
  type: 'observation',
  content: 'User requested feature X',
  tags: ['feature', 'request'],
  metadata: { priority: 'high' }
});

// V3: Store entry (enhanced)
await memory.store({
  content: 'User requested feature X',
  type: 'episodic',
  category: 'observations',
  tags: ['feature', 'request'],
  metadata: {
    priority: 'high',
    sessionId: 'session-1',
    agentId: 'agent-1'
  },
  importance: 0.8,
  ttl: 86400000  // 24 hours
});

Query Migration

typescript
// V2: Query entries
const results = await memory.query({
  namespace: 'default',
  search: 'feature request',
  type: 'observation',
  limit: 10
});

// V3: Query entries (enhanced with semantic search)
const results = await memory.search({
  query: 'feature request',
  searchType: 'hybrid',  // semantic + keyword
  type: 'episodic',
  limit: 10,
  minRelevance: 0.7
});

// V3: Fluent query builder
import { QueryBuilder } from '@claude-flow/memory';
const results = await new QueryBuilder(memory)
  .semantic('feature request')
  .type('episodic')
  .tags(['feature'])
  .minRelevance(0.7)
  .limit(10)
  .execute();

Vector Search (V3 Only)

typescript
// V3: HNSW vector search
import { HNSWIndex } from '@claude-flow/memory';

const index = new HNSWIndex({
  dimensions: 384,
  maxElements: 100000,
  efConstruction: 200,
  M: 16
});

// Add vectors
await index.add(vectorId, embedding);

// Search
const results = await index.search(queryEmbedding, k: 10);
// Returns: [{ id, distance, similarity }]

Data Migration Script

typescript
// Migrate V2 data to V3
import { migrateMemoryData } from '@claude-flow/memory/migration';

await migrateMemoryData({
  source: {
    type: 'v2-sqlite',
    path: './.claude-flow/memory.db'
  },
  target: {
    type: 'v3-hybrid',
    sqlitePath: './.claude-flow/v3-memory.db',
    agentdbPath: './.claude-flow/v3-vectors'
  },
  options: {
    generateEmbeddings: true,
    preserveTimestamps: true,
    batchSize: 1000
  }
});

Neural System Comparison

V2 Neural Architecture

v2/src/
├── neural/
│   ├── NeuralDomainMapper.ts  # GNN domain mapping
│   └── integration.ts         # Hooks integration
├── services/agentic-flow-hooks/
│   └── neural-hooks.ts        # Training hooks
└── reasoningbank/
    └── reasoningbank-adapter.js  # Via agentic-flow

V3 Neural Architecture

v3/@claude-flow/neural/
├── src/
│   ├── index.ts              # NeuralLearningSystem
│   ├── sona-manager.ts       # SONA modes
│   ├── sona-integration.ts   # @ruvector/sona
│   ├── reasoning-bank.ts     # Native ReasoningBank
│   ├── pattern-learner.ts    # Pattern extraction
│   ├── types.ts
│   ├── algorithms/           # RL algorithms
│   │   ├── ppo.ts
│   │   ├── dqn.ts
│   │   ├── a2c.ts
│   │   ├── decision-transformer.ts
│   │   ├── q-learning.ts
│   │   ├── sarsa.ts
│   │   └── curiosity.ts
│   ├── modes/                # Learning modes
│   │   ├── real-time.ts
│   │   ├── balanced.ts
│   │   ├── research.ts
│   │   ├── edge.ts
│   │   └── batch.ts
│   └── domain/               # DDD structure
│       ├── entities/
│       └── services/

Neural Feature Migration

V2 Features

FeatureFileStatus in V3
Neural Domain MapperNeuralDomainMapper.ts⚠️ Partial (pattern-learner)
Neural Hooksneural-hooks.tssona-manager.ts
ReasoningBank Adapterreasoningbank-adapter.js✅ Native reasoning-bank.ts

V3 New Features ✅

FeatureFileDescription
SONA Managersona-manager.ts5 learning modes
Learning Modesmodes/*.tsReal-time, Balanced, Research, Edge, Batch
Native ReasoningBankreasoning-bank.ts4-step pipeline
Pattern Learnerpattern-learner.tsTrajectory extraction
PPO Algorithmalgorithms/ppo.tsProximal Policy Optimization
DQN Algorithmalgorithms/dqn.tsDeep Q-Network
A2C Algorithmalgorithms/a2c.tsAdvantage Actor-Critic
Decision Transformeralgorithms/decision-transformer.tsTransformer RL
Q-Learningalgorithms/q-learning.tsClassic Q-Learning
SARSAalgorithms/sarsa.tsOn-policy TD
Curiosity Modulealgorithms/curiosity.tsIntrinsic motivation
LoRA Weightssona-manager.tsLow-rank adaptation
EWCsona-manager.tsElastic weight consolidation

Neural Migration Code

Initialize Neural System

typescript
// V2: Neural integration
import { NeuralDomainMapper } from 'claude-flow/neural';
import { registerNeuralHooks } from 'claude-flow/neural/integration';

const mapper = new NeuralDomainMapper();
registerNeuralHooks(mapper);

// V3: Neural learning system
import { NeuralLearningSystem } from '@claude-flow/neural';

const neural = new NeuralLearningSystem({
  mode: 'balanced',  // real-time | balanced | research | edge | batch
  reasoningBank: {
    maxPatterns: 10000,
    retrievalK: 5
  },
  sona: {
    enableLoRA: true,
    enableEWC: true
  }
});
await neural.initialize();

Pattern Learning

typescript
// V2: Train patterns
await mapper.train(domainGraph, {
  optimizer: 'adam',
  learningRate: 0.001,
  epochs: 100
});

// V3: Learn from trajectories
const trajectory = neural.startTrajectory('implement-feature');
trajectory.addStep({
  action: 'analyze',
  context: { files: ['src/app.ts'] },
  result: { success: true }
});
trajectory.addStep({
  action: 'implement',
  context: { changes: 5 },
  result: { success: true }
});
await trajectory.complete({ quality: 0.9 });

// Patterns automatically extracted and stored

ReasoningBank

typescript
// V2: Via agentic-flow adapter
import { ReasoningBankAdapter } from 'claude-flow/reasoningbank';
const rb = new ReasoningBankAdapter();
await rb.store(memory);
const patterns = await rb.retrieve(query);

// V3: Native implementation
import { ReasoningBank } from '@claude-flow/neural';

const rb = new ReasoningBank({
  memory: agentDbBackend,  // Uses AgentDB for 150x faster search
  maxPatterns: 10000
});

// Store trajectory
await rb.store(trajectory);

// 4-step pipeline
const patterns = await rb.retrieve(query, { k: 5 });  // RETRIEVE
const judged = await rb.judge(trajectory);            // JUDGE
const distilled = await rb.distill(trajectories);     // DISTILL
await rb.consolidate();                               // CONSOLIDATE

SONA Learning Modes

typescript
// V3 only: Configure learning modes
import { SONAManager, LearningMode } from '@claude-flow/neural';

const sona = new SONAManager();

// Real-time mode: 2200 ops/sec, micro-LoRA
sona.setMode(LearningMode.REAL_TIME);

// Balanced mode: +25% quality, standard LoRA
sona.setMode(LearningMode.BALANCED);

// Research mode: +55% quality, full fine-tuning
sona.setMode(LearningMode.RESEARCH);

// Edge mode: <5MB, aggressive quantization
sona.setMode(LearningMode.EDGE);

// Batch mode: High throughput, async processing
sona.setMode(LearningMode.BATCH);

RL Algorithms

typescript
// V3 only: Use RL algorithms
import { createPPO, createDQN, createA2C } from '@claude-flow/neural/algorithms';

// PPO for continuous action spaces
const ppo = createPPO({
  actor: { hiddenLayers: [64, 64] },
  critic: { hiddenLayers: [64, 64] },
  clipRatio: 0.2,
  gamma: 0.99
});

// DQN for discrete actions
const dqn = createDQN({
  hiddenLayers: [64, 64],
  epsilon: 0.1,
  targetUpdateFreq: 100
});

// A2C for parallel environments
const a2c = createA2C({
  numWorkers: 4,
  entropyCoeff: 0.01
});

Continual Learning

typescript
// V3 only: LoRA and EWC
import { SONAManager } from '@claude-flow/neural';

const sona = new SONAManager({
  enableLoRA: true,
  loraConfig: {
    rank: 8,
    alpha: 16,
    dropout: 0.1
  },
  enableEWC: true,
  ewcConfig: {
    lambda: 1000,
    decay: 0.99
  }
});

// LoRA adapts quickly, EWC prevents forgetting
await sona.train(newTask);

Performance Comparison

OperationV2V3Improvement
Vector SearchBrute-force O(n)HNSW O(log n)150x-12,500x
Memory StoreSQLite onlyHybridOptimized routing
Pattern RetrievalVia adapterNative~10x faster
Learning AdaptationManualSONA<0.05ms
Memory UsageFull vectorsQuantized4-32x reduction

Migration Checklist

Memory Migration

  • Update imports to @claude-flow/memory
  • Configure hybrid backend
  • Run data migration script
  • Generate embeddings for existing entries
  • Update query code to use semantic search
  • Test HNSW index performance

Neural Migration

  • Update imports to @claude-flow/neural
  • Initialize NeuralLearningSystem
  • Configure SONA mode
  • Migrate training code to trajectories
  • Update pattern retrieval to ReasoningBank
  • Enable LoRA/EWC if needed