v3/implementation/adrs/ADR-009-IMPLEMENTATION.md
This implementation provides a HybridBackend that combines SQLite (structured queries) and AgentDB (vector search) as per ADR-009. The hybrid approach leverages the strengths of both backends:
┌─────────────────────────────────────────────────────────────┐
│ HybridBackend │
│ (Intelligent Router) │
└───────────────────┬────────────────────┬────────────────────┘
│ │
┌───────────▼──────────┐ ┌─────▼──────────────────┐
│ SQLiteBackend │ │ AgentDBAdapter │
│ ───────────────── │ │ ────────────────── │
│ • Exact matches │ │ • Vector search │
│ • Prefix queries │ │ • HNSW indexing │
│ • Complex SQL │ │ • Semantic similarity │
│ • ACID transactions │ │ • LRU caching │
│ • Full-text search │ │ • 150x-12,500x faster │
└──────────────────────┘ └────────────────────────┘
/src/sqlite-backend.ts (788 lines)Complete SQLite implementation with:
/src/hybrid-backend.ts (747 lines)Hybrid backend combining both storage systems:
queryStructured() - Routes to SQLitequerySemantic() - Routes to AgentDBqueryHybrid() - Combines both/src/hybrid-backend.test.ts (380 lines)Comprehensive test suite covering:
/src/index.tsSQLiteBackend, HybridBackend, and related typescreateHybridService() factory function/package.jsonbetter-sqlite3 dependency (v11.0.0)@types/better-sqlite3 dev dependencyimport { HybridBackend } from '@claude-flow/memory';
const backend = new HybridBackend({
sqlite: {
databasePath: './data/memory.db',
walMode: true,
optimize: true,
},
agentdb: {
dimensions: 1536,
cacheEnabled: true,
hnswM: 16,
hnswEfConstruction: 200,
},
embeddingGenerator: async (text) => {
// Your embedding function (OpenAI, etc.)
return embeddings.embed(text);
},
dualWrite: true, // Write to both backends
routingStrategy: 'auto', // Auto-route based on query type
});
await backend.initialize();
// Exact key match - goes to SQLite
const user = await backend.getByKey('users', '[email protected]');
// Prefix query - goes to SQLite
const adminUsers = await backend.queryStructured({
namespace: 'users',
keyPrefix: 'admin-',
limit: 10,
});
// Time-based query - goes to SQLite
const recentDocs = await backend.queryStructured({
namespace: 'documents',
createdAfter: Date.now() - 86400000, // Last 24 hours
limit: 20,
});
// Semantic search - goes to AgentDB with HNSW
const similar = await backend.querySemantic({
content: 'authentication best practices',
k: 10,
threshold: 0.8, // Minimum similarity
});
// Semantic with filters
const securityDocs = await backend.querySemantic({
content: 'security vulnerabilities',
k: 20,
filters: {
namespace: 'security',
tags: ['critical'],
createdAfter: Date.now() - 604800000, // Last week
},
});
// Combine semantic + structured
const results = await backend.queryHybrid({
semantic: {
content: 'user authentication patterns',
k: 10,
threshold: 0.7,
},
structured: {
namespace: 'architecture',
keyPrefix: 'auth-',
createdAfter: Date.now() - 2592000000, // Last 30 days
},
combineStrategy: 'semantic-first', // Prefer semantic results
});
// Union strategy - all results from both backends
const unionResults = await backend.queryHybrid({
semantic: { content: 'database optimization', k: 5 },
structured: { namespace: 'performance', limit: 5 },
combineStrategy: 'union',
});
// Intersection strategy - only common results
const intersectionResults = await backend.queryHybrid({
semantic: { content: 'security patterns', k: 20 },
structured: { tags: ['security', 'critical'] },
combineStrategy: 'intersection',
});
// The backend automatically routes based on query properties
// This goes to SQLite (exact match)
const exactMatch = await backend.query({
type: 'exact',
key: 'user-123',
namespace: 'users',
limit: 1,
});
// This goes to AgentDB (semantic)
const semanticMatch = await backend.query({
type: 'semantic',
content: 'authentication patterns',
limit: 10,
});
// This uses hybrid (has both semantic and structured components)
const hybridMatch = await backend.query({
type: 'hybrid',
content: 'security best practices',
namespace: 'security',
tags: ['critical'],
limit: 15,
});
The HybridBackend intelligently routes queries:
| Query Type | Backend | Reason |
|---|---|---|
exact | SQLite | Optimized for exact key lookups with indexes |
prefix | SQLite | Optimized for LIKE queries with indexes |
tag | SQLite | Efficient JSON filtering |
semantic | AgentDB | HNSW vector search (150x-12,500x faster) |
hybrid | Both | Combines results from both backends |
| Auto (has embedding) | AgentDB | Semantic search capability |
| Auto (has key/prefix) | SQLite | Structured query capability |
| Auto (default) | AgentDB | Has caching, good default |
interface HybridBackendConfig {
// SQLite configuration
sqlite?: {
databasePath: string; // Path or ':memory:'
walMode: boolean; // Enable WAL (default: true)
optimize: boolean; // Auto-optimize (default: true)
maxEntries: number; // Max entries (default: 1M)
verbose: boolean; // SQL logging (default: false)
};
// AgentDB configuration
agentdb?: {
dimensions: number; // Vector dimensions (default: 1536)
cacheEnabled: boolean; // Enable LRU cache (default: true)
cacheSize: number; // Cache size (default: 10K)
cacheTtl: number; // Cache TTL ms (default: 5 min)
hnswM: number; // HNSW M parameter (default: 16)
hnswEfConstruction: number; // HNSW ef (default: 200)
};
// Hybrid configuration
embeddingGenerator?: (text: string) => Promise<Float32Array>;
routingStrategy?: 'auto' | 'sqlite-first' | 'agentdb-first';
dualWrite?: boolean; // Write to both (default: true)
semanticThreshold?: number; // Similarity threshold (default: 0.7)
hybridMaxResults?: number; // Max per backend (default: 100)
}
// Get statistics from both backends
const stats = await backend.getStats();
console.log('Total entries:', stats.totalEntries);
console.log('SQLite queries:', stats.routingStats.sqliteQueries);
console.log('AgentDB queries:', stats.routingStats.agentdbQueries);
console.log('Hybrid queries:', stats.routingStats.hybridQueries);
console.log('Avg query time:', stats.avgQueryTime, 'ms');
console.log('Memory usage:', stats.memoryUsage, 'bytes');
// Health check both backends
const health = await backend.healthCheck();
console.log('Overall status:', health.status);
console.log('SQLite health:', health.components.sqlite);
console.log('AgentDB health:', health.components.agentdb);
console.log('Issues:', health.issues);
console.log('Recommendations:', health.recommendations);
For existing systems using only AgentDB:
import { HybridBackend, AgentDBAdapter } from '@claude-flow/memory';
// Old: AgentDB only
const oldBackend = new AgentDBAdapter(config);
// New: Hybrid (backward compatible)
const newBackend = new HybridBackend({
agentdb: config, // Same config
dualWrite: true, // Enable dual-write
});
// Gradually migrate queries to use structured when appropriate
// const user = await backend.getByKey('users', 'id'); // Now uses SQLite
Run the comprehensive test suite:
cd /workspaces/claude-flow/v3/@claude-flow/memory
npm test src/hybrid-backend.test.ts
Test coverage includes:
Best of Both Worlds
Performance Optimization
Data Consistency
Flexibility
Production Ready
Batch Operations (ADR-006 extension)
bulkInsert(): 4-phase parallel embedding generationbulkGet(): Parallel retrieval via Promise.all()bulkUpdate(): Batch updates with parallel processingbulkDelete(): Parallel deletionPerformance Results
@claude-flow/[email protected]Last Updated: 2026-01-07
The HybridBackend implementation successfully achieves ADR-009 goals:
This provides Claude Flow V3 with a flexible, performant, and reliable memory system that adapts to different query patterns while maintaining consistency across both backends.