Back to Ruflo

ADR-014: Cross-Platform Workers System

v3/implementation/adrs/ADR-014-workers-system.md

3.6.3013.8 KB
Original Source

ADR-014: Cross-Platform Workers System

Status

Implemented ✅ (Extended with CLI Integration 2026-01-06)

Date

2026-01-05

Last Updated

2026-01-06

Context

V3 needs a robust background worker system for:

  1. Continuous monitoring of system health, security, and performance
  2. Automatic DDD and ADR compliance tracking
  3. Pattern consolidation and learning optimization
  4. Git status tracking and swarm coordination
  5. Cache cleanup and resource management

V2 relies on shell scripts (.claude/helpers/) which are:

  • Platform-specific (Linux/macOS only)
  • Difficult to test
  • Not integrated with the TypeScript codebase
  • Lacking persistence and historical tracking

Decision

1. Create TypeScript Worker System in @claude-flow/hooks

A cross-platform worker system with:

  • 10 Built-in Workers: performance, health, security, adr, ddd, patterns, learning, cache, git, swarm
  • WorkerManager Class: Central orchestration with EventEmitter pattern
  • Persistence: State saved to disk, survives restarts
  • Historical Metrics: Track trends over time (max 1000 entries)
  • Alert System: Threshold-based notifications
  • Statusline Integration: Real-time metrics for display

2. Architecture

@claude-flow/hooks/src/workers/
├── index.ts           # WorkerManager, all worker implementations
├── mcp-tools.ts       # MCP tool definitions for workers
├── session-hook.ts    # Claude Code session integration
└── __tests__/         # Comprehensive test suite
    └── workers.test.ts

3. Worker Manager Features

typescript
class WorkerManager extends EventEmitter {
  // Core methods
  register(name: string, handler: WorkerHandler): void;
  async initialize(): Promise<void>;
  async start(options?: StartOptions): Promise<void>;
  async stop(): Promise<void>;
  async runWorker(name: string): Promise<WorkerResult>;
  async runAll(concurrency?: number): Promise<WorkerResult[]>;

  // Persistence
  async loadState(): Promise<boolean>;
  async saveState(): Promise<void>;

  // Alerts
  setThresholds(worker: string, thresholds: AlertThreshold[]): void;
  getAlerts(limit?: number): WorkerAlert[];
  clearAlerts(): void;

  // History
  getHistory(worker?: string, limit?: number): HistoricalMetric[];

  // Statusline
  getStatuslineData(): StatuslineData;
  getStatuslineString(): string;
  async exportStatusline(): Promise<void>;
}

4. Built-in Workers

WorkerIntervalDescription
performance5 minMemory, CPU, V3 code stats
health5 minSystem health monitoring
security30 minSecret/vulnerability scanning
adr15 minADR compliance checking
ddd10 minDDD pattern tracking
patterns15 minPattern consolidation
learning30 minSONA optimization
cache1 hourTemp file cleanup
git5 minBranch/commit status
swarm1 minAgent coordination

5. Alert Thresholds

typescript
const DEFAULT_THRESHOLDS = {
  health: [
    { metric: 'memory.usedPct', warning: 80, critical: 95, comparison: 'gt' },
    { metric: 'disk.usedPct', warning: 85, critical: 95, comparison: 'gt' },
  ],
  security: [
    { metric: 'secrets', warning: 1, critical: 5, comparison: 'gt' },
    { metric: 'vulnerabilities', warning: 10, critical: 50, comparison: 'gt' },
  ],
  adr: [
    { metric: 'compliance', warning: 70, critical: 50, comparison: 'lt' },
  ],
};

6. MCP Tools

8 MCP tools for Claude Code integration:

  • worker/run - Run specific worker
  • worker/status - Get worker status
  • worker/alerts - Get recent alerts
  • worker/history - Get historical metrics
  • worker/statusline - Get statusline data
  • worker/run-all - Run all workers
  • worker/start - Start scheduling
  • worker/stop - Stop and save

7. Session Integration

typescript
// Auto-start on session begin
export async function onSessionStart(config?: SessionHookConfig): Promise<SessionHookResult>;

// Clean shutdown on session end
export async function onSessionEnd(manager: WorkerManager): Promise<void>;

8. Security Hardening

Implemented safeguards:

  • safePath() - Path traversal prevention
  • safeReadFile() - File size limits (10MB)
  • MAX_RECURSION_DEPTH - Depth limit (20)
  • MAX_CONCURRENCY - Batch limit (5)
  • Symlink skipping
  • Cache deletion path validation

Consequences

Positive

  • ✅ Cross-platform (Linux, macOS, Windows)
  • ✅ Type-safe TypeScript implementation
  • ✅ Comprehensive test coverage
  • ✅ Persistence across restarts
  • ✅ Historical trend tracking
  • ✅ Threshold-based alerting
  • ✅ MCP tool integration
  • ✅ Statusline export for shell consumption
  • ✅ Security hardened

Negative

  • Requires Node.js runtime (not shell-only)
  • Additional memory for history storage
  • Slightly more complex than shell scripts

Trade-offs

  • JSON persistence vs SQLite: Chose JSON for simplicity and portability
  • Fixed intervals vs dynamic: Chose fixed for predictability
  • In-memory history vs disk: Chose in-memory with periodic save for performance

Performance Metrics

MetricTargetAchieved
All workers run time<2s✅ 527ms
Individual worker<500ms✅ Max 301ms (security)
State save time<100ms✅ <50ms
Memory overhead<50MB✅ ~5MB

Test Coverage

Tests:       45 passed (45)
Duration:    1.2s

Coverage:
- WorkerManager: Unit tests
- Alert System: Threshold tests
- Historical Metrics: Recording tests
- Statusline: Export tests
- Persistence: Save/load tests
- Security: Boundary tests
- Built-in Workers: Integration tests

Usage Examples

Basic Usage

typescript
import { createWorkerManager } from '@claude-flow/hooks';

const manager = createWorkerManager('/path/to/project');
await manager.initialize();
await manager.start();

// Run specific worker
const result = await manager.runWorker('security');
console.log(result.data);

// Get alerts
const alerts = manager.getAlerts(10);

// Get statusline
const statusline = manager.getStatuslineString();
// "👷0/10 │ 🟢15% │ 🛡️0 │ 📋71% │ 🏗️13% │ ⚡1.0x"

MCP Integration

typescript
import { createWorkerToolHandler, workerMCPTools } from '@claude-flow/hooks';

// Register tools with MCP server
const handler = createWorkerToolHandler(manager);

// Handle tool call
const result = await handler('worker/run', { worker: 'health' });

Session Hook

typescript
import { onSessionStart, formatSessionStartOutput } from '@claude-flow/hooks';

const result = await onSessionStart({
  projectRoot: '/path/to/project',
  autoStart: true,
  runInitialScan: true,
  workers: ['health', 'security', 'git'],
});

console.log(formatSessionStartOutput(result));
// [Workers] System initialized
//   ✓ Health: healthy
//   ✓ Security: clean (0 issues)
//   ├─ Branch: v3
//   └─ Uncommitted: 5
// [Workers] Background scheduling started

References

  • V2 Shell Scripts: .claude/helpers/worker-manager.sh
  • ADR-002: Domain-Driven Design Structure
  • ADR-006: Unified Memory Service
  • ADR-012: MCP Security Features

Extension: CLI Integration (2026-01-06)

CLI Hooks Worker Subcommand

Extended the worker system with CLI integration via hooks worker command in @claude-flow/cli.

New Worker Types (12 Total)

In addition to the original system workers, the CLI exposes 12 trigger-based workers:

WorkerPriorityEst. TimeDescription
ultralearnnormal60sDeep knowledge acquisition and learning
optimizehigh30sPerformance optimization and tuning
consolidatelow20sMemory consolidation and cleanup
predictnormal15sPredictive preloading and anticipation
auditcritical45sSecurity analysis and vulnerability scanning
mapnormal30sCodebase mapping and architecture analysis
preloadlow10sResource preloading and cache warming
deepdivenormal60sDeep code analysis and examination
documentnormal45sAuto-documentation generation
refactornormal30sCode refactoring suggestions
benchmarknormal60sPerformance benchmarking
testgapsnormal30sTest coverage analysis

CLI Commands

bash
# List all available workers
claude-flow hooks worker list

# Detect triggers from prompt text (<5ms target)
claude-flow hooks worker detect --prompt "optimize performance"

# Auto-dispatch when triggers match (confidence ≥0.6)
claude-flow hooks worker detect --prompt "deep dive" --auto-dispatch --min-confidence 0.6

# Manually dispatch a worker
claude-flow hooks worker dispatch --trigger refactor --context "auth module"

# Check worker status
claude-flow hooks worker status

# Cancel a running worker
claude-flow hooks worker cancel --id worker_refactor_1_abc123

MCP Tools Added

5 new MCP tools in @claude-flow/cli/src/mcp-tools/hooks-tools.ts:

  • hooks/worker-list - List all 12 background workers
  • hooks/worker-dispatch - Dispatch a worker by trigger type
  • hooks/worker-status - Get status of running workers
  • hooks/worker-detect - Detect worker triggers from prompt text
  • hooks/worker-cancel - Cancel a running worker

UserPromptSubmit Integration

Workers are automatically triggered via the UserPromptSubmit hook in .claude/settings.json:

json
{
  "hooks": {
    "UserPromptSubmit": [{
      "matcher": ".*",
      "hooks": [{
        "type": "command",
        "timeout": 6000,
        "command": "claude-flow hooks worker detect --prompt \"$USER_PROMPT\" --auto-dispatch --min-confidence 0.6"
      }]
    }]
  }
}

Parser Enhancement

Fixed nested subcommand routing in parser.ts to support 3 levels of subcommands:

  • Level 1: hooks
  • Level 2: worker
  • Level 3: list, dispatch, status, detect, cancel

Performance Targets

MetricTargetStatus
Trigger detection<5ms
Worker spawn<50ms
Max concurrent10

Extension: Node.js Worker Daemon (2026-01-07)

Daemon Service Architecture

Extended the worker system with a full Node.js daemon service in @claude-flow/cli/src/services/worker-daemon.ts. This replaces the shell-based helpers in .claude/helpers/ with a cross-platform TypeScript implementation.

Key Components

ComponentLocationPurpose
WorkerDaemonservices/worker-daemon.tsEventEmitter-based daemon service
daemon commandcommands/daemon.tsCLI with start/stop/status/trigger/enable
Session integrationhooks-tools.tsAuto-start on SessionStart, auto-stop on SessionEnd
Init settingsinit/settings-generator.tsDaemon config in v3 init output

Daemon CLI Commands

bash
# Start the daemon (runs workers on intervals)
npx claude-flow@v3alpha daemon start
npx claude-flow@v3alpha daemon start --quiet  # Run once and exit

# Stop the daemon
npx claude-flow@v3alpha daemon stop

# Check status and worker history
npx claude-flow@v3alpha daemon status

# Manually trigger a worker
npx claude-flow@v3alpha daemon trigger <worker>
npx claude-flow@v3alpha daemon trigger map --force

# Enable/disable workers
npx claude-flow@v3alpha daemon enable map audit optimize
npx claude-flow@v3alpha daemon enable --all

Worker Intervals (5 Enabled by Default)

WorkerIntervalPriorityDescription
map5minnormalCodebase structure mapping
audit10mincriticalSecurity vulnerability scanning
optimize15minhighPerformance optimization analysis
consolidate30minlowMemory consolidation and cleanup
testgaps20minnormalTest coverage gap analysis
predict10minnormalPredictive preloading (disabled by default)
document30minlowAuto-documentation (disabled by default)

Metrics Output

Workers write JSON metrics to .claude-flow/metrics/:

.claude-flow/metrics/
├── codebase-map.json      # map worker output
├── security-audit.json    # audit worker output
├── performance.json       # optimize worker output
├── consolidation.json     # consolidate worker output
├── test-gaps.json         # testgaps worker output
├── agent-metrics.json     # Agent performance data
└── task-metrics.json      # Task execution data

State Persistence

Daemon state is persisted to .claude-flow/daemon-state.json:

typescript
interface DaemonState {
  workers: {
    [key: string]: {
      enabled: boolean;
      runCount: number;
      successCount: number;
      failureCount: number;
      lastRun?: Date;
      lastError?: string;
    };
  };
  pid?: number;
  startedAt?: string;
}

Session Integration

typescript
// Auto-start on SessionStart hook
hooks.SessionStart = [{
  hooks: [{
    type: 'command',
    command: 'npx claude-flow@v3alpha daemon start --quiet 2>/dev/null || true',
    timeout: 5000,
    continueOnError: true,
  }]
}];

Performance Characteristics

MetricTargetAchieved
Daemon startup<500ms✅ ~200ms
Worker execution<500ms✅ ~1ms per worker
State persistence<50ms✅ ~10ms
Memory overhead<50MB✅ ~5MB

Package Integration

The root package.json now links claude-flow@v3alpha to the V3 CLI:

json
{
  "name": "claude-flow",
  "bin": {
    "claude-flow": "./v3/@claude-flow/cli/bin/cli.js"
  },
  "publishConfig": {
    "access": "public",
    "tag": "v3alpha"
  }
}

This means all V3 CLI commands (including daemon) are available via:

  • npx claude-flow@v3alpha daemon start
  • npx claude-flow@v3alpha daemon status
  • npx claude-flow@v3alpha hooks ...
  • etc.

Document Maintained By: Architecture Team Last Updated: 2026-01-07