Worker Service

The worker service is a long-running HTTP API built with Express.js and managed natively by Bun. It processes observations through the Claude Agent SDK separately from hook execution to prevent timeout issues.

Overview

Technology: Express.js HTTP server
Runtime: Bun (auto-installed if missing)
Process Manager: Native Bun process management via ProcessManager
Port: Per-user default 37700 + (uid % 100) (override with CLAUDE_MEM_WORKER_PORT). The active port is stored in ~/.claude-mem/settings.json and reported by GET /api/health.
Location: src/services/worker-service.ts
Built Output: plugin/scripts/worker-service.cjs
Model: Configurable via CLAUDE_MEM_MODEL (default: claude-haiku-4-5-20251001)

REST API Endpoints

The worker service exposes 22 HTTP endpoints organized into six categories:

Viewer & Health Endpoints

1. Viewer UI

GET /

Purpose: Serves the web-based viewer UI (v5.1.0+)

Response: HTML page with embedded React application

Features:

Real-time memory stream visualization
Infinite scroll pagination
Project filtering
SSE-based live updates
Theme toggle (light/dark mode) as of v5.1.2

2. Health Check

GET /health

Purpose: Worker health status check

Response:

json

{
  "status": "ok",
  "uptime": 12345,
  "port": 37742
}

The port value is the actual worker port for the current user — per-user default 37700 + (uid % 100), or whatever CLAUDE_MEM_WORKER_PORT is set to. The example above is illustrative; your value will differ.

3. Server-Sent Events Stream

GET /stream

Purpose: Real-time updates for viewer UI

Response: SSE stream with events:

observation-created: New observation added
session-summary-created: New summary generated
user-prompt-created: New prompt recorded

Event Format:

event: observation-created
data: {"id": 123, "title": "...", ...}

Data Retrieval Endpoints

4. Get Prompts

GET /api/prompts?project=my-project&limit=20&offset=0

Purpose: Retrieve paginated user prompts

Query Parameters:

project (optional): Filter by project name
limit (default: 20): Number of results
offset (default: 0): Pagination offset

Response:

json

{
  "prompts": [{
    "id": 1,
    "session_id": "abc123",
    "prompt": "User's prompt text",
    "prompt_number": 1,
    "created_at": "2025-11-06T10:30:00Z"
  }],
  "total": 150,
  "hasMore": true
}

5. Get Observations

GET /api/observations?project=my-project&limit=20&offset=0

Purpose: Retrieve paginated observations

Query Parameters:

project (optional): Filter by project name
limit (default: 20): Number of results
offset (default: 0): Pagination offset

Response:

json

{
  "observations": [{
    "id": 123,
    "title": "Fix authentication bug",
    "type": "bugfix",
    "narrative": "...",
    "created_at": "2025-11-06T10:30:00Z"
  }],
  "total": 500,
  "hasMore": true
}

6. Get Summaries

GET /api/summaries?project=my-project&limit=20&offset=0

Purpose: Retrieve paginated session summaries

Query Parameters:

project (optional): Filter by project name
limit (default: 20): Number of results
offset (default: 0): Pagination offset

Response:

json

{
  "summaries": [{
    "id": 456,
    "session_id": "abc123",
    "request": "User's original request",
    "completed": "Work finished",
    "created_at": "2025-11-06T10:30:00Z"
  }],
  "total": 100,
  "hasMore": true
}

7. Get Observation by ID

GET /api/observation/:id

Purpose: Retrieve a single observation by its ID

Path Parameters:

id (required): Observation ID

Response:

json

{
  "id": 123,
  "sdk_session_id": "abc123",
  "project": "my-project",
  "type": "bugfix",
  "title": "Fix authentication bug",
  "narrative": "...",
  "created_at": "2025-11-06T10:30:00Z",
  "created_at_epoch": 1730886600000
}

Error Response (404):

json

{
  "error": "Observation #123 not found"
}

8. Get Observations by IDs (Batch)

POST /api/observations/batch

Purpose: Retrieve multiple observations by their IDs in a single request

Request Body:

json

{
  "ids": [123, 456, 789],
  "orderBy": "date_desc",
  "limit": 10,
  "project": "my-project"
}

Body Parameters:

ids (required): Array of observation IDs
orderBy (optional): Sort order - date_desc or date_asc (default: date_desc)
limit (optional): Maximum number of results to return
project (optional): Filter by project name

Response:

json

[
  {
    "id": 789,
    "sdk_session_id": "abc123",
    "project": "my-project",
    "type": "feature",
    "title": "Add new feature",
    "narrative": "...",
    "created_at": "2025-11-06T12:00:00Z",
    "created_at_epoch": 1730891400000
  },
  {
    "id": 456,
    "sdk_session_id": "abc124",
    "project": "my-project",
    "type": "bugfix",
    "title": "Fix authentication bug",
    "narrative": "...",
    "created_at": "2025-11-06T10:30:00Z",
    "created_at_epoch": 1730886600000
  }
]

Error Responses:

400 Bad Request: {"error": "ids must be an array of numbers"}
400 Bad Request: {"error": "All ids must be integers"}

Use Case: This endpoint is used by the get_observations MCP tool to efficiently retrieve multiple observations in a single request, avoiding the overhead of multiple individual requests.

9. Get Session by ID

GET /api/session/:id

Purpose: Retrieve a single session by its ID

Path Parameters:

id (required): Session ID

Response:

json

{
  "id": 456,
  "sdk_session_id": "abc123",
  "project": "my-project",
  "request": "User's original request",
  "completed": "Work finished",
  "created_at": "2025-11-06T10:30:00Z"
}

Error Response (404):

json

{
  "error": "Session #456 not found"
}

10. Get Prompt by ID

GET /api/prompt/:id

Purpose: Retrieve a single user prompt by its ID

Path Parameters:

id (required): Prompt ID

Response:

json

{
  "id": 1,
  "session_id": "abc123",
  "prompt": "User's prompt text",
  "prompt_number": 1,
  "created_at": "2025-11-06T10:30:00Z"
}

Error Response (404):

json

{
  "error": "Prompt #1 not found"
}

12. Get Stats

GET /api/stats

Purpose: Get database statistics by project

Response:

json

{
  "byProject": {
    "my-project": {
      "observations": 245,
      "summaries": 12,
      "prompts": 48
    },
    "other-project": {
      "observations": 156,
      "summaries": 8,
      "prompts": 32
    }
  },
  "total": {
    "observations": 401,
    "summaries": 20,
    "prompts": 80,
    "sessions": 20
  }
}

13. Get Projects

GET /api/projects

Purpose: Get list of distinct projects from observations

Response:

json

{
  "projects": ["my-project", "other-project", "test-project"]
}

Settings Endpoints

14. Get Settings

GET /api/settings

Purpose: Retrieve user settings

Response:

json

{
  "sidebarOpen": true,
  "selectedProject": "my-project",
  "theme": "dark"
}

15. Save Settings

POST /api/settings

Purpose: Persist user settings

Request Body:

json

{
  "sidebarOpen": false,
  "selectedProject": "other-project",
  "theme": "light"
}

Response:

json

{
  "success": true
}

Queue Management Endpoints

16. Get Pending Queue Status

GET /api/pending-queue

Purpose: View current processing queue status and identify stuck messages

Response:

json

{
  "queue": {
    "messages": [
      {
        "id": 123,
        "session_db_id": 45,
        "claude_session_id": "abc123",
        "message_type": "observation",
        "status": "pending",
        "retry_count": 0,
        "created_at_epoch": 1730886600000,
        "started_processing_at_epoch": null,
        "completed_at_epoch": null
      }
    ],
    "totalPending": 5,
    "totalProcessing": 2,
    "totalFailed": 0,
    "stuckCount": 1
  },
  "recentlyProcessed": [
    {
      "id": 122,
      "session_db_id": 44,
      "status": "processed",
      "completed_at_epoch": 1730886500000
    }
  ],
  "sessionsWithPendingWork": [44, 45, 46]
}

Status Definitions:

pending: Message queued, not yet processed
processing: Message currently being processed by SDK agent
processed: Message completed successfully
failed: Message failed after max retry attempts (3 by default)

Stuck Detection: Messages in processing status for >5 minutes are considered stuck and included in stuckCount

Use Case: Check queue health after worker crashes or restarts to identify unprocessed observations

17. Trigger Manual Recovery

POST /api/pending-queue/process

Purpose: Manually trigger processing of pending queues (replaces automatic recovery in v5.x+)

Request Body:

json

{
  "sessionLimit": 10
}

Body Parameters:

sessionLimit (optional): Maximum number of sessions to process (default: 10, max: 100)

Response:

json

{
  "success": true,
  "totalPendingSessions": 15,
  "sessionsStarted": 10,
  "sessionsSkipped": 2,
  "startedSessionIds": [44, 45, 46, 47, 48, 49, 50, 51, 52, 53]
}

Response Fields:

totalPendingSessions: Total sessions with pending messages in database
sessionsStarted: Number of sessions we started processing this request
sessionsSkipped: Sessions already actively processing (not restarted)
startedSessionIds: Database IDs of sessions started

Behavior:

Processes up to sessionLimit sessions with pending work
Skips sessions already actively processing (prevents duplicate agents)
Starts non-blocking SDK agents for each session
Returns immediately with status (processing continues in background)

Use Case: Manually recover stuck observations after worker crashes, or when automatic recovery was disabled

Recovery Strategy Note: As of v5.x, automatic recovery on worker startup is disabled by default. Users must manually trigger recovery using this endpoint or the CLI tool (bun scripts/check-pending-queue.ts) to maintain explicit control over reprocessing.

Session Management Endpoints

19. Initialize Session

POST /sessions/:sessionDbId/init

Request Body:

json

{
  "sdk_session_id": "abc-123",
  "project": "my-project"
}

Response:

json

{
  "success": true,
  "session_id": "abc-123"
}

20. Add Observation

POST /sessions/:sessionDbId/observations

Request Body:

json

{
  "tool_name": "Read",
  "tool_input": {...},
  "tool_result": "...",
  "correlation_id": "xyz-789"
}

Response:

json

{
  "success": true,
  "observation_id": 123
}

21. Generate Summary

POST /sessions/:sessionDbId/summarize

Request Body:

json

{
  "trigger": "stop"
}

Response:

json

{
  "success": true,
  "summary_id": 456
}

22. Session Status

GET /sessions/:sessionDbId/status

Response:

json

{
  "session_id": "abc-123",
  "status": "active",
  "observation_count": 42,
  "summary_count": 1
}

23. Delete Session

DELETE /sessions/:sessionDbId

Response:

json

{
  "success": true
}

Note: As of v4.1.0, the cleanup hook no longer calls this endpoint. Sessions are marked complete instead of deleted to allow graceful worker shutdown.

Bun Process Management

Overview

The worker is managed by the native ProcessManager class which handles:

Process spawning with Bun runtime
PID file tracking at ~/.claude-mem/worker.pid
Health checks with automatic retry
Graceful shutdown with SIGTERM/SIGKILL fallback

Commands

bash

# Start worker (auto-starts on first session)
npm run worker:start

# Stop worker
npm run worker:stop

# Restart worker
npm run worker:restart

# View logs
npm run worker:logs

# Check status
npm run worker:status

Auto-Start Behavior

The worker service auto-starts when the SessionStart hook fires. Manual start is optional.

Bun Requirement

Bun is required to run the worker service. If Bun is not installed, npx claude-mem install (and npx claude-mem repair) installs it globally during setup, with a visible clack spinner:

Windows: powershell -c "irm bun.sh/install.ps1 | iex"
macOS/Linux: curl -fsSL https://bun.sh/install | bash

You can also install manually via:

winget install Oven-sh.Bun (Windows)
brew install oven-sh/bun/bun (macOS)

Claude Agent SDK Integration

The worker service routes observations to the Claude Agent SDK for AI-powered processing:

Processing Flow

Observation Queue: Observations accumulate in memory
SDK Processing: Observations sent to Claude via Agent SDK
XML Parsing: Responses parsed for structured data
Database Storage: Processed observations stored in SQLite

SDK Components

Prompts (src/sdk/prompts.ts): Builds XML-structured prompts
Parser (src/sdk/parser.ts): Parses Claude's XML responses
Worker (src/sdk/worker.ts): Main SDK agent loop

Model Configuration

Set the Claude model used for compression via environment variable or ~/.claude-mem/settings.json:

bash

export CLAUDE_MEM_MODEL=claude-haiku-4-5-20251001

Allowed values:

claude-haiku-4-5-20251001 - default, fast and cheap (best for compression)
claude-sonnet-4-6 - balanced quality and cost
claude-opus-4-7 - highest quality, most expensive

Port Allocation

The worker uses a per-user default port so different OS users on the same machine never collide:

Default: 37700 + (uid % 100) (set in src/shared/SettingsDefaultsManager.ts)
Override: Set CLAUDE_MEM_WORKER_PORT (env or ~/.claude-mem/settings.json)
Discovery: GET /api/health returns the active port; ~/.claude-mem/settings.json stores the configured value

If the chosen port is occupied, the worker fails to start — pin a different port via CLAUDE_MEM_WORKER_PORT and restart.

Data Storage

The worker service stores data in the user data directory:

~/.claude-mem/
├── claude-mem.db           # SQLite database (bun:sqlite)
├── worker.pid              # PID file for process tracking
├── settings.json           # User settings
└── logs/
    └── worker-YYYY-MM-DD.log  # Daily rotating logs

Error Handling

The worker implements graceful degradation:

Database Errors: Logged but don't crash the service
SDK Errors: Retried with exponential backoff
Network Errors: Logged and skipped
Invalid Input: Validated and rejected with error response

Performance

Async Processing: Observations processed asynchronously
In-Memory Queue: Fast observation accumulation
Batch Processing: Multiple observations processed together
Connection Pooling: SQLite connections reused

Troubleshooting

See Troubleshooting - Worker Issues for common problems and solutions.