plans/2026-07-17-endless-mode-v1.md
Date: 2026-07-17
Status: Ready to build
Background research (optional reading): plans/2026-07-16-endless-mode-message-in-a-bottle.md — codebase investigation notes with file:line references. This document stands alone; read that one only if you want the receipts.
When a Claude Code session runs long enough, the harness performs compaction: it summarizes the conversation into a few paragraphs, throws the original context away, and continues with the summary as the model's only memory of everything before. The session survives; its memory doesn't. After a compact, the model loses its plan, its exact wording, its knowledge of what was already tried — users experience it as the assistant getting dumber mid-session.
The summary is generated by Claude Code and cannot be replaced or suppressed by a plugin. But a plugin can inject additional context at exactly the right moment, and claude-mem already stores everything needed to reconstruct the session far more faithfully than any summary.
Compaction only destroys the context window. Two durable records of the session sit on disk, untouched:
~/.claude/projects/<dashed-project-path>/<sessionId>.jsonl. Compaction does not truncate or delete it.So nothing needs to happen before a compact. There is no state to rescue. When the fresh context starts, we rebuild the session record from disk and hand the model a pointer to it.
Endless Mode: at the start of every post-compact (or resumed) context, claude-mem writes a markdown file — the bottle — reconstructing the session, and injects a short instruction: read this file and continue.
The bottle is:
bottle = merge( transcript.jsonl , claude-mem observations )
Result: a session record roughly 10–20× smaller than the real context, with the conversational spine preserved word-for-word. The model continues from its own last words instead of from a paraphrase.
The bottle is derived, never maintained: a pure function of the two sources, regenerated whole on demand, written atomically. The file on disk is a cache, not a source of truth. No hook appends to it; nothing can drift.
Claude Code hooks. Plugins register handlers for lifecycle events. Each hook is a subprocess that receives a JSON payload on stdin and may emit a JSON response on stdout. Relevant here:
SessionStart — fires when a context begins, with source = "startup" (fresh session), "compact" (right after compaction, same session_id), "resume" (user reopened a session), or "clear" (user ran /clear). Payload includes session_id, transcript_path, cwd. A hook injects context by printing:
{"hookSpecificOutput": {"hookEventName": "SessionStart", "additionalContext": "<text>"}}Stop — fires at the end of each assistant turn. claude-mem's existing Stop handler (src/cli/handlers/summarize.ts) already reads the full transcript on every Stop, so full-transcript reads per turn are the established cost baseline.src/cli/hook-command.ts).claude-mem storage (SQLite, ~/.claude-mem/claude-mem.db):
sdk_sessions — one row per Claude Code session, keyed by content_session_id (Claude Code's session id — stable across compaction). Holds the current memory_session_id; the observations and session_summaries foreign keys cascade on update, so one join reaches everything regardless of worker restarts:
content_session_id → sdk_sessions → memory_session_id → observations + session_summariesobservations — typed records (title, narrative, facts, files, concepts) generated asynchronously from tool activity by the worker's SDK agent. Each carries prompt_number — which user prompt it happened under — and a timestamp already anchored to tool time.user_prompts — every user prompt, verbatim (capped at 4,000 chars, ~3.5% of prompts hit the cap), keyed by content_session_id with prompt_number.session_summaries — LLM-generated per-turn summaries (request/investigated/learned/completed/next_steps/notes). The only assistant-perspective record that outlives the transcript — but generated prose, not the assistant's words.The worker is a local Express daemon owning the DB and the SDK agent. Routes live in src/services/worker/http/routes/, extend BaseRouteHandler, take services by constructor injection, and are registered in worker-service.ts. Long work follows a queue-and-return shape: respond immediately, do the work async, log failures as non-blocking.
Privacy contract. Every piece of text claude-mem persists passes through stripMemoryTags (src/utils/tag-stripping.ts), which removes <private>-tagged content and host-injected noise. Any new path that writes conversation text to disk must do the same.
Transcript retention. Claude Code prunes old transcripts (default ~30 days). On a typical long-lived install, most stored sessions no longer have a transcript — but their prompts, observations, and summaries survive in SQLite. The renderer must handle both cases (§6).
Four pieces. One is optional.
BottleRenderer (new worker service)Input: contentSessionId, optional transcriptPath. Output: the bottle file, written atomically (write temp sibling, rename() over target — precedent in src/shared/atomic-json.ts).
~/.claude-mem/bottles/<contentSessionId>.md; archived to ~/.claude-mem/bottles/archive/<ts>-<id>.md on /clear.tool_use, tool_result, thinking). From user entries keep genuine messages only (skip tool results and injected envelopes like ``, <task-notification>, command wrappers). Skip a torn final line — the file is being appended to live.sdk_sessions join. Position observations by prompt_number (group under the user prompt they belong to), ordered by created_at_epoch within a group.stripMemoryTags to every rendered message. Hard requirement — the verbatim render is the one path that would otherwise bypass the privacy contract.Degraded mode — transcript missing. Detect with existsSync(transcriptPath); when no path was provided, try the convention glob CLAUDE_CONFIG_DIR/projects/*/<contentSessionId>.jsonl (the convention is already codified in src/services/context/ObservationCompiler.ts). If the transcript is gone, render from SQLite alone:
user_prompts → verbatim user messages. Prompts that hit the 4,000-char cap get an explicit [truncated at 4000 chars] marker.observations → same one-line entries, positioned by prompt_number.session_summaries → rendered as a visibly generated block:
> Session summary (generated by claude-mem — not verbatim)
Never rendered under an **Assistant** heading. Summary prose reads like assistant voice; put it in the assistant's mouth and the resumed model will "remember" saying things it never said.mode: reconstructed — assistant messages not preserved.If the session has neither a transcript nor any stored prompts/observations, the renderer reports failure and the hook falls back to today's behavior (§5.3).
POST /api/sessions/render-bottle (new worker route)Body: { contentSessionId, transcript_path?, wait? }.
{status:"queued"} immediately, render async, .catch() → non-blocking error log.wait: true: render synchronously, respond { bottlePath, mode }. SessionStart uses this — the file must exist before the pointer is injected.sourcecompact / resume: call render-bottle with wait: true (~10s timeout). On success, inject the pointer block (§7). On any failure — worker down, render error, nothing to render — inject the existing timeline context exactly as today. Endless Mode can never leave the user worse off than current behavior.clear: archive the bottle, inject nothing. /clear means the user asked to forget.startup: unchanged.No bookkeeping table, no consumed-once state: the injection is a pointer to an idempotent file. If SessionStart double-fires (it can), the model gets the same 20 lines twice — harmless.
In the Stop handler (summarize.ts), after the existing summarize call:
try {
await executeWithWorkerFallback('/api/sessions/render-bottle', 'POST',
{ contentSessionId, transcript_path }, { timeoutMs: 5000 });
} catch { /* never escalate */ }
Await it — hook processes call process.exit, which kills un-awaited requests mid-flight (documented in hook-command.ts). This keeps a live, human-readable session diary on disk at all times. Nice to have; correctness does not depend on it.
# Session bottle — <contentSessionId>
project: <cwd> · started: <ts> · rendered: <ts>
mode: full
## Original request
<first genuine user prompt, verbatim>
---
**User** (10:32)
<verbatim>
*What happened:*
- [#102401] Explored hooks/ — every platform hook wraps 5 shared cores
- [#102402] Read start.mjs — 6 self-heal layers before MCP boot
**Assistant** (10:41)
<verbatim>
---
[#id] title — first clause of narrative. The id makes any line expandable through claude-mem's search tools; never inline full narratives.Full mode:
# [claude-mem] Endless Mode — session continuation
Before doing anything else, Read this file and continue the session from
where it ends:
<absolute bottle path>
- It is the authoritative session record: verbatim conversation, with
observations in place of tool activity. Where it conflicts with the
summary above, the bottle wins.
- The final assistant message in it was already delivered to the user.
Do not repeat it or redo work it describes as done — continue from its
end state.
- The newest tool activity may still be settling into observations;
check the timeline if the last few minutes look thin.
Current task: <one line, from the latest session summary or last user prompt>
Degraded mode swaps the middle bullets:
- It is a partial reconstruction: only the user's messages are verbatim.
Your own replies were not preserved — do not assume prior phrasings.
Session summary blocks are generated, not your words.
- The last session summary describes where you left off; continue from there.
Notes that matter:
Read, which lands in context as a tool result — exactly the category the next compaction evicts first. The bottle never accumulates across compactions; each new context re-reads fresh.| Not building | Why |
|---|---|
| A PreCompact hook, or any "capture before the wipe" step | The sources (transcript, DB) survive compaction; there is nothing to rescue. Render lazily at SessionStart. |
| Waiting for the observation queue to drain before rendering | Observations are generated asynchronously and the freshest tool events may not have them yet — that's fine. The verbatim final assistant message covers the tail narrative, the injection carries a one-line disclaimer, and any later render includes them. Blocking on an LLM-backed queue buys almost nothing. |
| A bottle-state table / consumed-once injection markers | The file is the state; the injection is an idempotent pointer. |
| Incremental transcript parsing (byte cursors) + render mutex | The Stop hook already reads the full transcript every turn today. Full re-parse per render matches the baseline; renders are pure functions of the sources, so concurrent renders are last-writer-wins of equally valid content. |
New schema columns (tool-call anchors, tool_use_id threading) | prompt_number — already on both user_prompts and observations — positions everything exactly. Revisit with the next schema bump if per-tool-call placement is ever wanted. |
<private> content in user and assistant messages → absent from the bottle on every render path.mode: reconstructed header, prompt_number ordering, summaries as quoted generated blocks (never under **Assistant**), truncation markers present./clear: bottle archived, nothing injected.npx claude-mem bottle <contentSessionId> — render any historical session and resume it in a fresh context (the degraded path already makes this possible for sessions whose transcripts are long gone).tool_use_id threading, anchor columns).