Back to Claude Mem

TODO (next): make the observer IGNORE output it can't parse — kill the "poisoned" respawn heuristic

plans/2026-06-20-todo-ignore-unparseable-observer-output.md

13.8.06.2 KB
Original Source

TODO (next): make the observer IGNORE output it can't parse — kill the "poisoned" respawn heuristic

Status: Not started. Spec for a follow-up cleanup pass. Owner principle (from @thedotmack): The observer parser should IGNORE any output it can't parse. That is the whole job. It must NOT inspect the content of unparseable output and act on it.


The problem

When the observer SDK returns something that isn't parseable <observation>/<summary> XML, the code is supposed to drop it. Instead, a substring classifier labels some of it poisoned and kills + respawns the SDK session. "Poisoned" is decided by a case-insensitive includes() against 11 hardcoded English phrases (session exhausted, context window, prompt is too long, this session has ended, …) — see src/sdk/output-classifier.ts:20-32.

This is a brittle keyword match standing in for "ignore what you can't parse," and it over-fires badly.

Evidence it's broken (live PostHog, 30d, project CMEM)

poisoned aborts as a share of all compression turns, by model:

Modelpoisoned % of turnscontext
haiku (4.5)41.1%200K
claude-haiku-4-5-2025100133.9% (9.0M turns)200K
claude-sonnet-4-517.7%200K–1M
claude-sonnet-4-69.6%200K–1M
claude-opus-4-75.9%200K
gemini-2.5-flash-lite0.1%1M
openai/gpt-oss-120b:free0.1%
gemini-3-flash-preview / xiaomi mimo-flash:free~0%small

Total ≈ 13.2M poisoned aborts / 30d ≈ 22% of all turns.

This is the inverse of real context exhaustion. If "poisoned" meant running out of room, the small-context models would top the list. Instead they're ~0% and the large-context Claude models are 6–41%. The classifier is matching Claude's closure-phrase wording (e.g. "I cannot continue this session"), not session health. Net effect: claude-mem kills+respawns ~1/3 of Haiku sessions on a keyword false positive, throwing away work and re-spending tokens.

(Note: the word "poisoned" is also just wrong — it implies tainted/corrupted input, not a wedged session. The whole concept should go.)


The fix — "parse XML or drop it," nothing content-based

Where it lives

  • src/services/worker/agents/ResponseProcessor.ts
    • :56-120 — the !parsed.valid branch. The correct behavior already exists at :115-120 ("Plain-text skip responses are intentionally ignored" → confirmClaimedMessages drops the batch, returns).
    • :75-77 — the offending trigger:
      js
      const mustRespawn =
        outputClass === 'poisoned' ||                                  // ← remove
        session.consecutiveInvalidOutputs >= INVALID_OUTPUT_RESPAWN_THRESHOLD; // ← see decision
      
    • :79-113 — the respawn + telemetry block. :26INVALID_OUTPUT_RESPAWN_THRESHOLD = 3.
  • src/sdk/output-classifier.tsPOISONED_MARKERS (:20-32), poisoned in the ObserverOutputClass union (:13), the precedence check (:67-73).
  • src/services/worker/SessionManager.tsrespawnPoisonedSession (:252), session.abortReason = 'poisoned' (:273).
  • src/services/worker/http/routes/SessionRoutes.ts:37,44abort_reason enum mapping incl. poisoned.
  • src/services/telemetry/scrub.ts:90-99 — enum doc comments + whitelisted keys invalid_output_class, abort_reason, respawn_triggered, consecutive_invalid_outputs.
  • src/npx-cli/commands/telemetry.ts:71,74 — field docs listing poisoned.
  • Tests: output-classifier tests + any ResponseProcessor respawn tests.
  1. output-classifier.ts — delete POISONED_MARKERS and the poisoned class. Classifier collapses to xml (parseable) vs not. Keep previewOutput() for log visibility only — it must never drive behavior.
  2. ResponseProcessor.ts — remove the whole mustRespawn/respawn block (:71-113). Unparseable output always falls through to the existing drop path (:115-120). Keep the warn-log + previewOutput so drops stay visible.
  3. SessionManager.respawnPoisonedSession — remove if it becomes dead code (confirm no other callers; respawnPoisonedSession is referenced from the Phase-2 buffer-flush guards — do NOT flush a rollup for it; just delete the call path cleanly).
  4. scrub.ts / SessionRoutes.ts / CLI docs — drop the poisoned enum value and, if respawn goes entirely, the now-unused respawn_triggered / invalid_output_class / abort_reason keys (or keep them whitelisted but unused — safer to keep keys, remove only the poisoned value from docs).
  5. Tests — delete poison-classification + immediate-respawn tests; add a test asserting unparseable output is dropped (batch confirmed, no respawn) regardless of content.

Why pure-ignore is loop-safe: the drop path calls confirmClaimedMessages — the batch is NOT re-queued, so a genuinely-stuck session can't loop "until quota exhausted." The 3-strikes respawn was guarding a loop that the drop path already prevents.

If a safety net is still wanted (fallback option)

Keep only the content-agnostic consecutiveInvalidOutputs >= 3 structural respawn; remove just the outputClass === 'poisoned' || clause and the POISONED_MARKERS list. This kills the 33–41% false positives but retains a structural circuit-breaker. (Owner leans toward pure-ignore above.)


Verification

  • bun test (output-classifier + ResponseProcessor + telemetry suites) green; tsc --noEmit clean.
  • Grep guard: grep -rin "poison" src returns nothing behavioral (only history/comments if intentionally kept).
  • Manual: feed the observer non-XML output containing the string "context window" → asserted ignored/dropped, session NOT respawned.
  • Post-ship PostHog: invalid_output_class='poisoned' / abort_reason='poisoned' volume goes to zero on updated installs; Haiku turn throughput should rise (fewer needless respawns).

Dashboard follow-up (separate, non-blocking)

The "Session abort reasons" and "Compression invalid-output classes" tiles on dashboard 1739781 will lose the poisoned slice as installs update — expected. No tile change needed; the decay itself is the confirmation signal.