packages/agent/docs/assistant-durability.md
This document specifies durable partial assistant messages for ordinary assistant generation and deferred-response polling. It builds on:
values.md;AssistantMessageFrame, AssistantMessageFrameEncoder, and reduceAssistantMessageFrames() from @earendil-works/pi-ai;harness.md.The design persists compact replayable stream frames without making them operation-state authority and without storing a growing full partial message on every update.
effect_pending state authoritative for recovery.done/error events separately from response settlement.partial snapshots.Built-in address constructor in session/values.ts:
export const pendingAssistantFrames = (
operationId: string,
responseEntryId: string,
) => list<AssistantMessageFrame>(
"pi.pending.assistant_frame",
`${operationId}:${responseEntryId}`,
);
The procedure binds one exact address for ordinary generation or a deferred poll:
const frames = pendingAssistantFrames(operationId, responseEntryId);
responseEntryId is already reserved in assistant/deferred effect_pending state. No frame count, cursor, or list identity is stored in operation state, and later list operations receive only frames—never another key.
Each list element is one AssistantMessageFrame. The storage transaction's global write sequence orders frames. The list is auxiliary:
Create one pi-ai encoder per provider stream and feed it every event in order:
const encoder = new AssistantMessageFrameEncoder();
const frame = encoder.encode(event);
partial is the provider's shared live response-so-far helper, not an event-time snapshot. The encoder keeps per-open-block counters and trims text/thinking delta prefixes already represented by an advanced block-start snapshot. It temporarily buffers only the raw JSON prefix needed to synchronize an already-advanced tool call, then emits one checkpoint and resumes compact deltas. It never clones the growing full message on every event.
start produces an empty-content metadata frame;done and error produce no frame because final response settlement is separate;error before start is valid and produces no frames;Do not define a second harness frame codec or reducer. Persistence stores the exported pi-ai value directly; hydration calls reduceAssistantMessageFrames().
Provider event blocks may interleave. Encoding and reduction rely on contentIndex, never block contiguity. Text and ordinary thinking blocks must be empty when *_start is published and then append only through matching deltas until end; redacted thinking may be complete at start and emit no deltas. Streaming tool calls start with empty arguments and emit their complete raw JSON through deltas; a provider that starts with complete arguments must emit a cumulative delta prefix that parses to that snapshot at an event boundary before later argument deltas.
The simplest scheduling is deliberate: one list-append transaction per converted frame, enqueued without awaiting storage in the provider loop.
For every start or update event:
encode the event against the per-stream frame encoder
→ when a frame is returned, synchronously enqueue invocation-fenced appendList(frames, frame)
→ attach the ordinary harness-fault observer to the returned promise
→ replace process-local latestFrameWrite promise reference
→ emit and await the existing message_start/message_update event
→ consume the next provider event
appendList() is called synchronously for each returned frame, so all frame mutations enter the Session line in provider-event order. The procedure does not await each write. Every promise is immediately observed for fault propagation before the latest reference replaces its predecessor. Covered queued events allocate no durable frame or write. Encoder state is proportional to open block count plus the unsynchronized prefix of an active tool-call JSON stream; it retains no second full assistant message. Model output limits bound queued frame bytes to bounded output plus frame/transaction overhead.
The event side retains existing behavior: AssistantStreamObserver.start/update awaits events.emit() for every event. There is no separate latest event-delivery promise because no assistant event delivery remains outstanding when the provider loop advances.
When the provider stream settles:
stop frame admission
→ await latestFrameWrite when present
→ run after_response
→ emit message_end
→ classify and commit final response settlement
The Session mutation line is FIFO, so completion of latestFrameWrite implies every earlier append completed. There is no array of promises, timer, batcher, active/waiting state, coalescer, or public/internal flush method.
A failed frame append faults the harness before after_response starts. The complete final response remains process-local and does not commit after a storage fault.
Every assistant response settlement that used a frame list deletes that exact list in the same transaction as the immutable response entry, usage, tip, and next operation state:
TX[
insert response entry R,
insert usage U,
setValue(branchTip(lane), R),
deleteList(frames),
setValue(operationState(operationId), classified next state)
]
This applies to:
error/aborted responses;after_response may transform the final response before settlement. Frames preserve provider-stream observation, while the immutable entry remains the post-hook canonical result.
A crash after the final frame append but before settlement still restores effect_pending; frames do not turn a complete-looking draft into a settled response.
An orphaned assistant generation effect_pending has no surviving provider stream. Activation:
frames = pendingAssistantFrames(operationId, responseEntryId) from current typed state and reads bounded pages from that exact address;reduceAssistantMessageFrames();stopReason: "error", zero usage, and an explicit interruption/unknown-outcome errorMessage/diagnostic;Required warning meaning:
The provider request was interrupted. The preceding content is the latest
committed partial response; newer live output may be missing, and the external
request outcome is unknown.
If no start frame committed, the harness constructs the same synthetic error with empty content from the captured model/API identity.
The synthetic response follows ordinary assistant error classification:
Error responses remain durable transcript history but are omitted from later provider context by the existing projection rule. Partial tool calls inside the interrupted error response never execute.
Recovery does not run after_response: there is no trustworthy complete provider result to transform.
A live cancelled provider stream settles through its ordinary final aborted response. All accepted frames are awaited first, and normal settlement deletes the list.
For restored cancelled assistant/deferred effect_pending, cancellation reconciliation:
aborted response preserving the committed partial content;Cancellation wins classification even if reduced content appears complete.
A deferred poll that returns an AssistantMessageEventStream uses the same pendingAssistantFrames(operationId, responseEntryId) address constructor.
Frames never change poll-number rules.
Structural summary-generation streams remain process-local. They do not emit public assistant-message lifecycle and may span multiple nested provider requests before one structural publication. Existing attempt-level retry and usage recovery remains authoritative.
Do not store their intermediate text at a pendingAssistantFrames(...) address in this slice. If structural partial diagnostics become a requirement, add a separate explicitly scoped consumer rather than silently reusing transcript-assistant semantics.
LaneSnapshot.streamingMessage means the latest observed partial assistant message, not proof that a provider stream is currently attached.
Precedence:
effect_pending, the value reduced from committed frame pages;A restored lane may therefore be suspended with a non-undefined streamingMessage. The field remains outside transcript; only entry_added moves a complete response into transcript history and clears the partial.
Snapshot hydration constructs the exact bound frame address from trusted typed operation state and enforces the assistant consumer's total frame/page budget. It does not scan arbitrary lists or perform a broad semantic restore audit.
Reconnect replays no historical message_start or message_update events. The snapshot carries the durable partial. Recovery later emits the ordinary recovery-tagged synthetic message lifecycle for the response it actually settles.
A started generation retains this live event ordering:
message_start
→ message_update* each listener delivery awaited by provider loop
→ await latest frame write
→ after_response
→ message_end
→ atomic response settlement + frame-list delete
→ entry_added
→ usage
A request setup failure may produce error before start; that path emits no message_start or frame and proceeds through after_response, message_end, and ordinary error settlement. Successful done and updates before start are protocol defects.
Frame commits emit ordinary storage telemetry only. There is no public frame event and no claim that a message_update was durable. entry_added remains the only proof that the final assistant entry committed.
A crash may occur after a live update event but before its asynchronously queued frame append commits. Reconnect then shows the latest committed frame prefix, which may be older than the last live event.
Close is a controlled crash:
effect_pending state.A frame-commit storage failure faults the harness. No later response settlement commits in that process.
Authorized external finalization deletes the operation-owned frame-list address in its terminal transaction. Every append mutation verifies current operation/response ownership on the Session line:
Normal/synthetic response settlement should already delete its exact frame address. The operation terminal transaction also defensively constructs and deletes the current operation-owned frame address when state is assistant/deferred effect_pending.
Idle forks never copy lists in the pi.pending.assistant_frame address family. Precise rewrites and migrations page frame lists and preserve element sequences when retaining them.
A migration changing AssistantMessageFrame shape must map every surviving element or explicitly delete the whole list and leave effect_pending recovery with no partial. It must never infer completion from legacy frames.
JSONL retains deleted frame bytes until snapshot compaction. Logical deletion is immediate.
| Race | Required result |
|---|---|
| frame append vs next frame | synchronous lane enqueue preserves provider-event order |
| frame append vs stream settlement | settlement awaits the latest promise; all accepted appends finish first |
| live update event vs frame commit | either may finish first; event is observation and reconnect uses only committed frames |
| append vs external finalization | append first is deleted by cleanup; finalization first fences the append |
| process loss with queued writes | only the committed prefix restores |
| final frame vs response settlement | frame commits first; settlement atomically deletes the list and inserts final entry |
| activation vs snapshot | both reduce the same committed sequence prefix; activation may then settle and clear it |
| unknown generation vs retry | synthetic partial error commits under old reserved IDs before later attempt starts |
| unknown deferred poll vs replacement | old list is deleted with fresh replacement intent; new response ID gets a new list |
AssistantMessageFrame.done/error events are never stored as frames.streamingMessage but never in transcript before settlement.error/aborted.pendingAssistantFrames(...) list in this slice.done/error append nothing, including pre-generation error;after_response and faults the harness.after_response never runs for restored synthetic settlement.streamingMessage on suspended effect-pending lane;entry_added;Expected runtime areas:
pendingAssistantFrames(operationId, responseEntryId) address constructor in session/values.ts;values.md;Implement bound typed value/list addresses first, then frame enqueue/settlement, then recovery/snapshots. Update harness.md with this complete lifecycle before implementing runtime assistant parity.