.agents/skills/agent-tracing/SKILL.md
@lobechat/agent-tracing is a zero-config local dev tool that records agent execution snapshots to disk and provides a CLI to inspect them.
In NODE_ENV=development, AgentRuntimeService.executeStep() automatically records each step to .agent-tracing/ as partial snapshots. When the operation completes, the partial is finalized into a complete ExecutionSnapshot JSON file.
Data flow: executeStep loop -> build StepPresentationData -> write partial snapshot to disk -> on completion, finalize to .agent-tracing/{timestamp}_{traceId}.json
Context engine capture: In RuntimeExecutors.ts, the call_llm executor calls ctx.tracingContextEngine(input, output) after serverMessagesEngine() processes messages. AgentRuntimeService.executeStep buffers the call per step and forwards it to OperationTraceRecorder.appendStep as the typed contextEngine field. CE flows through this side channel rather than the events array so its heavy payload (agentDocuments, systemRole, …) never enters the Redis state pipeline (LOBE-9110).
packages/agent-tracing/
src/
types.ts # ExecutionSnapshot, StepSnapshot, SnapshotSummary
store/
types.ts # ISnapshotStore interface
file-store.ts # FileSnapshotStore (.agent-tracing/*.json)
recorder/
index.ts # appendStepToPartial(), finalizeSnapshot()
viewer/
index.ts # Terminal rendering: renderSnapshot, renderStepDetail, renderMessageDetail, renderSummaryTable, renderPayload, renderPayloadTools, renderMemory
cli/
index.ts # CLI entry point (#!/usr/bin/env bun)
inspect.ts # Inspect command (default)
partial.ts # Partial snapshot commands (list, inspect, clean)
index.ts # Barrel exports
.agent-tracing/{ISO-timestamp}_{traceId-short}.json.agent-tracing/latest.json.agent-tracing/_partial/{operationId}.jsonFileSnapshotStore resolves from process.cwd() — run CLI from the repo rootAll commands run from the repo root:
# View latest trace (tree overview, `inspect` is the default command)
agent-tracing
agent-tracing inspect
agent-tracing inspect <traceId>
agent-tracing inspect latest
# List recent snapshots
agent-tracing list
agent-tracing list -l 20
# Inspect specific step (-s is short for --step)
agent-tracing inspect <traceId> -s 0
# View messages (-m is short for --messages)
agent-tracing inspect <traceId> -s 0 -m
# View full content of a specific message (by index shown in -m output)
agent-tracing inspect <traceId> -s 0 --msg 2
agent-tracing inspect <traceId> -s 0 --msg-input 1
# View tool call/result details (-t is short for --tools)
agent-tracing inspect <traceId> -s 1 -t
# View raw events (-e is short for --events)
agent-tracing inspect <traceId> -s 0 -e
# View runtime context (-c is short for --context)
agent-tracing inspect <traceId> -s 0 -c
# View context engine input overview (-p is short for --payload)
agent-tracing inspect <traceId> -p
agent-tracing inspect <traceId> -s 0 -p
# View available tools in payload (-T is short for --payload-tools)
agent-tracing inspect <traceId> -T
agent-tracing inspect <traceId> -s 0 -T
# View user memory (-M is short for --memory)
agent-tracing inspect <traceId> -M
agent-tracing inspect <traceId> -s 0 -M
# Raw JSON output (-j is short for --json)
agent-tracing inspect <traceId> -j
agent-tracing inspect <traceId> -s 0 -j
# List in-progress partial snapshots
agent-tracing partial list
# Inspect a partial (use `inspect` directly — all flags work with partial IDs)
agent-tracing inspect <partialOperationId>
agent-tracing inspect <partialOperationId> -T
agent-tracing inspect <partialOperationId> -p
# Clean up stale partial snapshots
agent-tracing partial clean
| Flag | Short | Description | Default Step |
|---|---|---|---|
--step <n> | -s | Target a specific step | — |
--messages | -m | Messages context (CE input → params → LLM payload) | — |
--tools | -t | Tool calls & results (what agent invoked) | — |
--events | -e | Raw events (llm_start, llm_result, etc.) | — |
--context | -c | Runtime context & payload (raw) | — |
--system-role | -r | Full system role content | 0 |
--env | Environment context | 0 | |
--payload | -p | Context engine input overview (model, knowledge, tools summary, memory summary, platform context) | 0 |
--payload-tools | -T | Available tools detail (plugin manifests + LLM function definitions) | 0 |
--memory | -M | Full user memory (persona, identity, contexts, preferences, experiences) | 0 |
--diff <n> | -d | Diff against step N (use with -r or --env) | — |
--msg <n> | Full content of message N from Final LLM Payload | — | |
--msg-input <n> | Full content of message N from Context Engine Input | — | |
--json | -j | Output as JSON (combinable with any flag above) | — |
Flags marked "Default Step: 0" auto-select step 0 if --step is not provided. All flags support latest or omitted traceId.
# 1. Trigger an agent operation in the dev UI
# 2. See the overview
agent-tracing inspect
# 3. List all traces, get traceId
agent-tracing list
# 4. Quick overview of what was fed into context engine
agent-tracing inspect -p
# 5. Inspect a specific step's messages to see what was sent to the LLM
agent-tracing inspect TRACE_ID -s 0 -m
# 6. Drill into a truncated message for full content
agent-tracing inspect TRACE_ID -s 0 --msg 2
# 7. Check available tools vs actual tool calls
agent-tracing inspect -T # available tools
agent-tracing inspect -s 1 -t # actual tool calls & results
# 8. Inspect user memory injected into the conversation
agent-tracing inspect -M
# 9. Diff system role between steps (multi-step agents)
agent-tracing inspect TRACE_ID -r -d 2
interface ExecutionSnapshot {
traceId: string;
operationId: string;
model?: string;
provider?: string;
startedAt: number;
completedAt?: number;
completionReason?:
| 'done'
| 'error'
| 'interrupted'
| 'max_steps'
| 'cost_limit'
| 'waiting_for_human';
totalSteps: number;
totalTokens: number;
totalCost: number;
error?: { type: string; message: string };
steps: StepSnapshot[];
}
interface StepSnapshot {
stepIndex: number;
stepType: 'call_llm' | 'call_tool';
executionTimeMs: number;
content?: string; // LLM output
reasoning?: string; // Reasoning/thinking
inputTokens?: number;
outputTokens?: number;
toolsCalling?: Array<{ apiName: string; identifier: string; arguments?: string }>;
toolsResult?: Array<{
apiName: string;
identifier: string;
isSuccess?: boolean;
output?: string;
}>;
messages?: any[]; // DB messages before step
context?: { phase: string; payload?: unknown; stepContext?: unknown };
events?: Array<{ type: string; [key: string]: unknown }>;
contextEngine?: {
input?: unknown; // contextEngineInput minus messages + toolsConfig (reconstructible from baseline)
output?: unknown; // processed messages array (final LLM payload)
};
}
When using --messages, the output shows three sections (if context engine data is available):
[0], [1], ... indices. Use --msg-input N to view full content.[0], [1], ... indices. Use --msg N to view full content.src/server/services/agentRuntime/AgentRuntimeService.ts — in the executeStep() method, after building stepPresentationData, writes partial snapshot in dev modesrc/server/modules/AgentRuntime/RuntimeExecutors.ts — in call_llm executor, after serverMessagesEngine() returns, calls ctx.tracingContextEngine(input, output). AgentRuntimeService.executeStep buffers it per step and passes it to traceRecorder.appendStep as the typed contextEngine field (kept off the events array to stay out of Redis state).FileSnapshotStore reads/writes to .agent-tracing/ relative to process.cwd()