Qwen Code Runtime Diagnostics Benchmark Report

Date: 2026-05-19

Scope

This run repeats the previous Qwen Code benchmark shapes with the new opt-in runtime diagnostics enabled. It only tests Qwen Code, not Claude Code.

Initial model matrix:

• pai/glm-5
• qwen3.6-plus

Additional PR-size follow-up:

• DeepSeek/deepseek-v4-pro through Anthropic-compatible protocol

Cases:

• small GitHub PR review: PR #4268
• code navigation: compression / compaction related code search and reads
• synthetic local diff: about 94.6 KiB
• synthetic local diff: about 968.5 KiB
• synthetic local diff: about 4.84 MiB

The run used the local bundled CLI from the diagnostics branch, with QWEN_CODE_PROFILE_RUNTIME=1 and a temporary CLI home. Global MCP servers and hooks were not loaded for this benchmark.

Important caveat: these absolute RSS numbers are lower than the previous PATH-resolved qwen runs because this run used node dist/cli.js from the local branch plus a stripped temporary config. Treat this report as an internal diagnostics distribution run, not a direct replacement for the earlier installed CLI RSS comparison.

Installed CLI vs Local Bundle Sanity Check

A follow-up sanity check used the same minimal prompt, model, and non-interactive mode across the installed CLI and the local diagnostics bundle. The only intentional variable was whether Qwen Code loaded a stripped temporary CLI home or the normal user config.

This check changes the attribution: the earlier 1 GiB user-visible peak is reproducible with the normal config even on the local diagnostics bundle. It is therefore not primarily explained by the local branch including PR #4186.

At the normal-config peak, the local process-tree sample was dominated by multiple Node/MCP processes rather than the Qwen root process alone:

PR #4186 is present in the local diagnostics branch, but it is a V8 heap pressure auto-compaction safety net. It triggers at about 70% V8 heap pressure; on this environment the Node heap limit is about 4.1 GiB, while the stripped benchmark end heap was about 99-143 MiB. Based on these numbers, the lower stripped-config RSS is not caused by #4186 actively compressing context during these benchmark runs.

Bare Mode Config Attribution Check

A second follow-up used qwen3.6-plus with the same PR-review prompt shape on both the installed CLI and the local bundle. This is not a normal end-to-end business benchmark. It is a controlled attribution check for startup/config memory only.

--bare changes the runtime inputs: it skips normal global settings discovery, MCP startup, hooks, implicit context, skills, and other startup integrations. It can therefore fail or behave differently when a model provider is configured only in global settings. For this run, model credentials were supplied only through the child-process environment because bare mode intentionally does not load the normal provider settings. Nothing was written back to the user's global config.

This run did not produce useful token/tool-call statistics: the model completed in one turn and did not call the requested shell command. Do not use these rows as normal task benchmark results, and do not compare their token/tool-call behavior with the matrix above. They are only useful for estimating how much process-tree RSS comes from normal config and configured child processes.

The result confirms the process-tree hypothesis for startup/config attribution. On this machine, normal config adds roughly 0.50-0.59 GiB of user-visible process-tree RSS over --bare, while root RSS stays in the same 0.21-0.25 GiB band. At the normal-config peak, the extra RSS again came from additional Node/MCP child processes, including a Chrome DevTools MCP process and its setup wrapper. --bare removes those startup/config children and brings installed/local runs back into the 0.45-0.53 GiB tree-RSS range.

Temporary Settings MCP / Hooks Isolation

Because --bare changes too many runtime inputs to be treated as a normal benchmark, a follow-up used temporary QWEN_HOME directories with generated settings files derived from the normal settings. The run stayed on the normal settings-loading path, but toggled only two config dimensions:

• MCP disabled: mcpServers cleared and MCP allow/exclude lists emptied.
• Hooks disabled: disableAllHooks set to true.

No global settings were modified. The case used qwen3.6-plus and a minimal startup prompt, so it measures startup/config process-tree cost, not task reasoning quality.

Interpretation:

1. Disabling MCP is the dominant change. It removes 4 MCP servers, reduces the advertised tool count by about 29 tools, and lowers process-tree RSS by about 0.42-0.47 GiB in this startup/config case.
2. Disabling hooks alone barely changes RSS in this case. That is expected because the prompt did not produce tool calls, so PreToolUse / PostToolUse hooks were not executed.
3. The root process stays around 0.21-0.25 GiB across all rows. The large difference is again process-tree composition, not root Qwen RSS.

Two attempted code-navigation follow-ups with qwen3.6-plus and pai/glm-5 also reproduced the same MCP-vs-no-MCP memory split, but neither model produced tool calls in those runs. Those rows are therefore not used as hooks execution evidence. A valid hooks benchmark still needs a task/model combination that reliably emits tool calls.

Per-MCP Isolation

The previous row showed MCP as a group is the dominant startup/config memory factor. A follow-up isolated each configured MCP server while keeping hooks disabled for all rows. This keeps the test on the normal settings-loading path but changes only the MCP server subset.

Configured MCP server names:

• approval-bridge
• env-center
• chrome-devtools
• code

Single-pass isolation:

Because startup RSS has some variance, the key variants were repeated twice:

Interpretation:

1. chrome-devtools is the dominant MCP contributor in this environment. It is sufficient by itself to reproduce the high process-tree RSS.
2. Removing chrome-devtools from the full MCP set returns RSS to the no-MCP band. Removing other MCPs while keeping chrome-devtools does not.
3. The advertised tool count follows the same pattern: baseline is 19 tools, while chrome-devtools raises the tool count to 48. That means this MCP is also likely to increase request tool schema size and token pressure, not just process-tree RSS.
4. approval-bridge, env-center, and code individually stay near the no-MCP baseline in these startup/config runs. They emitted startup warnings in this environment, so this result should be interpreted as "no persistent startup RSS owner observed" rather than proof that they have zero cost in all workflows.

Runtime Summary

Average by model:

Overlapping small PR #4268 model snapshot:

Request And Tool Diagnostics

Observations

1. Process-tree RSS is almost the same as root RSS in this local bundle run. The root/tree gap is usually below 10 MiB. That means these runs did not show a persistent child-process memory owner. The dominant process is the main Node process.
2. The local bundle run peaks around 0.36-0.41 GiB, not the earlier 0.83-1.04 GiB, because the matrix used a stripped temporary config. A follow-up normal-config sanity check reproduced about 1.1 GiB tree RSS on both PATH qwen and local dist/cli.js, with the extra memory coming from child MCP/Node processes in the process tree.
3. V8 heap is much smaller than RSS. End heap is about 99-143 MiB while end RSS is about 216-411 MiB. The remaining footprint is likely loaded modules, native allocations, external buffers, or runtime overhead outside live JS heap.
4. Static request overhead is large and repeated. The system prompt is about 39-51 KiB per request, and tool schema is about 37-40 KiB per request. This explains why even small tasks can produce high accumulated token counts when the model takes several turns.
5. Large diff output is capped before it reaches the model request. The 968 KiB and 4.84 MiB diff cases produced around 772-799 KiB of captured tool result, but the largest model-facing function response in a request stayed around 25-41 KiB, and max wire body stayed around 105-122 KiB. This points to truncation / saved-output handling working on the model-facing path.
6. Memory still increases on large-output cases even though wire body remains bounded. For example, the 4.84 MiB GLM run reached 414.2 MiB tree RSS and 410.7 MiB end RSS, and the 4.84 MiB qwen3.6-plus run ended with 142.8 MiB heap. That suggests large tool output can still affect local capture, normalization, or retained runtime state even when the final request payload is capped.
7. Model choice changed turns and token totals more than RSS in this run. qwen3.6-plus averaged more tokens and turns than pai/glm-5, but its average tree RSS peak was slightly lower. This supports the earlier conclusion that model choice is not the main explanation for process memory.

Updated Working Inference

The new diagnostics make the earlier hypothesis more precise:

• The installed-CLI user-visible 1 GiB peak is now reproducible with the normal config on the local diagnostics bundle. The stripped run should be used for internal Qwen runtime attribution; the normal-config run should be used for user-visible process-tree attribution.
• The largest observed difference between stripped and normal config is process-tree shape: normal config starts additional MCP/Node child processes. Those children explain most of the absolute jump from about 0.35-0.55 GiB to about 1.1 GiB in the minimal prompt sanity check.
• The --bare follow-up confirms the same direction on qwen3.6-plus: normal config costs about 0.50-0.59 GiB more process-tree RSS than bare mode for the same prompt shape, while root RSS changes only slightly.
• The temporary-settings isolation is a better attribution test than --bare: disabling MCP alone reduces process-tree RSS by about 0.42-0.47 GiB while keeping the normal settings-loading path. Disabling hooks alone does not show a meaningful RSS change in no-tool-call cases.
• Per-MCP isolation points to chrome-devtools as the dominant MCP contributor: it is enough by itself to reproduce the high RSS band, and removing it returns the run near the no-MCP baseline.
• Within the local Qwen runtime, the most suspicious areas are no longer "raw diff bytes sent to the model". The model-facing request body is bounded.
• The stronger suspects are static per-request context cost, repeated request rounds, tool schema size, and local retention/capture of large tool outputs before or outside model-facing truncation.
• Because RSS remains much higher than V8 heap, the next profiling layer should include module/startup accounting, external memory, and heap snapshots around tool execution and final response emission.

RSS Attribution From Current Diagnostics

The current counters do not identify an exact retained object or source file, but they do narrow what is and is not driving RSS in these local runs:

Working attribution: in the stripped local bundle benchmark, the RSS floor looks mostly like task-time runtime/module/native footprint, with large tool output adding incremental pressure. In the normal-config run, the user-visible 1 GiB tree peak is mostly process-tree composition: Qwen root plus MCP/Node child processes. The next targeted measurement should split Qwen root diagnostics from configured MCP server diagnostics, then add startup/module/external-memory checkpoints inside the Qwen root process.

Progress Snapshot

Current confirmed signals:

1. The user-visible 1 GiB startup/config peak is reproducible with both the installed CLI and the local diagnostics bundle when the normal config is loaded. It is not primarily explained by the diagnostics branch or PR #4186.
2. In this environment, that 1 GiB peak is mostly process-tree composition: Qwen root process plus relaunch child process plus MCP child processes.
3. chrome-devtools is the dominant configured MCP contributor in the current config. It is enough by itself to reproduce the high process-tree RSS band, even when the prompt does not explicitly use that MCP.
4. The no-MCP normal relaunch shape still sits around 0.45 GiB process-tree RSS. A single Qwen runtime process without the relaunch parent is closer to 0.22-0.24 GiB in the startup attribution check. This means the 0.45 GiB baseline is not a single-process root RSS number.
5. In stripped non-interactive task runs, model choice changes turns, token totals, latency, and request sizes more clearly than RSS. RSS stayed in a relatively narrow range across pai/glm-5, qwen3.6-plus, and DeepSeek/deepseek-v4-pro.
6. Current short-task diagnostics show model-facing tool/function responses are bounded, but local tool-result capture and runtime state can still increase heap/RSS on large-output cases. This keeps large-output retention on the investigation path.

Current gaps:

1. The short-task benchmark matrix is still short-lived. A later interactive long-review run did reproduce a 41.9 min failure, but it is still one sample and needs repeat runs plus heap/object attribution.
2. The current counters are enough to attribute process-tree RSS and request size, but not enough to name the retained JS object graph during long sessions.
3. Startup/config RSS and long-session OOM must remain separate tracks. MCP and relaunch explain a large idle/startup RSS band; they do not by themselves explain V8 heap OOM after long tasks.
4. Interactive TUI memory still needs a separate run from non-interactive mode, because UI history and Ink static output are not exercised the same way.

Long-Task OOM Evidence From Issues And PRs

Issue/PR evidence points to several different OOM shapes, not one single failure mode:

Initial interpretation:

• Unused configured MCP can consume memory because normal startup connects to configured MCP servers and advertises their tools before the task needs them. In the measured config, chrome-devtools starts extra Node/npm MCP processes and also increases the tool schema count from 19 to 48. This explains a large startup/config RSS band and can also increase repeated request overhead.
• The long-session OOM reports are a different layer. GC logs where Mark-Compact frees very little memory suggest the heap is full of reachable state. The strongest candidates are retained history/tool/UI objects, full-history clones, compression intermediates, and streaming/logging accumulators.
• PR #4186 is a useful mitigation because it can compact based on heap pressure before token thresholds trigger, and it removes one unnecessary full-history clone. It should not be treated as proof that large tool-output retention, UI history retention, or compression peak memory is already solved.

Long-Task Validation Plan

The next benchmark should keep two tracks separate:

1. Startup/config attribution: normal config vs MCP-disabled vs chrome-devtools-only vs no-relaunch attribution. This explains what users see before meaningful work begins.
2. Long-task runtime growth: repeated tool calls, large outputs, compression, resume, and interactive UI history. This explains OOM after real work.

Recommended long-task cases:

Metrics that should be recorded per turn or per sampling interval:

• Root RSS current/peak and process-tree RSS current/peak.
• Child process count and top child command shapes.
• V8 heapUsed, heapTotal, heap_size_limit, external, and arrayBuffers.
• Turn count, request count, tool-call count, and tool-call rounds.
• Input/output/cache/total tokens by request and by whole task.
• Request body bytes, system prompt bytes, tool schema bytes, and function response bytes.
• Tool-result count, total captured tool-result bytes, max tool-result bytes, and retained tool-result bytes if available.
• Conversation history message count and approximate history byte size.
• Interactive-only UI history item count and approximate retained display size.
• Compression attempts, compression trigger reason, tokens before/after, heap pressure before/after, and compression failure status.
• Heap snapshot or bounded memory timeline artifacts when heap pressure crosses a configured threshold.

Validation criteria:

1. Repeat at least the key long-task cases twice. Startup RSS has visible variance, so single-run conclusions should be avoided.
2. Report root RSS and process-tree RSS separately. User-facing memory pressure can come from child processes, while V8 OOM comes from the Qwen root heap.
3. Treat a flat RSS line as important evidence. If tokens and tool calls grow but heap/RSS stays flat, the issue is likely elsewhere.
4. When RSS or heap grows, correlate the growth with a specific signal: tool-result bytes, history bytes, UI history count, compression event, streaming accumulator size, or MCP process start.
5. If a heap snapshot is taken, write a structured diagnostics JSON first, then the snapshot. Heap snapshots may be large and can contain sensitive strings, so they should remain opt-in and local.

Interactive Long-Review Reproduction

After the short non-interactive prompts kept finishing before the target window, an interactive TUI benchmark was run with remote input. The CLI process stayed alive in one session while a controller submitted one real PR-review turn at a time. The next turn was only submitted after the assistant emitted that turn's completion marker. This avoids treating a short one-shot prompt as a long-task reproduction.

Setup:

• Installed Qwen Code 0.15.11, model qwen-latest-series-invite-beta-v28.
• Temporary CLI home derived from the normal settings, with MCP and hook config removed. No global config was modified.
• Interactive TUI mode with dual JSON event output and remote JSONL input.
• Static PR review only. The prompt disallowed dependency install, build, test, Playwright, Docker, and other long external build commands.
• External RSS samplers recorded both process-tree RSS and the Qwen Node root RSS every 5 seconds.

Outcome:

The process exited with:

libc++abi: terminating due to uncaught exception of type std::__1::system_error: thread constructor failed: Resource temporarily unavailable

This is a thread exhaustion error, not a V8 heap OOM. The failure mechanism is distinct: the OS refused to create a new thread, likely due to per-process resource limits (RLIMIT_NPROC) or memory fragmentation preventing stack allocation. It is still relevant because it occurred in a disabled-MCP, no-build/test, interactive long-session review where the Qwen Node process itself crossed about 1 GiB RSS. The failure happened during the final summary phase, after the controller had already completed six review turns.

Turn timeline and sampled Qwen root RSS:

Token and tool distribution:

Tool-call telemetry by function:

The top visible TUI token counter for a single agent reached about 3.83M tokens. Telemetry also shows the heaviest subagent at about 4.05M total tokens with a 215K-token max input request. That makes subagent amplification the dominant signal in this reproduction.

Interpretation:

1. This run separates long-session growth from MCP startup/config memory. MCP was disabled and there were no MCP tool calls, yet the Qwen root process still reached about 1 GiB RSS.
2. The late memory peak aligns with subagent-heavy review turns and final summary/merge-back, not with external build/test child processes.
3. The RSS curve is not a simple linear leak. It falls after early turns, then rises sharply after later subagent turns and remains high near exit.
4. The failure mode is native resource exhaustion rather than a V8 heap-limit stack, so the next run should add heap/external/arrayBuffer/thread-count sampling. RSS alone cannot distinguish JS heap from native allocations or thread-resource pressure.
5. The strongest code paths to inspect remain subagent transcript retention, agent-result merge-back, full-history cloning, checkpoint/session recording, and final summary/history assembly.

Deterministic Huge-Task Clone-Pressure Reproduction

A deterministic stress harness was added as scripts/memory-pressure-repro.mjs. It does not call a model. Instead, it constructs a Qwen-like long-session object graph with root review turns, subagent transcripts, large tool results, checkpoint JSON, and retained structuredClone() copies. This gives a repeatable reproduction for the clone and checkpoint peak suspected from the user-provided OOM stack.

The harness has a lightweight script test:

npx vitest run --config ./scripts/tests/vitest.config.ts \
  scripts/tests/memory-pressure-repro.test.js

Result: passed, 1 test.

Controlled runs used node --max-old-space-size=256 unless otherwise noted.

The failing huge-clone run produced:

FATAL ERROR: Reached heap limit Allocation failed - JavaScript heap out of memory

The native stack included:

• v8::internal::ValueDeserializer::ReadObjectInternal
• v8::internal::ValueDeserializer::ReadDenseJSArray
• node::worker::Message::Deserialize
• node::worker::StructuredClone

This matches the same stack family as the user-provided OOM log. The controlled reproduction also shows why 4 GiB / 8 GiB user reports are plausible: the failure is not caused by a single large object, but by large retained history/tool-result/subagent state plus one or more full-history clone or checkpoint copies. Raising --max-old-space-size can delay the crash while preserving the same amplification pattern.

Important attribution from this deterministic run:

1. Building a 76.2 MiB parent history JSON can succeed under the reduced heap. The OOM appears when additional full-history clone/checkpoint copies are retained.
2. A single checkpoint copy can push RSS close to 1 GiB even before OOM.
3. Removing retained subagent transcripts from the parent hot history changes the same generated workload from OOM to a small 136.8 MiB RSS run. That is the clearest mitigation signal so far.
4. This reproducer is synthetic and intentionally adversarial, but it exercises the same object-graph shape as the long interactive review: parent session, subagents, large tool outputs, transcript merge-back, and full-history clone pressure.

DeepSeek PR-Size Follow-Up

After the initial model matrix, an additional Qwen Code-only run tested DeepSeek/deepseek-v4-pro across three real PR sizes. This model is configured through the Anthropic-compatible protocol; OpenAI-compatible execution returned 404 in a smoke check, so the successful benchmark uses --auth-type anthropic.

The diagnostics branch was extended to record Anthropic wire request summaries with the same privacy rule as the OpenAI path: aggregate counts and byte sizes only, no prompt text, diff content, tool arguments, headers, base URL, or API key.

PR sizes:

Runtime:

Request and tool diagnostics:

DeepSeek observations:

1. PR size scaled turns, tokens, Anthropic wire body size, and tool result size clearly, but did not scale RSS proportionally. The small/medium/large tree RSS peaks stayed in a narrow 340.7-360.0 MiB band.
2. The large PR was expensive mostly in model rounds and token volume: 8 requests and 386,891 total tokens. Its max Anthropic body was 289.5 KiB, much larger than the OpenAI-compatible runs, but RSS still stayed near the same local-bundle band.
3. The static Anthropic request cost is also visible: system prompt is about 50.8 KiB and tool schema about 47.6 KiB per request. Repeated rounds are therefore a major token amplifier.
4. The large PR produced 235.0 KiB of captured tool results and 182.4 KiB max function response in a request. This is higher than the earlier small PR / code-navigation cases and shows large PRs still put pressure on local tool-result handling and request assembly, even when RSS does not spike.
5. The DeepSeek run reinforces the model-choice conclusion: provider/model choice strongly changes turns, latency, token volume, and wire payload shape, but the local bundle RSS peak remains dominated by Qwen Code runtime shape rather than scaling linearly with PR size.

Long-Review JSONL Replay: History Clone Pressure

A recent long PR-review chat record was analyzed as a post-mortem shape for the reported OOM class. The raw JSONL is not included here because it contains prompt and tool output text. The aggregate shape is:

This shape does not support MCP as the primary OOM cause for this case. Only 4 of 507 tool-call telemetry events were MCP, and all four recorded content_length=0. The dominant shape is long-session/subagent amplification: 15 agent calls produced 313 subagent API responses and 403 subagent tool-call events.

The replay then rebuilt the chat Content[] message shape from the JSONL and ran controlled clone/stringify pressure tests. The base retained message payload is small, so it is not itself enough to OOM:

The scaled replay is not a user-data claim; it is a controlled amplification of the observed JSONL shape to test whether full-history clone and checkpoint serialization can create the same failure mode as the reports.

A low-heap reproduction with --max-old-space-size=256 confirms the mechanism:

The repeated-clone OOM stack contains ValueDeserializer::ReadObjectInternal, ValueDeserializer::ReadDenseJSArray, node::worker::Message::Deserialize, and node::worker::StructuredClone, matching the same stack family seen in the user-provided OOM log. This proves that full-history structuredClone() can be the immediate OOM trigger without any MCP server involvement.

Current working hypothesis for this JSONL class:

1. MCP can explain normal-config startup RSS in separate benchmarks, but it is not the likely trigger for this long-review OOM shape.
2. Long task growth comes from retained chat history, large tool outputs, subagent histories, observable agent messages, and UI/tool-result state.
3. The immediate OOM trigger can be a full-history clone or checkpoint-style double serialization after the heap is already high.
4. Compression can mitigate retained history, but compression itself may create a temporary peak if it first clones or serializes large history.

Local Mitigation Validation: Disabled-MCP PR Review Case

Two targeted mitigations were applied locally and validated before rerunning a disabled-MCP PR review case:

1. checkNextSpeaker() now reads only the last curated message with getHistoryTail(1, true) and sends only that message to the next-speaker side query. The next-speaker prompt only asks about the immediately previous model response, so sending full history was unnecessary clone and token pressure.
2. AgentToolInvocation no longer retains full responseParts arrays inside the live task_execution.toolCalls display. The real response parts still flow through transcript/history paths, but the parent UI display now keeps only a bounded text summary for nested tool-result streaming instead of holding another full copy of large subagent tool outputs during long runs.
3. GeminiChat.sendMessageStream() now builds model request contents through an internal curated-history view instead of calling public getHistory(true). Public getHistory() still returns a defensive structuredClone() for external callers, but the request hot path no longer deep-clones the whole retained chat history before every model call.

TDD checks added for these mitigations:

Additional reproduction and fix validation:

This narrows the earlier "same stack family" statement: the deterministic synthetic OOM still proves retained full-history clones can fail in the same V8 stack family as the user log, while the new GeminiChat red/green test proves one real production request-setup path no longer reaches that clone point. Checkpoint/resume and compression internals still need separate long-run validation because they can legitimately need durable copied history.

Verification commands:

The full root npm run build was not clean in this worktree because the vscode-ide-companion package hit pre-existing import/no-internal-modules lint errors. The core/CLI build and bundle needed for the local runtime test completed successfully.

The same PR review prompt was then run with a temporary config where MCP and hooks were disabled. Both rows were interrupted after a bounded long-run window instead of waiting for a full review to finish. Caveat: the two runs are confounded by workload size (79K vs 390K tokens) and cannot be compared as a controlled experiment. The comparison only shows directional evidence.

This is not a deterministic apples-to-apples model benchmark: the patched run did more work and consumed substantially more total tokens before the manual cutoff. The useful signal is narrower: under a disabled-MCP review case with more observed work, root max RSS did not increase and was about 47.2 MiB lower. That supports the mitigation direction, but it does not prove the whole long-task OOM class is fixed.

Remaining high-risk clone/retention paths to inspect next:

1. Compression still calls full getHistory(true) before summarization. If the heap is already high, the compression attempt can create the peak that trips OOM.
2. Checkpoint creation can hold original history, cloned client history, and a serialized checkpoint payload at the same time.
3. Fork subagents still seed from parent history with getHistory(true).
4. ACP/history export/summary/copy paths still call full getHistory() and should be audited separately from the normal review loop.

Version timing:

The issue history does not prove that 0.15.10 introduced the OOM class; similar reports existed in March and April. It does support a recent cluster beginning around 2026-05-13, overlapping v0.15.10/v0.15.11 releases. The relevant diff between v0.15.9 and v0.15.10 touched subagent runtime, non-interactive execution, GeminiChat, and compression code heavily, so this range is a reasonable first bisect window.

Notes

• The first code-navigation prompt allowed open-ended exploration and hit maxSessionTurns; the successful rows above use a constrained command list.
• The first synthetic-diff attempt used a relative bundle path from inside the temporary repositories; those failed immediately and are excluded from the tables. The successful rows use the absolute local bundle path.
• Raw JSONL streams are not committed because they contain prompts, tool commands, and tool output. The report only includes aggregate diagnostics.