Back to Dd Trace Js

README

benchmark/sirun/plugin-claude-agent-sdk/README.md

6.1.01.3 KB
Original Source

This benchmark measures Claude Agent SDK lifecycle extraction cost.

The stream-scan variant models the fragile approach where each tool lifecycle is rediscovered by scanning the remaining event stream for task_started and tool_result chunks.

The hook-indexed variant models the hook-first approach: SDK hooks provide the semantic lifecycle record for each tool, while a single pass over the event stream builds cheap chunk indexes for LLM IO enrichment.

The hook-indexed variant uses the same compact fast path as the integration: it first checks the next few chunks for adjacent lifecycle data and only builds the full stream index when that local lookup misses. The stream index is still rebuilt inside each measured operation when needed, so delayed/noisy traces pay the per-trace indexing cost instead of benchmarking only Map lookups against repeated stream scans.

Two stream shapes are covered:

  • compact: task_started and tool_result chunks are adjacent to each tool use. This is the best case for repeated stream scanning.
  • delayed-noisy: tool lifecycle chunks are delayed until later in the turn and separated by unrelated assistant chunks. This models larger agent traces where reconstructing lifecycle from the stream repeatedly has to cross unrelated LLM/subagent output.