docs/issue-2037-p0-local-corpus-findings.md
~/.codex/sessions + ~/.codex/archived_sessions (this machine)docs/issue-2037-fork-family-provenance-spec.md (rev 8a+)| Metric | Value |
|---|---|
| JSONL files | ~81+ (grew during Ultra P0 attempts) |
Logical sessions (first session_meta.id) | dozens |
Real fork edges (forked_from_id ≠ self) | several families |
token_count events surveyed | ~6k+ |
Observed families (prefixes):
019f33ce → 019f3869 → 019f38ec (parent → child → grandchild)019f32c8 → 019f331e (mid-session / partial prefix fork)019f3cec → 019f3e32 (partial prefix; parent continued after fork)019f4d90 → 019f52bf / 019f52e6 / 019f52e7 / 019f52f3 (Sol/Terra, multi-agent v2)| Candidate | Survives copy? | Event-level cardinality? | Verdict |
|---|---|---|---|
Stable event / request / message ID on token_count | n/a | Absent on token_count rows | Do not use as event key |
turn_id | Present on other events (event_msg, turn_context) | Not on token_count | Lineage / priority join hint only — never sole cross-completion key |
Envelope timestamp on token_count | No — rewritten at fork into a tight burst | n/a | Not copy-stable; exclude from fingerprint match |
| File-local ordinal / line number | No | Tie-break only | Exclude from fingerprint |
Normalized last_token_usage + total_token_usage vector | Yes (field-equal, ordered) | Unique within observed parent streams, but not proven unique across distinct sibling work | Copy-candidate evidence only; insufficient destructive event identity |
| Byte-identical JSONL line | No (0 byte-identical parent↔child lines) | n/a | Prefer normalized equality |
Lock: For this Codex shape, there is no stable per-token_count event ID. Ordered exact-prefix matching of normalized usage vectors under forked_from_id ancestry identifies the copied-prefix candidate in the sanitized corpus, but token equality alone must not suppress runtime billing.
Copied ancestor token_count rows are field-equal after normalization, not byte-identical. Matching inputs that worked locally:
last_token_usage (all int fields)total_token_usage (all int fields)Do not require equal timestamps.
Corpus oracle rule (not a sufficient runtime event key): Contiguous ordered usage-fingerprint prefix.
token_count streams in file order.N = longest k such that child[0..k) equals parent[0..k) under the normalized usage fingerprint.child[0..N) is the copied prefix (state-only on the child; parent owns billing when present).child[N..) is child-local work.Corroboration (not the classifier): leaf session_meta.timestamp (fork_ts) lands within ~0–1s of the end of the rewritten prefix / start of unique work on observed mid-session forks. Useful for family graph ordering; do not classify prefix via parentEventTimestamp <= fork_ts because copied child timestamps are rewritten and are not the parent’s original times.
Evidence:
| Edge | Parent token_counts | Child | Prefix N | Parent continued after fork? |
|---|---|---|---|---|
33ce→3869 | 135 | 158 | 135 (entire parent) | No |
3869→38ec | 158 | 158 | 158 (entire parent; no unique child tokens in archive) | n/a |
32c8→331e | 270 | 276 | 243 | Yes (+27 parent events) |
3cec→3e32 | 387 | 581 | 226 | Yes |
4d90→52bf | 293 (live; continued) | 196 | 180 | Yes (parent continued after copy) |
last_token_usage semanticsOn ordinary in-session streams, Δ total_token_usage.total_tokens == last_token_usage.total_tokens on nearly every step (hundreds of matches, ~0 positive mismatches in sampled files).
Caveat: the first post-fork child event can show last > 0 while total is still flat vs the last prefix event. After provenance, billing should follow the corpus-locked per-event policy carefully; total-delta alone undercounts that row, last alone may overcount vs cumulative. Flag for golden oracles — do not invent a hybrid in P0 prose beyond “measure on goldens.”
Oracle units: hand oracles that sum last.total_tokens are not identical to scanner-priced units. Scanner accumulates input + cached_input + output from each billed delta. Integration tests must compare scanner units (sum(last.input+cached+output) with prefix suppressed), not raw total_tokens field sums.
session_meta in the file (the leaf). Later session_meta rows often re-embed ancestors.session_meta chains (leaf, then parent, then grandparent, …). Graph builder must not treat every embedded meta as a separate root file identity.session_meta.id only in the sense that archives are named by leaf id; always prefer leaf meta id.Common: active sessions/… rollout + archived_sessions/… for related rollouts. Same-session union must be ordered overlap, not unordered fingerprint sets. Multi-file groups often share a leaf id across rollouts; treat carefully (some “same id” groups are true active/archive; some are chained fork archives — use first session_meta + forked_from_id).
For 019f33ce + child 019f3869, summing last_token_usage.total_tokens over both files without prefix dedupe vs parent-owns-prefix:
| Method | Sum of last.total_tokens |
|---|---|
| Naive parent+child | ~28.8M |
| Dedupe (parent owns prefix) | ~15.4M |
| Inflation ratio | ~1.88× |
This is the inter-file overcount shape on ordinary forks (not yet Ultra interleave).
Attempts to produce intra-file interleaved cumulative streams (multiple rising total_token_usage counters mixed so totals drop mid-file):
| Attempt | What appeared | Total drops ≥50k? |
|---|---|---|
Sol/Terra + multi-agent v2 forks (4d90 family) | Fork copies, monotonic totals | No |
| Parallel-worker P0 prompt (fixture sanitization) | New forks 52e6/52e7/52f3, proactive mode sometimes | No |
| Corpus-wide scan after those runs | — | 0 files |
Conclusion: Harder tasks and parallel Ultra workers on this Codex build produced useful fork-copy logs and Sol/Terra multi-agent metadata, but not the reporter’s interleaved gap-recount shape. Token drops are not required to proceed on cross-file provenance. Interleaved Ultra remains reporter-corpus-dependent.
“Ultra” string hits in older logs were mostly chat/tool text / schemas, not a structural Ultra mode flag on token_count.
logs_2 turn surcharge metadata on these token_count rows (not visible in the JSONL token payload here)usage_fp collision rate at scale (sample showed collisions across parent/child as expected; token equality is not allowed to become destructive identity)token_count.(last_token_usage, total_token_usage) under ancestry.N; session_meta fork timestamp for graph ordering only.session_meta is leaf; ignore ancestor meta rows as competing file identities.#2066 containment remains file-local and non-closing.| Path | Role |
|---|---|
Tests/.../Fixtures/CostUsage/Issue2037/archived-fork-33ce-3869/ | Sanitized parent+child from local 33ce→3869; usage+meta only |
.../live-fork-4d90-52bf/ | Second parent-present golden from 4d90→52bf; parent truncated to prefix N=180 |
.../missing-parent-siblings/ | Two children, shared prefix, no parent file |
.../ORACLE.md + manifest.json | Hand oracle: naive vs parent-owns-prefix / provisional owner |
.../harness-smoke/ | Tiny install harness smoke |
Issue2037FixtureSupport.swift | Shared load/install + sanitized event parsing |
Issue2037ProvenanceFixtureTests.swift | Prefix length, oracle arithmetic, field allowlist |
Issue2037FixtureHarnessTests.swift | Install into CostUsageTestEnvironment |
Issue2037ScannerIntegrationTests.swift | End-to-end loadDailyReport vs scanner-unit oracle |
Fixture timestamps were rewritten to midday UTC on distinct calendar days so local timezones do not map parent rows outside a Jan 1–2 report window (midnight UTC previously made parent appear as the prior local day and dropped it from the report).
archived-fork-33ce-3869 into an isolated Codex home (sessions/ + archived_sessions/ roots as the scanner expects).CostUsageScanner.loadDailyReport(provider: .codex, …, forceRescan: true) over the fixture days.input + cacheRead + output to:
last.input+cached+output over parent and all child eventsWith the parent file present in-window, current #1164 inherited-totals accounting already matches the parent-owns-prefix scanner-unit oracle.
Issue2037ScannerIntegrationTests locks this as a regression, not as a new ledger.since/until then silently omits the parent and the report shows only child unique growth (~3.3M in one probe) — looking like undercount, not overcount.sum(last.total_tokens) ≠ scanner priced units when total_tokens omits cached input; always compare scanner units in integration tests.archived_sessions listing allows undated names (parent.jsonl); dated filenames are range-filtered — fine for fixtures without dates in the name.#1164 does not cover (next work)| Gap | Why inheritance is insufficient |
|---|---|
| Missing parent + two siblings sharing a copied prefix | No parent snapshot to inherit; each child can bill the full prefix |
| Provisional family billable owner | Spec §4.4.2 / §4.5.1 — needs family ledger, not per-file baseline alone |
| Intra-file interleaved totals | No drops in local Ultra logs; containment (#2066) is a separate guard |
| True event-key ledger across arbitrary ancestry depth | Inheritance is fork-point baseline, not per-event ownership migration |
Repro probes were ephemeral Python over ~/.codex/**/*.jsonl, hashing normalized usage vectors and comparing ordered prefixes. Raw production JSONL was not copied into the repo; fixtures are sanitized aliases only.
#1164-only gap.missing-parent-siblings). Runtime cross-file suppression is deferred until copy-stable event identity is proven.4d90→52bf as a second parent-present golden.live-fork-4d90-52bf; parent truncated to copied prefix for a clean #1164 regression). Locks a real corpus quirk: parent ordinal 120 has non-zero last with flat total — scanner units must follow total deltas, not sum(last).Fixture missing-parent-siblings: two children, shared 135-event prefix, no parent file.
| Metric | Scanner units |
|---|---|
| Naive (both bill prefix) | ~54.4M |
| Target parent-owns-prefix oracle | ~28.8M |
The provisional token-vector prefix suppression was removed before merge. Distinct sibling events can have identical last_token_usage / total_token_usage vectors at the same ordinal; treating that equality as event identity can silently suppress legitimate work. Adding envelope timestamps is invalid because copied timestamps are rewritten, and model is not reliably present on token_count rows.
Runtime fail-open rule: when a parent file is missing, do not perform destructive cross-file dedupe from token counters alone. Each sibling keeps its file-local unresolved-parent accounting. This can retain copied-prefix overcount, but it cannot undercount distinct work merely because counters collide. Issue2037ScannerIntegrationTests locks this with two missing-parent siblings whose distinct models/timestamps have equal token vectors.
The sanitized fixture and arithmetic remain useful as the desired family-ledger oracle. They do not authorize suppression until the runtime has a corpus-proven copy-stable event key, collision handling, cross-window family closure, and an exposed ambiguous/contained quality path.
Required before revisiting runtime suppression:
reasoning_output_tokens / total_tokens only reduce collision probability and are not sufficient identity.