Back to Next Js

Statistical methodology

.agents/skills/sandbox-bench/references/methodology.md

16.3.05.1 KB
Original Source

Statistical methodology

The design decisions below are what make the numbers trustworthy. They follow standard rigorous-benchmarking practice (Kalibera & Jones, "Rigorous Benchmarking in Reasonable Time"; Georges et al., "Statistically Rigorous Java Performance Evaluation"; JMH's forks model) — don't relax them to save wall-clock without understanding what they protect against.

The VM boot is the unit of replication

JIT state, code layout, GC rhythm, host hardware, and phase order are random effects fixed at boot time. Iterations within a boot share all of them, so they are not independent samples — treating them as independent produces confident nonsense (a p<1e-4 effect in one run that vanishes in the next). The harness therefore:

  • computes each metric's delta per boot (mean of that boot's paired within-boot deltas), and
  • runs t-based inference across boots (bench-stats.mjs); the CI and p-value you see are boot-level.

The within-run p printed in brackets is a diagnostic only. It tells you the pairing worked; it cannot support a claim.

Allocation follows from this: more boots × fewer runs beats fewer boots × more runs at fixed budget, and boots run in parallel — adding them costs money, not wall-clock, so don't be shy with them when sensitivity matters (CI shrinks with sqrt of boots). Defaults: e2e 16 VMs × 1 block × 2 runs. Both arms always run in the SAME VM, interleaved ABBA, paired per (vm, run) — hosts differ by up to ~20%, so cross-VM comparisons are meaningless; VMs exist to replicate, not to compare.

Claims

  • A claim requires boot-level p < 0.01, reported with its CI, on a team/config whose boot-level inference an A/A run has validated.
  • Anything that will drive a decision (merge, revert, ship) gets one independent confirmation cell first (standard perf-CI practice — Mozilla/Chrome retrigger, MongoDB change-point detection — nobody alerts off one detection).
  • Everything else is "noise-compatible", reported as no detected difference — not as a small effect.

Percentiles

A per-run p95/p99 from a small load phase is a near-max order statistic, not a distribution estimate. Compare percentiles only at boot level; p99 needs ~1k+ samples per boot to mean anything at all. Tail effects are also where phase carryover lives (a route's serial tail depends on the previous route's load phase) — investigate tails with --isolate-routes, which restarts the server between routes.

A/A validation

Before claiming anything on a new team/config/suite, run an A/A cell (same ref as both arms) at default allocation:

sh
node scripts/sandbox-e2e.mjs --arms base=<ref>,cand=<ref> --label aa
node scripts/sandbox-ssr.mjs --arms base=<ref>,cand=<ref> --label ssr-aa

The A/A validates the inference itself. With identical arms, expect about (number of metric-cells x 0.01) cells at p < 0.01 by chance — under 1 for the e2e suite's ~24 cells, roughly 1 for the ssr suite's ~88. The pass criterion is therefore two runs: a suite fails its A/A when any cell reaches p < 0.01 with the same sign in both (a bias tied to the arm label recurs; chance does not), or when any single cell lands beyond the familywise bar (p < 0.01 / number of independent variant-x-phase cells). A failing cell means boots are sharing label-correlated conditions and the CIs are too narrow — find the mechanism before claiming; a magnitude gate is not a substitute. The A/A CIs also tell you what effect sizes this config can resolve. Re-run the A/A when the platform changes (VM hardware generation, node version, bench app or fixture changes).

Validated so far: e2e (twice, 0/24 cells), ssr (two-run criterion: one chance cell at p=0.0024 in run 1 dissolved to p=0.86 in run 2, nothing recurred; runs ssr-aa2/ssr-aa3 in the results db).

Interim progress display

The launcher's progress digest shows, per route/phase, the running rps effect and a directional confidence — P(candidate is actually faster/slower | boots so far), the Student-t posterior under a flat prior (not a p-value). It appears from 4 completed boots. Display only: runs complete their planned allocation, and claims come only from the final fixed-n analysis.

Fingerprints

Every result row records a build fingerprint (dist-file hash + version string) read from the tree that actually served requests. The analysis header verifies them: inconsistent fingerprints within an arm invalidate the run (a VM measured the wrong build). Identical fingerprints across arms with different version strings can be legitimate — the hash covers the compiled server files, so client-only changes don't move it; check the version strings before declaring an accidental A/A.

Honest reporting

  • State the platform (e.g. "x86 Xeon sandbox VMs") next to any number. Magnitudes are platform-dependent — GC's share of request time scales with CPU speed — so direction and mechanism transfer between platforms, percentages do not.
  • Print per-boot values; if boots disagree in sign, the mean is not a finding, and say so.
  • Never conclude anything from the live streaming estimates. Peeking at sequential results and stopping when p looks good is a classic way to manufacture false positives.