tools/llm-sequential-upgrade/perf-benchmark/optimized-reference/METHODOLOGY.md
These files contain the optimized reference versions of the sendMessage handlers.
There are only two comparison points:
Same features as AI-generated. Implementation changes only:
userIdentity.filter(ctx.sender) instead of roomId.filter(roomId) + [...spread] + toHexString() string allocationSame features as AI-generated. Implementation changes only:
Same features as AI-generated. Implementation changes only:
Same features as AI-generated. Implementation changes only (produced by a clean
claude-sonnet-4-6 first-principles pass — goal-only prompt, no access to the PG/STDB
optimized references; it independently chose a different optimization set):
maxPoolSize 5 → 20 (default pool was the bottleneck under burst load)POST /messages: send the HTTP response right after the DB insert; defer the socket fan-outtrackMessageActivity: deferred global emit via setImmediate; Date[] → number[]; amortised trim instead of per-message filter()getActivityLevel: single counting loop instead of two filter() allocationsGET /messages, ephemeral cleanup): added .lean() (skip Mongoose hydration)perMessageDeflate: false (compression overhead > savings for small chat payloads){ roomId: 1, parentId: 1, createdAt: 1 } to satisfy the room-message query+sort in one B-tree scan| Version | STDB avg | PG avg | Ratio |
|---|---|---|---|
| Raw | 5,267 msgs/sec | 694 msgs/sec | 7.6x |
| Optimized (this dir) | 25,278 msgs/sec | 1,139 msgs/sec | 22x |
Stress throughput (writer-count sweep; peak shown), measured on the 20260616 machine:
| Version | Mongo peak | vs optimized STDB |
|---|---|---|
| Raw | ~800 msgs/sec (peak 796 @ 200 writers) | — |
| Optimized | ~1,400 msgs/sec (peak 1,394 @ 100 writers) | ~18x slower |
Optimization gain ~1.7x — in line with PG's 1.6x (both hand-built stacks gain modestly; STDB's 4.8x reflects more architectural headroom). Optimized Mongo (~1,400) ≈ optimized PG (1,139); both ~18–22x under optimized STDB (25,278).
Caveats (do not drop these when citing the Mongo numbers):
20260406 run;
the Mongo figures are from 20260616 on a different machine. The within-Mongo ratio
(raw→optimized, ~1.7x) is clean; absolute cross-backend numbers are not strictly
same-sitting. A same-machine re-run of all three would remove this.