Profiling renderer typing lag

Workflow for empirically measuring (and fixing) typing/submit lag in the desktop chat composer.

Quick boot for profiling

Vite 8 + plugin-react 6 has a known issue where the React Fast Refresh preamble script isn't injected into index.html, so opening Electron at http://127.0.0.1:5174 throws $RefreshReg$ is not defined on every TSX module and the React tree never mounts. Workaround: run vite with HMR off.

# Terminal A — start dev server without HMR
cd apps/desktop
node scripts/dev-no-hmr.mjs

# Terminal B — start Electron with CDP exposed
cd apps/desktop
XCURSOR_SIZE=24 HERMES_DESKTOP_DEV_SERVER=http://127.0.0.1:5174 \
  ../../node_modules/.bin/electron --remote-debugging-port=9222 .

Terminal C is yours to run the harnesses.

Harnesses

All zero-dep — Node 24 built-in WebSocket + fetch.

Typing latency — measure-latency.mjs

Per-keystroke keypress → next paint latency, p50/p90/p99/max. Synthesizes keystrokes via Input.dispatchKeyEvent so the run is reproducible.

node apps/desktop/scripts/measure-latency.mjs --chars=120 --cps=20

Anything > 16ms is a dropped frame. On a freshly-loaded session (scripts/click-session.mjs 'Phaser particle') we currently see:

Roughly even on a quick session — patches don't fix typing latency under benign synthetic conditions because the existing baseline is already snappy on synthetic input. The real wins are in the leak counters (see below). If the user reports typing jank, capture a profile + heap diff during their actual usage and compare against the synthetic baseline to identify what condition (long thread, popover open, paste, etc.) makes the path slow.

Leak counters — leak-typing.mjs

Types N chars per round, clears, force-GCs, captures Performance.getMetrics deltas. Reveals leaked event listeners, heap drift, document node growth, and forced-layout counts.

# After clicking into a real session (e.g. via click-session.mjs):
node apps/desktop/scripts/leak-typing.mjs --rounds=8 --chars=200 --cps=50

Real-session numbers (Phaser thread, 8 rounds × 200 chars):

The forced-layout count is the load-bearing number — typing into a real session was triggering ~7 layouts per character on the unpatched build (scrollHeight reads + per-px CSS var writes + FadeText scrollWidth reads all stacking up). After the patches it's down to ~2.35/char, which is Blink's natural cost for a 1px/char-growing contentEditable and can't be lowered further without architectural changes.

The initial "+35 listeners/round leak" I called out on the first unpatched run turned out to be transient warm-up (popovers initializing, etc.); steady-state listener growth was 0 both before and after.

CPU profile + heap snapshot — profile-typing.mjs

Records a CPU profile while typing, plus before/after heap snapshots so you can do a comparison diff in Chrome DevTools Memory tab.

node apps/desktop/scripts/profile-typing.mjs \
  --chars=400 --cps=30 --out=/tmp/hermes-typing
# → /tmp/hermes-typing.cpuprofile  (open in Chrome DevTools Performance)
# → /tmp/hermes-typing.before.heapsnapshot
# → /tmp/hermes-typing.after.heapsnapshot

Loading the cpuprofile: Chrome DevTools → Performance tab → drag the file in, or VS Code → open the .cpuprofile directly.

For heap diff: Chrome DevTools → Memory → Load snapshot → load "before", then Comparison view → load "after". Sort by # Delta. Stay alert for detached DOM, FiberNodes (unmounted), and listener growth.

Helpers

• probe-renderer.mjs — dump page state (URL, composer mounted?, body text)
• click-session.mjs <title> — click a sidebar session by partial title match
• reload-renderer.mjs — force Page.reload via CDP (no HMR available)
• dump-state.mjs — richer state dump (thread message count, sticky session, etc.)
• probe-console.mjs — dump recent console errors / exceptions

Findings

See commit message for apps/desktop/src/app/chat/composer/index.tsx edits. Three changes:

1. Per-keystroke scrollHeight read removed. The expansion useEffect used to read editorRef.current.scrollHeight on every draft change (forces synchronous layout). Replaced with a draft.length > 60 heuristic; the ResizeObserver catches anything the heuristic misses.

2. Bucketed CSS custom-property writes. syncComposerMetrics used to setProperty('--composer-measured-height', height + 'px') on every observed resize, invalidating computed style for the whole tree. Now writes only when the height crosses an 8 px bucket, so typing in a fixed-height row produces no style invalidation at all.

3. Removed dead $composerDraft → aui.composer().setText round-trip. Nothing outside the composer subscribed to $composerDraft (verified via grep). The two useEffects that pushed draft → store and store → composer were pure overhead per keystroke. reconcileComposerTerminalSelections was also called per keystroke; can be deferred to submit time (it's a stale-pruning step, not a correctness one — terminalContextBlocksFromDraft walks the current text directly at submit and ignores stale labels).

4. refreshTrigger fast-bails when no @// in draft. Previously textBeforeCaret() did range.toString() (O(n)) on every keystroke even when no trigger char was present.

The biggest win is the listener leak in (3) — without it, each round of typing leaked ~35 event listeners until a steady state.

Submit / TTFT stall (open)

User reports a perceived stall after Enter, before the assistant starts streaming. scripts/measure-submit.mjs measures enter → composer-cleared → user-message-rendered → first-paint. The script triggers a real prompt submission, so use it on a throwaway session. Not enabled in CI.

Streaming "5fps" investigation (May 21, 2026)

User complaint: "the streaming must bring fps to like 5? lol" — felt hitches during assistant streaming on long threads.

Tooling added

• src/app/chat/perf-probe.tsx — dev-only side-effect import (guarded by import.meta.env.MODE !== 'production' in main.tsx). Attaches two helpers to window:
  • __PERF_PROBE__ — React <Profiler> recorder. Currently inert because Vite is serving the production React build (see "Vite dev-build issue" below); kept for when that's fixed.
  • __PERF_DRIVE__ — synthetic stream driver. Pushes tokens through the live $messages atom at a fixed cadence, so the assistant-ui runtime, incremental repository, Streamdown markdown renderer, and React commit pipeline all see the same workload they'd see from a real LLM stream — but with no LLM call (and no credit cost).
• scripts/measure-synthetic-stream.mjs — drives __PERF_DRIVE__, records rAF frame intervals, PerformanceObserver({entryTypes:['longtask']}) entries, MutationObserver cadence on the live message, and optional type-while-streaming keystroke latency.
• scripts/profile-synth-stream.mjs — CPU profile during a synthetic stream; writes a .cpuprofile (open in Chrome DevTools Performance panel) and a top-30 self-time table.
• scripts/measure-real-stream.mjs — same harness as the synthetic but fires a real LLM prompt. Use when you have credits and want to confirm the synthetic predictions hold.
• scripts/profile-real-stream.mjs — CPU profile over the duration of a real LLM stream.

Helpers: scripts/eval.mjs (one-shot CDP eval), scripts/reload.mjs (hard reload renderer over CDP).

Findings

Measured on the Cloud Shadows session (7 turns, ~11k px scrollHeight) and the 34 MB session session_20260514_215353_fe0ac8.json (110 FadeText instances, lots of historical tool calls).

A single real-LLM stream on Cloud Shadows (gpt-4o-mini, 39s window) saw 12 longtasks totalling 1.26 s — same cadence the synthetic predicted (~1 hitch per 3.25 s, max 123 ms). So the synthetic stream is a faithful proxy for the real one and is fine for iterating on fixes without paying for tokens.

CPU profile during streaming (synthetic, markdown content)

Top self-time costs (5 s window, 400 tokens at 125 tok/s, markdown chunks):

That chunk-BO2N2NFS is the vendored bundle containing micromark, mdast-util-from-markdown, mdast-util-to-hast, rehype-raw, hast-util-sanitize, etc. — i.e. Streamdown's markdown pipeline, re-parsing the entire growing assistant message on every token append. Cost scales linearly with message length.

Compare plain-text (no markdown) — the chunk-BO2N… entries drop out of the top 30 entirely; total work per 5 s window halves.

Fix landed: FadeText memo

FadeText is used in tool-fallback.tsx (110 instances on a tool-heavy thread). Before: each parent re-render during streaming triggered a useEffect([children]) that forced a scrollWidth layout read — even when the title text was unchanged. The useResizeObserver already covers the genuine resize case, so the effect was strictly redundant.

After: wrapped in React.memo with a custom comparator that compares children (scalar fast-path), className, fadeWidth, and style field-by-field. Verified via temporary render counter: 122 renders during a 2 s synthetic stream vs ~11 000 without memo (110 instances × ~100 stream updates). Doesn't move the longtask needle on its own — Streamdown dwarfs it — but eliminates a class of forced layouts and removes a steady CPU floor.

Also landed: MarkdownText plugins memo + upstream flush floor

Two smaller follow-ups in the same investigation:

1. MarkdownText plugins object useMemo'd. The inline plugins={{ math: mathPlugin, ...(isStreaming ? {} : { code }) }} was constructing a new object on every render, which churns <Streamdown>'s outer memo and forces its internal rehypePlugins / remarkPlugins arrays to rebuild. CPU profile after the change shows parser self-time dropping out of the top 10, compile cut roughly in half, and bn$1 / m$1 (micromark internals) dropping off the top entries.

2. use-message-stream.scheduleDeltaFlush got a real minimum floor. Previously the rAF-only path effectively meant "at most one flush per frame," but at typical LLM token rates of 30-80 tok/sec each token arrives slower than rAF cadence and gets its own React commit. With STREAM_DELTA_FLUSH_MS = 33 (two frames) and a lastFlushAt-tracked floor, slower streams now coalesce ~2 tokens per commit, halving markdown re-parses. React's auto-batching already covers part of this probabilistically; the floor makes the batching deterministic so the max-longtask number tightens up.

A/B on the 34 MB session, 300 tokens at 50 tok/sec, markdown chunks (3 trials each):

Modest. inter-mutation p50 tightens from 22-28 ms to a clean 33 ms, which is what you'd expect from a deterministic floor.

Also landed: useDeferredValue at the streamdown-text boundary

The longtask CPU was unavoidable inside the block-memo pattern — the live tail re-parses every commit, scales linearly with current length, and nothing about Streamdown's architecture changes that without forking. The fix is to stop having that work block the main thread.

<DeferStreamingText> in markdown-text.tsx is a 12-line wrapper that reads the message-part state via useMessagePartText, runs it through useDeferredValue, and re-publishes via assistant-ui's <TextMessagePartProvider>. The inner StreamdownTextPrimitive reads the deferred value through the normal useMessagePartText hook — no fork, no internal-path imports, fully on the assistant-ui public API.

What React's concurrent scheduler now does:

• When a new token arrives mid-render, the in-flight deferred render is abandoned and a fresh one starts with the latest text.
• When the main thread has urgent work (typing, scroll, layout), the Streamdown render gets deprioritized — input stays responsive even while a 100 ms parse is queued.

Streamdown already uses useTransition internally for its block-array setState; useDeferredValue here just lifts the deferral all the way up to the consumer text boundary, so the whole pipeline — preprocess, block split, repair, parse, render — runs at low priority during streaming. This is the industry-standard approach (see Streamdown architecture analysis and Chrome's LLM-response render best practices).

A/B on the 34 MB session, 300 tokens at 50 tok/sec, markdown chunks (four trials each, prod-throttle (33 ms) on for both):

Longtask count and max LT are unchanged — useDeferredValue doesn't reduce CPU, only its priority. The avgFps lift and p99 frame drop are the proof that the existing CPU is no longer blocking 60 fps cadence: when React can defer the parse, frames stay clean. One particularly clean run logged MUTATIONS=0 — React skipped every intermediate text state and only committed the final one, the textbook useDeferredValue behaviour.

Not fixed: Streamdown markdown re-parse cost (the elephant)

Total CPU spent in micromark/mdast/hast pipeline per 5 s window is still the same ~700 ms. With useDeferredValue that work no longer blocks input, but if you watch a CPU profile you'll see the same hot functions (Tn$1, bn$1, m$1, parser, compile).

The path to actually reduce that cost (not just defer it) is to replace the parser with a state machine like Flowdown — process each character exactly once, emit DOM ops directly, no re-parse of the prefix on every token. Claimed ~2,000× over marked. Trades: not a react-markdown-compatible API, no rehype security pipeline, would require replacing Streamdown wholesale. Worth investigating only if even the deferred work shows up in user-perceptible ways (e.g. trackpad-scrolling a stream-in-progress stutters).

The synthetic harness now mirrors the real upstream pipeline via the flushMinMs option in __PERF_DRIVE__.stream({ flushMinMs: 33 }), so future Streamdown / Flowdown experiments can A/B without LLM credit cost. The synthetic numbers tracked the one real-LLM run we caught within noise, so it's a reliable proxy.

Possible approaches (none implemented here):

1. Coalesce/throttle Streamdown updates — render at most every 32 ms instead of every set-state. Reduces parses but doesn't reduce per-parse cost; trades latency for smoothness.
2. Memoize per-prefix — diff the new text against the prior parsed version; only re-parse the changed suffix.
3. Render in stable segments — close-form historical paragraphs as immutable React nodes; only the live tail goes through markdown each token. Probably the highest-impact change but requires forking or patching @assistant-ui/react-streamdown.
4. Move parsing to a Web Worker — main thread no longer blocks on markdown. Largest surgery; requires double-buffered hast.

Vite dev-build issue (separate)

http://127.0.0.1:5174/node_modules/.vite/deps/react.js resolves to react/cjs/react.production.js, and react-dom_client.js → react-dom-client.production.js. As a result:

• <React.Profiler> onRender is never called (production build is a no-op).
• import.meta.env.DEV is false, PROD is true even under vite dev (hence MODE !== 'production' as the workaround in main.tsx).
• All the React 19 dev-only warnings/devtools backend hooks are absent.

Root cause likely sits in vite.config.ts aliasing + dedupe + Vite 8's new optimizeDeps defaults. Worth a separate fix pass — when it's resolved, the <PerfProbe> blocks in perf-probe.tsx become useful (per-id commit timings) instead of inert.