ShakaPerf Setup and Test Files

These are the exact setup/test files used for the RSC FOUC ShakaPerf runs.

Tests

Purpose	File
Deterministic first-paint visual test	ab-tests/rsc-fouc.abtest.ts
Natural first-visible assertion test	ab-tests/natural-first-visible-assertion.abtest.ts

Config

Purpose	File
Main ShakaPerf config	config/abtests.config.ts
Natural first-visible assertion config	config/natural-first-visible-commit.abtests.config.ts

Twin server setup

File	Purpose
twin-servers/Dockerfile	Docker build used by ShakaPerf twin servers.
twin-servers/Dockerfile.dockerignore	Docker ignore file.
twin-servers/Procfile	Process command used inside each ShakaPerf server container.
twin-servers/PROJECT_PROFILE.md	Generated project profile.

Artifact hardening notes

• Docker caching: The Dockerfile still copies the filtered repo context before dependency installs. This is intentional for the archived monorepo artifact because Bundler and pnpm workspace resolution need package and gemspec files across the checkout. The paired Dockerfile-specific dockerignore is the cache hardening mechanism; a manifest-first cache split would need a maintained allowlist for both Ruby and pnpm workspace metadata.
• Runtime secrets: The fixed SECRET_KEY_BASE is a dummy for this hermetic SQLite test image. It is no longer baked into Docker ENV; the Procfile passes it when the Rails process starts, and the build uses SECRET_KEY_BASE_DUMMY=1 for the assets:precompile, db:prepare, and db:seed commands that need Rails boot. Because the dummy is inline in the Procfile command, it is still visible in the container process list; use a wrapper script or secret file instead if adapting this pattern for a non-dummy value.
• Renderer URL: Docker sets REACT_RENDERER_URL=http://localhost:3800 in the image ENV, and the Procfile repeats the same literal for Rails startup. Do not use shell parameter expansion in the Procfile for this value: Overmind evaluates Procfile lines through a host shell before run-overmind-command enters the container, so a host REACT_RENDERER_URL can override the container-local renderer URL. A host export is not a supported override for Docker runs; change the Dockerfile ENV and both Procfile Rails literals together if adapting the artifact to a different renderer URL. The literal also keeps direct host Procfile runs pointed at the paired renderer process.
• Parallelism: The config derives default parallelism from os.availableParallelism() rather than os.cpus() so container CPU quotas are respected when Node can see them. That API requires Node >= 18.14; this artifact Dockerfile pins Node 22.12.0. Set SHAKAPERF_ARTIFACT_PARALLELISM=1 for quota-constrained containers or noisy shared hosts. Fractional values are floored after validation, and invalid or zero values fall back to the detected default.
• Chromium sandbox: Local non-container ShakaPerf runs omit Chromium --no-sandbox by default. Set SHAKAPERF_CHROMIUM_NO_SANDBOX=true only when the ShakaPerf browser runner itself is inside Docker and Chromium cannot use its sandbox.
• ESLint future-proofing: These archived setup TypeScript files remain outside repo ESLint. If they are brought back under ESLint, keep the configs typed through defineConfig instead of adding unsafe-assignment suppressions; derived configs should extend the raw exported config object, not the already-defined default export.
• networkidle dependency: The first-paint test waits for networkidle after installRequestBlocking aborts /webpack/test/js/ requests. If request blocking is changed to stall or continue those requests, the wait can time out.

Generated ShakaPerf instructions followed

These were generated by shaka-perf init and used as instruction docs.

Skill	File
AB servers	SKILL.md
Setup docker servers	SKILL.md, references
Discover AB tests	SKILL.md, references
Assess AB test quality	SKILL.md

Review target

The main review question is whether these tests are the right ShakaPerf shape for RSC FOUC:

1. rsc-fouc.abtest.ts blocks app JS before navigation to isolate server-rendered first paint.
2. natural-first-visible-assertion.abtest.ts does not block JS and asserts the first visible computed style with RAF polling.

The first test gives a clean visual old-vs-fixed screenshot. The second test proves the natural browser path is actually unstyled on the old side.