Back to Sanity

Sanity Visual Regression Testing

.agents/skills/sanity-visual-regression/SKILL.md

6.10.14.9 KB
Original Source

Sanity Visual Regression Testing

Visual regression runs on Chromatic, wired via .github/workflows/chromatic.yml. Three snapshot sources, one Chromatic project each:

SourceChromatic projectRepo secretStatus
dev/storybook storiessanityCHROMATIC_PROJECT_TOKEN_STORYBOOKActive
Vitest browser tests (in place)vitest projectCHROMATIC_PROJECT_TOKEN_VITESTDormant (early access) — see activation below
Playwright e2e takeSnapshot()sanity_e2eCHROMATIC_PROJECT_TOKEN_E2EActive, curated opt-in

All checks are non-gating during burn-in (exitZeroOnChanges); merges to main auto-accept baselines. Review diffs on the Chromatic build linked from the PR check.

Quick start: add visual coverage for a component

  1. Add a co-located *.stories.tsx file to the owning package's src tree, usually in the same __tests__ directory as the component or harness. Storybook discovers story files in workspace package src trees. Two patterns:
    • Plain component states (ui-components wrappers, tone/card sentinels): render variants directly — see Button.stories.tsx. Put many variants in one story (a grid) to keep snapshot count low.
    • Studio-context states (form inputs, anything needing workspace/i18n/layers): wrap in the browser-test harness TestWrapper (+ TestForm for form inputs) — see Dialog.stories.tsx and the Portable Text stories. If a vitest browser test already has a *Story.tsx harness, put the story beside it and reuse it (never fork it): the harness stays shared between the test and the story.
  2. Verify locally: pnpm dev:storybook (port 6006), then pnpm --filter sanity-storybook test (every story runs as a vitest browser-mode test via @storybook/addon-vitest).
  3. Push — the Chromatic / Storybook visual tests check snapshots only affected stories (TurboSnap) and links the build for review.

Migration priority: card and tone-related components first (tones cascade through everything), box primitives later. Snapshot the wrapper components in packages/sanity/src/ui-components and vanilla-extract-migrated components (change indicators, DocumentLayout) as sentinels.

Determinism rules for stories

  • Harness stories are deterministic by construction (mock client/workspace, no network).
  • Never render live timestamps, random ids, or unfinished loading states. Chromatic pauses CSS animations automatically.
  • Knobs via parameters.chromatic per story/meta: delay (ms before capture — Portable Text stories use 300 for editor boot), diffThreshold, disableSnapshot: true (story stays in Storybook + addon-vitest but is never snapshotted), modes (viewport/theme matrix — the global 1280×900 desktop mode in preview.tsx matches the vitest browser viewport).

Vitest integration activation runbook (when early access lands)

@chromatic-com/vitest is pre-wired but dormant. To activate:

  1. Create the Vitest-type Chromatic project (requires Chromatic early access enablement).
  2. Add its token as the CHROMATIC_PROJECT_TOKEN_VITEST repo secret.
  3. Done — the Chromatic / Vitest browser visual tests job self-activates on the next run. No code changes. Every browser test's end state becomes a snapshot; the first build is the full baseline.

Afterwards, consider slimming co-located harness-reuse stories that overlap with end-of-test snapshots (keep them if the browsable workbench view is worth the snapshot spend). See REFERENCE.md for local capture runs, takeSnapshot()/configure() usage inside tests (only valid once the plugin is active — takeSnapshot() THROWS in normal runs, so never commit calls to it while the integration is dormant), and cost controls.

Playwright e2e snapshots

e2e/studio-test.ts wraps @chromatic-com/playwright with auto-snapshots disabled globally — the suite runs against per-PR staging datasets (live timestamps, presence, parallel mutations), so blanket end-of-test snapshots would be pure diff noise. Opt in per spec with takeSnapshot() at deterministic moments only. See REFERENCE.md before adding e2e snapshots.

More

REFERENCE.md: architecture and rationale, local Chromatic runs, CLI flags, snapshot cost management, Vercel deployment (studio-storybook.sanity.dev), troubleshooting.