doc/connections/SMOKE-LAB-TUTORIAL.md
A guided, click-by-click walkthrough of the Smoke Lab for a person sitting at the board. You'll turn on the experimental flag, start the deterministic fixture services, drive every integration path (P1–P7) through its full governed lifecycle, and read the results in the matrix and the dashboard card. Nothing here touches a real vendor or a real credential — the OAuth provider and the MCP servers are local fakes.
Every "You should see" below was checked against the real screens; where a button or label is quoted, that's the exact text in the product.
Companion docs: the automated counterparts live in
SMOKE-LAB-BROWSER-RUNNER.md(the agent-driven browser runner) andtests/e2e/smoke-lab.spec.ts(the headless CI mirror). The daily recurring routine that runs the browser smoke for you is described in §8.
The Smoke Lab fail-closes on public deployments. It runs anywhere else — you do
not need a special local_trusted box or any extra environment variables.
Turning on the flag is all the setup there is.
| Requirement | Where |
|---|---|
Smoke Lab experimental flag ON | Instance settings → Experimental |
deployment exposure not public (i.e. not internet-facing) | how the instance was started |
That's it. The everyday dev server works as-is: a local_trusted localhost box, an
authenticated instance behind Tailscale + login (e.g.
http://paperclip-dev:45439), and a pnpm dev server built with
NODE_ENV=production are all fine — those are private, so the Smoke Lab is
available. The auth mode and the Node build target no longer matter; only public
exposure is disallowed (the fake OAuth provider and fixture sidecars must never be
reachable from the open internet).
If the flag is off you'll see the tab say "Smoke Lab is turned off". If you're on a
public instance, API calls return 403 "Smoke lab is only available on private (non-public) deployments" — move to a private instance.
Throughout this tutorial, {PREFIX} is your company's short issue prefix (shown in
the URL bar, e.g. PAP). Replace it in the example paths.
/{PREFIX}/settings/experimental)./{PREFIX}/apps/advanced/smoke-lab). The breadcrumb reads
Apps → Advanced setup → Smoke Lab.[email protected]smoke-password…/api/companies/{companyId}/smoke-lab/oauth/authorize); the provider runs
in-process, so there is no separate port.http://127.0.0.1:<port>/mcp
URL.Which fixture am I in? The Connections list shows both, and the stdio one may be listed first. If you open a fixture's Setup tab and there is no Connect with Smoke OAuth card — only the "Agents can use this app" toggle — you're in the stdio fixture. Go back and open Smoke Lab HTTP MCP fixture for the OAuth steps.
If Start services errors with a
403, re-check §0 — you're on apublic(internet-facing) instance. Any private instance works, including the everyday authenticated dev server.
Each path P1–P7 walks the same governed lifecycle. You drive it from a fixture
connection's pages — a small left-hand menu inside the app with Setup,
Review, Permissions, Activity, Test, and Advanced
(/{PREFIX}/apps/{connectionId}/{tab}).
Two things to know before you start:
todo.list renders as List synthetic todos. The table below
gives both.| Step | What you do | What you should see |
|---|---|---|
| connect | Open the fixture connection (for P1, complete the fake OAuth consent). | Connection shows Connected, with the action count. |
| discover-catalog | Open Permissions. | The action list includes the path's tools (e.g. List synthetic todos). |
| allowed-read | Set the read action to Allowed, then run it from the Test tab. | Decision badge Allowed; the call returns without error. |
| ask-first-write | Set the write action to Ask a human first, then run it from Test. | Decision Ask first; a pending request appears in Review. |
| approve | Review tab → approve the pending write. | The request clears; the call completes. |
| denied-call | Set the blocked action to Off, then run it from Test. | Decision Off; the call is refused with a reason. |
| schema-change / quarantine | Trigger the fixture schema flip (HTTP paths), then Refresh actions on Permissions. | A quarantine pill with the changed entries held back. |
| revoke | Setup → turn off the "Agents can use this app" toggle (or revoke the gateway session for P6). | The connection is paused; a revoked token is cut off (401). |
| audit-evidence | Activity tab. | Audit rows for the allowed, approved, denied, quarantine, and revoke decisions. |
(The results matrix in §6 folds approve into its Ask-first write column, so the matrix shows 8 columns for these 9 steps.)
The per-path tools are:
| read (allowed) | write (ask-first) | denied | schema-flip (quarantine) | |
|---|---|---|---|---|
| HTTP (P1, P2, P5, P6, P7) | todo.list — List synthetic todos | todo.add — Add synthetic todo | email.send — Send outbox email | fixture.schemaFlip — Fixture schema mutation |
| stdio (P3, P4) | time.now — Deterministic time | slow.ping — Slow stdio fixture | crash.now — Crashing stdio fixture | malicious.metadata — Malicious metadata fixture |
This is the richest path — do it by hand once and the rest are variations.
/{PREFIX}/apps), open Smoke Lab HTTP MCP
fixture (not the stdio one — see the callout in §2), then choose Setup.[email protected]). Type the password
smoke-password and click Authorize smoke test app.403.todo.list) and Add synthetic todo (todo.add) appear under
Action permissions.email.send) to
Off, then run it from Test. You should see: an Off badge and a
refusal carrying a reason code.fixture.schemaFlip) — it changes a tool's schema — then click Refresh
actions on the Permissions tab. You should see: a quarantine
pill (on Review and Permissions) — the changed entries are held back until you
explicitly turn them on.Prefer not to click all seven by hand? Use the automated browser smoke — §7 — which performs exactly these steps and leaves you screenshots to read, including a shot of the filled OAuth consent page.
Each path reuses the §3 lifecycle. Only the connect/transport and a couple of tools change.
time.now); the "denied" tool Crashing stdio
fixture (crash.now) is blocked by policy. Its Setup tab has no OAuth
card — just the "Agents can use this app" toggle. Quarantine evidence is
recorded via fixture metadata rather than an HTTP schema flip.Rather than click all seven paths by hand, let the agent-driven runner do it and read the evidence:
SMOKE_BASE=http://127.0.0.1:3251 \
node --experimental-strip-types tests/e2e/smoke-lab-browser-runner.mts
# SMOKE_ONLY=P1,P3 restricts to a subset; omit for the full P1–P7 sweep.
A recurring Paperclip routine — "Daily Smoke Lab integration smoke (P1-P7)" — runs the browser smoke for you every day and:
high-priority issue with the failing step and a
screenshot, assigned to the owning coder, and links it back to the run;It's driven by tests/e2e/smoke-lab-routine.mts. See that file's header and the
routine's own description for the runbook.
/{PREFIX}/dashboard).Ctrl-C) — its embedded
database is disposable.That's the whole loop: flag on → services up → fixtures installed → drive/observe the P1–P7 lifecycle → read the matrix and the dashboard card.