Quality Gates Reference

This document is the authoritative reference for all CI quality gates in OmniRoute. It describes each gate, what it validates, which CI job it runs in, whether it uses a ratchet baseline or a pass/fail policy, and whether it blocks the build or is advisory.

For a short summary and the allowlist policy, see the "Quality Gates & Ratchets" section in CLAUDE.md. For the critical assessment, maturity classification, and tool-agnostic replication plan of the same system, see the Quality Gate Playbook.

Gate Inventory (~50 scripts)

Scripts live under scripts/check/ (policy gates) and scripts/quality/ (ratchet engine). The CI source of truth is .github/workflows/ci.yml.

Release PR fast-path (quality.yml)

.github/workflows/quality.yml runs on PRs targeting release/**. It keeps contributor branches moving with path-filtered fast gates, plus one advisory production-build signal for code changes:

Forgotten sibling tests report

npm run check:forgotten-sibling-tests reuses the import resolver behind the test-impact map. For every changed production module, it reports deterministic changed module/symbol -> static consumer -> candidate sibling test chains when the candidate test is absent from the pull-request diff. The Markdown summary and JSON result are retained as the forgotten-sibling-tests workflow artifact for calibration before any blocking rollout.

Barrel re-exports and dynamic imports are resolution diagnostics only; they never create a blocking finding. Reviewed exceptions live in config/quality/forgotten-sibling-allowlist.json. Each entry must name the consumer and candidate test, give a specific rationale, and link a GitHub issue or pull request. Malformed entries fail closed. Exceptions cannot suppress a deleted candidate test or a diff that adds .skip/.todo; assertion weakening and other masking remain owned by the independently blocking check:test-masking gate.

Job: lint

Runs on every PR to main. Blocks merge on failure.

Job: quality-gate

Runs after test-coverage. Blocks merge on failure.

Job: quality-extended

Entire job is advisory (continue-on-error: true). The npm-based ratchets run for real; the external scanners install via gh release download and self-skip (exit 0) when a binary is still absent.

Job: docs-sync-strict

Runs on every PR to main. Blocks merge on failure.

Job: i18n-ui-coverage

Needs fetch-depth: 0 — the value-drift gate diffs en.json against the merge base.

check-ui-value-drift — stale-translation gate

Catches the one i18n regression the other gates structurally cannot see: an English value is rewritten and the translations derived from the previous English stay behind, so non-English users keep reading confidently-worded, now-wrong copy.

This shipped for real. oauthModal.googleOAuthWarning was rewritten when the Antigravity login helper landed (#5203); 39 of 43 locales kept text telling operators to "copy the full URL and paste it below" — a flow that cannot complete for that provider. It went unnoticed until #8463 because:

• sync-ui-keys only backfills keys that are absent, never ones that are stale;
• check-ui-keys-coverage counts key presence, so a stale translation scores as covered;
• check-translation-drift tracks the docs/i18n/<locale>/**.md documentation mirrors — it never reads src/i18n/messages/*.json.

Diff-aware, not baseline-backed. It compares en.json at the merge base against the working tree; for every key whose English value changed, any locale still holding an untouched translation is stale. This deliberately freezes pre-existing debt — a diff cannot reveal which old English a long-standing translation came from, so the gate judges only what the current change touches. The alternative (a per-key hash baseline) would cost a ~600 KB generated file, 3× the largest existing baseline, churning on every i18n PR.

Two ways to satisfy it:

1. update the affected translations, or
2. set them to __MISSING__:<new english> — the runtime then serves the corrected English (src/i18n/request.ts::deepMergeFallback, #7258) and the key queues for translation.

If the string's meaning changed, prefer renaming the key: a new key cannot inherit a stale translation. That is the pattern #8463 used.

npm run i18n:check-value-drift          # strict (what CI runs)
npm run i18n:check-value-drift:warn     # report only
BASE_REF=origin/release/vX.Y.Z npm run i18n:check-value-drift

Exits 0 with SKIP reason=base-unresolved when the base catalog cannot be read (shallow clone without the base ref), mirroring check-openapi-breaking.

Job: i18n

Full i18n validation matrix (one job per locale). Entire job is advisory.

Job: pr-test-policy

Runs on pull requests only.

Job: test-vitest

Runs after build. Blocks merge on failure.

Nightly workflows (scheduled, advisory)

These run on a cron schedule (and workflow_dispatch), never on PRs. All are advisory.

Ratchet Baseline (quality-baseline.json)

The ratchet engine (scripts/quality/check-quality-ratchet.mjs) reads quality-baseline.json and compares it against the freshly collected quality-metrics.json. Any metric that regresses beyond its epsilon fails the build.

Current tracked metrics:

To update the baseline after a genuine improvement:

npm run quality:ratchet -- --update
git add quality-baseline.json

The --update flag writes the current measured values into quality-baseline.json. Commit this file alongside the change that improved the metric. A PR that improves a metric without updating the baseline will be caught by --require-tighten (Fase 6A.5, pending implementation).

Test Retry Policy (WS5.4, v3.8.49)

Retry is per-runner, never a global blanket — a blanket retry converts real regressions into invisible flakes:

Target SLOs once flake telemetry lands (WS5.2/5.3): <1% flake rate per test ("fix now" threshold), ≥95% pass rate per pipeline. Industry reference values — recalibrate against our own measurements.

Release-Level Ratchet Drift (WS5.5, v3.8.49)

When a ratchet (file-size, complexity, eslint warnings) regresses on the PURE release tip — i.e. the COMBINATION of merges regressed it, and no single PR reproduces the regression on its own branch — the fix belongs to the release captain, once, on the release branch: prefer extraction/refactor; rebaseline only with the documented justification entry. Never push combination drift onto a contributor PR, and never rebaseline per-PR (that hides real regressions). Discriminate first: reproduce the red against the pure tip in a probe worktree before assuming your PR caused it.

Banking Ratchet Shrinks — the downward direction (#8584)

The ratchet is only half automatic, and it is the wrong half. Raising a cap is a manual JSON edit that takes ten seconds and is the fastest way to unblock a red PR. Lowering one requires someone to run --update and commit the result — and until the bank-ratchet-shrinks job landed, no workflow ran it. The measured consequence (2026-07-25): 18 frozen files already at or under the 800-line new-file cap, the worst at 132× (src/shared/validation/schemas.ts, 19 lines carrying a 2,523 cap); the complexity ceiling walked 1794 → 2169 across ~37 rebaseline notes with exactly one decrease (−1); and "tighten via --update next cycle" written 31 times and honoured once. A cap that outlives the code that earned it silently converts every completed decomposition into a growth allowance for whoever edits the file next.

nightly-release-green.yml → job bank-ratchet-shrinks closes that loop:

Banking is batched rather than per-push because it has no latency requirement (a shrink banked within 8h is fine) while a per-merge run would rebuild the PR branch repeatedly during merge campaigns and pay for a full ESLint walk each time. Detection stays on push (release-green); only banking is batched.

The safety verifier

The job writes to the baselines unattended, so verify-ratchet-bank.mjs is what makes that acceptable. It diffs the post---update tree against HEAD and aborts the job before any commit exists — opening no PR — unless every change is one of:

• a frozen / testFrozen numeric entry lowered or removed
• complexity-baseline.json → count lowered
• quality-baseline.json → metrics.cognitiveComplexity.value lowered

Anything else fails: raising a number, adding an entry, changing cap/testCap, or deleting/rewriting a _rebaseline_* note (those notes are the audit trail for why each ceiling exists and are stored inside the same frozen object as the file entries). A bot that could raise a cap would be strictly worse than the status quo. Regression guard: tests/unit/verify-ratchet-bank.test.ts.

The job never pushes to release/* — a human merges the PR, so a bad measurement cannot land unreviewed.

Allowlist Policy

Every gate that cannot fail on pre-existing violations uses a frozen allowlist (e.g., KNOWN_STALE_DOC_REFS, KNOWN_MISSING, KNOWN_RAW_SQL). The policy is:

Fix the root cause; use the allowlist only when the violation is pre-existing and cannot be fixed in the same PR.

When adding an entry to an allowlist:

1. Include a comment with the justification.
2. Reference the tracking issue (e.g., // #3498 — Phase 2 feature, not yet implemented).
3. Remove the entry in the same PR that fixes the violation — a stale entry that no longer suppresses an active violation is itself a defect (6A.3 stale-enforcement will fail the gate on an orphaned allowlist entry once implemented).

Do not add allowlist entries to make tests pass faster. A green gate with a growing allowlist is a false sense of quality.

When a gate fails on your PR

1. Read the gate output carefully — it tells you exactly which file or symbol violated the rule.
2. Fix the violation — most gates are deterministic filesystem checks that pass as soon as the code is correct.
3. If the violation is pre-existing (i.e., you did not introduce it but the gate now covers it): add an allowlist entry with a justification comment and a tracking issue.
4. If the gate is a ratchet (coverage, ESLint warnings, duplication, complexity): your change made the metric worse. Fix the underlying issue, or (rarely) run npm run quality:ratchet -- --update if the change is intentional and the metric degradation is acceptable — but document why in the PR description.
5. Advisory gates (continue-on-error: true) are informational — they do not block merge but appear in the CI summary. Fix them anyway.

Adding a New Gate

1. Create scripts/check/check-<name>.mjs (or .ts). Policy gates exit 0/1. Ratchet-style gates emit a metric to quality-metrics.json via collect-metrics.mjs.
2. Add "check:<name>": "node scripts/check/check-<name>.mjs" to package.json.
3. Wire it in .github/workflows/ci.yml under the appropriate job (policy → lint or docs-sync-strict; ratchet → quality-gate).
4. If it has an allowlist, apply reportStaleEntries() from scripts/check/lib/allowlist.mjs so stale entries are detected automatically.
5. Write a test in tests/unit/build/ covering the gate's detection logic.
6. Update this document (add a row to the relevant job table).

Agent tooling: LSP-in-the-loop (opt-in)

Beyond the CI gates, OmniRoute ships an opt-in agent-lsp scaffold (a project-level .mcp.json, Fase 7 Task 15). Create .mcp.json to expose a TypeScript language server to coding agents, so they resolve symbols / diagnostics before writing code — a compile-before-claim companion to typecheck:core that cuts "invented symbol" errors at the source. It is intentionally not auto-loaded (you pick and verify the MCP↔LSP bridge); a broken entry only logs a connection error and never breaks sessions.

Rationalization Backlog (ROI review — Fase 9 Onda 3)

This inventory was reconciled against ci.yml on 2026-06-17 (the prior version omitted audit:deps, check:tracked-artifacts, check:lockfile, check:licenses, check:dead-code, check:cognitive-complexity, check:type-coverage, check:codeql-ratchet, check:pr-evidence). An ROI review of the reconciled set identified the following rationalization candidates. The merges are mechanical CI changes; the flips/drops are policy decisions reserved for the operator. Nothing below is applied yet.

Also undocumented above (advisory, low signal): the docs-lint job (markdownlint + Vale, whole job continue-on-error) and the standalone scanner workflows semgrep.yml / codeql.yml / scorecard.yml. semgrepFindings: 0 is in quality-baseline.json but is not wired to a blocking ratchet in ci.yml — the metric is currently orphaned.

Merge / dedup (mechanical, lower risk)

Each candidate was validated against the live gate state on 2026-06-17 (trust-but-verify); several "obvious" merges turned out to hide debt and are not clean drop-ins.

• check:docs-sync runs twice — standalone in the lint job and again inside check:docs-all (docs-sync-strict) and the husky pre-commit hook. ✅ DONE — standalone lint invocation removed.
• CVE scanning — ❌ NOT a clean merge. audit:deps hard-fails on any high/critical CVE; check:vuln-ratchet (osv) only fails on a regression vs baseline (currently 1 MODERATE). Different semantics — dropping audit:deps would lose the absolute high/critical gate. Keep both.
• Cycle detection — ❌ NOT a clean merge. check:circular-deps (dpdm) reports 91 cycles (that is why it is advisory); it cannot be promoted to blocking without first resolving them, and it has a broader scope than the green, curated check:cycles. Keep check:cycles blocking; resolving the 91 dpdm cycles is its own backlog.
• Complexity — ✅ DONE (check:complexity-ratchets / eslint.complexity-ratchets.config.mjs): one ESLint walk, counts by ruleId so cyclomatic+max-lines and cognitive baselines stay independent; individual check:complexity / check:cognitive-complexity remain for local --update.
• /api anti-hallucination — ✅ DONE (check:api-docs-refs + scripts/check/lib/apiRoutes.mjs): one FS inventory of src/app/api, openapi-routes + docs-symbols still report independently; individuals remain for local runs.
• check:node-runtime runs in 11 jobs — ⚠️ low ROI. Each is a separate runner and the check is <1s; total savings ~10s, against losing a cheap per-job guard. Not worth the churn.
• typecheck:noimplicit:core on CI lint — ✅ removed from lint job (was advisory continue-on-error); blocking type surface is typecheck:core + check:type-coverage. Local script retained.

Flip / decide (operator policy)

• check:openapi-security-tiers (advisory) — ❌ NOT cleanly flippable. It exits 0 but warns that several traffic-inspector routes under LOCAL_ONLY_API_PREFIXES lack the x-loopback-only: true annotation. Enforcing it requires adding those annotations to openapi.yaml first.
• typecheck:noimplicit:core (advisory) — largely subsumed by the blocking check:type-coverage ratchet. Flip to a ratchet or drop the redundant second tsc pass.
• test:vitest:ui (now blocking) — pre-existing failures are explicitly excluded in vitest.config.ts with // #8618 tracking comments; new failures fail the job.
• check:secrets (gitleaks, blocking ratchet frozen at 3 documented false-positives) — allowlist the 3 to reach 0, or demote to advisory. Overlaps GitHub native secret-scanning + check:public-creds.
• check:pr-evidence (blocking, greps PR-body prose) — high false-positive risk; weakens Hard Rule #18 enforcement if dropped, so this is a genuine policy call.
• semgrep (advisory standalone) — overlaps CodeQL for the OWASP families; wire its baseline to a ratchet or drop.

Related Documentation

• Supply-chain (provenance, SBOM, Trivy, Scorecard): docs/security/SUPPLY_CHAIN.md