apps/server/docs/ai-context/verifications/llm-router.md
Verification artifacts for the in-process router. Scope tracked against
docs/plans/2026-05-15-001-feat-llm-tts-router-replacing-knoway-plan.md.
| User path | Code wired | Has fresh evidence |
|---|---|---|
| chat completions happy (router → OpenRouter) | ✅ | ✅ commit 3a88f4225, 2026-05-15 |
| chat completions fallback (key/upstream exhaustion) | ✅ | ❌ unit-test only, needs real-wire run |
TTS speech (Azure) via routeTts | ✅ | ⏳ pending (was knoway-fetch until 2026-05-15) |
TTS speech (dashscope-cosyvoice) via routeTts | ✅ | ⏳ pending |
TTS speech (Volcengine) via routeTts | ✅ | ⏳ pending |
/audio/voices from adapter catalog (no upstream) | ✅ | ⏳ pending (sanity curl) |
/livez | ✅ | ✅ commit cfad87757, 2026-05-15 |
/readyz | ✅ | ✅ commit cfad87757, 2026-05-15 |
The TTS paths and /audio/voices were missing from the prior revision of
this doc (which claimed shipped across U1-U9 while the route handlers
were still hitting GATEWAY_BASE_URL). The router-side wiring landed
2026-05-15; the table above tracks the real evidence backlog so the doc
stops asserting completion ahead of measurement.
LLM_ROUTER_CONFIG with one OpenRouter LLM
upstream, then invokes the router directly to call OpenRouter for a chat
completion. Validates envelope decrypt → configKV load → key rotation →
upstream fetch → response parse on the real wire path.scripts/seed-router-config.ts):
# 1. seed via the admin endpoint — requires an account whose email is in
# ADMIN_EMAILS and is verified.
curl -sS -X POST http://localhost:3000/api/admin/config/router \
-H "Authorization: Bearer $ADMIN_TOKEN" \
-H "Content-Type: application/json" \
-d '{
"mode": "merge",
"slices": [{
"kind": "openrouter",
"modelName": "chat-default",
"overrideModel": "openai/gpt-4o-mini",
"plaintextKey": "<OPENROUTER_KEY>"
}],
"defaults": { "chatModel": "chat-default" }
}' | jq
# 2. exercise the router via the in-process e2e harness.
cd apps/server
pnpm exec dotenvx run --env-file=.env.local -- \
tsx scripts/e2e-llm-router.ts
status 200, JSON body with choices[0].message.content
populated, and E2E PASS — router service successfully called OpenRouter and returned a usable response.3a88f4225, 2026-05-15):
→ calling router.route() with model=chat-default
fetch → POST https://openrouter.ai/api/v1/chat/completions
auth = Bearer sk-or-v1-bb1a38505a7309...
body = {"messages":[{"role":"user","content":"Say \"hello world\" in exactly 3 words, no period."}],"max_tokens":20,"model":"openai/gpt-4o-mini"}
← status 200 (2958ms)
Assistant response:
model: openai/gpt-4o-mini
text: "hello world!"
tokens: prompt=21 completion=3
E2E PASS — router service successfully called OpenRouter and returned a usable response.
3a88f4225, local dev (Node 26, pnpm 10, Postgres
.env.local with a generated
32-byte base64 LLM_ROUTER_MASTER_KEY.GET /livez returns 200 with {status: "live"} even
when external dependencies are degraded. K8s-style flat naming; legacy
/health and nested /healthz/live removed in this revision.curl -i http://localhost:3000/livez{"status":"live"}.cfad87757 + uncommitted route rename, 2026-05-15):
HTTP 200
{"status":"live"}
curl http://localhost:3000/health → HTTP 404 (legacy
endpoint removed); curl http://localhost:3000/healthz/live → HTTP 404
(nested form removed).GET /readyz returns 200 when Postgres + Redis both
respond; 503 otherwise. Gateway-internal key health does NOT block
readiness (R14).curl -i http://localhost:3000/readyz{"status":"ready","checks":{"db":"ok","redis":"ok"}}.cfad87757 + uncommitted route rename, 2026-05-15):
HTTP 200
{"status":"ready","checks":{"db":"ok","redis":"ok"}}
pnpm -F @proj-airi/server exec vitest run3a88f4225, 2026-05-15): 91 tests across 8 files
green; pnpm -F @proj-airi/server typecheck exits 0.POST /api/admin/config/router (see routes/admin/config/router/index.ts).
Covers the write path with audit-log fields on the structured logger,
envelope encryption in-process, and cross-instance invalidation publish.
The plan's ETag-based optimistic concurrency control and HMAC-signed
invalidate payload are still deferred; the config_write and
config_invalid_hmac counters described below remain producerless until
those land.llmRouter.route /
routeTts / listTtsVoices. The LLM_ROUTER_MASTER_KEY env var is
now required (no graceful skip).airi.gen_ai.gateway.*
counters … do not yet have panels for themotel/grafana/dashboards/build.ts generates three router
rows (Health / Trends / Admin Plane) covering the 7 gateway counters
that have live producers: fallback_count, upstream_errors,
key_exhausted, same_status_exhaustion, decrypt_failures,
config_reload, and subscriber_state (producer added in the same
PR — app.ts now emits connected / error / reconnecting from the
configkv:invalidate subscriber). The remaining two counters
(config_write, config_invalid_hmac) intentionally have no panels
because their producer is the ETag + HMAC slice of the U9 admin
endpoint that has not shipped (see the U9 entry above); they will
rejoin Rows 6.5 / 6.7 when that slice lands.
Alert rules (key.exhausted > 0, fallback ratio > 30%, single-key
error ratio > 80%) are still configured through Grafana UI, not
build.ts — IaC-ifying them is a separate follow-up./Users/luoling8192/Git/proj-airi/airi-railway/knoway/
airi-railway/docker-compose.yml
for at least 14 days without a P1+ incident before removing per plan R18.