.agents/features/benchmark-cli.md
activepieces benchmark load-tests a deployment's sync-webhook path and diagnoses where latency goes, so a self-hosted setup can be compared apples-to-apples against Activepieces' published reference numbers. It builds a webhook → data-mapper → return-response flow, then instead of taking a raw --concurrency, it auto-discovers the deployment's shape (GET /v1/worker-machines) and derives the load from the effective execution slot count. After the run it attributes latency into queue-wait vs service-time from each FlowRun.timeline, validates the machine specs/settings against the recommended production setup, measures CLI→server network RTT, and checks S3 signed-URL upload + latency. A healthy deploy shows ~zero queue-wait by construction; any queue-wait it reports is a real finding (usually driven concurrency > slots).
packages/cli/src/lib/commands/benchmark.ts — the whole command (auth, flow build, discovery, load, attribution, reporting)packages/server/api/src/app/health/health.service.ts + health.module.ts — the GET /v1/health/diagnostics endpoint (platform-admin) added for this feature: server-measured in-region DB/Redis/S3 round-trip latency + effective config + the app tier (every self-registered app replica) + a worker summary (count, per-worker cores/cpu%/ram%), which a cross-region CLI cannot observe. Partial overlap with /v1/health/system and /v1/worker-machines is deliberate: /diagnostics intentionally repeats the worker summary so a single platform-admin call is a self-contained support bundle — the round-trip latency it adds exists on no other route.packages/server/api/src/app/helper/app-machine-cache.ts — the App Instance Registry: apps have no inbound healthcheck to register over (unlike workers), so each app self-registers into a separate Redis hash appMachines on its existing systemSnapshot tick (helper/system-snapshot.ts). Keyed by hostname; list() drops rows untouched for >120s (2 missed ticks) and cleans them in the same read. Write is gated off on Cloud (symmetric with the read), so it never accumulates unread rows on a multi-tenant deployment. Deliberately separate from workerMachines so an app is never counted as a worker execution slot by worker-capacity.packages/core/shared/src/lib/core/health/index.ts — GetDiagnosticsResponse / InfraCheck / DeploymentConfig typesGET /v1/health/system (version + worker skew), GET /v1/health/diagnostics (infra latency + config), GET /v1/worker-machines (specs/slots/sandboxes), GET /v1/worker-machines/queue-metrics (live queue depth), GET /v1/flow-runs?flowId=&createdAfter= (per-run timeline + status outcomes), GET /v1/flags (config flags map)benchmark/run-gke.sh — GKE harness used to produce the published reference numbersCommunity, Enterprise, Cloud. The CLI authenticates with a platform API key only (AP_API_KEY / --api-key + --project-id), same as the other CLI commands — email/password login was removed. The diagnostic endpoints (GET /v1/worker-machines, /v1/health/system, /v1/health/diagnostics) accept a platform API key (PrincipalType.SERVICE), so a single API key gets the full bundle. The infra-diagnostics block is still self-hosted only (FEATURE_DISABLED on AP_EDITION=cloud).
To compare our reference deployment against a customer deployment apples-to-apples, the invariant is concurrency = each deployment's own execution slots (the CLI default), so neither side queues. The comparable numbers are then the server/worker-measured RUN/service-time and the infra round-trip — not a fixed concurrency (which would make a smaller deployment queue and reproduce the original "over-drove their slots" confusion). Edition note: the infra round-trip block is self-hosted only — GET /v1/health/diagnostics returns FEATURE_DISABLED on AP_EDITION=cloud (a Cloud platform-admin is a tenant, not the infra operator), and the CLI degrades gracefully to the rest of the report.
The CLI runs from a different host/region than the API and workers, so every client-side number (autocannon latency, RTT, any CLI→S3 fetch) is network-polluted and is reported as observational only. All authoritative latency is server/worker-measured: the per-run QUEUE/PROVISION/BOOT/RUN split comes from FlowRun.timeline (timed inside the worker via wideEvent.timed in sandbox.ts; RUN includes the end-of-run S3 log backup), and the in-region DB/Redis/S3 round-trip latency comes from GET /v1/health/diagnostics. Caveat surfaced in the report: the QUEUE phase absorbs app↔worker clock skew (can clamp to 0 or inflate), so it is cross-checked against the live BullMQ queue depth.
packages/server/worker/CONTEXT.mdQUEUE / PROVISION / BOOT / RUN) — packages/core/execution/src/lib/flow-run/flow-run.ts