ContextQMD
Back to Rocket Chat

Plan: Experimental REST API endpoints

docs/experimental-api-endpoints-plan.md

8.8.011.3 KB
Original Source

Plan: Experimental REST API endpoints

Goal

Allow REST endpoints to ship to production but change or be removed in any release without a major-version bump. The mechanism must be general-purpose (any team can use it), not specific to one feature.

The contract

Endpoints under /api/experimental/... are unstable. They may change shape or be removed in any release, without notice and without a deprecation cycle. No semver promise attaches to this namespace.

The namespace is the contract. A caller hitting /api/experimental/* has opted into instability by the URL alone. /v1 keeps its implicit semver-stability promise, untouched.

Only the new typed API is allowed on experimental routes. Endpoints must be registered with .get() / .post() / .put() / .delete() (with AJV body / query / response validators). The deprecated .addRoute() must not be used — new surface area should not be born on the legacy registration path. This is a documented rule, not a compiler-enforced one: API.experimental is a plain APIClass, and .addRoute() already carries @deprecated everywhere it is reachable.

Why this design (key findings from the current code)

  • createApi({ version }) turns the version string into the URL path segment, so a new instance with version: 'experimental' mounts at /api/experimental/<name> without changing any router internals — the new router still has to be mounted in startRestAPI (see Step 2). See apps/meteor/server/api/api.ts (the API object and createApi) and apps/meteor/server/api/ApiClass.ts (apiPath composition in the APIClass constructor).
  • The typed route methods .get()/.post()/.put()/.delete() are generic over TSubPathPattern extends string (ApiClass.ts, APIClass.method() and its per-verb wrappers) — they are not gated on keyof Endpoints. Routes accumulate outward into the instance's TOperations, read back by ExtractApiClassEndpoints (apps/meteor/server/api/api.ts). So a separate experimental instance works with the type system.
  • PathPattern, Method, Path, and the typed client are all derived from the Endpoints interface (packages/rest-typings/src/index.ts). Keeping experimental paths out of that interface keeps the stable client surface clean and forces explicit opt-in for experimental ones.
  • The deprecation framework already writes x-deprecation-* response headers (writeDeprecationHeader in apps/meteor/server/lib/deprecationWarningLogger.ts). We mirror that pattern for an x-experimental / Warning signal.
  • Auth, permissions, rate limiting, CORS, AJV validation, and metrics all come from createApi + the middleware chain in startRestAPI — experimental endpoints get them for free.

Commit constraint

Each implementation step below ships as exactly one commit. No step is split across multiple commits, and no commit spans more than one step. This keeps the history bisectable, makes each phase independently reviewable and revertable, and maps the PR review 1:1 onto the plan.

  • A step's commit must leave the tree in a compiling, lint-clean state (yarn lint --quiet passes) — partial work is squashed before committing.
  • Commit message subject names the step, e.g. feat(api): add experimental API instance (step 1).
  • If a step turns out to require a prerequisite not in the plan, add it to that step's single commit rather than introducing an out-of-band commit.

Implementation steps

Step 1 — Add the experimental API instance

File: apps/meteor/server/api/api.ts

  1. In the API object literal, add:

    ts
    experimental: createApi({ version: 'experimental', useDefaultAuth: true }),

    Place it between v1 and default.

  2. Add an experimental entry to the API type annotation so it is typed: APIClass<'/experimental'>. Hiding addRoute() from that type was considered and dropped: the typed methods return this, so a restricted surface only holds until the first chained registration, and closing that hole means duplicating every typed signature and widening the route-extraction types that pattern-match APIClass. Not worth it for a rule @deprecated already signals.

  3. Refreshing experimental routes when settings change is a required parity condition, not an optional extra — the contract above promises experimental endpoints get rate limiting "for free", which only holds if the refresh callbacks cover them. The settings.watch(...) callbacks in this file that must also update API.experimental:

    • API_Enable_Rate_Limiter_Limit_Time_Default → reloadRoutesToRefreshRateLimiter()
    • API_Enable_Rate_Limiter_Limit_Calls_Default → reloadRoutesToRefreshRateLimiter()
    • Accounts_CustomFields → setLimitedCustomFields()

    Known gap: the rate-limiter watchers are at parity; the Accounts_CustomFields watcher still updates API.v1 only. That is currently harmless — no experimental endpoint returns user objects — but it must be closed before one does.

Acceptance: API.experimental.get('ping', { ... }, handler) compiles and serves at GET /api/experimental/ping.

Commit (1 of 5): feat(api): add experimental API instance

Step 2 — Mount it in the request pipeline

File: apps/meteor/server/api/api.ts, startRestAPI

  1. Insert .use(API.experimental.router) into the chain, before .use(API.default.router). Order matters: default is the catch-all.
  2. Add a second metricsMiddleware block pointed at API.experimental so experimental traffic is measured. Metrics are the canary used later to decide whether an endpoint is ready for promotion to /v1. Because every block shares the same /api mount, each one needs a guard or a request is sampled more than once: the versioned blocks opt in via basePathRegex, and a catch-all block for API.default (/api/info, /api/docs/json, unmatched /api/*) opts out via excludePathRegex. Without that catch-all block the guards silently drop default-router traffic that used to be sampled.

Acceptance: experimental requests appear in the REST API Prometheus metrics labelled version=experimental — specifically that label, not merely a distinguishable one. A change that only adjusted the path regex while leaving experimental traffic under the v1 version label does not satisfy this. /api/v1/* and default-router traffic each still record exactly one sample under their own label.

Commit (2 of 5): feat(api): mount experimental router and metrics

Step 3 — Runtime "unstable" signal (mirror deprecation headers)

New file: apps/meteor/server/api/v1/middlewares/experimental.ts, colocated with the existing middlewares.

  1. Write a middleware that sets, on every response from the experimental instance: 
    Warning: 299 - "experimental: endpoint is unstable and may change without notice"
x-experimental: true
    x-experimental: true is the supported programmatic signal — clients should detect experimental responses with it. Warning: 299 is a legacy compatibility signal only: warn code 299 came from RFC 7234, which RFC 9111 has since obsoleted along with the Warning header itself, so modern clients are not expected to generate or interpret it. It is emitted for the benefit of tooling that still surfaces it, and may be dropped without it being a breaking change. Model the header-writing on writeDeprecationHeader in apps/meteor/server/lib/deprecationWarningLogger.ts.
  2. Register the middleware on the shared /api mount in startRestAPI, ahead of cors, scoped to /api/experimental by a basePathRegex (same shape as the metrics middleware guard). It cannot live on API.experimental.router: cors answers rejected preflights with 403/405 without calling next(), so a router-scoped middleware would never run for those responses.

Acceptance: every /api/experimental/* response carries both headers — including 404s and CORS preflight rejections; /api/v1/* responses do not.

Commit (3 of 5): feat(api): add experimental unstable-signal middleware

Step 4 — Separate, opt-in SDK typings

File: packages/rest-typings/src/index.ts (+ a new file for the declarations)

  1. Create packages/rest-typings/src/experimental/index.ts (new folder) and declare:
    ts
    export type ExperimentalEndpoints = {
  '/experimental/<name>': {
    GET: (params: ...) => ...;
  };
  // ...
};
    Follow the existing per-resource endpoint style (e.g. packages/rest-typings/src/v1/channels/channels.ts).
  2. Export ExperimentalEndpoints from the package root, but do NOT add it to the interface Endpoints extends ... union. This keeps PathPattern, Method, Path, and the stable typed client free of experimental paths.
  3. Consumers who want typed experimental calls import ExperimentalEndpoints explicitly.

Acceptance: import type { Endpoints } from '@rocket.chat/rest-typings' does NOT include experimental paths; import type { ExperimentalEndpoints } does.

Commit (4 of 5): feat(rest-typings): add opt-in ExperimentalEndpoints

Step 5 — Guardrails (because it is a general mechanism)

  1. No path in both unions. A type-level CI guard for this was considered and dropped: union keys are full paths, so /experimental/x and /v1/x never collide, and the transition window described below does not produce a collision either. Keep the rule in the docs instead — promotion means moving the declaration to a stable *Endpoints type, not leaving a copy behind.
  2. Promotion path: document that stabilizing an endpoint means copying it to /v1 (optionally keeping the experimental path forwarding for a transition window). Removal needs no deprecation cycle — but log removals for courtesy.
  3. Docs/CONTRIBUTING note: state the no-semver guarantee and how to add an experimental endpoint, so the mechanism is discoverable.
  4. OpenAPI/doc generation: decide deliberately whether generated API docs scan only Endpoints (experimental endpoints hidden — probably desirable) or also ExperimentalEndpoints.

Commit (5 of 5): chore(api): add experimental guardrails and docs

Test checklist

  • GET /api/experimental/<name> resolves and returns the x-experimental + Warning headers.
  • /api/v1/* responses are unchanged (no experimental headers).
  • Auth / permissions / rate limiting enforced on an experimental route exactly as on /v1.
  • Endpoints type does not include experimental paths; ExperimentalEndpoints does.
  • Experimental requests show up in REST API metrics.

Files touched (summary)

FileChange
apps/meteor/server/api/api.tsAdd experimental instance, type entry, mount in startRestAPI, metrics blocks
apps/meteor/server/api/v1/middlewares/experimental.ts (new)x-experimental / Warning header middleware
apps/meteor/server/api/v1/middlewares/metrics.tsbasePathRegex / excludePathRegex sampling guards
packages/rest-typings/src/experimental/index.ts (new)ExperimentalEndpoints type, NOT merged into Endpoints
packages/rest-typings/src/index.tsExport ExperimentalEndpoints
docs / CONTRIBUTINGDocument the contract + promotion path