Back to Pi Mono

Adding a New LLM Provider (packages/ai)

.pi/skills/add-llm-provider.md

0.75.52.8 KB
Original Source

Adding a New LLM Provider (packages/ai)

A new provider touches multiple files. Work through these steps in order.

1. Core Types (packages/ai/src/types.ts)

  • Add API identifier to Api type union (e.g. "bedrock-converse-stream").
  • Create options interface extending StreamOptions.
  • Add mapping to ApiOptionsMap.
  • Add provider name to KnownProvider type union.

2. Provider Implementation (packages/ai/src/providers/)

Create a provider file exporting:

  • stream<Provider>() returning AssistantMessageEventStream.
  • streamSimple<Provider>() for SimpleStreamOptions mapping.
  • Provider-specific options interface.
  • Message/tool conversion functions.
  • Response parsing that emits standardized events (text, tool_call, thinking, usage, stop).

3. Provider Exports and Lazy Registration

  • Add a package subpath export in packages/ai/package.json pointing at ./dist/providers/<provider>.js.
  • Add export type re-exports in packages/ai/src/index.ts for provider option types that should remain available from the root entry.
  • Register the provider in packages/ai/src/providers/register-builtins.ts via lazy loader wrappers; do not statically import provider implementation modules there.
  • Add credential detection in packages/ai/src/env-api-keys.ts.

4. Model Generation (packages/ai/scripts/generate-models.ts)

  • Add logic to fetch/parse models from the provider source.
  • Map to the standardized Model interface.

5. Tests (packages/ai/test/)

  • Always add the provider to stream.test.ts with at least one representative model, even if it reuses an existing API impl such as openai-completions.
  • Add the provider to the broader matrix where applicable: tokens.test.ts, abort.test.ts, empty.test.ts, context-overflow.test.ts, unicode-surrogate.test.ts, tool-call-without-result.test.ts, image-tool-result.test.ts, total-tokens.test.ts, cross-provider-handoff.test.ts.
  • For cross-provider-handoff.test.ts, add at least one provider/model pair. If the provider exposes multiple model families (e.g. GPT and Claude), add at least one pair per family.
  • For non-standard auth, create a utility (e.g. bedrock-utils.ts) with credential detection.

6. Coding Agent (packages/coding-agent/)

  • src/core/model-resolver.ts: add default model ID to defaultModelPerProvider.
  • src/core/provider-display-names.ts: add API-key login display name so /login and related UI show the provider for built-in API-key auth.
  • src/cli/args.ts: add env var documentation.
  • README.md: add provider setup instructions.
  • docs/providers.md: add setup instructions, env var, and auth.json key.

7. Documentation

  • packages/ai/README.md: add to providers table, document options/auth, add env vars.
  • packages/ai/CHANGELOG.md: add entry under ## [Unreleased].