Custom Models

Add custom providers and models (Ollama, vLLM, LM Studio, proxies) via ~/.pi/agent/models.json.

Table of Contents

• Minimal Example
• Full Example
• Supported APIs
• Provider Configuration
• Model Configuration
• Overriding Built-in Providers
• Per-model Overrides
• Anthropic Messages Compatibility
• OpenAI Compatibility

Minimal Example

For local models (Ollama, LM Studio, vLLM), only id is required per model:

{
  "providers": {
    "ollama": {
      "baseUrl": "http://localhost:11434/v1",
      "api": "openai-completions",
      "apiKey": "ollama",
      "models": [
        { "id": "llama3.1:8b" },
        { "id": "qwen2.5-coder:7b" }
      ]
    }
  }
}

The apiKey value is a placeholder because Ollama ignores it. pi still treats models as requiring auth before they appear in /model, so keyless local servers should keep a dummy value, save a key for that provider with /login, or pass --api-key when selecting the model.

Some OpenAI-compatible servers do not understand the developer role used for reasoning-capable models. For those providers, set compat.supportsDeveloperRole to false so pi sends the system prompt as a system message instead. If the server also does not support reasoning_effort, set compat.supportsReasoningEffort to false too.

You can set compat at the provider level to apply to all models, or at the model level to override a specific model. This commonly applies to Ollama, vLLM, SGLang, and similar OpenAI-compatible servers.

{
  "providers": {
    "ollama": {
      "baseUrl": "http://localhost:11434/v1",
      "api": "openai-completions",
      "apiKey": "ollama",
      "compat": {
        "supportsDeveloperRole": false,
        "supportsReasoningEffort": false
      },
      "models": [
        {
          "id": "gpt-oss:20b",
          "reasoning": true
        }
      ]
    }
  }
}

Full Example

Override defaults when you need specific values:

{
  "providers": {
    "ollama": {
      "baseUrl": "http://localhost:11434/v1",
      "api": "openai-completions",
      "apiKey": "ollama",
      "models": [
        {
          "id": "llama3.1:8b",
          "name": "Llama 3.1 8B (Local)",
          "reasoning": false,
          "input": ["text"],
          "contextWindow": 128000,
          "maxTokens": 32000,
          "cost": { "input": 0, "output": 0, "cacheRead": 0, "cacheWrite": 0 }
        }
      ]
    }
  }
}

The file reloads each time you open /model. Edit during session; no restart needed.

Google AI Studio Example

Use google-generative-ai with a baseUrl to add models from Google AI Studio, including custom Gemma 4 entries:

{
  "providers": {
    "my-google": {
      "baseUrl": "https://generativelanguage.googleapis.com/v1beta",
      "api": "google-generative-ai",
      "apiKey": "$GEMINI_API_KEY",
      "models": [
        {
          "id": "gemma-4-31b-it",
          "name": "Gemma 4 31B",
          "input": ["text", "image"],
          "contextWindow": 262144,
          "reasoning": true
        }
      ]
    }
  }
}

The baseUrl is required when adding custom models to the google-generative-ai API type.

Supported APIs

Set api at provider level (default for all models) or model level (override per model).

Provider Configuration

For providers with models, non-built-in provider configs need baseUrl and an api value at either provider or model level. apiKey is not required to load the file: models become available when auth is configured through /login/auth.json, CLI --api-key, or provider apiKey. If no auth is configured, the models load but stay unavailable in /model and --list-models.

Value Resolution

The apiKey and headers fields support command execution, environment interpolation, and literals:

• Shell command: "!command" at the start executes the whole value as a command and uses stdout
  jsonCopy

  "apiKey": "!security find-generic-password -ws 'anthropic'"
  "apiKey": "!op read 'op://vault/item/credential'"
  

• Environment interpolation: "$ENV_VAR" or "${ENV_VAR}" uses the value of the named variable. Interpolation works inside larger literals.
  jsonCopy

  "apiKey": "$MY_API_KEY"
  "apiKey": "${KEY_PREFIX}_${KEY_SUFFIX}"
  

  $FOO_BAR is the variable FOO_BAR; use ${FOO}_BAR when BAR is literal text. Missing environment variables make the value unresolved.
• Escapes: "$$" emits a literal "$"; "$!" emits a literal "!" without triggering command execution.
  jsonCopy

  "apiKey": "$$literal-dollar-prefix"
  "apiKey": "$!literal-bang-prefix"
  

• Literal value: Used directly. Plain uppercase strings such as MY_API_KEY are literals; use $MY_API_KEY for environment variables.
  jsonCopy

  "apiKey": "sk-..."
  

For models.json, shell commands are resolved at request time. pi intentionally does not apply built-in TTL, stale reuse, or recovery logic for arbitrary commands. Different commands need different caching and failure strategies, and pi cannot infer the right one.

If your command is slow, expensive, rate-limited, or should keep using a previous value on transient failures, wrap it in your own script or command that implements the caching or TTL behavior you want.

/model availability checks use configured auth presence and do not execute shell commands.

Custom Headers

{
  "providers": {
    "custom-proxy": {
      "baseUrl": "https://proxy.example.com/v1",
      "apiKey": "$MY_API_KEY",
      "api": "anthropic-messages",
      "headers": {
        "x-portkey-api-key": "$PORTKEY_API_KEY",
        "x-secret": "!op read 'op://vault/item/secret'"
      },
      "models": [...]
    }
  }
}

Model Configuration

A cost tier supplies a complete alternate rate set and applies to the full request when total input usage (input + cacheRead + cacheWrite) exceeds inputTokensAbove. When multiple tiers match, the highest threshold wins.

{
  "cost": {
    "input": 5,
    "output": 30,
    "cacheRead": 0.5,
    "cacheWrite": 6.25,
    "tiers": [
      {
        "inputTokensAbove": 272000,
        "input": 10,
        "output": 45,
        "cacheRead": 1,
        "cacheWrite": 12.5
      }
    ]
  }
}

Current behavior:

• /model, --list-models, and the interactive footer display entries by model id.
• The configured name is used for model matching and secondary model detail text. It does not replace the footer/status-bar model id.

Sampling Parameters

samplingParams is a free-form object merged verbatim into every request body for the model, after the fields pi sets itself, so its keys win. Use it to send sampling parameters pi does not model — including server-specific ones like llama.cpp's min_p or vLLM's top_k:

{
  "id": "deepseek-v4-flash",
  "samplingParams": {
    "temperature": 1.0,
    "top_p": 0.95,
    "top_k": 0,
    "min_p": 0.0
  }
}

Only OpenAI-compatible APIs apply it (openai-completions, openai-responses, azure-openai-responses); other APIs ignore it. Keys override pi's named request fields (for example a temperature key here beats the request-level temperature), so prefer it as the single source of sampling truth for a model. In modelOverrides, samplingParams merges per key with the base model's value.

Thinking Level Map

Use thinkingLevelMap on a model to describe model-specific thinking controls. Keys are pi thinking levels: off, minimal, low, medium, high, xhigh, max. Maps may contain holes; for example, a model can expose high and max without exposing xhigh.

Values are tristate:

Example for a model that only supports off, high, and max reasoning:

{
  "id": "deepseek-v4-pro",
  "reasoning": true,
  "thinkingLevelMap": {
    "minimal": null,
    "low": null,
    "medium": null,
    "high": "high",
    "xhigh": null,
    "max": "max"
  }
}

Example for a model where thinking cannot be disabled:

{
  "id": "always-thinking-model",
  "reasoning": true,
  "thinkingLevelMap": {
    "off": null
  }
}

Migration: older configs that used compat.reasoningEffortMap should move that mapping to model-level thinkingLevelMap. Use null for levels that should not appear in the UI.

Overriding Built-in Providers

Route a built-in provider through a proxy without redefining models:

{
  "providers": {
    "anthropic": {
      "baseUrl": "https://my-proxy.example.com/v1"
    }
  }
}

All built-in Anthropic models remain available. Existing OAuth or API key auth continues to work.

To merge custom models into a built-in provider, include the models array:

{
  "providers": {
    "anthropic": {
      "baseUrl": "https://my-proxy.example.com/v1",
      "apiKey": "$ANTHROPIC_API_KEY",
      "api": "anthropic-messages",
      "models": [...]
    }
  }
}

Merge semantics:

• Built-in models are kept.
• Custom models are upserted by id within the provider.
• If a custom model id matches a built-in model id, the custom model replaces that built-in model.
• If a custom model id is new, it is added alongside built-in models.

Per-model Overrides

Use modelOverrides to customize built-in models and matching extension-registered models without replacing the provider's full model list.

{
  "providers": {
    "openrouter": {
      "modelOverrides": {
        "anthropic/claude-sonnet-4": {
          "name": "Claude Sonnet 4 (Bedrock Route)",
          "compat": {
            "openRouterRouting": {
              "only": ["amazon-bedrock"]
            }
          }
        }
      }
    }
  }
}

modelOverrides supports these fields per model: name, reasoning, thinkingLevelMap, input, cost (partial), contextWindow, maxTokens, samplingParams (merged per key), headers, compat.

Direct OpenAI GPT-5.6 Sol, Terra, and Luna default to a 272000 context window so requests remain within OpenAI's short-context pricing tier. To opt into OpenAI's 1.05M context window, increase it for each model you use:

{
  "providers": {
    "openai": {
      "modelOverrides": {
        "gpt-5.6-sol": {
          "contextWindow": 1050000
        }
      }
    }
  }
}

The override preserves the built-in pricing metadata. Requests with more than 272K total input tokens use GPT-5.6's long-context rates for the entire request. Apply the same override to gpt-5.6-terra or gpt-5.6-luna when needed.

Behavior notes:

• modelOverrides are applied to built-in provider models and matching extension-registered provider models.
• Unknown model IDs are ignored.
• You can combine provider-level baseUrl/headers with modelOverrides.
• Overriding name changes model matching and secondary detail text only; the footer and primary model lists continue to show the model id.
• If models is also defined for a provider, custom models are merged after built-in overrides. A custom model with the same id replaces the overridden built-in model entry.

Anthropic Messages Compatibility

For providers or proxies using api: "anthropic-messages", use compat to control Anthropic-specific request compatibility.

By default pi sends per-tool eager_input_streaming: true. If a proxy or Anthropic-compatible backend rejects that field, set supportsEagerToolInputStreaming to false. Pi will omit tools[].eager_input_streaming and send the legacy fine-grained-tool-streaming-2025-05-14 beta header for tool-enabled requests instead.

Some Anthropic models require adaptive thinking (thinking.type: "adaptive" plus output_config.effort) instead of the legacy budget-based thinking payload. Built-in models set this automatically. For custom providers or aliases that route to those models, set forceAdaptiveThinking to true.

Some Anthropic-compatible providers emit thinking blocks with empty signatures and still expect them on replay. Set allowEmptySignature to true only for those providers; real Anthropic rejects empty thinking signatures.

Built-in Anthropic models enable supportsStrictTools in their model metadata. Custom Anthropic-compatible models must set it to true when their endpoint accepts strict JSON-schema tool definitions.

{
  "providers": {
    "anthropic-proxy": {
      "baseUrl": "https://proxy.example.com",
      "api": "anthropic-messages",
      "apiKey": "$ANTHROPIC_PROXY_KEY",
      "compat": {
        "supportsEagerToolInputStreaming": false,
        "supportsLongCacheRetention": true,
        "forceAdaptiveThinking": true,
        "allowEmptySignature": true
      },
      "models": [
        {
          "id": "claude-opus-4-7",
          "reasoning": true,
          "input": ["text", "image"]
        }
      ]
    }
  }
}

OpenAI Compatibility

For providers with partial OpenAI compatibility, use the compat field.

• Provider-level compat applies defaults to all models under that provider.
• Model-level compat overrides provider-level values for that model.

{
  "providers": {
    "local-llm": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "compat": {
        "supportsUsageInStreaming": false,
        "maxTokensField": "max_tokens"
      },
      "models": [...]
    }
  }
}

openrouter uses reasoning: { effort }. together uses reasoning: { enabled } and also reasoning_effort when supportsReasoningEffort is enabled. qwen uses top-level enable_thinking. Use qwen-chat-template for local Qwen-compatible servers that require chat_template_kwargs.enable_thinking and preserve_thinking. Use chat-template for vLLM/Hugging Face chat templates that need configurable chat_template_kwargs, such as chatTemplateKwargs: { "thinking": { "$var": "thinking.enabled" } } for DeepSeek V3.x templates. Use thinkingFormat: "baseten" with chatTemplateArgs for providers that expose toggle controls through chat_template_args and optionally support top-level reasoning_effort.

cacheControlFormat: "anthropic" is for OpenAI-compatible providers that expose Anthropic-style prompt caching through cache_control markers on text content and tool definitions.

Example:

{
  "providers": {
    "openrouter": {
      "baseUrl": "https://openrouter.ai/api/v1",
      "apiKey": "$OPENROUTER_API_KEY",
      "api": "openai-completions",
      "models": [
        {
          "id": "openrouter/anthropic/claude-3.5-sonnet",
          "name": "OpenRouter Claude 3.5 Sonnet",
          "compat": {
            "openRouterRouting": {
              "allow_fallbacks": true,
              "require_parameters": false,
              "data_collection": "deny",
              "zdr": true,
              "enforce_distillable_text": false,
              "order": ["anthropic", "amazon-bedrock", "google-vertex"],
              "only": ["anthropic", "amazon-bedrock"],
              "ignore": ["gmicloud", "friendli"],
              "quantizations": ["fp16", "bf16"],
              "sort": {
                "by": "price",
                "partition": "model"
              },
              "max_price": {
                "prompt": 10,
                "completion": 20
              },
              "preferred_min_throughput": {
                "p50": 100,
                "p90": 50
              },
              "preferred_max_latency": {
                "p50": 1,
                "p90": 3,
                "p99": 5
              }
            }
          }
        }
      ]
    }
  }
}

Vercel AI Gateway example:

{
  "providers": {
    "vercel-ai-gateway": {
      "baseUrl": "https://ai-gateway.vercel.sh/v1",
      "apiKey": "$AI_GATEWAY_API_KEY",
      "api": "openai-completions",
      "models": [
        {
          "id": "moonshotai/kimi-k2.5",
          "name": "Kimi K2.5 (Fireworks via Vercel)",
          "reasoning": true,
          "input": ["text", "image"],
          "cost": { "input": 0.6, "output": 3, "cacheRead": 0, "cacheWrite": 0 },
          "contextWindow": 262144,
          "maxTokens": 262144,
          "compat": {
            "vercelGatewayRouting": {
              "only": ["fireworks", "novita"],
              "order": ["fireworks", "novita"]
            }
          }
        }
      ]
    }
  }
}