Model Configuration Reference

This document explains the JSON field conventions used in conf/models/*.json and conf/all_models.json, and the decimal vs. binary conventions used by different model vendors.

Table of Contents

• JSON Fields
• Field Relationship Diagram
• Migration Note
• Decimal vs. Binary Conventions
• Vendor Breakdown
• Aggregators & Platforms
• How to Add a New Model
• How to Update an Existing Model
• Quick Reference
• Troubleshooting

JSON Fields

Each model entry in a provider JSON file (conf/models/<provider>.json) or in the global catalog (conf/all_models.json) supports the following fields:

Example Entry

{
  "name": "claude-opus-4-8",
  "content_length": 1000000,
  "max_output": 128000,
  "model_types": ["chat", "vision"],
  "thinking": {
    "default_value": true,
    "clear_thinking": true
  },
  "tools": {
    "support": true
  }
}

Thinking Object

{
  "thinking": {
    "default_value": true,    // Whether thinking mode is enabled by default
    "clear_thinking": true    // Whether the API can disable thinking per-request
  }
}

Tools Object

{
  "tools": {
    "support": true           // Whether the model supports function/tool calling
  }
}

Field Relationship Diagram

┌─────────────────────────────────────────────────────┐
│                  content_length                      │
│  (total context window: input + output combined)    │
│                                                     │
│  ┌─────────────────────────────────────────────┐    │
│  │           prompt tokens (input)              │    │
│  └─────────────────────────────────────────────┘    │
│                                                     │
│  ┌─────────────────────────────────────────────┐    │
│  │        max_output (generated tokens)         │    │
│  │  May be fixed OR dynamic (context - input)   │    │
│  └─────────────────────────────────────────────┘    │
└─────────────────────────────────────────────────────┘

• content_length is the total budget (input + output).
• max_output is the generation budget alone.
• For most models, max_output <= content_length. Some vendors set them equal (output can fill the entire window).
• Dynamic max_output: Some models (e.g. Kimi K2.6) define max_output as content_length - input_tokens. In these cases, the configured max_output represents the upper bound; the actual available output decreases as the prompt grows.

Migration Note

Before PR #17807, a single max_tokens field served double duty — it was documented as the context window but often used as the output cap at runtime. The split into content_length + max_output removes this ambiguity:

• Old max_tokens → used only as migration context; do not copy it blindly.
• New content_length → set the vendor-documented context window.
• New max_output → set the vendor-documented generation cap.

Every migrated model must define both content_length and max_output, each taken from the official vendor model specification.

Decimal vs. Binary Conventions

Different vendors express context windows using different numerical conventions. This configuration preserves the exact numbers from each vendor's official documentation, even when vendors disagree on whether "128K" means 128,000 or 131,072.

How to Identify

A quick test: if n & (n-1) == 0, the value is a power of 2 (binary). Otherwise, it is decimal.

Vendor Breakdown

Key Takeaways

1. Never round or convert a value to match a different convention. If Anthropic says 200K, write 200000 — not 2097152 or 262144.
2. OpenAI, Google, Meta, NVIDIA, DeepSeek, Qwen, Kimi, Mistral all use binary (powers of 2).
3. Anthropic, xAI, GLM/Zhipu, MiniMax, Cohere, Baichuan, Amazon use decimal (powers of 10, or vendor-specific round numbers).
4. Some vendors mix conventions within their own catalog (e.g. Anthropic uses decimal for context but binary for output).
5. When in doubt, check the official API documentation linked above. The number in this config should match the vendor's stated limit exactly.

Aggregators & Platforms

The following providers are aggregators — they host models from multiple upstream creators. Their content_length / max_output values inherit from the underlying model, not from a native convention of their own. When updating an aggregator's model entry, refer to the upstream creator's documentation (see table above).

How to Add a New Model

1. Determine the model's content_length (context window) and max_output (generation cap) from the official API documentation.
2. Use the exact number stated — do not convert between decimal and binary.
3. For embedding-type models, also determine batch_size — the provider's documented maximum number of inputs per request — and add it to the entry.
4. Add the entry to the appropriate conf/models/<provider>.json file.
5. If the model is also listed in conf/all_models.json, update that entry too (or add it).
6. Run go test ./internal/entity/models/... to verify the config loads correctly.

How to Update an Existing Model

1. Find the latest official spec from the vendor's documentation.
2. Update content_length and/or max_output to match.
3. If the model is an embedding model, update batch_size to the provider's documented per-request input limit.
4. If the model appears in multiple provider files (e.g. DeepSeek models appear in deepseek.json, ppio.json, qiniu.json), update all copies.
5. Update conf/all_models.json if the model has an entry there.
6. Run go test ./internal/entity/models/... to verify.

Quick Reference

Common model_types Values

Token Count Rule of Thumb

Example: A 10,000-character English document ≈ 3,000 tokens.

Validation Command

go test ./internal/entity/models/...

This loads all provider configs and conf/all_models.json, checking for:

• Valid JSON syntax
• Unique aliases across all models
• Correct field types

Troubleshooting

Duplicate Alias Error

InitProviderManager: duplicate alias "X" for models "A" and "B"

Cause: Two models share the same alias. Aliases must be globally unique.

Fix: In conf/all_models.json, find the conflicting entries and remove or rename the duplicate alias. Also check conf/models/*.json files for the same alias.

Model Not Found

Cause: Model name or alias mismatch between tenant configuration and provider catalog.

Fix: Check both conf/all_models.json (aliases) and the specific conf/models/<provider>.json for the model name.

Context Length Mismatch

Symptom: API returns errors about exceeding context limits.

Cause: content_length in config does not match the vendor's actual limit.

Fix: Verify against official vendor documentation and update accordingly.