Model Providers

DB-GPT supports API providers and local runtimes. For a first run, use an API provider unless you specifically want local inference.

:::info Quick pick Not sure which provider to choose? Start with OpenAI or DeepSeek for the fastest setup (API proxy, no GPU needed). Use Ollama if you want to run models locally without complex setup. :::

Provider comparison

How model configuration works

All models are configured in TOML files under configs/. Each config file defines:

• LLM(s) — The language model(s) for chat and reasoning
• Embedding(s) — The embedding model(s) for RAG and knowledge search
• Reranker(s) — Optional re-ranking models for better retrieval

[models]

# Language model
[[models.llms]]
name = "model-name"
provider = "provider-type"
api_key = "your-api-key"

# Embedding model
[[models.embeddings]]
name = "embedding-model-name"
provider = "provider-type"
api_key = "your-api-key"

:::tip Environment variables You can use environment variable syntax in TOML configs: "${env:VARIABLE_NAME:-default_value}". This keeps secrets out of config files. :::

Provider guides

• OpenAI — fastest default for a first setup
• DeepSeek — strong reasoning, OpenAI-compatible proxy pattern
• Qwen (Tongyi) — Alibaba Cloud / DashScope
• SiliconFlow — China-hosted API options
• Ollama — simple local model runtime
• vLLM — GPU-backed local inference for heavier workloads

More providers

DB-GPT also supports additional providers through its proxy system. See Advanced LLM Configuration for:

• Azure OpenAI
• Google Gemini
• Anthropic Claude
• Baichuan
• Spark (iFlyTek)
• And more