docs/user/multi-custom-models_en.md
Prompt Optimizer now supports configuring unlimited number of custom models, allowing you to use multiple local models or self-hosted API services simultaneously.
Use the following format to configure multiple custom models:
VITE_CUSTOM_API_KEY_<suffix>=your-api-key # Required
VITE_CUSTOM_API_BASE_URL_<suffix>=your-base-url # Required
VITE_CUSTOM_API_MODEL_<suffix>=your-model-name # Required
VITE_CUSTOM_API_PARAMS_<suffix>=json-object-string # Optional extra request parameters
# Qwen 2.5 Model
VITE_CUSTOM_API_KEY_qwen25=ollama-dummy-key
VITE_CUSTOM_API_BASE_URL_qwen25=http://localhost:11434/v1
VITE_CUSTOM_API_MODEL_qwen25=qwen2.5:7b
# Qwen 3 Model
VITE_CUSTOM_API_KEY_qwen3=ollama-dummy-key
VITE_CUSTOM_API_BASE_URL_qwen3=http://localhost:11434/v1
VITE_CUSTOM_API_MODEL_qwen3=qwen3:8b
# Claude API
VITE_CUSTOM_API_KEY_claude=sk-ant-your-claude-key
VITE_CUSTOM_API_BASE_URL_claude=https://api.anthropic.com/v1
VITE_CUSTOM_API_MODEL_claude=claude-3-sonnet-20240229
VITE_CUSTOM_API_PARAMS_claude={"temperature":0.3,"top_p":0.8}
# Custom OpenAI Compatible Service
VITE_CUSTOM_API_KEY_custom=your-custom-api-key
VITE_CUSTOM_API_BASE_URL_custom=https://api.example.com/v1
VITE_CUSTOM_API_MODEL_custom=custom-model-name
VITE_CUSTOM_API_PARAMS_custom={"temperature":0.7,"top_p":0.9,"max_tokens":4096}
# Local model
VITE_CUSTOM_API_KEY_local=dummy-key
VITE_CUSTOM_API_BASE_URL_local=http://localhost:11434/v1
VITE_CUSTOM_API_MODEL_local=llama2:7b
# Cloud service
VITE_CUSTOM_API_KEY_cloud=real-api-key
VITE_CUSTOM_API_BASE_URL_cloud=https://api.service.com/v1
VITE_CUSTOM_API_MODEL_cloud=gpt-4-turbo
# Development environment
VITE_CUSTOM_API_KEY_dev=dev-api-key
VITE_CUSTOM_API_BASE_URL_dev=https://dev-api.example.com/v1
VITE_CUSTOM_API_MODEL_dev=dev-model
VITE_CUSTOM_API_PARAMS_dev={"temperature":0.4}
VITE_CUSTOM_API_PARAMS_<suffix> is useful when you need to:
temperature, top_p, or max_tokenschat_template_kwargsExample JSON payload:
{
"chat_template_kwargs": {
"enable_thinking": true
},
"temperature": 0.6,
"top_p": 0.95,
"max_tokens": 16384
}
Notes:
PARAMS must be a JSON object stringmodel, messages, and stream are reserved and will be ignored automaticallytimeout is allowed and can be used to override request timeout behaviorCreate .env.local file in project root:
# Basic models
VITE_OPENAI_API_KEY=your-openai-key
VITE_GEMINI_API_KEY=your-gemini-key
# Custom models
VITE_CUSTOM_API_KEY_ollama=dummy-key
VITE_CUSTOM_API_BASE_URL_ollama=http://localhost:11434/v1
VITE_CUSTOM_API_MODEL_ollama=qwen2.5:7b
VITE_CUSTOM_API_PARAMS_ollama={"temperature":0.7}
docker run -d -p 8081:80 \
-e VITE_OPENAI_API_KEY=your-openai-key \
-e VITE_CUSTOM_API_KEY_ollama=dummy-key \
-e VITE_CUSTOM_API_BASE_URL_ollama=http://host.docker.internal:11434/v1 \
-e VITE_CUSTOM_API_MODEL_ollama=qwen2.5:7b \
-e 'VITE_CUSTOM_API_PARAMS_ollama={"temperature":0.7}' \
-e VITE_CUSTOM_API_KEY_claude=your-claude-key \
-e VITE_CUSTOM_API_BASE_URL_claude=https://api.anthropic.com/v1 \
-e VITE_CUSTOM_API_MODEL_claude=claude-3-sonnet \
-e 'VITE_CUSTOM_API_PARAMS_claude={"temperature":0.3,"top_p":0.8}' \
--restart unless-stopped \
--name prompt-optimizer \
linshen/prompt-optimizer
Create .env file:
VITE_OPENAI_API_KEY=your-openai-key
VITE_CUSTOM_API_KEY_ollama=dummy-key
VITE_CUSTOM_API_BASE_URL_ollama=http://host.docker.internal:11434/v1
VITE_CUSTOM_API_MODEL_ollama=qwen2.5:7b
VITE_CUSTOM_API_PARAMS_ollama={"temperature":0.7}
VITE_CUSTOM_API_KEY_qwen3=your-qwen3-key
VITE_CUSTOM_API_BASE_URL_qwen3=http://host.docker.internal:11434/v1
VITE_CUSTOM_API_MODEL_qwen3=qwen3:8b
VITE_CUSTOM_API_PARAMS_qwen3={"temperature":0.6,"top_p":0.95}
Run with environment file:
docker run -d -p 8081:80 --env-file .env \
--restart unless-stopped \
--name prompt-optimizer \
linshen/prompt-optimizer
Modify docker-compose.yml to add env_file configuration:
services:
prompt-optimizer:
image: linshen/prompt-optimizer:latest
env_file:
- .env # Read environment variables from .env file
ports:
- "8081:80"
restart: unless-stopped
Then configure variables in .env file (same as Method 2).
Desktop version automatically reads environment variables from system or .env.local file.
MCP server supports all custom model configurations and automatically maps environment variables.
The system automatically converts suffix names to friendly display names:
| Suffix | Display Name |
|---|---|
qwen25 | Qwen25 |
claude_local | Claude Local |
my_model_v2 | My Model V2 |
test123 | Test123 |
Recommended:
ollama - Local Ollama serviceclaude - Claude APIqwen25 - Qwen 2.5 modellocal_llama - Local Llama modeldev_model - Development environment modelNot Recommended:
model.v1 - Contains dotsmy model - Contains spacestest@api - Contains special charactersThe system automatically validates configurations:
A: Check the following:
A: Verify:
A: Check browser console or application logs for:
[scanCustomModelEnvVars] Found X valid custom models: [model1, model2, ...]
[generateDynamicModels] Generated model: custom_modelname (Display Name)
If you configured PARAMS, inspect the outgoing request payload in browser DevTools to verify the extra fields are present.
A: Theoretically unlimited, but recommend reasonable configuration based on actual needs to avoid UI clutter.
A: Remove corresponding environment variables and restart the application.
A: Yes, custom models support all features including prompt optimization, comparison testing, etc.
A: Use different suffixes for different environments:
# Production
VITE_CUSTOM_API_KEY_prod=prod-key
VITE_CUSTOM_API_BASE_URL_prod=https://prod-api.com/v1
VITE_CUSTOM_API_MODEL_prod=prod-model
# Development
VITE_CUSTOM_API_KEY_dev=dev-key
VITE_CUSTOM_API_BASE_URL_dev=https://dev-api.com/v1
VITE_CUSTOM_API_MODEL_dev=dev-model
custom_<suffix>v1.2.6: Code quality fixes and performance optimization
v1.4.0: Added multiple custom models support