ContextQMD
Back to Qwen3 Coder

Table of Supported Models

qwencoder-eval/tool_calling_eval/berkeley-function-call-leaderboard/SUPPORTED_MODELS.md

latest18.3 KB
Original Source

Table of Supported Models

Below is a comprehensive table of models supported for running leaderboard evaluations. Each model entry indicates whether it supports native Function Calling (FC) or requires a special prompt format to generate function calls. Models marked with 💻 are intended to be hosted locally (using vllm or sglang), while models without the 💻 icon are accessed via API calls. To quickly see all available models, you can also run the bfcl models command.

Function Calling (FC) vs. Prompt Mode

  • Function Calling (FC) Mode:
    Models with native tool/function calling capabilities. For example, OpenAI GPT in FC mode uses the tools section as documented in the OpenAI function calling guide.

  • Prompt Mode:
    Models without native function calling capabilities rely on traditional prompt-based interactions to produce function calls in the desired format, and we supply the function definitions in the system prompt section as opposed to a dedicated tools section. Prompt mode also serve as an alternative approach for models that support FC mode but do not fully leverage its function calling ability (i.e., we only use its normal text generation capability).

Understanding Versioned Models

For model names containing {...}, multiple versions are available. For example, meta-llama/Llama-3.1-{8B,70B}-Instruct means we support both models: meta-llama/Llama-3.1-8B-Instruct and meta-llama/Llama-3.1-70B-Instruct.

Base ModelTypeProviderModel ID on BFCL
Amazon-Nova-Lite-v1:0Function CallingAWSnova-lite-v1.0
Amazon-Nova-Micro-v1:0Function CallingAWSnova-micro-v1.0
Amazon-Nova-Pro-v1:0Function CallingAWSnova-pro-v1.0
Arch-Agent-{1.5B, 3B, 7B, 32B}Function CallingSelf-hosted 💻katanemo/Arch-Agent-{1.5B, 3B, 7B, 32B}
Bielik-11B-v2.3-InstructPromptSelf-hosted 💻speakleash/Bielik-11B-v2.3-Instruct
BitAgent-8BPromptSelf-hosted 💻BitAgent/BitAgent-8B
BitAgent-Bounty-8BFunction CallingSelf-hosted 💻BitAgent/BitAgent-Bounty-8B
Claude-3.5-haiku-20241022Function CallingAnthropicclaude-3-5-haiku-20241022-FC
Claude-3.5-haiku-20241022PromptAnthropicclaude-3-5-haiku-20241022
Claude-Opus-4-20250514Function CallingAnthropicclaude-opus-4-20250514
Claude-Opus-4-20250514PromptAnthropicclaude-opus-4-20250514
Claude-Sonnet-4-20250514Function CallingAnthropicclaude-sonnet-4-20250514
Claude-Sonnet-4-20250514PromptAnthropicclaude-sonnet-4-20250514
CoALM-{8B, 70B, 405B}Function CallingSelf-hosted 💻uiuc-convai/CoALM-{8B,70B,405B}
Command AFunction CallingCoherecommand-a-03-2025-FC
Command R7BFunction CallingCoherecommand-r7b-12-2024-FC
Command-R-PlusFunction CallingCoherecommand-r-plus-FC
DBRX-InstructPromptDatabricksdatabricks-dbrx-instruct
DeepSeek-R1PromptSelf-hosted 💻deepseek-ai/DeepSeek-R1
DeepSeek-R1-0528PromptDeepSeekDeepSeek-R1-0528
DeepSeek-R1-0528Function CallingDeepSeekDeepSeek-R1-0528-FC
DeepSeek-V3-0324Function CallingDeepSeekDeepSeek-V3-0324-FC
DM-Cito-8B-v2PromptMininglampDM-Cito-8B-v2
Falcon3-{1B,3B,7B,10B}-InstructFunction CallingSelf-hosted 💻tiiuae/Falcon3-{1B,3B,7B,10B}-Instruct
FireFunction-v2Function CallingFireworks AIfirefunction-v2-FC
Functionary-{Small,Medium}-v3.1Function CallingMeetKaimeetkai/functionary-{small,medium}-v3.1-FC
Gemini-2.5-FlashFunction CallingGooglegemini-2.5-flash-FC
Gemini-2.5-FlashPromptGooglegemini-2.5-flash
Gemini-2.5-Flash-Lite-Preview-06-17Function CallingGooglegemini-2.5-flash-lite-preview-06-17-FC
Gemini-2.5-Flash-Lite-Preview-06-17PromptGooglegemini-2.5-flash-lite-preview-06-17
Gemini-2.5-ProFunction CallingGooglegemini-2.5-pro-FC
Gemini-2.5-ProPromptGooglegemini-2.5-pro
Gemma-3-{1b,4b,12b,27b}-itPromptSelf-hosted 💻google/gemma-3-{1b,4b,12b,27b}-it
GLM-4-9b-ChatFunction CallingSelf-hosted 💻THUDM/glm-4-9b-chat
GoGoAgentPromptBitAgentBitAgent/GoGoAgent
Gorilla-OpenFunctions-v2Function CallingGorilla LLMgorilla-openfunctions-v2
GPT-4.1-2025-04-14Function CallingOpenAIgpt-4.1-2025-04-14-FC
GPT-4.1-2025-04-14PromptOpenAIgpt-4.1-2025-04-14
GPT-4.1-mini-2025-04-14Function CallingOpenAIgpt-4.1-mini-2025-04-14-FC
GPT-4.1-mini-2025-04-14PromptOpenAIgpt-4.1-mini-2025-04-14
GPT-4.1-nano-2025-04-14Function CallingOpenAIgpt-4.1-nano-2025-04-14-FC
GPT-4.1-nano-2025-04-14PromptOpenAIgpt-4.1-nano-2025-04-14
GPT-4.5-Preview-2025-02-27Function CallingOpenAIgpt-4.5-preview-2025-02-27-FC
GPT-4.5-Preview-2025-02-27PromptOpenAIgpt-4.5-preview-2025-02-27
GPT-4o-2024-11-20Function CallingOpenAIgpt-4o-2024-11-20-FC
GPT-4o-2024-11-20PromptOpenAIgpt-4o-2024-11-20
GPT-4o-mini-2024-07-18Function CallingOpenAIgpt-4o-mini-2024-07-18-FC
GPT-4o-mini-2024-07-18PromptOpenAIgpt-4o-mini-2024-07-18
Granite-20b-FunctionCallingFunction CallingSelf-hosted 💻ibm-granite/granite-20b-functioncalling
Granite-3.1-8B-InstructFunction CallingSelf-hosted 💻ibm-granite/granite-3.1-8b-instruct
Grok-3-betaFunction CallingxAIgrok-3-beta-FC
Grok-3-betaPromptxAIgrok-3-beta
Grok-3-mini-betaFunction CallingxAIgrok-3-mini-beta-FC
Grok-3-mini-betaPromptxAIgrok-3-mini-beta
Haha-7BPromptSelf-hosted 💻ZJared/Haha-7B
Hammer2.1-{7b,3b,1.5b,0.5b}Function CallingSelf-hosted 💻MadeAgents/Hammer2.1-{7b,3b,1.5b,0.5b}
Ling-lite-v1.5PromptAnt GroupLing/ling-lite-v1.5
Llama-3.1-{8B,70B}-InstructFunction CallingSelf-hosted 💻meta-llama/Llama-3.1-{8B,70B}-Instruct-FC
Llama-3.1-{8B,70B}-InstructPromptSelf-hosted 💻meta-llama/Llama-3.1-{8B,70B}-Instruct
Llama-3.1-Nemotron-Ultra-253B-v1PromptNvidianvidia/llama-3.1-nemotron-ultra-253b-v1
Llama-3.2-{1B,3B}-InstructFunction CallingSelf-hosted 💻meta-llama/Llama-3.2-{1B,3B}-Instruct-FC
Llama-3.3-70B-InstructFunction CallingSelf-hosted 💻meta-llama/Llama-3.3-70B-Instruct-FC
Llama-4-Maverick-17B-128E-Instruct-FP8PromptNovita AImeta-llama/llama-4-maverick-17b-128e-instruct-fp8-novita
Llama-4-Maverick-17B-128E-Instruct-FP8Function CallingNovita AImeta-llama/llama-4-maverick-17b-128e-instruct-fp8-FC-novita
Llama-4-Maverick-17B-128E-Instruct-FP8Function CallingSelf-hosted 💻meta-llama/Llama-4-Maverick-17B-128E-Instruct-FP8-FC
Llama-4-Scout-17B-16E-InstructPromptNovita AImeta-llama/llama-4-scout-17b-16e-instruct-novita
Llama-4-Scout-17B-16E-InstructFunction CallingNovita AImeta-llama/llama-4-scout-17b-16e-instruct-FC-novita
Llama-4-Scout-17B-16E-InstructFunction CallingSelf-hosted 💻meta-llama/Llama-4-Scout-17B-16E-Instruct-FC
MiniCPM3-4BPromptSelf-hosted 💻openbmb/MiniCPM3-4B
MiniCPM3-4B-FCFunction CallingSelf-hosted 💻openbmb/MiniCPM3-4B-FC
Ministral-8B-Instruct-2410Function CallingSelf-hosted 💻mistralai/Ministral-8B-Instruct-2410
mistral-large-2411Function CallingMistral AImistral-large-2411-FC
mistral-large-2411PromptMistral AImistral-large-2411
Mistral-Medium-2505PromptMistral AImistral-medium-2505
Mistral-Medium-2505Function CallingMistral AImistral-medium-2505-FC
Mistral-small-2503Function CallingMistral AImistral-small-2503-FC
Mistral-Small-2503PromptMistral AImistral-small-2503
Nemotron-4-340b-instructPromptNvidianvidia/nemotron-4-340b-instruct
Nexusflow-Raven-v2Function CallingNexusflowNexusflow-Raven-v2
O3-2025-04-16PromptOpenAIo3-2025-04-16
O3-2025-04-16Function CallingOpenAIo3-2025-04-16-FC
O4-mini-2025-04-16PromptOpenAIo4-mini-2025-04-16
O4-mini-2025-04-16Function CallingOpenAIo4-mini-2025-04-16-FC
Open-Mistral-Nemo-2407PromptMistral AIopen-mistral-nemo-2407
Open-Mistral-Nemo-2407Function CallingMistral AIopen-mistral-nemo-2407-FC
palmyra-x-004Function CallingWriterpalmyra-x-004
phi-4PromptSelf-hosted 💻microsoft/phi-4
Phi-4-mini-instructPromptSelf-hosted 💻microsoft/Phi-4-mini-instruct
Phi-4-mini-instructFunction CallingSelf-hosted 💻microsoft/Phi-4-mini-instruct-FC
Qwen3-{0.6B,1.7B,4B,8B,14B,32B}PromptAlibaba Cloudqwen3-{0.6b,1.7b,4b,8b,14b,32b}
Qwen3-{0.6B,1.7B,4B,8B,14B,32B}PromptSelf-hosted 💻Qwen/Qwen3-{0.6B,1.7B,4B,8B,14B,32B}
Qwen3-{0.6B,1.7B,4B,8B,14B,32B}Function CallingAlibaba Cloudqwen3-{0.6b,1.7b,4b,8b,14b,32b}-FC
Qwen3-{0.6B,1.7B,4B,8B,14B,32B}Function CallingSelf-hosted 💻Qwen/Qwen3-{0.6B,1.7B,4B,8B,14B,32B}-FC
Qwen3-{30B-A3B,235B-A22B}PromptAlibaba Cloudqwen3-{30b-a3b, 235b-a22b}
Qwen3-{30B-A3B,235B-A22B}PromptSelf-hosted 💻Qwen/Qwen3-{30B-A3B,235B-A22B}
Qwen3-{30B-A3B,235B-A22B}Function CallingAlibaba Cloudqwen3-{30b-a3b, 235b-a22b}-FC
Qwen3-{30B-A3B,235B-A22B}Function CallingSelf-hosted 💻Qwen/Qwen3-{30B-A3B,235B-A22B}-FC
QwQ-32BFunction CallingAlibaba Cloudqwq-32b-FC
QwQ-32BFunction CallingNovita AIqwen/qwq-32b-FC-novita
QwQ-32BPromptAlibaba Cloudqwq-32b
QwQ-32BPromptNovita AIqwen/qwq-32b-novita
Sky-T1-32B-PreviewPromptSelf-hosted 💻NovaSky-AI/Sky-T1-32B-Preview
Snowflake/snowflake-arctic-instructPromptSnowflakesnowflake/arctic
ThinkAgent-1BFunction CallingSelf-hosted 💻ThinkAgents/ThinkAgent-1B
ToolACE-2-8BFunction CallingSelf-hosted 💻Team-ACE/ToolACE-2-8B
watt-tool-{8B,70B}Function CallingSelf-hosted 💻watt-ai/watt-tool-{8B,70B}
xiaoming-14BPromptMininglampxiaoming-14B
xLAM-2-1b-fc-rFunction CallingSelf-hosted 💻Salesforce/xLAM-2-1b-fc-r
xLAM-2-32b-fc-rFunction CallingSelf-hosted 💻Salesforce/xLAM-2-32b-fc-r
xLAM-2-3b-fc-rFunction CallingSelf-hosted 💻Salesforce/xLAM-2-3b-fc-r
xLAM-2-70b-fc-rFunction CallingSelf-hosted 💻Salesforce/Llama-xLAM-2-70b-fc-r
xLAM-2-8b-fc-rFunction CallingSelf-hosted 💻Salesforce/Llama-xLAM-2-8b-fc-r
yi-largeFunction Calling01.AIyi-large-fc

Additional Requirements for Certain Models

  • Gemini Models: For Gemini models, we use the Google AI Studio API for inference. Ensure you have set the GOOGLE_API_KEY in your .env file.

  • Databricks Models: For databrick-dbrx-instruct, you must create an Azure Databricks workspace and set up a dedicated inference endpoint. Provide the endpoint URL via DATABRICKS_AZURE_ENDPOINT_URL in .env.

  • Nova Models (AWS Bedrock): For Nova models, set your AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY in your .env file. Make sure the necessary AWS Bedrock permissions are granted in the us-east-1 region.

For more details and a summary of feature support across different models, see the Berkeley Function Calling Leaderboard blog post.