doc/development/ai_features/foundational_chat_agents.md
Foundational agents are specialized agents that are created and maintained by GitLab, providing more accurate responses for specific use cases. These agents are available by default on any place chat and GitLab Duo chat are available, including groups, and are supported on GitLab Duo Self-Hosted.
There are two ways of creating a foundational agent, using the AI Catalog or GitLab Duo Workflow Service. AI Catalog provides a user-friendly interface, and it is the preferred approach, but writing a definition on GitLab Duo Workflow Service provides more flexibility for complex cases.
Create your agent on the AI Catalog, and note its ID. Make sure the agent is set to public. Example: Planner Agent has ID 348.
Agents created on the AI Catalog need to be bundled into GitLab Duo Workflow Service, so they can be available to self-hosted setups that do not have access to our SaaS. To achieve this, open an MR to GitLab Duo Workflow Service adding the ID of the agent:
# https://gitlab.com/gitlab-org/modelops/applied-ml/code-suggestions/ai-assist/-/blob/main/Dockerfile
- RUN poetry run fetch-foundational-agents "https://gitlab.com" "$GITLAB_TOKEN" "348" \
+ RUN poetry run fetch-foundational-agents "https://gitlab.com" "$GITLAB_TOKEN" "duo_planner:348,<agent-reference>:<agent-catalog-id>" \
The command above can also be executed locally for testing purposes. Agent reference must be lowercase without spaces (example: 'test_agent').
To make the agent be selectable, add it to the FoundationalChatAgentsDefinitions.rb.
Use the reference used in the Dockerfile:
{
id: 3,
reference: '<agent-reference>',
version: 'experimental',
name: 'Test Agent',
description: "An agent for testing"
}
Update user facing documentation.
Create a flow configuration file in /duo_workflow_service/agent_platform/v1/flows/configs/ (located either on your GDK under PATH-TO-YOUR-GDK/gdk/gitlab-ai-gateway or on the ai-assist repository):
File: /duo_workflow_service/agent_platform/v1/flows/configs/foundational_pirate_agent.yml
version: "v1"
environment: chat-partial
components:
- name: "foundational_pirate_agent"
type: AgentComponent
prompt_id: "foundational_pirate_agent_prompt"
inputs:
- from: "context:goal"
as: "goal"
- from: "context:project_id"
as: "project_id"
toolset: []
ui_log_events: []
prompts:
- name: Foundational Pirate Agent
prompt_id: "foundational_pirate_agent_prompt"
model:
params:
model_class_provider: anthropic
max_tokens: 2_000
prompt_template:
system: |
You are a seasoned pirate from the Golden Age of Piracy. You speak exclusively in pirate dialect, using nautical
terms, pirate slang, and colorful seafaring expressions. Transform any input into authentic pirate speak while
maintaining the original meaning. Use terms like 'ahoy', 'matey', 'ye', 'aye', 'landlubber', 'scallywag',
'doubloons', 'plunder', etc. Add pirate exclamations like 'Arrr!', 'Shiver me timbers!', and 'Yo ho ho!' where
appropriate. Refer to yourself in the first person as a pirate would.
user: |
{{goal}}
placeholder: history
routers: []
flow:
entry_point: "foundational_pirate_agent"
Add your agent definition to FoundationalChatAgentsDefinitions.rb:
# frozen_string_literal: true
module Ai
module FoundationalChatAgentsDefinitions
extend ActiveSupport::Concern
ITEMS = [
{
id: 1,
reference: 'chat',
version: '',
name: 'GitLab Duo Agent',
description: "GitLab Duo is your general development assistant"
},
{
id: 2,
reference: 'foundational_pirate_agent',
version: 'v1',
name: 'Foundational Pirate Agent',
description: "A most important agent that speaks like a pirate"
}
].freeze
end
end
Update user facing documentation.
Tips:
FoundationalChatAgentsDefinitions.rb should be experimental.
When creating the definition in GitLab Duo Workflow Service, the version should be v1.Control the release of new foundational agents with feature flags:
# ee/app/graphql/resolvers/ai/foundational_chat_agents_resolver.rb
def resolve(*, project_id: nil, namespace_id: nil)
project = GitlabSchema.find_by_gid(project_id)
filtered_agents = []
filtered_agents << 'foundational_pirate_agent' if Feature.disabled?(:my_feature_flag, project)
# filtered_agents << 'foundational_pirate_agent' if Feature.disabled?(:my_feature_flag, current_user)
::Ai::FoundationalChatAgent
.select {|agent| filtered_agents.exclude?(agent.reference) }
.sort_by(&:id)
end
This also allows making a foundational agent available to a specific tier.
Not every agent is useful in every area. For example, some agents operate in projects, while others are more useful or have more capabilities in groups. Scoping is not supported. See issue 577395.
Triggers are not supported for foundational chat agents. However, if they are defined on AI Catalog, users can still add it to their project at which point they can be used through triggers.
Versioning of agents is not yet supported. Consider potential breaking changes to older GitLab versions before doing changes to an agent.
For AI catalog created agents, you need to sync the agents locally. To do so, either create the agent in the local AI Catalog or on GitLab.com AI Catalog.
Fetch agents from GitLab.com
On $GDK/gitlab-ai-gateway, run the following command:
poetry run fetch-foundational-agents "http://gdk.test:3000 or https://gitlab.com" "<token-to-your-local-gdk>" \
"<agent-reference>:<agent-id-in-local-catalog>" --flow-registry-version v1
An example pulling duo_planner and security_analyst_agent from GitLab.com would look like this:
poetry run fetch-foundational-agents "https://gitlab.com" "<token-to-your-local-gdk>" \
"duo_planner:348,security_analyst_agent:356" --flow-registry-version v1
Where:
348 is the GitLab Duo Planner catalog ID on GitLab.com356 is the Security Analyst Agent catalog ID on GitLab.comAfter fetching the configurations, restart the service:
gdk restart duo-workflow-service
Verify the setup
Foundational agents are saved in $GDK/gitlab-ai-gateway/duo_workflow_service/agent_platform/v1/flows/configs/ as .yml files.
For example if you used the above poetry command to pull duo_planner and security_analyst_agent, you can run the following:
ls duo_workflow_service/agent_platform/v1/flows/configs/ | grep -e "duo_planner" -e "security_analyst"
You then should see the following output:
duo_planner.yml
security_analyst_agent.yml
Alternatively to check in the GDK UI:
FoundationalChatAgentsDefinitions.rb, you can now select your foundational agent in the web chat locally.--flow-registry-version v1This section describes how to test the pipeline used to sync foundational agents in your local GDK. For developing the foundational flows or pulling the latest flows refer to the Developing foundational agents locally section above.
api scope ($GDK_PAT_WITH_API_SCOPE)gitlab-ai-gateway repository in your GDKFirst, identify which foundational agents are defined in the monolith but missing from your local AI Catalog:
Check the foundational agents definitions:
# In your GDK's gitlab directory
cat ee/lib/ai/foundational_chat_agents_definitions.rb
List existing agents in your local AI Catalog:
curl --header "Authorization: Bearer $GDK_PAT_WITH_API_SCOPE" \
--header "Content-Type: application/json" \
"http://gdk.test:3000/api/graphql" \
--data '{"query": "query { aiCatalogItems { nodes { id name description } } }"}'
Compare the results to identify missing foundational agents (typically duo_planner and security_analyst_agent).
If foundational agents are missing from your local AI Catalog, create them programmatically:
Get a project ID for hosting the agents:
If you've run the duo setup script with bundle exec rake gitlab:duo:setup, you can use the project with ID 1000000 as the foundational agents owning project. If not, you can pick any Premium or Ultimate project in your GDK and use that project's ID.
# If you haven't run the duo setup script, get any project ID
curl --header "Authorization: Bearer $GDK_PAT_WITH_API_SCOPE" \
"http://gdk.test:3000/api/v4/projects" | jq '.[0].id'
Create the Planner agent:
curl --header "Authorization: Bearer $GDK_PAT_WITH_API_SCOPE" \
--header "Content-Type: application/json" \
"http://gdk.test:3000/api/graphql" \
--data '{
"query": "mutation { aiCatalogAgentCreate(input: { projectId: \"gid://gitlab/Project/YOUR_PROJECT_ID\", name: \"Planner\", description: \"Get help with planning and workflow management. Organize, edit, create, and track work more effectively in GitLab.\", public: true, systemPrompt: \"You are a helpful planning assistant that helps users organize, edit, create, and track work more effectively in GitLab.\" }) { item { id name } errors } }"
}'
Create the Security Analyst agent:
curl --header "Authorization: Bearer $GDK_PAT_WITH_API_SCOPE" \
--header "Content-Type: application/json" \
"http://gdk.test:3000/api/graphql" \
--data '{
"query": "mutation { aiCatalogAgentCreate(input: { projectId: \"gid://gitlab/Project/YOUR_PROJECT_ID\", name: \"Security Analyst\", description: \"Automate vulnerability management and security workflows. The Security Analyst Agent acts as an AI team member that can autonomously analyze, triage, and remediate security vulnerabilities.\", public: true, systemPrompt: \"You are a security analyst AI that helps with vulnerability management and security workflows. You can analyze, triage, and help remediate security vulnerabilities.\" }) { item { id name } errors } }"
}'
Replace YOUR_PROJECT_ID with the actual project ID from step 1.
After creating the agents, get their local catalog IDs:
# Get Planner agent ID
curl --header "Authorization: Bearer $GDK_PAT_WITH_API_SCOPE" \
--header "Content-Type: application/json" \
"http://gdk.test:3000/api/graphql" \
--data '{"query": "query { aiCatalogItems(search: \"Planner\") { nodes { id name } } }"}'
# Get Security Analyst agent ID
curl --header "Authorization: Bearer $GDK_PAT_WITH_API_SCOPE" \
--header "Content-Type: application/json" \
"http://gdk.test:3000/api/graphql" \
--data '{"query": "query { aiCatalogItems(search: \"Security Analyst\") { nodes { id name } } }"}'
Note the numeric IDs from the responses (for example, 10 and 11).
In your gitlab-ai-gateway directory, fetch the agent configurations using the local IDs:
# For v1 flow registry (recommended)
poetry run fetch-foundational-agents "http://gdk.test:3000" "$GDK_PAT_WITH_API_SCOPE" \
"duo_planner:10,security_analyst_agent:11" \
--flow-registry-version v1
# For experimental flow registry (alternative)
poetry run fetch-foundational-agents "http://gdk.test:3000" "$GDK_PAT_WITH_API_SCOPE" \
"duo_planner:10,security_analyst_agent:11" \
--flow-registry-version experimental \
--output-path duo_workflow_service/agent_platform/experimental/flows/configs
Replace 10 and 11 with the actual agent IDs from step 3.
gdk restart duo-workflow-service
With the changes to FoundationalChatAgentsDefinitions.rb and the fetched configurations, you can now select your foundational agents in the web chat locally.
Missing agents: If agents don't appear in chat after following these steps, verify:
fetch-foundational-agentsFlow registry version: Use v1 for production-like behavior, experimental for testing new features
Permission errors: Ensure your API token has the api scope and sufficient project permissions
GraphQL errors: Check the exact mutation parameters using GraphQL introspection:
curl --header "Authorization: Bearer $GDK_PAT_WITH_API_SCOPE" \
--header "Content-Type: application/json" \
"http://gdk.test:3000/api/graphql" \
--data '{"query": "query { __type(name: \"AiCatalogAgentCreatePayload\") { fields { name type { name } } } }"}'
Foundational agents have an integration test harness that runs the full agent loop end-to-end using real LLM calls. Use it to verify that an agent correctly selects tools, passes the right arguments, and produces valid responses — without needing a live GitLab instance or real tool backends.
The tests live in the ai-assist repository and run as CI jobs (for example, the Data Analyst agent tests run on changes to the agent prompt, otherwise manual).
assert_called_tool and assert_llm_validates
to express expected behavior clearly.@pytest.mark.asyncio
async def test_how_many_open_issues(
analytics_agent,
initial_state,
mock_gitlab_client,
):
"""Agent must call run_glql_query and report the count."""
mock_glql_response(mock_gitlab_client, glql_response(SAMPLE_ISSUES, count=42))
result = await ask_agent(
analytics_agent,
initial_state,
"How many open issues are there in the gitlab-org group?",
)
(result.assert_has_tool_calls().assert_called_tool("run_glql_query"))
await result.assert_llm_validates(
[
"Response says 42 open issues",
]
)
ask_agent runs the agent loop with the given prompt and returns a result object.
assert_called_tool verifies the agent invoked the expected tool.
assert_llm_validates asks an LLM to check the response against plain-language criteria,
so you don't need brittle substring matches.
Integration test jobs are defined in .gitlab/ci/test.gitlab-ci.yml. Available configuration variables:
EXECUTION_MODEL: The model used to run the agent during the test.VALIDATION_MODEL: The model used by assert_llm_validates to judge responses.To reuse or extend this harness for your agent, see these merge requests for reference:
Foundational Chat Agents are developed by GitLab and must be available to all GitLab deployments (GitLab.com, Self-Managed, and Dedicated).
The architecture of how Foundational Agents are made available avoids connecting to AI Catalog to fetch definitions at runtime and allows GitLab engineering teams full control over when they are released.
This design could also be extended to support Foundational flows.
Defining foundational agents in the monolith serves two purposes: backwards compatibility support and release control.
With FoundationalChatAgentsDefinitions
The FoundationalChatAgentsDefinitions
module manages agent versioning based on the GitLab instance version.
affecting older GitLab versions, similar to prompt versioning.
Additionally, on FoundationalChatAgentsResolver,
teams are able to select which conditions can make a foundational chat agent available, for situations like:
If we relied exclusively on AI Catalog or Duo Workflow Service, such flexibility wouldn't be possible
Agent versions are resolved based on the version field in FoundationalChatAgentsDefinitions.rb,
which maps to a
folder in GitLab Duo Workflow Service (for example, v1, experimental).
In the future, version resolution will be based on semantic versioning. This will allow:
This approach ensures backward compatibility while enabling continuous improvement of foundational agents.
Bundling agents into GitLab Duo Workflow Service makes agents defined in AI Catalog available on all deployments, including self-hosted setups. With semantic versioning support, the latest version of each major release will be bundled, along with specific pinned versions of each foundational agent.
The alternative to this would be to ship the YAML definitions themselves as part of GitLab monolith, but that comes with the downside of not being able to quickly ship fixes to cloud-connected self-managed instances.
Eventually, if labels are implemented on AI Catalog, teams wouldn't need to add their entries to the Dockerfile, versions could be fetched by the correct labels.
%%{init: {"sequence": {"actorMargin": 50}}}%%
sequenceDiagram
accTitle: Foundational agent creation flow
accDescr: Sequence diagram showing the process of creating a foundational agent from AI Catalog through to GitLab monolith
participant Team
participant AI Catalog
participant DWS Repo as DWS Repository
participant CI
participant Monolith
Team->>AI Catalog: Create foundational agent
Team->>DWS Repo: Add agent ID to Dockerfile
DWS Repo->>CI: Trigger build
CI->>AI Catalog: Pull agent definitions
AI Catalog->>CI: Returns all required versions
CI->>CI: Store definitions in DWS image
CI->>CI: Ships images with definitions
Team->>Monolith: Add agent to FoundationalChatAgentsDefinitions.rb
%%{init: {"sequence": {"actorMargin": 50}}}%%
sequenceDiagram
accTitle: Foundational agent usage flow
accDescr: Sequence diagram showing how users interact with foundational agents through GitLab monolith and Duo Workflow Service
participant User
participant Monolith
participant DWS as GitLab Duo Workflow Service
User->>Monolith: Request to foundational agent
Monolith->>DWS: Request specific agent version
DWS->>DWS: Resolve agent version
DWS->>DWS: Process request
DWS->>Monolith: Return response
Monolith->>User: Return response
The execution flows are the same whether the user is using a local monolith, GitLab.com, the cloud-connected DWS or a local installation of DWS.