doc/development/ai_features/model_migration.md
The table below shows current open issues labeled with AI Model Migration. This provides a live view of ongoing model migration work across GitLab.
display: table
fields: title, author, assignee, milestone, labels, updated
limit: 10
query: label = "AI Model Migration" AND opened = true
[!note] This table is dynamically generated using GitLab Query Language (GLQL) when viewing the rendered documentation. It shows up to 10 open issues with the AI Model Migration label, sorted by most recently updated.
LLM models are constantly evolving, and GitLab needs to regularly update our AI features to support newer models. This guide provides a structured approach for migrating AI features to new models while maintaining stability and reliability.
[!note] GitLab strives to leverage the latest AI model capabilities to help provide optimal performance and features, which means model updates from existing GitLab subprocessors might occur without specific customer notifications beyond documentation updates.
Model migrations typically follow these general timelines:
Several factors can impact migration timelines:
[!note] While some migrations can technically be completed quickly, we typically plan for longer timelines to ensure proper testing and staged rollouts. This approach helps maintain system stability and reliability.
Model migrations involve several teams working together. This section clarifies which teams are responsible for different aspects of the migration process.
| Task | AI Framework | Feature Teams | Product | Infrastructure |
|---|---|---|---|---|
| Model configuration file creation | R/A | C | I | I |
| Infrastructure compatibility | R/A | I | I | C |
| Feature-specific prompt adjustments | C | R/A | I | I |
| Evaluations & testing | C | R/A | I | I |
| Feature flag implementation | C | R/A | I | I |
| Rollout planning | C | R/A | C | I |
| Documentation updates | C | R/A | C | I |
| Monitoring & incident response | C | R/A | I | C |
R = Responsible, A = Accountable, C = Consulted, I = Informed
[!note] Model Mapping Resource: You can see which features use which models and versions via the GitLab AI Features - Default GitLab AI Vendor Models page.
Initialization
AI Model Migration - Provider/Model/VersionAI Model Migration labelFeature Team Implementation
Testing & Validation
Deployment
Completion
Identification & Planning
Replace discontinued [model] with [replacement]AI Model Migration labelEvaluation
Implementation
Testing
Deployment
Completion
Before starting a model migration:
Create an issue under the AI Model Version Migration Initiative epic:
group::ai framework and AI Model MigrationVerify model support in AI Gateway:
ai_gateway/models/v2/container.pyai_gateway/models/anthropic.py.env filehttp://localhost:5052/docsEnsure access to testing environments and monitoring tools
Complete model evaluation using the Centralized Evaluation Framework
For model deprecations:
Create an epic when a deprecation is announced:
group::ai framework and AI Model MigrationEvaluate replacement models:
Create migration timeline:
[!note] Documentation of model changes and deprecations is crucial for tracking impact and future troubleshooting. Always create an issue before beginning any migration process.
Feature teams should use the AI Model Rollout template to implement model migrations. See an example from our Claude 3.7 Sonnet Code Generation Rollout Plan.
AI Framework Team:
Feature Team:
[!note] While we're moving toward AI Gateway holding the prompts, feature flag implementation still requires a GitLab release.
AI Framework Team:
Feature Team:
For implementing feature flags, refer to our Feature Flags Development Guidelines.
[!note] Feature flag implementations will affect self-hosted cloud-connected customers. These customers won't receive the model upgrade until the feature flag is removed from the AI Gateway codebase, as they won't have access to the new GitLab release.
Implement model selection logic in:
ee/lib/gitlab/llm/chain/requests/ai_gateway.rb)Example: Upgrading from Claude 3.5 to Claude 3.7
AI Framework Team:
Feature Teams:
Example: Adding AWS Bedrock models
AI Framework Team:
Feature Teams:
Example: Replacing discontinued Vertex AI Code Gecko v2
AI Framework Team:
Feature Teams:
If you encounter prompt compatibility issues:
Analyze Errors:
Resolve Issues:
For Claude 3.7 migrations:
The AI Framework team should create a main migration issue following this template:
# [Model Name] Model Upgrade
## Overview
[Brief description of the new model and its improvements]
## Features to Update
[List of features affected by this migration, organized by category]
### Generally Available Features
- [Feature 1]
- [Feature 2]
### Beta Features
- [Beta Feature 1]
### Experimental Features
- [Experimental Feature 1]
## Required Changes
- Add model configuration file for model flexibility
- New prompt definition created to use the new model
- Feature flag created for controlled rollout
## Technical Details
- [Any technical specifics about this migration]
- [Impact on GitLab.com and GitLab Self-Managed instances]
## Implementation Steps
- [ ] Update model configurations in each feature
- [ ] Verify performance improvements
- [ ] Deploy updates
- [ ] Update documentation
## Timeline
Priority: [Priority level]
## References
- [Model Announcement]
- [Model Documentation]
- [GitLab Documentation]
- [Other relevant links]
## Proposed Solution
[Description of the high-level implementation approach]
## Implementation Details
Follow the issues below with the associated rollout plans:
| Feature | DRI | ETA | Issue Link |
|---------|-----|-----|------------|
| [Feature 1] | [@username] | [Date] | [Issue link] |
| [Feature 2] | [@username] | [Date] | [Issue link] |
See an example in our Claude 3.7 Model Upgrade issue.