doc/development/cicd/templates.md
[!note] With the introduction of the CI/CD Catalog, GitLab is no longer accepting contributions of new CI/CD templates to the codebase. Instead, we encourage team members to create CI/CD components for the catalog. This transition enhances the modularity and maintainability of our shared CI/CD resources, and avoids the complexities of contributing new CI/CD templates. If you need to update an existing template, you must also update the matching CI/CD component. If no component exists that matches the CI/CD template yet, consider creating the matching component. This ensures that template and component functionality remain in sync, aligning with our new development practices.
This document explains how to develop GitLab CI/CD templates.
Before submitting a merge request with a new or updated CI/CD template, you must:
*.gitlab-ci.yml format..gitlab-ci.yml syntax. Verify it's valid
with the CI/CD lint tool.All template files are saved in lib/gitlab/ci/templates. Save general templates
in this directory, but certain template types have a specific directory reserved for
them. The ability to select a template in new file UI
is determined by the directory it is in:
| Subdirectory | Selectable in UI | Template type |
|---|---|---|
/* (root) | Yes | General templates. |
/AWS/* | No | Templates related to Cloud Deployment (AWS). |
/Jobs/* | No | Templates related to Auto DevOps. |
/Pages/* | Yes | Sample templates for using Static site generators with GitLab Pages. |
/Security/* | Yes | Templates related to Security scanners. |
/Terraform/* | No | Templates related to infrastructure as Code (Terraform). |
/Verify/* | Yes | Templates related to Testing features. |
/Workflows/* | No | Sample templates for using the workflow: keyword. |
Use the following guidelines to ensure your template submission follows standards:
Templates have two different types that impact the way the template should be written and used. The style in a template should match one of these two types:
A pipeline template provides an end-to-end CI/CD workflow that matches a project's
structure, language, and so on. It usually should be used by itself in projects that
don't have any other .gitlab-ci.yml files.
When authoring pipeline templates:
image
or before_script in a default
section at the top of the template.includes keyword in an existing
.gitlab-ci.yml file or not.A job template provides specific jobs that can be added to an existing CI/CD
workflow to accomplish specific tasks. It usually should be used by adding it to
an existing .gitlab-ci.yml file by using the includes
keyword. You can also copy and paste the contents into an existing .gitlab-ci.yml file.
Configure job templates so that users can add them to their current pipeline with very few or no modifications. It must be configured to reduce the risk of conflicting with other pipeline configuration.
When authoring job templates:
default
keywords. When a root .gitlab-ci.yml includes a template, global or default keywords
might be overridden and cause unexpected behavior. If a job template requires a
specific stage, explain in the code comments that users must manually add the stage
to the main .gitlab-ci.yml configuration.includes keyword or copied into an existing configuration.includes can break pipelines for all projects using the template.Additional points to keep in mind when authoring templates:
| Template design points | Pipeline templates | Job templates |
|---|---|---|
Can use global keywords, including stages. | Yes | No |
| Can define jobs. | Yes | Yes |
| Can be selected in the new file UI | Yes | No |
Can include other job templates with include | Yes | No |
Can include other pipeline templates with include. | No | No |
To make templates easier to follow, templates should all use clear syntax styles, with a consistent format.
The before_script, script, and after_script keywords of every job are linted
using ShellCheck and should follow the
Shell scripting standards and style guidelines
as much as possible.
ShellCheck assumes that the script is designed to run using Bash.
Templates which use scripts for shells that aren't compatible with the Bash ShellCheck
rules can be excluded from ShellCheck linting. To exclude a script, add it to the
EXCLUDED_TEMPLATES list in scripts/lint_templates_bash.rb.
Use $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH
instead of a hardcoded main branch, and never use master:
job:
rules:
if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH
script:
echo "example job"
rules instead of only or exceptAvoid using only or except if possible.
Only and except is not being developed any more, and rules
is now the preferred syntax:
job2:
script:
- echo
rules:
- if: $CI_COMMIT_BRANCH
If a command is very long, or has many command line flags, like -o or --option:
For example, with a long command with short CLI flags like
docker run --e SOURCE_CODE="$PWD" -v "$PWD":/code -v /var/run/docker.sock:/var/run/docker.sock "$CODE_QUALITY_IMAGE" /code:
job1:
script:
- docker run
--env SOURCE_CODE="$PWD"
--volume "$PWD":/code
--volume /var/run/docker.sock:/var/run/docker.sock
"$CODE_QUALITY_IMAGE" /code
You can also use the | and > YAML operators to split up multi-line commands.
You can access template contents from the new file menu, and this might be the only place users see information about the template. It's important to clearly document the behavior of the template directly in the template itself.
The following guidelines cover the basic comments expected in all template submissions. Add additional comments as needed if you think the comments can help users or template reviewers.
Give the details on how to use the template in # comments at the top of the file.
This includes:
include keyword in an existing pipeline.# Use this template to publish an application that uses the ABC server.
# You can copy and paste this template into a new `.gitlab-ci.yml` file.
# You should not add this template to an existing `.gitlab-ci.yml` file by using the `include:` keyword.
#
# Requirements:
# - An ABC project with content saved in /content and tests in /test
# - A CI/CD variable named ABC-PASSWORD saved in the project CI/CD settings. The value
# should be the password used to deploy to your ABC server.
# - An ABC server configured to listen on port 12345.
#
# You must change the URL on line 123 to point to your ABC server and port.
#
# For more information, see https://gitlab.com/example/abcserver/README.md
job1:
...
If the template uses variables, explain them in # comments where they are first
defined. You can skip the comment when the variable is trivially clear:
variables: # Good to have a comment here, for example:
TEST_CODE_PATH: <path/to/code> # Update this variable with the relative path to your Ruby specs
job1:
variables:
ERROR_MESSAGE: "The $TEST_CODE_PATH path is invalid" # (No need for a comment here, it's already clear)
script:
- echo ${ERROR_MESSAGE}
If you are expecting a variable to be provided via the CI/CD settings, or via the
variables keyword, that variable must use all-caps naming with underscores (_)
separating words.
.with_login:
before_script:
# SECRET_TOKEN should be provided via the project settings
- echo "$SECRET_TOKEN" | docker login -u my-user --password-stdin my-registry
Lowercase naming can optionally be used for variables which are defined locally in
one of the script keywords:
job1:
script:
- response="$(curl "https://example.com/json")"
- message="$(echo "$response" | jq -r .message)"
- 'echo "Server responded with: $message"'
A template might be dynamically included with the include:template: keyword. If
you make a change to an existing template, you must make sure that it doesn't break
CI/CD in existing projects.
For example, changing a job name in a template could break pipelines in an existing project.
In this example, a template named Performance.gitlab-ci.yml has the following content:
performance:
image: registry.gitlab.com/gitlab-org/verify-tools/performance:v0.1.0
script: ./performance-test $TARGET_URL
and users include this template with passing an argument to the performance job.
This can be done by specifying the CI/CD variable TARGET_URL in their .gitlab-ci.yml:
include:
template: Performance.gitlab-ci.yml
performance:
variables:
TARGET_URL: https://awesome-app.com
If the job name performance in the template is renamed to browser-performance,
the user's .gitlab-ci.yml immediately causes a lint error because there
are no such jobs named performance in the included template anymore. Therefore,
users have to fix their .gitlab-ci.yml that could annoy their workflow.
Read versioning section for introducing breaking change safely.
To introduce a breaking change without affecting the existing projects that depend on the current template, use stable and latest versioning.
Stable templates usually only receive breaking changes in major version releases, while latest templates can receive breaking changes in any release. In major release milestones, the latest template is made the new stable template (and the latest template might be deleted).
Adding a latest template is safe, but comes with a maintenance burden:
Before adding a new latest template, see if the change can be made to the stable template instead, even if it's a breaking change. If the template is intended for copy-paste usage only, it might be possible to directly change the stable version. Before changing the stable template with a breaking change in a minor milestone, make sure:
includes.include.A stable CI/CD template is a template that only introduces breaking changes in major
release milestones. Name the stable version of a template as <template-name>.gitlab-ci.yml,
for example Jobs/Deploy.gitlab-ci.yml.
You can make a new stable template by copying the latest template
available in a major milestone release of GitLab like 15.0. All breaking changes must be announced
on the Deprecations and removals by version page.
You can change a stable template version in a minor GitLab release like 15.1 if:
Templates marked as latest can be updated in any release, even with
breaking changes. Add .latest to the template name if
it's considered the latest version, for example Jobs/Deploy.latest.gitlab-ci.yml.
When you introduce a breaking change, you must test and document the upgrade path. In general, we should not promote the latest template as the best option, as it could surprise users with unexpected problems.
If the latest template does not exist yet, you can copy the stable template.
Users may want to use an older stable template that is not bundled in the current GitLab package. For example, the stable templates in GitLab 15.0 and GitLab 16.0 could be so different that a user wants to continue using the GitLab 15.0 template even after upgrading to GitLab 16.0.
You can add a note in the template or in documentation explaining how to use include:remote
to include older template versions. If other templates are included with include: template,
they can be combined with the include: remote:
# To use the v13 stable template, which is not included in v14, fetch the specific
# template from the remote template repository with the `include:remote:` keyword.
# If you fetch from the GitLab canonical project, use the following URL format:
# https://gitlab.com/gitlab-org/gitlab/-/raw/<version>/lib/gitlab/ci/templates/<template-name>
include:
- template: Auto-DevOps.gitlab-ci.yml
- remote: https://gitlab.com/gitlab-org/gitlab/-/raw/v13.0.1-ee/lib/gitlab/ci/templates/Jobs/Deploy.gitlab-ci.yml
There is an open issue about introducing versioning concepts in GitLab CI/CD templates. You can check that issue to follow the progress.
Each CI/CD template must be tested to make sure that it's safe to be published.
It's always good practice to test the template in a minimal demo project. To do so, follow the following steps:
.gitlab-ci.yml to the project with the proposed template.This is useful information for reviewers to make sure the template is safe to be merged.
Templates located under some directories are also selectable in the New file UI. When you add a template into one of those directories, make sure that it correctly appears in the dropdown list:
You should write an RSpec test to make sure that pipeline jobs are generated correctly:
spec/lib/gitlab/ci/templates/<template-category>/<template-name>_spec.rbCi::CreatePipelineService.When you introduce a breaking change to a latest template,
you must:
This information is important for users when a stable template is updated in a major version GitLab release.
Every CI/CD template must also have metrics defined to track their use. The CI/CD template monthly usage report can be found in Sisense (GitLab team members only). Select a template to see the graph for that single template.
To add a metric definition for a new template:
Install and start the GitLab GDK.
In the gitlab directory in your GDK, check out the branch that contains the new template.
Add the new template event name to the weekly and monthly CI/CD template total count metrics:
Use the same event name as above as the last argument in the following command to add new metric definitions:
bundle exec rails generate gitlab:usage_metric_definition:redis_hll ci_templates <template_metric_event_name>
The output should look like:
$ bundle exec rails generate gitlab:usage_metric_definition:redis_hll ci_templates p_ci_templates_my_template_name
create config/metrics/counts_7d/20220120073740_p_ci_templates_my_template_name_weekly.yml
create config/metrics/counts_28d/20220120073746_p_ci_templates_my_template_name_monthly.yml
Edit both newly generated files as follows:
name: and performance_indicator_type:: Delete (not needed).
introduced_by_url:: The URL of the MR adding the template.
data_source:: Set to redis_hll.
description: Add a short description of what this metric counts, for example: Count of pipelines using the latest Auto Deploy template
product_*: Set to section, stage, group, and feature category
as per the metrics dictionary guide.
If you are unsure what to use for these keywords, you can ask for help in the merge request.
Add the following to the end of each file:
options:
events:
- p_ci_templates_my_template_name
Commit and push the changes.
For example, these are the metrics configuration files for the 5 Minute Production App template:
A template could contain malicious code. For example, a template that contains the export shell command in a job
might accidentally expose secret project CI/CD variables in a job log.
If you're unsure if it's secure or not, you must ask security experts for cross-validation.
After your CI/CD template MR is created and labeled with ci::templates, DangerBot
suggests one reviewer and one maintainer that can review your code. When your merge
request is ready for review, mention
the reviewer and ask them to review your CI/CD template changes. See details in the merge request that added
a DangerBot task for CI/CD template MRs.