doc/ci/components/_index.md
{{< details >}}
{{< /details >}}
{{< history >}}
ci_namespace_catalog_experimental. Disabled by default.ci_namespace_catalog_experimental removed in GitLab 16.3.{{< /history >}}
A CI/CD component is a reusable single pipeline configuration unit. Use components to create a small part of a larger pipeline, or even to compose a complete pipeline configuration.
A component can be configured with input parameters for more dynamic behavior.
CI/CD components are similar to the other kinds of configuration added with the include keyword,
but have several advantages:
Instead of creating your own components, you can also search for published components that have the functionality you need in the CI/CD Catalog.
<i class="fa-youtube-play" aria-hidden="true"></i> For an introduction and hands-on examples, see Efficient DevSecOps workflows with reusable CI/CD components.
<!-- Video published on 2024-01-22. DRI: Developer Relations, <https://gitlab.com/groups/gitlab-com/marketing/developer-relations/-/epics/399> -->For common questions and additional support, see the FAQ: GitLab CI/CD Catalog blog post.
{{< history >}}
{{< /history >}}
A component project is a GitLab project with a repository that hosts one or more components. All components in the project are versioned together, with a maximum of 30 components per project.
If a component requires different versioning from other components, the component should be moved to a dedicated component project.
To create a component project, you must:
Create a new project with a README.md file:
Components published to the CI/CD catalog use both the description and avatar when displaying the component project's summary.
Add a YAML configuration file for each component, following the required directory structure. For example:
spec:
inputs:
stage:
default: test
---
component-job:
script: echo job 1
stage: $[[ inputs.stage ]]
You can use the component immediately, but you might want to consider publishing the component to the CI/CD catalog.
The repository must contain:
README.md Markdown file documenting the details of all the components in the repository.templates/ directory that contains all the component configurations.
In this directory, you can:
.yml for each component, like templates/secret-detection.yml.template.yml for each component,
like templates/secret-detection/template.yml. Only the template.yml file is used by other projects
using the component. Other files in these directories are not released with the component,
but can be used for things like tests or building container images.[!note] Optionally, each component can also have its own
README.mdfile that provides more detailed information, and can be linked from the top-levelREADME.mdfile. This helps to provide a better overview of your component project and how to use it.
You should also:
.gitlab-ci.yml to test the components
and release new versions.LICENSE.md file with a license of your choice that covers the usage of your component.
For example the MIT or Apache 2.0
open source licenses.For example:
If the project contains a single component, the directory structure should be similar to:
├── templates/
│ └── my-component.yml
├── LICENSE.md
├── README.md
└── .gitlab-ci.yml
If the project contains multiple components, then the directory structure should be similar to:
├── templates/
│ ├── my-component.yml
│ └── my-other-component/
│ ├── template.yml
│ ├── Dockerfile
│ └── test.sh
├── LICENSE.md
├── README.md
└── .gitlab-ci.yml
In this example:
my-component component's configuration is defined in a single file.my-other-component component's configuration contains multiple files in a directory.
Only the template.yml file can be used by other projects using the component.Prerequisites:
If you are a member of a parent group that contains the current group or project:
To add a component to a project's CI/CD configuration, use the include: component
keyword. The component reference is formatted as <fully-qualified-domain-name>/<project-path>/<component-name>@<specific-version>,
for example:
include:
- component: $CI_SERVER_FQDN/my-org/security-components/[email protected]
inputs:
stage: build
In this example:
$CI_SERVER_FQDN is a predefined variable
for the fully qualified domain name (FQDN) matching the GitLab host.
You can only reference components in the same GitLab instance as your project.my-org/security-components is the full path of the project containing the component.secret-detection is the component name that is defined as either a single file templates/secret-detection.yml
or as a directory templates/secret-detection/ containing a template.yml.1.0.0 is the version of the component.Pipeline configuration and component configuration are not processed independently. When a pipeline starts, any included component configuration merges into the pipeline's configuration. If your pipeline and the component both contain configuration with the same name, they can interact in unexpected ways.
For example, two jobs with the same name would merge together into a single job.
Similarly, a component using extends for configuration with the same name as a job in your pipeline
could extend the wrong configuration. Make sure your pipeline and the component do not share
any configuration with the same name, unless you intend to override
the component's configuration.
To use GitLab.com components on a GitLab Self-Managed instance, you must mirror the component project.
[!warning] If a component requires the use of tokens, passwords, or other sensitive data to function, be sure to audit the component's source code to verify that the data is only used to perform actions that you expect and authorize. You should also use tokens and secrets with the minimum permissions, access, or scope required to complete the action.
In order of highest priority first, the component version can be:
e3262fdd0914fa823210cdb79a8c421e2cef79d8.1.0.0. If a tag and commit SHA exist with the same name,
the commit SHA takes precedence over the tag. Components released to the CI/CD Catalog
must be tagged with a semantic version.main. If a branch and tag exist with the same name,
the tag takes precedence over the branch.~latest or a partial semantic version, which selects the latest version within the specified pattern
published in the CI/CD Catalog. Use ~latest only if you want to use the absolute
latest version at all times, which could include breaking changes. ~latest
does not include pre-releases, for example 1.0.1-rc, which are not considered
production-ready.You can use any version supported by the component, but using a version published to the CI/CD catalog is recommended. The version referenced with a commit SHA or branch name might not be published in the CI/CD catalog, but could be used for testing.
{{< history >}}
{{< /history >}}
You can use partial semantic version numbers and the keyword ~latest when referencing
a CI/CD catalog component to select the latest published version that matches your specification.
These formats only work with published CI/CD catalog components, not with regular project components.
This ensures that when you use formats like 1.2 or ~latest, you only pull components that have been validated and published to the catalog, rather than potentially untested code from any repository.
This approach offers significant benefits for both consumers and authors of components:
Use:
1.2 to select the latest 1.2.* version1 to select the latest 1.*.* version~latest to select the latest released versionFor example, a component has versions: 1.0.0, 1.1.0, 1.1.1, 1.2.0, 2.0.0, 2.0.1, 2.1.0
When referencing the component:
1 selects 1.2.01.1 selects 1.1.1~latest selects 2.1.0Pre-release versions are never fetched when using partial version selection. To fetch
a pre-release version, specify the full version, for example 1.0.1-rc.
{{< history >}}
ci_component_context_interpolation. Enabled by default.ci_component_context_interpolation removed.{{< /history >}}
Components can access metadata about themselves with a component context CI/CD expression. Use this expression in component templates to reference the version, commit SHA, and other metadata dynamically.
To use component context in a component, you must:
spec:component header.
spec:component supports name, sha, version, and reference fields.$[[ component.field-name ]] in the component template.For example, a component that references a Docker image built with the same version:
spec:
component: [name, version, reference]
inputs:
stage:
default: build
---
build-image:
stage: $[[ inputs.stage ]]
image: registry.example.com/$[[ component.name ]]:$[[ component.version ]]
script:
- echo "Building with component version $[[ component.version ]]"
- echo "Component reference: $[[ component.reference ]]"
You can also use component context to reference versioned resources.
spec sectionThe spec section in a component template defines the component's configuration and inputs.
You can use the following keywords in the spec section:
description: Provide a short description of the component that is displayed in the CI/CD Catalog.inputs: Define input parameters for users to customize component configuration.component: Declare which component context fields to make available for interpolation (like name, sha, version, and reference).[!note] You cannot use
spec:includein components. Components should be self-contained and not depend on external files. Define inputs directly in the component instead of including them from separate files.
This section describes some best practices for creating high quality component projects.
While it's possible for a component to use other components in turn, make sure to carefully select the dependencies. To manage dependencies, you should:
include:local is a good way
to ensure the same Git SHA is used across multiple files.~latest or a Git reference. Using a release or Git SHA guarantees that you are fetching the same revision
all the time and that consumers of your component get consistent behavior.README.mdEach component project should have clear and comprehensive documentation. To
write a good README.md file:
## Components section with sub-sections like ### Component A for each component.spec:inputs:description to
document any variables or secrets the component uses.README. Inputs appear automatically on the component page.
Instead, link to the published component.## Contribute section if contributions are welcome.If a component needs more instructions, add additional documentation in a Markdown file
in the component directory and link to it from the main README.md file. For example:
README.md # with links to the specific docs.md
templates/
├── component-1/
│ ├── template.yml
│ └── docs.md
└── component-2/
├── template.yml
└── docs.md
For an example, see the AWS components README.
Testing CI/CD components as part of the development workflow is strongly recommended and helps ensure consistent behavior.
Test changes in a CI/CD pipeline (like any other project) by creating a .gitlab-ci.yml
in the root directory. Make sure to test both the behavior and potential side-effects
of the component. You can use the GitLab API if needed.
For example:
include:
# include the component located in the current project from the current SHA
- component: $CI_SERVER_FQDN/$CI_PROJECT_PATH/my-component@$CI_COMMIT_SHA
inputs:
stage: build
stages: [build, test, release]
# Check if `component job of my-component` is added.
# This example job could also test that the included component works as expected.
# You can inspect data generated by the component, use GitLab API endpoints, or third-party tools.
ensure-job-added:
stage: test
image: badouralix/curl-jq
# Replace "component job of my-component" with the job name in your component.
script:
- |
route="${CI_API_V4_URL}/projects/${CI_PROJECT_ID}/pipelines/${CI_PIPELINE_ID}/jobs"
count=`curl --silent --header "JOB-TOKEN: ${CI_JOB_TOKEN}" "$route" | jq 'map(select(.name | contains("component job of my-component"))) | length'`
if [ "$count" != "1" ]; then
exit 1; else
echo "Component Job present"
fi
# If the pipeline is for a new tag with a semantic version, and all previous jobs succeed,
# create the release.
create-release:
stage: release
image: registry.gitlab.com/gitlab-org/cli:latest
script: echo "Creating release $CI_COMMIT_TAG"
rules:
- if: $CI_COMMIT_TAG
release:
tag_name: $CI_COMMIT_TAG
description: "Release $CI_COMMIT_TAG of components repository $CI_PROJECT_PATH"
After committing and pushing changes, the pipeline tests the component, then creates a release if the earlier jobs pass.
[!note] Authentication is necessary if the project is private.
In some cases, components require source files to interact with. For example, a component that builds Go source code likely needs some samples of Go to test against. Alternatively, a component that builds Docker images likely needs some sample Dockerfiles to test against.
You can include sample files like these directly in the component project, to be used during component testing.
You can learn more in examples for testing a component.
When using another component in your component, use $CI_SERVER_FQDN
instead of your instance's Fully Qualified Domain Name (like gitlab.com).
When accessing the GitLab API in your component, use the $CI_API_V4_URL instead of the
full URL and path for your instance (like https://gitlab.com/api/v4).
These predefined variables ensure that your component also works when used on another instance, for example when using a GitLab.com component on a GitLab Self-Managed instance.
Ensure that the component and its testing pipeline work also on GitLab Self-Managed. While some API resources of public projects on GitLab.com could be accessed with unauthenticated requests, on a GitLab Self-Managed instance a component project could be mirrored as private or internal project.
It's important that an access token can optionally be provided via inputs or variables to authenticate requests on GitLab Self-Managed instances.
Avoid using global keywords in a component.
Using these keywords in a component affects all jobs in a pipeline, including jobs
directly defined in the main .gitlab-ci.yml or in other included components.
As an alternative to global keywords:
extends keyword in the component, but use
unique names that reduce the risk of naming conflicts when the component is merged
into the configuration.For example, avoid using the default global keyword:
# Not recommended
default:
image: ruby:3.0
rspec-1:
script: bundle exec rspec dir1/
rspec-2:
script: bundle exec rspec dir2/
Instead, you can:
Add the configuration to each job explicitly:
rspec-1:
image: ruby:3.0
script: bundle exec rspec dir1/
rspec-2:
image: ruby:3.0
script: bundle exec rspec dir2/
Use extends to reuse configuration:
.rspec-image:
image: ruby:3.0
rspec-1:
extends:
- .rspec-image
script: bundle exec rspec dir1/
rspec-2:
extends:
- .rspec-image
script: bundle exec rspec dir2/
Avoid using hardcoded values in CI/CD components. Hardcoded values might force component users to need to review the component's internal details and adapt their pipeline to work with the component.
A common keyword with problematic hard-coded values is stage. If a component job's
stage is hardcoded, all pipelines using the component must either define
the exact same stage, or override
the configuration.
The preferred method is to use the input keyword for dynamic
component configuration. The component user can specify the exact value they need.
For example, to create a component with stage configuration that can be defined by users:
In the component configuration:
spec:
inputs:
stage:
default: test
---
unit-test:
stage: $[[ inputs.stage ]]
script: echo unit tests
integration-test:
stage: $[[ inputs.stage ]]
script: echo integration tests
In a project using the component:
stages: [verify, release]
include:
- component: $CI_SERVER_FQDN/myorg/ruby/[email protected]
inputs:
stage: verify
Similar to the values for the stage keyword, you should avoid hard-coding job names
in CI/CD components. When your component's users can customize job names, they can prevent conflicts
with the existing names in their pipelines. Users could also include a component
multiple times with different input options by using different names.
Use inputs to allow your component's users to define a specific job name, or a prefix for the
job name. For example:
spec:
inputs:
job-prefix:
description: "Define a prefix for the job name"
job-name:
description: "Alternatively, define the job's name"
job-stage:
default: test
---
"$[[ inputs.job-prefix ]]-scan-website":
stage: $[[ inputs.job-stage ]]
script:
- scan-website-1
"$[[ inputs.job-name ]]":
stage: $[[ inputs.job-stage ]]
script:
- scan-website-2
When using CI/CD variables in a component, evaluate if the inputs keyword
should be used instead. Avoid asking users to define custom variables to configure
components when inputs is a better solution.
Inputs are explicitly defined in the component's spec section, and have better validation than variables.
For example, if a required input is not passed to the component, GitLab returns a pipeline error.
By contrast, if a variable is not defined, its value is empty, and there is no error.
For example, use inputs instead of variables to configure a scanner's output format:
In the component configuration:
spec:
inputs:
scanner-output:
default: json
---
my-scanner:
script: my-scan --output $[[ inputs.scanner-output ]]
In the project using the component:
include:
- component: $CI_SERVER_FQDN/path/to/project/[email protected]
inputs:
scanner-output: yaml
In other cases, CI/CD variables might still be preferred. For example:
{{< details >}}
{{< /details >}}
{{< history >}}
{{< /history >}}
The CI/CD Catalog is a list of projects with published CI/CD components you can use to extend your CI/CD workflow.
Anyone can create a component project and add it to the CI/CD Catalog, or contribute to an existing project to improve the available components.
For a click-through demo, see the CI/CD Catalog beta Product Tour.
<!-- Demo published on 2024-01-24 -->To access the CI/CD Catalog and view the published components that are available to you:
Alternatively, if you are already in the pipeline editor in your project, you can select CI/CD Catalog.
Visibility of components in the CI/CD catalog follows the component source project's visibility setting. Components with source projects set to:
{{< details >}}
{{< /details >}}
{{< history >}}
{{< /history >}}
If you maintain CI/CD catalog resources, you can view usage analytics to understand how your components are being adopted across projects.
Prerequisites:
To view catalog resource analytics:
The Analytics view displays the catalog resources where you have the Maintainer or Owner role. This view shows:
For example:
You can use this information to:
To publish a component project in the CI/CD catalog, you must:
To make published versions of a component project visible in the CI/CD catalog, you must set the project as a catalog project.
Prerequisites:
To set the project as a catalog project:
The project only becomes findable in the catalog after you publish a new release.
To use automation to enable this setting, you can use the mutationcatalogresourcescreate
GraphQL endpoint. Issue 463043
proposes to expose this in the REST API as well.
CI/CD components can be used without being listed in the CI/CD catalog. However, publishing a component's releases in the catalog makes it discoverable to other users.
Prerequisites:
README.md file in the root directory for the commit SHA of the tag being released.templates/ directory
for the commit SHA of the tag being released.release keyword in a CI/CD job to create the release,
not the Releases API.To publish a new version of the component to the catalog:
Add a job to the project's .gitlab-ci.yml file that uses the release
keyword to create the new release when a tag is created.
You should configure the tag pipeline to test the components before
running the release job. For example:
create-release:
stage: release
image: registry.gitlab.com/gitlab-org/cli:latest
script: echo "Creating release $CI_COMMIT_TAG"
rules:
- if: $CI_COMMIT_TAG
release:
tag_name: $CI_COMMIT_TAG
description: "Release $CI_COMMIT_TAG of components in $CI_PROJECT_PATH"
Create a new tag for the release, which should trigger a tag pipeline that contains the job responsible for creating the release. The tag must use semantic versioning.
After the release job completes successfully, the release is created and the new version is published to the CI/CD catalog.
{{< history >}}
{{< /history >}}
When tagging and releasing new versions of components to the Catalog, you must use semantic versioning. Semantic versioning is the standard for communicating that a change is a major, minor, patch, or other kind of change.
For example, 1.0.0, 2.3.4, and 1.0.0-alpha are all valid semantic versions.
To remove a component project from the catalog, turn off the CI/CD Catalog resource toggle in the project settings.
[!warning] This action destroys the metadata about the component project and its versions published in the catalog. The project and its repository still exist, but are not visible in the catalog.
To publish the component project in the catalog again, you need to publish a new release.
{{< history >}}
{{< /history >}}
Some CI/CD components are badged with an icon to show that the component was created and is maintained by users verified by GitLab or the instance administrator:
GitLab-maintained ({{< icon name="tanuki-verified" >}}): GitLab.com components that are created and maintained by GitLab.
GitLab Partner ({{< icon name="partner-verified" >}}): GitLab.com components that are independently created and maintained by a GitLab-verified partner.
GitLab partners can contact a member of the GitLab Partner Alliance to have their namespace on GitLab.com flagged as GitLab-verified. Then any CI/CD components located in the namespace are badged as GitLab Partner components. The Partner Alliance member creates an internal request issue (GitLab team members only) on behalf of the verified partner.
[!warning] GitLab Partner-created components are provided as-is, without warranty of any kind. An end user's use of a GitLab Partner-created component is at their own risk and GitLab shall have no indemnification obligations nor any liability of any type with respect to the end user's use of the component. The end user's use of such content and any liability related thereto shall be between the publisher of the content and the end user.
Verified creator ({{< icon name="check-sm" >}}): Components created and maintained by a user verified by an administrator.
{{< details >}}
{{< /details >}}
{{< history >}}
{{< /history >}}
A GitLab administrator can set a CI/CD component as created and maintained by a verified creator:
Open GraphiQL in the instance with your administrator account, for example at: https://gitlab.example.com/-/graphql-explorer.
Run this query, replacing root-level-group with the root namespace of the component to verify:
mutation {
verifiedNamespaceCreate(input: { namespacePath: "root-level-group",
verificationLevel: VERIFIED_CREATOR_SELF_MANAGED
}) {
errors
}
}
After the query completes, all components in projects in the root namespace are verified. The Verified creator badge displays next to the component names in the CI/CD catalog.
To remove the badge from a component, repeat the query with UNVERIFIED for verificationLevel.
Any existing CI/CD template that you use in projects by using the include: syntax
can be converted to a CI/CD component:
.gitlab-ci.yml in the components repository to test changes to the component.You can learn more by following a practical example for migrating the Go CI/CD template to CI/CD component.
{{< details >}}
{{< /details >}}
The CI/CD catalog of a fresh install of a GitLab instance starts with no published CI/CD components. To populate your instance's catalog, you can:
To mirror a GitLab.com component in your GitLab Self-Managed instance:
gitlab.com.components).As anyone can publish components to the catalog, you should carefully review components before using them in your project. Use of GitLab CI/CD components is at your own risk and GitLab cannot guarantee the security of third-party components.
When using third-party CI/CD components, consider the following security best practices:
latest.CI_JOB_TOKEN) project access and permissions
for projects using CI/CD components.To maintain secure and trustworthy CI/CD components and ensure the integrity of the pipeline configuration you deliver to users, follow these best practices:
No one for protected branches.latest: Avoid including examples in your README.md that use @latest.content not found messageYou might receive an error message similar to the following when using the ~latest or a partial semantic
version qualifier to reference a component hosted by a catalog project:
This GitLab CI configuration is invalid: Component 'gitlab.com/my-namespace/my-project/my-component@~latest' - content not found
The ~latest behavior was updated
in GitLab 16.10. It now refers to the latest semantic version of the catalog resource. To resolve this issue, create a new release.
Build component error: Spec must be a valid json schemaIf a component has invalid formatting, you might not be able to create a release
and could receive an error like Build component error: Spec must be a valid json schema.
This error can be caused by an empty spec:inputs section. If your configuration
does not use any inputs, you can make the spec section empty instead. For example:
spec:
---
my-component:
script: echo