doc/user/application_security/container_scanning/_index.md
{{< details >}}
{{< /details >}}
Security vulnerabilities in container images create risk throughout your application lifecycle. Container scanning detects these risks early, before they reach production environments. When vulnerabilities appear in your base images or operating system's packages, container scanning identifies them and provides a remediation path for those that it can.
Container scanning is often considered part of Software Composition Analysis (SCA). SCA can contain aspects of inspecting the items your code uses. These items typically include application and system dependencies that are almost always imported from external sources, rather than sourced from items you wrote yourself.
GitLab offers both container scanning and dependency scanning to ensure coverage for all these dependency types. To cover as much of your risk area as possible, use all the security scanners. For a comparison of these features, see Dependency scanning compared to container scanning.
GitLab integrates with the Trivy security scanner to perform vulnerability static analysis in containers.
[!warning] The Grype analyzer is no longer maintained, except for limited fixes as explained in the GitLab statement of support. The existing current major version for the Grype analyzer image will continue to be updated with the latest advisory database, and operating system packages until GitLab 19.0, at which point the analyzer will stop working.
| Features | In Free and Premium | In Ultimate |
|---|---|---|
| Customize Settings (Variables, Overriding, offline environment support, etc) | {{< yes >}} | {{< yes >}} |
| View JSON Report as a CI job artifact | {{< yes >}} | {{< yes >}} |
| Generate a CycloneDX SBOM JSON report as a CI job artifact | {{< yes >}} | {{< yes >}} |
| Ability to enable container scanning via an MR in the GitLab UI | {{< yes >}} | {{< yes >}} |
| UBI Image Support | {{< yes >}} | {{< yes >}} |
| Support for Trivy | {{< yes >}} | {{< yes >}} |
| End-of-life Operating System Detection | {{< yes >}} | {{< yes >}} |
| Inclusion of GitLab advisory database | Limited to the time-delayed content from GitLab advisories-communities project | Yes - all the latest content from Gemnasium DB |
| Presentation of Report data in Merge Request and Security tab of the CI pipeline job | {{< no >}} | {{< yes >}} |
| Solutions for vulnerabilities (auto-remediation) | {{< no >}} | {{< yes >}} |
| Support for the vulnerability allow list | {{< no >}} | {{< yes >}} |
| Access to dependency list page | {{< no >}} | {{< yes >}} |
Enable the container scanning analyzer in your CI/CD pipeline. When a pipeline runs, the images your application depends on are scanned for vulnerabilities. You can customize container scanning by using CI/CD variables.
Prerequisites:
.gitlab-ci.yml file.docker or kubernetes executor on
Linux/amd64. If you're using the instance runners on GitLab.com, this is enabled by default.CS_REGISTRY_USER and CS_REGISTRY_PASSWORD. For more
details on how to use these variables, see
authenticate to a private external registry.To enable the analyzer, either:
.gitlab-ci.yml file manually.This method automatically prepares a merge request that includes the container scanning template
in the .gitlab-ci.yml file. You then merge the merge request to enable container scanning.
[!note] This method works best with no existing
.gitlab-ci.ymlfile, or with a minimal configuration file. If you have a complex GitLab configuration file it might not be parsed successfully, and an error might occur. In that case, use the manual method instead.
Prerequisites:
test stage exists in the .gitlab-ci.yml file.docker or kubernetes executor on Linux/amd64. If you're using the instance runners on GitLab.com, this is enabled by default.CS_REGISTRY_USER and CS_REGISTRY_PASSWORD. For more
details, see authenticate to a private external registry.To enable container scanning:
Pipelines now include a container scanning job.
.gitlab-ci.yml file manuallyThis method requires you to manually edit the existing .gitlab-ci.yml file. Use this method if
you have a complex GitLab configuration file or you need to use non-default options.
Prerequisites:
test stage exists in the .gitlab-ci.yml file.docker or kubernetes executor on Linux/amd64. If you're using the instance runners on GitLab.com, this is enabled by default.CS_REGISTRY_USER and CS_REGISTRY_PASSWORD. For more
details, see authenticate to a private external registry.To enable container scanning:
In the top bar, select Search or go to and find your project.
Select Build > Pipeline editor.
If no .gitlab-ci.yml file exists, select Configure pipeline, then delete the example
content.
Copy and paste the following to the bottom of the .gitlab-ci.yml file. If an include line
already exists, add only the template line below it.
include:
- template: Jobs/Container-Scanning.gitlab-ci.yml
Select the Validate tab, then select Validate pipeline.
The message Simulation completed successfully confirms the file is valid.
Select the Edit tab.
Complete the fields. Do not use the default branch for the Branch field.
Select the Start a new merge request with these changes checkbox, then select Commit changes.
Complete the fields according to your standard workflow, then select Create merge request.
Review and edit the merge request according to your standard workflow, wait until the pipeline passes, then select Merge.
Pipelines now include a container scanning job.
Prerequisites:
You can review vulnerabilities in a pipeline:
For more details, see Pipeline security report.
Additional ways to see container scanning results:
GitLab provides two approaches for container scanning:
After you are confident in the container scanning results for a single project, you can extend its implementation to additional projects:
The following Linux distributions are supported:
GitLab also offers FIPS-enabled Red Hat UBI
versions of the container-scanning images. You can therefore replace standard images with FIPS-enabled
images. To configure the images, set the CS_IMAGE_SUFFIX to -fips or modify the CS_ANALYZER_IMAGE variable to the
standard tag plus the -fips extension.
[!note] The
-fipsflag is automatically added toCS_ANALYZER_IMAGEwhen FIPS mode is enabled in the GitLab instance.
Container scanning of images in authenticated registries is not supported when FIPS mode
is enabled. When CI_GITLAB_FIPS_MODE is "true", and CS_REGISTRY_USER or CS_REGISTRY_PASSWORD is set,
the analyzer exits with an error and does not perform the scan.
To customize container scanning, use CI/CD variables.
Enable verbose output when you need to see in detail what the dependency scanning job does, for example when troubleshooting.
In the following example, the container scanning template is included and verbose output is enabled.
include:
- template: Jobs/Container-Scanning.gitlab-ci.yml
variables:
SECURE_LOG_LEVEL: 'debug'
The CS_DISABLE_LANGUAGE_VULNERABILITY_SCAN CI/CD variable controls whether the scan reports
findings related to programming languages. For more information about the supported languages, see Language-specific Packages in the Trivy documentation.
By default, the report only includes packages managed by the Operating System (OS) package manager
(for example, yum, apt, apk, tdnf). To report security findings in non-OS packages, set
CS_DISABLE_LANGUAGE_VULNERABILITY_SCAN to "false":
include:
- template: Jobs/Container-Scanning.gitlab-ci.yml
container_scanning:
variables:
CS_DISABLE_LANGUAGE_VULNERABILITY_SCAN: "false"
When you enable this feature, you might see duplicate findings in the vulnerability report if dependency scanning is enabled for your project. This happens because GitLab can't automatically deduplicate findings across different types of scanning tools. To understand which types of dependencies are likely to be duplicated, see Dependency scanning compared to container scanning.
See Use security scanning tools with merge request pipelines.
To customize container scanning, use CI/CD variables. The following table lists CI/CD variables specific to container scanning. You can also use any of the predefined CI/CD variables.
[!warning] Test customization of GitLab analyzers in a merge request before merging these changes to the default branch. Failure to do so can give unexpected results, including a large number of false positives.
| CI/CD Variable | Default | Description |
|---|---|---|
ADDITIONAL_CA_CERT_BUNDLE | "" | Bundle of CA certs that you want to trust. See Using a custom SSL CA certificate authority for more details. |
CI_APPLICATION_REPOSITORY | $CI_REGISTRY_IMAGE/$CI_COMMIT_REF_SLUG | Docker repository URL for the image to be scanned. |
CI_APPLICATION_TAG | $CI_COMMIT_SHA | Docker repository tag for the image to be scanned. |
CS_ANALYZER_IMAGE | registry.gitlab.com/security-products/container-scanning:8 | Docker image of the analyzer. Do not use the :latest tag with analyzer images provided by GitLab. |
CS_DEFAULT_BRANCH_IMAGE | "" | The name of the CS_IMAGE on the default branch. See Setting the default branch image for more details. |
CS_DISABLE_DEPENDENCY_LIST | "false" | {{< icon name="warning" >}} Removed in GitLab 17.0. |
CS_DISABLE_LANGUAGE_VULNERABILITY_SCAN | "true" | Disable scanning for language-specific packages installed in the scanned image. |
CS_DOCKER_INSECURE | "false" | Allow access to secure Docker registries using HTTPS without validating the certificates. |
CS_DOCKERFILE_PATH | Dockerfile | The path to the Dockerfile to use for generating remediations. By default, the scanner looks for a file named Dockerfile in the root directory of the project. You should configure this variable only if your Dockerfile is in a non-standard location, such as a subdirectory. See Solutions for vulnerabilities for more details. |
CS_INCLUDE_LICENSES | "" | If set, this variable includes licenses for each component. It is only applicable to cyclonedx reports and those licenses are provided by trivy |
CS_IGNORE_STATUSES | "" | Force the analyzer to ignore findings with specified statuses in a comma-delimited list. The following values are allowed: unknown,not_affected,affected,fixed,under_investigation,will_not_fix,fix_deferred,end_of_life. <sup>1</sup> |
CS_IGNORE_UNFIXED | "false" | Ignore findings that are not fixed. Ignored findings are not included in the report. |
CS_IMAGE | $CI_APPLICATION_REPOSITORY:$CI_APPLICATION_TAG | The Docker image to be scanned. If set, this variable overrides the $CI_APPLICATION_REPOSITORY and $CI_APPLICATION_TAG variables. |
CS_IMAGE_SUFFIX | "" | Suffix added to CS_ANALYZER_IMAGE. If set to -fips, FIPS-enabled image is used for scan. See FIPS-enabled images for more details. |
CS_QUIET | "" | If set, this variable disables output of the vulnerabilities table in the job log. |
CS_REGISTRY_INSECURE | "false" | Allow access to insecure registries (HTTP only). Should only be set to true when testing the image locally. Works with all scanners, but the registry must listen on port 80/tcp for Trivy to work. |
CS_REGISTRY_PASSWORD | $CI_REGISTRY_PASSWORD | Password for accessing a Docker registry requiring authentication. The default is only set if $CS_IMAGE resides at $CI_REGISTRY. Not supported when FIPS mode is enabled. |
CS_REGISTRY_USER | $CI_REGISTRY_USER | Username for accessing a Docker registry requiring authentication. The default is only set if $CS_IMAGE resides at $CI_REGISTRY. Not supported when FIPS mode is enabled. |
CS_REPORT_OS_EOL | "false" | Enable EOL detection |
CS_REPORT_OS_EOL_SEVERITY | "Medium" | Severity level assigned to EOL OS findings when CS_REPORT_OS_EOL is enabled. EOL findings are always reported regardless of CS_SEVERITY_THRESHOLD. Supported levels are UNKNOWN, LOW, MEDIUM, HIGH, and CRITICAL. |
CS_SEVERITY_THRESHOLD | UNKNOWN | Severity level threshold. The scanner outputs vulnerabilities with severity level higher than or equal to this threshold. Supported levels are UNKNOWN, LOW, MEDIUM, HIGH, and CRITICAL. |
CS_TRIVY_JAVA_DB | "registry.gitlab.com/gitlab-org/security-products/dependencies/trivy-java-db" | Specify an alternate location for the trivy-java-db vulnerability database. |
CS_TRIVY_DETECTION_PRIORITY | "precise" | Scan using the defined Trivy detection priority. The following values are allowed: precise or comprehensive. |
SECURE_LOG_LEVEL | info | Set the minimum logging level. Messages of this logging level or higher are output. From highest to lowest severity, the logging levels are: fatal, error, warn, info, debug. |
TRIVY_TIMEOUT | 5m0s | Set the timeout for the scan. |
TRIVY_PLATFORM | linux/amd64 | Set platform in the format os/arch if image is multi-platform capable. |
Footnotes:
CS_IGNORE_STATUSES can lead to
false positive or false negative filtering of findings when this setting is enabled.In addition to the GitLab-specific CS_* variables listed above, you can configure Trivy directly
by setting any of its native environment variables
in your container_scanning job. The GitLab container scanning analyzer passes all environment
variables through to Trivy automatically.
For example, to manually specify the OS distribution for a container image where Trivy cannot auto-detect it (such as a customized base image):
include:
- template: Jobs/Container-Scanning.gitlab-ci.yml
container_scanning:
variables:
GIT_STRATEGY: fetch
TRIVY_DISTRO: "alma/10"
[!note] The
--distroflag used byTRIVY_DISTROis experimental in Trivy. Results may vary depending on the Trivy version and the distribution specified.
The following TRIVY_* variables are set internally by the GitLab container scanning analyzer.
They are controlled by the corresponding GitLab CI/CD variables and cannot be overridden directly:
| Trivy variable | GitLab CI/CD variable |
|---|---|
TRIVY_CACHE_DIR | (internal, not exposed) |
TRIVY_USERNAME | CS_REGISTRY_USER |
TRIVY_PASSWORD | CS_REGISTRY_PASSWORD |
TRIVY_DEBUG | SECURE_LOG_LEVEL |
TRIVY_INSECURE | CS_DOCKER_INSECURE |
TRIVY_NON_SSL | CS_REGISTRY_INSECURE |
TRIVY_DB_REPOSITORYSetting TRIVY_DB_REPOSITORY to point Trivy to a custom vulnerability database has no effect.
The GitLab container scanning analyzer bundles the vulnerability database inside the analyzer
image and passes --skip-db-update to Trivy at runtime, so Trivy never downloads a database
regardless of this variable. To use a custom database location, see
Use a Trivy Java database mirror for the Java database,
or offline environment for general offline setups.
If you want to override the job definition (for example, to change properties like variables), you
must declare and override a job after the template inclusion, and then
specify any additional keys.
This example sets GIT_STRATEGY to fetch:
include:
- template: Jobs/Container-Scanning.gitlab-ci.yml
container_scanning:
variables:
GIT_STRATEGY: fetch
By default, container scanning scans images in the GitLab container registry. You can also scan images in external registries.
To scan an image in an external registry, configure the CS_IMAGE variable with the full path to
the image.
include:
- template: Jobs/Container-Scanning.gitlab-ci.yml
container_scanning:
variables:
CS_IMAGE: <example.com>/<user>/<image>:<tag>
If the external registry requires authentication, provide credentials using the CS_REGISTRY_USER
and CS_REGISTRY_PASSWORD CI/CD variables.
[!note] Scanning images in an external private registry is not supported when FIPS mode is enabled.
For example, to scan an image in Google Container Registry:
Add a CI/CD variable for GCP_CREDENTIALS containing the JSON key, as described in the
Google Cloud Platform Container Registry documentation.
Add the following to the .gitlab-ci.yml file.
include:
- template: Jobs/Container-Scanning.gitlab-ci.yml
container_scanning:
variables:
CS_REGISTRY_USER: _json_key
CS_REGISTRY_PASSWORD: "$GCP_CREDENTIALS"
CS_IMAGE: "gcr.io/<path-to-your-registry>/<image>:<tag>"
For example, to scan an image in AWS Elastic Container Registry:
Add the following to the .gitlab-ci.yml file:
container_scanning:
before_script:
- ruby -r open-uri -e "IO.copy_stream(URI.open('https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip'), 'awscliv2.zip')"
- unzip awscliv2.zip
- sudo ./aws/install
- export AWS_ECR_PASSWORD=$(aws ecr get-login-password --region <region>)
include:
- template: Jobs/Container-Scanning.gitlab-ci.yml
variables:
CS_IMAGE: <aws_account_id>.dkr.ecr.<region>.amazonaws.com/<image>:<tag>
CS_REGISTRY_USER: AWS
CS_REGISTRY_PASSWORD: "$AWS_ECR_PASSWORD"
AWS_DEFAULT_REGION: <region>
When the trivy scanner is used and a jar file is encountered in a container image being scanned,
trivy downloads an additional trivy-java-db vulnerability database. By default, the
trivy-java-db database is hosted as an OCI artifact at
ghcr.io/aquasecurity/trivy-java-db:1. If this registry is
not accessible or responds with
TOOMANYREQUESTS, one solution is to mirror the trivy-java-db to a more accessible container
registry:
mirror trivy java db:
image:
name: ghcr.io/oras-project/oras:v1.1.0
entrypoint: [""]
script:
- oras login -u $CI_REGISTRY_USER -p $CI_REGISTRY_PASSWORD $CI_REGISTRY
- oras pull ghcr.io/aquasecurity/trivy-java-db:1
- oras push $CI_REGISTRY_IMAGE:1 --config /dev/null:application/vnd.aquasec.trivy.config.v1+json javadb.tar.gz:application/vnd.aquasec.trivy.javadb.layer.v1.tar+gzip
The vulnerability database is not a regular Docker image, so you cannot pull it by using
docker pull. The image shows an error if you view it in the GitLab UI.
If the container registry is gitlab.example.com/trivy-java-db-mirror, then the container scanning
job should be configured in the following way. Do not add the tag :1 at the end, it is added by
trivy:
include:
- template: Jobs/Container-Scanning.gitlab-ci.yml
container_scanning:
variables:
CS_TRIVY_JAVA_DB: gitlab.example.com/trivy-java-db-mirror
By default, container scanning assumes that the image naming convention stores any branch-specific identifiers in the image tag rather than the image name. When the image name differs between the default branch and the non-default branch, previously-detected vulnerabilities show up as newly detected in merge requests.
When the same image has different names on the default branch and a non-default branch, you can use
the CS_DEFAULT_BRANCH_IMAGE variable to indicate what that image's name is on the default branch.
GitLab then correctly determines if a vulnerability already exists when running scans on non-default
branches.
As an example, suppose the following:
$CI_REGISTRY_IMAGE/$CI_COMMIT_BRANCH:$CI_COMMIT_SHA.$CI_REGISTRY_IMAGE:$CI_COMMIT_SHA.In this example, you can use the following CI/CD configuration to ensure that vulnerabilities aren't duplicated:
include:
- template: Jobs/Container-Scanning.gitlab-ci.yml
container_scanning:
variables:
CS_DEFAULT_BRANCH_IMAGE: $CI_REGISTRY_IMAGE:$CI_COMMIT_SHA
before_script:
- export CS_IMAGE="$CI_REGISTRY_IMAGE/$CI_COMMIT_BRANCH:$CI_COMMIT_SHA"
- |
if [ "$CI_COMMIT_BRANCH" == "$CI_DEFAULT_BRANCH" ]; then
export CS_IMAGE="$CI_REGISTRY_IMAGE:$CI_COMMIT_SHA"
fi
CS_DEFAULT_BRANCH_IMAGE should remain the same for a given CS_IMAGE. If it changes, then a
duplicate set of vulnerabilities are created, which must be manually dismissed.
When using Auto DevOps, CS_DEFAULT_BRANCH_IMAGE is
automatically set to $CI_REGISTRY_IMAGE/$CI_DEFAULT_BRANCH:$CI_APPLICATION_TAG.
You can use the ADDITIONAL_CA_CERT_BUNDLE CI/CD variable to configure a custom SSL CA certificate authority, which is used to verify the peer when fetching Docker images from a registry which uses HTTPS. The ADDITIONAL_CA_CERT_BUNDLE value should contain the text representation of the X.509 PEM public-key certificate. For example, to configure this value in the .gitlab-ci.yml file, use the following:
container_scanning:
variables:
ADDITIONAL_CA_CERT_BUNDLE: |
-----BEGIN CERTIFICATE-----
MIIGqTCCBJGgAwIBAgIQI7AVxxVwg2kch4d56XNdDjANBgkqhkiG9w0BAQsFADCB
...
jWgmPqF3vUbZE0EyScetPJquRFRKIesyJuBFMAs=
-----END CERTIFICATE-----
The ADDITIONAL_CA_CERT_BUNDLE value can also be configured as a custom variable in the UI, either
as a file, which requires the path to the certificate, or as a variable, which requires the text
representation of the certificate.
You can use the TRIVY_PLATFORM CI/CD variable to configure the container scan to run against a specific
operating system and architecture. For example, to configure this value in the .gitlab-ci.yml file, use
the following:
container_scanning:
# Use an arm64 SaaS runner to scan this natively
tags: ["saas-linux-small-arm64"]
variables:
TRIVY_PLATFORM: "linux/arm64"
{{< details >}}
{{< /details >}}
Prerequisites:
To allowlist specific vulnerabilities, follow these steps:
GIT_STRATEGY: fetch in your .gitlab-ci.yml file by following the instructions in
overriding the container scanning template.vulnerability-allowlist.yml. This must use
the format described in vulnerability-allowlist.yml data format.vulnerability-allowlist.yml file to the root folder of your project's Git repository.vulnerability-allowlist.yml data formatThe vulnerability-allowlist.yml file is a YAML file that specifies a list of CVE IDs of vulnerabilities that are allowed to exist, because they're false positives, or they're not applicable.
If a matching entry is found in the vulnerability-allowlist.yml file, the following happens:
gl-container-scanning-report.json file.Example vulnerability-allowlist.yml file:
generalallowlist:
CVE-2019-8696:
CVE-2014-8166: cups
CVE-2017-18248:
images:
registry.gitlab.com/gitlab-org/security-products/dast/webgoat-8.0@sha256:
CVE-2018-4180:
your.private.registry:5000/centos:
CVE-2015-1419: libxml2
CVE-2015-1447:
This example excludes from gl-container-scanning-report.json:
CVE-2019-8696, CVE-2014-8166, CVE-2017-18248.registry.gitlab.com/gitlab-org/security-products/dast/webgoat-8.0@sha256 container image with CVE ID CVE-2018-4180.your.private.registry:5000/centos container with CVE IDs CVE-2015-1419, CVE-2015-1447.generalallowlist block allows you to specify CVE IDs globally. All vulnerabilities with matching CVE IDs are excluded from the scan report.
images block allows you to specify CVE IDs for each container image independently. All vulnerabilities from the given image with matching CVE IDs are excluded from the scan report. The image name is retrieved from one of the environment variables used to specify the Docker image to be scanned, such as $CI_APPLICATION_REPOSITORY:$CI_APPLICATION_TAG or CS_IMAGE. The image provided in this block must match this value and must not include the tag value. For example, if you specify the image to be scanned using CS_IMAGE=alpine:3.7, then you would use alpine in the images block, but you cannot use alpine:3.7.
You can specify container image in multiple ways:
centos).your.private.registry:5000/centos).registry.gitlab.com/gitlab-org/security-products/dast/webgoat-8.0@sha256).[!note] The string after CVE ID (
cupsandlibxml2in the previous example) is an optional comment format. It has no impact on the handling of vulnerabilities. You can include comments to describe the vulnerability.
You can verify the results of your scan and the correctness of your vulnerability-allowlist.yml file by looking
at the logs that are produced by the container scanning analyzer in container_scanning job details.
The log contains a list of found vulnerabilities as a table, for example:
+------------+-------------------------+------------------------+-----------------------+------------------------------------------------------------------------+
| STATUS | CVE SEVERITY | PACKAGE NAME | PACKAGE VERSION | CVE DESCRIPTION |
+------------+-------------------------+------------------------+-----------------------+------------------------------------------------------------------------+
| Approved | High CVE-2019-3462 | apt | 1.4.8 | Incorrect sanitation of the 302 redirect field in HTTP transport metho |
| | | | | d of apt versions 1.4.8 and earlier can lead to content injection by a |
| | | | | MITM attacker, potentially leading to remote code execution on the ta |
| | | | | rget machine. |
+------------+-------------------------+------------------------+-----------------------+------------------------------------------------------------------------+
| Unapproved | Medium CVE-2020-27350 | apt | 1.4.8 | APT had several integer overflows and underflows while parsing .deb pa |
| | | | | ckages, aka GHSL-2020-168 GHSL-2020-169, in files apt-pkg/contrib/extr |
| | | | | acttar.cc, apt-pkg/deb/debfile.cc, and apt-pkg/contrib/arfile.cc. This |
| | | | | issue affects: apt 1.2.32ubuntu0 versions prior to 1.2.32ubuntu0.2; 1 |
| | | | | .6.12ubuntu0 versions prior to 1.6.12ubuntu0.2; 2.0.2ubuntu0 versions |
| | | | | prior to 2.0.2ubuntu0.2; 2.1.10ubuntu0 versions prior to 2.1.10ubuntu0 |
| | | | | .1; |
+------------+-------------------------+------------------------+-----------------------+------------------------------------------------------------------------+
| Unapproved | Medium CVE-2020-3810 | apt | 1.4.8 | Missing input validation in the ar/tar implementations of APT before v |
| | | | | ersion 2.1.2 could result in denial of service when processing special |
| | | | | ly crafted deb files. |
+------------+-------------------------+------------------------+-----------------------+------------------------------------------------------------------------+
Vulnerabilities in the log are marked as Approved when the corresponding CVE ID is added to the vulnerability-allowlist.yml file.
{{< details >}}
{{< /details >}}
To run container scanning in an offline environment, you must do an initial setup and perform ongoing maintenance.
Initial setup:
docker or kubernetes executor is available).Ongoing maintenance:
Configure runner (ensure the docker or kubernetes executor is available). For details,
see getting started.
By default the runner pulls container images from the GitLab container registry even if a local
copy is available. If you prefer to use only local container images, you can change the
pull_policy can be set to if-not-present.
However, you should keep the pull_policy setting to the default unless you have good reason to
change it.
Import the following image from the GitLab.com container registry into your local container registry. This image must be accessible from your offline GitLab instance.
registry.gitlab.com/security-products/container-scanning:8
The process for importing images into a local container registry depends on your network security policy. Consult your IT staff to find an accepted and approved process by which you can import or temporarily access external resources.
[!note] These configuration changes do not apply to container scanning for registry because it does not reference the
.gitlab-ci.ymlfile. To configure automatic container scanning for registry in an offline environment, define theCS_ANALYZER_IMAGEvariable in the GitLab UI instead.
For all projects using container scanning, edit the CI/CD configuration in all locations where it's applied. This might include:
.gitlab-ci.yml filesUpdate your container scanning configuration with the following variables:
CS_ANALYZER_IMAGE to the container scanning image in your local container registry.CS_REGISTRY_USER and
CS_REGISTRY_PASSWORD to match your registry credentials.CS_DOCKER_INSECURE: "true".Example .gitlab-ci.yml configuration for an offline environment:
include:
- template: Jobs/Container-Scanning.gitlab-ci.yml
container_scanning:
variables:
# Container scanning-specific variables
CS_ANALYZER_IMAGE: <hostname>:<port>/analyzers/container-scanning:8
CS_REGISTRY_USER: <username>
CS_REGISTRY_PASSWORD: <password>
CS_DOCKER_INSECURE: "true"
The container scanning image is periodically updated and pushed to the GitLab.com registry. In an offline environment, you must update the container scanning image in your local container registry, either automatically (recommended) or manually.
The following .gitlab-ci.yml extract demonstrates how to automatically update the container
scanning image in a local registry. This method defines variables for the source image and target
image, then uses the Docker CLI to pull the image from the GitLab.com registry and push it to your
local registry.
If you're using a non-GitLab registry, update the CI_REGISTRY value and configure authentication
by setting CI_REGISTRY_USER and CI_REGISTRY_PASSWORD variables to match your local registry
credentials.
variables:
SOURCE_IMAGE: registry.gitlab.com/security-products/container-scanning:8
TARGET_IMAGE: $CI_REGISTRY/namespace/container-scanning
image: docker:cli
update-scanner-image:
services:
- docker:dind
script:
- docker pull $SOURCE_IMAGE
- docker tag $SOURCE_IMAGE $TARGET_IMAGE
- echo "$CI_REGISTRY_PASSWORD" | docker login $CI_REGISTRY --username $CI_REGISTRY_USER --password-stdin
- docker push $TARGET_IMAGE
{{< history >}}
{{< /history >}}
Container scanning supports images in archive formats (.tar, .tar.gz).
Such images may be created, for example, using docker save or docker buildx build.
To scan an archive file, set the environment variable CS_IMAGE to the format archive://path/to/archive:
archive:// scheme prefix specifies that the analyzer is to scan an archive.path/to/archive specifies the path to the archive to scan, whether an absolute path or a relative path.Container scanning supports tar image files following the Docker Image Specification. OCI tarballs are not supported. For more information regarding supported formats, see Trivy tar file support.
Container scanning uses metadata from the tar file for image naming. When building tar image files, ensure the image is tagged:
# Pull or build an image with a name and a tag
docker pull image:latest
# OR
docker build . -t image:latest
# Then export to tar using docker save
docker save image:latest -o image-latest.tar
# Or build an image with a tag using buildx build
docker buildx create --name container --driver=docker-container
docker buildx build -t image:latest --builder=container -o type=docker,dest=- . > image-latest.tar
# With podman
podman build -t image:latest .
podman save -o image-latest.tar image:latest
Container scanning determines the image name by first evaluating the archive's manifest.json and using the first item in RepoTags.
If this is not found, index.json is used to fetch the io.containerd.image.name annotation. If this is not found, the archive filename
is used instead.
manifest.json is defined in Docker Image Specification v1.1.0
and created by using the command docker save.index.json format is defined in the OCI image specification v1.1.1.
io.containerd.image.name is available in containerd v1.3.0 and later
when using ctr image export.To scan an archive built in a CI/CD job, you must pass the archive artifact from the build job to the container scanning job.
Use the artifacts:paths and dependencies keywords to pass artifacts from one job to a following one:
build_job:
script:
- docker build . -t image:latest
- docker save image:latest -o image-latest.tar
artifacts:
paths:
- "image-latest.tar"
container_scanning:
variables:
CS_IMAGE: "archive://image-latest.tar"
dependencies:
- build_job
To scan an archive found in your project repository, ensure that your Git strategy enables access to your repository.
Set the GIT_STRATEGY keyword to either clone or fetch in the container_scanning job because it is set to none by default.
container_scanning:
variables:
GIT_STRATEGY: fetch
It's possible to run the GitLab container scanning tool against a Docker container without needing to run it within the context of a CI job. To scan an image directly, follow these steps:
Run Docker Desktop or Docker Machine.
Run the analyzer's Docker image, passing the image and tag you want to analyze in the
CI_APPLICATION_REPOSITORY and CI_APPLICATION_TAG variables:
docker run \
--interactive --rm \
--volume "$PWD":/tmp/app \
-e CI_PROJECT_DIR=/tmp/app \
-e CI_APPLICATION_REPOSITORY=registry.gitlab.com/gitlab-org/security-products/dast/webgoat-8.0@sha256 \
-e CI_APPLICATION_TAG=bc09fe2e0721dfaeee79364115aeedf2174cce0947b9ae5fe7c33312ee019a4e \
registry.gitlab.com/security-products/container-scanning
The results are stored in gl-container-scanning-report.json.
The container scanning tool emits JSON reports which the GitLab Runner recognizes through the
artifacts:reports keyword in the CI/CD configuration file.
After the CI/CD job finishes, the runner uploads these reports to GitLab, which are then available in the CI/CD job artifacts. In GitLab Ultimate, these reports can be viewed in the corresponding pipeline and the vulnerability report.
These reports must comply with the container scanning report schema.
Example container scanning report.
In addition to the JSON report file, the container scanning tool outputs a
CycloneDX Software Bill of Materials (SBOM) for the scanned image. This
CycloneDX SBOM is named gl-sbom-report.cdx.json and is saved in the same directory as the JSON report file. This feature is only supported when the Trivy analyzer is used.
This report can be viewed in the dependency list.
You can download CycloneDX SBOMs the same way as other job artifacts.
{{< history >}}
{{< /history >}}
Container scanning can include license information in CycloneDX reports. This feature is disabled by default to maintain backward compatibility.
To enable license scanning in your container scanning results:
CS_INCLUDE_LICENSES variable in your .gitlab-ci.yml file:container_scanning:
variables:
CS_INCLUDE_LICENSES: "true"
It is important to mention that only SPDX licenses are supported. However, licenses that are non-compliant with SPDX will still be ingested without any user-facing error.
Container scanning includes the ability to detect and report when your container images are using operating systems that have reached their end-of-life (EOL). Operating systems that have reached EOL no longer receive security updates, leaving them vulnerable to newly discovered security issues.
The EOL detection feature uses Trivy to identify operating systems that are no longer supported by their respective distributions. When an EOL operating system is detected, it's reported as a vulnerability in your container scanning report alongside other security findings.
To enable EOL detection, set CS_REPORT_OS_EOL to "true".
{{< details >}}
{{< /details >}}
{{< history >}}
enable_container_scanning_for_registry. Disabled by default.enable_container_scanning_for_registry removed.{{< /history >}}
When a container image is pushed with the latest tag, a container scanning job is automatically triggered by the security policy bot in a new pipeline against the default branch.
Unlike regular container scanning, the scan results do not include a security report. Instead, container scanning for registry relies on continuous vulnerability scanning to inspect the components detected by the scan.
When security findings are identified, GitLab populates the vulnerability report with these findings. Vulnerabilities can be viewed under the Container registry vulnerabilities tab of the vulnerability report page.
Container scanning for registry populates the vulnerability report only when a new advisory is published to the GitLab advisory database. Support for populating the vulnerability report with all present advisory data, instead of only newly-detected data, is proposed in epic 11219.
[!warning] Vulnerabilities detected by container scanning for registry cannot be automatically marked as resolved when you update or remove vulnerable components. These vulnerabilities remain visible indefinitely because this feature only generates SBOMs, not the security reports required for vulnerability resolution.
Prerequisites:
50 scans per project per day.To turn on container scanning for the GitLab container registry:
To use container scanning for registry in an offline or air-gapped environment, you must use a local copy of the container scanning analyzer image. Because this feature is managed by the GitLab Security Policy Bot, the analyzer image cannot be configured by editing the .gitlab-ci.yml file.
Instead, you must override the default scanner image by setting the CS_ANALYZER_IMAGE CI/CD
variable in the GitLab UI. The dynamically-created scanning job inherits variables defined in the
UI. You can use a project, group, or instance CI/CD variable.
Prerequisites:
To configure a custom scanner image:
CS_ANALYZER_IMAGEmy.local.registry:5000/analyzers/container-scanning:7.The GitLab Security Policy Bot will use the specified image when it triggers a scan.
All analyzer images are updated daily.
The images use data from upstream advisory databases:
In addition to the sources provided by these scanners, GitLab maintains the following vulnerability databases:
In the GitLab Ultimate tier, the data from the GitLab advisory database is merged in to augment the data from the external sources. In the GitLab Premium and Free tiers, the data from the GitLab Advisory Database (Open Source Edition) is merged in to augment the data from the external sources. This augmentation only applies to the analyzer images for the Trivy scanner.
Database update information for other analyzers is available in the maintenance table.
{{< details >}}
{{< /details >}}
Some vulnerabilities can be fixed by applying the solution that GitLab automatically generates.
To enable remediation support, the scanning tool must have access to the Dockerfile specified by
the CI/CD variable CS_DOCKERFILE_PATH. To ensure that the scanning tool
has access to this
file, it's necessary to set GIT_STRATEGY: fetch in
your .gitlab-ci.yml file by following the instructions described in this document's
overriding the container scanning template section.
Read more about the solutions for vulnerabilities.