Make Docker-in-Docker builds faster with Docker layer caching

{{< details >}}

• Tier: Free, Premium, Ultimate
• Offering: GitLab.com, GitLab Self-Managed, GitLab Dedicated

{{< /details >}}

When using Docker-in-Docker, Docker downloads all layers of your image every time you create a build. Recent versions of Docker (Docker 1.13 and later) can use a pre-existing image as a cache during the docker build step. This significantly accelerates the build process.

In Docker 27.0.1 and later, the default docker build driver only supports cache backends when the containerd image store is enabled.

To use Docker caching with Docker 27.0.1 and later, do one of the following:

• Enable the containerd image store in your Docker daemon configuration.
• Select a different build driver.

For more information, see Cache storage backends.

How Docker caching works

When running docker build, each command in Dockerfile creates a layer. These layers are retained as a cache and can be reused if there have been no changes. Change in one layer causes the recreation of all subsequent layers.

To specify a tagged image to be used as a cache source for the docker build command, use the --cache-from argument. Multiple images can be specified as a cache source by using multiple --cache-from arguments.

Docker inline caching example

This example .gitlab-ci.yml file shows how to use Docker caching with the inline cache backend with the default docker build command.

default:
  image: docker:27.4.1-cli
  services:
    - docker:27.4.1-dind
  before_script:
    - docker login -u $CI_REGISTRY_USER -p $CI_REGISTRY_PASSWORD $CI_REGISTRY

variables:
  # Use TLS https://docs.gitlab.com/ci/docker/using_docker_build/#tls-enabled
  DOCKER_HOST: tcp://docker:2376
  DOCKER_TLS_CERTDIR: "/certs"

build:
  stage: build
  script:
    - docker pull $CI_REGISTRY_IMAGE:latest || true
    - docker build --build-arg BUILDKIT_INLINE_CACHE=1 --cache-from $CI_REGISTRY_IMAGE:latest --tag $CI_REGISTRY_IMAGE:$CI_COMMIT_SHA --tag $CI_REGISTRY_IMAGE:latest .
    - docker push $CI_REGISTRY_IMAGE:$CI_COMMIT_SHA
    - docker push $CI_REGISTRY_IMAGE:latest

In the script section for the build job:

1. The first command tries to pull the image from the registry so that it can be used as a cache for the docker build command. Any image that's used with the --cache-from argument must be pulled (using docker pull) before it can be used as a cache source.
2. The second command builds a Docker image by using the pulled image as a cache (see the --cache-from $CI_REGISTRY_IMAGE:latest argument) if available, and tags it. The --build-arg BUILDKIT_INLINE_CACHE=1 tells Docker to use inline caching, which embeds the build cache into the image itself.
3. The last two commands push the tagged Docker images to the container registry so that they can also be used as cache for subsequent builds.

Docker registry caching example

You can cache your Docker builds directly to a dedicated cache image in the registry.

This example .gitlab-ci.yml file shows how to use Docker caching with the docker buildx build command and the registry cache backend. For more advanced caching options, see Cache storage backends.

default:
  image: docker:27.4.1-cli
  services:
    - docker:27.4.1-dind
  before_script:
    - docker login -u $CI_REGISTRY_USER -p $CI_REGISTRY_PASSWORD $CI_REGISTRY

variables:
  # Use TLS https://docs.gitlab.com/ci/docker/using_docker_build/#tls-enabled
  DOCKER_HOST: tcp://docker:2376
  DOCKER_TLS_CERTDIR: "/certs"

build:
  stage: build
  script:
    - docker context create my-builder
    - docker buildx create my-builder --driver docker-container --use
    - docker buildx build --push -t $CI_REGISTRY_IMAGE:$CI_COMMIT_SHA
      --cache-to type=registry,ref=$CI_REGISTRY_IMAGE/cache-image,mode=max
      --cache-from type=registry,ref=$CI_REGISTRY_IMAGE/cache-image .

The build job's script:

1. Creates and configures the docker-container BuildKit driver, which supports the registry cache backend.

2. Builds and pushes a Docker image using:

   • A dedicated cache image with --cache-from type=registry,ref=$CI_REGISTRY_IMAGE/cache-image.
   • Cache updates with --cache-to type=registry,ref=$CI_REGISTRY_IMAGE/cache-image,mode=max, where max mode caches intermediate layers.