Docker build caching in GitLab

As a software developer, you’re probably familiar with the time it takes to build a Docker image, especially if your application is complex and requires many dependencies. But did you know that you can speed up your builds by using caching strategies?

In this article, we’ll explore the different ways to optimize your Docker builds in GitLab, by using the BUILDKIT_INLINE_CACHE, tagging with both latest and sha_commit, and using the docker-compose cache from option.

BUILDKIT_INLINE_CACHE

The BUILDKIT_INLINE_CACHE environment variable can be used to enable build caching in GitLab CI/CD pipelines. When set to 1, it allows GitLab to cache the intermediate build layers, which can significantly speed up subsequent builds. To enable this feature, simply add the following line to your .gitlab-ci.yml file:

variables:   DOCKER_BUILDKIT: 1

Tagging with both latest and sha_commit

Tagging your Docker images with both the latest and the sha_commit (the unique identifier for each commit) can also improve your build times. By tagging with the latest tag, you can ensure that your latest build is always accessible, and by tagging with the sha_commit, you can guarantee that you can access specific builds at any time.

image: docker:stable  
stages:   
    - build  
build:   
    stage: build   
    script:     
        - docker build -t myimage:latest .     
        - docker build -t myimage:$CI_COMMIT_SHA .

Using docker-compose cache from

The docker-compose cache from option allows you to cache intermediate build layers when using docker-compose. This can be particularly useful when building multi-container applications, as it allows you to cache the dependencies and intermediate layers for each container. To use this feature, simply add the following line to your docker-compose.yml file:

version: '3.7' 
services:   
    web:     
        build: .     
        cache_from:       
            - web:latest

What to expect

One of the benefits of combining these caching strategies is that you can leverage the cache without prior pulling of all the last images' layers. This means that you don't need to wait for the entire image to be pulled from the repository to start the build process. Instead, GitLab can use the cached layers from previous builds to start the build process, reducing the time it takes to complete the build.

By using the BUILDKIT_INLINE_CACHE environment variable, GitLab can cache the intermediate build layers, which can be used in subsequent builds. This means that if you make changes to your code, docker will only have to build the layers that have changed, instead of starting from scratch. This can significantly speed up your builds and reduce the time it takes to complete a build.

The docker-compose cache from option also works in a similar way, allowing you to cache the dependencies and intermediate layers for each container in your multi-container application. This means that if you make changes to one container, you don't need to rebuild all the other containers from scratch, as the cached layers can be used.

In summary, by leveraging these caching strategies, you can significantly speed up your Docker builds in GitLab, without having to wait for the entire image to be pulled from the repository. This can save you time, reduce the number of build failures, and improve the overall performance of your CI/CD pipelines.