TL;DR

This post provides hands-on tutorial on how to use BuildKit from Docker Compose! Below is a sample command.

docker compose build --builder=container

BuildKit and Docker

BuildKit is an open-source project that provides a more flexible and efficient way to build Docker images compared to the traditional Docker build system. Now Docker itself is trying to take advantage of BuildKit, so BuildKit is the default builder for users on Docker Desktop, and Docker Engine as of version 23.0. If you’re interested in BuildKit, please check links below!

docker docs: BuildKit
GitHub Repository: moby/buildkit

Build drivers

Before I talk about Docker Compose, let me share about Build drivers. Build drivers are configurations for how and where the BuildKit backend runs. There are four types of drivers. docker driver is default driver but it has a lot of limitation, in other words, we cannot use many BuildKit’s features with docker driver.

So, if you’d like to use full features of BuildKit, you need to set up drivers other than docker driver. The following hands-on tutorial will show you how to set up the docker-container driver.

BuildKit and Docker Compose

Docker Compose V2 was released at 2022 April. At that point, BuildKit support is native within V2 and enabled by default, as it is in the Docker CLI. For more details of V2, please check docker’s blog titled Announcing Compose V2 General Availability.

However, unfortunately, there are few documents and blog posts that explain about how to use BuildKit from Docker Compose. So, I carefully checked docker compose’s documents and found some options that related to BuildKit. In this post, I will share them!

docker compose build has --builder option. The following hands-on tutorial will show you how to set BuildKit (a.k.a docker-container driver) as the builder.
Compose Build Specification has several attributes that can be used by BuildKit such as ssh, cache_from and cache_to. The following hands-on tutorial will show you how to use cache_from and cache_to options.

Hands-on: Use BuildKit from Docker Compose

Preparation

Please check your Docker Compose version. If it’s V1, please install V2 before start hand-on.

Here’s my Docker Compose Docker and macOS versions.

$ docker compose version
Docker Compose version v2.23.3-desktop.2

$ docker version
Client:
 Cloud integration: v1.0.35+desktop.5
 Version:           24.0.7
 API version:       1.43
 Go version:        go1.20.10
 Git commit:        afdd53b
 Built:             Thu Oct 26 09:04:20 2023
 OS/Arch:           darwin/arm64
 Context:           desktop-linux

Server: Docker Desktop 4.26.1 (131620)
 Engine:
  Version:          24.0.7
  API version:      1.43 (minimum version 1.12)
  Go version:       go1.20.10
  Git commit:       311b9ff
  Built:            Thu Oct 26 09:08:15 2023
  OS/Arch:          linux/arm64
  Experimental:     false
 containerd:
  Version:          1.6.25
  GitCommit:        d8f198a4ed8892c764191ef7b3b06d8a2eeb5c7f
 runc:
  Version:          1.1.10
  GitCommit:        v1.1.10-0-g18a0cb0
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0

$ sw_vers
ProductName:		macOS
ProductVersion:		13.6.3
BuildVersion:		22G436

Set up docker-container driver

Let’s set up docker-container driver so that we can use full features of BuildKit by following this document. I recommend you to add the --debug flag so that you can view debug logs of BuildKit in later phases.

docker buildx create --name=container \
--driver=docker-container \
--buildkitd-flags '--debug' \
--use --bootstrap

docker-container driver is running as a container on Docker. So, you can find it by docker ps command!

docker ps | grep buildkit

d9967cdf94e6 moby/buildkit:buildx-stable-1 "buildkitd --debug" 2 minutes ago Up 2 minutes buildx_buildkit_container0

Clone the repository from GitHub

I already prepared contents for hands-on, so please clone it from my repository named buildkit–101.

git clone https://github.com/yukinakanaka/buildkit-101.git

Then, move to docker-compose directory

cd buildkit-101/docker-compose

Walk through the contents

In this folder, there are two projects. service-python is Python project, service-rust is Rust project. Both of them have their own Dockerfile.

$ tree . -L 2
.
├── docker-compose.yaml
├── service-python
│   ├── Dockerfile.python.yaml
│   ├── app
│   └── requirements.txt
└── service-rust
    ├── Cargo.lock
    ├── Cargo.toml
    ├── Dockerfile.rust.yaml
    ├── src
    └── target

docker-compose.yaml contains the two projects. Also, it uses build.cache_from and build.cache_to attributions. These are attributes that allow BuildKit to import and export caches to and from the host machine!

$ cat docker-compose.yaml                                           
version: '3'
services:
  service-python:
    build:
      context: ./service-python
      dockerfile: Dockerfile.python.yaml
      cache_from:
        - type=local,src=./.buildkit-cache/service-python
      cache_to:
        - type=local,dest=./.buildkit-cache/service-python,mode=max
    ports:
      - 10080:80
  service-rust:
    build:
      context: ./service-rust
      dockerfile: Dockerfile.rust.yaml
      cache_from:
        - type=local,src=./.buildkit-cache/service-rust
      cache_to:
        - type=local,dest=./.buildkit-cache/service-rust,mode=max
    ports:
      - 1142:6142

docker compose build

Let’s build two images! We can set builder explicitly with --builder option.

docker compose build --builder=container

Exported local cache

Cache is a big feature of BuildKit. There are many types of caches, such as Inline, Local, Registory, etc. Local cache is a good choice if you’re just testing, or if you want the flexibility to self-manage a shared storage solution.

If you can see cache in .buildkit-cache directory, it worked will!

$ tree .buildkit-cache                           
.buildkit-cache
├── service-python
│   ├── blobs
│   │   └── sha256
│   │       ├── 14aea17807c4c653827ca820a0526d96552bda685bf29293e8be90d1b05662f6
│   │       ├── 35f10c4cf03d43a7afd688184e68c55cc3adc2049e78dffcc419a28f8ee8ac68
│   │       ├── 3fe349e029970270c77c6cc4e8c8159d6390de76a4c30666e585e9ac57e54212
│   │       ├── 40e83eb7650a9e09b6eb95f802f94ab7013cdcefbd6888200e238dcf0427c714
│   │       ├── 4ce6952016f067edb48d6b3297029a381f891967919888f2f066829e54e0e6da
│   │       ├── 6fbfb7c9bfe8763afda1ac7dd5e1d68f77507f66442f105873450be0f71fdea8
│   │       ├── 7399277d91f4520aebebe70097c48af726818282fbbc79c1fb34b53f088eb090
│   │       ├── 78086c75b8f792d954528b9af8387e6c8a01a92f4e33ab8af846310785a54b61
│   │       ├── 955cf54fa69dc2e5bc556d834f3659cedcda24b97bf9892c74849213c44345b0
│   │       ├── c912bacf30db5e2e041614253badd199094e4bdada55d5540dd3cf56aa4d708e
│   │       └── d191be7a3c9fa95847a482db8211b6f85b45096c7817fdad4d7661ee7ff1a421
│   ├── index.json
│   ├── ingest
│   └── oci-layout
└── service-rust
    ├── blobs
    │   └── sha256
    │       ├── 409bf65580386de8886329931c5566015d671b2c917cb4a9266d70342c3c65f6
    │       ├── 4319945d07eb9b83dc46a64480fc19ca523ce682bbf967d230af22f960ffffea
    │       ├── 5cdd9a70365f741a6b9f7a4e32cdb7d4aa29ac73da0b78ca0a83e937f285fdd5
    │       ├── 76a133b23caf160b26813cfb3565ada34c18cf0f74be5c466bb1039450c4deb2
    │       ├── 76b614c6afefd3c24889ccc48f1fd709eb45a92b3e91f43015778642627646d3
    │       ├── 8c94e81ed9ab6e9bf199821d8c2ca4f9ffe5fb550b79b20fab1437d1e699b7e6
    │       ├── 8d647f1dd7e741209a8a75083ccc889e39cb3e94c17f45441eae96e1a679d971
    │       ├── 8eb092f1267503e0a7fbaeec4819656955a5c83581d73039b34f78e78806d401
    │       ├── 95089c600b361807380090316c250b0b8eaf4fa2175b11ac8f49bb7581c61125
    │       └── df2021ddb7d686bdbb125598b2a6163d63035f080356b3014595f354ea0b40d6
    ├── index.json
    ├── ingest
    └── oci-layout

9 directories, 25 files

BuildKit Log

If you’d like to check BuildKit’s behaviors, you can see its logs by the following command.

docker logs buildx_buildkit_container0 --tail 5 

time="2024-01-16T12:16:23Z" level=debug msg="content garbage collected" d=1.319875ms
time="2024-01-16T12:16:23Z" level=debug msg="removed snapshot" key=buildkit/285/nr0yvdcqg7i1zeif0571ss6zj snapshotter=overlayfs
time="2024-01-16T12:16:23Z" level=debug msg="removed snapshot" key=buildkit/286/95loxyg9wz2nf6jppq9eidcyo snapshotter=overlayfs
time="2024-01-16T12:16:23Z" level=debug msg="removed snapshot" key=buildkit/287/qa7ci42nvtetdp06rk06xir8s snapshotter=overlayfs
time="2024-01-16T12:16:23Z" level=debug msg="snapshot garbage collected" d=3.903917ms snapshotter=overlayfs

FYI: Cache export is not supported for the docker driver

Cache export is not supported for the docker driver in Docker Compose version v2.23.3-desktop.2.

$ docker compose build --builder=desktop-linux
[+] Building 0.0s (0/0)                                                                                                                                                                                docker:desktop-linux
Cache export is not supported for the docker driver.
Switch to a different driver, or turn on the containerd image store, and try again.
Learn more at https://docs.docker.com/go/build-cache-backends/

docker compose up –build

Set the docker-container builder to default

Unlike docker build command, docker compose up doesn’t have --builder option. So, in order to use docker-container driver in docker compose up, we have to set it to default by

docker buildx use container

Let’s run docker compose up with –option!

docker compose up --build

If build runs and you can see containers running, it’s OK!

Wrap up

In this post, I explained how to use BuildKit from Docker Compose command! Docker is now trying to introduce BuildKit features into Docker and Docker Compose now. I’m looking for new features of them! I hope this post may help someone. 🐳