How to debug Docker image intermediate layers

Published June 4, 2024

I use Docker buildx since the early days, it adds a lot of values compared with its predecessor, better parallelization and various performance optimization. It is also based on a library called buildkit used by many other opensource projects and commercial solutions. Generally this is an added value because it is tested under difference circumstances.

It worked so well that I didn’t have any reason to move back to the internal docker build system until today where I was moving a Rust based application from rust:latest to chainguard rust, the build was failing and I was used to take the ID of an intermediate layer and jump straight via docker run. That’s when I noticed that such ID was not there anymore, buildx doe snot exposes it. So I started looking around to see why and what’s the new way to troubleshoot images.

As often happens regressions are opportunities for new project and Brandon Mitchell a fellow Docker Captain suggested to look at buildg and interactive debugger from Docker Images, he didn’t try it yet, some here I am sure it works but I was looking for my old an easy layer ID as part of the docker build output. So my solution is to disable buildx when I need to do this operation, not what I would like to do but there is an issue tracking this problem: “Improved debugging supportĀ #1472”

DOCKER_BUILDKIT=0 docker build -t sssql-chainguard -f Dockerfile.chainguard .
[sudo] password for gianarb:
DEPRECATED: The legacy builder is deprecated and will be removed in a future release.
            BuildKit is currently disabled; enable it by removing the DOCKER_BUILDKIT=0
            environment-variable.

Sending build context to Docker daemon  100.9kB
Step 1/8 : FROM cgr.dev/chainguard/rust as builder
latest: Pulling from chainguard/rust
e0698bbbb4a2: Already exists
Digest: sha256:96f6890a4cb69ac3fcded5f4b559a30871b03e1ad969713eca5b980e4af4e07b
Status: Downloaded newer image for cgr.dev/chainguard/rust:latest

The layer e0698bbbb4a2 can be used to run a container and inspect what’s inside:

docker run -it --rm e0698bbbb4a2

Are you having trouble figuring out your way to building automation, release and troubleshoot your software? Let's get actionables lessons learned straight to you via email.