Handling Docker Build Secrets the secure way

TL;DR

Use the RUN --mount=type=secret,id=secret-key,target=/root/secret/ command in the Dockerfile and assign it at for building via --secret

Problem

Consider you have something that you need to download software. But as you are in a company network, you need to go via a proxy with your personal credentials or any other credentials inside your build pipeline.

After you downloaded the software, you remove the credentials from the layer and are done. Unfortunatly, this is not a save method to handle secrets.

Lets start with a bad Dockerfile that we will use for the demo:

FROM ubuntu:latest
WORKDIR /root
RUN apt-get update -y
# add personal secrets, assume that we need it because of proxy
COPY secrets.json /root/secrets.json
RUN apt-get install -y htop
# remove personal secrets after install
RUN rm /root/secrets.json

This Dockerfile is the one that exposes secrets via its layer. Using this approach, we are able to restore the credentials. I will show you how:

Build if with:

docker build -t docker-secrets .

You will not find the credentials when you run the image:

docker run -it docker-secrets ls

The output is empty. But they are still present in the layers. Docker layers are compressed tar archives that you can expect.

Dive

For easy access, a tool was created for that purpose: dive. You can download it from github.com: https://github.com/wagoodman/dive

With dive you can search for all changes in each layer, so you know what you are looking for to extracted it case you don't have access to the Dockerfile that was used. Ctrl+F will allow you to search for a particular file in the scope of a layer.

As we just created the image, we search the /root/secrets.json file. But you can't really take a look at the content of the file with dive - but we know now the path.

Get the secrets

Export the just created image as tar archive

docker save docker-secrets -o docker-secrets.tar

Unpack the tar archive using:

tar xf docker-secrets.tar

Use following snippet to search for the file in the extracted docker layers (tar archives):

export FILENAME="secrets.json"
for layer in */layer.tar; do tar -tf $layer | grep $FILENAME && echo $layer; done

Now you have the layer hashes where the file is placed. In my example:

root/.wh.secrets.json
7ffab2fd67ed1e5948017e7856340e6629baf1a486f6cbd713db55f2a66517b5/layer.tar
root/secrets.json
bd6f414da0ffb5996c9ddb9e1e8da783256666d15ec802b1045941103803513a/layer.tar

To get the content:

tar xf bd6f414da0ffb5996c9ddb9e1e8da783256666d15ec802b1045941103803513a/layer.tar root/secrets.json

Now you will find the extracted secrets file under root/secrets.json.

Prevent Secret in layers

Use the relativly new RUN with the secrets option to make sure the secret is not added to the layer. An example clean Dockerfile looks like this:

FROM ubuntu:latest
WORKDIR /root
RUN apt-get update -y
RUN --mount=type=secret,id=secret-key,target=/root/secrets.json
RUN apt-get install -y htop

Note that you need to use buildkit engine and not the classic one. See the following, full example of the usage:

DOCKER_BUILDKIT=1 docker build \
--secret id=secret-key,src=$HOME/secrets.json \
-t docker-secrets-fixed \
-f fixed.Dockerfile .

docker save docker-secrets-fixed -o docker-secrets-fixed.tar
tar xf docker-secrets-fixed.tar
export FILENAME="secrets.json"
for layer in */layer.tar; do tar -tf $layer | grep $FILENAME && echo $layer; done

Your output is now empty.

For a complete list of features (or a good recap when you are familiar with Dockerfiles), I recommend the official Dockerfile Documentation that can be found here: https://docs.docker.com/engine/reference/builder/

Thanks for reading.