Advanced multi-stage build templates

Original author: Tõnis Tiigi
  • Transfer

image


The multistage assembly function in Dockerfile files allows you to create small images of containers with a higher level of caching and a smaller amount of protection. In this article, I’ll show several advanced templates — something more than copying files between build and execute steps. They allow you to maximize the effectiveness of the function. However, if you are a beginner in the field of multi-stage assembly, then first, probably, it would not be superfluous to read the usage guide .


Version Compatibility


Support for multi-stage build was added to Docker in version v17.05. All templates work with any subsequent version, but some are much more efficient, thanks to the build using server-side BuildKit . For example, BuildKit effectively skips unused stages and, if possible, creates stages at the same time (I singled out these examples separately). Currently, BuildKit is being added to Moby as an experimental server part of the build and should be available in Docker CE v18.06. It can also be used autonomously or as part of the img project .




Inheritance from the stage


Multi-stage build adds several new syntax concepts. First of all, you can assign a FROMname to the step starting with the command AS stagenameand use the option --from=stagenamein the command COPYto copy the files from this step. In fact, the team FROMand the label --fromhave much more in common, no wonder they have the same name. Both take the same argument, recognize it and either start a new phase from this point, or use it as a source to copy the file. That is, to use the previous stage in the original image quality for the current stage, you can take not only the --from=stagenamestage nameFROM stagename. It is useful if you use the same common parts in several commands in the Dockerfile: reduces the common code and simplifies its maintenance, keeping the child stages separate. Thus, rebuilding one stage does not affect the build cache for others. Accordingly, each stage can be collected individually using the label --targetwhen calling docker build.


FROM ubuntu AS base
RUN apt-get update && apt-get install git
FROM base AS src1
RUN git clone …
FROM base as src2
RUN git clone …

In this example, the second and third stages in BuildKit are built simultaneously.


Direct use of images


Instead of using assembly step names in commands FROMthat previously only supported image references, you can directly use images — using a label --from. It turns out to copy files directly from these images. For example, linuxkit/ca-certificatesimagethe following code directly copies the TLS CA roots into the current stage.


FROM alpine
COPY --from=linuxkit/ca-certificates / /

Common nickname


The build step does not necessarily include any commands; it may consist of a single line FROM. If an image is used in several places, it will make reading easier and make it so that if you need to update a shared image, you only need to change one line.


FROM alpine:3.6 AS alpine
FROM alpine
RUN …
FROM alpine
RUN …

In this example, every place that uses the alpine image is actually fixed on alpine:3.6, not alpine:latest. When the time comes to upgrade to alpine:3.7, you will need to change a single line, and there is no doubt: now the updated version is used in all elements of the assembly.


This is all the more important when the assembly argument is used in the alias. The following example is similar to the previous one, but allows the user to redefine all instances of the assembly in which the alpine image is involved, by setting the option --build-arg ALPINE_VERSION=value. Remember: any arguments used in commands FROMneed to be defined before the first build stage .


ARG ALPINE_VERSION=3.6
FROM alpine:${ALPINE_VERSION} AS alpine
FROM alpine
RUN …

Using assembly arguments in "- from"


The value specified in the --fromcommand label COPYmust not contain assembly arguments. For example, the following example is invalid.


// THIS EXAMPLE IS INTENTIONALLY INVALID
FROM alpine AS build-stage0
RUN …
FROM alpine
ARG src=stage0
COPY --from=build-${src} . .

This is due to the fact that the dependencies between the stages need to be determined even before the assembly begins. Then the constant evaluation of all teams is not required. For example, an environment variable defined in an image alpinecan affect the value estimate --from. The reason we can evaluate team arguments FROMis because these arguments are defined globally before the start of any stage. Fortunately, as we found out earlier, it is enough to define the stage of the pseudonym with the help of one command FROMand refer to it.


ARG src=stage0
FROM alpine AS build-stage0
RUN …
FROM build-${src} AS copy-src
FROM alpine
COPY --from=copy-src . .

Now, if you override the assembly argument src, the initial stage for the final element will COPYswitch. Please note: if some stages are no longer used, then only BuildKit-based linkers will be able to skip them.


Conditions using assembly arguments


We were asked to add support for the style conditions in the Dockerfile IF/ELSE. We still do not know whether we will add something similar, but in the future we will try - using the support of the client part in BuildKit. Meanwhile, to achieve a similar behavior, you can use the current multi-stage concept (with some planning).


// THIS EXAMPLE IS INTENTIONALLY INVALID
FROM alpine
RUN …
ARG BUILD_VERSION=1
IF $BUILD_VERSION==1
RUN touch version1
ELSE IF $BUILD_VERSION==2
RUN touch version2
DONE
RUN …

The previous example shows pseudo-code for recording conditions with IF/ELSE. To achieve similar behavior with the current multi-stage builds, you may need to define different branch conditions as separate steps and use an argument to choose the right dependency path.


ARG BUILD_VERSION=1
FROM alpine AS base
RUN …
FROM base AS branch-version-1
RUN touch version1
FROM base AS branch-version-2
RUN touch version2
FROM branch-version-${BUILD_VERSION} AS after-condition
FROM after-condition 
RUN …

The last step in the Dockerfile is based on a step after-conditionthat is an alias of the image (recognized based on the assembly argument BUILD_VERSION). Depending on the value BUILD_VERSION, one or another stage of the middle section is selected.


Please note: only BuildKit-based linkers can skip unused branches. In previous versions of linkers, all stages would be built, but before creating the final image, their results would be discarded.


Development / Testing Assistant for Minimum Production Stage


Finally, let's take a look at an example of combining previous templates to demonstrate how to create a Dockerfile that creates a minimal production image and can then use its contents to test and create a development image. Let's start with the basic Dockerfile example:


FROM golang:alpine AS stage0
…
FROM golang:alpine AS stage1
…
FROM scratch
COPY --from=stage0 /binary0 /bin
COPY --from=stage1 /binary1 /bin

When a minimal production image is created, this is a fairly common option. But what if you also need to get an alternative developer image or run tests with these binaries at the final stage? The first thing that comes to mind is simply to copy similar binaries during the testing and development phases. The problem is this: there is no guarantee that you will test all production binaries in the same combination. At the final stage, something may change, and you will forget to make similar changes at other stages or make an error in the path to copy binary files. In the end, we are not testing a separate binary file, but a final image.


An alternative is to define the development and testing phase after the production phase and copy the entire contents of the production phase. Then use one command for the production phase FROMto again make the default production phase the last step.


FROM golang:alpine AS stage0
…
FROM scratch AS release
COPY --from=stage0 /binary0 /bin
COPY --from=stage1 /binary1 /bin
FROM golang:alpine AS dev-env
COPY --from=release / /
ENTRYPOINT ["ash"]
FROM golang:alpine AS test
COPY --from=release / /
RUN go test …
FROM release

By default, this Dockerfile will continue to build the minimum default image, while, for example, an assembly with the option --target=dev-envwill create an image with a shell containing all the binaries of the final version.




I hope this was helpful and suggested how to create more efficient multistage Dockerfile files. If you participate in DockerCon2018 and want to learn more about multi-stage builds, Dockerfiles, BuildKit, or related topics, subscribe to the Hallway track linker or track Docker’s internal meetings on the Contribute and Collaborate tracks or Black Belt .


Also popular now: