Speed up CI/CD pipelines with the brand new AWS CodeBuild Docker Server functionality

Beginning immediately, you should utilize AWS CodeBuild Docker Server functionality to provision a devoted and protracted Docker server instantly inside your CodeBuild challenge. With Docker Server functionality, you’ll be able to speed up your Docker picture builds by centralizing picture constructing to a distant host, which reduces wait instances and will increase total effectivity.

From my benchmark, with this Docker Server functionality, I decreased the full constructing time by 98 %, from 24 minutes and 54 seconds to 16 seconds. Right here’s a fast have a look at this characteristic from my AWS CodeBuild initiatives.

AWS CodeBuild is a completely managed steady integration service that compiles supply code, runs assessments, and produces software program packages prepared for deployment. Constructing Docker photographs is among the commonest use instances for CodeBuild prospects, and the service has progressively improved this expertise over time by releasing options similar to Docker layer caching and reserved capability options to enhance Docker construct efficiency.

With the brand new Docker Server functionality, you’ll be able to scale back construct time to your purposes by offering a persistent Docker server with constant caching. When enabled in a CodeBuild challenge, a devoted Docker server is provisioned with persistent storage that maintains your Docker layer cache. This server can deal with a number of concurrent Docker construct operations, with all builds benefiting from the identical centralized cache.

Utilizing AWS CodeBuild Docker Server
Let me stroll you thru an illustration that showcases the advantages with the brand new Docker Server functionality.

For this demonstration, I’m constructing a fancy, multi-layered Docker picture primarily based on the official AWS CodeBuild curated Docker photographs repository, particularly the Dockerfile for constructing a customary Ubuntu picture. This picture accommodates quite a few dependencies and instruments required for contemporary steady integration and steady supply (CI/CD) pipelines, making it a great instance of the kind of giant Docker builds that improvement groups commonly carry out.

# Copyright 2020-2024 Amazon.com, Inc. or its associates. All Rights Reserved.
#
# Licensed below the Amazon Software program License (the "License"). You could not use this file besides in compliance with the License.
# A duplicate of the License is positioned at
#
#    http://aws.amazon.com/asl/
#
# or within the "license" file accompanying this file.
# This file is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, categorical or implied.
# See the License for the precise language governing permissions and limitations below the License.
FROM public.ecr.aws/ubuntu/ubuntu:20.04 AS core

ARG DEBIAN_FRONTEND="noninteractive"

# Set up git, SSH, Git, Firefox, GeckoDriver, Chrome, ChromeDriver,  stunnel, AWS Instruments, configure SSM, AWS CLI v2, env instruments for runtimes: Dotnet, NodeJS, Ruby, Python, PHP, Java, Go, .NET, Powershell Core,  Docker, Composer, and different utilities
COMMAND REDACTED FOR BREVITY
# Activate runtime variations particular to picture model.
RUN n $NODE_14_VERSION
RUN pyenv  world $PYTHON_39_VERSION
RUN phpenv world $PHP_80_VERSION
RUN rbenv  world $RUBY_27_VERSION
RUN goenv world  $GOLANG_15_VERSION

# Configure SSH
COPY ssh_config /root/.ssh/config
COPY runtimes.yml /codebuild/picture/config/runtimes.yml
COPY dockerd-entrypoint.sh /usr/native/bin/dockerd-entrypoint.sh
COPY authorized/bill_of_material.txt /usr/share/doc/bill_of_material.txt
COPY amazon-ssm-agent.json /and so forth/amazon/ssm/amazon-ssm-agent.json

ENTRYPOINT ["/usr/local/bin/dockerd-entrypoint.sh"]

This Dockerfile creates a complete construct atmosphere with a number of programming languages, construct instruments, and dependencies – precisely the kind of picture that will profit from persistent caching.

Within the construct specification (buildspec), I take advantage of the docker buildx construct . command:

model: 0.2
phases:
  construct:
    instructions:
      - cd ubuntu/customary/5.0
      - docker buildx construct -t codebuild-ubuntu:newest .

To allow the Docker Server functionality, I navigate to the AWS CodeBuild console and choose Create challenge. I may allow this functionality when modifying present CodeBuild initiatives.

I fill in all particulars and configuration. Within the Surroundings part, I choose Further configuration.

Then, I scroll down and discover Docker server configuration and choose Allow docker server for this challenge. After I choose this selection, I can select a compute sort configuration for the Docker server. After I’m completed with the configurations, I create this challenge.

Now, let’s see the Docker Server functionality in motion.

The preliminary construct takes roughly 24 minutes and 54 seconds to finish as a result of it must obtain and compile all dependencies from scratch. That is anticipated for the primary construct of such a fancy picture.

For subsequent builds with no code adjustments, the construct takes solely 16 seconds and that reveals 98% discount in construct time.

Wanting on the logs, I can see that with Docker Server, most layers are pulled from the persistent cache:

The persistent caching supplied by the Docker Server maintains all layers between builds, which is especially helpful for giant, advanced Docker photographs with many layers. This demonstrates how Docker Server can dramatically enhance throughput for groups working quite a few Docker builds of their CI/CD pipelines.

Further issues to know
Listed here are a few issues to notice:

• Structure help – The characteristic is accessible for each x86 (Linux) and ARM builds.
• Pricing – To be taught extra about pricing for Docker Server functionality, check with the AWS CodeBuild pricing web page.
• Availability – This characteristic is accessible in all AWS Areas the place AWS CodeBuild is obtainable. For extra details about the AWS Areas the place CodeBuild is accessible, see the AWS Areas web page.

You’ll be able to be taught extra in regards to the Docker Server characteristic within the AWS CodeBuild documentation.

Completely happy constructing! —

Donnie Prakoso

How is the Information Weblog doing? Take this 1 minute survey!

(This survey is hosted by an exterior firm. AWS handles your info as described within the AWS Privateness Discover. AWS will personal the information gathered through this survey and won’t share the data collected with survey respondents.)