How to Enable Docker layer caching in Azure DevOps - azure-devops

Question

I am running the below yaml script to build docker images and push into kubernetes cluster but at the same time I wanted to enable docker layer caching in the azure DevOps while building the yaml script.Could you please explain how to enable or how to add the task in azure devops to do this.
Yaml:
# Starter pipeline
# Start with a minimal pipeline that you can customize to build and deploy your code.
# Add steps that build, run tests, deploy, and more:
# https://aka.ms/yaml
trigger:
- master
pool:
vmImage: 'ubuntu-latest'
variables:
tag: 'web'
DockerImageName: 'boiyaa/google-cloud-sdk-nodejs'
steps:
- task: Docker#2
inputs:
command: 'build'
Dockerfile: '**/Dockerfile'
tags: 'web'
- script: |
echo ${GCLOUD_SERVICE_KEY_STAGING} > ${HOME}/gcp-key.json
gcloud auth activate-service-account --key-file ${HOME}/gcp-key.json --project ${GCLOUD_PROJECT_ID_STAGING}
gcloud container clusters get-credentials ${GCLOUD_PROJECT_CLUSTER_ID_STAGING} \
--zone ${GCLOUD_PROJECT_CLUSTER_ZONE_STAGING} \
--project ${GCLOUD_PROJECT_ID_STAGING}
displayName: 'Setup-staging_credentials'
- bash: bash ./deploy/deploy-all.sh staging
displayName: 'Deploy_script_staging'

Answer 1

Here's how I fixed this. I just pull the latest version of the image from my registry (Azure Container Registry in my case) to the Azure DevOps hosted agent. Then I add --cache-from to the Docker build arguments pointing to this latest tag which it just downloaded to the local machine/cache.
- task: Docker#2
inputs:
containerRegistry: '$(ContainerRegistryName)'
command: 'login'
- script: "docker pull $(ACR_ADDRESS)/$(REPOSITORY):latest"
displayName: Pull latest for layer caching
continueOnError: true # for first build, no cache
- task: Docker#2
displayName: build
inputs:
containerRegistry: '$(ContainerRegistryName)'
repository: '$(REPOSITORY)'
command: 'build'
Dockerfile: './dockerfile '
buildContext: '$(BUILDCONTEXT)'
arguments: '--cache-from=$(ACR_ADDRESS)/$(REPOSITORY):latest'
tags: |
$(Build.BuildNumber)
latest
- task: Docker#2
displayName: "push"
inputs:
command: push
containerRegistry: "$(ContainerRegistryName)"
repository: $(REPOSITORY)
tags: |
$(Build.BuildNumber)
latest

Answer 2

Docker layer caching is not supported in Azure DevOps currently. The reason is stated as below:
In the current design of Microsoft-hosted agents, every job is dispatched to a newly provisioned virtual machine. These virtual machines are cleaned up after the job reaches completion, not persisted and thus not reusable for subsequent jobs. The ephemeral nature of virtual machines prevents the reuse of cached Docker layers.
However:
Docker layer caching is possible using self-hosted agents. You can try creating your on-premise agents to run your build pipeline.
You may need to disable the Job's option 'Allow scripts to access the OAuth token'. For $(System.AccessToken) is passed to docker build using a --build-arg ACCESS_TOKEN=$(System.AccessToken), and its value varies for every run, which will invalidate the cache.
You can also you use Cache task and docker save/load commands to upload the saved Docker layer to Azure DevOps server and restore it on the future run. Check this thread for more information.
Another workaround as described in this blog is to use --cache-from and --target in your Dockerfile.
If the above workarounds are not satisfying, you can submit a feature request to Microsoft Develop Team. Click Suggest a Feature and choose Azure DevOps.

Answer 3

Edit: as pointed out in the comments, this feature is actually available without BuildKit. There's an example here on how to use a Docker image as the cache source during a build.
By adding the variable DOCKER_BUILDKIT: 1 (see this link) to the pipeline job and installing buildx, I managed to achieve layer caching by storing the cache as a separate image. See this link for some basics
Here's an example step in Azure DevOps
- script: |
image="myreg.azurecr.io/myimage"
tag=$(Build.SourceBranchName)-$(Build.SourceVersion)
cache_tag=cache-$(Build.SourceBranchName)
docker buildx create --use
docker buildx build \
-t "${image}:${tag}"
--cache-from=type=registry,ref=${image}:${cache_tag}\
--cache-to=type=registry,ref=${image}:${cache_tag},mode=max \
--push \
--progress=plain \
.
displayName: Build & push image using remote BuildKit layer cache
This of course will require each run to download the image cache, but for images that have long-running installation steps in the Docker build process this is definitely faster (in our case from about 8 minutes to 2).

Answer 4

It looks like Microsoft introduced Pipeline Caching for Azure Devops a while ago and it's possible to cache docker layers. See this link.

Answer 5

You can also set up "local" Docker layer caching in your Pipeline VM if you don't want to push up your cache to a container registry. You'll need the following steps:
- task: Docker#2
displayName: Login to ACR
inputs:
command: login
containerRegistry: $(SERVICE_NAME)
- task: Cache#2
displayName: Cache task
inputs:
key: 'docker | "$(Agent.OS)" | "$(Build.SourceVersion)"'
path: /tmp/.buildx-cache
restoreKeys: 'docker | "$(Agent.OS)"'
- bash: |
docker buildx create --driver docker-container --use
docker buildx build --cache-to type=local,dest=/tmp/.buildx-cache-new --cache-from type=local,src=/tmp/.buildx-cache --push --target cloud --tag $REGISTRY_NAME/$IMAGE_NAME:$TAG_NAME .
displayName: Build Docker image
# optional: set up deploy steps here
- task: Docker#2
displayName: Logout of ACR
inputs:
command: logout
containerRegistry: $(SERVICE_NAME)
The key here is to set up Docker buildx and run it with the --cache-to and --cache-from flags instead of using the Azure Docker task. You'll also need to use the Cache task to make sure the Docker cache is reloaded in subsequent pipeline runs, and you'll have to set up a manual swap step where the newly-generated cache replaces the old cache.

Answer 6

Had the same problem. Turns out it was the task "npm authenticate" that was breaking the caching by inserting a new token each time. I just put a static .npmrc file into the Pipeline > Library > SecureFiles depot and everything became unbelievably fast:
- task: DownloadSecureFile#1
name: 'npmrc'
displayName: 'Download of the npmrc authenticated'
inputs:
secureFile: '.npmrc'
- task: CopyFiles#2
inputs:
SourceFolder: $(Agent.TempDirectory)
contents: ".npmrc"
TargetFolder: $(Build.SourcesDirectory)/Code
OverWrite: true
displayName: 'Import of .npmrc'
- task: Docker#2
displayName: Build an image
inputs:
command: build
dockerfile: '$(Build.SourcesDirectory)/Dockerfile'
tags: |
$(tag)
The only drawback of this is that personnal access token last a year. So you need to replace your securefile every year...