If I run a Docker Compose command in GitHub Actions which uses a bind mount, it says the source directory doesn't exist. Here's the error.
Cannot create container for service chat: invalid mount config for type "bind": bind source path does not exist: /__w/omni-chat/omni-chat
I think the issue is that the root directory is incorrectly being passed to GitHub Actions. I specified the absolute path as the conventional ., but I don't know what caveats GitHub Actions has regarding that.
Here's a simplified version of my workflow.
on: push
jobs:
test-server:
runs-on: ubuntu-latest
container: docker/compose
steps:
- uses: actions/checkout#v2
- run: docker-compose run --rm chat gradle test
Here's a simplified version of my Docker Compose file.
version: '3.7'
services:
chat:
image: gradle:6.3-jdk8
command: bash
volumes:
- type: bind
source: .
target: /home/gradle
- type: volume
source: gradle-cache
target: /home/gradle/.gradle
volumes:
gradle-cache:
If you need the full details, here's the exact run.
It turns out that you should use preinstalled Docker Compose installation. So simply removing the specified container will allow bind mounts to work since it's no longer a Docker-in-Docker scenario.
Related
First of all - no, I cannot switch from Bitbucket pipelines to something appropriate, unfortunately, it is direct requirement.
[x] I have searched other SO questions and google, the following two questions are related:
Bitbucket Pipeline - docker compose error (no answer)
How to use docker compose V2 in Bitbucket Pipelines (answer not working even when literally copied to pipeline definition for one of reasons below)
Working v1 main pipeline (only significant step and job, of course, it is larger)
image: python:3.10
definitions:
steps:
- step: &run-tests
name: Test
image: docker/compose:debian-1.29.2
caches:
- docker
services:
- docker
script:
- COMPOSE_DOCKER_CLI_BUILD=1 DOCKER_BUILDKIT=1 docker-compose --project-name reporting --env-file .env.ci -f docker-compose.ci.yaml up -d --build
# - ... (wait until ready and run tests, ignored, because error happens earlier)
pipelines:
default:
- parallel:
- step: *run-tests
Encountered errors
I'll to refer to them multiple times, so let's define short aliases:
403
+ COMPOSE_DOCKER_CLI_BUILD=1 DOCKER_BUILDKIT=1 docker compose --project-name reporting --env-file .env.ci -f docker-compose.ci.yaml up -d --build
listing workers for Build: failed to list workers: Unavailable: connection error: desc = "transport: Error while dialing unable to upgrade to h2c, received 403"
priviliged
+ COMPOSE_DOCKER_CLI_BUILD=1 DOCKER_BUILDKIT=1 docker compose --project-name reporting --env-file .env.ci -f docker-compose.ci.yaml up -d --build
#1 [internal] booting buildkit
#1 pulling image moby/buildkit:buildx-stable-1
#1 pulling image moby/buildkit:buildx-stable-1 2.8s done
#1 creating container buildx_buildkit_default 0.0s done
#1 ERROR: Error response from daemon: authorization denied by plugin pipelines: --privileged=true is not allowed
------
> [internal] booting buildkit:
------
Error response from daemon: authorization denied by plugin pipelines: --privileged=true is not allowed
Unfortunately, there is no docker/compose v2 image, and our deployment uses v2, so some inconsistencies happen. I'm trying to use v2 in pipeline now. I replaced docker-compose references with docker compose and try to prevent this command from crashing. Important thing to note: I need docker buildkit and cannot go without it, because I'm using Dockerfile.name.dockerignore files which are separate for prod and dev, and docker without buildkit does not support it (builds will simply fail).
Things I tried (debug smts like docker version and docker compose version were always working OK in these cases):
using image: linuxserver/docker-compose:2.10.2-v2. Result: 403.
using image: library/docker:20.10.18.
No more changes. Result: privileged.
Add docker buildx create --driver-opt image=moby/buildkit:v0.10.4-rootless --use as a step. Result: privileged (logs show that this image is actually used: pulling image moby/buildkit:v0.10.4-rootless 6.3s done).
using no explicit image (relying on bitbucket docker installation).
with official compose installation method (result: 403):
- mkdir -p /usr/local/lib/docker/cli-plugins/
- wget -O /usr/local/lib/docker/cli-plugins/docker-compose https://github.com/docker/compose/releases/download/v2.10.2/docker-compose-linux-x86_64
- chmod +x /usr/local/lib/docker/cli-plugins/docker-compose
with solution from 2nd link above (result: 403, but with some portion of success: downloaded two services that do not require building - postgres and redis - and failed only then)
If it is important, compose file for CI (only healthchecks trimmed, everything else not touched):
# We need this file without volumes due to bitbucket limitations.
version: '3.9'
services:
db:
image: mariadb:10.8.3-jammy
env_file: .env.ci
volumes:
- ./tests/db_init/:/docker-entrypoint-initdb.d
networks:
- app_network
redis:
image: redis:alpine
environment:
- REDIS_REPLICATION_MODE=master
networks:
- app_network
app:
build:
context: .
args:
- APP_USER=reporting
- APP_PORT
env_file: .env.ci
depends_on:
- db
- redis
networks:
- app_network
nginx:
build:
context: .
dockerfile: configs/Dockerfile.nginx
env_file: .env.ci
environment:
- APP_HOST=app
ports:
- 80:80
depends_on:
- app
networks:
- app_network
networks:
app_network:
driver: bridge
For now I reverted everything and keep using v1. The limitations of bitbucket pipelines drive me mad, I can easily run the same stuff in github actions, but now have to remove one service (that uses docker directory mounting, so cannot run on bitbucket) and spend whole day trying to upgrade compose. Sorry for this tone, this really makes me desire to quit bitbucket forever and never touch it again.
Since I'm developing on an M1 mac, my Docker builds will build ARM. I want to build x86_64 images which I know I can build with the --platform flag but I want to do that with my Skaffold configuration.
Thanks to #p10l putting me on the right track, I was able to figure it out. Skaffold can't set the platform using the Docker Engine API, so we need to pass --platform=linux/x86_64 to the command line by setting useDockerCLI to true and adding cliFlags to our Docker configuration.
apiVersion: skaffold/v2beta26
kind: Config
build:
# ... other unrelated config ...
artifacts:
- image: my-image
context: ./
docker:
cliFlags:
- --platform=linux/x86_64
local:
useDockerCLI: true # the only way to set platform is with cliFlags so we need to enable the Docker CLI
Whoever is coming here from google and confront this problem, skaffold almost finished developing multi-arch support but as the version of v1.39.1, you can use
skaffold run --platform linux/amd64
or simply to configure it in skaffold.yaml file
Skaffold has buildCommand field to which you can pass a custom script.
So, for example
...
build:
artifacts:
- image: "foo"
context: .
custom:
buildCommand: ./build.sh
...
build.sh
docker buildx build \
--platform linux/amd64
... # any other flags
Disclaimer: Everything below is a speculation. I'm currently unable to test this, but I'm sure someone will correct me if I'm wrong.
There is also build.artifacts.docker field (currently in beta). It may be possible to use this field to pass arguments to Docker build.
...
build:
artifacts:
- image: "foo"
context: .
docker:
dockerfile: <Dockerfile relative to workspace>
target: <Dockerfile target name to build>
buildArgs:
platform: linux/amd64
local:
useDockerCLI: true #this is needed to use docker build CLI rather than Docker Engine API
It may also be required to set buildx as a default builder for Docker. This can be achieved with
docker buildx install
I am trying to substitue variable in app.yaml with a cloud build trigger.
I Added substitution variable in build trigger.
Add environment variables to app.yaml in a way they can be easily substituted with build trigger variables. Like this:
env_variables:
SECRET_KEY: %SECRET_KEY%
Add a step in cloudbuild.yaml to substitute all %XXX% variables inside app.yaml with their values from build trigger.
steps:
- name: node:10.15.1
entrypoint: npm
args: ["install"]
- name: 'gcr.io/cloud-builders/gcloud'
entrypoint: bash
args:
- '-c'
- |
sed -i 's/%SESSION_SECRET%/'${_SESSION_SECRET}'/g' app.yaml
timeout: "1600s"
The problem is that Gcloud Build throw an exception :
Already have image (with digest): gcr.io/cloud-builders/gcloud
bash: _L/g: No such file or directory
Why ? How can I make a substitution of my app.yaml ?
I have a app.yaml to the root of the project at the same level of the cloudbuild.yaml
UPDATED
I am trying to build and debug gcloud locally with this command:
sudo cloud-build-local --config=cloudbuild.yaml --write-workspace=../workspace --dryrun=false --substitutions=_SESSION_SECRET=test --push .
When I take a look into the app.yaml file, the substitution worked as expected and there is no exception at all.
What is the difference with the gcloud build environment ?
OK I finally decided to use github action instead of google cloud triggers.
Since Google cloud triggers aren't able to find its own app.yaml and manage the freaking environment variable by itself.
Here is how to do it:
My environment :
App engine,
standard (not flex),
Nodejs Express application,
a PostgreSQL CloudSql
First the setup :
1. Create a new Google Cloud Project (or select an existing project).
2. Initialize your App Engine app with your project.
[Create a Google Cloud service account][sa] or select an existing one.
3. Add the the following Cloud IAM roles to your service account:
App Engine Admin - allows for the creation of new App Engine apps
Service Account User - required to deploy to App Engine as service account
Storage Admin - allows upload of source code
Cloud Build Editor - allows building of source code
[Download a JSON service account key][create-key] for the service account.
4. Add the following [secrets to your repository's secrets][gh-secret]:
GCP_PROJECT: Google Cloud project ID
GCP_SA_KEY: the downloaded service account key
The app.yaml
runtime: nodejs14
env: standard
env_variables:
SESSION_SECRET: $SESSION_SECRET
beta_settings:
cloud_sql_instances: SQL_INSTANCE
Then the github action
name: Build and Deploy to GKE
on: push
env:
PROJECT_ID: ${{ secrets.GKE_PROJECT }}
DATABASE_URL: ${{ secrets.DATABASE_URL}}
jobs:
setup-build-publish-deploy:
name: Setup, Build, Publish, and Deploy
runs-on: ubuntu-latest
steps:
- uses: actions/checkout#v2
- uses: actions/setup-node#v2
with:
node-version: '12'
- run: npm install
- uses: actions/checkout#v1
- uses: ikuanyshbekov/app-yaml-env-compiler#v1.0
env:
SESSION_SECRET: ${{ secrets.SESSION_SECRET }}
- shell: bash
run: |
sed -i 's/SQL_INSTANCE/'${{secrets.DATABASE_URL}}'/g' app.yaml
- uses: actions-hub/gcloud#master
env:
PROJECT_ID: ${{ secrets.GKE_PROJECT }}
APPLICATION_CREDENTIALS: ${{ secrets.GCLOUD_AUTH }}
CLOUDSDK_CORE_DISABLE_PROMPTS: 1
with:
args: app deploy app.yaml
To add secrets into git hub action you must go to : Settings/secrets
Take note that I could handle all the substitution with the bash script. So I would not depend on the github project "ikuanyshbekov/app-yaml-env-compiler#v1.0"
It's a shame that GAE doesn't offer an easiest way to handle environment variable for the app.yaml. I don't want to use KMS since I need to update the beta-settings/cloud sql instance.. I really needed to substitute everything into the app.yaml.
This way I can make a specific action for the right environment and manage the secrets.
The entrypoint should be an executable, use /bin/bash or /bin/sh.
How to inspect inside the image (in general):
$ docker pull gcr.io/cloud-builders/gcloud
Using default tag: latest
latest: Pulling from cloud-builders/gcloud
...
$ docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
gcr.io/cloud-builders/gcloud latest 8499764c4ef6 About an hour ago 4.01GB
$ docker run -ti --entrypoint '/bin/bash' 8499764c4ef6
root#60354dfb588a:/#
You can test your commands from there to test without having to sending it to Cloud Build each time.
I want to run a skopeo container as a container job.
I keep getting this error message during the Initialize containers step
Error response from daemon: Container 7e741e4aafb30bb89e1dfb830c1cb69fa8d47d219f28cc7b8e57727253632256 is not running
my pipeline looks like this:
- job: publish_branch_image
pool:
vmImage: ubuntu-latest
container: docker.io/ananace/skopeo:latest
steps:
- script: |
# clean branchname for imagename
export COMMIT_IMAGE="$(Image.TagName)"
export TARGET_IMAGE="$(Image.Name)":$(echo $(Build.SourceBranch) | sed 's./.-.g')
echo "Pushing to ${TARGET_IMAGE}"
skopeo copy docker://${COMMIT_IMAGE} docker://${TARGET_IMAGE} --src-creds="$(Registry.USER):$(Registry.PASSWORD)" --dest-creds="$(Registry.USER):$(Registry.PASSWORD)"
displayName: publish-branch-release-image
According to the error message, it seems that the container is not running, we could run the cmd docker pull docker.io/ananace/skopeo:latest to pull the image and run it via docker run docker.io/ananace/skopeo:latest, then we could use it.
Update1
Thanks for michiel sharing, according to the doc Endpoints and Linux-based containers:
Containers can be hosted on registries other than Docker Hub. To host an image on Azure Container Registry or another private container registry, add a service connection to the private registry. Then you can reference it in a container spec:
container:
image: xxx/xxx:tag
endpoint: xxx
Is there any way to cache docker-compose so that it will not build again and again?
here is my action workflow file:
name: Github Action
on:
push:
branches:
- staging
jobs:
test:
runs-on: ubuntu-18.04
steps:
- uses: actions/checkout#v1
- name: Bootstrap app on Ubuntu
uses: actions/setup-node#v1
with:
node-version: '12'
- name: Install global packages
run: npm install -g yarn prisma
- name: Install project deps
if: steps.cache-yarn.outputs.cache-hit != 'true'
run: yarn
- name: Build docker-compose
run: docker-compose -f docker-compose.test.prisma.yml up --build -d
I want to cache the docker build step. I have tried using if: steps.cache-docker.outputs.cache-hit != 'true' then only build but didn't work.
What you are referring to is called "docker layer caching", and it is not yet natively supported in GitHub Actions.
This is discussed extensively in several places, like:
Cache docker image forum thread
Cache a Docker image built in workflow forum thread
Docker caching issue in actions/cache repository
As mentioned in the comments, there are some 3rd party actions that provide this functionality (like this one), but for such a core and fundamental feature, I would be cautious with anything that is not officially supported by GitHub itself.
For those arriving here via Google, this now "supported". Or at least it is working: https://github.community/t/use-docker-layer-caching-with-docker-compose-build-not-just-docker/156049.
The idea is to build the images using docker (and its cache) and then use docker compose to run (up) them.
If using docker/bake-action or docker/build-push-action & want to access a cached image in subsequent steps -
Use load:true to save the image
Use the same image name as the cached image across steps in order to skip rebuilds.
Example:
...
name: Build and push
uses: docker/bake-action#master
with:
push: false
load: true
set: |
web.cache-from=type=gha
web.cache-to=type=gha
-
name: Test via compose
command: docker compose run web tests
...
services:
web:
build:
context: .
image: username/imagename
command: echo "Test run successful!"
See the docker team's responses;
How to access the bake-action cached image in subsequent steps?
How to use this plugin for a docker-compose?
How to share layers with Docker Compose?`
Experiment on caching docker compose images in GitHub Actions