Github Actions newbie here :).
The workflow file:
name: ga-poc-sb CI
on: [push]
jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout#v3
- name: Set up JDK 17
uses: actions/setup-java#v3
with:
java-version: '17'
distribution: 'temurin'
- name: Validate Gradle wrapper
uses: gradle/wrapper-validation-action#e6e38bacfdf1a337459f332974bb2327a31aaf4b
- name: Build
uses: gradle/gradle-build-action#67421db6bd0bf253fb4bd25b31ebb98943c375e1
with:
arguments: build
- name: Test
uses: gradle/gradle-build-action#67421db6bd0bf253fb4bd25b31ebb98943c375e1
with:
arguments: test
Till yesterday, I was able to see the workflow logs but now cannot, perhaps due to an existing issue in the Github UI. Still, I recall some logs from yesterday, and they match with my local (act) logging as follows:
[ga-poc-sb CI/build] ✅ Success - Main Build
[ga-poc-sb CI/build] ⭐ Run Main Test
[ga-poc-sb CI/build] 🐳 docker cp src=/Users/oj/.cache/act/gradle-gradle-build-action#67421db6bd0bf253fb4bd25b31ebb98943c375e1/ dst=/var/run/act/actions/gradle-gradle-build-action#67421db6bd0bf253fb4bd25b31ebb98943c375e1/
[ga-poc-sb CI/build] 🐳 docker exec cmd=[node /var/run/act/actions/gradle-gradle-build-action#67421db6bd0bf253fb4bd25b31ebb98943c375e1/dist/main/index.js] user= workdir=
[ga-poc-sb CI/build] 💬 ::debug::Determined user.home from java -version output: '/root'
| Gradle setup only performed on first gradle-build-action step in workflow.
| [command]/Volumes/Development/work-directories/repositories/dev/ga-poc-sb/gradlew test
| > Task :compileJava UP-TO-DATE
| > Task :processResources UP-TO-DATE
| > Task :classes UP-TO-DATE
| > Task :compileTestJava UP-TO-DATE
| > Task :processTestResources NO-SOURCE
| > Task :testClasses UP-TO-DATE
| > Task :test UP-TO-DATE
|
| BUILD SUCCESSFUL in 1s
| 4 actionable tasks: 4 up-to-date
[ga-poc-sb CI/build] ✅ Success - Main Test
[ga-poc-sb CI/build] ⭐ Run Post Test
[ga-poc-sb CI/build] 🐳 docker exec cmd=[node /var/run/act/actions/gradle-gradle-build-action#67421db6bd0bf253fb4bd25b31ebb98943c375e1/dist/post/index.js] user= workdir=
| Gradle setup post-action only performed for first gradle-build-action step in workflow.
[ga-poc-sb CI/build] ✅ Success - Post Test
[ga-poc-sb CI/build] ⭐ Run Post Build
[ga-poc-sb CI/build] 🐳 docker exec cmd=[node /var/run/act/actions/gradle-gradle-build-action#67421db6bd0bf253fb4bd25b31ebb98943c375e1/dist/post/index.js] user= workdir=
| Stopping all Gradle daemons
| Stopping Gradle daemons in /root/.gradle/wrapper/dists/gradle-7.6-bin/9l9tetv7ltxvx3i8an4pb86ye/gradle-7.6
| [command]/root/.gradle/wrapper/dists/gradle-7.6-bin/9l9tetv7ltxvx3i8an4pb86ye/gradle-7.6/bin/gradle --stop
| Stopping Daemon(s)
| 1 Daemon stopped
| In final post-action step, saving state and writing summary
| Cache is read-only: will not save state for use in subsequent builds.
[ga-poc-sb CI/build] ✅ Success - Post Build
[ga-poc-sb CI/build] ⭐ Run Post Set up JDK 17
[ga-poc-sb CI/build] 🐳 docker exec cmd=[node /var/run/act/actions/actions-setup-java#v3/dist/cleanup/index.js] user= workdir=
[ga-poc-sb CI/build] ✅ Success - Post Set up JDK 17
[ga-poc-sb CI/build] 🏁 Job succeeded
This caught my eye:
[ga-poc-sb CI/build] 🐳 docker exec cmd=[node /var/run/act/actions/gradle-gradle-build-action#67421db6bd0bf253fb4bd25b31ebb98943c375e1/dist/post/index.js] user= workdir=
| Gradle setup post-action only performed for first gradle-build-action step in workflow.
Question: Why aren't the unit tests triggered(I see the results when I do a normal Gradle test in my IntelliJ setup).
Do I need to add another job named 'Test'(which again installs Java, and clones repo) to trigger the unit tests?
Related
I'm building an app that needs MariaDB 10.3 for testing in a CI pipeline. I'm trying to set this up with Github actions but I'm having trouble getting MariaDB to start. The ubuntu runner includes MySQL 8.0 by default so my workflow removes that first and then installs MariaDB, but MariaDB fails to start. At first I saw this error:
2022-09-17T01:24:30.8867144Z + sudo cat /var/log/mysql/error.log
2022-09-17T01:24:30.8926847Z 2022-09-17 1:24:20 0 [ERROR] InnoDB: Invalid flags 0x4800 in ./ibdata1
2022-09-17T01:24:30.8927599Z 2022-09-17 1:24:20 0 [ERROR] InnoDB: Plugin initialization aborted with error Data structure corruption
2022-09-17T01:24:30.8928456Z 2022-09-17 1:24:21 0 [ERROR] Plugin 'InnoDB' init function returned error.
2022-09-17T01:24:30.8929215Z 2022-09-17 1:24:21 0 [ERROR] Plugin 'InnoDB' registration as a STORAGE ENGINE failed.
2022-09-17T01:24:30.8930300Z 2022-09-17 1:24:21 0 [ERROR] Unknown/unsupported storage engine: InnoDB
2022-09-17T01:24:30.8930738Z 2022-09-17 1:24:21 0 [ERROR] Aborting
I think this is because MySQL 8.0 leaves behind some old files, so I added a step to remove /var/lib/mysql but now the action stalls after the MariaDB installation.
I made a copy in a new public repo to show the issue here: https://github.com/llamafilm/mariadb_test/actions/runs/3071684353/jobs/4962586417
The workflow is like this:
jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout#v3
- name: Install dependencies
run: |
set -x
sudo apt update
sudo apt autoremove mysql*
- name: Look around
run: |
set -x
sudo ls -l /var/lib/mysql
sudo rm -rf /var/lib/mysql
- name: Install MariaDB
run: |
sudo apt install -y mariadb-server-10.3
- name: Validate DB
run: |
set -x
sudo cat /var/log/mysql/error.log
sudo ls -l /var/lib/mysql
mysql -e "SHOW STATUS"
I am trying to configure my etc/pip.conf file to download a private PyPi artifactory while using a secret variable on my dockerfile.
Dockerfile
FROM python
WORKDIR ./app
COPY . /app
RUN pip install --upgrade pip
RUN pip install -r pre-requirements.txt
RUN echo ${{ secrets.PIP }} > etc/pip.conf
RUN pip install -r post-requirements.txt
CMD ["python", "./simpleflask.py"]
docker-image.yml
name: CI
on:
push:
branches: [ "main" ]
pull_request:
branches: [ "main" ]
workflow_dispatch:
jobs:
build:
runs-on: ubuntu-latest
steps:
- name: Setup JFrog CLI
uses: jfrog/setup-jfrog-cli#v2
env:
JF_ARTIFACTORY_SERVER: ${{ secrets.JFROG_CLI }}
- name: Checkout
uses: actions/checkout#v3
- name: Build
run: |
docker build -t simple-flask .
docker tag simple-flask awakzdev.jfrog.io/docker-local/simple-flask:latest
docker push awakzdev.jfrog.io/docker-local/simple-flask:latest
pretty simple and straightfoward but my pipeline returns the following
Step 6/8 : RUN echo ${{ secrets.PIP }} > etc/pip.conf
---> Running in deb3e3f4167f
/bin/sh: 1: Bad substitution
The command '/bin/sh -c echo ${{ secrets.PIP }} > etc/pip.conf' returned a non-zero code: 2
Error: Process completed with exit code 2.
Edit :
Trying a slightly difference approach and went to install dependencies in the pipeline
my .yml looks like this now
jobs:
build:
runs-on: ubuntu-latest
steps:
- name: Setup JFrog CLI
uses: jfrog/setup-jfrog-cli#v2
env:
JF_ARTIFACTORY_SERVER: ${{ secrets.JFROG_CLI }}
- name: Checkout
uses: actions/checkout#v3
- name: install dependencies
run: |
pip config -v list
echo "${{ secrets.PIP }}" > /etc/pip.conf
pip install ganesha-experimental==2.0.1
- name: Build
run: |
docker build -t simple-flask .
docker tag simple-flask awakzdev.jfrog.io/docker-local/simple-flask:latest
docker push awakzdev.jfrog.io/docker-local/simple-flask:latest
but the following error is being returned:
1s
Run pip config -v list
For variant 'global', will try loading '/etc/xdg/pip/pip.conf'
For variant 'global', will try loading '/etc/pip.conf'
For variant 'user', will try loading '/home/runner/.pip/pip.conf'
For variant 'user', will try loading '/home/runner/.config/pip/pip.conf'
For variant 'site', will try loading '/usr/pip.conf'
/home/runner/work/_temp/09382b8f-ce09-4646-816f-fb337f40ad4b.sh: line 2: /etc/pip.conf: Permission denied
Error: Process completed with exit code 1.
I've placed the secret on my .yml file instead.
as for the broken pip permissions I used
sudo chown runner /etc/
echo ${{ secrets.PIP }} > /etc/pip.conf
which resulted in another error with the contents of the pip.conf file (it was configured correctly through secrets)
so I found you can specify the url like so
ganesha_experimental==5.0.0 --find-links=https://awakzdev.jfrog.io/artifactory/
I want to build docker images and push into ECR for that I have written below buildspect.yml file and build my project using AWS build project service.
My buildspec.yml file is as below:
version: 0.2
phases:
install:
runtime-versions:
docker: 18
commands:
- nohup /usr/local/bin/dockerd --host=unix:///var/run/docker.sock --host=tcp://localhost:2375 --storage-driver=overlay2&
- timeout 15 sh -c "until docker info; do echo .; sleep 1; done"
pre_build:
commands:
- $(aws ecr get-login --no-include-email --region ${AWS_DEFAULT_REGION})
- REPOSITORY_URI_SERVER=<accountnumber>.dkr.ecr.${AWS_DEFAULT_REGION}.amazonaws.com/${IMAGE_NAME}
build:
commands:
- docker-compose build
post_build:
commands:
- docker-compose push
While I do build using AWS build project service then I will get an error like:
ERROR: Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?
errors pretty printing info
[Container] 2020/07/10 01:57:15 Command did not exit successfully timeout 15 sh -c "until docker info; do echo .; sleep 1; done" exit status 124
[Container] 2020/07/10 01:57:15 Phase complete: INSTALL State: FAILED
[Container] 2020/07/10 01:57:15 Phase context status code: COMMAND_EXECUTION_ERROR Message: Error while executing command: timeout 15 sh -c "until docker info; do echo .; sleep 1; done". Reason: exit status 124
I have tried to write buildspect.yml file as below
I have added an artifact tag in my file that is the place where my project build is a store.
post_build:
commands:
- docker-compose push
- printf '[{"name":"<name of container>","imageUri":"<path of your image"}]' > imagedefinitions.json
- cat imagedefinitions.json
artifacts:
files: imagedefinitions.json
I got success in it.
Does anyone have a working azure-pipelines.yaml file that utilizes containers only to build Dockerfiles which I have in my repo.
resources:
containers:
- container: docker-image
image: docker:18.09.6
jobs:
- job: Build
container: docker-image
steps:
- script: |
echo hello from Linux
Docker --version
The above ends up with the error:
##[section]Starting: Initialize containers
##[command]/usr/bin/docker version --format '{{.Server.APIVersion}}'
'1.40'
Docker daemon API version: '1.40'
##[command]/usr/bin/docker version --format '{{.Client.APIVersion}}'
'1.40'
Docker client API version: '1.40'
##[command]/usr/bin/docker ps --all --quiet --no-trunc --filter "label=f38b39"
##[command]/usr/bin/docker network prune --force --filter "label=f38b39"
##[command]/usr/bin/docker network create --label f38b39 vsts_network_b78f6272ef304388ac849f3f37071eea
fda578a22eeb5aae1fcd41a4b816a279d20cfa552dbe3ad366fa2a009ab35313
##[command]/usr/bin/docker pull docker:18.09.6
18.09.6: Pulling from library/docker
e7c96db7181b: Already exists
5297bd381816: Pulling fs layer
3a664477889c: Pulling fs layer
a9b893dcc701: Pulling fs layer
48bf7c1cb0dd: Pulling fs layer
555b6ea27ad2: Pulling fs layer
48bf7c1cb0dd: Waiting
555b6ea27ad2: Waiting
3a664477889c: Verifying Checksum
3a664477889c: Download complete
5297bd381816: Download complete
48bf7c1cb0dd: Verifying Checksum
48bf7c1cb0dd: Download complete
5297bd381816: Pull complete
555b6ea27ad2: Verifying Checksum
555b6ea27ad2: Download complete
a9b893dcc701: Verifying Checksum
a9b893dcc701: Download complete
3a664477889c: Pull complete
a9b893dcc701: Pull complete
48bf7c1cb0dd: Pull complete
555b6ea27ad2: Pull complete
Digest: sha256:bf929409251faa1d1fcf63af68d02bb942054cfbe14e6ad9dca0dc4a01cbffad
Status: Downloaded newer image for docker:18.09.6
docker.io/library/docker:18.09.6
##[command]/usr/bin/docker inspect --format="{{index .Config.Labels \"com.azure.dev.pipelines.agent.handler.node.path\"}}" docker:18.09.6
##[command]/usr/bin/docker create --name docker-image_docker18096_794554 --label f38b39 --network vsts_network_b78f6272ef304388ac849f3f37071eea -v "/var/run/docker.sock":"/var/run/docker.sock" -v "/home/vsts/work/1":"/__w/1" -v "/home/vsts/work/_temp":"/__w/_temp" -v "/opt/hostedtoolcache":"/__t" -v "/home/vsts/work/_tasks":"/__w/_tasks" -v "/home/vsts/agents/2.150.3/externals":"/__a/externals":ro -v "/home/vsts/work/.taskkey":"/__w/.taskkey" docker:18.09.6 "/__a/externals/node/bin/node" -e "setInterval(function(){}, 24 * 60 * 60 * 1000);"
b8856a42131206302453903ae8fbc85957368b2e377ee5a9d1b44627130006b0
##[command]/usr/bin/docker start b8856a42131206302453903ae8fbc85957368b2e377ee5a9d1b44627130006b0
b8856a42131206302453903ae8fbc85957368b2e377ee5a9d1b44627130006b0
##[command]/usr/bin/docker ps --all --filter id=b8856a42131206302453903ae8fbc85957368b2e377ee5a9d1b44627130006b0 --filter status=running --no-trunc --format "{{.ID}} {{.Status}}"
b8856a42131206302453903ae8fbc85957368b2e377ee5a9d1b44627130006b0 Up Less than a second
##[command]/usr/bin/docker exec b8856a42131206302453903ae8fbc85957368b2e377ee5a9d1b44627130006b0 sh -c "command -v bash"
OCI runtime exec failed: exec failed: cannot exec a container that has stopped: unknown
##[error]Docker exec fail with exit code 126
##[section]Finishing: Initialize containers
that's because that container doesnt run endlessly if you just start it, it exits, change docker:18.09.6 to ubuntu or something that doesnt exit immediately and it will work.
So i recently tried Docker and Gitlab Runner but it seems i cant get it to work.
This is the log i have:
Running with gitlab-runner 10.0.2 (a9a76a50)
on my-docker (c588e5e2)
Using Docker executor with image docker:git ...
Starting service docker:dind ...
Pulling docker image docker:dind ...
Using docker image docker:dind ID=sha256:b9145b364a203c0afc538ca615b3470e41729edfb7338017f5d4eeb5b13b2d90 for docker service...
Waiting for services to be up and running...
Using docker image sha256:7961fbf38d6f827265aed22fe41a1db889c54913283b678a8623efdda9573977 for predefined container...
Pulling docker image docker:git ...
Using docker image docker:git ID=sha256:5917639be9495ab183f357e8bafafea82449f0c4b12b745eef8bd23d474220ca for build container...
Running on runner-c588e5e2-project-1-concurrent-0 via gitlabServer...
Cloning repository...
Cloning into '<Project name>'...
Checking out ed0ce69e as master...
Skipping Git submodules setup
$ # Auto DevOps variables and functions # collapsed multi-line command
$ setup_docker
$ build
Building Heroku-based application using gliderlabs/herokuish docker image...
**docker: Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?.**
See 'docker run --help'.
ERROR: Job failed: exit code 125
What could be the error? Docker itsself is running as it seems, but the docker inside does not seem to work.
This is my .toml file:
[[runners]]
name = my name
url = my url
token = my token
executor = "docker"
[runners.docker]
tls_verify = false
image = "docker:latest"
privileged = true
disable_cache = false
volumes = ["/cache"]
shm_size = 0
[runners.cache]
Thanks in advance for help!
Edit: Thats what "docker ps" gave as output:
ED STATUS PORTS NAMES
e66e844481b7 7961fbf38d6f "gitlab-runner-ser..." 2 sec onds ago Up Less than a second runner-73520410-proje ct-1-concurrent-0-docker-0-wait-for-service
4f659dba7bac b9145b364a20 "dockerd-entrypoin..." 2 sec onds ago Up 1 second 2375/tcp runner-73520410-proje ct-1-concurrent-0-docker-0
73776d4638b9 gitlab/gitlab-runner:latest "/usr/bin/dumb-ini..." 19 mi nutes ago Up 19 minutes gitlab-runner
Edit 2: My gitlab-ci.yaml
#ruby 2.2
rspec:ruby2.2:
image: ruby:2.2
script:
- bundle exec rspec spec
tags:
- ruby
except:
- tags
#ruby 2.1
rspec:ruby2.1:
image: ruby:2.1
script:
- bundle exec rspec spec
tags:
- ruby
except:
- tags
.go: &go_definition
before_script:
- apt-get update -qq && apt-get install -y ruby
- ruby -v
script:
- go version
- which go
- bin/compile
- support/go-test
- support/go-format check
go:1.8:
<<: *go_definition
image: golang:1.8
codeclimate:
before_script: []
image: docker:latest
variables:
DOCKER_DRIVER: overlay
services:
- docker:dind
script:
- docker pull codeclimate/codeclimate
- docker run --env CODECLIMATE_CODE="$PWD" --volume "$PWD":/code --volume /var/run/docker.sock:/var/run/docker.sock --volume /tmp/cc:/tmp/cc codeclimate/codeclimate analyze -f json > codeclimate.json
artifacts:
paths: [codeclimate.json]
This is my registration command, I think you are missing to pass the privileged during registration and also make sure the gitlab-runner user is part of the docker group:
gitlab-runner register \
--template-config /tmp/gitlab-config.toml \
--config /etc/gitlab-runner/config.toml \
--non-interactive \
--url "$gitlab_url" \
--registration-token "$runner_registration_token" \
--name "$runner_name" \
--tag-list "$runner_tags" \
--run-untagged="$runner_run_untagged" \
--locked="$runner_locked" \
--access-level="$runner_access" \
--maximum-timeout="$maximum_timeout" \
--executor "docker" \
--docker-privileged \
--docker-volumes "/cache" \
--docker-volumes "/certs/client" \
--docker-image "$runner_image"
sudo usermod -aG docker gitlab-runner
# concurrent global can't be setup in registration
# See: https://gitlab.com/gitlab-org/gitlab/-/issues/332497
sed -i "s/concurrent.*/concurrent = $concurrent/" /etc/gitlab-runner/config.toml
# prometheus port for GitLab Runner is 9252 as defined here https://github.com/prometheus/prometheus/wiki/Default-port-allocations
echo -e "listen_address = \":9252\"\n$(cat /etc/gitlab-runner/config.toml)" > /etc/gitlab-runner/config.toml