I am using CircleCI with a GameCI docker image in order to build a Unity project. The build works, but I am trying to make use of the h-matsuo/github-release orb in order to create a release on GitHub for the build. I have created a new separate job for this, so I needed to share data between the jobs. I am using persist_to_workspace in order to do that, as specified in the documentation, but the solution doesn't seem to work. I get the following error:
Could not ensure that workspace directory /root/project/Zipped exists
For the workspace persist logic, I've added the following lines of code in my config.yml file:
working_directory: /root/project - Inside the executor of the main job
persist_to_workspace - As a last command inside my main job's steps
attach_workspace - As a beginning command inside my second job's steps
Here's my full config.yml file:
version: 2.1
orbs:
github-release: h-matsuo/github-release#0.1.3
executors:
unity_exec:
docker:
- image: unityci/editor:ubuntu-2019.4.19f1-windows-mono-0.9.0
environment:
BUILD_NAME: speedrun-circleci-build
working_directory: /root/project
.build: &build
executor: unity_exec
steps:
- checkout
- run: mkdir -p /root/project/Zipped
- run:
name: Git submodule recursive
command: git submodule update --init --recursive
- run:
name: Remove editor folder in shared project
command: rm -rf ./Assets/Shared/Movement/Generic/Attributes/Editor/
- run:
name: Converting Unity license
command: chmod +x ./ci/unity_license.sh && ./ci/unity_license.sh
- run:
name: Building game binaries
command: chmod +x ./ci/build.sh && ./ci/build.sh
- run:
name: Zipping build
command: apt update && apt -y install zip && zip -r "/root/project/Zipped/build.zip" ./Builds/
- store_artifacts:
path: /root/project/Zipped/build.zip
- run:
name: Show all files
command: find "$(pwd)"
- persist_to_workspace:
root: Zipped
paths:
- build.zip
jobs:
build_windows:
<<: *build
environment:
BUILD_TARGET: StandaloneWindows64
release:
description: Build project and publish a new release tagged `v1.1.1`.
executor: github-release/default
steps:
- attach_workspace:
at: /root/project/Zipped
- run:
name: Show all files
command: sudo find "/root/project"
- github-release/create:
tag: v1.1.1
title: Version v1.1.1
description: This release is version v1.1.1.
file-path: ./build.zip
workflows:
version: 2
build:
jobs:
- build_windows
- release:
requires:
- build_windows
Can somebody help me with this please?
If somebody ever encounters the same issue, try to avoid making use of the /root path. I've stored the artifacts somewhere inside /tmp/, and before storing artifacts, I've manually created the folder with chmod 777 by using mkdir with the -m flag to specify chmod permissions.
Related
I have a multi-stage build that uses a docker image which contains all the node_modules needed for a build and other dependencies. I used that in a multi-stage docker build to build a production dist for a project. In the 2nd stage I use a much smaller docker to store the dist files so it's easier to distribute.
Dockerfile for the build:
FROM ghcr.io/dancing-star/packages-image-full:latest AS builder
# Make the dir
WORKDIR /var/dancing-star
COPY . .
# Set the cache location
RUN npm config set cache /tmp/node-cache --global \
&& npm run prod \
&& rm -rf src \
&& ls -l
# Dist stage
FROM alpine
WORKDIR /var/dancing-star
COPY --from=builder ["/var/dancing-star/dist", "/var/dancing-star/assets", "/var/dancing-star/config", "/var/dancing-star/scripts", "/var/dancing-star/*.json", "/var/dancing-star/*.md", "/var/dancing-star/*.js", "/var/dancing-star/"]
RUN ls -l
#CMD [ "pm2-runtime", "start", "app.pm2.json", "--env", "production" ]
The ghcr.io/dancing-star/packages-image-full contains the node_modules and works fine. I used a github action to cache ghcr.io/dancing-star/packages-image-full but it doesn't seem to be caching it very well since it's constantly pulling it again.
.github/workflows/docky.yaml
name: Build Docker
on:
push:
branches: [ main ]
tags:
- v*
env:
IMAGE_NAME: project-docker
BUILD_IMAGE_NAME: ghcr.io/naologic/dancing-star/packages-image-full
BUILD_IMAGE_PATH: ghcr.io/naologic/dancing-star/packages-image-full:latest
jobs:
build:
runs-on: ubuntu-latest
permissions:
packages: write
contents: read
steps:
- name: Checkout
uses: actions/checkout#v3
- name: Set up Docker Buildx
uses: docker/setup-buildx-action#v2
- name: Login to GitHub Container Registry
uses: docker/login-action#v2
with:
registry: ghcr.io
username: ${{ github.actor }}
password: ${{ secrets.DOCKER_DEPLOY_GITHUB }}
- name: Build and push
uses: docker/build-push-action#v4
with:
context: .
file: Dockerfile
push: true
tags: ${{ env.BUILD_IMAGE_PATH }}
cache-from: type=registry,ref=${{ env.BUILD_IMAGE_NAME }}:buildcache
cache-to: type=registry,ref=${{ env.BUILD_IMAGE_NAME }}:buildcache,mode=max
IMPORTANT: ghcr.io/dancing-star/packages-image-full is shared between many builds so ideally it would be a shared cache.
How should I use actions to cache ghcr.io/dancing-star/packages-image-full for the build?
My solution doesn't seem to be working very well.
I tried the current action, cache still pulls for 2 minutes before starting
I tried without cache, build time is about the same
I want to cache the Android NDK in my Github Actions workflow. The reason is that I require a specific version of the NDK and CMake which aren't pre-installed on MacOS runners.
I tried to use the following workflow job to achieve this:
jobs:
build:
runs-on: macos-latest
steps:
- name: Cache NDK
id: cache-primes
uses: actions/cache#v1
with:
path: ${{ env.ANDROID_NDK_HOME }}
key: ${{ runner.os }}-ndk-${{ hashFiles(env.ANDROID_NDK_HOME) }}
- name: Install NDK
run: echo "y" | $ANDROID_HOME/tools/bin/sdkmanager "ndk;21.0.6113669" "cmake;3.10.2.4988404"
The problem with this is that the env context doesn't contain the ANDROID_NDK_HOME variable. So this means build.steps.with.path evaluates empty.
The regular environment variable is present and prints the correct path if I debug using the following step:
jobs:
build:
steps:
- name: Debug print ANDROID_NDK_HOME
run: echo $ANDROID_NDK_HOME
But the regular environment variable can only be used in shell scripts and not in build.steps.with as far as I understand.
- name: Prepare NDK dir for caching ( workaround for https://github.com/actions/virtual-environments/issues/1337 )
run: |
sudo mkdir -p /usr/local/lib/android/sdk/ndk
sudo chmod -R 777 /usr/local/lib/android/sdk/ndk
sudo chown -R $USER:$USER /usr/local/lib/android/sdk/ndk
- name: NDK Cache
id: ndk-cache
uses: actions/cache#v2
with:
path: /usr/local/lib/android/sdk/ndk
key: ndk-cache-21.0.6113669-v2
- name: Install NDK
if: steps.ndk-cache.outputs.cache-hit != 'true'
run: echo "y" | sudo /usr/local/lib/android/sdk/tools/bin/sdkmanager --install "ndk;21.0.6113669"
Here is a config I use in my project.
Couple of note:
You need to create ndk directory and change permission to workaround https://github.com/actions/virtual-environments/issues/1337
Make sure
you use proper id (ndk-cache in example above) in if statement so
you can actually use cache
You can easily specify the NDK installation directory to be cached.
- name: Cache (NDK)
uses: actions/cache#v2
with:
path: ${ANDROID_HOME}/ndk/21.0.6113669
key: ndk-cache
- name: Install NDK
run: echo "y" | sudo ${ANDROID_HOME}/tools/bin/sdkmanager --install 'ndk;21.0.6113669'
As long as you're using an NDK version that is pre-installed with Github Actions runners, then you no longer need to worry about caching your NDK :)
Find the runners list here:
https://docs.github.com/en/actions/using-github-hosted-runners/about-github-hosted-runners#supported-software
Example runner list:
https://github.com/actions/runner-images/blob/main/images/linux/Ubuntu2204-Readme.md
For the Ubuntu 22.04 runner, it comes with three NDKs pre-installed
23.2.8568313
24.0.8215888
25.2.9519653 (default)
I have recently built a selenium test project. Now i want to host it on circle CI.
Can someone please provide me the step by step details with the configuration file.
I already have a Github and circle CI account. Im new to this integration aspect.
#
# Check https://circleci.com/docs/2.0/language-java/ for more details
#
version: 2
jobs:
build:
docker:
- image: circleci/openjdk:8-jdk-stretch-browsers
environment:
# Customize the JVM maximum heap limit
MAVEN_OPTS: -Xmx3200m
steps:
- checkout
- run: mkdir test-reports
- run:
name: Download Selenium
command: |
curl -O http://selenium-release.storage.googleapis.com/3.5/selenium-server-standalone-3.5.3.jar
- run:
name: Start Selenium
command: |
java -jar selenium-server-standalone-3.5.3.jar -log test-reports/selenium.log
background: true
- run: mvn dependency:go-offline
- save_cache:
paths:
- ~/.m2
key: v1-dependencies-{{ checksum "pom.xml" }}
# run tests!
- run: mvn clean integration-test
- run:
name: Save test results
command: |
mkdir -p ~/testng/results/
find . -type f -regex "./test-output/testng-results.xml" -exec cp {} ~/testng/results/ \;
when: always
- store_test_results:
path: ~/testng/results/
- store_artifacts:
path: ~/testng/results/
- store_artifacts:
path: testng/results/
After a lot of digging the internet I was finally able to create my own config file. I searched crazily but I could'nt anthying for testng. I started building from scratch. Hopefully this helps someone in the future.
I'm trying to build a pipeline for a gradle java app which is using an embedded mongo instance. I have built a container to which has java and mongo. However, I keep getting the following error for all my test that require the embedded mongo instance.
org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'embeddedMongoServer'
defined in class path resource [org/springframework/boot/autoconfigure/mongo/embedded/EmbeddedMongoAutoConfiguration.class]:
Invocation of init method failed; nested exception is java.io.IOException:
Cannot run program "/tmp/extract-f816c11c-614b-46d7-ad29-68923ca9d624extractmongod": error=2, No such file or directory
My gitlab-ci.yml looks like this:
image: java:latest
services:
- docker:dind
variables:
GRADLE_OPTS: "-Dorg.gradle.daemon=false"
DOCKER_DRIVER: overlay
SPRING_PROFILES_ACTIVE: gitlab-ci
stages:
- build
- test
build:
stage: build
script: ./gradlew --build-cache assemble
cache:
key: "$CI_COMMIT_REF_NAME"
policy: push
paths:
- build
- .gradle
test:
stage: test
image: registry.gitlab.com/path/to/explorer-ci:1.0.0
script: ./gradlew check --debug
cache:
key: "$CI_COMMIT_REF_NAME"
policy: pull
paths:
- build
- .gradle
The build job works fine, the test job fails. My explorer-ci container is built using the following Dockerfile
FROM openjdk:8-jdk-alpine
RUN apk update && \
apk add --no-cache \
mongodb \
bash
VOLUME /data/db
VOLUME log
RUN ["mongod", "--smallfiles", "--fork", "--logpath", "log/mongodb.log"]
I've spent a week with a bunch of different configs but can't seem to hack it. Just to note, builds/tests run fine on my local machine. Any ideas of what I'm doing wrong?
On reflection, as I am using an embedded mongo instance, I do not have a dependency on mongodb to build or test. I am now using the following gitlab-ci.yaml and it works fine.
image: openjdk:8-jdk
variables:
GRADLE_OPTS: "-Dorg.gradle.daemon=false"
DOCKER_DRIVER: overlay
SPRING_PROFILES_ACTIVE: gitlab-ci
stages:
- build
- test
build:
stage: build
script: ./gradlew --build-cache assemble
cache:
key: "$CI_COMMIT_REF_NAME"
policy: push
paths:
- build
- .gradle
test:
stage: test
script: ./gradlew check
cache:
key: "$CI_COMMIT_REF_NAME"
policy: pull
paths:
- build
- .gradle
The issue was on base docker image for your project runner, You need to use jdk version not alpine.
Try to change base image to this image: openjdk:8-jdk and it will work fine.
In our case the solution described at Fladoodle embedded MongoDB on the issues here helped:
https://github.com/flapdoodle-oss/de.flapdoodle.embed.mongo/issues/281
In short - we added this to our Gitlab script (or do this in container setup itself if possible) before running our tests:
apk --no-cache add ca-certificates
wget -q -O /etc/apk/keys/sgerrand.rsa.pub https://alpine-pkgs.sgerrand.com/sgerrand.rsa.pub
wget https://github.com/sgerrand/alpine-pkg-glibc/releases/download/2.29-r0/glibc-2.29-r0.apk
wget https://github.com/sgerrand/alpine-pkg-glibc/releases/download/2.29-r0/glibc-bin-2.29-r0.apk
apk add glibc-2.29-r0.apk glibc-bin-2.29-r0.apk
wget https://github.com/sgerrand/alpine-pkg-glibc/releases/download/2.29-r0/glibc-i18n-2.29-r0.apk
apk add glibc-bin-2.29-r0.apk glibc-i18n-2.29-r0.apk
/usr/glibc-compat/bin/localedef -i en_US -f UTF-8 en_US.UTF-8
just updating the dependency should resolve:
<dependency>
<groupId>de.flapdoodle.embed</groupId>
<artifactId>de.flapdoodle.embed.mongo</artifactId>
<version>2.2.0</version>
<scope>test</scope>
</dependency>
I have an application which is in a private github repo, and am wondering if the releases could be made public so that the app can auto-update itself from github instead of us having to host it.
Additionally I'm wondering if it's possible to use the github api from the deployed app to check for new updates.
A workaround would be to create a public repo, composed of:
empty commits (git commit --allow-empty)
each commit tagged
each tag with a release
each release with the deliveries (the binaries of your private app)
That way, you have a visible repo dedicated for release hosting, and a private repos for source development.
As #VonC mentioned we have to create a second Repository for that. This is not prohibited and i am doing it already. With github workflows i automated this task, I'm using a develop / master branching, so always when I'm pushing anything to the master branch a new version is build and pushed to the public "Realease" Repo.
In my specific use case I'm building an android apk and releasing it via unoffical github api "hub". Some additional advantage of this is you can have an extra issue tracker for foreign issues and bugs.
name: Master CI CD
# using checkout#v2 instead of v1 caus it needs further configuration
on:
pull_request:
types: [closed]
jobs:
UnitTest:
runs-on: ubuntu-latest
if: github.event.pull_request.merged
steps:
- uses: actions/checkout#v2
- name: make executable
run: chmod +x gradlew
- name: Unit tests
run: |
./gradlew test
IncrementVersionCode:
needs: UnitTest
runs-on: ubuntu-latest
steps:
- uses: actions/checkout#v2
- name: set up JDK 1.8
uses: actions/setup-java#v1
with:
java-version: 1.8
- name: make executable
run: chmod +x gradlew
- name: increment version
run: ./gradlew incrementVersionCode
- name: Push new version to master
run: |
git config --local user.email "workflow#bot.com"
git config --local user.name "WorkflowBot"
git commit -m "Increment Build version" -a
# maybe better amend commits to avoid bot commits
BuildArtifacts:
needs: IncrementVersionCode
runs-on: ubuntu-latest
steps:
- uses: actions/checkout#v2
- name: set up JDK 1.8
uses: actions/setup-java#v1
with:
java-version: 1.8
- name: make executable
run: chmod +x gradlew
- name: Build with Gradle
run: ./gradlew build -x lint
- name: Rename artifacts
run: |
cp app/build/outputs/apk/release/app-release.apk MyApp.apk
- name: Upload Release
uses: actions/upload-artifact#master
with:
name: Release Apk
path: MyApp.apk
- name: Upload Debug
uses: actions/upload-artifact#master
with:
name: Debug Apk
path: app/build/outputs/apk/debug/app-debug.apk
# https://dev.to/ychescale9/running-android-emulators-on-ci-from-bitrise-io-to-github-actions-3j76
E2ETest:
needs: BuildArtifacts
runs-on: macos-latest
strategy:
matrix:
api-level: [21, 27]
arch: [x86]
steps:
- name: checkout
uses: actions/checkout#v2
- name: Make gradlew executable
run: chmod +x ./gradlew
- name: run tests
uses: reactivecircus/android-emulator-runner#v2
with:
api-level: ${{ matrix.api-level }}
arch: ${{ matrix.arch }}
script: ./gradlew connectedCheck
Deploy:
needs: E2ETest
runs-on: ubuntu-latest
if: github.ref == 'refs/heads/master'
steps:
- uses: actions/checkout#v2 # Needed for gradle file to get version information
- name: Get Hub
run: |
curl -fsSL https://github.com/github/hub/raw/master/script/get | bash -s 2.14.1
cd bin
chmod +x hub
cd ..
- name: Get Apk
uses: actions/download-artifact#master
with:
name: Release Apk
- name: Publish
env:
GITHUB_TOKEN: "${{ secrets.RELEASE_REPO_SECRET }}"
run: |
APP_NAME=MyApp
VERSION_NAME=`grep -oP 'versionName "\K(.*?)(?=")' ./app/build.gradle`
VERSION_CODE=`cat version.properties | grep "VERSION_CODE" | cut -d'=' -f2`
FILENAME="${APP_NAME}-v${VERSION_NAME}-${VERSION_CODE}"
TAG="v${VERSION_NAME}-${VERSION_CODE}"
TAG="latest-master"
echo $APP_NAME
echo $VERSION_NAME
echo $VERSION_CODE
echo $FILENAME
echo $TAG
git clone https://github.com/MyUser/MyApp-Releases
cd MyApp-Releases
./../bin/hub release delete "${TAG}" || echo "Failed deleting TAG: ${TAG}" # If release got lost catch error with message
./../bin/hub release create -a "../${APP_NAME}.apk" -m "Current Master Build: ${FILENAME}" -p "${TAG}"
EvaluateCode:
needs: Deploy
runs-on: ubuntu-latest
steps:
- name: Get Hub
run: |
echo "TDOO: Run Jacoco for coverage, and other profiling tools"
The 2022 answer to this question is even more straight-forward.
You'd just need to use the pre-installed gh CLI:
gh release create v0.0.1 foobar.zip -R https://github.com/your/repo-here
This command will create a tag v0.0.1 and a release with the local file foobar.zip attached on the public repository. You can run this in the GitHub Action of any private repository.
The -R argument points to the repository you'd like to create a tag/release on. foobar.zip would be located in your local directory.
One thing is important here: GITHUB_TOKEN must still be set as the token of the repository you'd like to release on!
Full example:
- name: Publish
env:
GITHUB_TOKEN: "${{ secrets.RELEASE_REPO_SECRET }}"
run: |
gh release create v0.0.1 foobar.zip -R https://github.com/your/repo-here
If you're planning to re-release and override existing versions, there is gh release delete as well. The -d flag creates a release as a draft etc. pp. Please take a look at the docs.
I'm using a slightly more advanced approach by setting:
shell: bash
run: $GITHUB_ACTION_PATH/scripts/publish.sh
And in file scripts/publish.sh:
#!/usr/bin/env node
const cp = require('child_process')
const fs = require('fs');
const path = require('path');
const APP_VERSION = JSON.parse(fs.readFileSync('package.json', { encoding: 'utf8' })).version
const TAG = `v${APP_VERSION}`
cp.execSync(`gh release create ${TAG} foobar.zip -R https://github.com/your/repo-name`, { stdio: 'inherit' })
This approach enables you to be able to for example, use Node.js or any other programming language available, to extract a version from the project management file of choice (e.g. package.json) and automatically come up with the right tag version and name.
A simple way to duplicate releases from a private repo to a public one may be this Release & Assets Github Action which can: Create a release, upload release assets, and duplicate a release to other repository.
Then you can use the regular electron-builder updater's support for public repos.