I want to cache the Android NDK in my Github Actions workflow. The reason is that I require a specific version of the NDK and CMake which aren't pre-installed on MacOS runners.
I tried to use the following workflow job to achieve this:
jobs:
build:
runs-on: macos-latest
steps:
- name: Cache NDK
id: cache-primes
uses: actions/cache#v1
with:
path: ${{ env.ANDROID_NDK_HOME }}
key: ${{ runner.os }}-ndk-${{ hashFiles(env.ANDROID_NDK_HOME) }}
- name: Install NDK
run: echo "y" | $ANDROID_HOME/tools/bin/sdkmanager "ndk;21.0.6113669" "cmake;3.10.2.4988404"
The problem with this is that the env context doesn't contain the ANDROID_NDK_HOME variable. So this means build.steps.with.path evaluates empty.
The regular environment variable is present and prints the correct path if I debug using the following step:
jobs:
build:
steps:
- name: Debug print ANDROID_NDK_HOME
run: echo $ANDROID_NDK_HOME
But the regular environment variable can only be used in shell scripts and not in build.steps.with as far as I understand.
- name: Prepare NDK dir for caching ( workaround for https://github.com/actions/virtual-environments/issues/1337 )
run: |
sudo mkdir -p /usr/local/lib/android/sdk/ndk
sudo chmod -R 777 /usr/local/lib/android/sdk/ndk
sudo chown -R $USER:$USER /usr/local/lib/android/sdk/ndk
- name: NDK Cache
id: ndk-cache
uses: actions/cache#v2
with:
path: /usr/local/lib/android/sdk/ndk
key: ndk-cache-21.0.6113669-v2
- name: Install NDK
if: steps.ndk-cache.outputs.cache-hit != 'true'
run: echo "y" | sudo /usr/local/lib/android/sdk/tools/bin/sdkmanager --install "ndk;21.0.6113669"
Here is a config I use in my project.
Couple of note:
You need to create ndk directory and change permission to workaround https://github.com/actions/virtual-environments/issues/1337
Make sure
you use proper id (ndk-cache in example above) in if statement so
you can actually use cache
You can easily specify the NDK installation directory to be cached.
- name: Cache (NDK)
uses: actions/cache#v2
with:
path: ${ANDROID_HOME}/ndk/21.0.6113669
key: ndk-cache
- name: Install NDK
run: echo "y" | sudo ${ANDROID_HOME}/tools/bin/sdkmanager --install 'ndk;21.0.6113669'
As long as you're using an NDK version that is pre-installed with Github Actions runners, then you no longer need to worry about caching your NDK :)
Find the runners list here:
https://docs.github.com/en/actions/using-github-hosted-runners/about-github-hosted-runners#supported-software
Example runner list:
https://github.com/actions/runner-images/blob/main/images/linux/Ubuntu2204-Readme.md
For the Ubuntu 22.04 runner, it comes with three NDKs pre-installed
23.2.8568313
24.0.8215888
25.2.9519653 (default)
Related
I'm trying to set my Github secrets as env variables via Actions but for some reasons they aren't being set (Using Ubuntu 22.04.1 LTS).
name: My app
on:
push:
branches: [ "main" ]
env:
JWT_SECRET_KEY: ${{ secrets.JWT_SECRET_KEY }}
JWT_REFRESH_SECRET: ${{ secrets.JWT_REFRESH_SECRET }}
MONGO_CONNECTION_STRING: ${{ secrets.MONGO_CONNECTION_STRING }}
PLAI_APP_URL: ${{ secrets.APP_URL }}
SENDGRID_API_KEY: ${{ secrets.SENDGRID_API_KEY }}
permissions:
contents: read
jobs:
build:
runs-on: self-hosted
steps:
- name: check out latest repo
uses: actions/checkout#v3
with:
ref: main
- name: Set up Python 3.10
uses: actions/setup-python#v3
with:
python-version: "3.10"
- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install -r requirements.txt
Workflow runs successfully without errors but when I check my variables via env they are not listed. Any idea what I'm doing wrong?
You could follow the same idea as in "Managing secrets in an open-sourced flutter web app" from Mahesh Jamdade:
encode your confidential file to base64 and store the encoded output to GitHub secrets.
In your GitHub workflow decode the encrypted secret and convert it back to a file and then run your build scripts.
You can see an example in "How to access secrets when using flutter web with github actions".
Example workflow: maheshmnj/vocabhub .github/workflows/firebase-hosting-merge.yml.
I created a workflow for my Python repo as follows:
name: Python package
on: [push, pull_request]
jobs:
build:
runs-on: [ubuntu-latest, macos-latest]
strategy:
fail-fast: false
matrix:
python-version: ["3.7", "3.8", "3.9", "3.10"]
steps:
- uses: actions/checkout#v3
- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python#v3
with:
python-version: ${{ matrix.python-version }}
- name: Install dependencies
run: |
python -m pip install --upgrade pip
python -m pip install flake8 pytest semver
if [ -f requirements.txt ]; then pip install -r requirements.txt; fi
- name: Lint with flake8
run: |
# stop the build if there are Python syntax errors or undefined names
flake8 . --count --select=E9,F63,F7,F82 --show-source --statistics
# exit-zero treats all errors as warnings.
flake8 . --count --exit-zero --max-complexity=10 --ignore=E501 --statistics
- name: Test with pytest
run: |
pytest
Unfortunately, the action never runs and times out with the error:
This request was automatically failed because there were no enabled runners online to process the request for more than 1 days.
Did I do something silly in the configuration file?
I'm currently on a free GitHub account. Are GitHub-hosted runners available on free accounts? If so how do I enable one of those?
Turns out
runs-on: [ubuntu-latest, macos-latest]
doesn't run the action on each platforms. Instead it tries to find a runner that satisfies both conditions, i.e. running on ubuntu-latest and macos-latest which is, of course, never found.
The way to so what I originally intended is to, instead, do a two-dimensional matrix for os and python-version.
I couldn't make this workflow work, I'm always receiving this error every step must define a "uses" or "run" key, but looking at my script I don't see any issues, can someone pls help me fix this? It does not seem like a - problem as it usually is.
# This is a basic workflow to help you get started with Actions
name: Build Digital Ocean Image
# Controls when the workflow will run
on:
# Triggers the workflow on push or pull request events but only for the "master" branch
push:
branches:
- '*'
- '*/*'
- '**'
- '!master'
pull_request:
branches: [ "master" ]
# Allows you to run this workflow manually from the Actions tab
workflow_dispatch:
# A workflow run is made up of one or more jobs that can run sequentially or in parallel
jobs:
packages:
runs-on: ubuntu-latest
steps:
- name: Install packages
run: |
apt-get install curl -y
curl -fsSL https://apt.releases.hashicorp.com/gpg | sudo apt-key add -
sudo apt-add-repository "deb [arch=amd64] https://apt.releases.hashicorp.com $(lsb_release -cs) main"
sudo apt-get update && sudo apt-get install packer
apt-get install ansible -y
# This workflow contains a single job called "build"
build:
# The type of runner that the job will run on
# runs-on: ubuntu-latest
# Steps represent a sequence of tasks that will be executed as part of the job
steps:
# Checks-out your repository under $GITHUB_WORKSPACE, so your job can access it
- name: Checkout
uses: actions/checkout#v3
# Runs a single command using the runners shell
- name: Packer init
env:
DIGITALOCEAN_TOKEN=${{ secrets.DIGITALOCEAN_TOKEN }}
run: packer init
working-directory: /home/runner/work/packer-do-custom-images/demo
# Runs a set of commands using the runners shell
- name: Packer build
run: packer build .
working-directory: /home/runner/work/packer-do-custom-images/demo
The runs-on directive is missing (commented!), just add/uncomment it after the name, like:
# This workflow contains a single job called "build"
build:
# The type of runner that the job will run on
runs-on: ubuntu-latest
# Steps represent a sequence of tasks that will be executed as part of the job
steps:
To fix the issue.
I would suggest to use the actionlint tool to discover error like this, as example:
> actionlint .github/workflows/test.yaml
.github/workflows/test.yaml:35:3: "runs-on" section is missing in job "build" [syntax-check]
|
35 | build:
| ^~~~~~
UPDATE:
As side notes also the env variable as issue: you should use : instead of = like:
- name: Packer init
env:
DIGITALOCEAN_TOKEN: ${{ secrets.DIGITALOCEAN_TOKEN }}
I am using CircleCI with a GameCI docker image in order to build a Unity project. The build works, but I am trying to make use of the h-matsuo/github-release orb in order to create a release on GitHub for the build. I have created a new separate job for this, so I needed to share data between the jobs. I am using persist_to_workspace in order to do that, as specified in the documentation, but the solution doesn't seem to work. I get the following error:
Could not ensure that workspace directory /root/project/Zipped exists
For the workspace persist logic, I've added the following lines of code in my config.yml file:
working_directory: /root/project - Inside the executor of the main job
persist_to_workspace - As a last command inside my main job's steps
attach_workspace - As a beginning command inside my second job's steps
Here's my full config.yml file:
version: 2.1
orbs:
github-release: h-matsuo/github-release#0.1.3
executors:
unity_exec:
docker:
- image: unityci/editor:ubuntu-2019.4.19f1-windows-mono-0.9.0
environment:
BUILD_NAME: speedrun-circleci-build
working_directory: /root/project
.build: &build
executor: unity_exec
steps:
- checkout
- run: mkdir -p /root/project/Zipped
- run:
name: Git submodule recursive
command: git submodule update --init --recursive
- run:
name: Remove editor folder in shared project
command: rm -rf ./Assets/Shared/Movement/Generic/Attributes/Editor/
- run:
name: Converting Unity license
command: chmod +x ./ci/unity_license.sh && ./ci/unity_license.sh
- run:
name: Building game binaries
command: chmod +x ./ci/build.sh && ./ci/build.sh
- run:
name: Zipping build
command: apt update && apt -y install zip && zip -r "/root/project/Zipped/build.zip" ./Builds/
- store_artifacts:
path: /root/project/Zipped/build.zip
- run:
name: Show all files
command: find "$(pwd)"
- persist_to_workspace:
root: Zipped
paths:
- build.zip
jobs:
build_windows:
<<: *build
environment:
BUILD_TARGET: StandaloneWindows64
release:
description: Build project and publish a new release tagged `v1.1.1`.
executor: github-release/default
steps:
- attach_workspace:
at: /root/project/Zipped
- run:
name: Show all files
command: sudo find "/root/project"
- github-release/create:
tag: v1.1.1
title: Version v1.1.1
description: This release is version v1.1.1.
file-path: ./build.zip
workflows:
version: 2
build:
jobs:
- build_windows
- release:
requires:
- build_windows
Can somebody help me with this please?
If somebody ever encounters the same issue, try to avoid making use of the /root path. I've stored the artifacts somewhere inside /tmp/, and before storing artifacts, I've manually created the folder with chmod 777 by using mkdir with the -m flag to specify chmod permissions.
I have an application which is in a private github repo, and am wondering if the releases could be made public so that the app can auto-update itself from github instead of us having to host it.
Additionally I'm wondering if it's possible to use the github api from the deployed app to check for new updates.
A workaround would be to create a public repo, composed of:
empty commits (git commit --allow-empty)
each commit tagged
each tag with a release
each release with the deliveries (the binaries of your private app)
That way, you have a visible repo dedicated for release hosting, and a private repos for source development.
As #VonC mentioned we have to create a second Repository for that. This is not prohibited and i am doing it already. With github workflows i automated this task, I'm using a develop / master branching, so always when I'm pushing anything to the master branch a new version is build and pushed to the public "Realease" Repo.
In my specific use case I'm building an android apk and releasing it via unoffical github api "hub". Some additional advantage of this is you can have an extra issue tracker for foreign issues and bugs.
name: Master CI CD
# using checkout#v2 instead of v1 caus it needs further configuration
on:
pull_request:
types: [closed]
jobs:
UnitTest:
runs-on: ubuntu-latest
if: github.event.pull_request.merged
steps:
- uses: actions/checkout#v2
- name: make executable
run: chmod +x gradlew
- name: Unit tests
run: |
./gradlew test
IncrementVersionCode:
needs: UnitTest
runs-on: ubuntu-latest
steps:
- uses: actions/checkout#v2
- name: set up JDK 1.8
uses: actions/setup-java#v1
with:
java-version: 1.8
- name: make executable
run: chmod +x gradlew
- name: increment version
run: ./gradlew incrementVersionCode
- name: Push new version to master
run: |
git config --local user.email "workflow#bot.com"
git config --local user.name "WorkflowBot"
git commit -m "Increment Build version" -a
# maybe better amend commits to avoid bot commits
BuildArtifacts:
needs: IncrementVersionCode
runs-on: ubuntu-latest
steps:
- uses: actions/checkout#v2
- name: set up JDK 1.8
uses: actions/setup-java#v1
with:
java-version: 1.8
- name: make executable
run: chmod +x gradlew
- name: Build with Gradle
run: ./gradlew build -x lint
- name: Rename artifacts
run: |
cp app/build/outputs/apk/release/app-release.apk MyApp.apk
- name: Upload Release
uses: actions/upload-artifact#master
with:
name: Release Apk
path: MyApp.apk
- name: Upload Debug
uses: actions/upload-artifact#master
with:
name: Debug Apk
path: app/build/outputs/apk/debug/app-debug.apk
# https://dev.to/ychescale9/running-android-emulators-on-ci-from-bitrise-io-to-github-actions-3j76
E2ETest:
needs: BuildArtifacts
runs-on: macos-latest
strategy:
matrix:
api-level: [21, 27]
arch: [x86]
steps:
- name: checkout
uses: actions/checkout#v2
- name: Make gradlew executable
run: chmod +x ./gradlew
- name: run tests
uses: reactivecircus/android-emulator-runner#v2
with:
api-level: ${{ matrix.api-level }}
arch: ${{ matrix.arch }}
script: ./gradlew connectedCheck
Deploy:
needs: E2ETest
runs-on: ubuntu-latest
if: github.ref == 'refs/heads/master'
steps:
- uses: actions/checkout#v2 # Needed for gradle file to get version information
- name: Get Hub
run: |
curl -fsSL https://github.com/github/hub/raw/master/script/get | bash -s 2.14.1
cd bin
chmod +x hub
cd ..
- name: Get Apk
uses: actions/download-artifact#master
with:
name: Release Apk
- name: Publish
env:
GITHUB_TOKEN: "${{ secrets.RELEASE_REPO_SECRET }}"
run: |
APP_NAME=MyApp
VERSION_NAME=`grep -oP 'versionName "\K(.*?)(?=")' ./app/build.gradle`
VERSION_CODE=`cat version.properties | grep "VERSION_CODE" | cut -d'=' -f2`
FILENAME="${APP_NAME}-v${VERSION_NAME}-${VERSION_CODE}"
TAG="v${VERSION_NAME}-${VERSION_CODE}"
TAG="latest-master"
echo $APP_NAME
echo $VERSION_NAME
echo $VERSION_CODE
echo $FILENAME
echo $TAG
git clone https://github.com/MyUser/MyApp-Releases
cd MyApp-Releases
./../bin/hub release delete "${TAG}" || echo "Failed deleting TAG: ${TAG}" # If release got lost catch error with message
./../bin/hub release create -a "../${APP_NAME}.apk" -m "Current Master Build: ${FILENAME}" -p "${TAG}"
EvaluateCode:
needs: Deploy
runs-on: ubuntu-latest
steps:
- name: Get Hub
run: |
echo "TDOO: Run Jacoco for coverage, and other profiling tools"
The 2022 answer to this question is even more straight-forward.
You'd just need to use the pre-installed gh CLI:
gh release create v0.0.1 foobar.zip -R https://github.com/your/repo-here
This command will create a tag v0.0.1 and a release with the local file foobar.zip attached on the public repository. You can run this in the GitHub Action of any private repository.
The -R argument points to the repository you'd like to create a tag/release on. foobar.zip would be located in your local directory.
One thing is important here: GITHUB_TOKEN must still be set as the token of the repository you'd like to release on!
Full example:
- name: Publish
env:
GITHUB_TOKEN: "${{ secrets.RELEASE_REPO_SECRET }}"
run: |
gh release create v0.0.1 foobar.zip -R https://github.com/your/repo-here
If you're planning to re-release and override existing versions, there is gh release delete as well. The -d flag creates a release as a draft etc. pp. Please take a look at the docs.
I'm using a slightly more advanced approach by setting:
shell: bash
run: $GITHUB_ACTION_PATH/scripts/publish.sh
And in file scripts/publish.sh:
#!/usr/bin/env node
const cp = require('child_process')
const fs = require('fs');
const path = require('path');
const APP_VERSION = JSON.parse(fs.readFileSync('package.json', { encoding: 'utf8' })).version
const TAG = `v${APP_VERSION}`
cp.execSync(`gh release create ${TAG} foobar.zip -R https://github.com/your/repo-name`, { stdio: 'inherit' })
This approach enables you to be able to for example, use Node.js or any other programming language available, to extract a version from the project management file of choice (e.g. package.json) and automatically come up with the right tag version and name.
A simple way to duplicate releases from a private repo to a public one may be this Release & Assets Github Action which can: Create a release, upload release assets, and duplicate a release to other repository.
Then you can use the regular electron-builder updater's support for public repos.