I have this reusable workflow:
`
test:
name: Test
runs-on: ubuntu-latest
steps:
- uses: actions/checkout#v3
- uses: actions/setup-python#v3
with:
python-version: '3.8'
- uses: aws-actions/setup-sam#v2
- name: build
run: sam build --beta-features
`
I call this workflow with a matrix from outside. However the setup=-sam#v2 is taking a lot of time: around 26secs. Is it possible to cache this between matrix runs?
Thanks
Trying to fasten the setup of sam in github workflow
Matrix strategy is used per job, so one possibility is to pull this
- name: build
run: sam build --beta-features
into a separate job instance. Then you will use with matrix strategy only this part:
- name: build
run: sam build --beta-features
If you want to use caching try sth like:
test:
name: Test
runs-on: ubuntu-latest
steps:
- uses: actions/checkout#v3
- uses: actions/setup-python#v3
with:
python-version: '3.8'
- name: Cache sam
id: cache-sam
uses: actions/cache#v3
with:
path: <specify the path>
key: ${{ runner.os }}-sam
- uses: aws-actions/setup-sam#v2
if: steps.cache-sam.outputs.cache-hit != 'true'
- name: build
run: sam build --beta-features
However, we have to provide a path in the above code snipped. Maybe the installation path will be good.
Please check out the following links:
https://github.com/actions/cache
How do I cache steps in GitHub actions?
Related
I have this GitHub Actions workflow working:
name: Test Bootstrap
on: push
jobs:
bootstrap-on-mac:
runs-on: macos-latest
steps:
- name: Checkout Code
uses: actions/checkout#v2
- name: Bootstrap
run: ./bootstrap.sh
bootstrap-on-ubuntu:
runs-on: ubuntu-latest
steps:
- name: Checkout Code
uses: actions/checkout#v2
- name: Bootstrap
run: ./bootstrap.sh
But is there a way to combine the jobs into a single job that gets executed on multiple operating systems?
So something like this?:
name: Test Bootstrap
on: push
jobs:
bootstrap-on-mac:
runs-on:
- macos-latest
- ubuntu-latest
steps:
- name: Checkout Code
uses: actions/checkout#v2
- name: Bootstrap
run: ./bootstrap.sh
I couldn't get the above working because I think GitHub interprets it as requiring an OS that's labelled macos-latest, ubuntu-latest, which doesn't exist.
I tried to use the matrix strategy, but I kept on getting errors and none of the example matrices are 1 dimensional. All of the examples are 2 dimensions or higher.
Thank you for your time 🙏
This is answered here: https://github.com/orgs/community/discussions/39131#discussioncomment-4146138
Essentially, the matrix feature works and can implemented like so:
jobs:
bootstrap:
strategy:
matrix:
os:
- macos-latest
- ubuntu-latest
runs-on: ${{ matrix.os }}
steps:
- ...
I have a main job like the below and two other parallel jobs are dependent on the first job including secret generation and node module installation like secret setup and install node module.
I tried to make it work with needs but all the environment setup is gone with needs.
And reusable workflow seems to just setup keys.
name: build
on: [push]
jobs:
codepull:
runs-on: ubuntu-latest
steps:
- uses: actions/setup-node#v3
with:
node-version: '16.16.0'
- name: install node module
run |
yarn
- name: secrets
run |
yarn secrets
codepull-ios:
- name: build ios
run |
...
codepull-ios:
runs-on: ubuntu-latest
steps:
...
codepull-android:
runs-on: ubuntu-latest
steps:
...
I checked reusable workflow but those seems only for setting up env variables.
Anyone tried to do similar things?
jobs runs in it's own environment so do not share nothing by default.
By you can define jobs output and use it in the dependant job, like:
name: build
on: [push]
jobs:
codepull:
runs-on: ubuntu-latest
outputs: # define here the job output
one-secret: ${{ steps.secret.outputs.my-secret }}
steps:
- uses: actions/setup-node#v3
with:
node-version: '16.16.0'
- name: install node module
run |
yarn
- name: secrets
id: secret
run |
yarn secrets
secretkey=$(cat password.txt) # stupid example to take some var from somewhere
echo "::set-output name=my-secret::$secretkey"
codepull-ios:
- name: build ios
run |
...
codepull-ios:
runs-on: ubuntu-latest
needs:
- codepull
steps:
-run:
echo needs.codepull.outputs.one-secret # use the secrets
...
codepull-android:
runs-on: ubuntu-latest
steps:
...
Some link to the doc about jobs output
As the title suggests, I am building a MEVN (Vue) Stack application and am facing this issue:
Earlier to this, I had been only building the frontedn part of my app & it was successfully getting deployed. Only when did I integrate my backend & changed the folder structure a bit, did I start getting these errors.
Below is my Deploy.yml :
name: Deploy
on:
push:
branches:
- main
jobs:
build:
name: Build
runs-on: ubuntu-latest
steps:
- name: Checkout repo
uses: actions/checkout#v2
- name: Setup Node
uses: actions/setup-node#v1
with:
node-version: 16
- name: Install dependencies
uses: bahmutov/npm-install#v1
- name: Build project
run: npm run build --prefix client
- name: Upload production-ready build files
uses: actions/upload-artifact#v2
with:
name: production-files
path: ./dist
deploy:
name: Deploy
needs: build
runs-on: ubuntu-latest
if: github.ref == 'refs/heads/main'
steps:
- name: Download artifact
uses: actions/download-artifact#v2
with:
name: production-files
path: ./dist
- name: Deploy to GitHub Pages
uses: peaceiris/actions-gh-pages#v3
with:
github_token: ${{ secrets.GITHUB_TOKEN }}
publish_dir: ./dist
Please help.
I'm creating a Github Action Workflow.
There is a phing build file that creates multiple zips depending on product variations. This is done with a matrix.
Before the product is build there is a JS File that needs to be build. That's done with a rollup build file.
Everything works fine, but the JS build only needs to be done once and not with every matrix combination.
I don't know how that should be done. Maybe with a single separate workflow running at start, but how would the finished JS file be pushed into the next workflow? Or maybe one workflow is fine?
name: Build
on: [push]
jobs:
build:
runs-on: ubuntu-latest
env:
VERSION: 1.0.1
strategy:
matrix:
edition: ["product1", "product2", "product3"]
limits: [100, 200, 300]
exclude:
- edition: product1
limits: 100
- edition: product2
limits: 100
- edition: product1
limits: 200
- edition: product2
limits: 200
steps:
- uses: actions/checkout#v2
- name: Build JS
uses: actions/setup-node#v1
with:
node-version: 12.x
- name: Cache Node.js modules
uses: actions/cache#v2
with:
path: ~/.npm
key: ${{ runner.OS }}-node-${{ hashFiles('**/package-lock.json') }}
restore-keys: |
${{ runner.OS }}-node-
${{ runner.OS }}-
- run: npm ci
- run: npm run build # builds main.js
- name: Phing Build
uses: phingofficial/phing-github-action#main
with:
repo-token: "${{ secrets.GITHUB_TOKEN }}"
buildfile: build/phing/build.xml
targets: main
user-properties: editions=${{ matrix.edition }} limits=${{ matrix.limits }}
version: 3.0.0-alpha4
- name: Archive code
uses: actions/upload-artifact#v2
with:
name: file-${{ matrix.edition }}${{ matrix.limits }}-v${{ env.VERSION }}
path: build/${{ matrix.edition }}/zip/
retention-days: 1
You can execute creating the JS File in a separate job (let's name it prepare) and then reference that job in the job with the matrix definition (build) using the needs keyword. Sharing files across jobs is something that GitHub is very specific about: not each job needs all files from a previous job, so you will need to upload and download that file yourself.
Example setup:
jobs:
prepare:
runs-on: ubuntu-latest
steps:
...
# upload the file here for later use
- uses: actions/upload-artifact
build:
needs: prepare
strategy:
matrix:
edition: ["product1", "product2", "product3"]
steps:
# download the file here so you can use it
- uses: actions/download-artifact
See the docs on the needs keyword here.
Switching from Travis CI to GitHub Actions, I was wondering if there is a way to share common steps between jobs. For a project, I need each job to start with 3 actions: check out repository code, install Node.js v12, restore node_modules from the cache (if available). Actually I have added these 3 actions for each job, which works but is a bit verbose. Is there a way to say: "Each job must run these actions first" or something like that?
name: ci
on: [push, workflow_dispatch]
jobs:
setup:
runs-on: ubuntu-latest
steps:
- name: Check out repository code
uses: actions/checkout#v2
- name: Setup node
uses: actions/setup-node#v2
with:
node-version: '12'
- name: Cache node_modules
id: cache-node-modules
uses: actions/cache#v2
with:
path: node_modules
key: node-modules-${{ hashFiles('package-lock.json') }}
- name: Install dependencies
run: npm install
test_mysql:
runs-on: ubuntu-latest
needs: setup
steps:
- name: Check out repository code
uses: actions/checkout#v2
- name: Setup node
uses: actions/setup-node#v2
with:
node-version: '12'
- name: Restore node_modules
id: cache-node-modules
uses: actions/cache#v2
with:
path: node_modules
key: node-modules-${{ hashFiles('package-lock.json') }}
- name: Test MySQL 5
run: npm run test-mysql
env:
DOCKER_MYSQL_TAG: 5
- name: Test MySQL 8
run: npm run test-mysql
env:
DOCKER_MYSQL_TAG: 8
test_postgres:
runs-on: ubuntu-latest
needs: setup
steps:
- name: Check out repository code
uses: actions/checkout#v2
- name: Setup node
uses: actions/setup-node#v2
with:
node-version: '12'
- name: Restore node_modules
id: cache-node-modules
uses: actions/cache#v2
with:
path: node_modules
key: node-modules-${{ hashFiles('package-lock.json') }}
- name: Test Postgres 10
run: npm run test-postgres
env:
DOCKER_POSTGRES_TAG: 10
- name: Test Postgres 11
run: npm run test-postgres
env:
DOCKER_POSTGRES_TAG: 11
- name: Test Postgres 12
run: npm run test-postgres
env:
DOCKER_POSTGRES_TAG: 12
test_mariadb:
runs-on: ubuntu-latest
needs: setup
steps:
- name: Check out repository code
uses: actions/checkout#v2
- name: Setup node
uses: actions/setup-node#v2
with:
node-version: '12'
- name: Restore node_modules
id: cache-node-modules
uses: actions/cache#v2
with:
path: node_modules
key: node-modules-${{ hashFiles('package-lock.json') }}
- name: Test MariaDB 10.4
run: npm run test-mariadb
env:
DOCKER_MARIADB_TAG: 10.4.12
test_mssql:
runs-on: ubuntu-latest
needs: setup
steps:
- name: Check out repository code
uses: actions/checkout#v2
- name: Setup node
uses: actions/setup-node#v2
with:
node-version: '12'
- name: Restore node_modules
id: cache-node-modules
uses: actions/cache#v2
with:
path: node_modules
key: node-modules-${{ hashFiles('package-lock.json') }}
- name: Test MSSQL 2017
run: npm run test-mssql
env:
DOCKER_MSSQL_TAG: 2017-CU17-ubuntu
- name: Test MSSQL 2019
run: npm run test-mssql
env:
DOCKER_MSSQL_TAG: 2019-latest
test_sqlite:
runs-on: ubuntu-latest
needs: setup
steps:
- name: Check out repository code
uses: actions/checkout#v2
- name: Setup node
uses: actions/setup-node#v2
with:
node-version: '12'
- name: Restore node_modules
id: cache-node-modules
uses: actions/cache#v2
with:
path: node_modules
key: node-modules-${{ hashFiles('package-lock.json') }}
- name: Test SQLite
run: npm run test-sqlite
publish:
runs-on: ubuntu-latest
needs: [test_mysql, test_postgres, test_mariadb, test_mssql, test_sqlite]
steps:
- name: Check out repository code
uses: actions/checkout#v2
- name: Setup node
uses: actions/setup-node#v2
with:
node-version: '12'
- name: Restore node_modules
id: cache-node-modules
uses: actions/cache#v2
with:
path: node_modules
key: node-modules-${{ hashFiles('package-lock.json') }}
- name: Build
run: npm run build
- name: Check version changes
uses: EndBug/version-check#v1
id: check
- name: Publish
if: steps.check.outputs.changed == 'true' && github.ref == 'refs/heads/master'
run: |
npm set registry "https://registry.npmjs.org"
npm set //registry.npmjs.org/:_authToken ${{ secrets.NPM_PUBLISH_TOKEN }}
npm publish
Now is possible to use uses in composite actions - please check this link
Composite action:
name: "Publish to Docker"
description: "Pushes built artifacts to Docker"
inputs:
registry_username:
description: “Username for image registry”
required: true
registry_password:
description: “Password for image registry”
required: true
runs:
using: "composite"
steps:
- uses: docker/setup-buildx-action#v1
- uses: docker/login-action#v1
with:
username: ${{inputs.registry_username}}
password: ${{inputs.registry_password}}
- uses: docker/build-push-action#v2
with:
context: .
push: true
tags: user/app:latest
And then:
on: [push]
jobs:
publish:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout#v2
- uses: my-org/publish-docker#v1
with:
registry_username: ${{secrets.REGISTRY_USERNAME}}
registry_password: ${{secrets.REGISTRY_PASSWORD}}
Original reply:
It looks that at the moment is not possible if among steps you want to share are uses.
It should be handled by composite actions but
What does composite run steps currently support?
For each run step in a composite action, we support:
name
id
run
env
shell
working-directory
In addition, we support mapping input and outputs throughout the action.
See docs for more info.
What does Composite Run Steps Not Support
We don't support setting conditionals, continue-on-error, timeout-minutes, "uses", and secrets on individual steps within a composite action right now.