Github Action: run one step before matrix combinations - github

I'm creating a Github Action Workflow.
There is a phing build file that creates multiple zips depending on product variations. This is done with a matrix.
Before the product is build there is a JS File that needs to be build. That's done with a rollup build file.
Everything works fine, but the JS build only needs to be done once and not with every matrix combination.
I don't know how that should be done. Maybe with a single separate workflow running at start, but how would the finished JS file be pushed into the next workflow? Or maybe one workflow is fine?
name: Build
on: [push]
jobs:
build:
runs-on: ubuntu-latest
env:
VERSION: 1.0.1
strategy:
matrix:
edition: ["product1", "product2", "product3"]
limits: [100, 200, 300]
exclude:
- edition: product1
limits: 100
- edition: product2
limits: 100
- edition: product1
limits: 200
- edition: product2
limits: 200
steps:
- uses: actions/checkout#v2
- name: Build JS
uses: actions/setup-node#v1
with:
node-version: 12.x
- name: Cache Node.js modules
uses: actions/cache#v2
with:
path: ~/.npm
key: ${{ runner.OS }}-node-${{ hashFiles('**/package-lock.json') }}
restore-keys: |
${{ runner.OS }}-node-
${{ runner.OS }}-
- run: npm ci
- run: npm run build # builds main.js
- name: Phing Build
uses: phingofficial/phing-github-action#main
with:
repo-token: "${{ secrets.GITHUB_TOKEN }}"
buildfile: build/phing/build.xml
targets: main
user-properties: editions=${{ matrix.edition }} limits=${{ matrix.limits }}
version: 3.0.0-alpha4
- name: Archive code
uses: actions/upload-artifact#v2
with:
name: file-${{ matrix.edition }}${{ matrix.limits }}-v${{ env.VERSION }}
path: build/${{ matrix.edition }}/zip/
retention-days: 1

You can execute creating the JS File in a separate job (let's name it prepare) and then reference that job in the job with the matrix definition (build) using the needs keyword. Sharing files across jobs is something that GitHub is very specific about: not each job needs all files from a previous job, so you will need to upload and download that file yourself.
Example setup:
jobs:
prepare:
runs-on: ubuntu-latest
steps:
...
# upload the file here for later use
- uses: actions/upload-artifact
build:
needs: prepare
strategy:
matrix:
edition: ["product1", "product2", "product3"]
steps:
# download the file here so you can use it
- uses: actions/download-artifact
See the docs on the needs keyword here.

Related

Github actions cache between jobs but not between workflows

Currently I am trying to have 1 job scanning SonarQube and 1 job checking for the quality gate in github actions. In order to get the report from gradle of the quality check in the second job, I have to cache it (or atleast with the limited knowledge I have). But I don't want when I rerun the workflow on the same PR or on different PRs use the same cache, since the report is only valid for a current workflow and not futher ones (those need to always create a new report and give the new report to the second job).
Here is my workflow:
name: SonarQube
on:
push:
branches:
- master # or the name of your main branch
pull_request:
types: [opened, synchronize, reopened]
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
jobs:
scan:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout#v3
with:
fetch-depth: 0 # Shallow clones should be disabled for a better relevancy of analysis
- name: Set up JDK 11
uses: actions/setup-java#v3
with:
java-version: 11
distribution: corretto
cache: gradle
- name: Build and analyze
env:
SONAR_TOKEN: ${{ secrets.SONAR_TOKEN }}
SONAR_HOST_URL: ${{ secrets.SONAR_HOST_URL }}
run: ./gradlew --info sonar -Dsonar.host.url=$SONAR_HOST_URL -Dsonar.login=$SONAR_TOKEN
- name: Cache report
uses: actions/cache#v3
with:
path: build/sonar/
key: report-task
qualityCheck:
needs: scan
runs-on: ubuntu-latest
steps:
- name: Get cache report
uses: actions/cache#v3
with:
path: build/sonar/
key: report-task
- name: Quality Gate check
id: sonarqube-quality-gate-check
uses: sonarsource/sonarqube-quality-gate-action#master
# Force to fail step after specific time.
timeout-minutes: 5
with:
scanMetadataReportFile: build/sonar/report-task.txt
env:
SONAR_TOKEN: ${{ secrets.SONAR_TOKEN }}
SONAR_HOST_URL: ${{ secrets.SONAR_HOST_URL }}
What needs to be changed to achieve this?
I have currently done this:
- name: Cache report
uses: actions/cache#v3
with:
path: build/sonar/
key: commit-${{ github.sha }}-workflow-${{ github.run_id }}-${{ github.run_number }}-report-task-${{ hashFiles('build/sonar/**') }}
This doesn't solve the problem that it keeps the cache after the workflow ends, but it now won't use the report of previous workflows.

caching sam setup in github workflow

I have this reusable workflow:
`
test:
name: Test
runs-on: ubuntu-latest
steps:
- uses: actions/checkout#v3
- uses: actions/setup-python#v3
with:
python-version: '3.8'
- uses: aws-actions/setup-sam#v2
- name: build
run: sam build --beta-features
`
I call this workflow with a matrix from outside. However the setup=-sam#v2 is taking a lot of time: around 26secs. Is it possible to cache this between matrix runs?
Thanks
Trying to fasten the setup of sam in github workflow
Matrix strategy is used per job, so one possibility is to pull this
- name: build
run: sam build --beta-features
into a separate job instance. Then you will use with matrix strategy only this part:
- name: build
run: sam build --beta-features
If you want to use caching try sth like:
test:
name: Test
runs-on: ubuntu-latest
steps:
- uses: actions/checkout#v3
- uses: actions/setup-python#v3
with:
python-version: '3.8'
- name: Cache sam
id: cache-sam
uses: actions/cache#v3
with:
path: <specify the path>
key: ${{ runner.os }}-sam
- uses: aws-actions/setup-sam#v2
if: steps.cache-sam.outputs.cache-hit != 'true'
- name: build
run: sam build --beta-features
However, we have to provide a path in the above code snipped. Maybe the installation path will be good.
Please check out the following links:
https://github.com/actions/cache
How do I cache steps in GitHub actions?

Github action executes an action one at the end of the other

I have the following two actions, how can I make the second action be executed at the end of the first after making the first one commit and push?
Action1
on:
workflow_dispatch:
inputs:
name: Scrape Data
jobs:
build:
name: Build
runs-on: ubuntu-latest
steps:
- uses: actions/checkout#master
- name: Build
run: npm install
- name: Scrape
run: npm run action
- uses: mikeal/publish-to-github-action#master
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} # GitHub sets this for you
Action2
on:
workflow_dispatch:
inputs:
name: Visit Data
jobs:
build:
name: Build
runs-on: ubuntu-latest
steps:
- uses: actions/checkout#master
- name: Build
run: npm install
- name: Scrape
run: npm run visit
- uses: mikeal/publish-to-github-action#master
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} # GitHub sets this for you
You could use the workflow_run trigger on the second workflow.
Example:
name: Visit Data
on:
workflow_run:
workflows: ['Scrape Data'] # First workflow name
types:
- completed # can also use 'requested'
jobs:
build:
name: Build
runs-on: ubuntu-latest
steps:
- uses: actions/checkout#master
- name: Build
run: npm install
- name: Scrape
run: npm run visit
- uses: mikeal/publish-to-github-action#master
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
Note that you can't use workflow inputs in that case (I observed you had it set, and if it's necessary you would need to use another trigger, for example through the Github API using a workflow dispatch event with a payload).

Cache implementation not working in github action

I've tried to follow the official documentation, several blogs and answers on stackoverflow but nothing helped me to find a solution for this problem.
With github-action I've created a script to run unit tests after every push is done in our github server. For now the project is in early stage, so our dependency list is quite short and npm install command is done quite quickly (around 40 seconds). Obviously in future we'll need to add more dependencies and we are expecting a serious increment of that time, so I've tried to implement the cache mechanism, but for some reason the process still need to download the dependencies and the time for that step still be pretty much the same.
Below there is the script I've used. Following the documentation online I've found 2 different options, I've tried both (once per time) but without luck.
name: unit
on:
push:
branches: ['*']
pull_request:
branches: ['*']
workflow_dispatch:
jobs:
build:
runs-on: macos-latest
steps:
- uses: actions/checkout#v2
- name: setup node
uses: actions/setup-node#v2
with:
node-version: '15'
cache: npm
# start option 1
- name: Setup cache
uses: actions/cache#v2
with:
path: app/node_modules
key: ${{ runner.os }}-node-${{ hashFiles('app/package-lock.json') }}
restore-keys: |
${{ runner.os }}-node-
# end option 1
# start option 2
- name: Cache node modules
uses: actions/cache#v2
env:
cache-name: cache-node-modules
with:
path: ~/.npm
key: ${{ runner.os }}-build-${{ env.cache-name }}-${{ hashFiles('**/package-lock.json') }}
restore-keys: |
${{ runner.os }}-build-${{ env.cache-name }}-
${{ runner.os }}-build-
${{ runner.os }}-
# end option 2
- name: Install dependencies
run: npm install
- name: run unit tests
run: npm run test
Why the cache is not working? Am I doing some mistake in the script?

github action failing? tar empty archive, docker run failed with exit code 1

This is the project structure:
- parent/
|- .github/workflows/
|- frontend/
|- ...
This is the .yml file within workflows:
name: CI
on:
push:
branches:
- master
jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout#v1
- uses: actions/setup-node#v1
with:
node-version: "12.x"
- name: Install dependencies
working-directory: frontend
run: npm install
- name: Build
working-directory: frontend
run: npm run build
- name: Deploy Files
uses: appleboy/scp-action#master
env:
HOST: ${{ secrets.aws_pull_host }}
USERNAME: ${{ secrets.aws_pull_username }}
KEY: ${{ secrets.aws_pull_private_key }}
with:
working-directory: frontend
source: build/
target: "/home/build/site/testDir/"
strip_components: 1
Whenever the action gets to the step Deploy Files, i get the error:
tar: empty archive
tar all files into /tmp/891353322/bC24rHhFAi.tar
exit status 1
##[error]Docker run failed with exit code 1
First time working with github actions so I am pretty lost with this error. Appreciate any help.
Kept messing with it, finally figured it out:
- name: Deploy Files
uses: appleboy/scp-action#master
env:
HOST: ${{ secrets.aws_pull_host }}
USERNAME: ${{ secrets.aws_pull_username }}
KEY: ${{ secrets.aws_pull_private_key }}
with:
source: frontend/build/
target: "/home/build/site/testDir/"
strip_components: 1