(GitHub Actions) Split step into two consecutive ones - github

I'd like to use GitHub Actions to publish an npm package. So far I am using quite a trivial script to do that. Now I'd like to split one step of the script into two consecutive ones.
here's an excerpt from my workflows/...yaml file:
steps:
# ...
- name: Build
run: |
cd src
npm install
tsc
# TODO split here
npm set registry https://npm.pkg.github.com
npm set //npm.pkg.github.com/:_authToken ${{ secrets.GITHUB_TOKEN }}
npm publish
env:
CI: true
Now, when I tried to have these as separate Steps, they ran in parallel which is not the behavior I hope for, since the first step yields results (creates a src/lib directory) that I depend on in step #2. (The one where I log into npm and publish this).
Can someone please help me to unravel this?

As #smac89 explains above, steps can't be executed in parallel, only jobs and workflows can.
The official Github Documentation shares this illustration about the Github Actions structure:
What probably happened in your case was that you created 2 jobs instead of 2 steps.
The syntax to split your steps should be something like this:
jobs:
my-job:
runs-on: ubuntu-latest #for example
steps:
# ...
- name: Build
run: |
cd src
npm install
tsc
env:
CI: true
- name: Publish
run: |
npm set registry https://npm.pkg.github.com
npm set //npm.pkg.github.com/:_authToken ${{ secrets.GITHUB_TOKEN }}
npm publish
env:
CI: true

Related

Github Actions: Run .sh without checking out whole project?

Due to bandwidth limits, I'm trying to checkout a subfolder of my project to Github Actions and found this Action: https://github.com/marketplace/actions/checkout-files
New (broken) Script:
name: Create Build Target
run-name: ${{ github.actor }} is creating ${{ github.ref_name }}
on: create
jobs:
Create:
runs-on: ubuntu-latest
steps:
- name: Checkout to access bash script
uses: Bhacaz/checkout-files#v2
with:
files: CICD
- name: Create Buildtarget info on Unity Cloud Build
env:
api_key: ${{ secrets.api_key }}
org_id: ${{ secrets.org_id }}
project_id: ${{ secrets.project_id }}
branch_name: ${{ github.ref_name }}
credential_id: ${{ secrets.credential_id }}
run: CICD/CreateBuildTarget.sh
I get an error in the Github Actions terminal when triggering the above .yaml file to check out a subdirectory instead of the whole project:
/home/runner/work/_temp/2ddc6165-7186-415a-8d87-bc4d746f659f.sh: line 1: CICD/CreateBuildTarget.sh: Permission denied
60
Error: Process completed with exit code 126.
I had this working before, and made sure the files had the correct permissions:
myUserName#myUserNames-MacBook-Pro cicd % ls -l
total 16
-rwxr-xr-x# 1 myUserName staff 2239 Feb 3 11:17 CreateBuildTarget.sh
-rwxr-xr-x# 1 myUserName staff 449 Feb 3 11:18 DeleteBuildTarget.sh
The only thing I changed was the checkout action.
Before (working):
- name: Checkout to access bash script
uses: actions/checkout#v2.6.0
After (not working):
- name: Checkout to access bash script
uses: Bhacaz/checkout-files#v2
with:
files: CICD
Is what I'm trying to do even possible? For now, I changed the script to trigger on pull request open/reopen, instead of create, but I still want to only check out a subdirectory instead of the whole project.
The action you're using isn't setting the execute bits on the .sh files. It's relatively simple to add them manually after restoring the files, but you might want to fork the action and make it do the right thing.
Try using a Sparse Checkout action instead, it would rely on Git to restore the files and has a lot more expected default behaviors built-in:
gogaille/sparse-checkout
While your files are executable locally (on your machine), they might not have been added to your Git repository with +x (the executable bit).
Which means, once the GitHub Action checks out your file (even limiting itself to one subfolder), said sh script files are not executable.
Try locally to do, using git add --chmod=+x:
cd /path/to/repository
cd CICD
git add --chmod=+x *.sh
git commit -m "Add executable bit in CICD folder"
git push
Then check if your GitHub Action has the same issue.

Github Action error every step must define a `uses` or `run` key

I couldn't make this workflow work, I'm always receiving this error every step must define a "uses" or "run" key, but looking at my script I don't see any issues, can someone pls help me fix this? It does not seem like a - problem as it usually is.
# This is a basic workflow to help you get started with Actions
name: Build Digital Ocean Image
# Controls when the workflow will run
on:
# Triggers the workflow on push or pull request events but only for the "master" branch
push:
branches:
- '*'
- '*/*'
- '**'
- '!master'
pull_request:
branches: [ "master" ]
# Allows you to run this workflow manually from the Actions tab
workflow_dispatch:
# A workflow run is made up of one or more jobs that can run sequentially or in parallel
jobs:
packages:
runs-on: ubuntu-latest
steps:
- name: Install packages
run: |
apt-get install curl -y
curl -fsSL https://apt.releases.hashicorp.com/gpg | sudo apt-key add -
sudo apt-add-repository "deb [arch=amd64] https://apt.releases.hashicorp.com $(lsb_release -cs) main"
sudo apt-get update && sudo apt-get install packer
apt-get install ansible -y
# This workflow contains a single job called "build"
build:
# The type of runner that the job will run on
# runs-on: ubuntu-latest
# Steps represent a sequence of tasks that will be executed as part of the job
steps:
# Checks-out your repository under $GITHUB_WORKSPACE, so your job can access it
- name: Checkout
uses: actions/checkout#v3
# Runs a single command using the runners shell
- name: Packer init
env:
DIGITALOCEAN_TOKEN=${{ secrets.DIGITALOCEAN_TOKEN }}
run: packer init
working-directory: /home/runner/work/packer-do-custom-images/demo
# Runs a set of commands using the runners shell
- name: Packer build
run: packer build .
working-directory: /home/runner/work/packer-do-custom-images/demo
The runs-on directive is missing (commented!), just add/uncomment it after the name, like:
# This workflow contains a single job called "build"
build:
# The type of runner that the job will run on
runs-on: ubuntu-latest
# Steps represent a sequence of tasks that will be executed as part of the job
steps:
To fix the issue.
I would suggest to use the actionlint tool to discover error like this, as example:
> actionlint .github/workflows/test.yaml
.github/workflows/test.yaml:35:3: "runs-on" section is missing in job "build" [syntax-check]
|
35 | build:
| ^~~~~~
UPDATE:
As side notes also the env variable as issue: you should use : instead of = like:
- name: Packer init
env:
DIGITALOCEAN_TOKEN: ${{ secrets.DIGITALOCEAN_TOKEN }}

How to optimise github actions in a simple setup-lint-test-e2e-build scenario?

If we were to use minimal resources, doing these steps would look something like:
setup --- checkout, install fixed versions via cache
then trigger in parallel 1 command to run:
lint
test
e2e
all in parallel.
Unfortunately, with the current options I've found out there, the best scenario I can find is to have a separate workflow file for lint, test, e2e that are called by a main workflow and upon success build an artifact.
The issue with this is.
checkout will run 3 times.
install command will run 3 times.
Which can be mitigated with caching, but you're still looking at 10-15 seconds extra per job for those steps.
Is there a way to optimise this and still keep the reusable steps?
Even if I would keep everything in 1 giant file, I'd still define the lint, test, e2e steps as separate jobs and not as 1 single step part of a job.
Can we call a 'workflow' and run it in the same environment on the spot somehow?
This example calls checkout and caches the repo using the $ {{ github.run_id }} to prevent the cache conflicting with other runs. The cache is then used by the child workflows in place of checking out. e2e, test and lint will wait for the cache to be created and then all run in parallel.
Depending on your use case it may be excessive to create a new cache for each run_id, you can adjust this as you need. If speed is what you're after I'd also suggest only caching the files that you need for your child workflows, and not take everything like I've done in the example below.
name: main_caller
on: workflow_dispatch
jobs:
cache_checkout:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout#v3
- uses: actions/upload-artifact#v3
with:
name: my-checkout-${{ github.run_id }}
path: ${{ github.workspace }}/**/*
run_lint:
needs: cache_checkout
uses: ./.github/workflows/lint.yml
run_test:
needs: cache_checkout
uses: ./.github/workflows/test.yml
./.github/workflows/lint.yml example: (e2e and test will follow this pattern)
name: lint
on: workflow_call
jobs:
lint:
runs-on: ubuntu-latest
steps:
- name: Get Cached Checkout
uses: actions/download-artifact#v3
with:
name: my-checkout-${{ github.run_id }}
- name: Do linting things
run: echo "I'm linting!"
Edited as per comments - all jobs in GitHub run in parallel, but to have cleaner example you could use matrix strategy. To make it work you need it to depend on a 'setup' job that will install everything and create the artifacts (this is done using the needs keyword).
In the following example the install job would setup everything and then the example job, which uses matrix strategy, will run the lint/test/e2e flow in separate parallel jobs after it downloads the required artifacts.
jobs:
install:
runs-on: ubuntu-latest
- uses: actions/checkout#v3
- name: Compile artifacts
run: *install fixed versions/create artifacts*
- uses: actions/upload-artifact#v3
with:
name: my-artifact
path: path/to/my_artifact
example:
runs-on: ubuntu-latest
needs: install
strategy:
matrix:
action: [lint, test, e2e]
steps:
- uses: actions/download-artifact#v3
with:
name: my-artifact
path: path/to/artifact
- name: lint
if: ${{ matrix.lint }}
run: echo "lint"
- name: test
if: ${{ matrix.test }}
run: echo "test"
- name: e2e
if: ${{ matrix.e2e }}
run: echo "e2e"
It will create a workflow like this:

Updating GitHub issues from GitHub Actions

I was trying to make a GitHub action using some simple scripts (which I already use locally) that I would like to run inside a docker container.
A new issue should trigger the event to update the said issue with its content based on some processing. An example of this might be:
Say I have a list of labels defined in my script and it checks the issue's title and adds a label to the issue.
I'm still reading the GitHub Action's documentation so I may be not completely informed but the issue I seem to have is that in my local machine these scripts use gh cli for doing such tasks (eg. adding labels). So I was wondering if I need to have the gh installed in that docker container or is there a better way to update the issue? I'm very much willing to make these scripts from scratch again using the GitHub's event payloads and stuff as long as I don't have to write in TypeScript.
I've looked around the documentation and couldn't find anything that talked about updating issues. Also couldn't find a similar question being asked here; it may be that I've missed something so if that is the case direct me to relevant material and I would very much appreciate it.
An option could be (as you said) to install GH in that docker container, and then run GH commands.
Example using a container:
jobs:
build:
runs-on: ubuntu-latest
container:
image: docker://myrepoandimagewithghinstalled
steps:
- name: Github CLI Authentication
run: gh auth login --hostname <your hostname>
- name: Github CLI commands execution samples
run: |
gh command1
gh command2
gh command3
Another option could be to install GH directly on the OS (for exemple ubuntu-latest), authenticate, and then use the "run" option to execute GH command.
Example installing GH on the OS:
jobs:
build:
runs-on: ubuntu-latest
steps:
- name: Install Github CLI
run: |
sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-key C99B11DEB97541F0
sudo apt-add-repository https://cli.github.com/packages
sudo apt update
sudo apt install gh
- name: Github CLI Authentication
run: gh auth login --hostname <your hostname>
- name: Github CLI commands execution samples
run: |
gh command1
gh command2
gh command3
Finally, you could also create a script consuming the Github API service to update an ISSUE and execute the script using the run option.
Example to execute a Python script in your workflow:
jobs:
build:
runs-on: ubuntu-latest
steps:
- name: checkout repo content
uses: actions/checkout#v2 # checkout the repository content to github runner.
- name: setup python
uses: actions/setup-python#v2
with:
python-version: 3.8 #install the python needed
- name: execute py script # run the run.py to get the latest data
run: |
python run.py
env:
key: ${{ secrets.key }} # if run.py requires passwords..etc, set it as secrets
- name: export index
.... # use crosponding script or actions to help export.

Deploy on Azure Function using GitHub Actions on push only if Function App is not executing

I have a function app, which basically scrapes data from the web. It is a long-running one, which generally takes 9 hrs a day.
I have configured on push event for build & deploy through GitHub Actions.
Problem: When we push any change to GitHub and the function app is running it will create a mess as the running function will be stopped and triggered after deployment.
I want a solution to deploy on every push but only when the function app is not running.
Content of yml file:
name: dev-workflow
on:
push:
branches:
- main
env:
AZURE_FUNCTIONAPP_NAME: test-github-actions
AZURE_FUNCTIONAPP_PACKAGE_PATH: '.'
PYTHON_VERSION: '3.7'
jobs:
build-and-deploy:
runs-on: ubuntu-latest
steps:
- name: 'Checkout GitHub Action'
uses: actions/checkout#main
- name: Setup Python ${{ env.PYTHON_VERSION }} Environment
uses: actions/setup-python#v1
with:
python-version: ${{ env.PYTHON_VERSION }}
- name: 'Resolve Project Dependencies Using Pip'
shell: bash
run: |
pushd './${{ env.AZURE_FUNCTIONAPP_PACKAGE_PATH }}'
python -m pip install --upgrade pip
pip install -r requirements.txt --target=".python_packages/lib/site-packages"
popd
- name: 'Run Azure Functions Action'
uses: Azure/functions-action#v1
id: fa
with:
app-name: ${{ env.AZURE_FUNCTIONAPP_NAME }}
package: ${{ env.AZURE_FUNCTIONAPP_PACKAGE_PATH }}
publish-profile: ${{ secrets.PUBLISH_PROFILE }}
According to this post response from chrispat (6 months ago) there isn't
any native feature to prevent duplicate workflows currently on Github Actions.
However, there is this action that might help to prevent duplicated actions to start.
Another option could be to limit concurrent workflows runs, you can find more about it in this post.