I'm trying to parametrize the jobs.myjob.container.image field. The documentation says the needs context is available there:
Contexts documentation
Specifically this:
Workflow key
Context
jobs.<job_id>.container
github, needs, strategy, matrix, env, secrets, inputs
But it doesn't work. My job output is an empty string, causing an error.
get_image:
name: get_image
runs-on: self-hosted
outputs:
image: ${{ steps.jq.image }}
needs:
- ...
steps:
- name: Checkout code
uses: actions/checkout#v3
- name: jq
id: jq
run: |
set -x
export TAG=$(jq -r '.${{ github.event.inputs.cluster }} | .tag' data.json)
echo "::set-output name=image::registry.com/mycontainer:$TAG"
job2:
name: job2
runs-on: self-hosted
needs:
- get_image
container:
image: ${{ needs.get_image.outputs.image }} <--- this is an empty string
credentials:
...
steps:
...
The error I'm getting is Error: The template is not valid. ...: Unexpected value ''.
Is the documentation lying to me or am I just reading it wrong?
Other questions lead me to think that the thing I want to do is not allowed.
parametrize container.image
github community discussion
You should use outputs here image: ${{ steps.jq.outputs.image }}.
Related
I'm not sure if it's possible, but I'm attempting to use outputs from one reusable workflow, in another, that's part of the same caller workflow. For reference, please see the configs below:
Caller Workflow:
jobs:
call-workflow-calver:
uses: ./.github/workflows/called-workflow1.yaml
secrets: inherit
call-workflow-echo:
needs: call-workflow-calver
uses: ./.github/workflows/called-workflow2.yaml
secrets: inherit
Job for creating the CalVer tag (it outputs as $VERSION as part of the action)
Called Workflow 1:
...
jobs:
calver:
name: Create CalVer tag
...
steps:
- name: Calver tag
uses: StephaneBour/actions-calver#1.4.4
if: ${{ github.ref == 'refs/heads/main' }}
id: calVer
with:
date_format: "%Y-%m-%d"
release: ${{ github.event_name == 'push' && github.ref == 'refs/heads/main' }}
...
Trying to use the CalVer $VERSION output from another caller workflow
Called Workflow 2:
...
- name: Echo
run: |
echo ${{needs.calver.outputs.VERSION}}
...
The basic concept is I'm trying to use outputs from one reusable workflow in another for setting CalVer versions in workflow 1, calling that in workflow 2 so I can set it as an image version. I will eventually use in a 3rd reusable workflow to deploy said image. If this is possible, that would be great!
Hopefully, this all makes a bit of sense, but if anything needs clarifying, please do let me know!
Many thanks in advance!
According to the official documentation, you can now declare outputs to reusable workflows.
These work just like job outputs and are available via needs.<reusable>.outputs.<output> format once you declare the output.
Example
1. Reusable workflow configuration:
name: Reusable workflow
on:
workflow_call:
# Map the workflow outputs to job outputs
outputs:
firstword:
description: "The first output string"
value: ${{ jobs.example_job.outputs.output1 }}
secondword:
description: "The second output string"
value: ${{ jobs.example_job.outputs.output2 }}
jobs:
example_job:
name: Generate output
runs-on: ubuntu-latest
# Map the job outputs to step outputs
outputs:
output1: ${{ steps.step1.outputs.firstword }}
output2: ${{ steps.step2.outputs.secondword }}
steps:
- id: step1
run: echo "::set-output name=firstword::hello"
- id: step2
run: echo "::set-output name=secondword::world"
2. Workflow using the reusable:
name: Call a reusable workflow and use its outputs
on:
workflow_dispatch:
jobs:
job1:
uses: octo-org/example-repo/.github/workflows/called-workflow.yml#v1
job2:
runs-on: ubuntu-latest
needs: job1
steps:
- run: echo ${{ needs.job1.outputs.firstword }} ${{ needs.job1.outputs.secondword }}
Note that if a reusable workflow that sets an output is executed with a matrix strategy, the output will be the output set by the last successful completing reusable workflow of the matrix which actually sets a value. That means if the last successful completing reusable workflow sets an empty string for its output, and the second last successful completing reusable workflow sets an actual value for its output, the output will contain the value of the second last completing reusable workflow.
I used this workflow as example here if you want to check the logs of the workflow run.
I hope someone can help me out with this github action issue.
I am trying to set a github action pipeline that parse the some tags and invoke another reusable workflow.
Here is what the code looks like:
name: release per tag
on:
release:
types: [ published ]
permissions:
actions: write
checks: write
contents: write
deployments: write
issues: write
packages: write
pull-requests: write
repository-projects: write
security-events: write
statuses: write
jobs:
get_project_folder:
name: "Find project folder"
runs-on: ubuntu-latest
outputs:
project_folder: ${{ steps.regex-match.outputs.name }}
steps:
- id: regex-match
uses: actions-ecosystem/action-regex-match#v2
with:
text: ${{ github.event.release.tag_name }}
regex: '.*(?=\-)'
- id: to-variable
run: echo "::set-output name=project_name::${{ steps.regex-match.outputs.match }}"
build_release_package:
name: "Invoke standard release tag yaml"
uses: ./.github/workflows/standard_release_tag.yaml
with:
project_name: ${{ needs.get_project_folder.outputs.project_folder }}
environment: Production
secrets:
INSTALL_PKG_PAT: ${{ secrets.INSTALL_PKG_PAT }}
HOST_URL: ${{ secrets.HOST_URL }}
ACCESS_TOKEN: ${{ secrets.ACCESS_TOKEN }}
ENV: prod
Somehow this pipeline is not being able to call the standard_release_tag.yaml even it exists on the repository.
Is this something happened to anyone? I googled but only saw this happen for a different trigger event and was fixed by Github team.
Thanks.
Solved by validating that the pipeline was invoking and old version of the yaml pipeline.
Building a GitHub action based on the commit message I'm trying to base a step on whether the commit message contains a particular string, set it to a variable and then in the next step check with a condition.
My current implementation of my action works:
name: Smoke Test
on:
push:
branches:
- main
permissions:
contents: read
issues: write
jobs:
smoking:
runs-on: [ubuntu-latest]
steps:
- name: Run smoke tests
if: ${{ !contains(github.event.head_commit.message, 'smoke_test') }}
run: |
echo 'Smoke Test not requested'
exit 1
stuff:
needs: smoking
runs-on: ubuntu-latest
steps:
- uses: actions/checkout#v2
- uses: JasonEtco/create-an-issue#v2
env:
GITHUB_TOKEN: ${{ secrets.TOKEN }}
with:
filename: .github/ISSUE_TEMPLATE/smoke-test.md
id: create-issue
- run: 'echo Created issue number ${{ steps.create-issue.outputs.number }}'
- run: 'echo Created ${{ steps.create-issue.outputs.url }}'
but with the implementation of:
exit 1
causes the action to indicate it error'ed out in the action panel and while that works that isn't technically accurate because I don't need it to error I just don't want the remaining steps to run.
I've tried setting a variable:
if: ${{ contains(github.event.head_commit.message, 'smoke_test') }}
with:
run-smoke-test: true
run: |
echo 'Smoke Test requested'
but it's not passing to the next step.
Research
Use environment variable in github action if
How to pass variable between two successive GitHub Actions jobs?
github-action: does the IF have an ELSE?
How to fail a job in GitHub Actions?
GitHub Actions - trigger another action after one action is completed
Without relying on another GitHub action is there a way in step smoking to set an env variable that step stuff would need to validate for before running the step?
Edit
After reading the answer and implementing job outputs I've written:
name: Smoke Test
on:
push:
branches:
- main
permissions:
contents: read
issues: write
jobs:
commitMessage:
runs-on: ubuntu-latest
outputs:
output1: ${{ steps.isSmoke.outputs.test }}
steps:
- id: isSmoke
if: ${{ contains(github.event.head_commit.message, 'smoke_test') }}
run: echo "::set-output name=test::true"
smokeTest:
runs-on: ubuntu-latest
needs: commitMessage
steps:
- uses: actions/checkout#v2
- uses: JasonEtco/create-an-issue#v2
if: steps.isSmoke.output.test == true
env:
GITHUB_TOKEN: ${{ secrets.DEV_TOKEN }}
with:
filename: .github/ISSUE_TEMPLATE/smoke-test.md
but when the commit message of smoke_test is used it bypasses create-an-issue:
and I'm basing my condition after reading "Run github actions step based on output condition" and reading:
Contexts
Expressions
Using conditions to control job execution
Can a condition come before a step and/or what is the correct way to run a step based off the previous step?
You are looking for job outputs, which allow you to send data to the following jobs.
I'm using composite GitHub actions, where I want to check the current branch name in composite action's some steps and make the decision on that condition.
e.g.
name: main
on:
push:
repository_dispatch:
types:
- manual-trigger
jobs:
build:
runs-on: windows-latest
steps:
- name: Checkout project
uses: actions/checkout#v2
- name: Fetch full project
run: git fetch --prune --unshallow
- name: Restore packages
run: nuget restore -ConfigFile "../Build/Nuget.config"
working-directory: Projects
env:
# ARTIFACTORY_PASSWORD is read by the nuget.config credentials section
ARTIFACTORY_PASSWORD: ${{ secrets.ARTIFACTORY_PASSWORD }}
- name: Composite Action - To build solution and execute SonarScanner
uses: ./Build/build-and-sonarscanner-execution
Where in Composite action - I do have a check that Sonar-Scanner should execute only for develop branch, else only project build will gets execute.
name: build-and-sonarscanner-execution
description: "Build the solution using msbuild and SonarScanner execution"
inputs:
# Set of parameters
runs:
using: "composite"
steps:
- name: Restore packages
run: nuget restore -ConfigFile "../Build/Nuget.config"
working-directory: ${{ inputs.solution-directory }}
env:
# ARTIFACTORY_PASSWORD is read by the nuget.config credentials section
ARTIFACTORY_PASSWORD: ${{ inputs.artifactory-password }}
shell: pwsh
- name: Install dotnet-sonarscanner package
if: {{ github.ref == 'ref/head/develop' }} This is the line where it is throwing syntax error as 'if' is not allowed under steps
run: dotnet tool install --global dotnet-sonarscanner --version 4.10.0
shell: pwsh
Here and here are the references I have looked for before applying this if condition here. And that works totally fine if I apply that main.yml, but in composite action yaml file that's not working.
Can someone please share that what am I missing in here?
The above syntax if: ${{ ... }} should be fine but it seems that the composite actions do not support conditions yet: https://github.com/actions/runner/blob/main/docs/adrs/0549-composite-run-steps.md
The workaround that Benjamin proposed is working for me:
run: |
if [[ "${{inputs.myVar}}" != "" ]]; then
aws ...
fi
exit 0
shell: bash
Try the below syntax:
if: ${{ github.ref == 'refs/heads/xxxx' }}
Does GitHub action support nested templates? For example, here is an example of Azure Pipeline yaml where it calls another yaml file:
- job: BuildFunctions
steps:
- ${{ each func in parameters.functionApps }}:
- template: yaml/build-functionapps.yml
parameters:
Is it possible to call a yaml file from another yaml file in GitHub actions?
You can use composite run steps actions. These are actions that are solely defined in YAML (documentation).
You can now specify containers, other composite actions (up to a depth of 9) and node actions in additional to the previously available run steps
node actions likely refers to leaf actions, i.e. actions that don't call any other actions.
Source: https://github.com/actions/runner/issues/646#issuecomment-901336347
Workflow
[...]
jobs:
job:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout#v2
- uses: ./.github/workflows/composite-action
[...]
Composite run steps action
.github/workflows/composite-action/action.yml (same repository as the workflow)
name: "My composite action"
description: "Checks out the repository and does something"
runs:
using: "composite"
steps:
- uses: actions/checkout#v2
- uses: actions/setup-node#v2
with:
node-version: 12
- run: npm test
shell: bash
- run: |
echo "Executing action"
shell: bash
Old limitations:
What does composite run steps currently support?
For each run step in a composite action, we support:
name
id
run
env
shell
working-directory
In addition, we support mapping input and outputs throughout the action.
See docs for more info.
What does Composite Run Steps Not Support
We don't support setting conditionals, continue-on-error, timeout-minutes, "uses" [remark: i.e. using other actions], and secrets on individual steps within a composite action right now.
(Note: we do support these attributes being set in workflows for a step that uses a composite run steps action)
Source: https://github.com/actions/runner/issues/646
I think using the composite action pattern, you can achieve what you want.
You need to define the steps which you think will be reused in other places, and make it parameterized, by providing inputs. In my opinion, it's more powerful than how templates work in gitlab or in other similar platforms.
This way, you are defining a function, which can take inputs, and get stuff done for you, based on those inputs.
Also, even though the docs suggest that, you should create your leaf action as a separate public repo, and use it in your base action- it's not necessary, you can simply have a structure like below(taken the example from one of our live workflow), and use those leaf actions in your workflow-
.github
- actions
- deploy-to-k8s
- action.yaml
- publish-image
- action.yaml
- workflows
- deploy-from-pr.yaml <-- this will make use of all the actions defined
Here's how the deploy-from-pr.yaml workflow looks like-
name: deploy-from-pr
on:
pull_request:
branches:
- master
env:
REGISTRY: ghcr.io
IMAGE_NAME: ${{ github.repository }}
jobs:
deploy-from-pr:
name: Deploy from PR to Development
runs-on: ubuntu-latest
steps:
- uses: actions/checkout#v2
- name: Set version
id: release
run: echo ::set-output name=version::$(git describe --always)
# custom action to build and push image
- name: Build & publish image
uses: ./.github/actions/publish-image # see how it's referred from same repo directly
with:
registry: ${{ env.REGISTRY }}
registry_username: ${{ github.REGISTRY_USERNAME }}
registry_password: ${{ secrets.REGISTRY_PASSWORD }}
image_name: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}
tag: ${{ steps.release.outputs.version }}
# custom action to deploy into kubernetes
- name: Deploy to kubernetes
uses: ./.github/actions/deploy-to-k8s # see how it's referred from same repo directly
with:
digitalocean_token: ${{ secrets.DIGITALOCEAN_TOKEN }}
cluster_name: ${{ secrets.CLUSTER_NAME }}
overlay_path: .k8s/overlays/dev
image_name: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}
tag: ${{ steps.release.outputs.version }}
Github Gist
You can check the deploy-to-k8s/action.yaml, to see how it's written.
No, it is not. I asked the exact same question in the GitHub Forum:
Is it possible to create / publish Actions without Docker or JS by having the code in the Workflow Syntax / YML?
As mentioned in the document: Currently, types of actions only lists
Docker container and JavaScript, so there is no such feature to
achieve your requirement.
Source: https://github.community/t/how-to-create-ready-to-use-action-without-docker-js/124889/2
This would have eased creating templates for users as system administrator.
You can also use reusable workflows.