Parameterize `runs-on` value for GitHub action - github

I have this GitHub workflow that I need to parameterize on which runners runs. So in the YAML file I tried:
# ...
jobs:
process:
name: Process
runs-on: ${{ secrets.GH_RUNNER_TAG }}
# ...
However, I get this error:
The workflow is not valid. .github/workflows/action.yml (Line: 12, Col: 14): Unrecognized named-value: 'secrets'. Located at position 1 within expression: secrets.GH_RUNNER_TAG
Is the secrets injection not available for this element? Is there some other alternative? The value does not need to be a secret but I need to have it in one place and not edit hundreds of YAML files everytime the runner tag would change...
EDIT1:
I've tried, as GuiFalourd suggested, to create an environment variable at the workflow level which would hold the secret:
env:
RUNNER_LABEL: ${{ secrets.GH_RUNNER_TAG }}
jobs:
analyze:
name: Analyze
runs-on: $RUNNER_LABEL
And it doesn't work. The action gets stuck. I tried using:
$RUNNER_LABEL -> gets stuck
"$RUNNER_LABEL" -> gets stuck, too
${{ env.RUNNER_LABEL }} -> action does not start, outputs error:
The workflow is not valid. .github/workflows/action.yml (Line: 14, Col: 14): Unrecognized named-value: 'env'. Located at position 1 within expression: env.RUNNER_LABEL
Furthermore, I've checked that the env var is properly assigned, by placing a valid, hard-coded value for runs-on and setting first step as:
steps:
- name: Test
run: echo "$RUNNER_LABEL"
which produces "***" - proof that a secret has been output and redacted automatically by GitHub.

This is achievable using Reusable Workflow by configuring the "called" workflow to accept inputs from the caller.
The main pipeline which we can name it as "process" will use a shared codebase/pipeline lets call it "common" which can accept inputs, one of these inputs can be the runs-on value.
For example
# common.yml
name: parameterized job
on:
workflow_call:
inputs:
runner:
required: true
type: string
jobs:
common:
name: Common
runs-on: ${{ inputs.runner }}
steps:
- run: echo "Hello World"
# process.yml
name: process
on:
push:
jobs:
process:
uses: username/repo/.github/workflows/common.yml#branch
with:
runner: machine_with_specific_label # using ${{ env.MY_RUNNER_LABEL }} is possible as well

Related

Sum expression in github action run-name

I am trying to set dynamic name to workflow run that has a number. The number should be calculated using an expression.
run-name: Perform Operation with ID [SOME_GITHUB_VAR + 1]
Is there a way to perform the above sum operation?
Thanks,
The run-name can include some expressions and can reference only the github and inputs contexts:
run-name: Deploy to ${{ inputs.deploy_target }} by #${{ github.actor }}
Unfortunately, GitHub Actions don't support math operations inside expressions. So, it's impossible to perform such a calculation for the value of the run-name option.
It's possible for the name value for a job step. You can add up these two numbers in a script and then set it as an environment variable.
Example:
name: 'Sum expression'
on:
workflow_dispatch:
jobs:
test:
runs-on: ubuntu-latest
steps:
- name: run number with offset
env:
NUM: ${{ github.run_number }}
run: |
echo "GITHUB_RUN_NUMBER_WITH_OFFSET=$(($NUM + 200))" >> $GITHUB_ENV
- name: 'Perform Operation with ID ${{ env.GITHUB_RUN_NUMBER_WITH_OFFSET }}'
run: echo "Success!"
Result:
For more details, read the Setting an environment variable article.
Reference: How to add two numbers.

Github Actions get output fom one job to matrix job

all.
I have this pipeline. Where I just basically want to get changed filed dirs and run helm push command on that dirs.
name: Helm Package and Push
on:
push:
branches:
- chartChange
paths:
- 'charts/**'
jobs:
detect_changes:
runs-on: ubuntu-latest
outputs:
list: ${{ steps.changed-files-specific.outputs.all_changed_files }}
steps:
- name: Checkout
uses: actions/checkout#v2
- name: Get changed files in the docs folder
id: changed-files-specific
uses: tj-actions/changed-files#v35
with:
json: true
files: charts/** # Alternatively using: `docs/**` or `docs`
- name: Run bash script
run: |
for i in ${{ steps.changed-files-specific.outputs.all_changed_files }}; do echo ${i%/*}; done
Test_from_first_job:
needs: detect_changes
runs-on: ubuntu-latest
strategy:
matrix:
folders: ${{fromJson(needs.detect_changes.outputs.list)}}
steps:
- name: Checkout
uses: actions/checkout#v2
- name: Run test step
run: |
echo ${{matrix.folders}}
But however my jobs failing with this error:
Error when evaluating 'strategy' for job 'Test_from_first_job'. .github/workflows/cm-
push.yml (Line: 37, Col: 22): Error parsing fromJson,.github/workflows/cm-push.yml
(Line: 37, Col: 22): Unexpected character encountered while parsing value: \. Path '',
line 1, position 1.,.github/workflows/cm-push.yml (Line: 37, Col: 22): Unexpected type
of value '', expected type: Sequence.
Any ideas how can i make it work ?
REgards
The matrix must be able to expand at the time the template is parsed and it can't rely on any runtime-defined values.
You'll need to rely on a script to trigger another workflow and pass the value in as a parameter. The parameter value IS available at parse time, so that can be used to spin up the matrix.
Or you must loop through the values using a runtime solution, such as a script block and a for-loop.

GitHub Actions - Reuse outputs from other reusable workflows

I'm not sure if it's possible, but I'm attempting to use outputs from one reusable workflow, in another, that's part of the same caller workflow. For reference, please see the configs below:
Caller Workflow:
jobs:
call-workflow-calver:
uses: ./.github/workflows/called-workflow1.yaml
secrets: inherit
call-workflow-echo:
needs: call-workflow-calver
uses: ./.github/workflows/called-workflow2.yaml
secrets: inherit
Job for creating the CalVer tag (it outputs as $VERSION as part of the action)
Called Workflow 1:
...
jobs:
calver:
name: Create CalVer tag
...
steps:
- name: Calver tag
uses: StephaneBour/actions-calver#1.4.4
if: ${{ github.ref == 'refs/heads/main' }}
id: calVer
with:
date_format: "%Y-%m-%d"
release: ${{ github.event_name == 'push' && github.ref == 'refs/heads/main' }}
...
Trying to use the CalVer $VERSION output from another caller workflow
Called Workflow 2:
...
- name: Echo
run: |
echo ${{needs.calver.outputs.VERSION}}
...
The basic concept is I'm trying to use outputs from one reusable workflow in another for setting CalVer versions in workflow 1, calling that in workflow 2 so I can set it as an image version. I will eventually use in a 3rd reusable workflow to deploy said image. If this is possible, that would be great!
Hopefully, this all makes a bit of sense, but if anything needs clarifying, please do let me know!
Many thanks in advance!
According to the official documentation, you can now declare outputs to reusable workflows.
These work just like job outputs and are available via needs.<reusable>.outputs.<output> format once you declare the output.
Example
1. Reusable workflow configuration:
name: Reusable workflow
on:
workflow_call:
# Map the workflow outputs to job outputs
outputs:
firstword:
description: "The first output string"
value: ${{ jobs.example_job.outputs.output1 }}
secondword:
description: "The second output string"
value: ${{ jobs.example_job.outputs.output2 }}
jobs:
example_job:
name: Generate output
runs-on: ubuntu-latest
# Map the job outputs to step outputs
outputs:
output1: ${{ steps.step1.outputs.firstword }}
output2: ${{ steps.step2.outputs.secondword }}
steps:
- id: step1
run: echo "::set-output name=firstword::hello"
- id: step2
run: echo "::set-output name=secondword::world"
2. Workflow using the reusable:
name: Call a reusable workflow and use its outputs
on:
workflow_dispatch:
jobs:
job1:
uses: octo-org/example-repo/.github/workflows/called-workflow.yml#v1
job2:
runs-on: ubuntu-latest
needs: job1
steps:
- run: echo ${{ needs.job1.outputs.firstword }} ${{ needs.job1.outputs.secondword }}
Note that if a reusable workflow that sets an output is executed with a matrix strategy, the output will be the output set by the last successful completing reusable workflow of the matrix which actually sets a value. That means if the last successful completing reusable workflow sets an empty string for its output, and the second last successful completing reusable workflow sets an actual value for its output, the output will contain the value of the second last completing reusable workflow.
I used this workflow as example here if you want to check the logs of the workflow run.

GitHub Action workflow_call does not use up to date input values

I am encountering a strange behaviour with GitHub Action workflow_call.
Basically, everything works fine after the initial setup, but when I edit the input parameters of the workflow files it does not use the updated values. Hard to explain in words, so let me show you an example.
Consider this basic setup:
File: start-pipeline.yml
name: Start Pipeline
on:
workflow_dispatch:
inputs:
release_type:
required: true
type: choice
options:
- patch
- minor
- major
jobs:
call-sub-workflow:
uses: codezalot/workflow-call-issue/.github/workflows/sub-workflow.yml#main
with:
release_type: ${{ github.event.inputs.release_type }}
File: sub-workflow.yml
name: Sub Workflow
on:
workflow_call:
inputs:
release_type:
type: string
required: true
jobs:
my-job:
runs-on: ubuntu-latest
steps:
- name: Print Inputs
run: |
echo "Release Type: ${{ github.event.inputs.release_type }}"
I then start the pipeline with the value patch and the sub-workflow prints the input just fine:
I then update the workflow files and add an additional input value like this:
File: start-pipeline.yml
...
jobs:
call-sub-workflow:
uses: codezalot/workflow-call-issue/.github/workflows/sub-workflow.yml#main
with:
release_type: ${{ github.event.inputs.release_type }}
some_other_value: ${{ github.event.inputs.release_type }}
File: sub-workflow.yml
...
inputs:
...
some_other_value:
type: string
required: true
jobs:
...
run: |
echo "Release Type: ${{ github.event.inputs.release_type }}"
echo "Release Type: ${{ github.event.inputs.some_other_value }}"
I then run the pipeline again, once with release_type patch, once with minor.
The value some_other_value is missing however, see:
.
I have been playing around with this issue for hours, but I do not understand what is going wrong. For it does seem to be using the latest SHA versions of the files, just the input parameters cause an issue.
Does anyone know what's going on here?
According to the documentation: "When a workflow is triggered with the workflow_call event, the event payload in the called workflow is the same event payload from the calling workflow." In other words, the event parameter in the github context is the same for the called workflow as for the original workflow.
Consequently, there is no github.event.input.some_other_value parameter since the original workflow did not have this input. For inputs defined in a reusable workflow you can use the inputs context which contains the input properties passed to the reusable workflow.
To sum up, here is a working sub-workflow.yml file:
name: Sub Workflow
on:
workflow_call:
inputs:
release_type:
type: string
required: true
some_other_value:
type: string
required: true
jobs:
my-job:
runs-on: ubuntu-latest
steps:
- name: Print Inputs
run: |
echo "Release Type: ${{ inputs.release_type }}"
echo "Release Type: ${{ inputs.some_other_value }}"
Result:

How can I invoke a YAML pipeline that has both variables and runtime parameters?

I have a scenario where I need to have both:
runtime parameters, so that the pipeline can be triggered manually from the UI, where users triggering it can choose from a predefined set of options (defined in YAML)
variables, so that the pipeline can be invoked via REST APIs
Regarding runtime parameters, I was able to create the following sample pipeline:
parameters:
- name: image
displayName: Pool Image
type: string
default: ubuntu-latest
values:
- windows-latest
- ubuntu-latest
trigger: none
stages:
- stage: A
jobs:
- job: A
steps:
- pwsh: |
echo "This should be triggering against image: $MY_IMAGE_NAME"
env:
MY_IMAGE_NAME: ${{ parameters.image }}
When I run it, I can see the dropdown list where I can choose the image name and it is reflected in the output message of the PowerShell script.
Regarding variables, I have defined one called "image" here (notice the value is empty):
The idea now is to invoke the pipeline from REST APIs and have the image name replaced by the value coming from the variable:
{
"definition": {
"id": 1
},
"sourceBranch": "master",
"parameters": "{\"image\": \"windows-latest\" }"
}
In order to make the step print the value I'm passing here, I need to correct the environment variable in some way. I thought it would be sufficient to write something like:
env:
MY_IMAGE_NAME: ${{ coalesce(variables.image, parameters.image) }}
That's because I want to give the priority to the variables, then to parameters, so that in case none is specified, I always have a default value the pipeline can use.
However, this approach doesn't work, probably because we're dealing with different expansion times for variables, but I don't really know what I should be writing instead (if there is a viable option, of course).
What I also tried is:
env:
MY_IMAGE_NAME: ${{ coalesce($(image), parameters.image) }}
MY_IMAGE_NAME: ${{ coalesce('$(image)', parameters.image) }}
MY_IMAGE_NAME: $[ coalesce(variables.image, parameters.image) ]
MY_IMAGE_NAME: $[ coalesce($(image), parameters.image) ]
None of those are working, so I suspect this may not be feasible at all.
There is a workaround that I'm currently thinking of, which is to create two different pipelines so that those can be invoked independently, but while this is quite easy for me to accomplish, given I'm using a lot of templates, I don't find it the right way to proceed, so I'm open to any suggestion.
I tested and found you might need to define a variable and assign the parameter's value to it (eg. Mimage: ${{parameters.image}}). And define another variable(eg. Vimage) and assign $[coalesce(variables.image, variables.Vimage)] to it. Then refer to $(Vimage) in the env field of powershell task. Please check out below yaml.
parameters:
- name: image
displayName: Pool Image
type: string
default: ubuntu-latest
values:
- windows-latest
- ubuntu-latest
trigger: none
stages:
- stage: A
jobs:
- job: A
variables:
Mimage: ${{parameters.image}}
Vimage: $[coalesce(variables.image, variables.Mimage)]
steps:
- pwsh: |
echo "This should be triggering against image: $env:MY_IMAGE_NAME"
env:
MY_IMAGE_NAME: $(Vimage)
Env field of powershell task is usually for mapping secret variables. You can directly refer to $(Vimage) in the powershell script: echo "This should be triggering against image: $(Vimage).
Note: To queue a build via REST API with provided parameters, you need to check Let users override this value when running this pipeline to make the varilabe to be settable at queue time.
Update:
You can try passing the variables to the parameters of the template to make the parameters for template dynamic. Please check below simple yaml.
jobs:
- template: template.yaml
parameters:
MTimage: ${{parameters.image}}
VTimage: $(Vimage)
template.yaml:
parameters:
MTimage:
VTimage:
jobs:
- job: buildjob
steps:
- powershell: |
echo "${{parameters.VTimage}}"
echo "${{parameters.MTimage}}"