github action expressions - split string - github

I'm trying to use a part of current branch name as a tag for publishing docker image in CI pipeline. The problem is that in my org, there is a convention to name branches like feature/foo, and docker tags can't contain slashes. So I want to split the string on '/' symbol and take the last part.
The full branch name, with slashes, should be in variable github.ref_name. Here is the expression I'm trying to evaluate: ${{ github.ref_name.split('/')[-1] }}.
I get this error: Unable to interpolate expression 'format('{0}', github.ref_name.split('/')[-1])': Failed to parse: unexpected token "(" while parsing arguments of function call. expecting ",", ")"
What are the options for manipulating strings in github actions expression? I didn't find it in docs https://docs.github.com/en/actions/learn-github-actions/expressions

IMPROVED:
This technique doesn't need to bring in anything from the actions marketplace.
Add this to steps: in an action:
- name: Split branch name
env:
BRANCH: ${{ github.ref_name }}
id: split
run: echo "::set-output name=fragment::${BRANCH##*/}"
It captures the output of a shell command. The only necessary actions are setting github.ref_name to an environmental variable and using parameter expansion to get the part of the branch name you want.
## greedily removes the subsequent pattern off the front
*/ matches anything followed by a /
So, ##*/ removes everything up to and including the final forward slash.
Because id is split, and name=fragment, you then reference your split string with steps.split.outputs.fragment. For example,
# Another step
- name: Test variable
run: |
echo ${{ steps.split.outputs.fragment }}
Other parameter expansion features would be useful here, like #, %, and %% in addition to ##.
220829: Final update
Originally I spawned another process with $(echo ${BRANCH##*/}) but that is unnecessary. The variable can be directly referenced.
ORIGINAL:
I'm doing something essentially the same and using an action was the easiest temporary workaround, however I also would have preferred something built-in like a string-splitting expression. For now here is what worked.
- uses: jungwinter/split#master
id: branch
with:
msg: ${{ github.ref_name }}
separator: "/"
maxsplit: -1
From its repo:
Outputs
_0, _1, ..., _n: Each result of a split
According to metadata syntax of outputs, it has _ prefix
Currently, support only 100 splits
length: Length of the splits
So, then I can reference each portion of the string with steps.branch.outputs._n, with branch being the id and n being the split index.
I was looking for the first instead of the last value in the branch name, so I could set maxsplit to 1 and just take _0. In your case you might need to do some magic with checking length and choosing the index from there.

I ended up using bash to do the split and publishing it as job output. In the retrospect should have went for just installing python (bash syntax is way more confusing and less maintainable).
build-docker:
container: docker:cli
outputs:
docker-tag: ${{steps.docker-tag.outputs.DOCKER_TAG}}
steps:
- name: add bash
run: |
apk update
apk upgrade
apk add bash
- name: determine docker tag by splitting branch name on slash
id: docker-tag
shell: bash
run: |
split=(${GITHUB_REF_NAME//\// })
index=$((${#split[#]}-1))
DOCKER_TAG=${split[$index]}
echo DOCKER_TAG=$DOCKER_TAG >> $GITHUB_ENV
echo "::set-output name=DOCKER_TAG::$DOCKER_TAG"
- name: Build and push
uses: docker/build-push-action#v3
file: ./docker/ci/Dockerfile
push: true
tags: |
artifactory.com/repo/image:${{ env.DOCKER_TAG }}
test-job:
needs: build-docker
runs-on: [self-hosted, Linux, docker]
container:
image: artifactory.com/repo/image:${{ needs.build-docker.outputs.docker-tag }}

Related

How to use env variable as default value for input in github actions?

I have a github action that has an input which should have a default value based on an env.variable. Because github actions doesn't support env variables in the default field, I was wondering if I could reassign the inputs.variable in the steps portion of my action.yml file.
Here's what I've tried so far:
Doesn't work:
...
inputs:
...
mono-build-tag:
description: Release tag to download from the mono-build-repo
# Github Actions complains that env is being used here
default: "${{ env.GODOT_MONO_BUILD_TAG }}"
runs:
using: "composite"
steps:
- name: Setup default inputs
run: |
if ${{ inputs.mono-build-repo == '' }}
...
Doesn't work:
...
inputs:
...
mono-build-tag:
description: Release tag to download from the mono-build-repo
default: ""
runs:
using: "composite"
steps:
- name: Setup default inputs
run: |
if ${{ inputs.mono-build-repo == '' }}; then
# How do I set inputs.mono-build-repo here???
fi
...
You could define the env variable as follow:
env:
GODOT_MONO_BUILD_TAG: ${{ github.event.inputs.mono-build-repo || "latest" }}
where latest should be the default value for the env var
If you were to use #Matteo's solution of creating an env variable with a default value I believe that default value would have to use single quotes as per the github actions expressions docs:
"You don't need to enclose strings in ${{ and }}. However, if you do, you must use single quotes (') around the string. To use a literal single quote, escape the literal single quote using an additional single quote (''). Wrapping with double quotes (") will throw an error."

Evaluating environment variables in github actions workflow

I'm positive that this is something easy but I'm not able to track down exactly what I'm looking for. I have a workflow that performs a build and creates an artifact. The artifact uses an environment variable in the filename. So far so good. Then when I try to pass this file name to S3 upload action, it isn't found because the environment variable isn't evaluated. Here is the relevant part of my workflow:
- name: Build project
run: ./build_project.sh
- run: ls -l "${GITHUB_WORKSPACE}/build/${FILE_NAME}.zip" # file exists in directory
- run: echo "${GITHUB_WORKSPACE}/build/${FILE_NAME}.zip" # echo returns the location properly
- uses: hkusu/s3-upload-action#v2
id: upload # specify some ID for use in subsequent steps
with:
aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
aws-region: "eu-west-2"
aws-bucket: ${{ secrets.AWS_BUCKET }}
file-path: "${GITHUB_WORKSPACE}/build/${FILE_NAME}.zip" # Error: file does not exist
output-file-url: "true" # specify true
- name: Show URL
run: echo '${{ steps.upload.outputs.file-url }}' # use this output
My actual question is how to replace "${GITHUB_WORKSPACE}/build/${FILE_NAME}.zip" with the file path and name when it actually runs the workflow. Also, I have tried a few of different combinations of things - no quotes, no curly braces, neither, both.
Since these parameters are passed through to the s3-upload-action, the behaviour depends on whether the action expands shell parameters or not, but the input value will be literally
${GITHUB_WORKSPACE}/build/${FILE_NAME}.zip
i.e., unexpanded.
You can use expressions to work around this:
file-path: ${{ github.workspace }}/build/${{ env.FILE_NAME }}.zip
You maybe have assumed that environment variables expand everywhere as they do when evaluated by a shell, such as in a run: step, but they don't.

GitHub Action: Pass Environment Variable to into Action using PowerShell

I am using trying to build a workflow that will run in PowerShell. I am setting an environment for my branch name to use in a step for checkout of a different repository.
run: |
$branchName = $Env:GITHUB_REF -replace "refs/heads/", ""
echo "CURRENT_BRANCH=${branchName}" | Out-File -FilePath $Env:GITHUB_ENV -Encoding utf8 -Append
In a later step, I'm trying to pass in the variable:
- name: Checkout repo
uses: actions/checkout#v2
with:
repository: 'MyOrg/MyRepo'
ref: ${env:CURRENT_BRANCH}
I've tried different formats involving curly brackets, but I keep getting output from the build that shows that exact text as the path. I'm not sure sure how to get it to evaluate.
When I do ${{ env:CURRENT_BRANCH }} I received the following error:
The workflow is not valid.
.github/workflows/publish.yml (Line: 54, Col: 14):
Unexpected symbol: 'env:CURRENT_BRANCH'. Located at position 1
within expression: env:CURRENT_BRANCH
To reference a variable from the given context (env in this case) in the GitHub Actions workflow we have to use a dot (.) character, but you used a colon (:). To fix the error above the workflow should be adjusted:
- name: Checkout repo
uses: actions/checkout#v2
with:
repository: 'MyOrg/MyRepo'
ref: ${{env.CURRENT_BRANCH}}
Additionally, you don't have to detect the current branch on and pass it to the checkout action. actions/checkout#v2 will use the current branch by default. So you only have to have:
- name: Checkout repo
uses: actions/checkout#v2
- name: Next Step
[...]
One solution I found elsewhere and will post it here as an option, although I would like to know if using Environment Variables is possibly in my scenario.
The solution is to use Outputs from a Step
- name: Output Variables
id: SetVariables
run: |
$branchName = $Env:GITHUB_REF -replace "refs/heads/", ""
echo "Branch: ${branchName}"
echo "::set-output name=branch::${branchName}"
- name: Checkout Repo 2
uses: actions/checkout#v2
with:
repository: 'MyOrg/MyRepo'
ref: ${{ steps.SetVariables.outputs.branch }}
This is a working method now, more info here:
echo "action_state=yellow" >> $GITHUB_ENV
The one below no longer works for me. A bit of research showed it is due to a security issue.
echo "::set-output name=action_state::yellow"

How to get pull request number within GitHub Actions workflow

I want to access the Pull Request number in a Github Actions workflow. I can access the GITHUB_REF environment variable that is available. Although on a Pull Request action it has the value: "refs/pull/125/merge". I need to extract just the "125".
I have found a similar post here that shows how to get the current branch using this variable. Although in this case, what I am parsing is different and I have been unable to isolate the Pull Request number.
I have tried using {GITHUB_REF##*/} which resolves to "merge"
I have also tried {GITHUB_REF#*/} which resolves to "pull/125/merge"
I only need the Pull Request number (which in my example is 125)
Although it is already answered, the easiest way I found is using the github context. The following example shows how to set it to an environment variable.
env:
PR_NUMBER: ${{ github.event.number }}
An alternative if you are trying to figure out which PR a commit is linked to on a push instead of a pull_request event is to use the gh CLI which is included in the standard GitHub Action images.
For example:
- name: Get Pull Request Number
id: pr
run: echo "::set-output name=pull_request_number::$(gh pr view --json number -q .number || echo "")"
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
Be sure to add pull_request: read permissions on the job as well.
Then in following steps, you can access it with the variable,
${{ steps.pr.outputs.pull_request_number }}
While the answer by #Samira worked correctly. I found out that there is a new way to do this and wanted to share it with anyone who might stumble upon this.
The solution is to add a stage at the beginning of your workflow which gets the PR number from the Github Token (event) and then set it as an environment variable for easy use throughout the rest of the workflow. Here is the code:
- name: Test
uses: actions/github-script#0.3.0
with:
github-token: ${{github.token}}
script: |
const core = require('#actions/core')
const prNumber = context.payload.number;
core.exportVariable('PULL_NUMBER', prNumber);
Now in any later stage, you can simply use $PULL_NUMBER to access the environment variable set before.
How about using awk to extract parts of GITHUB_REF instead of bash magick?
From awk manpage:
-F fs
--field-separator fs
Use fs for the input field separator (the value of the FS predefined variable).
As long you remember this, it's trivial to extract only part of variable you need. awk is available on all platforms, so step below will work everywhere:
- run: echo ::set-env name=PULL_NUMBER::$(echo "$GITHUB_REF" | awk -F / '{print $3}')
shell: bash
Just gonna drop what worked out for me
- id: find-pull-request
uses: jwalton/gh-find-current-pr#v1
with:
# Can be "open", "closed", or "all". Defaults to "open".
state: open
- name: create TODO/FIXME comment body
id: comment-body
run: |
yarn leasot '**/*.{js,ts,jsx,tsx}' --ignore 'node_modules/**/*' --exit-nicely --reporter markdown > TODO.md
body="$(sed 1,2d TODO.md)"
body="${body//'%'/'%25'}"
body="${body//$'\n'/'%0A'}"
body="${body//$'\r'/'%0D'}"
echo "::set-output name=body::$body"
- name: post TODO/FIXME comment to PR
uses: peter-evans/create-or-update-comment#v2
with:
issue-number: ${{ steps.find-pull-request.outputs.number }}
body: ${{ steps.comment-body.outputs.body }}
Here's a working snippet to get the issue number in both push and pull_request events within a GitHub Actions workflow by leveraging actions/github-script:
steps:
- uses: actions/github-script#v6
id: get_issue_number
with:
script: |
if (context.issue.number) {
// Return issue number if present
return context.issue.number;
} else {
// Otherwise return issue number from commit
return (
await github.rest.repos.listPullRequestsAssociatedWithCommit({
commit_sha: context.sha,
owner: context.repo.owner,
repo: context.repo.repo,
})
).data[0].number;
}
result-encoding: string
- name: Issue number
run: echo '${{steps.get_issue_number.outputs.result}}'
The script queries the list labels for an issue REST API endpoint via octokit/rest.js client.

Using output from a previous job in a new one in a GitHub Action

For (mainly) pedagogical reasons, I'm trying to run this workflow in GitHub actions:
name: "We 🎔 Perl"
on:
issues:
types: [opened, edited, milestoned]
jobs:
seasonal_greetings:
runs-on: windows-latest
steps:
- name: Maybe greet
id: maybe-greet
env:
HEY: "Hey you!"
GREETING: "Merry Xmas to you too!"
BODY: ${{ github.event.issue.body }}
run: |
$output=(perl -e 'print ($ENV{BODY} =~ /Merry/)?$ENV{GREETING}:$ENV{HEY};')
Write-Output "::set-output name=GREET::$output"
produce_comment:
name: Respond to issue
runs-on: ubuntu-latest
steps:
- name: Dump job context
env:
JOB_CONTEXT: ${{ jobs.maybe-greet.steps.id }}
run: echo "$JOB_CONTEXT"
I need two different jobs, since they use different context (operating systems), but I need to get the output of a step in the first job to the second job. I am trying with several combinations of the jobs context as found here but there does not seem to be any way to do that. Apparently, jobs is just the name of a YAML variable that does not really have a context, and the context job contains just the success or failure. Any idea?
Check the "GitHub Actions: New workflow features" from April 2020, which could help in your case (to reference step outputs from previous jobs)
Job outputs
You can specify a set of outputs that you want to pass to subsequent jobs and then access those values from your needs context.
See documentation:
jobs.<jobs_id>.outputs
A map of outputs for a job.
Job outputs are available to all downstream jobs that depend on this job.
For more information on defining job dependencies, see jobs.<job_id>.needs.
Job outputs are strings, and job outputs containing expressions are evaluated on the runner at the end of each job. Outputs containing secrets are redacted on the runner and not sent to GitHub Actions.
To use job outputs in a dependent job, you can use the needs context.
For more information, see "Context and expression syntax for GitHub Actions."
To use job outputs in a dependent job, you can use the needs context.
Example
jobs:
job1:
runs-on: ubuntu-latest
# Map a step output to a job output
outputs:
output1: ${{ steps.step1.outputs.test }}
output2: ${{ steps.step2.outputs.test }}
steps:
- id: step1
run: echo "test=hello" >> $GITHUB_OUTPUT
- id: step2
run: echo "test=world" >> $GITHUB_OUTPUT
job2:
runs-on: ubuntu-latest
needs: job1
steps:
- run: echo ${{needs.job1.outputs.output1}} ${{needs.job1.outputs.output2}}
Note the use of $GITHUB_OUTPUT, instead of the older ::set-output now (Oct. 2022) deprecated.
To avoid untrusted logged data to use set-state and set-output workflow commands without the intention of the workflow author we have introduced a new set of environment files to manage state and output.
Jesse Adelman adds in the comments:
This seems to not work well for anything beyond a static string.
How, for example, would I take a multiline text output of step (say, I'm running a pytest or similar) and use that output in another job?
either write the multi-line text to a file (jschmitter's comment)
or base64-encode the output and then decode it in the next job (Nate Karasch's comment)
Update: It's now possible to set job outputs that can be used to transfer string values to downstream jobs. See this answer.
What follows is the original answer. These techniques might still be useful for some use cases.
Write the data to file and use actions/upload-artifact and actions/download-artifact. A bit awkward, but it works.
Create a repository dispatch event and send the data to a second workflow. I prefer this method personally, but the downside is that it needs a repo scoped PAT.
Here is an example of how the second way could work. It uses repository-dispatch action.
name: "We 🎔 Perl"
on:
issues:
types: [opened, edited, milestoned]
jobs:
seasonal_greetings:
runs-on: windows-latest
steps:
- name: Maybe greet
id: maybe-greet
env:
HEY: "Hey you!"
GREETING: "Merry Xmas to you too!"
BODY: ${{ github.event.issue.body }}
run: |
$output=(perl -e 'print ($ENV{BODY} =~ /Merry/)?$ENV{GREETING}:$ENV{HEY};')
Write-Output "::set-output name=GREET::$output"
- name: Repository Dispatch
uses: peter-evans/repository-dispatch#v1
with:
token: ${{ secrets.REPO_ACCESS_TOKEN }}
event-type: my-event
client-payload: '{"greet": "${{ steps.maybe-greet.outputs.GREET }}"}'
This triggers a repository dispatch workflow in the same repository.
name: Repository Dispatch
on:
repository_dispatch:
types: [my-event]
jobs:
myEvent:
runs-on: ubuntu-latest
steps:
- run: echo ${{ github.event.client_payload.greet }}
In my case I wanted to pass an entire build/artifact, not just a string:
name: Build something on Ubuntu then use it on MacOS
on:
workflow_dispatch:
# Allows for manual build trigger
jobs:
buildUbuntuProject:
name: Builds the project on Ubuntu (Put your stuff here)
runs-on: ubuntu-latest
steps:
- uses: actions/checkout#v2
- uses: some/compile-action#v99
- uses: actions/upload-artifact#v2
# Upload the artifact so the MacOS runner do something with it
with:
name: CompiledProject
path: pathToCompiledProject
doSomethingOnMacOS:
name: Runs the program on MacOS or something
runs-on: macos-latest
needs: buildUbuntuProject # Needed so the job waits for the Ubuntu job to finish
steps:
- uses: actions/download-artifact#master
with:
name: CompiledProject
path: somewhereToPutItOnMacOSRunner
- run: ls somewhereToPutItOnMacOSRunner # See the artifact on the MacOS runner
It is possible to capture the entire output (and return code) of a command within a run step, which I've written up here to hopefully save someone else the headache. Fair warning, it requires a lot of shell trickery and a multiline run to ensure everything happens within a single shell instance.
In my case, I needed to invoke a script and capture the entirety of its stdout for use in a later step, as well as preserve its outcome for error checking:
# capture stdout from script
SCRIPT_OUTPUT=$(./do-something.sh)
# capture exit code as well
SCRIPT_RC=$?
# FYI, this would get stdout AND stderr
SCRIPT_ALL_OUTPUT=$(./do-something.sh 2>&1)
Since Github's job outputs only seem to be able to capture a single line of text, I also had to escape any newlines for the output:
echo "::set-output name=stdout::${SCRIPT_OUTPUT//$'\n'/\\n}"
Additionally, I needed to ultimately return the script's exit code to correctly indicate whether it failed. The whole shebang ends up looking like this:
- name: A run step with stdout as a captured output
id: myscript
run: |
# run in subshell, capturiing stdout to var
SCRIPT_OUTPUT=$(./do-something.sh)
# capture exit code too
SCRIPT_RC=$?
# print a single line output for github
echo "::set-output name=stdout::${SCRIPT_OUTPUT//$'\n'/\\n}"
# exit with the script status
exit $SCRIPT_RC
continue-on-error: true
- name: Add above outcome and output as an issue comment
uses: actions/github-script#v5
env:
STEP_OUTPUT: ${{ steps.myscript.outputs.stdout }}
with:
github-token: ${{ secrets.GITHUB_TOKEN }}
script: |
// indicates whather script succeeded or not
let comment = `Script finished with \`${{ steps.myscript.outcome }}\`\n`;
// adds stdout, unescaping newlines again to make it readable
comment += `<details><summary>Show Output</summary>
\`\`\`
${process.env.STEP_OUTPUT.replace(/\\n/g, '\n')}
\`\`\`
</details>`;
// add the whole damn thing as an issue comment
github.rest.issues.createComment({
issue_number: context.issue.number,
owner: context.repo.owner,
repo: context.repo.repo,
body: comment
})
Edit: there is also an action to accomplish this with much less bootstrapping, which I only just found.
2022 October update: GitHub is deprecating set-output and recommends to use GITHUB_OUTPUT instead. The syntax for defining the outputs and referencing them in other steps, jobs.
An example from the docs:
- name: Set color
id: random-color-generator
run: echo "SELECTED_COLOR=green" >> $GITHUB_OUTPUT
- name: Get color
run: echo "The selected color is ${{ steps.random-color-generator.outputs.SELECTED_COLOR }}"