Cannot retrieve and display Github env setup in previous step - github

I am trying to setup a variable in my CI pipeline that I will reuse later (eventually in another job, which I don't know if possible since I don't know if jobs shares variables.. but this is another problem). My pipeline is:
name: CI
on:
pull_request:
branches:
- main
jobs:
test-job:
runs-on: ubuntu-latest
name: test-job
steps:
- name: setup env variable
run: |
BRANCH_NAME=`echo "${{github.head_ref}}"'`
echo "BRANCH_NAME=$BRANCH_NAME" >> $GITHUB_ENV
echo ${{ env.BRANCH_NAME }}
that last echo doesn't show anything unfortunately. I am sure that BRANCH_NAME is correctly set because before pushing it into the $GITHUB_ENV" I did echo it and it contains data. Plus you can see the name of the branch in the console logs.
Console logs from Github are the following:
1. Run BRANCH_NAME=`echo "test_branch"'`
2. BRANCH_NAME=test_branch >> /home/runner/work/_temp/_runner_file_commands/set_env_9eeeac39-f573-4079-ba62-e1c2019f7aff
3.
So, that final echo ${{ env.BRANCH_NAME }} gives no result. What am I missing?
UPDATE:
As suggested in the comments, I started using workflow variables, in such a way that they are available throughout all the jobs.
The initial setup becomes:
name: CI
on:
pull_request:
branches:
- main
env:
BRANCH_NAME: ""
jobs:
...
I don't like the fact I need to give those variables an empty string value as placeholder and would have preferred declaring and assigning them in one of the jobs itself.. but still. So, now variables are declared before the jobs section, how do I assign a value to them in one of my steps? Meaning, I need to replace the
echo "BRANCH_NAME=$BRANCH_NAME" >> $GITHUB_ENV
tried already
echo "BRANCH_NAME=$BRANCH_NAME >> ${{ env.BRANCH_NAME }}
or
${{ env.BRANCH_NAME }}=$BRANCH_NAME
but both ways don't work.

Related

How can I run a GitHub Actions job based on a complex condition (specific label removed)?

I have two reusable workflows to deploy and destroy GCP resources, which I call from one workflow based on different conditions.
One workflow creates infra and is triggered when the label preview is added to a PR:
on:
pull_request:
types: [opened, reopened, labeled]
jobs:
create-infrastructure:
if: ${{ contains( github.event.pull_request.labels.*.name, 'preview') }}
# Call to a reusable workflow here
The second workflow I need to trigger when the PR is closed or when a specific label is removed; I tried this:
on:
pull_request:
types: [ closed, unlabeled ]
jobs:
destroy_preview:
if: ${{ contains( github.event.pull_request.labels.*.name, 'preview') }}
uses: myrepo/.github/workflows/preview-app-destroy.yml#v0.3.6
with:
project_id: xxx
I don't know how to define unlabeled for a specific label. It would be great if someone has any idea.
The pull request webhook payload doesn't contain the removed label, as far as I can tell, but you can fetch the list of issue events (which work for pull requests, too), filter by unlabeled events, and then look at the label name of the last one.
Using the GitHub CLI in a run step, that might look something like this:
name: Preview removed workflow
on:
pull_request:
types:
- unlabeled
jobs:
destroy_preview:
runs-on: ubuntu-20.04
steps:
- name: Check if "preview" was removed
env:
GITHUB_TOKEN: ${{ github.token }}
run: |
pr=${{ github.event.number }}
label=$(gh api "repos/$GITHUB_REPOSITORY/issues/$pr/events" \
--jq 'map(select(.event == "unlabeled"))[-1].label.name')
if [[ $label == 'preview' ]]; then
echo "The 'preview' label has been removed"
fi
where you'd replace the echo with your infrastructure commands.
Now, if you want to call a reusable workflow when that specific label is removed, you have to somehow find a way to add a condition to the job where the reusable workflow is called.
One option is to make two jobs, one to check the condition and setting the result as a job output. The other job is set up as depending on the first one, and its if condition checks if the output was set to true.
This would look something like this (omitting the name and trigger, as they are identical to above):
jobs:
checklabel:
runs-on: ubuntu-20.04
outputs:
waspreview: ${{ steps.check.outputs.preview }}
steps:
- name: Check if "preview" was removed
id: check
env:
GITHUB_TOKEN: ${{ github.token }}
run: |
pr=${{ github.event.number }}
label=$(gh api "repos/$GITHUB_REPOSITORY/issues/$pr/events" \
--jq 'map(select(.event == "unlabeled"))[-1].label.name')
if [[ $label == 'preview' ]]; then
echo "::set-output name=preview::true"
fi
destroy_preview:
needs: checklabel
if: needs.checklabel.outputs.waspreview
uses: myrepo/.github/workflows/preview-app-destroy.yml#v0.3.6
with:
project_id: xxx

GitHub actions: default branch variable

Is there any smart way to determine the default branch in GitHub actions?
Now I need to write something like:
on:
push:
branches:
- master
is there a way to write something like the code below?
on:
push:
branches:
- $default-branch
I tried to google but found nothing
I accidentally found a really nice way to solve this. That evalutes to the branch name, e.g. master.
${{ github.event.repository.default_branch }}
Also, found out the hard way that that always() is side-effecting: my job was getting skipped if always() was not called even though the other clause was true.
This works to run a job only when running on default branch
if: ${{ always() && format('refs/heads/{0}', ) == github.ref }}
$default-branch can be used in Workflow templates, but not in Workflows. The branch will become hard-coded in the Workflow upon initialization, and will have to be manually maintained. [1]
Blog post: https://github.blog/changelog/2020-07-22-github-actions-better-support-for-alternative-default-branch-names/
- if: github.ref == format('refs/heads/{0}', github.event.repository.default_branch)
run: echo "On the default branch"
- if: github.ref != format('refs/heads/{0}', github.event.repository.default_branch)
run: echo "Not on the default branch"
This is not possible at the moment. Please check this topic on github community
You simply can reach variable at this level
The workflow is not valid. .github/workflows/so-004-variables-in-trigger.yaml (Line: 7, Col: 9): Unrecognized named-value: 'env'. Located at position 1 within expression: env.default-branch
You may consider addition filterint based on the branch name like here but at the moment you can't do what you want.
Add this step to your job:
- name: Determine default branch
run: |
DEFAULT_BRANCH=$(git remote show origin | awk '/HEAD branch/ {print $NF}')
echo "default_branch=$DEFAULT_BRANCH" >> $GITHUB_ENV
echo "default_branch_ref=refs/heads/$DEFAULT_BRANCH" >> $GITHUB_ENV
That will add a default_branch and a default_branch_ref variable to the env enivronment variables.
You can then access the default branch name with ${{ env.default_branch }} in subsequent steps.
The default_branch_ref variable is useful for directly comparing against github.ref to determine whether you are on the default branch.
This method uses the current method of setting environment variables to use in later steps [1] and JoeLinux's method for determining the default branch name [2].
Full example workflow:
name: ci
on: [push, pull_request]
jobs:
ci:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout#v2
- name: Determine default branch
run: |
DEFAULT_BRANCH=$(git remote show origin | awk '/HEAD branch/ {print $NF}')
echo "default_branch=$DEFAULT_BRANCH" >> $GITHUB_ENV
echo "default_branch_ref=refs/heads/$DEFAULT_BRANCH" >> $GITHUB_ENV
- name: debug
run: echo ${{ env.default_branch }}
- name: Deploy
if: github.ref == env.default_branch_ref
run: echo "Run!"
You can use $default-branch in a template, and then when that template is rendered into a new repo, it will be replaced with the (then) default branch name for the repo, but that is a very limited use case and still does not help you when the name of the default branch changes. The best I have come up with is to list the all the default branch names in the organization, like this:
on:
push:
branches:
- master
- main
- root
- default
- production
and then you can either trust that the repos will not have non-default branches with those names, or start the jobs and then filter them by adding an if condition like
if: github.event.push.ref == format('refs/heads/{}', github.event.repository.default_branch)
Side note
For most events
${{ github.event.repository.default_branch }}
is available and works fine, but not when running schedule events via cron. When github.event_name == "schedule" the only element in github.event is schedule (the cron string that triggered the run).
When running inside a GitHub action on at GitHub runner with gh available, this more reliably gets you the default branch name:
gh repo view --json defaultBranchRef --jq .defaultBranchRef.name
However, this does not help the OP when you want to make the default branch the target that triggers the run.
Hopefully, there will be a better way to do this in the future. Until then, you can use the GitHub API and save the result in a named step output.
e.g.
- name: Extract default branch name
shell: bash
run: |
owner="my-org"
repo="repo_x"
branch=$(curl -L -H 'authorization: Bearer ${{ secrets.GITHUB_TOKEN }}' \
https://api.github.com/repos/${owner}/${repo} \
| jq .default_branch)
echo "##[set-output name=default_branch;]$(echo ${branch})"
id: repo_x
...
${{ steps.repo_x.outputs.default_branch }}

Condititionally set Environmental Variable in GitHub Actions Workflows [duplicate]

I currently have two GitHub actions workflow files that are pretty much identical, only one is configured to react to push/pull_requests on branch master the other on production.
The staging workflow starts like this:
env:
GCLOUD_PROJECT: my-project-stg
VERCEL_TARGET: staging
on:
push:
branches: [ master ]
pull_request:
branches: [ master ]
The production workflow starts like:
env:
GCLOUD_PROJECT: my-project-prd
VERCEL_TARGET: production
on:
push:
branches: [ production ]
pull_request:
branches: [ production ]
The rest of the workflow files is the same, so this is clearly not very DRY.
I would like to have 1 single workflow file and somehow switch between two sets of variables based on the branch name. Is there a way to achieve this or am I maybe approach this from the wrong angle?
If it were possible to extend both workflow files on a shared base definition that would also solve it I guess.
It is not possible to set workflow-level environment variables from a job. Each job runs in its own machine and setting an env variable there only affects all of the later steps in that job.
There are currently two ways to share data between jobs; either you create an artifact and use that, or you declare and set job outputs. The latter works for string values.
Here's an example:
name: "Main"
on:
push:
branches:
- master
- production
pull_request:
branches:
- master
- production
jobs:
init:
runs-on: ubuntu-latest
outputs:
gcloud_project: ${{ steps.setvars.outputs.gcloud_project }}
phase: ${{ steps.setvars.outputs.phase }}
steps:
- name: Cancel previous workflow
uses: styfle/cancel-workflow-action#0.4.0
with:
access_token: ${{ github.token }}
- name: Set variables
id: setvars
run: |
if [[ "${{github.base_ref}}" == "master" || "${{github.ref}}" == "refs/heads/master" ]]; then
echo "::set-output name=gcloud_project::my-project-dev"
echo "::set-output name=phase::staging"
fi
if [[ "${{github.base_ref}}" == "production" || "${{github.ref}}" == "refs/heads/production" ]]; then
echo "::set-output name=gcloud_project::my-project-prd"
echo "::set-output name=phase::production"
fi
print:
runs-on: ubuntu-latest
needs: init
steps:
- name: Print
run: echo "gcloud_project=${{needs.init.outputs.gcloud_project}}"
You can drop the global env statement, combine the event triggers to
on:
push:
branches:
- master
- production
pull_request:
branches:
- master
- production
and then add a first step that checks which branch the workflow is running on and set the environment there:
- name: Set environment for branch
run: |
if [[ $GITHUB_REF == 'refs/heads/master' ]]; then
echo "GLCOUD_PROJECT=my-project-stg" >> "$GITHUB_ENV"
echo "VERCEL_TARGET=staging" >> "$GITHUB_ENV"
else
echo "GLCOUD_PROJECT=my-project-prd" >> "$GITHUB_ENV"
echo "VERCEL_TARGET=production" >> "$GITHUB_ENV"
fi
Things could have changed but for anyone still searching for an answer, there is a way to set a variabe visible for the entire workflow.
jobs:
weekday_job:
runs-on: ubuntu-latest
env:
DAY_OF_WEEK: Mon
Then access by jobs.<job_id>.steps[*].env
More info in the docs
There's now a better way of achieving this using reusable workflows:
https://docs.github.com/en/actions/learn-github-actions/reusing-workflows

How to set workflow env variables depending on branch

I currently have two GitHub actions workflow files that are pretty much identical, only one is configured to react to push/pull_requests on branch master the other on production.
The staging workflow starts like this:
env:
GCLOUD_PROJECT: my-project-stg
VERCEL_TARGET: staging
on:
push:
branches: [ master ]
pull_request:
branches: [ master ]
The production workflow starts like:
env:
GCLOUD_PROJECT: my-project-prd
VERCEL_TARGET: production
on:
push:
branches: [ production ]
pull_request:
branches: [ production ]
The rest of the workflow files is the same, so this is clearly not very DRY.
I would like to have 1 single workflow file and somehow switch between two sets of variables based on the branch name. Is there a way to achieve this or am I maybe approach this from the wrong angle?
If it were possible to extend both workflow files on a shared base definition that would also solve it I guess.
It is not possible to set workflow-level environment variables from a job. Each job runs in its own machine and setting an env variable there only affects all of the later steps in that job.
There are currently two ways to share data between jobs; either you create an artifact and use that, or you declare and set job outputs. The latter works for string values.
Here's an example:
name: "Main"
on:
push:
branches:
- master
- production
pull_request:
branches:
- master
- production
jobs:
init:
runs-on: ubuntu-latest
outputs:
gcloud_project: ${{ steps.setvars.outputs.gcloud_project }}
phase: ${{ steps.setvars.outputs.phase }}
steps:
- name: Cancel previous workflow
uses: styfle/cancel-workflow-action#0.4.0
with:
access_token: ${{ github.token }}
- name: Set variables
id: setvars
run: |
if [[ "${{github.base_ref}}" == "master" || "${{github.ref}}" == "refs/heads/master" ]]; then
echo "::set-output name=gcloud_project::my-project-dev"
echo "::set-output name=phase::staging"
fi
if [[ "${{github.base_ref}}" == "production" || "${{github.ref}}" == "refs/heads/production" ]]; then
echo "::set-output name=gcloud_project::my-project-prd"
echo "::set-output name=phase::production"
fi
print:
runs-on: ubuntu-latest
needs: init
steps:
- name: Print
run: echo "gcloud_project=${{needs.init.outputs.gcloud_project}}"
You can drop the global env statement, combine the event triggers to
on:
push:
branches:
- master
- production
pull_request:
branches:
- master
- production
and then add a first step that checks which branch the workflow is running on and set the environment there:
- name: Set environment for branch
run: |
if [[ $GITHUB_REF == 'refs/heads/master' ]]; then
echo "GLCOUD_PROJECT=my-project-stg" >> "$GITHUB_ENV"
echo "VERCEL_TARGET=staging" >> "$GITHUB_ENV"
else
echo "GLCOUD_PROJECT=my-project-prd" >> "$GITHUB_ENV"
echo "VERCEL_TARGET=production" >> "$GITHUB_ENV"
fi
Things could have changed but for anyone still searching for an answer, there is a way to set a variabe visible for the entire workflow.
jobs:
weekday_job:
runs-on: ubuntu-latest
env:
DAY_OF_WEEK: Mon
Then access by jobs.<job_id>.steps[*].env
More info in the docs
There's now a better way of achieving this using reusable workflows:
https://docs.github.com/en/actions/learn-github-actions/reusing-workflows

Using output from a previous job in a new one in a GitHub Action

For (mainly) pedagogical reasons, I'm trying to run this workflow in GitHub actions:
name: "We 🎔 Perl"
on:
issues:
types: [opened, edited, milestoned]
jobs:
seasonal_greetings:
runs-on: windows-latest
steps:
- name: Maybe greet
id: maybe-greet
env:
HEY: "Hey you!"
GREETING: "Merry Xmas to you too!"
BODY: ${{ github.event.issue.body }}
run: |
$output=(perl -e 'print ($ENV{BODY} =~ /Merry/)?$ENV{GREETING}:$ENV{HEY};')
Write-Output "::set-output name=GREET::$output"
produce_comment:
name: Respond to issue
runs-on: ubuntu-latest
steps:
- name: Dump job context
env:
JOB_CONTEXT: ${{ jobs.maybe-greet.steps.id }}
run: echo "$JOB_CONTEXT"
I need two different jobs, since they use different context (operating systems), but I need to get the output of a step in the first job to the second job. I am trying with several combinations of the jobs context as found here but there does not seem to be any way to do that. Apparently, jobs is just the name of a YAML variable that does not really have a context, and the context job contains just the success or failure. Any idea?
Check the "GitHub Actions: New workflow features" from April 2020, which could help in your case (to reference step outputs from previous jobs)
Job outputs
You can specify a set of outputs that you want to pass to subsequent jobs and then access those values from your needs context.
See documentation:
jobs.<jobs_id>.outputs
A map of outputs for a job.
Job outputs are available to all downstream jobs that depend on this job.
For more information on defining job dependencies, see jobs.<job_id>.needs.
Job outputs are strings, and job outputs containing expressions are evaluated on the runner at the end of each job. Outputs containing secrets are redacted on the runner and not sent to GitHub Actions.
To use job outputs in a dependent job, you can use the needs context.
For more information, see "Context and expression syntax for GitHub Actions."
To use job outputs in a dependent job, you can use the needs context.
Example
jobs:
job1:
runs-on: ubuntu-latest
# Map a step output to a job output
outputs:
output1: ${{ steps.step1.outputs.test }}
output2: ${{ steps.step2.outputs.test }}
steps:
- id: step1
run: echo "test=hello" >> $GITHUB_OUTPUT
- id: step2
run: echo "test=world" >> $GITHUB_OUTPUT
job2:
runs-on: ubuntu-latest
needs: job1
steps:
- run: echo ${{needs.job1.outputs.output1}} ${{needs.job1.outputs.output2}}
Note the use of $GITHUB_OUTPUT, instead of the older ::set-output now (Oct. 2022) deprecated.
To avoid untrusted logged data to use set-state and set-output workflow commands without the intention of the workflow author we have introduced a new set of environment files to manage state and output.
Jesse Adelman adds in the comments:
This seems to not work well for anything beyond a static string.
How, for example, would I take a multiline text output of step (say, I'm running a pytest or similar) and use that output in another job?
either write the multi-line text to a file (jschmitter's comment)
or base64-encode the output and then decode it in the next job (Nate Karasch's comment)
Update: It's now possible to set job outputs that can be used to transfer string values to downstream jobs. See this answer.
What follows is the original answer. These techniques might still be useful for some use cases.
Write the data to file and use actions/upload-artifact and actions/download-artifact. A bit awkward, but it works.
Create a repository dispatch event and send the data to a second workflow. I prefer this method personally, but the downside is that it needs a repo scoped PAT.
Here is an example of how the second way could work. It uses repository-dispatch action.
name: "We 🎔 Perl"
on:
issues:
types: [opened, edited, milestoned]
jobs:
seasonal_greetings:
runs-on: windows-latest
steps:
- name: Maybe greet
id: maybe-greet
env:
HEY: "Hey you!"
GREETING: "Merry Xmas to you too!"
BODY: ${{ github.event.issue.body }}
run: |
$output=(perl -e 'print ($ENV{BODY} =~ /Merry/)?$ENV{GREETING}:$ENV{HEY};')
Write-Output "::set-output name=GREET::$output"
- name: Repository Dispatch
uses: peter-evans/repository-dispatch#v1
with:
token: ${{ secrets.REPO_ACCESS_TOKEN }}
event-type: my-event
client-payload: '{"greet": "${{ steps.maybe-greet.outputs.GREET }}"}'
This triggers a repository dispatch workflow in the same repository.
name: Repository Dispatch
on:
repository_dispatch:
types: [my-event]
jobs:
myEvent:
runs-on: ubuntu-latest
steps:
- run: echo ${{ github.event.client_payload.greet }}
In my case I wanted to pass an entire build/artifact, not just a string:
name: Build something on Ubuntu then use it on MacOS
on:
workflow_dispatch:
# Allows for manual build trigger
jobs:
buildUbuntuProject:
name: Builds the project on Ubuntu (Put your stuff here)
runs-on: ubuntu-latest
steps:
- uses: actions/checkout#v2
- uses: some/compile-action#v99
- uses: actions/upload-artifact#v2
# Upload the artifact so the MacOS runner do something with it
with:
name: CompiledProject
path: pathToCompiledProject
doSomethingOnMacOS:
name: Runs the program on MacOS or something
runs-on: macos-latest
needs: buildUbuntuProject # Needed so the job waits for the Ubuntu job to finish
steps:
- uses: actions/download-artifact#master
with:
name: CompiledProject
path: somewhereToPutItOnMacOSRunner
- run: ls somewhereToPutItOnMacOSRunner # See the artifact on the MacOS runner
It is possible to capture the entire output (and return code) of a command within a run step, which I've written up here to hopefully save someone else the headache. Fair warning, it requires a lot of shell trickery and a multiline run to ensure everything happens within a single shell instance.
In my case, I needed to invoke a script and capture the entirety of its stdout for use in a later step, as well as preserve its outcome for error checking:
# capture stdout from script
SCRIPT_OUTPUT=$(./do-something.sh)
# capture exit code as well
SCRIPT_RC=$?
# FYI, this would get stdout AND stderr
SCRIPT_ALL_OUTPUT=$(./do-something.sh 2>&1)
Since Github's job outputs only seem to be able to capture a single line of text, I also had to escape any newlines for the output:
echo "::set-output name=stdout::${SCRIPT_OUTPUT//$'\n'/\\n}"
Additionally, I needed to ultimately return the script's exit code to correctly indicate whether it failed. The whole shebang ends up looking like this:
- name: A run step with stdout as a captured output
id: myscript
run: |
# run in subshell, capturiing stdout to var
SCRIPT_OUTPUT=$(./do-something.sh)
# capture exit code too
SCRIPT_RC=$?
# print a single line output for github
echo "::set-output name=stdout::${SCRIPT_OUTPUT//$'\n'/\\n}"
# exit with the script status
exit $SCRIPT_RC
continue-on-error: true
- name: Add above outcome and output as an issue comment
uses: actions/github-script#v5
env:
STEP_OUTPUT: ${{ steps.myscript.outputs.stdout }}
with:
github-token: ${{ secrets.GITHUB_TOKEN }}
script: |
// indicates whather script succeeded or not
let comment = `Script finished with \`${{ steps.myscript.outcome }}\`\n`;
// adds stdout, unescaping newlines again to make it readable
comment += `<details><summary>Show Output</summary>
\`\`\`
${process.env.STEP_OUTPUT.replace(/\\n/g, '\n')}
\`\`\`
</details>`;
// add the whole damn thing as an issue comment
github.rest.issues.createComment({
issue_number: context.issue.number,
owner: context.repo.owner,
repo: context.repo.repo,
body: comment
})
Edit: there is also an action to accomplish this with much less bootstrapping, which I only just found.
2022 October update: GitHub is deprecating set-output and recommends to use GITHUB_OUTPUT instead. The syntax for defining the outputs and referencing them in other steps, jobs.
An example from the docs:
- name: Set color
id: random-color-generator
run: echo "SELECTED_COLOR=green" >> $GITHUB_OUTPUT
- name: Get color
run: echo "The selected color is ${{ steps.random-color-generator.outputs.SELECTED_COLOR }}"