How do I create a github workflow step name with a variable value.
I tried this but it does not work.
name: Publish
on:
push:
branches:
- main
env:
REGISTRY: ghcr.io
jobs:
Publish:
runs-on: ubuntu-latest
steps:
- name: Log into Container registry ${{ env.REGISTRY }}
I know you tried it, but reproducing the workflow here with your implementation (as below) actually worked for me.
name: Publish
on:
push:
env:
REGISTRY: ghcr.io
jobs:
Publish:
runs-on: ubuntu-latest
steps:
- name: Log into Container registry ${{ env.REGISTRY }}
run: echo "Ok"
The job step name was generated dynamically according to the workflow env variable set.
Here is the workflow run
Since this does not seem supported, it is better to add an echo which prints the variable, before one processing it:
name: Publish
on:
push:
branches:
- main
env:
REGISTRY: ghcr.io
jobs:
Publish:
runs-on: ubuntu-latest
steps:
- name: Log into Container registry
run: echo "registry is '$REGISTRY'"
After that, for runtime variables, you can add conditions:
on:
push:
branches:
- actions-test-branch
jobs:
Echo-On-Commit:
runs-on: ubuntu-latest
steps:
- name: "Checkout Repository"
uses: actions/checkout#v2
- name: "Set flag from Commit"
env:
COMMIT_VAR: ${{ contains(github.event.head_commit.message, '[commit var]') }}
run: |
if ${COMMIT_VAR} == true; then
echo "flag=true" >> $GITHUB_ENV
echo "flag set to true"
else
echo "flag=false" >> $GITHUB_ENV
echo "flag set to false"
fi
- name: "Use flag if true"
if: env.flag
run: echo "Flag is available and true"
I have a workflow that checks files on push and pull_request.
It has 2 jobs: One to list the changed files that match a pattern (Dockerfiles), and a second job with a matrix strategy to be executed for every file.
The jobs:
jobs:
get-files:
name: Get changed files
runs-on: ubuntu-latest
outputs:
dockerfiles: ${{ steps.filter.outputs.dockerfiles_files }}
steps:
- uses: dorny/paths-filter#v2
id: filter
with:
list-files: json
filters: |
dockerfiles:
- "**/Dockerfile"
check:
name: Check Dockerfiles
needs: get-files
strategy:
matrix:
dockerfile: ${{ fromJson(needs.get-files.outputs.dockerfiles) }}
runs-on: ubuntu-latest
steps:
- id: get-directory
# Remove last path segment to only keep the Dockerfile directory
run: |
directory=$(echo ${{matrix.dockerfile}} | sed -r 's/\/[^\/]+$//g')
echo "::set-output name=directory::$directory"
- run: echo "${{steps.get-directory.outputs.directory}}"
- uses: actions/checkout#v2
- name: Build Dockerfile ${{ matrix.dockerfile }}
run: docker build ${{steps.get-directory.outputs.directory}} -f ${{ matrix.dockerfile }}
My problem is, that the "get-files" ("Get changed files") job appears as a check in the pull requests:
Is there any way to hide it in the PR checks?
If not, is there a better way to have a check per modified file?
The workflow in file inrisk.packages.ci.yml generates a tag and a realise of the code when a push is done in the develop branch. The below works as expected.
name: Code Int
on:
push:
paths:
- 'infra/**'
jobs:
ci:
runs-on: ubuntu-latest
steps:
# Checks-out to $GITHUB_WORKSPACE
- uses: actions/checkout#v2
- name: Basic Checks
run: |
whoami
ls -lah
pwd
- uses: actions/setup-node#v1
# Create a new release when on develop which triggers the deployment
- name: Bump version and push tag
if: github.ref == 'refs/heads/develop'
uses: mathieudutour/github-tag-action#v4.5
id: tag_version
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
- name: Create Release
if: github.ref == 'refs/heads/develop'
id: create_release
uses: actions/create-release#v1
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
with:
tag_name: ${{ steps.tag_version.outputs.new_tag }}
release_name: Release ${{ steps.tag_version.outputs.new_tag }}
draft: false
prerelease: false
The below workflow in file inrisk.packages.cd.yml and is suppose to be triggered when ever a tag/realise is created/published.
name: Code Deploy
on:
push:
tags:
- 'v*'
release:
types:
- published
- created
- released
jobs:
build-and-deploy:
runs-on: ubuntu-latest
steps:
# Checks-out to $GITHUB_WORKSPACE
- uses: actions/checkout#v2
- uses: actions/setup-node#v1
- name: Install Yarn
run: npm install -g yarn
- uses: chrislennon/action-aws-cli#v1.1
- name: Install, Build and Deploy
run: |
whoami
ls -lah
pwd
The second workflow Code Deploy dose not get trigger after Code Int publishes/created a tag/realise
However when I manually create a realise/tag the second workflow Code Deploy get triggered
This seems to be by design as stated here .This is to stop recursive workflow runs.
I used this article to get around the problem
I'm trying to dynamically pull back a GitHub secret using GitHub Actions at runtime:
Let's say I have two GitHub Secrets:
SECRET_ORANGES : "This is an orange secret"
SECRET_APPLES : "This is an apple secret"
In my GitHub Action, I have another env variable which will differ between branches
env:
FRUIT_NAME: APPLES
Essentially I want to find a way to do some sort of variable substitution to get the correct secret. So in one of my child jobs, I want to do something like:
env:
FRUIT_SECRET: {{ 'SECRET_' + env.FRUIT_NAME }}
I've tried the following approaches with no luck:
secrets['SECRET_$FRUIT_NAME'] }}
I even tried a simpler approach without concatenation just to try and get it working
secrets['$FRUIT_NAME'] }}
and
{{ secrets.$FRUIT_NAME }}
None of the above worked.
Apologies if I have not explained this very well. I tried to keep my example as simple as possible.
Anyone have any idea of how to achieve this?
Alternatively, what I am trying to do is to store secrets on a per-branch basis
For example:
In customer1 code branch:
SECRET_CREDENTIAL="abc123"
In customer2 code branch:
SECRET_CREDENTIAL="def456"
Then I can access the correct value for SECRET_CREDENTIAL depending on which branch I am in.
Thanks!
Update: I'm getting a bit closer to what I am trying to achieve:
name: Test
env:
CUSTOMER: CUSTOMER1
jobs:
build:
runs-on: ubuntu-latest
env:
AWS_ACCESS_KEY_ID: ${{ env.CUSTOMER }}_AWS_ACCESS_KEY_ID
steps:
- uses: actions/checkout#v2
- run: |
AWS_ACCESS_KEY_ID=${{ secrets[env.AWS_ACCESS_KEY_ID] }}
echo "AWS_ACCESS_KEY_ID = $AWS_ACCESS_KEY_ID"
There is a much cleaner option to achieve this using the format function.
Given set secrets DEV_A and TEST_A, the following two jobs will use those two secrets:
name: Secrets
on: [push]
jobs:
dev:
name: dev
runs-on: ubuntu-18.04
env:
ENVIRONMENT: DEV
steps:
- run: echo ${{ secrets[format('{0}_A', env.ENVIRONMENT)] }}
test:
name: test
runs-on: ubuntu-18.04
env:
ENVIRONMENT: TEST
steps:
- run: echo ${{ secrets[format('{0}_A', env.ENVIRONMENT)] }}
This also works with input provided through manual workflows (the workflow_dispatch event):
name: Secrets
on:
workflow_dispatch:
inputs:
env:
description: "Environment to deploy to"
required: true
jobs:
secrets:
name: secrets
runs-on: ubuntu-18.04
steps:
- run: echo ${{ secrets[format('{0}_A', github.event.inputs.env)] }}
Update - July 2021
I found a better way to prepare dynamic secrets in a job, and then consume those secrets as environment variables in other jobs.
Here's how it looks like in GitHub Actions.
My assumption is that each secret should be fetched according to the branch name. I'm getting the branch's name with this action rlespinasse/github-slug-action.
Go through the inline comments to understand how it all works together.
name: Dynamic Secret Names
# Assumption:
# You've created the following GitHub secrets in your repository:
# AWS_ACCESS_KEY_ID_master
# AWS_SECRET_ACCESS_KEY_master
on:
push:
env:
AWS_REGION: "eu-west-1"
jobs:
prepare:
name: Prepare
runs-on: ubuntu-20.04
steps:
- uses: actions/checkout#v2
- name: Inject slug/short variables
uses: rlespinasse/github-slug-action#v3.x
- name: Prepare Outputs
id: prepare-step
# Sets this step's outputs, that later on will be exported as the job's outputs
run: |
echo "::set-output name=aws_access_key_id_name::AWS_ACCESS_KEY_ID_${GITHUB_REF_SLUG}";
echo "::set-output name=aws_secret_access_key_name::AWS_SECRET_ACCESS_KEY_${GITHUB_REF_SLUG}";
# Sets this job's, that will be consumed by other jobs
# https://docs.github.com/en/actions/reference/workflow-syntax-for-github-actions#jobsjob_idoutputs
outputs:
aws_access_key_id_name: ${{ steps.prepare-step.outputs.aws_access_key_id_name }}
aws_secret_access_key_name: ${{ steps.prepare-step.outputs.aws_secret_access_key_name }}
test:
name: Test
# Must wait for `prepare` to complete so it can use `${{ needs.prepare.outputs.{output_name} }}`
# https://docs.github.com/en/actions/reference/context-and-expression-syntax-for-github-actions#needs-context
needs:
- prepare
runs-on: ubuntu-20.04
env:
# Get secret names
AWS_ACCESS_KEY_ID_NAME: ${{ needs.prepare.outputs.aws_access_key_id_name }}
AWS_SECRET_ACCESS_KEY_NAME: ${{ needs.prepare.outputs.aws_secret_access_key_name }}
steps:
- uses: actions/checkout#v2
- name: Test Application
env:
# Inject secret values to environment variables
AWS_ACCESS_KEY_ID: ${{ secrets[env.AWS_ACCESS_KEY_ID_NAME] }}
AWS_SECRET_ACCESS_KEY: ${{ secrets[env.AWS_SECRET_ACCESS_KEY_NAME] }}
run: |
printenv | grep AWS_
aws s3 ls
Update - August 2020
Following some hands-on experience with this project terraform-monorepo, here's an example of how I managed to use secret names dynamically
Secrets names are aligned with environments names and branches names - development, staging and production
$GITHUB_REF_SLUG comes from the Slug GitHub Action which fetches the name of the branch
The commands which perform the parsing are
- name: set-aws-credentials
run: |
echo "::set-env name=AWS_ACCESS_KEY_ID_SECRET_NAME::AWS_ACCESS_KEY_ID_${GITHUB_REF_SLUG}"
echo "::set-env name=AWS_SECRET_ACCESS_KEY_SECRET_NAME::AWS_SECRET_ACCESS_KEY_${GITHUB_REF_SLUG}"
- name: terraform-apply
run: |
export AWS_ACCESS_KEY_ID=${{ secrets[env.AWS_ACCESS_KEY_ID_SECRET_NAME] }}
export AWS_SECRET_ACCESS_KEY=${{ secrets[env.AWS_SECRET_ACCESS_KEY_SECRET_NAME] }}
Full example
name: pipeline
on:
push:
branches: [development, staging, production]
paths-ignore:
- "README.md"
jobs:
terraform:
runs-on: ubuntu-latest
env:
### -----------------------
### Available in all steps, change app_name to your app_name
TF_VAR_app_name: tfmonorepo
### -----------------------
steps:
- uses: actions/checkout#v2
- name: Inject slug/short variables
uses: rlespinasse/github-slug-action#v2.x
- name: prepare-files-folders
run: |
mkdir -p ${GITHUB_REF_SLUG}/
cp live/*.${GITHUB_REF_SLUG} ${GITHUB_REF_SLUG}/
cp live/*.tf ${GITHUB_REF_SLUG}/
cp live/*.tpl ${GITHUB_REF_SLUG}/ 2>/dev/null || true
mv ${GITHUB_REF_SLUG}/backend.tf.${GITHUB_REF_SLUG} ${GITHUB_REF_SLUG}/backend.tf
- name: install-terraform
uses: little-core-labs/install-terraform#v1
with:
version: 0.12.28
- name: set-aws-credentials
run: |
echo "::set-env name=AWS_ACCESS_KEY_ID_SECRET_NAME::AWS_ACCESS_KEY_ID_${GITHUB_REF_SLUG}"
echo "::set-env name=AWS_SECRET_ACCESS_KEY_SECRET_NAME::AWS_SECRET_ACCESS_KEY_${GITHUB_REF_SLUG}"
- name: terraform-apply
run: |
export AWS_ACCESS_KEY_ID=${{ secrets[env.AWS_ACCESS_KEY_ID_SECRET_NAME] }}
export AWS_SECRET_ACCESS_KEY=${{ secrets[env.AWS_SECRET_ACCESS_KEY_SECRET_NAME] }}
cd ${GITHUB_REF_SLUG}/
terraform version
rm -rf .terraform
terraform init -input=false
terraform get
terraform validate
terraform plan -out=plan.tfout -var environment=${GITHUB_REF_SLUG}
terraform apply -auto-approve plan.tfout
rm -rf .terraform
After reading this - Context and expression syntax for GitHub Actions
, focusing on
env object, I found out that:
As part of an expression, you may access context information using one of two syntaxes.
Index syntax: github['sha']
Property dereference syntax: github.sha
So the same behavior applies to secrets, you can do secrets[secret_name], so you can do the following
- name: Run a multi-line script
env:
SECRET_NAME: A_FRUIT_NAME
run: |
echo "SECRET_NAME = $SECRET_NAME"
echo "SECRET_NAME = ${{ env.SECRET_NAME }}"
SECRET_VALUE=${{ secrets[env.SECRET_NAME] }}
echo "SECRET_VALUE = $SECRET_VALUE"
Which results in
SECRET_NAME = A_FRUIT_NAME
SECRET_NAME = A_FRUIT_NAME
SECRET_VALUE = ***
Since the SECRET_VALUE is redacted, we can assume that the real secret was fetched.
Things that I learned -
You can't reference env from another env, so this won't work
env:
SECRET_PREFIX: A
SECRET_NAME: ${{ env.SECRET_PREFIX }}_FRUIT_NAME
The result of SECRET_NAME is _FRUIT_NAME, not good
You can use context expressions in your code, not only in env, you can see that in SECRET_VALUE=${{ secrets[env.SECRET_NAME] }}, which is cool
And of course - here's the workflow that I tested - https://github.com/unfor19/gha-play/runs/595345435?check_suite_focus=true - check the Run a multi-line script step
In case this can help, after reading the above answers which truly helped, the strategy I decided to use consists of storing my secrets as follow:
DB_USER_MASTER
DB_PASSWORD_MASTER
DB_USER_TEST
DB_PASSWORD_TEST
Where MASTER is the master branch for the prod environment and TEST is the test branch for the test environment.
Then, using the suggested solutions in this thread, the key is to dynamically generate the keys of the secrets variable. Those keys are generated via an intermediate step (called vars in the sample below) using outputs:
name: Pulumi up
on:
push:
branches:
- master
- test
jobs:
up:
name: Update
runs-on: ubuntu-latest
steps:
- name: Create variables
id: vars
run: |
branch=${GITHUB_REF##*/}
echo "::set-output name=DB_USER::DB_USER_${branch^^}"
echo "::set-output name=DB_PASSWORD::DB_PASSWORD_${branch^^}"
- uses: actions/checkout#v2
with:
fetch-depth: 1
- uses: docker://pulumi/actions
with:
args: up -s ${GITHUB_REF##*/} -y
env:
AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
GOOGLE_CREDENTIALS: ${{ secrets.GOOGLE_CREDENTIALS }}
PULUMI_ACCESS_TOKEN: ${{ secrets.PULUMI_ACCESS_TOKEN }}
DB_USER: ${{ secrets[steps.vars.outputs.DB_USER] }}
DB_PASSWORD: ${{ secrets[steps.vars.outputs.DB_PASSWORD] }}
Notice the hack to get the branch on uppercase: ${branch^^}. This is required because GitHub forces secrets to uppercase.
I was able to achieve this using the workflow name as the branch specific variable.
For each branch I create, I simply update this single value at the top of the YML file, then add GitHub Secrets to match the workflow name:
name: CUSTOMER1
jobs:
build:
runs-on: ubuntu-latest
env:
AWS_ACCESS_KEY_ID: ${{ github.workflow }}_AWS_ACCESS_KEY_ID
steps:
- uses: actions/checkout#v2
- run: echo "::set-env name=AWS_ACCESS_KEY_ID::${{ secrets[env.AWS_ACCESS_KEY_ID] }}"
- run: echo $AWS_ACCESS_KEY_ID
Don't use ::set-env, it is depreacated.
Use instead
echo "env_key=env_value" >> $GITHUB_ENV
You can set the env variable on a branch basis by setting env as in this example.
Suppose you have at least two secrets with different prefixes in your repository, like this: (DEV_SERVER_IP, OTHER_SERVER_IP)
I use 'format', '$GITHUB_ENV' which are workflow commands and function provide on Github.
- name: Set develop env
if: ${{ github.ref == 'refs/heads/develop' }}
run: echo "branch_name=DEVELOP" >> $GITHUB_ENV
- name: Set other env
if: ${{ github.ref == 'refs/heads/other' }}
run: echo "branch_name=OTHER" >> $GITHUB_ENV
- name: SSH Test
env:
SERVER_IP: ${{ secrets[format('{0}_SERVER_IP', env.branch_name)] }}
run: ssh -T user#$SERVER_IP
New solution as of December 2020
If you are reading this question because you need to use different secret values based on the environment you are deploying to, GitHub Actions now has a new feature called "Environments" in beta: https://docs.github.com/en/free-pro-team#latest/actions/reference/environments
This allows us to define environment secrets, and allow only jobs that are assigned to the environment to access them. This not only leads to better user experience as a developer, but also to better security and isolation of different deployment jobs.
Below is an example for how to dynamically determine the environment that should be used, based on the branch name:
jobs:
get-environment-name:
name: "Extract environment name"
runs-on: ubuntu-latest
outputs:
environment: ${{ steps.extract.outputs.environment }}
steps:
- id: extract
# You can run any logic you want here to map refs to environment names.
# The GITHUB_REF will look like this: refs/heads/my-branchname
# The example logic here simply removes "refs/heads/deploy-" from the beginning,
# so a branch name deploy-prod would be mapped to the environment "prod"
run: echo "::set-output name=environment::$(echo $GITHUB_REF | sed -e '/^refs\/heads\/deploy-\(.*\)$/!d;s//\1/')"
- env:
EXTRACTED: ${{ steps.extract.outputs.environment }}
run: 'echo "Extracted environment name: $EXTRACTED"'
deploy:
name: "Deploy"
if: ${{ github.event_name == 'push' && needs.get-environment-name.outputs.environment }}
needs:
- get-environment-name
# - unit-tests
# - frontend-tests
# ... add your unit test jobs here, so they are executed before deploying anything
environment: ${{ needs.get-environment-name.outputs.environment }}
runs-on: ubuntu-latest
steps:
- uses: actions/checkout#v2
# ... Run your deployment actions here, with full access to the environment's secrets
Note that in the if: clause of the deployment job, it's not possible to use any environment variables or bash scripts. So using a previous job that extracts the environment name from the branch name is the simplest I could make it at the current time.
I came across this question when trying to implement environment-based secret selection for a Github action.
This variable-mapper action (https://github.com/marketplace/actions/variable-mapper) implements the desired concept of mapping a key variable or an environment name to secrets or other pre-defined values.
The example use of it shows this:
on: [push]
name: Export variables corresponding to regular expression-matched keys
jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: kanga333/variable-mapper#v1
with:
key: ${{GITHUB_REF#refs/heads/}}
map: |
{
"master": {
"environment": "production",
"AWS_ACCESS_KEY_ID": ${{ secrets.PROD_AWS_ACCESS_KEY_ID }},
"AWS_SECRET_ACCESS_KEY": ${{ secrets.PROD_AWS_ACCESS_KEY_ID }}
},
"staging": {
"environment": "staging",
"AWS_ACCESS_KEY_ID": ${{ secrets.STG_AWS_ACCESS_KEY_ID }},
"AWS_SECRET_ACCESS_KEY": ${{ secrets.STG_AWS_ACCESS_KEY_ID }}
},
".*": {
"environment": "development",
"AWS_ACCESS_KEY_ID": ${{ secrets.DEV_AWS_ACCESS_KEY_ID }},
"AWS_SECRET_ACCESS_KEY": ${{ secrets.DEV_AWS_ACCESS_KEY_ID }}
}
}
- name: Echo environment
run: echo ${{ env.environment }}
Currently on my GitHub repository, I have the following workflow that releases a nightly snapshot every day, and uses the current date as release name and tag name:
name: Nightly Snapshot
on:
schedule:
- cron: "59 23 * * *"
jobs:
build:
name: Release
runs-on: ubuntu-latest
steps:
- name: Get current date
id: date
run: echo "::set-output name=date::$(date +'%Y-%m-%d')"
- name: Checkout branch "master"
uses: actions/checkout#v2
with:
ref: 'master'
- name: Release snapshot
id: release-snapshot
uses: actions/create-release#latest
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
with:
tag_name: ${{ steps.date.outputs.date }}
release_name: ${{ steps.date.outputs.date }}
draft: false
prerelease: false
GitHub labels all snapshots created this way as the latest release. However, I want to avoid this, and achieve something akin to what Swift's snapshots are like: the snapshots are only tags; although they appear among the releases, they're treated differently.
How should I modify my workflow file to make this happen? Thanks!
Another option is to use GitHub Script. This creates a lightweight tag called <tagname> (replace this with the name of your tag):
- name: Create tag
uses: actions/github-script#v5
with:
script: |
github.rest.git.createRef({
owner: context.repo.owner,
repo: context.repo.repo,
ref: 'refs/tags/<tagname>',
sha: context.sha
})
Edit: Michael Ganß's solution is better.
I found this GitHub action that tags on demand. Using it, my workflow can be revised as such:
name: Nightly Snapshot
on:
schedule:
- cron: "59 23 * * *"
jobs:
tag:
name: Tag
runs-on: ubuntu-latest
steps:
- name: Get current date
id: date
run: echo "::set-output name=date::$(date +'%Y-%m-%d')"
- name: Checkout branch "master"
uses: actions/checkout#v2
with:
ref: 'master'
- name: Tag snapshot
uses: tvdias/github-tagger#v0.0.1
with:
repo-token: ${{ secrets.GITHUB_TOKEN }}
tag: ${{ steps.date.outputs.date }}
I succeed with only : git tag + git push
I'm using gitVersion to automatically generate the tag
semver:
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout#v2
with:
fetch-depth: 0
- name: Install GitVersion
uses: gittools/actions/gitversion/setup#v0.9.7
with:
versionSpec: '5.x'
- name: Determine Version
uses: gittools/actions/gitversion/execute#v0.9.7
- name: Display SemVer
run: |
echo "SemVer: $GITVERSION_SEMVER" && echo "$version" && echo "$major.$minor.$patch"
- name: Create git tag
run: |
git tag $GITVERSION_SEMVER
- name: Push git tag
run: git push origin $GITVERSION_SEMVER
Building on Michael Ganß's solution, here is an example of how to create a variable dynamically.
- name: Set Dist Version
run: |
BUILD_NUMBER="${{ github.run_id }}-${{ github.run_number }}-${{ github.run_attempt }}"
echo "${BUILD_NUMBER}"
VERSION="$(mvn -q -U -Dexpression=project.build.finalName help:evaluate -DforceStdout=true -DbuildNumber=${BUILD_NUMBER})"
echo "DIST_VERSION=${VERSION}" >> $GITHUB_ENV
- name: Create Tag
uses: actions/github-script#v6
with:
script: |
const {DIST_VERSION} = process.env
github.rest.git.createRef({
owner: context.repo.owner,
repo: context.repo.repo,
ref: `refs/tags/${DIST_VERSION}`,
sha: context.sha
})