Dynamically retrieve GitHub Actions secret - github

Question

I'm trying to dynamically pull back a GitHub secret using GitHub Actions at runtime:
Let's say I have two GitHub Secrets:
SECRET_ORANGES : "This is an orange secret"
SECRET_APPLES : "This is an apple secret"
In my GitHub Action, I have another env variable which will differ between branches
env:
FRUIT_NAME: APPLES
Essentially I want to find a way to do some sort of variable substitution to get the correct secret. So in one of my child jobs, I want to do something like:
env:
FRUIT_SECRET: {{ 'SECRET_' + env.FRUIT_NAME }}
I've tried the following approaches with no luck:
secrets['SECRET_$FRUIT_NAME'] }}
I even tried a simpler approach without concatenation just to try and get it working
secrets['$FRUIT_NAME'] }}
and
{{ secrets.$FRUIT_NAME }}
None of the above worked.
Apologies if I have not explained this very well. I tried to keep my example as simple as possible.
Anyone have any idea of how to achieve this?
Alternatively, what I am trying to do is to store secrets on a per-branch basis
For example:
In customer1 code branch:
SECRET_CREDENTIAL="abc123"
In customer2 code branch:
SECRET_CREDENTIAL="def456"
Then I can access the correct value for SECRET_CREDENTIAL depending on which branch I am in.
Thanks!
Update: I'm getting a bit closer to what I am trying to achieve:
name: Test
env:
CUSTOMER: CUSTOMER1
jobs:
build:
runs-on: ubuntu-latest
env:
AWS_ACCESS_KEY_ID: ${{ env.CUSTOMER }}_AWS_ACCESS_KEY_ID
steps:
- uses: actions/checkout#v2
- run: |
AWS_ACCESS_KEY_ID=${{ secrets[env.AWS_ACCESS_KEY_ID] }}
echo "AWS_ACCESS_KEY_ID = $AWS_ACCESS_KEY_ID"

Answer 1

There is a much cleaner option to achieve this using the format function.
Given set secrets DEV_A and TEST_A, the following two jobs will use those two secrets:
name: Secrets
on: [push]
jobs:
dev:
name: dev
runs-on: ubuntu-18.04
env:
ENVIRONMENT: DEV
steps:
- run: echo ${{ secrets[format('{0}_A', env.ENVIRONMENT)] }}
test:
name: test
runs-on: ubuntu-18.04
env:
ENVIRONMENT: TEST
steps:
- run: echo ${{ secrets[format('{0}_A', env.ENVIRONMENT)] }}
This also works with input provided through manual workflows (the workflow_dispatch event):
name: Secrets
on:
workflow_dispatch:
inputs:
env:
description: "Environment to deploy to"
required: true
jobs:
secrets:
name: secrets
runs-on: ubuntu-18.04
steps:
- run: echo ${{ secrets[format('{0}_A', github.event.inputs.env)] }}

Answer 2

Update - July 2021
I found a better way to prepare dynamic secrets in a job, and then consume those secrets as environment variables in other jobs.
Here's how it looks like in GitHub Actions.
My assumption is that each secret should be fetched according to the branch name. I'm getting the branch's name with this action rlespinasse/github-slug-action.
Go through the inline comments to understand how it all works together.
name: Dynamic Secret Names
# Assumption:
# You've created the following GitHub secrets in your repository:
# AWS_ACCESS_KEY_ID_master
# AWS_SECRET_ACCESS_KEY_master
on:
push:
env:
AWS_REGION: "eu-west-1"
jobs:
prepare:
name: Prepare
runs-on: ubuntu-20.04
steps:
- uses: actions/checkout#v2
- name: Inject slug/short variables
uses: rlespinasse/github-slug-action#v3.x
- name: Prepare Outputs
id: prepare-step
# Sets this step's outputs, that later on will be exported as the job's outputs
run: |
echo "::set-output name=aws_access_key_id_name::AWS_ACCESS_KEY_ID_${GITHUB_REF_SLUG}";
echo "::set-output name=aws_secret_access_key_name::AWS_SECRET_ACCESS_KEY_${GITHUB_REF_SLUG}";
# Sets this job's, that will be consumed by other jobs
# https://docs.github.com/en/actions/reference/workflow-syntax-for-github-actions#jobsjob_idoutputs
outputs:
aws_access_key_id_name: ${{ steps.prepare-step.outputs.aws_access_key_id_name }}
aws_secret_access_key_name: ${{ steps.prepare-step.outputs.aws_secret_access_key_name }}
test:
name: Test
# Must wait for `prepare` to complete so it can use `${{ needs.prepare.outputs.{output_name} }}`
# https://docs.github.com/en/actions/reference/context-and-expression-syntax-for-github-actions#needs-context
needs:
- prepare
runs-on: ubuntu-20.04
env:
# Get secret names
AWS_ACCESS_KEY_ID_NAME: ${{ needs.prepare.outputs.aws_access_key_id_name }}
AWS_SECRET_ACCESS_KEY_NAME: ${{ needs.prepare.outputs.aws_secret_access_key_name }}
steps:
- uses: actions/checkout#v2
- name: Test Application
env:
# Inject secret values to environment variables
AWS_ACCESS_KEY_ID: ${{ secrets[env.AWS_ACCESS_KEY_ID_NAME] }}
AWS_SECRET_ACCESS_KEY: ${{ secrets[env.AWS_SECRET_ACCESS_KEY_NAME] }}
run: |
printenv | grep AWS_
aws s3 ls
Update - August 2020
Following some hands-on experience with this project terraform-monorepo, here's an example of how I managed to use secret names dynamically
Secrets names are aligned with environments names and branches names - development, staging and production
$GITHUB_REF_SLUG comes from the Slug GitHub Action which fetches the name of the branch
The commands which perform the parsing are
- name: set-aws-credentials
run: |
echo "::set-env name=AWS_ACCESS_KEY_ID_SECRET_NAME::AWS_ACCESS_KEY_ID_${GITHUB_REF_SLUG}"
echo "::set-env name=AWS_SECRET_ACCESS_KEY_SECRET_NAME::AWS_SECRET_ACCESS_KEY_${GITHUB_REF_SLUG}"
- name: terraform-apply
run: |
export AWS_ACCESS_KEY_ID=${{ secrets[env.AWS_ACCESS_KEY_ID_SECRET_NAME] }}
export AWS_SECRET_ACCESS_KEY=${{ secrets[env.AWS_SECRET_ACCESS_KEY_SECRET_NAME] }}
Full example
name: pipeline
on:
push:
branches: [development, staging, production]
paths-ignore:
- "README.md"
jobs:
terraform:
runs-on: ubuntu-latest
env:
### -----------------------
### Available in all steps, change app_name to your app_name
TF_VAR_app_name: tfmonorepo
### -----------------------
steps:
- uses: actions/checkout#v2
- name: Inject slug/short variables
uses: rlespinasse/github-slug-action#v2.x
- name: prepare-files-folders
run: |
mkdir -p ${GITHUB_REF_SLUG}/
cp live/*.${GITHUB_REF_SLUG} ${GITHUB_REF_SLUG}/
cp live/*.tf ${GITHUB_REF_SLUG}/
cp live/*.tpl ${GITHUB_REF_SLUG}/ 2>/dev/null || true
mv ${GITHUB_REF_SLUG}/backend.tf.${GITHUB_REF_SLUG} ${GITHUB_REF_SLUG}/backend.tf
- name: install-terraform
uses: little-core-labs/install-terraform#v1
with:
version: 0.12.28
- name: set-aws-credentials
run: |
echo "::set-env name=AWS_ACCESS_KEY_ID_SECRET_NAME::AWS_ACCESS_KEY_ID_${GITHUB_REF_SLUG}"
echo "::set-env name=AWS_SECRET_ACCESS_KEY_SECRET_NAME::AWS_SECRET_ACCESS_KEY_${GITHUB_REF_SLUG}"
- name: terraform-apply
run: |
export AWS_ACCESS_KEY_ID=${{ secrets[env.AWS_ACCESS_KEY_ID_SECRET_NAME] }}
export AWS_SECRET_ACCESS_KEY=${{ secrets[env.AWS_SECRET_ACCESS_KEY_SECRET_NAME] }}
cd ${GITHUB_REF_SLUG}/
terraform version
rm -rf .terraform
terraform init -input=false
terraform get
terraform validate
terraform plan -out=plan.tfout -var environment=${GITHUB_REF_SLUG}
terraform apply -auto-approve plan.tfout
rm -rf .terraform
After reading this - Context and expression syntax for GitHub Actions
, focusing on
env object, I found out that:
As part of an expression, you may access context information using one of two syntaxes.
Index syntax: github['sha']
Property dereference syntax: github.sha
So the same behavior applies to secrets, you can do secrets[secret_name], so you can do the following
- name: Run a multi-line script
env:
SECRET_NAME: A_FRUIT_NAME
run: |
echo "SECRET_NAME = $SECRET_NAME"
echo "SECRET_NAME = ${{ env.SECRET_NAME }}"
SECRET_VALUE=${{ secrets[env.SECRET_NAME] }}
echo "SECRET_VALUE = $SECRET_VALUE"
Which results in
SECRET_NAME = A_FRUIT_NAME
SECRET_NAME = A_FRUIT_NAME
SECRET_VALUE = ***
Since the SECRET_VALUE is redacted, we can assume that the real secret was fetched.
Things that I learned -
You can't reference env from another env, so this won't work
env:
SECRET_PREFIX: A
SECRET_NAME: ${{ env.SECRET_PREFIX }}_FRUIT_NAME
The result of SECRET_NAME is _FRUIT_NAME, not good
You can use context expressions in your code, not only in env, you can see that in SECRET_VALUE=${{ secrets[env.SECRET_NAME] }}, which is cool
And of course - here's the workflow that I tested - https://github.com/unfor19/gha-play/runs/595345435?check_suite_focus=true - check the Run a multi-line script step

Answer 3

In case this can help, after reading the above answers which truly helped, the strategy I decided to use consists of storing my secrets as follow:
DB_USER_MASTER
DB_PASSWORD_MASTER
DB_USER_TEST
DB_PASSWORD_TEST
Where MASTER is the master branch for the prod environment and TEST is the test branch for the test environment.
Then, using the suggested solutions in this thread, the key is to dynamically generate the keys of the secrets variable. Those keys are generated via an intermediate step (called vars in the sample below) using outputs:
name: Pulumi up
on:
push:
branches:
- master
- test
jobs:
up:
name: Update
runs-on: ubuntu-latest
steps:
- name: Create variables
id: vars
run: |
branch=${GITHUB_REF##*/}
echo "::set-output name=DB_USER::DB_USER_${branch^^}"
echo "::set-output name=DB_PASSWORD::DB_PASSWORD_${branch^^}"
- uses: actions/checkout#v2
with:
fetch-depth: 1
- uses: docker://pulumi/actions
with:
args: up -s ${GITHUB_REF##*/} -y
env:
AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
GOOGLE_CREDENTIALS: ${{ secrets.GOOGLE_CREDENTIALS }}
PULUMI_ACCESS_TOKEN: ${{ secrets.PULUMI_ACCESS_TOKEN }}
DB_USER: ${{ secrets[steps.vars.outputs.DB_USER] }}
DB_PASSWORD: ${{ secrets[steps.vars.outputs.DB_PASSWORD] }}
Notice the hack to get the branch on uppercase: ${branch^^}. This is required because GitHub forces secrets to uppercase.

Answer 4

I was able to achieve this using the workflow name as the branch specific variable.
For each branch I create, I simply update this single value at the top of the YML file, then add GitHub Secrets to match the workflow name:
name: CUSTOMER1
jobs:
build:
runs-on: ubuntu-latest
env:
AWS_ACCESS_KEY_ID: ${{ github.workflow }}_AWS_ACCESS_KEY_ID
steps:
- uses: actions/checkout#v2
- run: echo "::set-env name=AWS_ACCESS_KEY_ID::${{ secrets[env.AWS_ACCESS_KEY_ID] }}"
- run: echo $AWS_ACCESS_KEY_ID

Answer 5

Don't use ::set-env, it is depreacated.
Use instead
echo "env_key=env_value" >> $GITHUB_ENV
You can set the env variable on a branch basis by setting env as in this example.
Suppose you have at least two secrets with different prefixes in your repository, like this: (DEV_SERVER_IP, OTHER_SERVER_IP)
I use 'format', '$GITHUB_ENV' which are workflow commands and function provide on Github.
- name: Set develop env
if: ${{ github.ref == 'refs/heads/develop' }}
run: echo "branch_name=DEVELOP" >> $GITHUB_ENV
- name: Set other env
if: ${{ github.ref == 'refs/heads/other' }}
run: echo "branch_name=OTHER" >> $GITHUB_ENV
- name: SSH Test
env:
SERVER_IP: ${{ secrets[format('{0}_SERVER_IP', env.branch_name)] }}
run: ssh -T user#$SERVER_IP

Answer 6

New solution as of December 2020
If you are reading this question because you need to use different secret values based on the environment you are deploying to, GitHub Actions now has a new feature called "Environments" in beta: https://docs.github.com/en/free-pro-team#latest/actions/reference/environments
This allows us to define environment secrets, and allow only jobs that are assigned to the environment to access them. This not only leads to better user experience as a developer, but also to better security and isolation of different deployment jobs.
Below is an example for how to dynamically determine the environment that should be used, based on the branch name:
jobs:
get-environment-name:
name: "Extract environment name"
runs-on: ubuntu-latest
outputs:
environment: ${{ steps.extract.outputs.environment }}
steps:
- id: extract
# You can run any logic you want here to map refs to environment names.
# The GITHUB_REF will look like this: refs/heads/my-branchname
# The example logic here simply removes "refs/heads/deploy-" from the beginning,
# so a branch name deploy-prod would be mapped to the environment "prod"
run: echo "::set-output name=environment::$(echo $GITHUB_REF | sed -e '/^refs\/heads\/deploy-\(.*\)$/!d;s//\1/')"
- env:
EXTRACTED: ${{ steps.extract.outputs.environment }}
run: 'echo "Extracted environment name: $EXTRACTED"'
deploy:
name: "Deploy"
if: ${{ github.event_name == 'push' && needs.get-environment-name.outputs.environment }}
needs:
- get-environment-name
# - unit-tests
# - frontend-tests
# ... add your unit test jobs here, so they are executed before deploying anything
environment: ${{ needs.get-environment-name.outputs.environment }}
runs-on: ubuntu-latest
steps:
- uses: actions/checkout#v2
# ... Run your deployment actions here, with full access to the environment's secrets
Note that in the if: clause of the deployment job, it's not possible to use any environment variables or bash scripts. So using a previous job that extracts the environment name from the branch name is the simplest I could make it at the current time.

Answer 7

I came across this question when trying to implement environment-based secret selection for a Github action.
This variable-mapper action (https://github.com/marketplace/actions/variable-mapper) implements the desired concept of mapping a key variable or an environment name to secrets or other pre-defined values.
The example use of it shows this:
on: [push]
name: Export variables corresponding to regular expression-matched keys
jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: kanga333/variable-mapper#v1
with:
key: ${{GITHUB_REF#refs/heads/}}
map: |
{
"master": {
"environment": "production",
"AWS_ACCESS_KEY_ID": ${{ secrets.PROD_AWS_ACCESS_KEY_ID }},
"AWS_SECRET_ACCESS_KEY": ${{ secrets.PROD_AWS_ACCESS_KEY_ID }}
},
"staging": {
"environment": "staging",
"AWS_ACCESS_KEY_ID": ${{ secrets.STG_AWS_ACCESS_KEY_ID }},
"AWS_SECRET_ACCESS_KEY": ${{ secrets.STG_AWS_ACCESS_KEY_ID }}
},
".*": {
"environment": "development",
"AWS_ACCESS_KEY_ID": ${{ secrets.DEV_AWS_ACCESS_KEY_ID }},
"AWS_SECRET_ACCESS_KEY": ${{ secrets.DEV_AWS_ACCESS_KEY_ID }}
}
}
- name: Echo environment
run: echo ${{ env.environment }}