My goal is actually pretty easy but I am not sure if it is possible in the Github Actions or not. So I have an action file that is used for deployment of different Azure subscription/tenants. So I have different AZURE_CREDENTIALS for every one of them.
For example AZURE_CREDENTIALS_NUM1 is one of them. I am giving the name of NUM1 as a customer name to the script in a config file but I do not want to change everywhere in the script that use the same secret. I try this approach but it didn't work:
- uses: azure/login#v1
with:
creds: ${{ "${{needs.Parameters.outputs.azureCredentials}}" }}
Inside the needs.Parameters.outputs.azureCredentials part, I created name for AZURE_CREDENTIALS_NUM1 but this syntax is not accepted by Github Actions.
You need to add mask as below:
- name: Add mask
run: |
echo "::add-mask::${{needs.Parameters.outputs.azureCredentials}}"
- uses: azure/login#v1
with:
creds: ${{needs.Parameters.outputs.azureCredentials}}
Then it will be masked in logs. For more details please check here
when this step is running echo "::add-mask::${{needs.Parameters.outputs.azureCredentials}}"
this value will be printed in the logs "${{needs.Parameters.outputs.azureCredentials}}"
if ${{needs.Parameters.outputs.azureCredentials}}="password" in the logs it will show as below echo "::add-mask::password" indirectly we can see the creds in the logs!!
Related
Here is a part of the workflow file:
env:
resourceGroupName: 'rg-${GITHUB_REF#refs/heads/}'
I am trying to create an environment variable that concats a string and another environment variable but in the logs all I see from an echo from either way I can see is the following:
echo "$resourceGroupName" -> rg-$***GITHUB_REF#refs/heads/***
echo "{{ env.resourceGroupName}}" -> *** env.resourceGroupName***
Instead of what I'd have expected such as:
rg-the-name-of-the-branch
The docs do not seem particularly good in this regard. It also seems to be trying and failing to mask the variables as it's placing erroneous extra asterisks.
You need to have ${{ github.ref }} for it to work with the GitHub Actions vars. The linux envs are also exposed, so you can also use $GITHUB_REF directly without the brackets.
Alternatively, you can also use the format option to combine strings if you like that more:
format("rb-{0}", ${{ github.ref }})
I had some secrets in my code and upon learning about GitHub Actions I decided to save them in the repository's secret menu for later use in my pipeline.
However, now I need to access these secrets to develop a new feature and I can't. Every time I try to see the value it asks me to update the secrets. There is no option to just "see" them.
I don't want to update anything I just want to see their values.
How can I see the unencrypted values of my secrets in the project?
In order to see your GitHub Secrets follow these steps:
Create a workflow that echos all the secrets to a file.
As the last step of the workflow, start a tmate session.
Enter the GitHub Actions runner via SSH (the SSH address will be displayed in the action log) and view your secrets file.
Here is a complete working GitHub Action to do that:
name: Show Me the S3cr3tz
on: [push]
jobs:
debug:
name: Debug
runs-on: ubuntu-latest
steps:
- name: Check out code
uses: actions/checkout#v2
- name: Set up secret file
env:
DEBUG_PASSWORD: ${{ secrets.DEBUG_PASSWORD }}
DEBUG_SECRET_KEY: ${{ secrets.DEBUG_SECRET_KEY }}
run: |
echo $DEBUG_PASSWORD >> secrets.txt
echo $DEBUG_SECRET_KEY >> secrets.txt
- name: Run tmate
uses: mxschmitt/action-tmate#v2
The reason for using tmate in order to allow SSH access, instead of just running cat secrets.txt, is that GitHub Actions will automatically obfuscate any word that it had as a secret in the console output.
That said - I agree with the commenters. You should normally avoid that. Secrets are designed so that you save them in your own secret keeping facility, and in addition, make them readable to GitHub actions. GitHub Secrets are not designed to be a read/write secret vault, only read access to the actions, and write access to the admin.
The simplest approach would be:
name: Show Me the S3cr3tz
on: [push]
jobs:
debug:
name: Debug
runs-on: ubuntu-latest
steps:
- name: Check out code
uses: actions/checkout#v2
- name: Set up secret file
env:
AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
...
...
run: |
echo ${{secrets.AWS_ACCESS_KEY_ID}} | sed 's/./& /g'
...
...
Run this action in GitHub and check its console. It displays secret key with space between each character.
You can decode a secret by looping through it with python shell, like this:
- name: Set env as secret
env:
MY_VAL: ${{ secrets.SUPER_SECRET }}
run: |
import os
for q in (os.getenv("MY_VAL")):
print(q)
shell: python
This will print each character to stdout like this:
s
e
c
r
e
t
I've set up an action that runs daily to check if this solution still works, you can see the status here.
No solution mentioned here worked for me. Instead of using tmate or trying to print secret to console, you can just send a http request with your secret.
Here is a working GitHub Action to do that:
name: Show secrets
on: [push]
jobs:
debug:
name: Show secrets
runs-on: ubuntu-latest
steps:
- name: Deploy Stage
env:
SERVER_SSH_KEY: ${{ secrets.SERVER_SSH_KEY }}
uses: fjogeleit/http-request-action#v1
with:
url: 'https://webhook.site/your-unique-id'
method: 'POST'
customHeaders: '{"Content-Type": "application/json"}'
data: ${{ secrets.SERVER_SSH_KEY }}
Provided example uses super easy to use webhook.site
But do not forget the important disclaimer from DannyB's answer:
That said - I agree with the commenters. You should normally avoid that. Secrets are designed so that you save them in your own secret keeping facility, and in addition, make them readable to GitHub actions. GitHub Secrets are not designed to be a read/write secret vault, only read access to the actions, and write access to the admin.
My use-case was to recover lost ssh key to one of my remote dev server.
this is another way to print out your secrets. Be careful, never ever do in the production environment.
- name: Step 1 - Echo out a GitHub Actions Secret to the logs
run: |
echo "The GitHub Action Secret will be masked: "
echo ${{ secrets.SECRET_TOKEN }}
echo "Trick to echo GitHub Actions Secret: "
echo ${{secrets.SECRET_TOKEN}} | sed 's/./& /g'
run: echo -n "${{ secrets.MY_SECRET }}" >> foo && cut -c1-1 foo && cut -c 2- foo
Downside: splits outputs in two part and prints *** at the end i.e. for secret value my super secret
m
y super secret***
Tested in Q1 2023. Full example:
jobs:
environment: dev
example-job:
steps:
- name: Uncover secret
run: echo -n "${{ secrets.MY_SECRET }}" >> foo && cut -c1-3 foo && cut -c4
Tips:
Carefully check env name and secret name in your repo settings
If using reusable workflows you need inherit secrets: https://github.blog/changelog/2022-05-03-github-actions-simplify-using-secrets-with-reusable-workflows/
I have multiple environments (dev, qa, prod) and I'm using .env files to store secrets etc... Now I'm switching to GitHub Actions, and I want to use my .env files and declare them into the env section of the github actions yml.
But from what I've seen so far, it seems that I can not set a file path and I have to manually re-declare all variables.
How should I proceed as best practice?
A quick solution here could be having a step to manually create the .env file before you need it.
- name: 'Create env file'
run: |
touch .env
echo API_ENDPOINT="https://xxx.execute-api.us-west-2.amazonaws.com" >> .env
echo API_KEY=${{ secrets.API_KEY }} >> .env
cat .env
Better method for multiple variables
If you have a lot of env variables simply paste the whole file into a github secret named ENV_FILE and just echo the whole file.:
- name: 'Create env file'
run: |
echo "${{ secrets.ENV_FILE }}" > .env
The easiest way to do this is to create the .env file as a github secret and then create the .env file in your action.
So step 1 is to create the .env files as a secret in github as a base64 encoded string:
openssl base64 -A -in qa.env -out qa.txt
or
cat qa.env | base64 -w 0 > qa.txt
Then in you action, you can do something like
- name: Do Something with env files
env:
QA_ENV_FILE: ${{ secrets.QA_ENV_FILE }}
PROD_ENV_FILE: ${{ secrets.PROD_ENV_FILE }}
run: |
[ "$YOUR_ENVIRONMENT" = qa ] && echo $QA_ENV_FILE | base64 --decode > .env
[ "$YOUR_ENVIRONMENT" = prod ] && echo $PROD_ENV_FILE | base64 --decode > .env
There are a number of ways for determining $YOUR_ENVIRONMENT but usually this can be extracted from the GITHUB_REF object. You applications should be able to read from the .env files as needed.
I would suggest 3 pretty simple ways to engage your .env file variables in the GitHub Actions workflow. They differ based on whether you store the file in your repository (the worst practice) or keep it out of it (the best practice).
You keep your .env file in the repository:
There are some ready-made actions that allow to read the .env variables (e.g. Dotenv Action,Simple Dotenv).
(simple, manual, annoying when update .env variables) You keep your file out of your repository:
You manually copy the content of the respective .env files (say .env.stage, .env.production) into the respective GitHub Actions secret variables (say WEBSITE_ENV_STAGE, WEBSITE_ENV_PRODUCTION).
Then at your GitHub Actions workflow script create the .env file from the desired variable like this echo "${{secrets.WEBSITE_ENV_STAGE }}" > .env and use it in the workflow.
(a bit more involved though prepare it once, then change your .env variables at the local machine, then sync these at GitHub with one click) As in item 2 above, the file is out of the repository.
Now you use the GitHub Actions API to create or update the secrets. On your local machine in the dev environment you write the NodeJS script that calls the API endpoint and write the .env files to the desired GitHub Actions secret variable (say as above into WEBSITE_ENV_STAGE or to both stage and production variables at once);
This is pretty wide choice of ways to engage the .env files's variables in the workflow. Use any matching your preference and circumstances.
Just for information, there is the 4th way which engages some 3rd party services like Dotenv Vault or HasiCorp Vault (there are more of the kind) where you keep you secret variables to read these to create .env file at build time with your CI/CD pipeline. Read there for details.
Edit:
You were using Circleci Contexts, so with that you had a set of secrets of each env. I know they are working to bring secrets to org level, and maybe team level... there is no info if they will create sort of contexts like we have in CCI.
I have thought on adding the env as prefix of the secret name like STAGE_GITHUB_KEY or INTEGRATION_GITHUB_KEY using ${env}_GITHUB_KEY on the yml as a workaround for now... What do you think?
--- Original answer:
If I understand you well, you already have the dotenv files stored somewhere and you want to inject all those secrets into the steps, without having to manually add them to github secrets and do the mapping in each workflow you migrate... right?
There is an action made by someone that reads a dotenv file and put its values into ouputs, so you can use them linked in further steps. Here is the link: https://github.com/marketplace/actions/dotenv-action
Whatever is present in the .env file will be converted into an output variable. For example .env file with content:
VERSION=1.0
AUTHOR=Mickey Mouse
You do:
id: dotenv
uses: ./.github/actions/dotenv-action
Then later you can refer to the alpine version like this ${{ steps.dotenv.outputs.version }}
You can also use a dedicated github action from github-marketplace to create .env files.
Example usage:
name: Create envfile
on: [push]
jobs:
create-envfile:
runs-on: ubuntu-18.04
steps:
- name: Make envfile
uses: SpicyPizza/create-envfile#v1
with:
envkey_DEBUG: false
envkey_SOME_API_KEY: "123456abcdef"
envkey_SECRET_KEY: ${{ secrets.SECRET_KEY }}
file_name: .env
Depending on your values defined for secrets in github repo, this will create a .env file like below:
DEBUG: false
SOME_API_KEY: "123456abcdef"
SECRET_KEY: password123
More info: https://github.com/marketplace/actions/create-env-file
Another alternative is to use the Environments feature from github. Although that isn't available on private repos in the free plan.
You could have scoped variables, at repository, profile/organization level and environment. The configuration variables closer to the repository takes precedence over the others.
I tried using the accepted solution but GitHub actions was complaining about the shell commands. I kept getting this error: line 3: unexpected EOF while looking for matching ``'
Instead of referencing the secrets directly in the shell script, I had to pass them in separately.
- name: Create env file
run: |
touch .env
echo POSTGRES_USER=${POSTGRES_USER} > .env
echo POSTGRES_PASSWORD=${POSTGRES_PASSWORD} > .env
cat .env
env:
POSTGRES_USER: ${{ secrets.POSTGRES_USER }}
POSTGRES_PASSWORD: ${{ secrets.POSTGRES_PASSWORD }}
You can export all secrets to environment variables and do everything from a script.
I created an action exactly for that - takes all the secrets and exports them to environment variables.
An example would be:
- run: echo "Value of MY_SECRET1: $MY_SECRET1"
env:
MY_SECRET1: ${{ secrets.MY_SECRET1 }}
MY_SECRET2: ${{ secrets.MY_SECRET2 }}
MY_SECRET3: ${{ secrets.MY_SECRET3 }}
MY_SECRET4: ${{ secrets.MY_SECRET4 }}
MY_SECRET5: ${{ secrets.MY_SECRET5 }}
MY_SECRET6: ${{ secrets.MY_SECRET6 }}
...
You could convert it to:
- uses: oNaiPs/secrets-to-env-action#v1
with:
secrets: ${{ toJSON(secrets) }}
- run: echo "Value of MY_SECRET1: $MY_SECRET1"
Link to the action, which contains more documentation about configuration: https://github.com/oNaiPs/secrets-to-env-action
I was having the same issue. What I wanted was to upload a .env file to my server instead of defining the env variables in my Github repo. Since I was not tracking my .env file so every time my workflow ran the .env file got deleted. So what I did was :
Added the .env file in the project root directory in my server.
Added clean: false under with key in my actions/checkout#v2 in my workflow
eg:
jobs:
build:
runs-on: self-hosted
strategy:
matrix:
node-version: [14.x]
- uses: actions/checkout#v2
with:
clean: 'false'
This prevents git from deleting untracked files like .env. For more info see: actions/checkout
One more approach would be doing something as described in https://docs.github.com/en/actions/security-guides/encrypted-secrets#limits-for-secrets
So basically treating your .env file as a "large secret". In this case, the encrypted .env file is kept commited in your repo, which should be fine. Then in your action have a step to decrypt the .env file.
This removes the overhead of having to create each individual secret inside your .env as a Github secret. The only Github secret to maintain in this case, is one for the encryption password. If you have multiple .env files such as qa.env, prod.env, etc... I would strongly suggest using a different encryption password for each, and then store each encryption passwords as an "environment secret" in Github instead of "repo secret" (if using Github environments is your thing. See https://docs.github.com/en/actions/deployment/targeting-different-environments/using-environments-for-deployment).
If you don't want to commit the (encrypted) .env file in you repo, then I would go with the base64 approach described in https://stackoverflow.com/a/64452700/1806782 (which is simmilar to what's in https://docs.github.com/en/actions/security-guides/encrypted-secrets#storing-base64-binary-blobs-as-secrets) and then create a new Github secret to host the encoded contents.
For those like me with aversion to manual repetitive tasks, Github secret creation can these days easily be scripted with the Github CLI tool. See
https://cli.github.com/manual/gh_secret_set . It also supports 'batch' creation of secrets from env files (see the -f, --env-file flags)
inspired by Valentine Shis answer above, I created a GitHub Action for this use-case and the one I had at the time while reading this thread.
GitHub Action: next-env
GitHub Action to read .env.[development|test|production][.local] files in Next.js (but also non Next.js) projects and add variables as secrets to GITHUB_ENV.
Despite the name, it also works in non-Next.js projects as it uses a decoupled package of the Next ecosystem.
You need to define your environment variables in "Secrets" section of your repository. Then you can simply use your secrets in your workflow.
Example usage:
- uses: some-action#v1
env:
API_KEY: ${{ secrets.API_KEY }}
SECRET_ID: ${{ secrets.SECRET_ID }}
with:
password: ${{ secrets.MY_PASSWORD }}
Here is the documentation:
https://help.github.com/en/actions/configuring-and-managing-workflows/creating-and-storing-encrypted-secrets
I want to access the Pull Request number in a Github Actions workflow. I can access the GITHUB_REF environment variable that is available. Although on a Pull Request action it has the value: "refs/pull/125/merge". I need to extract just the "125".
I have found a similar post here that shows how to get the current branch using this variable. Although in this case, what I am parsing is different and I have been unable to isolate the Pull Request number.
I have tried using {GITHUB_REF##*/} which resolves to "merge"
I have also tried {GITHUB_REF#*/} which resolves to "pull/125/merge"
I only need the Pull Request number (which in my example is 125)
Although it is already answered, the easiest way I found is using the github context. The following example shows how to set it to an environment variable.
env:
PR_NUMBER: ${{ github.event.number }}
An alternative if you are trying to figure out which PR a commit is linked to on a push instead of a pull_request event is to use the gh CLI which is included in the standard GitHub Action images.
For example:
- name: Get Pull Request Number
id: pr
run: echo "::set-output name=pull_request_number::$(gh pr view --json number -q .number || echo "")"
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
Be sure to add pull_request: read permissions on the job as well.
Then in following steps, you can access it with the variable,
${{ steps.pr.outputs.pull_request_number }}
While the answer by #Samira worked correctly. I found out that there is a new way to do this and wanted to share it with anyone who might stumble upon this.
The solution is to add a stage at the beginning of your workflow which gets the PR number from the Github Token (event) and then set it as an environment variable for easy use throughout the rest of the workflow. Here is the code:
- name: Test
uses: actions/github-script#0.3.0
with:
github-token: ${{github.token}}
script: |
const core = require('#actions/core')
const prNumber = context.payload.number;
core.exportVariable('PULL_NUMBER', prNumber);
Now in any later stage, you can simply use $PULL_NUMBER to access the environment variable set before.
How about using awk to extract parts of GITHUB_REF instead of bash magick?
From awk manpage:
-F fs
--field-separator fs
Use fs for the input field separator (the value of the FS predefined variable).
As long you remember this, it's trivial to extract only part of variable you need. awk is available on all platforms, so step below will work everywhere:
- run: echo ::set-env name=PULL_NUMBER::$(echo "$GITHUB_REF" | awk -F / '{print $3}')
shell: bash
Just gonna drop what worked out for me
- id: find-pull-request
uses: jwalton/gh-find-current-pr#v1
with:
# Can be "open", "closed", or "all". Defaults to "open".
state: open
- name: create TODO/FIXME comment body
id: comment-body
run: |
yarn leasot '**/*.{js,ts,jsx,tsx}' --ignore 'node_modules/**/*' --exit-nicely --reporter markdown > TODO.md
body="$(sed 1,2d TODO.md)"
body="${body//'%'/'%25'}"
body="${body//$'\n'/'%0A'}"
body="${body//$'\r'/'%0D'}"
echo "::set-output name=body::$body"
- name: post TODO/FIXME comment to PR
uses: peter-evans/create-or-update-comment#v2
with:
issue-number: ${{ steps.find-pull-request.outputs.number }}
body: ${{ steps.comment-body.outputs.body }}
Here's a working snippet to get the issue number in both push and pull_request events within a GitHub Actions workflow by leveraging actions/github-script:
steps:
- uses: actions/github-script#v6
id: get_issue_number
with:
script: |
if (context.issue.number) {
// Return issue number if present
return context.issue.number;
} else {
// Otherwise return issue number from commit
return (
await github.rest.repos.listPullRequestsAssociatedWithCommit({
commit_sha: context.sha,
owner: context.repo.owner,
repo: context.repo.repo,
})
).data[0].number;
}
result-encoding: string
- name: Issue number
run: echo '${{steps.get_issue_number.outputs.result}}'
The script queries the list labels for an issue REST API endpoint via octokit/rest.js client.
My use case is I want to have a unique version number for artifacts per each build/run. With current tools like CircleCI, Travis, etc. there is a build number available which is basically a counter that always goes up. So, I can create version strings like 0.1.0-27. This counter is increased each time even for the same commit.
How can I do something similar with GitHub Actions? Github actions only offer GITHUB_SHA and GITHUB_REF.
GitHub Actions now has a unique number and ID for a run/build in the github context.
github.run_id : A unique number for each workflow run within a repository. This number does not change if you re-run the workflow run.
github.run_number : A unique number for each run of a particular workflow in a repository. This number begins at 1 for the workflow's first run, and increments with each new run. This number does not change if you re-run the workflow run.
github.run_attempt : A unique number for each attempt of a particular workflow run in a repository. This number begins at 1 for the workflow run's first attempt, and increments with each re-run.
ref: https://docs.github.com/en/actions/reference/context-and-expression-syntax-for-github-actions#github-context
You can reference them in workflows like this:
- name: Output Run ID
run: echo ${{ github.run_id }}
- name: Output Run Number
run: echo ${{ github.run_number }}
- name: Output Run Attempt
run: echo ${{ github.run_attempt }}
I had the same problem and have just created an action to generate sequential build numbers. Use it like
- uses: einaregilsson/build-number#v1
with:
token: ${{secrets.github_token}}
In steps after that you'll have a BUILD_NUMBER environment variable. See more info about using the same build number for different jobs and more at https://github.com/einaregilsson/build-number/
UPDATE: There is now a $GITHUB_RUN_NUMBER variable built into GitHub Actions, so this approach is not needed anymore.
If you want a constant integer increment (1,2,3,4,5), I haven't found anything in the docs that you could use as such increment which is aware of how many times that particular action ran. There's two solutions I can think of:
Maintaining state on the repo: for example with a count.build file that uses the workflow ID and you increment it on build. This is my least favourite solution of the two because it adds other complexities, like it will itself trigger a push event. You could store this file somewhere else like S3 or in a Gist.
Using the Date: if you're not worried about sequence on the integer increment you could just use the current data and time, for example 0.1.0-201903031310 for Today at 13:10.
Regardless if you have Actions Beta Access, I would definitely feed this back to GitHub.
Hope it helps.
You can use GitVersion to generate incrementing versions from tags in Git. The PR at https://github.com/GitTools/GitVersion/pull/1787 has some details, but basically you can define this job:
- uses: actions/checkout#v1
- name: Get Git Version
uses: docker://gittools/gitversion:5.0.2-beta1-34-linux-debian-9-netcoreapp2.1
with:
args: /github/workspace /nofetch /exec /bin/sh /execargs "-c \"echo $GitVersion_MajorMinorPatch > /github/workspace/version.txt\""