Value not set using $GITHUB_OUTPUT - powershell

I have been previously using set-output for setting values, but we now get thee "deprecated feature" messages and I'm using $GITHUB_OUTPUT as prescribed.
I replace all instances of
run: echo ::set-output name=Key::Value
with
run: "Key=Value" >> $GITHUB_OUTPUT
but Key does not appear to be set.
My runner is on Windows, version 2.299.1 and the workflow is using CMD.
All calls to set-output work, and all using $GITHUB_OUTPUT do not.
Simplified action code
defaults:
run:
shell: cmd
jobs:
EnvSetup:
name: Publish Base Environment Vars
runs-on: [self-hosted, Windows, myLabel]
outputs:
var_Project: ${{ steps.set-Project.outputs.Project }}
var_Val1: ${{ steps.set-Val1.outputs.Val1 }}
var_Val2: ${{ steps.set-Val2.outputs.Val2 }}
steps:
- name: Project
id: set-Project
run: echo ::set-output name=Project::Larry
- name: Val1
id: set-Val1
run: echo "Val1=Curly" >> $GITHUB_OUTPUT
- name: Val2
id: set-Val2
run: echo "Val2=Moe" >> $GITHUB_OUTPUT
...
Testing:
name: ShowStuff
runs-on: [self-hosted, Windows, myLabel]
needs: [EnvSetup]
env:
MyProject: ${{ needs.EnvSetup.outputs.var_Project }}_ABC
steps:
- name: Print environment variables
run: |
echo "Project: ${{ needs.EnvSetup.outputs.var_Project }}" ^
echo "MyProject: ${{ env.MyProject }}" ^
echo "Val1: ${{ needs.EnvSetup.outputs.var_Val1 }}" ^
echo "Val2: ${{ needs.EnvSetup.outputs.var_Val2 }}"
The output:
echo "Project: Larry"
echo "MyProject: Larry_ABC"
echo "Val1: "
echo "Val2: "
From everything I've seen, the way to reference the values hasn't changed, just the set.
Has anyone else tried it using CMD? I'll go to PowerShell if I have to, but that's not a small change if I can avoid it.

Windows run the script task using PowerShell Core by default, not bash. So you need to use PowerShell syntax, or set the shell: bash property on the script action.
- name: Val2
id: set-Val2
run: echo "Val2=Moe" >> $GITHUB_OUTPUT
shell: bash
When using these commands with PowerShell, make sure you redirect to $env:GITHUB_OUTPUT:
- name: Val2
id: set-Val2
run: echo "Val2=Moe" >> $env:GITHUB_OUTPUT
shell: pwsh
I also explicitly added shell: pwsh above, as the "old PowerShell" needs to be told to write UTF-8:
- shell: powershell
run: |
"mypath" | Out-File -FilePath $env:GITHUB_PATH -Encoding utf8 -Append
When using shell: cmd you'd need to use %GITHUB_OUTPUT%, and change the codepage to Unicode:
#chcp 65001>nul
echo Val2=Moe >> %GITHUB_OUTPUT%

Related

Using external action output in outer git action step

I have this git action for my build
...
- name: Building S3 Instance
uses: charlie87041/s3-actions#main
id: s3
env:
AWS_S3_BUCKET: 'xxx'
AWS_ACCESS_KEY_ID: 'xxx'
AWS_SECRET_ACCESS_KEY: 'xxxxx'
AWS_REGION: 'xxx'
- name: Updating EC2 [Develop] instance
uses: appleboy/ssh-action#master
with:
host: ${{secrets.EC2HOST}}
key: ${{secrets.EC2KEY}}
username: xxx
envs: TESTING
script: |
cd ~/devdir
export BUCKET_USER=${{steps.s3.outputs.user_id}}
export BUCKET_USER_KEY=${{steps.s3.outputs.user_key}}
docker login
docker-compose down --remove-orphans
docker system prune -a -f
docker pull yyyy
docker-compose up -d
And this is the important function in charlie87041/s3-actions#main
generate_keys () {
RSP=$(aws iam create-access-key --user-name $USER);
BUCKET_ACCESS_ID=$(echo $RSP | jq -r '.AccessKey.AccessKeyId');
BUCKET_ACCESS_KEY=$(echo $RSP | jq -r '.AccessKey.SecretAccessKey');
echo "user_id=$BUCKET_ACCESS_ID" >> $GITHUB_OUTPUT
echo "user_key=$BUCKET_ACCESS_KEY" >> $GITHUB_OUTPUT
echo "::set-output name=BUCKET_ACCESS_KEY::$BUCKET_ACCESS_KEY"
echo "::set-output name=BUCKET_ACCESS_ID::$BUCKET_ACCESS_ID"
}
I need to update env variables in container with BUCKET_USER and BUCKET_USER_KEY, but these always return null when echo the container. How do I do this?
Not that set-output was deprecated recently (oct. 2022)
If you are using self-hosted runners make sure they are updated to version 2.297.0 or greater.
If you are using runner on github.com directly, you would need to change
echo "::set-output name=BUCKET_ACCESS_KEY::$BUCKET_ACCESS_KEY"
with
echo "BUCKET_ACCESS_KEY=$BUCKET_ACCESS_KEY" >> $GITHUB_OUTPUT
I am not sure an export within the script would work.
Using with directives, as in issue 154 might be more effective
with:
BUCKET_USER: ${{steps.s3.outputs.user_id}}
...
script: |
...

Introducing secret variables to dockerfile on Github Actions

I am trying to configure my etc/pip.conf file to download a private PyPi artifactory while using a secret variable on my dockerfile.
Dockerfile
FROM python
WORKDIR ./app
COPY . /app
RUN pip install --upgrade pip
RUN pip install -r pre-requirements.txt
RUN echo ${{ secrets.PIP }} > etc/pip.conf
RUN pip install -r post-requirements.txt
CMD ["python", "./simpleflask.py"]
docker-image.yml
name: CI
on:
push:
branches: [ "main" ]
pull_request:
branches: [ "main" ]
workflow_dispatch:
jobs:
build:
runs-on: ubuntu-latest
steps:
- name: Setup JFrog CLI
uses: jfrog/setup-jfrog-cli#v2
env:
JF_ARTIFACTORY_SERVER: ${{ secrets.JFROG_CLI }}
- name: Checkout
uses: actions/checkout#v3
- name: Build
run: |
docker build -t simple-flask .
docker tag simple-flask awakzdev.jfrog.io/docker-local/simple-flask:latest
docker push awakzdev.jfrog.io/docker-local/simple-flask:latest
pretty simple and straightfoward but my pipeline returns the following
Step 6/8 : RUN echo ${{ secrets.PIP }} > etc/pip.conf
---> Running in deb3e3f4167f
/bin/sh: 1: Bad substitution
The command '/bin/sh -c echo ${{ secrets.PIP }} > etc/pip.conf' returned a non-zero code: 2
Error: Process completed with exit code 2.
Edit :
Trying a slightly difference approach and went to install dependencies in the pipeline
my .yml looks like this now
jobs:
build:
runs-on: ubuntu-latest
steps:
- name: Setup JFrog CLI
uses: jfrog/setup-jfrog-cli#v2
env:
JF_ARTIFACTORY_SERVER: ${{ secrets.JFROG_CLI }}
- name: Checkout
uses: actions/checkout#v3
- name: install dependencies
run: |
pip config -v list
echo "${{ secrets.PIP }}" > /etc/pip.conf
pip install ganesha-experimental==2.0.1
- name: Build
run: |
docker build -t simple-flask .
docker tag simple-flask awakzdev.jfrog.io/docker-local/simple-flask:latest
docker push awakzdev.jfrog.io/docker-local/simple-flask:latest
but the following error is being returned:
1s
Run pip config -v list
For variant 'global', will try loading '/etc/xdg/pip/pip.conf'
For variant 'global', will try loading '/etc/pip.conf'
For variant 'user', will try loading '/home/runner/.pip/pip.conf'
For variant 'user', will try loading '/home/runner/.config/pip/pip.conf'
For variant 'site', will try loading '/usr/pip.conf'
/home/runner/work/_temp/09382b8f-ce09-4646-816f-fb337f40ad4b.sh: line 2: /etc/pip.conf: Permission denied
Error: Process completed with exit code 1.
I've placed the secret on my .yml file instead.
as for the broken pip permissions I used
sudo chown runner /etc/
echo ${{ secrets.PIP }} > /etc/pip.conf
which resulted in another error with the contents of the pip.conf file (it was configured correctly through secrets)
so I found you can specify the url like so
ganesha_experimental==5.0.0 --find-links=https://awakzdev.jfrog.io/artifactory/

How to package App.zip artifact in an installer in Github Actions

I'm a unity developer, and I am new to devops. I am using Github Actions to make a prototype CI/CD pipeline for a project. Currently I can build and attach the build artifact to a release after every push. When I download this from the release, I get a .zip file with my project in it. This is expected, as it is how Unity builds games natively.
I want to make an installer for the game, so users do not have to open up a .zip file to find the game executable. I have watched a couple videos on how to do this manually using Inno and InstallCreator2, but I do not know how to do this using my CI/CD pipe (nor do I know if either of those technologies would be best for my use case)
Here is my yaml code. I made it following documentation and tutorials. I have a vague idea of what individual lines, steps, and jobs are doing, but my understanding is rudimentary at best. If something is terribly inefficient or otherwise bad, I probably don't know. I have added a couple comments to aid in my own understanding.
TL;DR: What job/ steps do I add to make the .zip output to an installer.exe type file?
EDIT: (updated YAML code) So, over the past day I tried creating an installer with inno on my local machine, and it worked! Pretty neat. Then I quickly descended into madness trying to edit the hardcode fields in the .iss file in my build server using YAML. At the end of the day it seems that it is rather challenging to get Inno itself on the build server (they have some instructions on the Inno github, but I got stuck trying to figure out how to apt-get Embarcadero Delphi.) Additionally, about 8 hours in I had the thought that "maybe trying to change the filepaths on a script generated in windows, so it is compatible with linux isn't the best way to go." Is there a linux-to-windows installer script generator? I was looking at this link earlier, but I feel like I am lacking a bunch of information to actually implement it.
name: Build project
on:
[push] #Comment this in (and add more triggers if you want), to have the job run on the set triggers (instead of manual). if using, might want to have this run on main branch merge only.
#workflow_dispatch: {} #Comment this in for a manual push button in github actions
jobs:
buildForAllSupportedPlatforms:
name: Build for ${{ matrix.targetPlatform }}
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
targetPlatform:
#- StandaloneOSX # Build a macOS standalone (Intel 64-bit).
#- StandaloneWindows # Build a Windows standalone.
- StandaloneWindows64 # Build a Windows 64-bit standalone.
#- StandaloneLinux64 # Build a Linux 64-bit standalone.
#- iOS # Build an iOS player.
#- Android # Build an Android .apk standalone app.
#- WebGL # WebGL.
steps:
- name: checkout
uses: actions/checkout#v2
with:
fetch-depth: 0
lfs: true
- name: cache
uses: actions/cache#v2
with:
path: Library
key: Library-${{ matrix.targetPlatform }}
restore-keys: Library-
- name: build unity project
uses: game-ci/unity-builder#v2
env:
UNITY_LICENSE: ${{ secrets.UNITY_LICENSE }}
with:
targetPlatform: ${{ matrix.targetPlatform }}
- name: upload build artifact
uses: actions/upload-artifact#v2
with:
name: Build-${{ matrix.targetPlatform }}
path: build/${{ matrix.targetPlatform }}
- name: Set Up Enviorment Varibles for installers
run: |
echo "pathStartpoint=$(pwd)" >> $GITHUB_ENV
cd .installer
echo "installedDirPath=$(pwd)" >> $GITHUB_ENV
echo "appVersion=$(less VersionInfo.txt)" >> $GITHUB_ENV
echo "iconPath=$(ls -d *.ico | tail -n +1 | head -1)" >> $GITHUB_ENV
echo "appId=$(uuidgen)" >> $GITHUB_ENV
cd ../build
echo "outputDir=$(pwd)" >> $GITHUB_ENV
- name: Create Windows Installer using Inno script
if: matrix.targetPlatform == 'StandaloneWindows64' #Conditional task
run: |
cd ${{ env.pathStartpoint }}
cd build/StandaloneWindows64
echo 'refactoring build file arrangment for inno file'
sudo mkdir tempDir1 tempDir2
sudo mv ${{ matrix.targetPlatform }}_Data /tempDir1
sudo mv tempDir1 ${{ matrix.targetPlatform }}_Data
sudo mv MonoBleedingEdge /tempDir2
sudo mv tempDir2 MonoBleedingEdge
echo 'finished refactoring build file arrangment'
echo 'Making a copy of WindowsInstaller64.iss'
cd ../..
sudo cp ./.installer/WindowsInstaller64.iss ./build
echo 'Finished making copy'
echo 'setting installer values'
cd ./build
sudo sed -i 's/#define MyAppName "UNITYWINDOWSBUILD"/#define MyAppName "${{ github.event.repository.name }}"/g' WindowsInstaller64.iss
echo 'set MyAppName'
sudo sed -i 's/#define MyAppVersion "1.11"/#define MyAppVersion "${{ env.appVersion }}"/g' WindowsInstaller64.iss
echo 'set MyAppVersion'
sudo sed -i 's/AppId={{76999C4F-90E2-49D0-8EF2-C315F16CEAD6}/AppId={{"${{ env.appId }}"}/g' WindowsInstaller64.iss
echo 'set AppId'
sudo sed -i 's,LicenseFile=EXAMPLELicense.txt,LicenseFile=${{ env.installedDirPath }}/License.txt,g' WindowsInstaller64.iss
echo 'set LicenseFile path'
sudo sed -i 's,InfoBeforeFile=EXAMPLEPreInstall.txt,InfoBeforeFile=${{ env.installedDirPath }}/PreInstall.txt,g' WindowsInstaller64.iss
echo 'set PreInstallFile path'
sudo sed -i 's,InfoAfterFile=EXAMPLEPostInstall.txt,InfoAfterFile=${{ env.installedDirPath }}/PostInstall.txt,g' WindowsInstaller64.iss
echo 'set PostInstallFile path'
sudo sed -i 's,OutputDir=EXAMPLEDIR,OutputDir=${{ env.outputDir }},g' WindowsInstaller64.iss
echo 'set Output path'
sudo sed -i 's,OutputBaseFilename=EXAMPLEPROJ_Setup(x64),OutputBaseFilename=StandaloneWindows_Setup(x64),g' WindowsInstaller64.iss
echo 'set OutputFile name'
sudo sed -i 's,SetupIconFile=EXAMPLEICONPATH.ico,SetupIconFile=${{ env.installedDirPath }}/${{ env.iconPath }},g' WindowsInstaller64.iss
echo 'set Icon file path name'
sudo sed -i 's,Source: "EXAMPLEPATH/{#MyAppExeName}",Source: "${{ env.pathStartpoint }}/build/StandaloneWindows64/{#MyAppExeName}",g' WindowsInstaller64.iss
echo 'set executable path'
sudo sed -i 's,Source: "EXAMPLEPATH/UnityCrashHandler64.exe",Source: "${{ env.pathStartpoint }}/build/StandaloneWindows64/UnityCrashHandler64.exe",g' WindowsInstaller64.iss
echo 'set Crash Handler path'
sudo sed -i 's,Source: "EXAMPLEPATH/UnityPlayer.dll",Source: "${{ env.pathStartpoint }}/build/StandaloneWindows64/UnityPlayer.dll",g' WindowsInstaller64.iss
echo 'set unity player path'
sudo sed -i 's,Source: "EXAMPLEPATH/StandaloneWindows64_Data/*",Source: "${{ env.pathStartpoint }}/build/StandaloneWindows64/StandaloneWindows64_Data/*"",g' WindowsInstaller64.iss
echo 'set build Data path'
sudo sed -i 's,Source: "EXAMPLEPATH/MonoBleedingEdge/*",Source: "${{ env.pathStartpoint }}/build/StandaloneWindows64/MonoBleedingEdge/*",g' WindowsInstaller64.iss
echo 'set monoBleedingEdge path'
echo 'Finished setting installer values'
less WindowsInstaller64.iss
echo downloading Inno
cd ~
sudo git clone https://github.com/jrsoftware/issrc.git is
cd is
sudo git submodule init
sudo git submodule update
sudo iscc ${{ env.pathStartpoint }}/build/WindowsInstaller64.iss
#Add another upload build artifact step to upload the windows installer
release-project:
name: release for ${{ matrix.targetPlatform }}
runs-on: ubuntu-latest
needs: buildForAllSupportedPlatforms
strategy:
fail-fast: false
matrix:
targetPlatform:
#- StandaloneOSX # Build a macOS standalone (Intel 64-bit).
#- StandaloneWindows # Build a Windows standalone.
- StandaloneWindows64 # Build a Windows 64-bit standalone.
#- StandaloneLinux64 # Build a Linux 64-bit standalone.
#- iOS # Build an iOS player.
#- Android # Build an Android .apk standalone app.
#- WebGL # WebGL.
steps:
- name: Download Artifact
uses: actions/download-artifact#v3
with:
name: Build-${{ matrix.targetPlatform }}
- name: Archive Artifact Content
uses: thedoctor0/zip-release#master
with:
filename: ${{ matrix.targetPlatform }}.zip
- name: Create Github Release
id: create-new-release
uses: "marvinpinto/action-automatic-releases#latest" #NOTE: If this step breaks, it may be because the latest commit to marvinpinto/... broke the action. refactor this to use stable version instead of #latest
with:
repo_token: "${{ secrets.GITHUB_TOKEN }}" # This token is provided by Actions, you do not need to create your own token
automatic_release_tag: "latest-${{ matrix.targetPlatform }}"
prerelease: false
title: "Release-${{ matrix.targetPlatform }}-v${{ github.run_number }}"
- name: Upload Release Asset Versioned #This attaches the built .zip file to the release with the version in the name
uses: actions/upload-release-asset#v1
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
with:
upload_url: ${{ steps.create-new-release.outputs.upload_url }} # This pulls from the CREATE GITHUB RELEASE step above, referencing it's ID to get its outputs object, which include a `upload_url`. See this blog post for more info: https://jasonet.co/posts/new-features-of-github-actions/#passing-data-to-future-steps
asset_path: ./${{ matrix.targetPlatform }}.zip
asset_name: build_${{ matrix.targetPlatform }}-v${{ github.run_number }}.zip
asset_content_type: application/zip
- name: Upload Release Asset Static #This attaches the built .zip file to the release without the version in the name. This is good for a consitant download URL for sites
uses: actions/upload-release-asset#v1
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
with:
upload_url: ${{ steps.create-new-release.outputs.upload_url }} # This pulls from the CREATE GITHUB RELEASE step above, referencing it's ID to get its outputs object, which include a `upload_url`. See this blog post for more info: https://jasonet.co/posts/new-features-of-github-actions/#passing-data-to-future-steps
asset_path: ./${{ matrix.targetPlatform }}.zip
asset_name: build_${{ matrix.targetPlatform }}-latest.zip
asset_content_type: application/zip

How to limit pytest code coverage calculation to only specific files?

I am trying to setup continues integration in github using actions. I setup a action to get the code coverage automatically, this part works great, however it's calculating the code coverage for every file in my src directory, this is not preferred, I would like it to calculate the code coverage for only the files modified, how do I do that?
name: Pytest Coverage
on:
pull_request:
branches: [ main, dev ]
jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout#v2
- name: Set up Python 3.10
uses: actions/setup-python#v2
with:
python-version: "3.10"
- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install flake8 pytest pytest-cov
if [ -f requirements.txt ]; then pip install -r requirements.txt; fi
- name: Changed Files Exporter
id: files
uses: umani/changed-files#v3.3.0
with:
repo-token: ${{ github.token }}
pattern: '^src.*\.(py)$'
result-encoding: 'string'
- name: Build coverage file
run: |
echo "${{ steps.files.outputs.files_updated }} ${{ steps.files.outputs.files_created }}"
pytest --cache-clear --cov-branch --cov=src > pytest-coverage.txt
- name: Comment coverage
uses: coroo/pytest-coverage-commentator#v1.0.2
- name: Get Coverage %
run: |
LAST_LINE=$(tail -4 pytest-coverage.txt)
LAST_LINE=$(head -n 1 <<< "$LAST_LINE")
COVERAGE=$(sed 's/.*[[:space:]]\([0-9]\+\)%/\1/' <<< "$LAST_LINE")
echo "Overall code coverage is $COVERAGE"
echo 'SCORE<<EOF' >> $GITHUB_ENV
echo "$SCORE" >> $GITHUB_ENV
echo 'EOF' >> $GITHUB_ENV
The important 2 lines are here:
echo "${{ steps.files.outputs.files_updated }} ${{ steps.files.outputs.files_created }}"
pytest --cache-clear --cov-branch --cov=src > pytest-coverage.txt
The echo print out the correct files:
src/scripts/testing_script.py src/server.py
How do I change this line so it ONLY calculate the total coverage of thoese 2 modified files?
pytest --cache-clear --cov-branch --cov=src > pytest-coverage.txt

How to mask a environment set in GitHub Workflow

I am using the following to set a env variable:
- name: get root token
run: |
echo "ROOT_TOKEN=$(some command | base64 --decode)" >> $GITHUB_ENV
Then I use it in another run within a Python script:
- name: init
run: |
python3 scripts/create_entries.py
Actually this works great, but the value of ROOT_TOKEN is printed in the Workflow console:
Run python3 scripts/create_entries.py
python3 scripts/create_entries.py
shell: /usr/bin/bash -e ***0***
env:
DATA: ***
CONFIG: /home/debian/runner/_work/_temp/config_1646400032032
ROOT_TOKEN: <this is shown>
I tried to mask it using ::add-mask:: like this (but unfortunately that does not work):
- name: get root token
run: |
echo "::add-mask::ROOT_TOKEN=$(some command | base64 --decode)" >> $GITHUB_ENV
Does anyone know how to mask the value of ROOT_TOKEN in the Workflow console?