I have a self hosted Github Action runner on a Windows server, my goal is to migrate from Jenkins to GitHub's workflow. I have a very hard to time understand how to use the environment variables for some reason.
I have a folder on my self hosted server, where I want to place some generic build scripts, which is created in Powershell.
So my main.yml look like this
# This is a basic workflow to help you get started with Actions
name: Build integration
# Controls when the workflow will run
on:
# Triggers the workflow on push or pull request events but only for the "develop" branch
push:
branches: ["*"]
pull_request:
branches: ["*"]
# Allows you to run this workflow manually from the Actions tab
workflow_dispatch:
# A workflow run is made up of one or more jobs that can run sequentially or in parallel
jobs:
# This workflow contains a single job called "build"
continuous-integration:
# The type of runner that the job will run on
runs-on: [self-hosted, .net]
# Steps represent a sequence of tasks that will be executed as part of the job
steps:
# Checks-out your repository under $GITHUB_WORKSPACE, so your job can access it
- name: Checkout code
uses: actions/checkout#v3
# get environment
- name: Get system environments
uses: FranzDiebold/github-env-vars-action#v2
# show environment
- name: Show all enronment variables
run: SET
shell: cmd
# read meta file
- name: read project meta file
run: |
.\$BUILD_SCRIPTS_PATH}\Powershell\meta.ps1 -MetaFilePath ProjectMeta.json
shell: powershell
I have tried all kinds of variants to get the environment variable BUILD_SCRIPTS_PATH out.
.\${env.BUILD_SCRIPTS_PATH}\Powershell\meta.ps1 -MetaFilePath ProjectMeta.json
.\${{BUILD_SCRIPTS_PATH}}\Powershell\meta.ps1 -MetaFilePath ProjectMeta.json
.\$env.BUILD_SCRIPTS_PATH\Powershell\meta.ps1 -MetaFilePath ProjectMeta.json
.\$BUILD_SCRIPTS_PATH\Powershell\meta.ps1 -MetaFilePath ProjectMeta.json
I keep getting this error
+ ${BUILD_SCRIPTS_PATH}\Powershell\meta.ps1 -MetaFilePath ProjectMe ...
+ ~~~~~~~~~~~~~~~~~~~~
Unexpected token '\Powershell\meta.ps1' in expression or statement.
+ CategoryInfo : ParserError: (:) [], ParseException
+ FullyQualifiedErrorId : UnexpectedToken
a minor part of the output from the SET command
Run SET
ALLUSERSPROFILE=C:\ProgramData
APPDATA=C:\Users\Administrator\AppData\Roaming
BUILD_SCRIPTS_PATH=E:\github\BuildScripts
ChocolateyInstall=C:\ProgramData\chocolatey
I have looked into the documentation from GitHub GitHub Documentation but I don't really understand what i am doing wrong. I would really appreciate is someone could explain to me what i am doing wrong.
You can read more about using environment variables in GitHub actions here.
From what you've posted it doesn't seem like you're providing the BUILD_SCRIPTS_PATH to the workflow's environment.
You can add it like this:
# read meta file
- name: read project meta file
run: |
.\$BUILD_SCRIPTS_PATH\Powershell\meta.ps1 -MetaFilePath ProjectMeta.json
shell: powershell
env:
BUILD_SCRIPTS_PATH: ${{ secrets:build_scripts_path}}
Then just make sure to add the build_scripts_path as a GitHub secret.
Related
I am trying to setup my first GitHub Workflow and I am facing many YAML syntax issues even I am using the official documentation.
I am using the below YAML:
# This is a basic workflow to help you get started with Actions
name: TestWorkflowGithub
# Controls when the workflow will run
on:
# Triggers the workflow on push or pull request events but only for the "main" branch
pull_request:
branches:
- 'testbranch/**'
# A workflow run is made up of one or more jobs that can run sequentially or in parallel
jobs:
# The type of runner that the job will run on
runs-on: ubuntu-latest
# Steps represent a sequence of tasks that will be executed as part of the job
steps:
- name: Checkout the code
uses: actions/checkout#v3
- name: Install PMD
run: |
PMD_VERSION=`cat pmd/pmd-version.txt`
wget https://github.com/pmd/pmd/releases/download/pmd_releases%2F6.54.0/pmd-bin-6.54.0.zip
unzip pmd-bin-6.54.0.zip -d ~
mv ~/pmd-bin-$6.54.0 ~/pmd
~/pmd/bin/run.sh pmd --version
# Run PMD scandd
- name: Run PMD scan
run: ~/pmd/bin/run.sh pmd -d force-app -R pmd/ruleset.xml -f text
GitHub is showing me the below error:
You have an error in your yaml syntax on line 14
Note: the line 14 is "runs-on: ubuntu-latest"
Which is the syntax issue in the above YAML file?
You are missing the job identifier:
jobs:
foo: # <-- This
runs-on: ubuntu-latest
steps:
- name: Checkout the code
uses: actions/checkout#v3
steps:
You can use actionlint or vscode-yaml to avoid such syntax issues next time :)
I have this workflow in a repo called terraform-do-database and I'm trying to use a reusable workflow coming from the public repo foo/git-workflows/.github/workflows/tag_validation.yaml#master
name: Tag Validation
on:
pull_request:
branches: [master]
push:
branches:
- '*' # matches every branch that doesn't contain a '/'
- '*/*' # matches every branch containing a single '/'
- '**' # matches every branch
- '!master' # excludes master
# Allows you to run this workflow manually from the Actions tab
workflow_dispatch:
jobs:
tag_check:
uses: foo/git-workflows/.github/workflows/tag_validation.yaml#master
And this is the reusable workflow file from the public git-workflows repo that has the script that should run on it. What is happening is that the workflow is trying to use a script inside the repo terraform-do-database
name: Tag Validation
on:
pull_request:
branches: [master]
workflow_call:
jobs:
tag_check:
# The type of runner that the job will run on
runs-on: ubuntu-latest
# Steps represent a sequence of tasks that will be executed as part of the job
steps:
# Checks-out your repository under $GITHUB_WORKSPACE, so your job can access it
- uses: actions/checkout#v3
# Runs a single command using the runners shell
- name: Verify the tag value
run: ./scripts/tag_verify.sh
So the question: How can I make the workflow use the script stored in the git-worflows repo instead of the terraform-do-database?
I want to have a single repo where I can call the workflow and the scripts, I don't want to have everything duplicated inside all my repos.
I have found that if I wrap the script into a composite action. I can use GitHub context github.action_path to locate the scripts.
Example:
run: ${{ github.action_path }}/scripts/foo.sh
One way to go about this is perform a checkout inside your reusable workflow that essentially clones the content of the repo where your scripts are and only then you can access it. It's not the cleanest solution but it works.
Perform a second checkout, to clone your repo that has the reusable workflow into a dir reusable-workflow-repo
- name: Checkout reusable workflow dir
uses: actions/checkout#v3
with:
repository: <your-org>/terraform-do-database
token: ${{ secrets.GIT_ACCESS_TOKEN }}
path: reusable-workflow-repo
Now you have all the code you need inside reusable-workflow-repo. Use ${GITHUB_WORKSPACE} to find the current path and simply append the path to the script.
- name: Verify the tag value
run: ${GITHUB_WORKSPACE}/reusable-workflow-repo/scripts/tag_verify.sh
I was able to solve it adding a few more commands to manually download the script and execute it.
steps:
# Checks-out your repository under $GITHUB_WORKSPACE, so your job can access it
- uses: actions/checkout#v3
# Runs a single command using the runners shell
- name: Check current directory
run: pwd
- name: Download the script
run: curl -o $PWD/tag_verify.sh https://raw.githubusercontent.com/foo/git-workflows/master/scripts/tag_verify.sh
- name: Give script permissions
run: chmod +x $PWD/tag_verify.sh
- name: Execute script
run: $PWD/tag_verify.sh
Following Kaleby Cadorin example but for the case where the script is in a private repository
- name: Download & run script
run: |
curl --header "Authorization: token ${{ secrets.MY_PAT }}" \
--header 'Accept: application/vnd.github.raw' \
--remote-name \
--location https://raw.githubusercontent.com/COMPANY/REPO/BRANCH/PATH/script.sh
chmod +x script.sh
./script.sh
Note: GITHUB_TOKEN doesn't seem to work here, a PAT is required.
According to this thread on github-community the script needs to be downloaded/checked out separatly.
The "reusable" workflow you posted is not reusable in this sense, because since it is not downloading the script the workflow can only run within its own repository (or a repository that already has the script).
I want to add my github personal access token(for authenticating github graphql api) to my code but github keeps removing it from developer settings as soon as it is deployed. i tried adding it as an environment variable by adding it to secrets and declaring it under env: in the github actions CI script and then called it in the code using process.env.PA_TOKEN but it stil doesn't authenticate although i get a different error message in the console now saying Uncaught ReferenceError: process is not defined. What am i doing wrong?
Note: The token works fine locally before it is removed from developer settings after deployment
Github action workflow
# This is a basic workflow to help you get started with Actions
name: CI
# Controls when the action will run.
on:
# Triggers the workflow on push or pull request events but only for the main branch
push:
branches: [ main ]
pull_request:
branches: [ main ]
# Allows you to run this workflow manually from the Actions tab
workflow_dispatch:
# A workflow run is made up of one or more jobs that can run sequentially or in parallel
jobs:
# This workflow contains a single job called "build"
build:
# The type of runner that the job will run on
runs-on: ubuntu-latest
# Steps represent a sequence of tasks that will be executed as part of the job
steps:
# Checks-out your repository under $GITHUB_WORKSPACE, so your job can access it
- uses: actions/checkout#v2
# Runs a single command using the runners shell
- name: Run a one-line script
run: echo Hello, world!
# Runs a set of commands using the runners shell
- name: Run a multi-line script
run: |
echo Add other actions to build,
echo test, and deploy your project.
env:
PERSONAL_ACCESS_TOKEN: ${{ secrets.PERSONAL_ACCESS_TOKEN }}
Code implementation
const variables = {
githubToken: process.env.PERSONAL_ACCESS_TOKEN,
githubLogin: username
}
I have two ps1 scripts in Github Actions.
My scenario:
The first script executes before build
Project builds
The second script executes after build.
I need to set the value inside the first script and use it inside the second script.
So I decided to use BUILD_NUMBER environment variable and set it to 10 as a default value.
jobs:
Droid:
runs-on: windows-latest
env:
BUILD_NUMBER: "10"
Inside the first script I tried to set this variable in several ways but in the second script the value of BUILD_NUMBER was 10.
My attempts to set it:
[Environment]::SetEnvironmentVariable($env:BUILD_NUMBER, $buildNumber, 'Machine')
$env:BUILD_NUMBER: '123'
But inside the second script I was getting 10 value by this $newName = "${env:BUILD_NUMBER}"
The whole code of Github Actions side:
name: CI
# Controls when the action will run. Triggers the workflow on push or pull request
# events but only for the master branch
on:
push:
branches:
- 'master'
- 'develop'
- 'feature/*'
- 'rc/*'
pull_request:
branches:
- 'master'
- 'develop'
- 'feature/*'
- 'rc/*'
jobs:
Droid:
runs-on: windows-latest
env:
DOTNET_CLI_TELEMETRY_OPTOUT: 'true'
BUILD_NUMBER: "10"
steps:
- uses: actions/checkout#v1
- name: Run a calculate version and set sign in password script
run: .\Scripts\CalculateVersionAndSetSignPassword.ps1
shell: powershell
# Build goes here. It is skipped by me for testing purposes
- uses: actions/checkout#v1
- name: Run a change apk name script
run: |
.\Scripts\ChangeApkName.ps1
shell: powershell
set-env was depricated - please check GitHub Actions: Deprecating set-env and add-path commands
As a replacement you may use
echo "BUILD_NUMBER=yellow" >> $GITHUB_ENV
and then:
jobs:
show:
runs-on: ubuntu-latest
steps:
- name: Is variable exported?
run: |
echo "BUILD_NUMBER=yellow" >> $GITHUB_ENV
- name: PowerShell script
# You may pin to the exact commit or the version.
# uses: Amadevus/pwsh-script#25a636480c7bc678a60bbf4e3e5ac03aca6cf2cd
uses: Amadevus/pwsh-script#v2.0.0
continue-on-error: true
with:
# PowerShell script to execute in Actions-hydrated context
script: |
Write-Host $env:BUILD_NUMBER
- name: Read exported variable
run: |
echo "${{ env.BUILD_NUMBER}}"
To set environment variables in a step that can be referenced in another, you will need to use the ::set-env syntax.
In your case, your first script will have to run this command:
Write-Output "::set-env name=BUILD_NUMBER::$buildNumber"
And the second script should be able to reference it with $env:BUILD_NUMBER.
[6/20/20] Update with full example.
Action yaml file (Inline powershell will have similar behavior than with a ps1):
name: StackOverFlow
on:
push:
branches: [ master ]
jobs:
build:
runs-on: windows-latest
steps:
- run: |
$buildNumber = "12345"
Write-Output "::set-env name=BUILD_NUMBER::$buildNumber"
- run: Write-Output "Doing something else..."
- run: Write-Output "The build number is $env:BUILD_NUMBER"
Output logs:
2020-06-20T23:13:23.3209811Z ##[section]Starting: Request a runner to run this job
2020-06-20T23:13:23.5144969Z Can't find any online and idle self-hosted runner in current repository that matches the required labels: 'windows-latest'
2020-06-20T23:13:23.5145013Z Can't find any online and idle self-hosted runner in current repository's account/organization that matches the required labels: 'windows-latest'
2020-06-20T23:13:23.5145038Z Found online and idle hosted runner in current repository's account/organization that matches the required labels: 'windows-latest'
2020-06-20T23:13:23.6348644Z ##[section]Finishing: Request a runner to run this job
2020-06-20T23:13:29.9867339Z Current runner version: '2.263.0'
2020-06-20T23:13:29.9982614Z ##[group]Operating System
2020-06-20T23:13:29.9983190Z Microsoft Windows Server 2019
2020-06-20T23:13:29.9983380Z 10.0.17763
2020-06-20T23:13:29.9983515Z Datacenter
2020-06-20T23:13:29.9983691Z ##[endgroup]
2020-06-20T23:13:29.9983875Z ##[group]Virtual Environment
2020-06-20T23:13:29.9984067Z Environment: windows-2019
2020-06-20T23:13:29.9984247Z Version: 20200608.1
2020-06-20T23:13:29.9984524Z Included Software: https://github.com/actions/virtual-environments/blob/win19/20200608.1/images/win/Windows2019-Readme.md
2020-06-20T23:13:29.9984752Z ##[endgroup]
2020-06-20T23:13:29.9985890Z Prepare workflow directory
2020-06-20T23:13:30.0151643Z Prepare all required actions
2020-06-20T23:13:30.9154166Z ##[group]Run $buildNumber = "12345"
2020-06-20T23:13:30.9154566Z [36;1m$buildNumber = "12345"[0m
2020-06-20T23:13:30.9154784Z [36;1mWrite-Output "::set-env name=BUILD_NUMBER::$buildNumber"[0m
2020-06-20T23:13:30.9820753Z shell: C:\Program Files\PowerShell\7\pwsh.EXE -command ". '{0}'"
2020-06-20T23:13:30.9821156Z ##[endgroup]
2020-06-20T23:13:43.2981407Z ##[group]Run Write-Output "Doing something else..."
2020-06-20T23:13:43.2981812Z [36;1mWrite-Output "Doing something else..."[0m
2020-06-20T23:13:43.3022226Z shell: C:\Program Files\PowerShell\7\pwsh.EXE -command ". '{0}'"
2020-06-20T23:13:43.3022501Z env:
2020-06-20T23:13:43.3022706Z BUILD_NUMBER: 12345
2020-06-20T23:13:43.3022906Z ##[endgroup]
2020-06-20T23:13:43.8091340Z Doing something else...
2020-06-20T23:13:43.8671648Z ##[group]Run Write-Output "The build number is $env:BUILD_NUMBER"
2020-06-20T23:13:43.8671986Z [36;1mWrite-Output "The build number is $($env:BUILD_NUMBER)"[0m
2020-06-20T23:13:43.8717102Z shell: C:\Program Files\PowerShell\7\pwsh.EXE -command ". '{0}'"
2020-06-20T23:13:43.8717288Z env:
2020-06-20T23:13:43.8718175Z BUILD_NUMBER: 12345
2020-06-20T23:13:43.8718286Z ##[endgroup]
2020-06-20T23:13:44.4148124Z The build number is 12345
2020-06-20T23:13:44.4368449Z Cleaning up orphan processes
Found the resolution in Michael Stum`s repo that he provided in this question:
The key was Get-ChildItem Env: | Where-Object {$_.Name -Match "^MH_"} | %{ echo "::set-output name=$($_.Name)::$($_.Value)" } in .yml and $Env:MH_BUILD_VERSION = $version in .ps1 script file in his repository.
So I successfully retrieved an output from .ps1 script and used it in Github Actions.
I want to run periodically ipynb file in my github repository (Like every 30 minutes).
I know that I can use Github Action to create yml file for this progress but I have no idea how to reorganize yml file.
How can I do it?
Here is my test tml file defined below.
# This is a basic workflow to help you get started with Actions
name: CI
# Controls when the action will run. Triggers the workflow on push or pull request
# events but only for the master branch
on:
schedule:
- cron: '*/5 * * * *'
push:
branches: [ master ]
pull_request:
branches: [ master ]
# A workflow run is made up of one or more jobs that can run sequentially or in parallel
jobs:
# This workflow contains a single job called "build"
build:
# The type of runner that the job will run on
runs-on: ubuntu-latest
# Steps represent a sequence of tasks that will be executed as part of the job
steps:
# Checks-out your repository under $GITHUB_WORKSPACE, so your job can access it
- uses: actions/checkout#v2
# Runs a single command using the runners shell
- name: Run a one-line script
run: echo Hello, world!
# Runs a set of commands using the runners shell
- name: Run a multi-line script
run: |
echo Add other actions to build,
echo test, and deploy your project.
You can check out this GitHub action that runs your jupyter notebook and lets you upload the artifacts. As for how to organize your workflow file, you can read the documentation here.