Powershell command Compress-Archive incredibly slow - powershell

I'm new to windows development and I'm attempting using github actions to do a build/deploy. In the build step I compress my project and upload it
jobs:
build:
runs-on: windows-latest
steps:
- uses: actions/checkout#v2
- name: Set up Node.js version
uses: actions/setup-node#v1
with:
node-version: '16.x'
- name: npm install, build, and test
run: |
npm install
npm run build --if-present
npm run test --if-present
- name: Zip contents for upload
shell: powershell
run: |
Compress-Archive -Path . -DestinationPath nextjs-app.zip
- name: Upload artifact for deployment job
uses: actions/upload-artifact#v2
with:
name: nextjs-app
path: nextjs-app.7z
and then in my deploy step I download it, expand it, and then deploy it.
deploy:
runs-on: windows-latest
needs: build
environment:
name: 'Production'
url: ${{ steps.deploy-to-webapp.outputs.webapp-url }}
steps:
- name: Download artifact from build job
uses: actions/download-artifact#v2
with:
name: nextjs-app
- name: Unzip archive file
shell: powershell
run: |
Expand-Archive nextjs-app.zip -DestinationPath .
The issue is that it takes an incredible amount of time to do the compression/expansion. I had previously done this on linux and it took around 2 minutes for compression/expansion, but using powershell it's taking about 15 minutes to compress and 12 minutes to expand. Why are the Compress/Expand commands going so slow? Am I doing something wrong? The expanded folder size is around 200MB

You may experiment with compression parameters, e. g. -CompressionLevel Fastest.
Alternatively use .NET API directly, as Santiago Squarzon suggests. This requires PowerShell (Core) 7+:
[IO.Compression.ZipFile]::CreateFromDirectory( $sourceDirectory, $zipFileName, 'Fastest', $false )
Note that .NET API has a different current directory than PowerShell, so best practice is to pass only absolute paths to .NET API. The simplest way to do this is to prepend $PWD (PowerShell's current directory) to any path, e. g. "$PWD\SomeFile.xyz".

Related

Sonar scanner command found on Gitlab CI/CD

I am trying to integrate Sonarqube into my project CI/CD pipeline on Gitlab. I have followed the documentation on Gitlab and Sonarqube to the best of my understanding to get the job included in my yml file.
I am current experiencing the error as shown in the image below
This is my yml file script
build_project:
stage: build
script:
- xcodebuild clean -workspace TinggIOS/TinggIOS.xcworkspace -scheme TinggIOS | xcpretty
- xcodebuild test -workspace TinggIOS/TinggIOS.xcworkspace -scheme TinggIOS -destination 'platform=iOS Simulator,name=iPhone 11 Pro Max,OS=15' | xcpretty -s
tags:
- stage
image: macos-11-xcode-12
sonarqube-check:
stage: analyze
image:
name: sonarsource/sonar-scanner-cli:latest
entrypoint: [""]
variables:
SONAR_USER_HOME: "${CI_PROJECT_DIR}/.sonar" # Defines the location of the analysis task cache
GIT_DEPTH: "0" # Tells git to fetch all the branches of the project, required by the analysis task
cache:
key: "${CI_JOB_NAME}"
paths:
- .sonar/cache
script:
- sonar-scanner -Dsonar.qualitygate.wait=true
allow_failure: true
only:
- merge_requests
- feature/unit-test # or the name of your main branch
- develop
tags:
- stage
It looks like the
- sonar-scanner -Dsonar.qualitygate.wait=true
command is not found. Try to run that command on the machine you are setting up your pipeline (like ssh into that machine or ssh into it and try running that command). The issue might be that it isn't installed on there.

Why github actions cant timeout a single job

I have a workflow in which a run request runs infinitely. i want to stop that run after 5 minutes of it running.
my workflow file:-
name: MSBuild
on:
push:
branches: [ master ]
pull_request:
branches: [ master ]
env:
# Path to the solution file relative to the root of the project.
SOLUTION_FILE_PATH: ./genshincheat.sln
# Configuration type to build.
# You can convert this to a build matrix if you need coverage of multiple configuration types.
# https://docs.github.com/actions/learn-github-actions/managing-complex-workflows#using-a-build-matrix
BUILD_CONFIGURATION: Release
permissions:
contents: read
jobs:
build:
runs-on: windows-latest
steps:
- uses: actions/checkout#v1
with:
submodules: recursive
- name: Add MSBuild to PATH
uses: microsoft/setup-msbuild#v1.0.2
- name: Restore NuGet packages
working-directory: ${{env.GITHUB_WORKSPACE}}
run: nuget restore ${{env.SOLUTION_FILE_PATH}}
- name: Build
working-directory: ${{env.GITHUB_WORKSPACE}}
# Add additional options to the MSBuild command line here (like platform or verbosity level).
# See https://learn.microsoft.com/visualstudio/msbuild/msbuild-command-line-reference
run: msbuild /m /p:Configuration=${{env.BUILD_CONFIGURATION}} ${{env.SOLUTION_FILE_PATH}}
- uses: montudor/action-zip#v1
with:
args: zip -qq -r bin.zip dir
- uses: actions/checkout#v2
- run: mkdir -p path/to/artifact
- run: echo hello > path/to/artifact/world.txt
- uses: actions/upload-artifact#v3
with:
name: bin.zip
path: ./bin.zip
the "build" runs infinitely any way to stop it after 5 mins so it can carry out next jobs? it runs infinitely becauseafter build it runs the built program so i cant exit that ;-;. any help is appreciated
There are different fields that can help you achieve what you want.
At the job level: job.<id>.timeout-minutes (defining a job timeout)
At the step level: job.<id>.steps.timeout-minutes (defining a step timeout)
Which would look like this in your case:
At the job level:
build:
runs-on: windows-latest
timeout-minutes: 5
steps:
[...]
At the step which never ends (example):
- name: Build
timeout-minutes: 5
working-directory: ${{env.GITHUB_WORKSPACE}}
# Add additional options to the MSBuild command line here (like platform or verbosity level).
# See https://learn.microsoft.com/visualstudio/msbuild/msbuild-command-line-reference
run: msbuild /m /p:Configuration=${{env.BUILD_CONFIGURATION}} ${{env.SOLUTION_FILE_PATH}}
Another reference on the Github Community

GitHub Actions - How clean up unchanged/uncommited files before upload to SFTP Server

I´m tring to config a GitHub Action to deploy my application to SFTP file.
My application has 6700 files and I would like to upload only changed/commited files.
How can I remove unchanged and/or uncommited files before upload to SFTP?
This way, my one file modification deploy would be so faster than upload 6k files.
name: CI
on:
push:
branches: [ main ]
workflow_dispatch:
jobs:
deploy:
runs-on: ubuntu-latest
name: Deploy Job
steps:
- name: Checkout
uses: actions/checkout#v2
with:
fetch-depth: 2
- name: Deploy files
uses: wlixcc/SFTP-Deploy-Action#v1.0
with:
username: 'deploy_user'
server: 'server_ip'
ssh_private_key: ${{ secrets.SSH_PRIVATE_KEY }}
local_path: './www/*'
remote_path: '/www'
args: '-o ConnectTimeout=10'
To list down all the files that are updated/committed in the given commit, you can use this command:
$ git diff-tree --no-commit-id --name-only -r $GITHUB_SHA
index.html
src/application.js
Then you can use this to delete all the files that are not on this list. This needs some bash digging. One hack I can think of is to make a temporary directory to copy updated files and only upload these files.

Setting environment variable value from .ps1 script not working in Github Actions

I have two ps1 scripts in Github Actions.
My scenario:
The first script executes before build
Project builds
The second script executes after build.
I need to set the value inside the first script and use it inside the second script.
So I decided to use BUILD_NUMBER environment variable and set it to 10 as a default value.
jobs:
Droid:
runs-on: windows-latest
env:
BUILD_NUMBER: "10"
Inside the first script I tried to set this variable in several ways but in the second script the value of BUILD_NUMBER was 10.
My attempts to set it:
[Environment]::SetEnvironmentVariable($env:BUILD_NUMBER, $buildNumber, 'Machine')
$env:BUILD_NUMBER: '123'
But inside the second script I was getting 10 value by this $newName = "${env:BUILD_NUMBER}"
The whole code of Github Actions side:
name: CI
# Controls when the action will run. Triggers the workflow on push or pull request
# events but only for the master branch
on:
push:
branches:
- 'master'
- 'develop'
- 'feature/*'
- 'rc/*'
pull_request:
branches:
- 'master'
- 'develop'
- 'feature/*'
- 'rc/*'
jobs:
Droid:
runs-on: windows-latest
env:
DOTNET_CLI_TELEMETRY_OPTOUT: 'true'
BUILD_NUMBER: "10"
steps:
- uses: actions/checkout#v1
- name: Run a calculate version and set sign in password script
run: .\Scripts\CalculateVersionAndSetSignPassword.ps1
shell: powershell
# Build goes here. It is skipped by me for testing purposes
- uses: actions/checkout#v1
- name: Run a change apk name script
run: |
.\Scripts\ChangeApkName.ps1
shell: powershell
set-env was depricated - please check GitHub Actions: Deprecating set-env and add-path commands
As a replacement you may use
echo "BUILD_NUMBER=yellow" >> $GITHUB_ENV
and then:
jobs:
show:
runs-on: ubuntu-latest
steps:
- name: Is variable exported?
run: |
echo "BUILD_NUMBER=yellow" >> $GITHUB_ENV
- name: PowerShell script
# You may pin to the exact commit or the version.
# uses: Amadevus/pwsh-script#25a636480c7bc678a60bbf4e3e5ac03aca6cf2cd
uses: Amadevus/pwsh-script#v2.0.0
continue-on-error: true
with:
# PowerShell script to execute in Actions-hydrated context
script: |
Write-Host $env:BUILD_NUMBER
- name: Read exported variable
run: |
echo "${{ env.BUILD_NUMBER}}"
To set environment variables in a step that can be referenced in another, you will need to use the ::set-env syntax.
In your case, your first script will have to run this command:
Write-Output "::set-env name=BUILD_NUMBER::$buildNumber"
And the second script should be able to reference it with $env:BUILD_NUMBER.
[6/20/20] Update with full example.
Action yaml file (Inline powershell will have similar behavior than with a ps1):
name: StackOverFlow
on:
push:
branches: [ master ]
jobs:
build:
runs-on: windows-latest
steps:
- run: |
$buildNumber = "12345"
Write-Output "::set-env name=BUILD_NUMBER::$buildNumber"
- run: Write-Output "Doing something else..."
- run: Write-Output "The build number is $env:BUILD_NUMBER"
Output logs:
2020-06-20T23:13:23.3209811Z ##[section]Starting: Request a runner to run this job
2020-06-20T23:13:23.5144969Z Can't find any online and idle self-hosted runner in current repository that matches the required labels: 'windows-latest'
2020-06-20T23:13:23.5145013Z Can't find any online and idle self-hosted runner in current repository's account/organization that matches the required labels: 'windows-latest'
2020-06-20T23:13:23.5145038Z Found online and idle hosted runner in current repository's account/organization that matches the required labels: 'windows-latest'
2020-06-20T23:13:23.6348644Z ##[section]Finishing: Request a runner to run this job
2020-06-20T23:13:29.9867339Z Current runner version: '2.263.0'
2020-06-20T23:13:29.9982614Z ##[group]Operating System
2020-06-20T23:13:29.9983190Z Microsoft Windows Server 2019
2020-06-20T23:13:29.9983380Z 10.0.17763
2020-06-20T23:13:29.9983515Z Datacenter
2020-06-20T23:13:29.9983691Z ##[endgroup]
2020-06-20T23:13:29.9983875Z ##[group]Virtual Environment
2020-06-20T23:13:29.9984067Z Environment: windows-2019
2020-06-20T23:13:29.9984247Z Version: 20200608.1
2020-06-20T23:13:29.9984524Z Included Software: https://github.com/actions/virtual-environments/blob/win19/20200608.1/images/win/Windows2019-Readme.md
2020-06-20T23:13:29.9984752Z ##[endgroup]
2020-06-20T23:13:29.9985890Z Prepare workflow directory
2020-06-20T23:13:30.0151643Z Prepare all required actions
2020-06-20T23:13:30.9154166Z ##[group]Run $buildNumber = "12345"
2020-06-20T23:13:30.9154566Z [36;1m$buildNumber = "12345"[0m
2020-06-20T23:13:30.9154784Z [36;1mWrite-Output "::set-env name=BUILD_NUMBER::$buildNumber"[0m
2020-06-20T23:13:30.9820753Z shell: C:\Program Files\PowerShell\7\pwsh.EXE -command ". '{0}'"
2020-06-20T23:13:30.9821156Z ##[endgroup]
2020-06-20T23:13:43.2981407Z ##[group]Run Write-Output "Doing something else..."
2020-06-20T23:13:43.2981812Z [36;1mWrite-Output "Doing something else..."[0m
2020-06-20T23:13:43.3022226Z shell: C:\Program Files\PowerShell\7\pwsh.EXE -command ". '{0}'"
2020-06-20T23:13:43.3022501Z env:
2020-06-20T23:13:43.3022706Z BUILD_NUMBER: 12345
2020-06-20T23:13:43.3022906Z ##[endgroup]
2020-06-20T23:13:43.8091340Z Doing something else...
2020-06-20T23:13:43.8671648Z ##[group]Run Write-Output "The build number is $env:BUILD_NUMBER"
2020-06-20T23:13:43.8671986Z [36;1mWrite-Output "The build number is $($env:BUILD_NUMBER)"[0m
2020-06-20T23:13:43.8717102Z shell: C:\Program Files\PowerShell\7\pwsh.EXE -command ". '{0}'"
2020-06-20T23:13:43.8717288Z env:
2020-06-20T23:13:43.8718175Z BUILD_NUMBER: 12345
2020-06-20T23:13:43.8718286Z ##[endgroup]
2020-06-20T23:13:44.4148124Z The build number is 12345
2020-06-20T23:13:44.4368449Z Cleaning up orphan processes
Found the resolution in Michael Stum`s repo that he provided in this question:
The key was Get-ChildItem Env: | Where-Object {$_.Name -Match "^MH_"} | %{ echo "::set-output name=$($_.Name)::$($_.Value)" } in .yml and $Env:MH_BUILD_VERSION = $version in .ps1 script file in his repository.
So I successfully retrieved an output from .ps1 script and used it in Github Actions.

(GitHub Actions) Split step into two consecutive ones

I'd like to use GitHub Actions to publish an npm package. So far I am using quite a trivial script to do that. Now I'd like to split one step of the script into two consecutive ones.
here's an excerpt from my workflows/...yaml file:
steps:
# ...
- name: Build
run: |
cd src
npm install
tsc
# TODO split here
npm set registry https://npm.pkg.github.com
npm set //npm.pkg.github.com/:_authToken ${{ secrets.GITHUB_TOKEN }}
npm publish
env:
CI: true
Now, when I tried to have these as separate Steps, they ran in parallel which is not the behavior I hope for, since the first step yields results (creates a src/lib directory) that I depend on in step #2. (The one where I log into npm and publish this).
Can someone please help me to unravel this?
As #smac89 explains above, steps can't be executed in parallel, only jobs and workflows can.
The official Github Documentation shares this illustration about the Github Actions structure:
What probably happened in your case was that you created 2 jobs instead of 2 steps.
The syntax to split your steps should be something like this:
jobs:
my-job:
runs-on: ubuntu-latest #for example
steps:
# ...
- name: Build
run: |
cd src
npm install
tsc
env:
CI: true
- name: Publish
run: |
npm set registry https://npm.pkg.github.com
npm set //npm.pkg.github.com/:_authToken ${{ secrets.GITHUB_TOKEN }}
npm publish
env:
CI: true