I am using Azure Pipelines to build a Rakudo binary for Raku (previously aka Perl 6) in Windows.
This is my azure-pipelines.yml file:
jobs:
- job: Windows
pool:
vmImage: 'vs2017-win2016'
steps:
- bash: |
mkdir -p $(Build.SourcesDirectory)/rakudo-win
curl -L https://github.com/rakudo/rakudo/releases/download/2019.07.1/rakudo-2019.07.1.tar.gz | tar xz
mv rakudo-2019.07.1 rakudo
cd rakudo
C:/Strawberry/perl/bin/perl Configure.pl --gen-moar --gen-nqp --backends=moar --prefix=$(Build.SourcesDirectory)/rakudo-win
make
make install
- bash: |
echo "##vso[task.prependpath]$(Build.SourcesDirectory)/rakudo-win/bin"
- bash: |
perl6 -v
The pipeline script builds perl6 binary fine inside $(Build.SourcesDirectory)/rakudo-win/bin folder. There is indeed perl6.exe inside $(Build.SourcesDirectory)/rakudo-win/bin. To make it available, I set the path by prepending it in the bash script. But when I try to run command perl6 -v, the build fails at this step.
I searched for similar issues in SO here, here, here.
Still I could not solve my issue. Any help how to make perl6 binary available at PATH?
EDITED
Next thing I did was create another .yml script as follows:
jobs:
- job: Windows
pool:
vmImage: 'vs2017-win2016'
steps:
- script: |
call "C:\Program Files (x86)\Microsoft Visual Studio\2017\Enterprise\VC\Auxiliary\Build\vcvars64.bat"
- pwsh: |
mkdir -p C:\rakudo-win
Invoke-WebRequest -Uri "https://github.com/rakudo/rakudo/releases/download/2019.07.1/rakudo-2019.07.1.tar.gz" -OutFile "rakudo.tar.gz"
tar -xvf .\rakudo.tar.gz
cd rakudo-2019.07.1
C:\Strawberry\perl\bin\perl Configure.pl --gen-moar --gen-nqp --backends=moar --prefix=C:\rakudo-win
make
make install
- pwsh: |
$oldpath = (Get-ItemProperty -Path 'Registry::HKEY_LOCAL_MACHINE\System\CurrentControlSet\Control\Session Manager\Environment' -Name PATH).path
$newpath = "C:\rakudo-win\bin;$oldpath"
Set-ItemProperty -Path 'Registry::HKEY_LOCAL_MACHINE\System\CurrentControlSet\Control\Session Manager\Environment' -Name PATH -Value $newpath
- script: |
SET PATH=C:\rakudo-win\bin;%PATH%
- script: |
perl6 -v
and tried changing PATH twice once in powershell and another in cmdline. But still it throws following error:
'perl6' is not recognized as an internal or external command,
operable program or batch file.
Any help?
The best method I have found for setting the PATH for subsequent tasks in Azure Pipelines is by using the logging command syntax mentioned in the first of the three SO links you looked at. Since you are using PowerShell in your updated yaml pipeline, the command would be:
Write-Host "##vso[task.prependpath]$(Build.SourcesDirectory)/rakudo-win/bin"
Note that this only applies to subsequent tasks, if you try outputting the PATH variable in the current task it will not have updated.
In fact, you are very close to the correct solution. Your second powershell task has set the PATH successfully. You can add another separate task to print out the system PATH value to verify this.
- pwsh: |
$NewPathInRegistry = (Get-ItemProperty -Path 'Registry::HKEY_LOCAL_MACHINE\System\CurrentControlSet\Control\Session Manager\Environment' -Name PATH).path
Write-Host $NewPathInRegistry
To set the PATH programmatically, you can not use set command, this command can indeed set the environment variable PATH, but the disadvantage of set is the new PATH value is only available in the current command line terminal, it does not actually been added into the System Variable. That's why you were getting an unrecognized error in the next script step.
To permanently add the directory to environment variable PATH so that it can work for next others steps, you need use setx or add them into Registry by using reg add. But the usage of setx has limitation that the PATH value max to 1024 characters. So, here the best solution is updating PATH by modifying the Registry value.
BUT, updating Registry still has another issue, you must kill current process and run one new process to run perl6 so that it can read the new available Registry setting.
If run stop-process in Azure devops pipeline, it will make the task failed with exit code -1. This is the expected exit code, so you can set the continueOnError: true to step so that the next steps can continue.
why not just do this:
- script: |
PATH=$BUILD_SOURCESDIRECTORY/rakudo-win/bin:$PATH perl6 -v
Related
I define GENERATOR_PLATFORM as an empty environment variable, and then I want
to set it to something for my Windows build. But, the variable never gets set:
env:
GENERATOR_PLATFORM:
steps:
- name: windows-dependencies
if: startsWith(matrix.os, 'windows')
run: |
$generator= "-DCMAKE_GENERATOR_PLATFORM=x64"
echo "Generator: ${generator}"
echo "GENERATOR_PLATFORM=$generator" >> $GITHUB_ENV
- name: Configure CMake
shell: bash
working-directory: ${{github.workspace}}/build
run: cmake $GITHUB_WORKSPACE $GENERATOR_PLATFORM
If you are using a Windows/PowerShell environment, you have to use $env:GITHUB_ENV instead of $GITHUB_ENV:
echo "GENERATOR_PLATFORM=$generator" >> $env:GITHUB_ENV
This way, you can access your env var through $env:GENERATOR_PLATFORM, eg:
run: echo $env:GENERATOR_PLATFORM
To follow up on #soltex answer: The proposed solution only works if the encoding is set to utf-8. If your runner is using Windows PowerShell (i.e. not PowerShell v7+, which uses utf-8 by default), utf16-le is written to the environment file, which causes the variable to not being set.
The correct solution is this:
echo "GENERATOR_PLATFORM=$generator" | Out-File -FilePath $env:GITHUB_ENV -Encoding utf8 -Append
From: https://github.com/actions/runner-images/issues/5251#issuecomment-1071030822
Further reading: Changing PowerShell's default output encoding to UTF-8
In my dockerfile want to use the following sequence of commands to download and extract a large zip file:
RUN powershell -Command \
wget http://my_server/big_huge.zip \
-OutFile C:\big_huge.zip ; \
Expand-Archive -Path C:\big_huge.zip \
-DestinationPath C:\big_huge ; \
Remove-Item C:\big_huge.zip -Force
I don't want to use ADD to download the zip file isn't going to change and I want this step to be cached.
What I have above seems to work but I do not get any indication of the progress of the download like I normally would. That's a bummer because this is a large download. The progress of the download is obscured I suppose because Invoke-WebRequest which wget is an alias to is a cmdlet. Is there any way to pipe the output of a cmdlet to stdout so I can see it when I am running docker build?
I gave up on trying to do the download from the Dockerfile and instead wrote a separate script that pre-downloads the files I need and expands their archives if the files aren't already present. This script then calls docker build, docker run, etc. In the Dockerfile I am copying the directory where I expanded the archives.
I don't know Docker. But maybe you can pipe the output through the powershell cmdlet Out-Host. Type in help Out-Host for more information.
I'm currently trying to add GitHub actions workflow to a repo...
To do a C++/CMake/swig/python development (i.e. native python library dev), I need to download and install swigwin and have it available in the PATH...
Unfortunately it seems the $env:Path... command is not take into account during the next subsequent steps
Example
name: Python Windows CI
on: [push, pull_request]
jobs:
# Building using the GitHub runner environment directly.
build:
runs-on: windows-latest
steps:
- uses: actions/checkout#v2
- name: Check cmake
run: cmake --version
- name: Install swig
run: |
(New-Object System.Net.WebClient).DownloadFile("http://prdownloads.sourceforge.net/swig/swigwin-4.0.1.zip","swigwin-4.0.1.zip");
Expand-Archive .\swigwin-4.0.1.zip .;
$env:Path += ";.\swigwin-4.0.1";
swig -version;
- name: Check swig
run: swig -version # swig cmdlet not found...
Observed
> Set up job
> Run actions/checkout#v23s
> Check cmake
v Install swig
...
SWIG Version 4.0.1
...
v Check swig
swig -version
shell: C:\Program Files\PowerShell\6\pwsh.EXE -command ". '{0}'"
swig : The term 'swig' is not recognized as the name of a cmdlet, function, script file, or operable program.
Check the spelling of the name, or if a path was included, verify that the path is correct and try again.
At D:\a\_temp\0a8dc0e1-ec51-429b-abd0-cb3597e983ac.ps1:2 char:1
+ swig -version
+ ~~~~
+ CategoryInfo : ObjectNotFound: (swig:String) [], ParentContainsErrorRecordException
+ FullyQualifiedErrorId : CommandNotFoundException
##[error]Process completed with exit code 1.
The add-path and set-env commands have been deprecated the 1st October 2020 for security reasons: https://github.blog/changelog/2020-10-01-github-actions-deprecating-set-env-and-add-path-commands/
The recommended way to add to %PATH% is using environment files as follows:
Assuming you use Powershell, the default shell:
echo "C:\directory\to\add\to\path" | Out-File -FilePath $env:GITHUB_PATH -Encoding utf8 -Append
or alternatively for bash:
echo "C:\directory\to\add\to\path" >> $GITHUB_PATH
There is also a shorter way to achieve this in Powershell, which is the default shell that windows hosted runner uses:
Add-Content $env:GITHUB_PATH "C:\directory\to\add\to\path"
See Add-Content
The other answers are a bit out of date (due to GitHub Actions deprecating add-path as explained in #Kel Solaar's answer), here's a full example based on #Mizux answer:
- name: Install swig
if: "startsWith(runner.os, 'windows')"
run: |
(New-Object System.Net.WebClient).DownloadFile("http://prdownloads.sourceforge.net/swig/swigwin-4.0.1.zip","swigwin-4.0.1.zip");
Expand-Archive .\swigwin-4.0.1.zip .;
echo "$((Get-Item .).FullName)/swigwin-4.0.1" | Out-File -FilePath $env:GITHUB_PATH -Encoding utf8 -Append
Another difference to #Mizus answer is that the absolute path to the swig directory is used, this is to ensure it still works even though the working directory changes.
EDIT: GitHub have deprecated this, please see other answer...
ref: https://github.blog/changelog/2020-10-01-github-actions-deprecating-set-env-and-add-path-commands/
You must use the action syntax echo "::add-path::...", in your case:
...
- name: Install swig
run: |
(New-Object System.Net.WebClient).DownloadFile("http://prdownloads.sourceforge.net/swig/swigwin-4.0.1.zip","swigwin-4.0.1.zip");
Expand-Archive .\swigwin-4.0.1.zip .;
echo "::add-path::./swigwin-4.0.1"
- name: Check swig
run: swig -version
src: https://help.github.com/en/actions/reference/workflow-commands-for-github-actions#adding-a-system-path
i need to run single line powershell script like this one:
$a = Get-Content test.tmp; $b = [timespan]::fromseconds($a); "{0:HH:mm:ss,fff}" -f ([datetime]$b.Ticks)
in command line. When running it directly from powerhsell cli it work fine. but when trying run:
powershell "$a = Get-Content test.tmp; $b = [timespan]::fromseconds($a); "{0:HH:mm:ss,fff}" -f ([datetime]$b.Ticks)"
from command line cli i get error. I cannot run it from script in form of .ps1 file as i am not allowed to change restriction policy regarding to run powershell script.
Anybodoy would be able to point me what i have to change to run it properly from command line?
Many thanks
With proper quoting / parentheses you need no intermediate variables and only one command:
'{0:HH:mm:ss,fff}' -f ([datetime]([timespan]::fromseconds((Get-Content test.tmp))).Ticks)
In a batch:
powershell -C "'{0:HH:mm:ss,fff}' -f ([datetime]([timespan]::fromseconds((Get-Content test.tmp))).Ticks)"
Or to get the duration into a batch variable:
:: Q:\Test\2017\09\23\SO_46379453.cmd
#Echo off
For /f "tokens=1*" %%A in (
'powershell -C "\"{0:HH:mm:ss,fff}\" -f ([datetime]([timespan]::fromseconds((Get-Content test.tmp))).Ticks)" '
) Do Set "Duration=%%A,%%B"
set Duration
I'm just curios. Is there an equivalent for PowerShell that behaves equally to the liunx command listed in the title, i.e.
mkdir {folderA, folderB}
?
-- edit
the command listed above creates the folders "folderA" and "folderB" (just saw that I wrote file previously. Sorry, my fault) in the current working directory.
The mkdir command in PowerShell is a wrapper for the New-Item command. If you want to create multiple folders with a single command, then run:
mkdir c:\test,c:\test2;
Effectively, because of positional parameters in PowerShell, this passes the array c:\test,c:\test2 to the -Path parameter of the New-Item command.