I need to cache files in /home/vsts/.cache/torch/.
Is there a predefined variable for the home folder?
If by "Predefined variables" you mean one of the variables specified in the documentation then no.
However if you are using Hosted Linux Agents you can simply look at the environment variable $HOME, which works at least if you are in a script:, which may or may not fit your use case.
Example:
trigger:
- main
pool:
vmImage: ubuntu-latest
steps:
- script: echo $HOME
The output of the script task is /home/vsts
No, there is no such variable. You can use one of the /home/vsts/work paths and cache $(System.WorkFolder)/../.cache/torch/.
Values of predefined variables for an Ubuntu agent:
Agent.AcceptTeeEula True
Agent.BuildDirectory /home/vsts/work/1
Agent.HomeDirectory /home/vsts/agents/2.181.2
Agent.Id 32
Agent.JobName Job
Agent.JobStatus Succeeded
Agent.MachineName fv-az160-540
Agent.Name Azure Pipelines 2
Agent.OS Linux
Agent.OSArchitecture X64
Agent.RetainDefaultEncoding false
Agent.ReadOnlyVariables true
Agent.RootDirectory /home/vsts/work
Agent.TempDirectory /home/vsts/work/_temp
Agent.ToolsDirectory /opt/hostedtoolcache
Agent.Version 2.181.2
Agent.WorkFolder /home/vsts/work
Build.ArtifactStagingDirectory /home/vsts/work/1/a
Build.BinariesDirectory /home/vsts/work/1/b
Build.DefinitionName FooBar
Build.SourceBranch refs/heads/FOOBAR-123-caching
Build.SourceVersion 2b2c45223722fc7226eb23d752fc722bcdee2c54
Build.SourcesDirectory /home/vsts/work/1/s
Build.StagingDirectory /home/vsts/work/1/a
Common.TestResultsDirectory /home/vsts/work/1/TestResults
Pipeline.Workspace /home/vsts/work/1
System.AccessToken ***
System.ArtifactsDirectory /home/vsts/work/1/a
System.CollectionId c02d3c7b-9d74-4532-aa21-abccdf07c888
System.Culture en-US
System.DefaultWorkingDirectory /home/vsts/work/1/s
System.DefinitionId 26
System.EnableAccessToken SecretVariable
System.HostType build
System.JobAttempt 1
System.JobId 33f11733-54f2-5aa3-20dd-22fc7dcf5912
System.JobName __default
System.PhaseAttempt 1
System.PhaseDisplayName Job
System.PhaseName Job
System.ServerType Hosted
System.StageAttempt 1
System.StageName __default
System.TeamProject Foo Bar
System.TeamProjectId 3337a4f6-1411-40aa-beeb-0c7d86b9ecba
System.WorkFolder /home/vsts/work
Task.DisplayName print predefined variables
Related
I have a local building agent for a Unity|HoloLens-Project. Today I had to switch from VS2019 to VS2022 on my building agent and I came across the issue, that the MSBuild-Task-Version of DevOps 2019 is not compatible with VS2022.
So I found a workaround. That workaround is to use the following script instead of the MSBuild task. But using that script I ran into the following misbehavior:
If I use the MSBuild#1-task my pipeline crashes because of mentioned compatibility problem. But the pipeline uses the right building agent.
If I use the Script the pipeline crashes because it uses the wrong building agent. That agent has no unity installed and is not mentioned in the YAML.
Yaml:
- task: MSBuild#1
displayName: 'Build Appx Package - MSBuild#1'
inputs:
solution: '$(Build.BinariesDirectory)$(unity.outputPath)/*.sln'
msbuildVersion: 'latest'
platform: 'ARM64'
configuration: 'Release'
msbuildArguments: '/p:AppxBundle=Always /p:AppxPackageDir=$(Build.ArtifactStagingDirectory)/AppPackages'
# ----- or ------
- script: |
#echo off
setlocal enabledelayedexpansion
for /f "usebackq tokens=*" %%i in (`"!ProgramFiles(x86)!\Microsoft Visual Studio\Installer\vswhere.exe" -latest -products * -requires Microsoft.Component.MSBuild -find MSBuild\**\Bin\MSBuild.exe`) do (set msbuild_exe=%%i)
"!msbuild_exe!" "$(Build.BinariesDirectory)$(unity.outputPath)/*.sln" /p:Configuration=Release /p:Platform="ARM64" /p:AppxBundle=Always /p:AppxPackageDir=$(Build.ArtifactStagingDirectory)/AppPackages /t:rebuild
At the beginng of my YAML I set of course the pool and the name of the building agent:
- job: unity
pool:
name: 'My_Pool'
demand: Agent.name -equals My_Agent_Unity_1
I don't understand how removing the MSBuild task and adding the script can cause a different building agent to suddenly be taken.
I don't understand how removing the MSBuild task and adding the script
can cause a different building agent to suddenly be taken.
Because you entered the wrong keyword "demand" in your YAML. It should be demands.
- job: unity
pool:
name: 'My_Pool'
demands: Agent.name -equals My_Agent_Unity_1
Please see Manually entered demands and pool definition for details.
As per title above.
I am trying to mimic my automation setup on my local machine to match the automation i have running using Microsoft Hosted Agent machine.
This is how it set up in my local machine. therefore when the automation inserts "%repoic%" in a file explorer and the select folder button is pressed, it will open the desired folder
Now, I need to do the same using the Microsoft Hosted Agent to run my automation. Would anyone give me a clue on how this can be done? Is it simply do the following within my DevOps Pipeline's YAML file?
cheers!
From your requirement, you need to set the Environment variable and use it to navigate to path of the agent machine.
In Azure DevOps Pipeline, you can set the Pipeline variable as the screenshot show in the question.
Then you can use the format: $(variablename) to use the Pipeline variable.
Or you can use the format: %NAME% for batch and $env:NAME in PowerShell to use the Pipeline Environment Variable. Refer to this doc: Environment Variable
Here is an example:
pool:
vmImage: windows-latest
variables:
testpath: ./src/app/head
steps:
- powershell: Get-Location
workingDirectory: $(testpath)
Result:
OR you can use the environment variable in Pipeline.
Example:
pool:
vmImage: windows-latest
variables:
testpath: ./src/app/head
steps:
- task: CmdLine#2
inputs:
script: |
cd %testpath%
I have a standard .NET Core (Ubuntu) pipeline on Azure Devops and within my Test project, I use environment variables. Within my pipeline, I have defined my group variables like so
variables:
- group: MyApiVariables
Whenever I run the tests for my project
- task: DotNetCoreCLI#2
displayName: "Testing Application"
inputs:
command: test
projects: '**/*Tests/*.csproj'
arguments: '--configuration $(buildConfiguration)'
The actual environment variables aren't passed in. They are blank.
What am I missing to get this running? I've even defined variables in the edit pipeline page too with no luck
- task: Bash#3
inputs:
targetType: 'inline'
script: echo $AppConfigEndpoint
env:
AppConfigEndpoint: $(AppConfigEndpoint)
ApiConfigSection: $(ApiConfigSection)
Thanks!
CASING Strikes again! MyVariableName was turned into MYVARIABLENAME on Azure Devops. I changed my variable names in my group to all caps and it worked. I spent way too much time on this.
Mike, I see that you use variable groups. I assume it may cause your issue. Take a look at variable passing example I made:
First I had to create new variable group in Library:
Here is a pipeline code that reference created variables:
# Set variables group reference
variables:
- group: SampleVariableGroup
steps:
- powershell: 'Write-Host "Config variable=$(configuration) Platform variable=$(platform)"'
displayName: 'Display Sample Variable'
I used PowerShell task to verify if variables were properly passed to the job.
As you can see both configuration & platform values were displayed correctly.
In fact you can't go wrong that way, unless you start to mix variable groups with variables defined in a yaml. In such scenario you'll have to use name/value syntax for the individual (non-grouped) variables.
Please see Microsoft Variable Groups documentation. Such example is well explained there. I also suggest to take closer look at general Variables Documentation.
In case of referencing variables in other tasks here is a great example from MS (it should work everywhere in same manner):
# Set variables once
variables:
configuration: debug
platform: x64
steps:
# Use them once
- task: MSBuild#1
inputs:
solution: solution1.sln
configuration: $(configuration) # Use the variable
platform: $(platform)
# Use them again
- task: MSBuild#1
inputs:
solution: solution2.sln
configuration: $(configuration) # Use the variable
platform: $(platform)
Good luck!
My Azure DevOps Build Pipeline makes use of GitVersion for versioning the build artefacts. Having the task present sets the GitVersion-variable that gives access to GitVersion.MajorMinorPatch and so on. What I want to do though is concatenating this variable with the BuildID. Setting the artefact name with the following command works well:
name: $(GitVersion.MajorMinorPatch).$(BuildID)
In the next step I wanted to have the version to be written into various config files. I installed the ReplaceToken plugin that replaces a specific token (in my case #{Version}#) with the content of the variable "Version".
Creating a variable with the name Version and the value "$(GitVersion.MajorMinorPatch).$(BuildID)" simply replaces the variable with the string instead of the evaluated value. I already tried to set the variable in the job that runs after the GitVersion task, so it should already be present when the variable gets created.
eg.
- job: 'Frontend'
dependsOn: 'Preparation'
variables:
Version: $('GitVersion.MajorMinorPatch')
steps:
- bash: echo $(Version)
The "Preparation" job is currently running the GitVersion task so it's present during the other jobs.
How can I get the concatenated GitVersion/BuildId-Version into the config files?
The issue was with the Scope of the GitVersion variable. I could access it only in the job where the task was executed.
The solution:
Concatenate the Version (GitVersion.MajorMinorPatch + Build.BuildId) in a powershell script and export it in the job where the GitVersion is present.
- powershell: echo "##vso[task. variable=Version;isOutput=true]$env:VERSION.$env:BUILD_ID"
env:
VERSION: $(GitVersion.MajorMinorPatch)
BUILD_ID: $(Build.BuildId)
name: SetVersion
displayName: Set Version
Map the Version to a Variable in the dependend job:
- job: 'Frontend'
dependsOn: 'Preparation'
variables:
Version: $[ dependencies.Preparation.outputs['SetVersion.Version']]
Now the Variable Version should be present in the Job
I'm currently using Build in Visual Studio Team Services (was Visual Studio Online), and would like to be able to set a Build Variable in a Build Step so that the new value can be used in a subsequent Build Step.
Obviously you can set it before the Build starts but I'm looking to late bind the variable during a subsequent Build Step.
Is this possible?
When inside of a script you can update a variable by emitting the following in your ps1
"##vso[task.setvariable variable=testvar;]testvalue"
You can then pass the variable into the next script using $(testvar)
This doc from the API talks about what ##vso commands you can use.
Don't forget to set system.debug to true. It seems there is a bug that muted stdout and thus, all ##vso are not working.
https://github.com/Microsoft/vso-agent-tasks/blob/master/docs/authoring/commands.md
You can create a powershell script an reference it as a build task.
Then inside your powershell scripts add this:
"##vso[task.setvariable variable=key]value"
After that on all your tasks you can read the variable as $(key).
If you want to protect your variable, use:
"##vso[task.setvariable variable=secretVar;issecret=true]value"
And then use it as $(secretVar) in your next tasks.
I found this link helpful: https://learn.microsoft.com/en-us/azure/devops/pipelines/scripts/logging-commands?view=azure-devops&tabs=powershell
This has the complete options of what you can do: https://learn.microsoft.com/en-us/azure/devops/pipelines/process/variables?view=azure-devops&tabs=yaml%2Cbatch
You can reuse set variable from task to task, and also job to job. I couldn't find anything on stage to stage.
In summary:
jobs:
# Set an output variable from job A
- job: A
pool:
vmImage: 'vs2017-win2016'
steps:
- powershell: echo "##vso[task.setvariable variable=myOutputVar;isOutput=true]this is the value"
name: setvarStep
- script: echo $(setvarStep.myOutputVar)
name: echovar
# Map the variable into job B
- job: B
dependsOn: A
pool:
vmImage: 'ubuntu-16.04'
variables:
myVarFromJobA: $[ dependencies.A.outputs['setvarStep.myOutputVar'] ] # map in the variable
# remember, expressions require single quotes
steps:
- script: echo $(myVarFromJobA)
name: echovar