I'm currently using Build in Visual Studio Team Services (was Visual Studio Online), and would like to be able to set a Build Variable in a Build Step so that the new value can be used in a subsequent Build Step.
Obviously you can set it before the Build starts but I'm looking to late bind the variable during a subsequent Build Step.
Is this possible?
When inside of a script you can update a variable by emitting the following in your ps1
"##vso[task.setvariable variable=testvar;]testvalue"
You can then pass the variable into the next script using $(testvar)
This doc from the API talks about what ##vso commands you can use.
Don't forget to set system.debug to true. It seems there is a bug that muted stdout and thus, all ##vso are not working.
https://github.com/Microsoft/vso-agent-tasks/blob/master/docs/authoring/commands.md
You can create a powershell script an reference it as a build task.
Then inside your powershell scripts add this:
"##vso[task.setvariable variable=key]value"
After that on all your tasks you can read the variable as $(key).
If you want to protect your variable, use:
"##vso[task.setvariable variable=secretVar;issecret=true]value"
And then use it as $(secretVar) in your next tasks.
I found this link helpful: https://learn.microsoft.com/en-us/azure/devops/pipelines/scripts/logging-commands?view=azure-devops&tabs=powershell
This has the complete options of what you can do: https://learn.microsoft.com/en-us/azure/devops/pipelines/process/variables?view=azure-devops&tabs=yaml%2Cbatch
You can reuse set variable from task to task, and also job to job. I couldn't find anything on stage to stage.
In summary:
jobs:
# Set an output variable from job A
- job: A
pool:
vmImage: 'vs2017-win2016'
steps:
- powershell: echo "##vso[task.setvariable variable=myOutputVar;isOutput=true]this is the value"
name: setvarStep
- script: echo $(setvarStep.myOutputVar)
name: echovar
# Map the variable into job B
- job: B
dependsOn: A
pool:
vmImage: 'ubuntu-16.04'
variables:
myVarFromJobA: $[ dependencies.A.outputs['setvarStep.myOutputVar'] ] # map in the variable
# remember, expressions require single quotes
steps:
- script: echo $(myVarFromJobA)
name: echovar
Related
I've tried several articles and threads from Stackoverflow but can't seem to get anywhere. I am trying to take a variable from a .py file which is called in a YAML step and set that variable globally to be used.
In my .py file i have
print(f'##vso[task.setvariable variable=AMLPipelineId;isOutput=true]{pipelineId}')
Then in my YAML pipeline step i have
- task: AzurePowerShell#5
displayName: 'Run AML Pipeline'
inputs:
azureSubscription: '$(azureSubscription)'
ScriptType: 'InlineScript'
name: AmlPipeline
azurePowerShellVersion: 'LatestVersion'
pwsh: true
Inline: |
$username = "$(ARM_CLIENT_ID)"
$password = "$(ARM_CLIENT_SECRET)"
$tenantId = "$(ARM_TENANT_ID)"
python $(Pipeline.Workspace)/AML_Pipeline/build_aml_pipeline.py --wsName $(wsName) --resourceGroup $(ResourceGroupName) --subscriptionId $(subId)
$MLPipelineId = $AmlPipeline.AMLPipelineId
But it seems like this variable is empty. I know there are other ways of using the "set variable" but this is my latest attempt i.e. something like print('##vso[task.setvariable variable=version;]%s' % (version))
My current approach i followed: https://learn.microsoft.com/en-us/azure/devops/pipelines/process/variables?view=azure-devops&tabs=yaml%2Cbatch
You don't need isOutput=true - that's only needed for referencing variables between different jobs or stages.
"You cannot use the variable in the step that it is defined." - split that script into two steps: one that runs your .py file, second one that uses the newly defined variable.
I used print('##vso[task.setvariable variable=<Variable-in-Pipeline]+<output-variable>')
Variable-in-Pipeline // the given name should be used in Azure Devops pipeline and should be added to pipeline variables as an empty string
A very minimal example for everyone struggling with this. The documentation is kind of lacking on this for my taste. As #qbik said, dont set and use the variable in the same step, make it seperate steps.
set_variable.py
if __name__ == '__main__':
# set name of the variable
name = 'COLOR'
# set value of the variable
value = 'red'
# set variable
print(f'##vso[task.setvariable variable={name};]{value}')
azure-pipelines.yml
trigger:
- main
pool:
vmImage: ubuntu-latest
steps:
- task: UsePythonVersion#0
inputs:
versionSpec: '3.9'
displayName: 'Use Python 3.9'
# run the script to set the variable
- task: PythonScript#0
inputs:
scriptSource: filePath
scriptPath: set_variable.py
# now you can use the variable in the next step
- bash: echo my favorite color is: $(COLOR)
Now you can technically do all kinds of cool stuff in python, then set, and reference the variables in the following steps. In my case I have to extract specific package version numbers from a JSON/YAML file based on an id that is set earlier in the pipeline and parse the information as an args for a docker build. Hope that helps other people stumbling across this answer looking for a minimal working example :)
I am trying this for quite sometime now & not able to figure out how to proceed further. My requirement is to dynamically calculate a variable in a bash task & then in next step use the same as a parameter to the gulp task. Below are my two tasks as part of the build pipeline (removed lines for simplicity)
- task: Bash#3
name: SetVariableValue
inputs:
targetType: inline
script: >
// removed
myvariableValue=$(do something & calculate here, assume value will be 'abc')
// Set to an output variable
echo "##vso[task.setvariable variable=myVar;isOutput=true]$myvariableValue"
- task: gulp#0
displayName: Publish front-end
inputs:
gulpFile: $(Build.SourcesDirectory)/../gulpfile.js
targets: publish
arguments: >-
--buildId $(Build.BuildId) --buildNumber $(Build.BuildNumber) --sourceBranch $(Build.SourceBranch) --var $(myVar)
gulpjs: >-
$(Build.SourcesDirectory)/../node_modules/gulp/bin/gulp.js
enableCodeCoverage: false
I am using linux machines to build angular packages and the gulp publish command is used to package our code depending on the said variable.
With above steps all the rest parameters like BuildId, BuildNumber, SourceBranch are getting passed correctly but the 'var' parameter is being passed as $(myVar) only, rather abc
Can you please help me on what am I missing here? I tried multiple things like --var $(Build.myVar), but not able to make it work.
Thanks
Sanjay
Oh man all I needed was to correct the name of the step & refer it without mistyping :(
--var $(SetVariableValue.myVar) did the trick. Thanks.
I'm looking for a way to define a variable in my azure-pipelines.yml file where I can substring the 'Build.SourceVersion' -> Use the first 7 characters only.
Seems like there is no build-in function that can do such string operations in the documentation. Is there something I'm missing?
My other approach would be to use a bash task and overwrite the variable there but finding something build-in that can do this would be way better solution.
My other approach would be to use a bash task and overwrite the variable there but finding something build-in that can do this would be way better solution.
I agree with Lucas. There is no such built-in task to get the first 7 characters of $(Build.SourceVersion) in Azure DevOps.
We could use the command line/powershell task to split long sha into short sha:
echo $(Build.SourceVersion)
set TestVar=$(Build.SourceVersion)
set MyCustomVar=%TestVar:~0,7%
echo %MyCustomVar%
echo ##vso[task.setvariable variable=ShortSourceVersion]%MyCustomVar%
In this case, we could get the short versions of Build.SourceVersion and set it as environment variable.
Then we could set this command line task as a task group:
So, we could use this task to set the ShortSourceVersion directly.
Hope this helps.
Here is the shortest version that I use for this job;
- bash: |
longcommithash=$(Build.SourceVersion)
echo "##vso[task.setvariable variable=shorthash;]$(echo ${longcommithash::9})"
It gives you the output as shown below;
On Azure devops it's possible to be done using batch technique indeed - like other answers propose in here - however I wanted to remove string from the end of string - and with batch this gets more complex ( See following link ).
Then I've concluded to use Azure's built-in expressions, for example like this:
- name: sourceBranchName
value: ${{ replace(variables['Build.SourceBranch'], 'refs/heads/') }}
replace will have similar effect to substring - so it will remove unnecessary part from original string.
If however you want to concatenate some string and then remove it from original name - it's possible to be done using for instance format function:
- name: originBranchName
value: ${{ replace(variables.sourceBranchName, format('-to-{0}', variables.dynamicvar ) ) }}
If dynamicvar is a - then function will remove -to-a from branch name.
You are right, there is no native way to do it. You will have to write a script to transform the variable.
Here is an example:
trigger:
- master
resources:
- repo: self
stages:
- stage: Build
displayName: Build image
jobs:
- job: Build
displayName: Build
pool:
vmImage: 'ubuntu-latest'
steps:
- task: CmdLine#2
inputs:
script: ' x=`echo "$(Build.SourceVersion)" | head -c 7`; echo "##vso[task.setvariable variable=MyVar]$x"'
- task: CmdLine#2
inputs:
script: 'echo "$(MyVar)"'
Sure you can! If you absolutely must. Here is a runtime computation taking the first 7 characters of the Build.SourceVersion variable.
variables:
example: ${{ format('{0}{1}{2}{3}{4}{5}{6}', variables['Build.SourceVersion'][0], variables['Build.SourceVersion'][1], variables['Build.SourceVersion'][2], variables['Build.SourceVersion'][3], variables['Build.SourceVersion'][4], variables['Build.SourceVersion'][5], variables['Build.SourceVersion'][6]) }}
NB: I can't get it to work with $[...] syntax as the variable is apparently empty at the initial.
I am creating YAML pipeline in Azure DevOps that consists of two stages.
The first stage (Prerequisites) is responsible for reading the git commit and creates a comma separated variable containing the list of services that has been affected by the commit.
The second stage (Build) is responsible for building and unit testing the project. This Stage consists of many templates, one for each Service. In the template script, the job will check if the relevant Service in in the variable created in the previous stage. If the job finds the Service it will continue to build and test the service. However if it cannot find the service, it will skip that job.
Run.yml:
stages:
- stage: Prerequisites
jobs:
- job: SetBuildQueue
steps:
- task: powershell#2
name: SetBuildQueue
displayName: 'Set.Build.Queue'
inputs:
targetType: inline
script: |
## ... PowerShell script to get changes - working as expected
Write-Host "Build Queue Auto: $global:buildQueueVariable"
Write-Host "##vso[task.setvariable variable=buildQueue;isOutput=true]$global:buildQueueVariable"
- stage: Build
jobs:
- job: StageInitialization
- template: Build.yml
parameters:
projectName: Service001
projectLocation: src/Service001
- template: Build.yml
parameters:
projectName: Service002
projectLocation: src/Service002
Build.yml:
parameters:
projectName: ''
projectLocation: ''
jobs:
- job:
displayName: '${{ parameters.projectName }} - Build'
dependsOn: SetBuildQueue
continueOnError: true
condition: and(succeeded(), contains(dependencies.SetBuildQueue.outputs['SetBuildQueue.buildQueue'], '${{ parameters.projectName }}'))
steps:
- task: NuGetToolInstaller#1
displayName: 'Install Nuget'
Issue:
When the first stages runs it will create a variable called buildQueue which is populated as seen in the console output of the PowerShell script task:
Service001 Changed
Build Queue Auto: Service001;
However when it gets to stage two and it tries to run the build template, when it checks the conditions it returns the following output:
Started: Today at 12:05 PM
Duration: 16m 7s
Evaluating: and(succeeded(), contains(dependencies['SetBuildQueue']['outputs']['SetBuildQueue.buildQueue'], 'STARS.API.Customer.Assessment'))
Expanded: and(True, contains(Null, 'service001'))
Result: False
So my question is how do I set the dependsOn and condition to get the information from the previous stage?
It because you want to access the variable in a different stage from where you defined them. currently, it's impossible, each stage it's a new instance of a fresh agent.
In this blog you can find a workaround that involves writing the variable to disk and then passing it as a file, leveraging pipeline artifacts.
To pass the variable FOO from a job to another one in a different stage:
Create a folder that will contain all variables you want to pass; any folder could work, but something like mkdir -p $(Pipeline.Workspace)/variables might be a good idea.
Write the contents of the variable to a file, for example echo "$FOO" > $(Pipeline.Workspace)/variables/FOO. Even though the name could be anything you’d like, giving the file the same name as the variable might be a good idea.
Publish the $(Pipeline.Workspace)/variables folder as a pipeline artifact named variables
In the second stage, download the variables pipeline artifact
Read each file into a variable, for example FOO=$(cat $(Pipeline.Workspace)/variables/FOO)
Expose the variable in the current job, just like we did in the first example: echo "##vso[task.setvariable variable=FOO]$FOO"
You can then access the variable by expanding it within Azure Pipelines ($(FOO)) or use it as an environmental variable inside a bash script ($FOO).
My Azure DevOps Build Pipeline makes use of GitVersion for versioning the build artefacts. Having the task present sets the GitVersion-variable that gives access to GitVersion.MajorMinorPatch and so on. What I want to do though is concatenating this variable with the BuildID. Setting the artefact name with the following command works well:
name: $(GitVersion.MajorMinorPatch).$(BuildID)
In the next step I wanted to have the version to be written into various config files. I installed the ReplaceToken plugin that replaces a specific token (in my case #{Version}#) with the content of the variable "Version".
Creating a variable with the name Version and the value "$(GitVersion.MajorMinorPatch).$(BuildID)" simply replaces the variable with the string instead of the evaluated value. I already tried to set the variable in the job that runs after the GitVersion task, so it should already be present when the variable gets created.
eg.
- job: 'Frontend'
dependsOn: 'Preparation'
variables:
Version: $('GitVersion.MajorMinorPatch')
steps:
- bash: echo $(Version)
The "Preparation" job is currently running the GitVersion task so it's present during the other jobs.
How can I get the concatenated GitVersion/BuildId-Version into the config files?
The issue was with the Scope of the GitVersion variable. I could access it only in the job where the task was executed.
The solution:
Concatenate the Version (GitVersion.MajorMinorPatch + Build.BuildId) in a powershell script and export it in the job where the GitVersion is present.
- powershell: echo "##vso[task. variable=Version;isOutput=true]$env:VERSION.$env:BUILD_ID"
env:
VERSION: $(GitVersion.MajorMinorPatch)
BUILD_ID: $(Build.BuildId)
name: SetVersion
displayName: Set Version
Map the Version to a Variable in the dependend job:
- job: 'Frontend'
dependsOn: 'Preparation'
variables:
Version: $[ dependencies.Preparation.outputs['SetVersion.Version']]
Now the Variable Version should be present in the Job