I am trying this for quite sometime now & not able to figure out how to proceed further. My requirement is to dynamically calculate a variable in a bash task & then in next step use the same as a parameter to the gulp task. Below are my two tasks as part of the build pipeline (removed lines for simplicity)
- task: Bash#3
name: SetVariableValue
inputs:
targetType: inline
script: >
// removed
myvariableValue=$(do something & calculate here, assume value will be 'abc')
// Set to an output variable
echo "##vso[task.setvariable variable=myVar;isOutput=true]$myvariableValue"
- task: gulp#0
displayName: Publish front-end
inputs:
gulpFile: $(Build.SourcesDirectory)/../gulpfile.js
targets: publish
arguments: >-
--buildId $(Build.BuildId) --buildNumber $(Build.BuildNumber) --sourceBranch $(Build.SourceBranch) --var $(myVar)
gulpjs: >-
$(Build.SourcesDirectory)/../node_modules/gulp/bin/gulp.js
enableCodeCoverage: false
I am using linux machines to build angular packages and the gulp publish command is used to package our code depending on the said variable.
With above steps all the rest parameters like BuildId, BuildNumber, SourceBranch are getting passed correctly but the 'var' parameter is being passed as $(myVar) only, rather abc
Can you please help me on what am I missing here? I tried multiple things like --var $(Build.myVar), but not able to make it work.
Thanks
Sanjay
Oh man all I needed was to correct the name of the step & refer it without mistyping :(
--var $(SetVariableValue.myVar) did the trick. Thanks.
Related
I've tried several articles and threads from Stackoverflow but can't seem to get anywhere. I am trying to take a variable from a .py file which is called in a YAML step and set that variable globally to be used.
In my .py file i have
print(f'##vso[task.setvariable variable=AMLPipelineId;isOutput=true]{pipelineId}')
Then in my YAML pipeline step i have
- task: AzurePowerShell#5
displayName: 'Run AML Pipeline'
inputs:
azureSubscription: '$(azureSubscription)'
ScriptType: 'InlineScript'
name: AmlPipeline
azurePowerShellVersion: 'LatestVersion'
pwsh: true
Inline: |
$username = "$(ARM_CLIENT_ID)"
$password = "$(ARM_CLIENT_SECRET)"
$tenantId = "$(ARM_TENANT_ID)"
python $(Pipeline.Workspace)/AML_Pipeline/build_aml_pipeline.py --wsName $(wsName) --resourceGroup $(ResourceGroupName) --subscriptionId $(subId)
$MLPipelineId = $AmlPipeline.AMLPipelineId
But it seems like this variable is empty. I know there are other ways of using the "set variable" but this is my latest attempt i.e. something like print('##vso[task.setvariable variable=version;]%s' % (version))
My current approach i followed: https://learn.microsoft.com/en-us/azure/devops/pipelines/process/variables?view=azure-devops&tabs=yaml%2Cbatch
You don't need isOutput=true - that's only needed for referencing variables between different jobs or stages.
"You cannot use the variable in the step that it is defined." - split that script into two steps: one that runs your .py file, second one that uses the newly defined variable.
I used print('##vso[task.setvariable variable=<Variable-in-Pipeline]+<output-variable>')
Variable-in-Pipeline // the given name should be used in Azure Devops pipeline and should be added to pipeline variables as an empty string
A very minimal example for everyone struggling with this. The documentation is kind of lacking on this for my taste. As #qbik said, dont set and use the variable in the same step, make it seperate steps.
set_variable.py
if __name__ == '__main__':
# set name of the variable
name = 'COLOR'
# set value of the variable
value = 'red'
# set variable
print(f'##vso[task.setvariable variable={name};]{value}')
azure-pipelines.yml
trigger:
- main
pool:
vmImage: ubuntu-latest
steps:
- task: UsePythonVersion#0
inputs:
versionSpec: '3.9'
displayName: 'Use Python 3.9'
# run the script to set the variable
- task: PythonScript#0
inputs:
scriptSource: filePath
scriptPath: set_variable.py
# now you can use the variable in the next step
- bash: echo my favorite color is: $(COLOR)
Now you can technically do all kinds of cool stuff in python, then set, and reference the variables in the following steps. In my case I have to extract specific package version numbers from a JSON/YAML file based on an id that is set earlier in the pipeline and parse the information as an args for a docker build. Hope that helps other people stumbling across this answer looking for a minimal working example :)
I have an Azure DevOps project with a couple of YAML build pipelines that share a common template for some of their tasks.
I now want to add a DownloadSecureFile task to that template, but I can't find a way to get it to work.
The following snippet results in an error when added to the template, but works fine in the parent pipeline definition (Assuming I also replace the ${{ xx }} syntax for the variable names with the $(xx) version):
- task: DownloadSecureFile#1
name: securezip
displayName: Download latest files
inputs:
secureFile: xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
retryCount: 5
- task: ExtractFiles#1
displayName: Extract files
inputs:
archiveFilePatterns: ${{ variables.securezip.secureFilePath }}
destinationFolder: '${{ parameters.sourcesDir }}\secure\'
cleanDestinationFolder: true
The error occurs on the 'Extract File' step and is Input required: archiveFilePatterns, so it looks like it's just not finding the variable.
As a workaround, I could move the download task to the parent pipeline scripts and pass the file path as a parameter. However, that means duplicating the task, which seems like a bit of a hack.
Variables in dollar-double-curly-brackets are resolved at template expansion time. They are not the output of tasks.
Output variables from tasks are referenced by dollar-single-parentheses and they don't need to start with the word "variables."
So I believe the line you're looking for is like this, and it isn't affected by the template mechanism.
archiveFilePatterns: $(securezip.secureFilePath)
I'm looking for a way to define a variable in my azure-pipelines.yml file where I can substring the 'Build.SourceVersion' -> Use the first 7 characters only.
Seems like there is no build-in function that can do such string operations in the documentation. Is there something I'm missing?
My other approach would be to use a bash task and overwrite the variable there but finding something build-in that can do this would be way better solution.
My other approach would be to use a bash task and overwrite the variable there but finding something build-in that can do this would be way better solution.
I agree with Lucas. There is no such built-in task to get the first 7 characters of $(Build.SourceVersion) in Azure DevOps.
We could use the command line/powershell task to split long sha into short sha:
echo $(Build.SourceVersion)
set TestVar=$(Build.SourceVersion)
set MyCustomVar=%TestVar:~0,7%
echo %MyCustomVar%
echo ##vso[task.setvariable variable=ShortSourceVersion]%MyCustomVar%
In this case, we could get the short versions of Build.SourceVersion and set it as environment variable.
Then we could set this command line task as a task group:
So, we could use this task to set the ShortSourceVersion directly.
Hope this helps.
Here is the shortest version that I use for this job;
- bash: |
longcommithash=$(Build.SourceVersion)
echo "##vso[task.setvariable variable=shorthash;]$(echo ${longcommithash::9})"
It gives you the output as shown below;
On Azure devops it's possible to be done using batch technique indeed - like other answers propose in here - however I wanted to remove string from the end of string - and with batch this gets more complex ( See following link ).
Then I've concluded to use Azure's built-in expressions, for example like this:
- name: sourceBranchName
value: ${{ replace(variables['Build.SourceBranch'], 'refs/heads/') }}
replace will have similar effect to substring - so it will remove unnecessary part from original string.
If however you want to concatenate some string and then remove it from original name - it's possible to be done using for instance format function:
- name: originBranchName
value: ${{ replace(variables.sourceBranchName, format('-to-{0}', variables.dynamicvar ) ) }}
If dynamicvar is a - then function will remove -to-a from branch name.
You are right, there is no native way to do it. You will have to write a script to transform the variable.
Here is an example:
trigger:
- master
resources:
- repo: self
stages:
- stage: Build
displayName: Build image
jobs:
- job: Build
displayName: Build
pool:
vmImage: 'ubuntu-latest'
steps:
- task: CmdLine#2
inputs:
script: ' x=`echo "$(Build.SourceVersion)" | head -c 7`; echo "##vso[task.setvariable variable=MyVar]$x"'
- task: CmdLine#2
inputs:
script: 'echo "$(MyVar)"'
Sure you can! If you absolutely must. Here is a runtime computation taking the first 7 characters of the Build.SourceVersion variable.
variables:
example: ${{ format('{0}{1}{2}{3}{4}{5}{6}', variables['Build.SourceVersion'][0], variables['Build.SourceVersion'][1], variables['Build.SourceVersion'][2], variables['Build.SourceVersion'][3], variables['Build.SourceVersion'][4], variables['Build.SourceVersion'][5], variables['Build.SourceVersion'][6]) }}
NB: I can't get it to work with $[...] syntax as the variable is apparently empty at the initial.
I have a standard .NET Core (Ubuntu) pipeline on Azure Devops and within my Test project, I use environment variables. Within my pipeline, I have defined my group variables like so
variables:
- group: MyApiVariables
Whenever I run the tests for my project
- task: DotNetCoreCLI#2
displayName: "Testing Application"
inputs:
command: test
projects: '**/*Tests/*.csproj'
arguments: '--configuration $(buildConfiguration)'
The actual environment variables aren't passed in. They are blank.
What am I missing to get this running? I've even defined variables in the edit pipeline page too with no luck
- task: Bash#3
inputs:
targetType: 'inline'
script: echo $AppConfigEndpoint
env:
AppConfigEndpoint: $(AppConfigEndpoint)
ApiConfigSection: $(ApiConfigSection)
Thanks!
CASING Strikes again! MyVariableName was turned into MYVARIABLENAME on Azure Devops. I changed my variable names in my group to all caps and it worked. I spent way too much time on this.
Mike, I see that you use variable groups. I assume it may cause your issue. Take a look at variable passing example I made:
First I had to create new variable group in Library:
Here is a pipeline code that reference created variables:
# Set variables group reference
variables:
- group: SampleVariableGroup
steps:
- powershell: 'Write-Host "Config variable=$(configuration) Platform variable=$(platform)"'
displayName: 'Display Sample Variable'
I used PowerShell task to verify if variables were properly passed to the job.
As you can see both configuration & platform values were displayed correctly.
In fact you can't go wrong that way, unless you start to mix variable groups with variables defined in a yaml. In such scenario you'll have to use name/value syntax for the individual (non-grouped) variables.
Please see Microsoft Variable Groups documentation. Such example is well explained there. I also suggest to take closer look at general Variables Documentation.
In case of referencing variables in other tasks here is a great example from MS (it should work everywhere in same manner):
# Set variables once
variables:
configuration: debug
platform: x64
steps:
# Use them once
- task: MSBuild#1
inputs:
solution: solution1.sln
configuration: $(configuration) # Use the variable
platform: $(platform)
# Use them again
- task: MSBuild#1
inputs:
solution: solution2.sln
configuration: $(configuration) # Use the variable
platform: $(platform)
Good luck!
I'm currently using Build in Visual Studio Team Services (was Visual Studio Online), and would like to be able to set a Build Variable in a Build Step so that the new value can be used in a subsequent Build Step.
Obviously you can set it before the Build starts but I'm looking to late bind the variable during a subsequent Build Step.
Is this possible?
When inside of a script you can update a variable by emitting the following in your ps1
"##vso[task.setvariable variable=testvar;]testvalue"
You can then pass the variable into the next script using $(testvar)
This doc from the API talks about what ##vso commands you can use.
Don't forget to set system.debug to true. It seems there is a bug that muted stdout and thus, all ##vso are not working.
https://github.com/Microsoft/vso-agent-tasks/blob/master/docs/authoring/commands.md
You can create a powershell script an reference it as a build task.
Then inside your powershell scripts add this:
"##vso[task.setvariable variable=key]value"
After that on all your tasks you can read the variable as $(key).
If you want to protect your variable, use:
"##vso[task.setvariable variable=secretVar;issecret=true]value"
And then use it as $(secretVar) in your next tasks.
I found this link helpful: https://learn.microsoft.com/en-us/azure/devops/pipelines/scripts/logging-commands?view=azure-devops&tabs=powershell
This has the complete options of what you can do: https://learn.microsoft.com/en-us/azure/devops/pipelines/process/variables?view=azure-devops&tabs=yaml%2Cbatch
You can reuse set variable from task to task, and also job to job. I couldn't find anything on stage to stage.
In summary:
jobs:
# Set an output variable from job A
- job: A
pool:
vmImage: 'vs2017-win2016'
steps:
- powershell: echo "##vso[task.setvariable variable=myOutputVar;isOutput=true]this is the value"
name: setvarStep
- script: echo $(setvarStep.myOutputVar)
name: echovar
# Map the variable into job B
- job: B
dependsOn: A
pool:
vmImage: 'ubuntu-16.04'
variables:
myVarFromJobA: $[ dependencies.A.outputs['setvarStep.myOutputVar'] ] # map in the variable
# remember, expressions require single quotes
steps:
- script: echo $(myVarFromJobA)
name: echovar