Azure DevOps YAML If condition - azure-devops

I am attempting to use the if condition to execute a block of operations but it doesn't seem to be working for me. I'm not entirely sure if I understand the if condition correctly, but this is how I have been using it - or at least a simplified version of it.
# Starter pipeline
# Start with a minimal pipeline that you can customize to build and deploy your code.
# Add steps that build, run tests, deploy, and more:
# https://aka.ms/yaml
name: MyTest-$(date:yyyyMMdd)$(rev:.r)
trigger: none
resources:
- repo: self
clean: false
fetchDepth: 2
variables:
- group: TestVG
- name: EnableSigning
value: true
- name: SignDebugBuilds
value: false
stages:
- stage: Stage_Build
displayName: Build Stage
dependsOn: []
jobs:
- job: Job_Build
displayName: Build Job
condition: succeeded()
dependsOn: []
pool:
vmImage: 'windows-latest'
variables:
BuildPlatform: AnyCPU
strategy:
matrix:
Debug:
BuildConfiguration: Debug
Retail:
BuildConfiguration: Retail
steps:
- template: param-test.yml
parameters:
Value: 'Build Solution 1'
#- script: echo 'Build Solution 1'
- ${{ if and(eq(variables['EnableSigning'], True), or(ne(variables['BuildConfiguration'], 'Debug'), eq(variables['SignDebugBuilds'], True))) }}:
#- script: echo 'Build Solution 2 IF check'
- template: param-test.yml
parameters:
Value: 'Build Solution 2 IF check'
- script: echo 'Build Solution 2 COND check'
condition: and(eq(variables['EnableSigning'], True), or(ne(variables['BuildConfiguration'], 'Debug'), eq(variables['SignDebugBuilds'], True)))
The param-test.yml file is a simple file to echo a line for logging/debugging purposes.
parameters:
Value: string
steps:
- script: echo '${{ parameters.Value }}'
My expectation was that for the debug build I would see the first log and the other 2 logs would get skipped, while all 3 logs would be printed for retail build.
However, what I see is that for the debug build, the first and second logs are printed, while only the 3rd log is skipped (the one with the explicit condition in the script step), while all 3 logs are printed for retail build.
To clarify a few items, I used the 3rd log as a control to check if the condition was correct or not. And this is a highly simplified version of the actual pipeline that I have in our repo. We have a primary pipeline YAML file where all the variables are declared including the EnableSigning and SignDebugBuilds. This calls a template file passing in the EnableSigning and SignDebugBuilds flags to the same. This happens a few times around with subtemplates, till one of the sub-templates, the one responsible for the build, uses these from the parameters in the if check. In the actual pipeline, I have the condition use parameters instead of variables, but the result is still the same.
I have looked at a few documentation but it wasn't very clear as to what we could expect from the if statement. And since template tags don't support an explicit condition that we can pass in, this seems to be the only option aside from maintaining 2 separate versions of the template files corresponding to the flags

In your current situation, the 'if' keyword cannot get the variables['BuildConfiguration'], because this variable is created when the job is running. And 'if' key word needs to use the runtime parameters.
So, every time the job is running, the result of the variables['BuildConfiguration'] under the 'if' keyword is NULL. Because when the job is init, this variable has not been created. This variable will be created when the job is running and then the pipeline will help you create other two jobs under your job.
At present the work around is to split the matrix to different two jobs and use the parameters to instead of the variables.
Here is the demo I create:
name: MyTest-$(date:yyyyMMdd)$(rev:.r)
trigger: none
resources:
- repo: self
clean: false
fetchDepth: 2
parameters:
- name: EnableSigning
type: boolean
displayName: 'EnableSigning'
default: true
- name: SignDebugBuilds
type: boolean
displayName: 'SignDebugBuilds'
default: false
variables:
# - group: TestVG
# - name: EnableSigning
# value: true
# - name: SignDebugBuilds
# value: false
- name: system.debug
value: true
stages:
- stage: Stage_Build
displayName: Build Stage
dependsOn: []
jobs:
- job: Build_Job_Debug
pool:
# vmImage: 'windows-latest'
name: default
variables:
BuildPlatform: AnyCPU
BuildConfiguration: Debug
steps:
- template: param-test.yml
parameters:
Value: 'Build Solution 1'
- ${{ if and(eq(parameters.EnableSigning, true), eq(parameters.SignDebugBuilds, true))}}:
- template: param-test.yml
parameters:
Value: 'Build Solution 2 IF check'
- script: echo 'Build Solution 2 COND check and BuildConfiguration is $(BuildConfiguration)'
name: 'CMD3_check'
condition: eq('${{ parameters.EnableSigning }}', true)
- job: Build_Job_Retail
pool:
# vmImage: 'windows-latest'
name: default
variables:
BuildPlatform: AnyCPU
BuildConfiguration: Retail
steps:
- template: param-test.yml
parameters:
Value: 'Build Solution 1'
- ${{ if or(eq(parameters.EnableSigning, true), eq(parameters.SignDebugBuilds, false))}}:
- template: param-test.yml
parameters:
Value: 'Build Solution 2 IF check'
- script: echo 'Build Solution 2 COND check and BuildConfiguration is $(BuildConfiguration)'
name: 'CMD3_check'
condition: eq('${{ parameters.EnableSigning }}', true)
Also, if you need to use the variable BuildConfiguration, you can also define it under the different job.

Related

Azure DevOps: How do I expand variables in a template in the YML file referring to that template for use in a conditional?

I have two Azure DevOps .yml-based templates, one referring to another. I'm finding that variables specified in the one being referred to do not expand when I need them to in a conditional in the template referring to that template. Here is how they look:
Parent template:
jobs:
- deployment:
displayName: Release
variables:
- template: variables.yml
environment:
name: QA
resourceType: VirtualMachine
strategy:
runOnce:
deploy:
...
#this copies to xxMaster
- task: CopyFiles#2
#condition: eq('${{ parameters.appName }}', 'all')
condition: and(eq(variables['Build.SourceBranchName'], 'master'), eq('${{ configuration }}', '$(productionConfiguration)'))
inputs:
SourceFolder: 'C:\DevOps\$(Build.BuildNumber)\Content\D_C\a\1\s\xxTexas\obj\Release\Package\PackageTmp'
Contents: '**\*.*'
OverWrite: true
TargetFolder: 'C:\Production\Web Sites\xxMaster'
Child Template:
variables:
- name: solution
value: '**/*.sln'
- name: buildPlatform
value: 'Any CPU'
- name: buildConfiguration
value: "${{ parameters.configuration }}"
- name: productionConfiguration
value: 'Horsie'
In this case, when the conditional should work and ${{ configuration }} equals 'Horsie', the YAML is evaluated and comes up with
and(eq(variables['Build.SourceBranchName'], 'master'), eq('Horsie', '$(productionConfiguration)'))
That is, $(productionConfiguration), the variable specified in the referred-to template, is never expanded into 'Horsie.' What should I do to make it so the variable from that sub-template is expanded and my conditional works?
From your YAML sample, the cause of this issue is the format of the variable:$(productionConfiguration) in condition.
You need to use the format: variables['productionConfiguration'] to call the pipeline variable.
Here is an example:
stages:
- stage: deploy
jobs:
- deployment: DeployWeb
displayName: deploy Web App
variables:
- template: template.yml
pool:
vmImage: 'Ubuntu-latest'
# creates an environment if it doesn't exist
environment: 'smarthotel-dev'
strategy:
runOnce:
deploy:
steps:
- script: echo Hello world
condition: and(eq(variables['Build.SourceBranchName'], 'main'), eq('${{ configuration }}', variables['productionConfiguration']))

Using pipeline variables across stages with template jobs

Problem Description
I was having some problems trying to use variables created in one stage in another stage and managed to find various articles old and new describing how this can be done. The more recent articles/posts identifying the new syntax
$[stageDependencies.{stageName}.{jobName}.outputs['{stepName}.{variableName}']
Used like this:
variables:
myVariable: $[stagedependencies.CreateStageVarStage.CreateStageVarJob.outputs['SetValueStep.VariableFromFirstStage']]
This works great until you needed to use job templates.
None of the samples I found online covered the situation of templates. They just demonstrated how multiple stages in the same yaml file could obtain the value.
The syntax depends on being able to put the expression into a variable. Unfortunately, when you use a template for a job, it's not possible to declare variables and passing it as a parameter results in it being unevaluated.
- stage: UseJobTemplateStage
displayName: 'Use Job Template Stage'
dependsOn: CreateStageVarStage
jobs:
- template: templates/job-showstagevars.yml
parameters:
ValueToOutput: $[ stagedependencies.CreateStageVarStage.CreateStageVarJob.outputs['SetValueStep.VariableFromFirstStage'] ]
In this snippet, it comes through as-is. The value does not get substituted in.
Theoretically, you could set your job to have the expression present in the variables block but that sort of hard-coding undermines one of the main benefits of templates.
Related Articles
Share variables across stages in Azure DevOps Pipelines
Azure DevOps Release Notes - sprint 168
I know the question asks about template jobs, but for future reference I want to describe how it can be achieved with template stages as well.
It is done with a variable workaround as in the accepted answer, and with a reference to stagedependencies when no dependsOn exists. (Templates don't allow dependsOn). Somehow, this still works.
Example YAML using stage template (I have modified the code from the accepted answer):
stages:
- stage: CreateStageVarStage
displayName: 'Create StageVar Stage'
jobs:
- job: CreateStageVarJob
displayName: 'Create StageVar Job'
timeoutInMinutes: 5
pool:
name: 'Azure Pipelines'
vmImage: 'windows-2019'
steps:
- checkout: none
- pwsh: |
[string]$message = 'This is the value from the first stage'
Write-Host "Setting output variable 'VariableFromFirstStage' to '$message'"
Write-Output "##vso[task.setvariable variable=VariableFromFirstStage;isOutput=$true]$message"
name: SetValueStep
# stage template cannot use dependsOn, but is still allowed to refer to stagedependencies...
- template: templates/stage-showstagevars.yml
parameters:
ValueToOutput: $[ stagedependencies.CreateStageVarStage.CreateStageVarJob.outputs['SetValueStep.VariableFromFirstStage'] ]
Stage template:
parameters:
- name: ValueToOutput
type: string
stages:
- stage: ShowStageVarStage
variables:
- name: LocalVarOfValueToOutputParam
value: ${{ parameters.ValueToOutput }}
jobs:
- job: ShowStageVarJob
displayName: 'Show stage var'
pool:
name: 'Azure Pipelines'
vmImage: 'windows-2019'
steps:
- checkout: none
- pwsh: |
Write-Host "ValueToOutput parameter=${{ parameters.ValueToOutput }}"
Write-Host "LocalVarOfValueToOutputParam (pre-processor syntax)=${{ variables.LocalVarOfValueToOutputParam }}"
Write-Host "LocalVarOfValueToOutputParam (macro syntax)=$(LocalVarOfValueToOutputParam)"
displayName: 'Show StageVariable'
Solution
The answer isn't actually far away. The original expression just need to be passed through a variable in the template job. Basically, set a variable to be the value of the parameter and use the macro syntax to evaluate the variable.
parameters:
- name: ValueToOutput
type: string
...
variables:
- name: LocalVarOfValueToOutputParam
value: ${{ parameters.ValueToOutput }}
Using the macro syntax of $(LocalVarOfValueToOutputParam) will result in the value making its way into the template job correctly.
Example
If we have a yaml file for the build definition:
stages:
- stage: CreateStageVarStage
displayName: 'Create StageVar Stage'
jobs:
- job: CreateStageVarJob
displayName: 'Create StageVar Job'
timeoutInMinutes: 5
pool:
name: 'Azure Pipelines'
vmImage: 'windows-2019'
steps:
- checkout: none
- pwsh: |
[string]$message = 'This is the value from the first stage'
Write-Host "Setting output variable 'VariableFromFirstStage' to '$message'"
Write-Output "##vso[task.setvariable variable=VariableFromFirstStage;isOutput=$true]$message"
name: SetValueStep
- stage: UseJobTemplateStage
displayName: 'Use Job Template Stage'
dependsOn: CreateStageVarStage
jobs:
- template: templates/job-showstagevars.yml
parameters:
ValueToOutput: $[ stagedependencies.CreateStageVarStage.CreateStageVarJob.outputs['SetValueStep.VariableFromFirstStage'] ]
That uses the job template templates/job-showstagevars.yml
parameters:
- name: ValueToOutput
type: string
jobs:
- job: ShowStageVarJob
displayName: 'Show stage var'
timeoutInMinutes: 5
pool:
name: 'Azure Pipelines'
vmImage: 'windows-2019'
variables:
- name: LocalVarOfValueToOutputParam
value: ${{ parameters.ValueToOutput }}
steps:
- checkout: none
- pwsh: |
Write-Host "ValueToOutput parameter=${{ parameters.ValueToOutput }}"
Write-Host "LocalVarOfValueToOutputParam (pre-processor syntax)=${{ variables.LocalVarOfValueToOutputParam }}"
Write-Host "LocalVarOfValueToOutputParam (macro syntax)=$(LocalVarOfValueToOutputParam)"
displayName: 'Show StageVariable'
What we get in our output of the second stage is:
ValueToOutput parameter=$[ stagedependencies.CreateStageVarStage.CreateStageVarJob.outputs['SetValueStep.VariableFromFirstStage'] ]
LocalVarOfValueToOutputParam (pre-processor syntax)=$[ stagedependencies.CreateStageVarStage.CreateStageVarJob.outputs['SetValueStep.VariableFromFirstStage'] ]
LocalVarOfValueToOutputParam (macro syntax)=This is the value from the first stage

Deselect Stages By Default

In Azure Devops multistage YAML pipeline we got multiple environments.
In stages to run normally we do a build and deploy only in QA, so we need to deselect each stage manually. By default all stages are selected is it possible to have exact opposite, where all stages are deselected by default???
trigger: none
pr: none
stages:
- stage: 'Build'
jobs:
- deployment: 'Build'
pool:
name: Default
# testing
environment: INT
strategy:
runOnce:
deploy:
steps:
- checkout: none
- powershell: |
echo "Hello Testing"
Start-Sleep -Seconds 10
- stage: 'Sandbox'
jobs:
- job: 'Sandbox'
pool:
name: Default
steps:
- checkout: none
# testing
- powershell: |
echo "Hello Testing"
- stage: 'Test'
jobs:
- job: 'DEV'
pool:
name: Default
steps:
- checkout: none
- powershell: |
echo "Hello Testing"
- stage: 'QA'
dependsOn: ['Test','Test1','Test2']
jobs:
- job: 'QA'
pool:
name: Default
steps:
- checkout: none
# Testing
- powershell: |
echo "Hello Testing"
I am afraid that there is no UI (like stage to run) method that can meet your needs.
You could try to add parameters to your Yaml Sample.
Here is an example:
trigger: none
pr: none
parameters:
- name: stageTest
displayName: Run Stage test
type: boolean
default: false
- name: stageBuild
displayName: Run Stage build
type: boolean
default: false
stages:
- ${{ if eq(parameters.stageBuild, true) }}:
- stage: 'Build'
jobs:
- deployment: 'Build'
pool:
name: Default
environment: INT
strategy:
runOnce:
deploy:
steps:
- checkout: none
- powershell: |
echo "Hello Testing"
Start-Sleep -Seconds 10
- ${{ if eq(parameters.stageTest, true) }}:
- stage: Test
dependsOn: []
jobs:
- job: B1
steps:
- script: echo "B1"
The parameters are used to determine whether to run these stages. You could add expressions before the stage to check if the parameter value could meet expression.
The default value is false. This means that the stage will not run by default.
Here is the result:
You can select the stage you need to run by clicking the selection box.
Update
Workaround has some limitations. When the select stage has depenencies, you need to select all dependent stages to run.
For example:
- stage: 'QA'
dependsOn: ['Test','Test1','Test2']
On the other hand, I have created a suggestion ticket to report this feature request. Here is the suggestion ticket link: Pipeline Deselect Stages By Default You could vote and add comment in it .
I've used this solution to build a nuget-package, and:
always push packages from master
conditionally push packages from other branches
Using GitVersion ensures that the packages from other branches get prerelease version numbers, e.g. 2.2.12-my-branch-name.3 or 2.2.12-PullRequest7803.4. The main branch simply gets 2.2.12, so the master branch is recognized as a "regular" version.
The reason I'm repeating the answer above, is that I chose to make the stage conditional instead of using an if:
trigger:
- master
parameters:
- name: pushPackage
displayName: Push the NuGet package
type: boolean
default: false
stages:
- stage: Build
jobs:
- job: DoBuild
steps:
- script: echo "I'm building a NuGet package (versioned with GitVersion)"
- stage: Push
condition: and(succeeded('build'), or(eq('${{ parameters.pushPackage }}', true), eq(variables['build.sourceBranch'], 'refs/heads/master')))
jobs:
- job: DoPush
steps:
- script: echo "I'm pushing the NuGet package"
Like the other answer, this results in a dialog:
But what's different from the (equally valid) solution with '${{ if }}', is that the stage is always shown (even if it's skipped):

Azure Devops YAML pipeline - how to repeat a task

In my YAML pipeline I have a deployment job:
- stage: deployment_to_development
jobs:
- deployment: deployment_to_development
displayName: Deploy to Development
environment: Development
variables:
- template: migrate-vars.yml
strategy:
runOnce:
deploy:
steps:
- template: migrate-data.yml
The deployment template is a simple DbUp task:
steps:
- task: UpdateDatabaseWithDbUp#2
displayName: Migrate data
inputs:
ConnectionString: 'Data Source=$(DatabaseServer);Initial Catalog=$(DatabaseName);Integrated Security=SSPI'
ScriptPath: '$(Pipeline.Workspace)\data-migrations'
JournalSchemaName: dbo
JournalTableName: _SchemaVersions
LogScriptOutput: true
IncludeSubfolders: true
Order: FolderStructure
TransactionStrategy: SingleTransaction
The variables template defines the server and db name:
variables:
DatabaseServer: 'server'
DatabaseName: 'dbName'
Instances: '_1,_2'
This all works fine for a single database. However, I wish to repeat the task for each instance defined in the Instances variable, i.e for databases named dbName_1, dbName_2. This seemingly simple feat is proving difficult.
I have tried passing the instances as a parameter array and iterating them using
parameters:
param: []
steps:
- ${{each p in parameters.param}}:
but the 'p' variable isn't evaluated in the task.
There have been many more futile attempts. I must be missing something very obvious here. What is it?
I can't test UpdateDatabaseWithDbUp#2 but I have sth what explain how you can achieve your goal. First define template.yaml
parameters:
- name: 'instances'
type: object
default: {}
- name: 'server'
type: string
default: ''
steps:
- ${{ each instance in parameters.instances }}:
- script: echo ${{ parameters.server }}:${{ instance }}
then reuse this template in your build:
steps:
- template: template.yaml
parameters:
instances:
- test1
- test2
server: someServer
And here is the result:

Azure Devops pipeline finish with warning badge when a job failed

I have a pipeline (Azure Devops yaml) with some jobs.
They run in parallel.
But when one fails, the build ends with a warning and not an error badge.
Why?
I expected the pipeline to end in error.
Edit:
pipelines.yml
pool:
name: Default
jobs:
- job: FirstJob
steps:
- script: echo First!
- template: template.yml
parameters:
name: Second
- template: template.yml
parameters:
name: Third
- template: template.yml
parameters:
name: Fourth
- template: template.yml
parameters:
name: Fifth
- template: template.yml
parameters:
name: Sixth
- template: template.yml
parameters:
name: Seventh
template.yml:
parameters:
- name: name
type: string
default: --
jobs:
- job: ${{ parameters.name }}
dependsOn: FirstJob
continueOnError: true
variables:
BuildTag: ${{ parameters.name }}
steps:
- task : AddTag#0
inputs:
tags: '$(BuildTag)'
- task: DotNetCoreCLI#2
inputs:
command: 'build'
projects: 'foo'
condition: and(succeeded(), eq(variables['BuildTag'], 'Sixth'))
I see my mistake with continueOnError: true.
Azure Devops pipeline finish with warning badge when a job failed
Thanks for your comments and sample, which make me found out the reason for this issue:
The property continueOnError. This behavior is by designed and is not a bug. There is no way to fix it at present.
If we set the continueOnError: true, which will make the future jobs should run even if this job fails. In order to achieve it, Azure Devops will use a "deceptive way" to treat error as warning, so that the error will not prevent the building. That the reason why the job failed, but the pipeline show it as warning.
We could even just reproduce this issue with Control Options Continue on error in non-YAML task:
Besides, it does not affect the completion of the PR.
To resolve this issue, you could comment it in the YAML. If it is necessary for you, you could set the condition: always() for the future jobs:
jobs:
- job: ${{ parameters.name }}
dependsOn: FirstJob
variables:
BuildTag: ${{ parameters.name }}
steps:
...
- job: Second
condition: always()
- job: Third
condition: always()
Hope this helps.
Maybe u need to uncheck "advanced" => "FAIL ON STDERR" (If this option is selected, the build will fail when the remote commands or script write to STDERR).