I have an issue with a variable evaluating to Null on a Condition.
This is the pipeline:
I have a variable template that is pulled in by the main yml file:
variables:
- ${{ if startsWith(variables['Build.SourceBranch'], 'refs/heads/feature/') }}:
- name: is_sandbox
value: true
- ${{ if eq(variables['system.pullRequest.targetBranch'], 'refs/heads/master') }}:
- name: is_sandbox
value: true
- name: bob
value: a
Note that is_sandbox is not set anywhere else.
I then have a condition on job:
condition: or (eq(variables.is_sandbox, true), eq(variables.bob, 'a'))
But this fails to evaluate. In the log is see:
system.pullRequest.targetBranch : refs/heads/master
and:
Expanded: or(eq(Null, True), eq(Null, 'a'))
Result: False
So it appears that the variables template has not set correctly. Why would that be?
Stage variables will only be available to activities within the stage. This is beneficial when reusing variables with different values across different stages. Think potentially stage is tied to an environment and the environment values are different so thus can pass in different variables at the stage level.
I believe you are looking to define these at the root level so variable values will be available to anything in the pipelines unless it is overwritten by a lower level scope.
Related
I am trying to use if else conditions in Azure Devops yml pipeline with variable groups. I am trying to implement it as per latest Azure Devops yaml pipeline build.
Following is the sample code for the if else condition in my scenario. test is a variable inside my-global variable group.
variables:
- group: my-global
- name: fileName
${{ if eq(variables['test'], 'true') }}:
value: 'product.js'
${{ elseif eq(variables['test'], false) }}:
value: 'productCost.js'
jobs:
- job:
steps:
- bash:
echo test variable value $(fileName)
When the above code is executed, in echo statement we don't see any value for filename, i.e. it empty, meaning none of the above if else condition was executed, however when I test the if else condition with the following condition.
- name: fileName
${{ if eq('true', 'true') }}:
value: 'product.js'
Filename did echo the correct value, i.e. product.js. So my conclusion is that I am not able to refer the variables from the variable group correctly. So any suggestion will be helpful and appreciated.
Thanks!
Unfortunately there is no ternary operator in Azure DevOps Pipelines. And it seems unlikely considering the state of https://github.com/microsoft/azure-pipelines-yaml/issues/256 and https://github.com/microsoft/azure-pipelines-yaml/issues/278. So for the time being the only choices are :
conditional insertion : it works with parameters, and should work with variables according to the documentation (but it is difficult to use properly),
or the hacks you can find in this Stack Overflow question.
Another work-around has been posted by Simon Alling on GitHub (https://github.com/microsoft/azure-pipelines-yaml/issues/256#issuecomment-1077684972) :
format(
replace(replace(condition, True, '{0}'), False, '{1}'),
valueIfTrue,
valueIfFalse
)
It is similar to the solution provided by Tejas Nagchandi, but I find it a little bit better because the syntax looks closer to what it would be if there was a ternary operator.
I was able to achieve the goal using some dirty work-around, but I do agree that using parameters would be much better way unless ternary operators are available for Azure DevOps YAML pipeline.
The issue is that ${{ if condition }}: is compile time expression, thus the variables under variable group are not available.
I was able to use runtime expressions $[<expression>]
Reference: https://learn.microsoft.com/en-us/azure/devops/pipelines/process/expressions?view=azure-devops
My pipeline:
trigger:
- none
variables:
- group: Temp-group-for-testing
- name: fileName
value: $[replace(replace('True',eq(variables['test'], 'True'), 'value1'),'True','value2')]
stages:
- stage: test
jobs:
- job: testvar
continueOnError: false
steps:
- bash: echo $(fileName)
displayName: "echo variable"
Results are available on github
After detailed investigation I realized that if else doesnt work with variables in Az Devop yaml pipelines, it only works with parameters. However the solution posted by #Tejas Nagchandi is a workaround and might be able to accomplish the same logic of if else setting variable value with replace commands. Hats off to TN.
I am trying to do something like this
variables:
${{ if eq(variables['abc'], 'dev') }}:
someOtherVariable: '123'
with a variable defined via UI here:
It doesn't work. someOtherVariable is not defined after this.
Is there a way to use this variable in conditions? What should be the syntax?
thanks
The syntax is correct but doesn't seem to work for conditions that rely on variables within the block where variables are defined for custom variables.
It is one of the (many) quirks of the Azure Pipelines YAML processing pipeline. Some conditions, variables, templates, syntax is only available at specific stages of the YAML processing and it depends on whether you are in a pipeline, template, or decorator.
Simplest solution is to use a script step to set the variable and optionally make that step conditional:
${{ if eq(variables['condition'], 'true') }}:
script: echo '##vso[task.setvariable variable=someOtherVariable]123'
or rely on one of my tasks to do that on your behalf:
- task: SetVariable#1
inputs:
name: 'someOtherVariable'
value: '123'
condition: eq(variables['condition'], 'true')
or:
${{ if eq(variables['condition'], 'true') }}:
- task: SetVariable#1
inputs:
name: 'someOtherVariable'
value: '123'
I am quite new in Yaml and AzureDevOps and I am trying to initialize variable based on condition
variables:
- name: DisplayName
${{ if eq('$(env)', 'wh') }}:
value: 'DisplayNAME-WH'
${{ if eq('$(env)', 'lw') }}:
value: 'DisplayNAME-LW'
Where my 'env' is variable passed via UI in Azure DevOps
Issue what I am getting that "DisplayName" stays still empty (is not initialized).
Can you help me out?
Thanks.
Azure DevOps conditional initialization of variables is not working
According to the Understand variable syntax:
In a pipeline, template expression variables (${{ variables.var }})
get processed at compile time, before runtime starts. Macro syntax
variables ($(var)) get processed during runtime before a task runs.
So, we could not use the $(env) in the ${{ if eq('$(env)', 'wh') }}. That because the syntax of ${{ if eq('$(env)', 'wh') }} is parsed before the syntax of $(env).
To resolve this issue, we could to defined the variable as Runtime parameters:
parameters:
- name: 'env'
default: 'wh'
type: string
values:
- wh
- lw
variables:
${{ if eq(parameters.env, 'wh') }}:
DisplayName: 'DisplayNAME-WH'
${{ if eq(parameters.env, 'lw') }}:
DisplayName: 'DisplayNAME-LW'
steps:
- script:
echo $(DisplayName)
The test result:
In Azure DevOps (YAML pipeline), we have a stages that should be run only after another set of stages have been skipped.
In the example below, the parameter copyStages_UAT can be amended by users when triggering a manual run, meaning it's impossible to hard-code the dependsOn and condition properties, so necessitating the use of the directive each.
- template: ../Stages/stage--code--depoly-to-environment.yml
parameters:
name: Deploy_PRD_UKS
displayName: Deploy PRD - UK South
dependsOn:
- ${{ each uatStage in parameters.copyStages_UAT }}:
- Roll_Back_${{ uatStage.name }}
variables:
- template: ../Variables/variables--code--global.yml
- template: ../Variables/variables--code--prd.yml
environment: PRD
This stage above works in a pipeline, however because a successful run results in stages defined in dependsOn being skipped, sadly then Azure DevOps will also skip this stage.
To counter this, I'm trying to add a condition to check whether or not the previous stages were all skipped.
condition: >-
and(replace(
${{ each uatStage in parameters.copyStages_UAT }}:
eq(dependencies.Roll_Back_${{ uatStage.name }}.result, 'Skipped'),
), ', )', ' )')
Unfortunately though, it seems as though I cannot use the directive each in this context -
The directive 'each' is not allowed in this context. Directives are not supported for expressions that are embedded within a string. Directives are only supported when the entire value is an expression.
As condition can only be a string, how can I leverage expressions and/or directives to construct my desired condition?
Example of desired YAML
Assuming the following value was given for the parameter copyStages_UAT -
- name: UAT_UKS
displayName: UAT - UK South
- name: UAT_UKW
displayName: UAT - UK West
This is how the YAML should be compiled. I'm not worried out the format of the condition, as long as the relevant checks are included.
- template: ../Stages/stage--code--depoly-to-environment.yml
parameters:
name: Deploy_PRD_UKS
displayName: Deploy PRD - UK South
dependsOn:
- Roll_Back_UAT_UKS
- Roll_Back_UAT_UKW
condition: >-
and(
eq(dependencies.Roll_Back_UAT_UKS.result, 'Skipped'),
eq(dependencies.Roll_Back_UAT_UKW.result, 'Skipped')
)
variables:
- template: ../Variables/variables--code--global.yml
- template: ../Variables/variables--code--prd.yml
environment: PRD
Azure DevOps Pipelines does not have a particularly good way for solving this. However, and(...),
join(delimiter, ...) and 'Filtered Arrays' can be used to hackily accomplish this.
Observe that the following condition could be rearranged:
and(
eq(dependencies.Roll_Back_UAT_UKW.result, 'Skipped'),
eq(dependencies.Roll_Back_UAT_UKX.result, 'Skipped'),
eq(dependencies.Roll_Back_UAT_UKY.result, 'Skipped'),
eq(dependencies.Roll_Back_UAT_UKZ.result, 'Skipped')
)
and(
eq(dependencies.Roll_Back_
UAT_UKW
.result, 'Skipped'), eq(dependencies.Roll_Back_
UAT_UKX
.result, 'Skipped'), eq(dependencies.Roll_Back_
UAT_UKY
.result, 'Skipped'), eq(dependencies.Roll_Back_
UAT_UKZ
.result, 'Skipped')
)
Or more abstractly, where PREFIX=eq(dependencies.Roll_Back_ and SUFFIX=.result, 'Skipped'):
and(
<PREFIX>
UAT_UKW
<SUFFIX>, <PREFIX>
UAT_UKX
<SUFFIX>, <PREFIX>
UAT_UKY
<SUFFIX>, <PREFIX>
UAT_UKZ
<SUFFIX>
)
With filtered arrays (NAMES=parameters.parameterName.*.name) to extract the name, an aggregation could then be written:
and(<PREFIX>${{ join('<PREFIX>, <SUFFIX>', <NAMES>) }}<SUFFIX>)
Thus:
condition: |
and(
ne(dependencies.Roll_Back_${{ join('.result, ''Skipped''), ne(dependencies.Roll_Back_', parameters.copyStages_UAT.*.name) }}.result, 'Skipped')
)
But there are some obvious caveats with this:
If there is 0 elements in parameters.copyStages_UAT, then the expression would evaluate to and(dependencies.Roll_Back_.result, 'Skipped') which could be non-sensical.
and(...) requires a minimum of 2 arguments, so this could potentially fail if there is only 0 or 1 expression(s). To circumvent this, True can be supplied as the first and second argument such that the expression is always valid. If your logic instead requires or(...), then use False instead of True to keep the meaning consistent.
Therefore, you may need to protect against these scenarios occurring with a modified check:
${{ if eq(length(parameters.copyStages_UAT), 0) }}:
condition: false
${{ else }}:
condition: |
and(
True,
ne(dependencies.Roll_Back_${{ join('.result, ''Skipped''), ne(dependencies.Roll_Back_', parameters.copyStages_UAT.*.name) }}.result, 'Skipped')
)
Updated:
To summary your demand, you are looking for a expression that you can use it in condition while the dependsOn value are dynamic. And this stage should run only after another set of dependent stages are all skipped.
As far as I know and tested, this can not be achieved via each.
For further confirmation, I discussed this scenario with our pipeline PM who is more familiar with each and YAML pipeline.
Same as me, he also think this isn't possible to achieve. Because ${{ each }} expects to be the outermost part of a mapping key or value, so you can’t nest it into a condition string.
Work around:
You could fake it by having a hard-coded stage which depends on the dynamic list, figures out if they were all skipped, and sets an output variable. Then the real final job would only depend on that “decider” job, and its condition would depend on the contents of the output variable.
Figuring out that all upstream dependencies were skipped is something of an exercise for you. You might be able to dynamically construct one step per stage. Those steps map the stage’s status to a variable with a known name scheme. Then the final (hardcoded) step iterates the environment variables of the known name scheme and decides whether the next stage should proceed.
And... yes, I am aware how ugly that sounds ☹
I'm trying to assign one of 2 values to a variable in addition to variable group and can't find the reference that how to use IF ELSE.
Basically I need to convert this jerkins logic to azure DevOps.
Jenkins
if (branch = 'master') {
env = 'a'
} else if (branch = 'dev'){
env ='b'
}
I found 1 reference from the following one, but this one seems to work if the variables section doesn't have variable groups.
https://stackoverflow.com/a/57532526/5862540
But in my pipeline, I already have a variable group for secrets, so I have to use name/value convention and the example doesn't work with the errors like expected a mapping or A mapping was not expected or Unexpected value 'env'
variables:
- group: my-global
- name: env
value:
${{ if eq(variables['Build.SourceBranchName'], 'master') }}:
env: a
${{ if eq(variables['Build.SourceBranchName'], 'dev') }}:
env: b
or
variables:
- group: my-global
- name: env
value:
${{ if eq(variables['Build.SourceBranchName'], 'master') }}: a
${{ if eq(variables['Build.SourceBranchName'], 'dev') }}: b
This code works.
I'm doing similar with parameters.
variables:
- name: var1
${{ if eq(parameters.var1, 'custom') }}:
value: $(var1.manual.custom)
${{ if ne(parameters.var1, 'custom') }}:
value: ${{ parameters.var1 }}
Update 09/09/2021
We have now natively if else expression and we can write it like
variables:
- group: PROD
- name: env
${{ if eq(variables['Build.SourceBranchName'], 'master') }}:
value: a
${{ else }}:
value: b
steps:
- script: |
echo '$(name)'
echo '$(env)'
Original reply
Syntax with template expressions ${{ if ...... }} is not limited only to job/stage level. Both below pipeline does the same and produce the same output:
stages:
- stage: One
displayName: Build and restore
variables:
- group: PROD
- name: env
${{ if eq(variables['Build.SourceBranchName'], 'master') }}:
value: a
${{ if eq(variables['Build.SourceBranchName'], 'dev') }}:
value: b
jobs:
- job: A
steps:
- script: |
echo '$(name)'
echo '$(env)'
variables:
- group: PROD
- name: env
${{ if eq(variables['Build.SourceBranchName'], 'master') }}:
value: a
${{ if eq(variables['Build.SourceBranchName'], 'dev') }}:
value: b
steps:
- script: |
echo '$(name)'
echo '$(env)'
Microsoft a few weeks ago released a new feature for YAML pipeliens that just lets you do that: IF ELSE notation.
https://learn.microsoft.com/en-us/azure/devops/release-notes/2021/sprint-192-update#new-yaml-conditional-expressions
Writing conditional expressions in YAML files just got easier with the use of ${{ else }} and ${{ elseif }} expressions. Below are examples of how to use these expressions in YAML pipelines files.
steps:
- script: tool
env:
${{ if parameters.debug }}:
TOOL_DEBUG: true
TOOL_DEBUG_DIR: _dbg
${{ else }}:
TOOL_DEBUG: false
TOOL_DEBUG_DIR: _dbg
variables:
${{ if eq(parameters.os, 'win') }}:
testsFolder: windows
${{ elseif eq(parameters.os, 'linux' }}:
testsFolder: linux
${{ else }}:
testsFolder: mac
I wanted to have runtime condition evaluation, something similar to compile time:
variables:
VERBOSE_FLAG:
${{if variables['System.Debug']}}:
value: '--verbose'
${{else}}:
value: ''
but unfortunately Azure devops does not supports special kind of functions like if(condition, then case, else case) - so I've played around and find out that it's possible do double string replacement using replace function. It does looks bit hacky of course.
For example, one may want to tweak task inputs depending on whether system debugging is enabled or not. This cannot be done using "standard conditional insertion" (${{ if … }}:), because System.Debug isn't in scope in template expressions. So, runtime expressions to the rescue:
- job:
variables:
VERBOSE_FLAG: $[
replace(
replace(
eq(lower(variables['System.Debug']), 'true'),
True,
'--verbose'
),
False,
''
)
]
steps:
- task: cURLUploader#2
inputs:
# …
options: --fail --more-curl-flags $(VERBOSE_FLAG)
Note that using eq to check the value of System.Debug before calling replace is not redundant: Since eq always returns either True or False, we can then safely use replace to map those values to '--verbose' and '', respectively.
In general, I highly recommend sticking to boolean expressions (for example the application of a boolean-valued function like eq, gt or in) as the first argument of the inner replace application. Had we not done so and instead just written for example
replace(
replace(
lower(variables['System.Debug']),
'true',
'--verbose'
),
'false',
''
)
then, if System.Debug were set to e.g. footruebar, the value of VERBOSE_FLAG would have become foo--verbosebar.
I think for now you're going to need to use a task to customize with name/value syntax variables and conditional variable values. It looks like the object structure for name/value syntax breaks the parsing of expressions, as you have pointed out.
For me, the following is a reasonably clean implementation, and if you want to abstract it away from the pipeline, it seems that a simple template for your many pipelines to use should satisfy the desire for a central "global" location.
variables:
- group: FakeVarGroup
- name: env
value: dev
steps:
- powershell: |
if ($env:Build_SourceBranchName -eq 'master') {
Write-Host ##vso[task.setvariable variable=env;isOutput=true]a
return
} else {
Write-Host ##vso[task.setvariable variable=env;isOutput=true]b
}
displayName: "Set Env Value"
As far as I know, the best way to have conditional branch build is using "trigger" in your YAML, instead of implementing complex "if-else". It is also much safer, and you have more explicit controls on the branch triggers instead of relying on CI variables.
Example:
# specific branch build
jobs:
- job: buildmaster
pool:
vmImage: 'vs2017-win2016'
trigger:
- master
steps:
- script: |
echo "trigger for master branch"
- job: buildfeature
pool:
vmImage: 'vs2017-win2016'
trigger:
- feature
steps:
- script: |
echo "trigger for feature branch"
To have trigger with branches inclusion and exclusion, you could use more complex syntax of trigger with branches include and exclude.
Example:
# specific branch build
trigger:
branches:
include:
- master
- releases/*
exclude:
- releases/1.*
The official documentation of Azure DevOps Pipelines trigger in YAML is:
Azure Pipelines YAML trigger documentation
UPDATE 1:
I repost my comment here with additional notes:
I was thinking to have different pipelines because having the complexity of juggling between CI variables is not more maintainable than having multi jobs in one YAML with triggers. Having multijobs with triggers is also enforcing us to have clear distinction and provision on branch management. Triggers and conditional branches inclusions have been used for a year by my team because of these maintainability advantages.
Feel free to disagree, but to me having an embedded logic in any scripted in any steps to check which branch is currently in session and then does any further actions, are more like ad-hoc solutions. And this has given my team and me maintenance problems before.
Especially if the embedded logic tends to grow by checking other branches, the complexity is more complex later than having clear separations between branches. Also if the YAML file is going to be maintained for long time, it should have clear provisions and roadmaps across different branches. Redundancy is unavoidable, but the intention to separate specific logic will pay more in the long run for maintainability.
This is why I also emphasize branches inclusions and exclusions in my answer :)
Azure YAML if-else solution (when you have a group defined which required name/value notation use thereafter.
variables:
- group: my-global
- name: env
value: a # set by default
- name: env
${{ if eq(variables['Build.SourceBranchName'], 'master') }}:
value: b # will override default
Of if you don't have a group defined:
variables:
env: a # set by default
${{ if eq(variables['Build.SourceBranchName'], 'master') }}:
env: b # will override default