It seems that name is a special magic variable that somehow gets used for my output directory.
(Is this behavior documented anywhere?)
I'm trying to set it.
Given the extraordinary difficulty of writing Azure pipeline yml, it's highly unlikely that I'll get it right. In the absence of any form of debugging I want to add a print statement so that I can see the value.
How?
${{ if eq(variables['Build.SourceBranchName'], 'master') }}:
buildConfiguration: 'Release'
tag: ''
${{ if ne(variables['Build.SourceBranchName'], 'master') }}:
buildConfiguration: 'Debug'
tag: ${{ format('-{0}', variables['Build.SourceBranchName']) }}
# How do you do string concatenation in yml? Do I need to do `format` like above?
name: $(Build.BuildId)$(tag)
steps:
- script: echo "name is $(name)"
But the output is
Generating script.
Script contents:
echo "name is $(name)"
...
name is $(name)"
Is it possible to make this work? How?
The name variable is for the Build.BuildNumber value (see here).
So just print it:
- script: echo "name is $(Build.BuildNumber)"
Related
I have a pretty complex setup of my Pipelines in Azure DevOps for various reasons but I'm kind of stuck in a special scenario now. Let me explain a bit.
There is a Stage_A with Job_A setting a Variable_A. Now there is a Stage_B with Job_B, need to use the Variable_A from Stage_A.Job_A.
The variable in Job_A is set by this:
echo ##vso[task.setvariable variable=Variable_A;isOutput=true]$value
Now, Job_B in Stage_B can access the variable in a condition with
variables:
Variable_A_FromStageA: $[stageDependencies.Stage_A.Job_A.outputs['task_A.Variable_A']]
I can also do an echo on the variable by using
echo $(Variable_A_FromStageA)
the Question is now, how can I use this in an if-statement? I tried different approaches:
- ${{ if eq($(Variable_A_FromStageA), 'True') }}:
- ${{ if eq(variables.Variable_A_FromStageA, 'True') }}:
- ${{ if eq(variables['Variable_A_FromStageA'], 'True') }}:
- ${{ if eq(stageDependencies.Stage_A.Job_A.outputs['task_A.Variable_A'], "True") }}:
Nothing actually works. Either the system complains about syntax issues or it doesn't evaluate it correctly. I don't really know how to use the information in my if statement in the yaml file. The documentation is not really clear about it. It only mentions the usage of a stage dependency in a condition and that's it.
Hope anyone can help me here!
Cheers,
Frank
use stageDependencies in if-statements in Azure DevOps yaml
If you mean you want to use conditional insertion to use the variables output from the logging command, then answer is NO.
The reason is the conditional insertion needs compile time value(you must provide them before pipeline run.), but the variable that the logging command output is runtime. Conditional Insertion will be unable to get it.
The right way is to use "condition" instead of "Conditional Insertion". Using condition can achieve your situation.
I write a demo for you as below:
trigger:
- none
pool:
vmImage: ubuntu-latest
stages:
- stage: A
jobs:
- job: A1
steps:
- bash: echo "##vso[task.setvariable variable=shouldrun;isOutput=true]true"
# or on Windows:
# - script: echo ##vso[task.setvariable variable=shouldrun;isOutput=true]true
name: printvar
- stage: B
condition: and(succeeded(), eq(dependencies.A.outputs['A1.printvar.shouldrun'], 'true'))
variables:
myStageAVar: $[stageDependencies.A.A1.outputs['printvar.shouldrun']]
dependsOn: A
jobs:
- job: B1
steps:
- script: echo $(myStageAVar)
I have a BRANCH_NAME variable based on the trigger branch which is working great. Now based on my BRANCH_NAME value I have to select a json file. Im using starts with but not sure why my TARGET_ENV is not working
variables:
system.debug: 'false'
${{ if startsWith(variables['Build.SourceBranch'],
'refs/heads/') }}:
BRANCH_NAME: $[replace(variables['Build.SourceBranch'],
'refs/heads/', '')]
${{ if startsWith(variables['Build.SourceBranch'],
'refs/heads/feature') }}:
BRANCH_NAME: $[replace(variables['Build.SourceBranchName'],
'/', '-')]
###
${{ if eq(variables['BRANCH_NAME'], 'dev') }}:
TARGET_ENV: dev
${{ if startsWith(variables['BRANCH_NAME'], 'test') }}:
TARGET_ENV: test
${{ if or( startsWith(variables['BRANCH_NAME'], 'XX'), startsWith(variables['BRANCH_NAME'], 'VV') ) }}:
TARGET_ENV: feature
And the value of TARGET_ENV should be used in a script ( echo $TARGET_ENV )
The yaml file isn't executes as soon as an element is parted, the yaml file is parted in stages. During the template expansion stage only the variables set at queue time are available.
After the template has been expanded (e.g. the ${{}} blocks have been evaluated the yaml is interpreted top to bottom. So the value of BRANCH_NAME isn't available yet.
You can set the variable either from a pre (like my variable tasks) or from a script by printing one of the special command statements.
- bash: |
echo "##vso[task.setvariable variable=myVar;]foo"
condition: $[ eq(variables['BRANCH_NAME'], 'dev' ]
Or you can put all the logic in a single PowerShell block and let PowerShell evaluate the values of the variables.
- powershell: |
if ($env:BRANCH_NAME -eq 'dev') {
Write-host ##vso[task.setvariable variable=TARGET_ENV;]dev
}
...
...
This variable will only be available in the job the task runs. You can make it an output variable to make the task available in other jobs, but then you need to refer to it by its long name:
Dependencies.jobname.stepname.outputvariable
See also:
https://learn.microsoft.com/en-us/azure/devops/pipelines/process/set-variables-scripts?view=azure-devops&tabs=bash#about-tasksetvariable
https://stackoverflow.com/a/73609482/736079
I have a YAML file which forms an Azure DevOps pipeline. The pipeline itself defines four variables which are needed in the variables section of the YAML...
variables:
environmentIdentifier: "$(environmentIdentifier)"
keyVaultSourceName: "$(keyVaultSourceName)"
location: "$(location)"
locationIdentifier: "$(locationIdentifier)"
The variables are definitely set for each run of the pipeline, but when it runs I encounter errors further down in my script which indicate that these variables were not populated correctly...
ERROR: (InvalidResourceGroup) The provided resource group name 'rg-main-$(locationIdentifier)' has these invalid characters: '$:'. The name can only be a letter, digit, '-', '.', '(', ')' or '_'.
I've also tried...
$env:location
${{variables['location']}}
...but incurred the same error.
How should I correctly declare vars in the variables section of the pipeline definition, where their values are retrieved from the pipeline's variables?
You need to define as:
variables:
- name: location
value: 'Australia Southeast'
If you want them at a later stage as a template expression use:
${{ variables.location }}
and if you want to use them inside a script:
steps:
- bash: echo $(location)
- powershell: echo $(location)
- script: echo $(location)
Check this Link and the below extracted sample for more information.
variables:
- name: one
value: initialValue
steps:
- script: |
echo ${{ variables.one }} # outputs initialValue
echo $(one)
displayName: First variable pass
- bash: echo '##vso[task.setvariable variable=one]secondValue'
displayName: Set new variable value
- script: |
echo ${{ variables.one }} # outputs initialValue
echo $(one) # outputs secondValue
displayName: Second variable pass
I have a scenario where I need to have both:
runtime parameters, so that the pipeline can be triggered manually from the UI, where users triggering it can choose from a predefined set of options (defined in YAML)
variables, so that the pipeline can be invoked via REST APIs
Regarding runtime parameters, I was able to create the following sample pipeline:
parameters:
- name: image
displayName: Pool Image
type: string
default: ubuntu-latest
values:
- windows-latest
- ubuntu-latest
trigger: none
stages:
- stage: A
jobs:
- job: A
steps:
- pwsh: |
echo "This should be triggering against image: $MY_IMAGE_NAME"
env:
MY_IMAGE_NAME: ${{ parameters.image }}
When I run it, I can see the dropdown list where I can choose the image name and it is reflected in the output message of the PowerShell script.
Regarding variables, I have defined one called "image" here (notice the value is empty):
The idea now is to invoke the pipeline from REST APIs and have the image name replaced by the value coming from the variable:
{
"definition": {
"id": 1
},
"sourceBranch": "master",
"parameters": "{\"image\": \"windows-latest\" }"
}
In order to make the step print the value I'm passing here, I need to correct the environment variable in some way. I thought it would be sufficient to write something like:
env:
MY_IMAGE_NAME: ${{ coalesce(variables.image, parameters.image) }}
That's because I want to give the priority to the variables, then to parameters, so that in case none is specified, I always have a default value the pipeline can use.
However, this approach doesn't work, probably because we're dealing with different expansion times for variables, but I don't really know what I should be writing instead (if there is a viable option, of course).
What I also tried is:
env:
MY_IMAGE_NAME: ${{ coalesce($(image), parameters.image) }}
MY_IMAGE_NAME: ${{ coalesce('$(image)', parameters.image) }}
MY_IMAGE_NAME: $[ coalesce(variables.image, parameters.image) ]
MY_IMAGE_NAME: $[ coalesce($(image), parameters.image) ]
None of those are working, so I suspect this may not be feasible at all.
There is a workaround that I'm currently thinking of, which is to create two different pipelines so that those can be invoked independently, but while this is quite easy for me to accomplish, given I'm using a lot of templates, I don't find it the right way to proceed, so I'm open to any suggestion.
I tested and found you might need to define a variable and assign the parameter's value to it (eg. Mimage: ${{parameters.image}}). And define another variable(eg. Vimage) and assign $[coalesce(variables.image, variables.Vimage)] to it. Then refer to $(Vimage) in the env field of powershell task. Please check out below yaml.
parameters:
- name: image
displayName: Pool Image
type: string
default: ubuntu-latest
values:
- windows-latest
- ubuntu-latest
trigger: none
stages:
- stage: A
jobs:
- job: A
variables:
Mimage: ${{parameters.image}}
Vimage: $[coalesce(variables.image, variables.Mimage)]
steps:
- pwsh: |
echo "This should be triggering against image: $env:MY_IMAGE_NAME"
env:
MY_IMAGE_NAME: $(Vimage)
Env field of powershell task is usually for mapping secret variables. You can directly refer to $(Vimage) in the powershell script: echo "This should be triggering against image: $(Vimage).
Note: To queue a build via REST API with provided parameters, you need to check Let users override this value when running this pipeline to make the varilabe to be settable at queue time.
Update:
You can try passing the variables to the parameters of the template to make the parameters for template dynamic. Please check below simple yaml.
jobs:
- template: template.yaml
parameters:
MTimage: ${{parameters.image}}
VTimage: $(Vimage)
template.yaml:
parameters:
MTimage:
VTimage:
jobs:
- job: buildjob
steps:
- powershell: |
echo "${{parameters.VTimage}}"
echo "${{parameters.MTimage}}"
I'm trying to assign one of 2 values to a variable in addition to variable group and can't find the reference that how to use IF ELSE.
Basically I need to convert this jerkins logic to azure DevOps.
Jenkins
if (branch = 'master') {
env = 'a'
} else if (branch = 'dev'){
env ='b'
}
I found 1 reference from the following one, but this one seems to work if the variables section doesn't have variable groups.
https://stackoverflow.com/a/57532526/5862540
But in my pipeline, I already have a variable group for secrets, so I have to use name/value convention and the example doesn't work with the errors like expected a mapping or A mapping was not expected or Unexpected value 'env'
variables:
- group: my-global
- name: env
value:
${{ if eq(variables['Build.SourceBranchName'], 'master') }}:
env: a
${{ if eq(variables['Build.SourceBranchName'], 'dev') }}:
env: b
or
variables:
- group: my-global
- name: env
value:
${{ if eq(variables['Build.SourceBranchName'], 'master') }}: a
${{ if eq(variables['Build.SourceBranchName'], 'dev') }}: b
This code works.
I'm doing similar with parameters.
variables:
- name: var1
${{ if eq(parameters.var1, 'custom') }}:
value: $(var1.manual.custom)
${{ if ne(parameters.var1, 'custom') }}:
value: ${{ parameters.var1 }}
Update 09/09/2021
We have now natively if else expression and we can write it like
variables:
- group: PROD
- name: env
${{ if eq(variables['Build.SourceBranchName'], 'master') }}:
value: a
${{ else }}:
value: b
steps:
- script: |
echo '$(name)'
echo '$(env)'
Original reply
Syntax with template expressions ${{ if ...... }} is not limited only to job/stage level. Both below pipeline does the same and produce the same output:
stages:
- stage: One
displayName: Build and restore
variables:
- group: PROD
- name: env
${{ if eq(variables['Build.SourceBranchName'], 'master') }}:
value: a
${{ if eq(variables['Build.SourceBranchName'], 'dev') }}:
value: b
jobs:
- job: A
steps:
- script: |
echo '$(name)'
echo '$(env)'
variables:
- group: PROD
- name: env
${{ if eq(variables['Build.SourceBranchName'], 'master') }}:
value: a
${{ if eq(variables['Build.SourceBranchName'], 'dev') }}:
value: b
steps:
- script: |
echo '$(name)'
echo '$(env)'
Microsoft a few weeks ago released a new feature for YAML pipeliens that just lets you do that: IF ELSE notation.
https://learn.microsoft.com/en-us/azure/devops/release-notes/2021/sprint-192-update#new-yaml-conditional-expressions
Writing conditional expressions in YAML files just got easier with the use of ${{ else }} and ${{ elseif }} expressions. Below are examples of how to use these expressions in YAML pipelines files.
steps:
- script: tool
env:
${{ if parameters.debug }}:
TOOL_DEBUG: true
TOOL_DEBUG_DIR: _dbg
${{ else }}:
TOOL_DEBUG: false
TOOL_DEBUG_DIR: _dbg
variables:
${{ if eq(parameters.os, 'win') }}:
testsFolder: windows
${{ elseif eq(parameters.os, 'linux' }}:
testsFolder: linux
${{ else }}:
testsFolder: mac
I wanted to have runtime condition evaluation, something similar to compile time:
variables:
VERBOSE_FLAG:
${{if variables['System.Debug']}}:
value: '--verbose'
${{else}}:
value: ''
but unfortunately Azure devops does not supports special kind of functions like if(condition, then case, else case) - so I've played around and find out that it's possible do double string replacement using replace function. It does looks bit hacky of course.
For example, one may want to tweak task inputs depending on whether system debugging is enabled or not. This cannot be done using "standard conditional insertion" (${{ if … }}:), because System.Debug isn't in scope in template expressions. So, runtime expressions to the rescue:
- job:
variables:
VERBOSE_FLAG: $[
replace(
replace(
eq(lower(variables['System.Debug']), 'true'),
True,
'--verbose'
),
False,
''
)
]
steps:
- task: cURLUploader#2
inputs:
# …
options: --fail --more-curl-flags $(VERBOSE_FLAG)
Note that using eq to check the value of System.Debug before calling replace is not redundant: Since eq always returns either True or False, we can then safely use replace to map those values to '--verbose' and '', respectively.
In general, I highly recommend sticking to boolean expressions (for example the application of a boolean-valued function like eq, gt or in) as the first argument of the inner replace application. Had we not done so and instead just written for example
replace(
replace(
lower(variables['System.Debug']),
'true',
'--verbose'
),
'false',
''
)
then, if System.Debug were set to e.g. footruebar, the value of VERBOSE_FLAG would have become foo--verbosebar.
I think for now you're going to need to use a task to customize with name/value syntax variables and conditional variable values. It looks like the object structure for name/value syntax breaks the parsing of expressions, as you have pointed out.
For me, the following is a reasonably clean implementation, and if you want to abstract it away from the pipeline, it seems that a simple template for your many pipelines to use should satisfy the desire for a central "global" location.
variables:
- group: FakeVarGroup
- name: env
value: dev
steps:
- powershell: |
if ($env:Build_SourceBranchName -eq 'master') {
Write-Host ##vso[task.setvariable variable=env;isOutput=true]a
return
} else {
Write-Host ##vso[task.setvariable variable=env;isOutput=true]b
}
displayName: "Set Env Value"
As far as I know, the best way to have conditional branch build is using "trigger" in your YAML, instead of implementing complex "if-else". It is also much safer, and you have more explicit controls on the branch triggers instead of relying on CI variables.
Example:
# specific branch build
jobs:
- job: buildmaster
pool:
vmImage: 'vs2017-win2016'
trigger:
- master
steps:
- script: |
echo "trigger for master branch"
- job: buildfeature
pool:
vmImage: 'vs2017-win2016'
trigger:
- feature
steps:
- script: |
echo "trigger for feature branch"
To have trigger with branches inclusion and exclusion, you could use more complex syntax of trigger with branches include and exclude.
Example:
# specific branch build
trigger:
branches:
include:
- master
- releases/*
exclude:
- releases/1.*
The official documentation of Azure DevOps Pipelines trigger in YAML is:
Azure Pipelines YAML trigger documentation
UPDATE 1:
I repost my comment here with additional notes:
I was thinking to have different pipelines because having the complexity of juggling between CI variables is not more maintainable than having multi jobs in one YAML with triggers. Having multijobs with triggers is also enforcing us to have clear distinction and provision on branch management. Triggers and conditional branches inclusions have been used for a year by my team because of these maintainability advantages.
Feel free to disagree, but to me having an embedded logic in any scripted in any steps to check which branch is currently in session and then does any further actions, are more like ad-hoc solutions. And this has given my team and me maintenance problems before.
Especially if the embedded logic tends to grow by checking other branches, the complexity is more complex later than having clear separations between branches. Also if the YAML file is going to be maintained for long time, it should have clear provisions and roadmaps across different branches. Redundancy is unavoidable, but the intention to separate specific logic will pay more in the long run for maintainability.
This is why I also emphasize branches inclusions and exclusions in my answer :)
Azure YAML if-else solution (when you have a group defined which required name/value notation use thereafter.
variables:
- group: my-global
- name: env
value: a # set by default
- name: env
${{ if eq(variables['Build.SourceBranchName'], 'master') }}:
value: b # will override default
Of if you don't have a group defined:
variables:
env: a # set by default
${{ if eq(variables['Build.SourceBranchName'], 'master') }}:
env: b # will override default