Azure Yaml DevOps variable handling - azure-devops

So I have a YAML file where I am trying to do something like the following:
# top of the file, declare global variables
variables:
- name: MyName
value: 'apple'
...
parameters:
- name: SwitchName
type: boolean
default: false
...
stages:
- stage: Build
displayName: 'Build'
...
jobs:
variables:
- ${{ if eq(parameters.SwitchName, true) }}:
- name: MyName
value: '$(MyName)_pie'
...
steps:
- task: PowerShell#1
inputs:
scriptType: inlineScript
inlineScript: |
Write-Output $(MyName)
...
#end of script
The goal is to control the pipeline to print apple in default run and apple_pie when the user selects that parameter in the UI to be true.
I understand that parameters are compile-time and variables are run-time. I know that to overwrite a global variable you can create a variable at a job level and change it as yo see fit. Sadly, the way how template works do not let me redeclare that variable properly and I end up with $(MyName)_pie. For some reason, Yaml fails to see that inside that template there is a runtime variable that is needed to be defined.
What are my options here to achieve desired behavior?
Anything missing in my understanding here?

You can change to use the ${{ variables.MyName }} to call the variable in root level.
For example:
variables:
- name: MyName
value: 'apple'
parameters:
- name: SwitchName
type: boolean
default: false
stages:
- stage: Build
displayName: 'Build'
jobs:
- job: A
variables:
- ${{ if eq(parameters.SwitchName, true) }}:
- name: MyName
value: '${{ variables.MyName }}_pie'
steps:
- script: "echo $(MyName)"
Result:
Since you used an if expression to reassign value, the variable will be assigned at compile time.
Refer to this doc: Runtime expression syntax
You can use the template expression format: ${{ variables.var }}

Related

Variable from expression in YAML Pipeline with dynamic variable group

I am trying to assign value of the variable by coalescing (via expression) from parameter and variable that is in variable group.
Problem is, that the variable group is dynamic, depending on another parameter (like DEV, QA,...)
Pipeline can't see the value of the variable ResourceGroupName which comes from variable group MyAPIVarGroup-*. It comes empty.
Is this even achievable and I'm just doing something wrong? Or is this not possible with the current implementation of expressions and variable groups in YAML pipelines?
# Parameter of the target environment to deploy to
- name: targetEnvironmentParam
displayName: Target environment of the deployment
type: string
default: 'DEV'
values:
- DEV
- QA
# Parameter of the custom resource group suffix (suffix not used later if not filled)
- name: resGroupOverrideParam
displayName: "[OPTIONAL] Custom resource group name"
type: string
default: ' '
variables:
# Variable group to be used (based on environment)
- ${{ if eq(parameters.targetEnvironmentParam, 'DEV') }}:
- group: MyAPIVarGroup-DEV
- ${{ if eq(parameters.targetEnvironmentParam, 'QA') }}:
- group: MyAPIVarGroup-QA
- name: resGroupName
value: ${{ coalesce(parameters.resGroupOverrideParam, variables.ResourceGroupName, 'default is wrong') }}
So, what you are experiencing is the fact that ${{ }} expressions are evaluated before the pipeline is run so the variable from the variable group is empty at compile time (not known). See this article for more details on expressions and evaluation:
https://learn.microsoft.com/en-us/azure/devops/pipelines/process/expressions?view=azure-devops
There are also runtime expressions that you could use instead but they do not allow access to pipeline parameters so you would need to assign the override value to a variable for use in evaluating the coalesce() statement like this (note the replace function to trim the default value of the override parameter to an empty string):
parameters:
- name: targetEnvironmentParam
displayName: Target environment of the deployment
type: string
default: 'DEV'
values:
- DEV
- QA
- name: resGroupOverrideParam
displayName: "[OPTIONAL] Custom resource group name"
type: string
default: ' '
variables:
- ${{ if eq(parameters.targetEnvironmentParam, 'DEV') }}:
- group: MyAPIVarGroup-DEV
- ${{ if eq(parameters.targetEnvironmentParam, 'QA') }}:
- group: MyAPIVarGroup-QA
- name: resGroupOverride
value: ${{ replace(parameters.resGroupOverrideParam, ' ', '') }}
- name: resGroupName
value: $[ coalesce(variables['resGroupOverride'], variables.ResourceGroupName, 'default is wrong') ]
steps:
- pwsh: |
Write-Host $(resGroupName)
You could also stick with just compile time expressions with the following example instead which relies on overriding the same variable name from a group if the optional parameter is set with something other than a space:
parameters:
- name: targetEnvironmentParam
displayName: Target environment of the deployment
type: string
default: 'DEV'
values:
- DEV
- QA
- name: resGroupOverrideParam
displayName: "[OPTIONAL] Custom resource group name"
type: string
default: ' '
variables:
- group: MyAPIVarGroup-${{ parameters.targetEnvironmentParam }}
- ${{ if ne(parameters['resGroupOverrideParam'], ' ') }}:
- name: resGroupName
value: ${{ parameters.resGroupOverrideParam }}
steps:
- pwsh: |
Write-Host $(resGroupName)
This example pipeline always loads a variable group and both groups have a variable named the same thing. Also, if the override parameter is set then the pipeline itself will create a variable and set the value to the parameter which because it is declared after the variable group will override that value.

It doesn't work pass variable from one stage to another and iterate with an each loop

I need to do an each from a variable obtained with stageDependencies previously.
It doesn't work Pass variable from one stage to another and iterate with an each loop using a split function
What am I doing wrong?
Code:
task.yml
steps:
- task: Bash#3
name: env_string
inputs:
targetType: 'inline'
script: |
environmentsStr='dev,prd'
echo "##vso[task.setvariable variable=environmentsString;isOutput=true]$environmentsStr"
pipeline
- stage: stage1
jobs:
- job: job_stage1
steps:
- template: tasks.yml
- stage: stage2
dependsOn:
- stage1
variables:
- name: envFromVar
value: 'dev,prd'
- name: envFromStageDependencies
value: $[ stageDependencies.stage1.job_stage1.outputs['env_string.environmentsString'] ]
jobs:
- job: job_stage2
steps:
- template: stage3.yml
parameters:
envFromStageDependencies: $(envFromStageDependencies)
environmentsFromVar: ${{ variables.envFromVar }}
stage3.yml
parameters:
- name: envFromStageDependencies
type: string
- name: environmentsFromVar
type: string
steps:
- ${{ each environment in split(parameters.envFromStageDependencies, ',') }}:
- bash: |
echo "env ${{ environment }}" # OUTPUT FAIL; (env dev,prd)
- ${{ each environment in split(parameters.environmentsFromVar, ',') }}:
- bash: |
echo "envVar ${{ environment }}" # OUTPUT OK, 2 iterations envVar [dev,prd]
OUTPUT
BASH -> env dev,prd
BASH -> envVar dev
BASH -> envVar prd
You should change your pipeline YAML like as below:
. . .
jobs:
- job: job_stage2
steps:
- template: stage3.yml
parameters:
environments: ${{ variables.envFromStageDependencies }}
environmentsFromVar: ${{ variables.envFromVar }}
As #Daniel Mann has mentioned, the template is processed at compile time which is before runtime. So, to pass the variable values into the template, you should use the compile time variable syntax (${{ variables.varName }}) instead of runtime variable syntax ($(varName)).
For more details, you can reference the document "Understand variable syntax".

Evaluate an expression at template expansion time

I have an azure pipeline template that takes parameters, and I'd like to set one of the parameters based on an expression:
- template: templates/mytemplate.yml
parameters:
TestBackCompat: eq('${{ parameters.CIBuildId }}', 'MasterLatestBuildId')
CreateNupkg: ${{ parameters.CreateNupkg }}
As written, it doesn't work, because the expression isn't evaluated until runtime.
Is there a way to evaluate the expression at compile-time? Simple variable replacement (e.g., the usage of CreateNuPkg in the script above) works OK.
From this official document:
https://learn.microsoft.com/en-us/azure/devops/pipelines/process/templates?view=azure-devops#template-expressions
Template expressions can expand template parameters, and also
variables. You can use parameters to influence how a template is
expanded. The object works like the variables object in an expression.
How the variable object works:
https://learn.microsoft.com/en-us/azure/devops/pipelines/process/variables?view=azure-devops&tabs=yaml%2Cbatch#runtime-expression-syntax
So the usage in your situation should be like this:
give_parameters.yml
trigger:
- none
parameters:
- name: CIBuildId
type: string
default: 'MasterLatestBuildId'
extends:
template: using_parameters.yml
parameters:
TestBackCompat: ${{ eq(parameters.CIBuildId, 'MasterLatestBuildId')}}
using_parameters.yml
parameters:
- name: CIBuildId
type: string
default: 'x'
- name: TestBackCompat
type: string
default: 'x'
variables:
- name: test1
value: ${{ parameters.TestBackCompat}}
- name: test2
value: ${{ parameters.CIBuildId}}
steps:
- task: PowerShell#2
inputs:
targetType: 'inline'
script: |
Write-Host $(test1)
Write-Host $(test2)
Write-Host ${{ parameters.TestBackCompat}}
Result:

AzureDevops set variables based on parameter and pass to template at next stage

In Azure DevOps, I'm trying to create a pipeline which offers a simple selection of pre-set options to the user running it. These options will be converted into different combinations of parameters as specified by a templated stage (the definition of which, I have no control over). The idea of my pipeline is that frequently-used build configurations are easy to select correctly, rather than having to manually set 3 or 4 different parameters.
I need the "Build.Setup" from immutable_pipeline to print config_one, profile_one when the first radio is selected (buildType=type1), config_two, profile_two when buildType=type2, and so on.
Unfortunately I'm really struggling to get any variable value into the templated stage other than the defaults. Are ADO variables even mutable variables at all - or just constants?
I've read the MS docs extensively and understand the meaings of the different macro declaration types. I've tried many different combinations of syntaxes ${{...}}, $(...) and $[...], all behave differently but none seems able to deliver what's needed. Is this even possible? Is there a simple solution someone can suggest?
Pipeline:
name: $(Date:yyyyMMdd).$(Rev:r)
parameters:
- name: buildType
displayName: 'Type of build'
type: string
default: 'type3'
values: ['type1', 'type2', 'type3']
pool:
name: default
variables:
- name: config
value: 'defaultConfig'
- name: profile
value: 'defaultProfile'
stages:
- stage: Stage1
displayName: Prepare build config
jobs:
- job: Job1_1
steps:
- checkout: none
- task: Bash#3
name: SetVariables
inputs:
targetType: inline
script: |
p1='${{ parameters.buildType }}'
v1='$(config)'
v2='$(profile)'
echo -e "BEFORE: p1='${p1}'\n v1='${v1}'\n v2='${v2}'"
case ${p1} in
type1)
v1='config_one'
v2='profile_one'
;;
type2)
v1='config_two'
v2='profile_two'
;;
type3)
v1='config_three'
v2='profile_three'
;;
esac
echo -e "AFTER: p1='${p1}'\n v1='${v1}'\n v2='${v2}'"
echo "##vso[task.setvariable variable=config]${v1}"
echo "##vso[task.setvariable variable=profile;isOutput=True]${v2}"
- job: Job1_2
dependsOn: Job1_1
variables:
- name: variable1
value: $(config)
- name: variable2
value: $[ dependencies.Job1_1.outputs['SetVariables.profile']]
steps:
- task: Bash#3
name: GetVariables2
inputs:
targetType: inline
script: |
echo -e 'SAME STAGE: v1="$(variable1)"\n v2="$(variable2)"'
# Next stage - use computed values for "config" and "profile"
- template: templates/immutable_pipeline.yml
parameters:
config: $(config)
profile: ${{ variables.profile }}
templates/immutable_pipeline.yml:
Note that I don't have access to change this, I can't make it dependsOn: Stage1.Job1_1.
parameters:
- name: config
displayName: 'Config'
type: string
default: 'unset'
- name: profile
displayName: 'Profile'
type: string
default: 'unset'
stages:
- stage: Build
displayName: Templated build
jobs:
- job: Setup
pool:
name: default
demands:
- Agent.OS -equals Linux
steps:
- checkout: none
- script: |
echo '##[info] parameters.config=${{ parameters.config }}'
echo '##[info] parameters.profile=${{ parameters.profile }}'
I just found one solution (which is arguably simpler than using variables) using the ${{ if eq(...) }}: syntax:
name: $(Date:yyyyMMdd).$(Rev:r)
parameters:
- name: buildType
displayName: 'Type of build'
type: string
default: 'type3'
values: ['type1', 'type2', 'type3']
pool:
name: default
stages:
- template: templates/immutable_pipeline.yml
${{ if eq(parameters.buildType, 'type1') }}:
parameters:
config: config_one
profile: profile_one
${{ if eq(parameters.buildType, 'type2') }}:
parameters:
config: config_two
profile: profile_two
${{ if eq(parameters.buildType, 'type3') }}:
parameters:
config: config_three
profile: profile_three
Still interested in whether the original approach of setting variables is even possible, if only beause I've spent so much time on it.

Checking for null object type parameter in Azure YAML

I'm setting up a build template and can't figure out the syntax for an optional object type parameter. In my pipeline I'm calling the template like this:
stages:
- template: aspnet-core.yml#templates
parameters:
database:
name: 'SomeDatabase'
server: 'SomeServer'
I have the parameter defined like this in the template:
parameters:
database: null
I want to do a check like this in the template so I can run a task conditionally:
- ${{ if ne('${{ parameters.database }}', null) }}:
However, it's not liking the keyword null in the if statement, and I don't know how to represent the fact that it wasn't passed in. What are my options here?
You can use below expression to check if a parameter is empty. For below example
- ${{if parameters.database}}:
Below is my testing template and azure-pipeline.yml.
the script task will only get executed when database is evaluated to true. I tested and found database: "" and database: will be evalutated to false. If it is defined as database: {}, it will be evaluated to true.
Template: deploy-jobs.yaml
parameters:
database: {}
stages:
- stage: buildstage
pool: Hosted VS2017
jobs:
- job: secure_buildjob
steps:
- ${{if parameters.database}}:
- script: echo "will run if database is not empty"
displayName: 'Base: Pre-build'
azure-pipeline.yml:
stages:
- template: deploy-jobs.yaml
parameters:
database: ""
To execute some tasks if database is empty you can use below statement:
steps:
- ${{if not(parameters.database)}}:
- script: echo "will run if database is empty"
displayName: 'Base: Pre-build'
It looks like an alternative to the if syntax is to use conditions instead. This allows you to skip over the step. I had to check a property of the object though to see if it was actually passed, so not super ideal.
condition: and(succeeded(), ne('${{ parameters.database.name }}', ''))
I've found another solution to that, you can work with the length of the incoming object.
If the object is empty, it's length is 0
parameters:
- name: myObject
type: object
default: []
steps:
- ${{ if not(eq(length(parameters.myObject), 0)) }}:
- script: |
echo "hello world"
displayName: "next task"
Try to define the default as:
parameters:
database:
name: ''
server: ''
then check:
- ${{ if ne(parameters.database.name, '') }}:
<run-your-task>