Dynamic variable group in Azure DevOps pipeline - azure-devops

Question

I want to be able to use a variable group in Azure DevOps Pipeline based upon a generated name.
Like so: (where dtapName is a letter like d or t.)
variables:
- group: 'project-${{ dtapName }}'
I've seen it work in other pipelines but creating one myself has been a pain.
My current pipeline is as follows:
The variable group "global-d" contains a variable 'dtapName' with value 'd'.
And the variable group "project-t" contains URLs and such specific for our 'develop' landscape.
File "pipeline-infra.yaml":
trigger:
branches:
include:
- main
- develop
paths:
include:
- Project/pipelines
stages:
- stage: Develop
displayName: Develop deployment of Project
variables:
- name: environmentName
value: 'develop-o'
- group: 'global-o'
jobs:
- deployment: 'Debug_001'
environment: ${{ variables.environmentName }}
strategy:
runOnce:
deploy:
steps:
- script: echo [Testing job]
- script: echo v.environment = ${{ variables.environmentName }} # Does work
- script: echo v.dtapName = ${{ variables.dtapName }} #❗Fails, why? It is defined in the variable group 'global-o'
# script: echo environment = ${{ environmentName }} #❗Failed, why does it work in the template call below?
# script: echo dtapName = ${{ dtapName }} #❗Failed, why does it work in the template call below?
- script: echo env ENVIRONMENTNAME = $ENVIRONMENTNAME # Does work
- script: echo env DTAPNAME = $DTAPNAME # Does work
- template: pipeline-infra.extra.yaml
parameters:
environment: $(environmentName)
dtapName: $(dtapName)
File "pipeline-infra.extra.yaml"
parameters:
- name: environment
type: string
- name: dtapName
type: string
jobs:
- deployment: 'Debug_002'
# environment: ${{ parameters.environment }} #❗Fails with: Environment $(environmentName) could not be found. WHY?
environment: 'develop-o'
pool:
vmImage: 'ubuntu-latest'
variables:
- name: placeholder
value: 'none'
# - group: 'project-${{ parameters.dtapName }}' #❗Fails with: Variable group 'project-$(dtapName)' could not be found. How to do this?
strategy:
runOnce:
deploy:
steps:
- script: echo [ pipeline-infra.extra.yaml ]
- script: echo Environment = ${{ parameters.environment }} # WORKS
- script: echo "Environment (v) = ${{ variables.environmentName }}" #❗FAILS, this variable is declare on the stage of this job... why can't it be accessed?
- script: echo Parameter = ${{ parameters.dtapName }} # WORKS
- script: echo PARAMETER = $DTAPNAME # WORKS
- script: echo Variable = ${{ variables.dtapName }} #❗FAILS, would think this comes available when the problem in the calling stage is fixed
- task: Bash#3
displayName: VariableListing 2
inputs:
targetType: inline
script: 'env | sort'
Can anyone explain me (or point me to a description) why these problems occur?
I've read (most of) Define variables and documents linked from here.
It's probably a little thing, but for the past couple of days I'm not seeing it.

Answer 1

You have two errors in your main yaml:
You cannot invoke variables from Variable group with template expression ${{ variables.var }}. Template variables are processed at compile time, and are replaced before runtime starts. It cannot get variable value from Group, it can get value from predefined variable.
Template expression should have format as ${{ variables.var }} not ${{ var }}.
Please check doc: Understand variable syntax for the details. I add the comment on your yaml below:
trigger:
branches:
include:
- main
- develop
paths:
include:
- Project/pipelines
stages:
- stage: Develop
displayName: Develop deployment of Project
variables:
- name: environmentName
value: 'develop-o'
- group: 'global-o'
jobs:
- deployment: 'Debug_001'
environment: ${{ variables.environmentName }}
strategy:
runOnce:
deploy:
steps:
- script: echo [Testing job]
- script: echo v.environment = ${{ variables.environmentName }} # Does work, it use template expression syntax and can get the value as it's predefined variable.
- script: echo v.dtapName = ${{ variables.dtapName }} #❗Not work, Template variables are processed at compile time, and are replaced before runtime starts, it cannot get the Test Group variable value, should use macro syntax $(dtapName).
# script: echo environment = ${{ environmentName }} #❗Failed, syntax error, should be ${{ variables.environmentName }} or $(environmentName)
# script: echo dtapName = ${{ dtapName }} #❗Failed, syntax error, should be $(dtapName)
- script: echo env ENVIRONMENTNAME = $ENVIRONMENTNAME # Does work, the variable is set as environment value, you can use it with syntax $NAME.
- script: echo env DTAPNAME = $DTAPNAME # Does work,the variable is set as environment value, you can use it with syntax $NAME.
- template: pipeline-infra.extra.yaml
parameters:
environment: $(environmentName) # Environment creation happens at compile time, if you pass marco syntax to template, it will cause the error, should use template expreesion ${{ variables.environmentName }}.
dtapName: $(dtapName) # not work here, should define it out of variable group, and transfer with parameter expression same as above.
For your template yaml:
Environment creation happens at compile time, so if you define $(environmentName) in main yaml for the template, it cannot get the value, you should use template expression ${{ variables.environmentName }} in main yaml to transfer the value.
No variable defined in template, you cannot use ${{ variables.var }}, you should use parameters expression ${{ parameters.var }}.
parameters:
- name: environment
type: string
- name: dtapName
type: string
jobs:
- deployment: 'Debug_002'
# environment: ${{ parameters.environment }} #❗Fails, should use ${{ variables.environmentName }} in main yaml for template.
environment: 'develop-o'
pool:
vmImage: 'ubuntu-latest'
variables:
- name: placeholder
value: 'none'
# - group: 'project-${{ parameters.dtapName }}' #❗Fails, same as above, it cannot get the value
strategy:
runOnce:
deploy:
steps:
- script: echo [ pipeline-infra.extra.yaml ]
- script: echo Environment = ${{ parameters.environment }} # WORKS
- script: echo "Environment (v) = ${{ variables.environmentName }}" #❗FAILS, should use parameter expression as no variable defined.
- script: echo Parameter = ${{ parameters.dtapName }} # WORKS
- script: echo PARAMETER = $DTAPNAME # WORKS
- script: echo Variable = ${{ variables.dtapName }} #❗FAILS, same as above.
- task: Bash#3
displayName: VariableListing 2
inputs:
targetType: inline
script: 'env | sort'