Azure DevOps: How do I expand variables in a template in the YML file referring to that template for use in a conditional? - azure-devops

I have two Azure DevOps .yml-based templates, one referring to another. I'm finding that variables specified in the one being referred to do not expand when I need them to in a conditional in the template referring to that template. Here is how they look:
Parent template:
jobs:
- deployment:
displayName: Release
variables:
- template: variables.yml
environment:
name: QA
resourceType: VirtualMachine
strategy:
runOnce:
deploy:
...
#this copies to xxMaster
- task: CopyFiles#2
#condition: eq('${{ parameters.appName }}', 'all')
condition: and(eq(variables['Build.SourceBranchName'], 'master'), eq('${{ configuration }}', '$(productionConfiguration)'))
inputs:
SourceFolder: 'C:\DevOps\$(Build.BuildNumber)\Content\D_C\a\1\s\xxTexas\obj\Release\Package\PackageTmp'
Contents: '**\*.*'
OverWrite: true
TargetFolder: 'C:\Production\Web Sites\xxMaster'
Child Template:
variables:
- name: solution
value: '**/*.sln'
- name: buildPlatform
value: 'Any CPU'
- name: buildConfiguration
value: "${{ parameters.configuration }}"
- name: productionConfiguration
value: 'Horsie'
In this case, when the conditional should work and ${{ configuration }} equals 'Horsie', the YAML is evaluated and comes up with
and(eq(variables['Build.SourceBranchName'], 'master'), eq('Horsie', '$(productionConfiguration)'))
That is, $(productionConfiguration), the variable specified in the referred-to template, is never expanded into 'Horsie.' What should I do to make it so the variable from that sub-template is expanded and my conditional works?

From your YAML sample, the cause of this issue is the format of the variable:$(productionConfiguration) in condition.
You need to use the format: variables['productionConfiguration'] to call the pipeline variable.
Here is an example:
stages:
- stage: deploy
jobs:
- deployment: DeployWeb
displayName: deploy Web App
variables:
- template: template.yml
pool:
vmImage: 'Ubuntu-latest'
# creates an environment if it doesn't exist
environment: 'smarthotel-dev'
strategy:
runOnce:
deploy:
steps:
- script: echo Hello world
condition: and(eq(variables['Build.SourceBranchName'], 'main'), eq('${{ configuration }}', variables['productionConfiguration']))

Related

Selecting Azure DevOps service connection based on if else logic for tenant selection

I am trying to select a service connection based on if else logic.
I have following two pipelines azure-caller.yml and template.yaml.
azure-caller.yml
parameters:
- name: directory
displayName: 'select appropriate environment directory'
values:
- Dev
- Stage
stages:
- template: job-templates/template.yml
parameters:
selectdirectory: $(select-directory)
template.yml
parameters:
selectdirectory: default
stages:
- stage: A
variables:
- name: service-connection
${{ if eq(parameters.selectdirectory, 'Dev') }}:
value: Dev-Service-connection
${{ elseif eq(parameters.selectdirectory, 'Stage') }}:
value: Stage-Service-connection
jobs:
- job: Install_terraform
displayName: Intialize Terraform
continueOnError: true
steps:
- checkout: self
- task: TerraformInstaller#0
inputs:
terraformVersion: '1.0.5'
- job: Validate_terraform_Plan
displayName: Validate Terraform
continueOnError: true
dependsOn: Install_terraform
steps:
- checkout: self
- task: TerraformTaskV2#2
inputs:
provider: 'azurerm'
command: 'init'
backendServiceArm: ${{ variables.service-connection }}
backendAzureRmResourceGroupName: 'any-rg'
backendAzureRmStorageAccountName: 'any-storage'
backendAzureRmContainerName: 'statefile'
backendAzureRmKey: abcd.tfstate
- task: TerraformTaskV2#2
displayName: 'plan'
inputs:
provider: 'azurerm'
command: 'plan'
commandOptions: '-out="plan.out"'
environmentServiceNameAzureRM: ${{ variables.service-connection }}
My tasks are failing because it fetches the value of the service connection at compile time, which is empty.
Azure Caller is a wrapper/main pipeline, and the template is the only one that is allowed to use Dev or Stage service connections for deployment purposes to achieve security. Is there a way to accomplish this?
Your input will be helpful. thx
It is not supported to dynamically specify value of pipeline variable conditionally.
In the YMAL pipeline, you can use the if conditional to judge using or not using a pipeline variable. You can not use it to judge specifying a value or another one to a pipeline variable.
However, you can dynamically set the input conditionally on a task. See "Conditionally set a task input".
You also can dynamically specify value of parameter conditionally.
Below is an example to dynamically specify value of parameter conditionally. You can reference it to change your YAML files.
template.yml
parameters:
- name: ServiceConnection
type: string
stages:
- stage: A
displayName: 'Stage A'
jobs:
- job: A1
displayName: 'Job A1'
pool:
vmImage: ubuntu-latest
steps:
- task: Bash#3
displayName: 'Show parameter value'
inputs:
targetType: inline
script: echo "The value of ServiceConnection is ${{ parameters.ServiceConnection }}"
azure-pipelines.yml
parameters:
- name: directory
displayName: 'Select appropriate environment directory:'
type: string
values:
- Dev
- Stage
stages:
- template: template.yml
parameters:
${{ if eq(parameters.directory, 'Dev') }}:
ServiceConnection: Dev-Service-connection
${{ if eq(parameters.directory, 'Stage') }}:
ServiceConnection: Stage-Service-connection
Result
[UPDATE]
Dynamically set the input conditionally on the task.
template.yml
parameters:
- name: selectdirectory
type: string
stages:
- stage: A
displayName: 'Stage A'
jobs:
- job: Validate_terraform_Plan
displayName: 'Validate Terraform'
. . .
steps:
. . .
- task: TerraformTaskV2#2
inputs:
provider: 'azurerm'
command: 'init'
${{ if eq(parameters.selectdirectory, 'Dev') }}:
backendServiceArm: Dev-Service-connection
${{ if eq(parameters.selectdirectory, 'Stage') }}:
backendServiceArm: Stage-Service-connection
backendAzureRmResourceGroupName: 'any-rg'
backendAzureRmStorageAccountName: 'any-storage'
backendAzureRmContainerName: 'statefile'
backendAzureRmKey: abcd.tfstate
. . .
azure-pipelines.yml
parameters:
- name: directory
displayName: 'Select appropriate environment directory:'
type: string
values:
- Dev
- Stage
stages:
- template: template.yml
parameters:
selectdirectory: ${{ parameters.directory }}
Have you checked this answer out: AzureDevops: Can we pass dynamic value as service connection in yaml
Essentially rather then pass in your service connection name via an if statement have the service connection name stored in a variable.yml file that is determined off your selected directory and will load the appropriate service connection name.

Azure Devops YML Template Conditions and Dependencies

I have a requirement for a couple of job templates but I can’t get the conditions and dependencies configured to run how I want. I have the basic yaml below:
parameters:
- name: environment
displayName: Environment
- name: action
values:
- deploy
- undeploy
- redeploy
default: redeploy
stages:
- stage: app
displayName: App
jobs:
- deployment: kickoff
environment:
name: ${{ parameters.environment }}
resourceType: virtualMachine
tags: app
strategy:
runOnce:
deploy:
steps:
- checkout: none
- powershell: Write-Host "Run kickoff tasks"
- template: undeploy.yml
parameters:
environment: ${{ parameters.environment }}
action: ${{ parameters.action }}
- template: deploy.yml
parameters:
environment: ${{ parameters.environment }}
action: ${{ parameters.action }}
The requirements I have are the below:
Undeploy.yml: if Action = ‘Deploy’, does not run. If Action = ‘Undeploy’, only run this template. Action = ‘Redeploy’, run both templates but undeploy.yml must run first.
Deploy.yml: If Action = ‘Deploy’, only run this template. If Action = ‘Undeploy’, does not run. If Action = ‘Redeploy’, run both templates but deploy.yml must run second.
Closest I can get to is setting the templates yaml as below:
undeploy.yml
parameters:
- name: environment
default: environmentToDeployTo
- name: action
values:
- deploy
- undeploy
- redeploy
default: redeploy
jobs:
- deployment: undeploy
dependsOn: kickoff
condition: ne ('${{parameters.action}}', 'deploy')
environment:
name: ${{ parameters.environment }}
resourceType: virtualMachine
tags: app
strategy:
runOnce:
deploy:
steps:
- checkout: none
#undeploy steps here#
deploy.yml
parameters:
- name: environment
default: environmentToDeployTo
- name: action
values:
- deploy
- undeploy
- redeploy
default: redeploy
jobs:
- deployment: dploy
dependsOn: undeploy
condition: ne ('${{parameters.action}}', 'undeploy')
environment:
name: ${{ parameters.environment }}
resourceType: virtualMachine
tags: app
strategy:
runOnce:
deploy:
steps:
- checkout: none
#dploy steps here#
This results only affects deploy.yml but it is not able to run in isolation - the undeploy.yml job template will have to run beforehand. However, without the "dependsOn", I can see no other way to ensure that undeploy.yml runs first.
Is there anyway to achieve this as per requirements outlined above?
Thanks in advance
dependsOn takes the name of a stage or job as input, not the name of a template file. Seeing more of your templates would be useful, but if you could use the name of the stage or job in dependsOn, you might get the result you're looking for.
Think of the expanded template as your guide here - you can refer to a stage or job that's declared in a different template, so long as it expands to the right place when the pipeline runs.

Why can't I use a variable to define the environment property in the Azure Pipeline YAML config file?

I'm trying to create a deploy pipeline YAML template for all environments/stages. I've set up the Environments on Azure DevOps so that I can add checks and approvals on the Test and Prod environments before they get deployed. I've set up a library group for each stage and each one of them has a variable called 'env' which defines the current stage running in the pipeline. For some reason, the environment property under the deployment job (see code snippet below) doesn't read that variable.
Has anyone faced this issue before, or is there a reason why the variable won't be read for that specific property?
Note: I've tested the variables and they do work, for example, the stage property outputs as 'deploy-dev/test/prod' (depending on the environment)
- stage: deploy-$(env)
jobs:
- deployment: DeployWeb
displayName: deploy Web App
pool:
vmImage: 'Ubuntu-latest'
# creates an environment if it doesn't exist
environment: 'smarthotel-$(env)'
strategy:
runOnce:
deploy:
steps:
- script: echo Hello world
You can't do this because it have to be know at compilation phase.
But you can try this (lets name this file deploy.yml):
parameters:
- name: env
type: string
default: 'dev'
stages:
- stage: deploy-${{ parameters.env }}
jobs:
- deployment: DeployWeb
displayName: deploy Web App
pool:
vmImage: 'Ubuntu-latest'
# creates an environment if it doesn't exist
environment: 'smarthotel-${{ parameters.env }}'
strategy:
runOnce:
deploy:
steps:
- script: echo Hello world
and then you need to run as follows (in build.yml file):
stages:
- template: deploy.yml
parameters:
env: dev
- template: deploy.yml
parameters:
env: qa
- template: deploy.yml
parameters:
env: prod

Using pipeline variables across stages with template jobs

Problem Description
I was having some problems trying to use variables created in one stage in another stage and managed to find various articles old and new describing how this can be done. The more recent articles/posts identifying the new syntax
$[stageDependencies.{stageName}.{jobName}.outputs['{stepName}.{variableName}']
Used like this:
variables:
myVariable: $[stagedependencies.CreateStageVarStage.CreateStageVarJob.outputs['SetValueStep.VariableFromFirstStage']]
This works great until you needed to use job templates.
None of the samples I found online covered the situation of templates. They just demonstrated how multiple stages in the same yaml file could obtain the value.
The syntax depends on being able to put the expression into a variable. Unfortunately, when you use a template for a job, it's not possible to declare variables and passing it as a parameter results in it being unevaluated.
- stage: UseJobTemplateStage
displayName: 'Use Job Template Stage'
dependsOn: CreateStageVarStage
jobs:
- template: templates/job-showstagevars.yml
parameters:
ValueToOutput: $[ stagedependencies.CreateStageVarStage.CreateStageVarJob.outputs['SetValueStep.VariableFromFirstStage'] ]
In this snippet, it comes through as-is. The value does not get substituted in.
Theoretically, you could set your job to have the expression present in the variables block but that sort of hard-coding undermines one of the main benefits of templates.
Related Articles
Share variables across stages in Azure DevOps Pipelines
Azure DevOps Release Notes - sprint 168
I know the question asks about template jobs, but for future reference I want to describe how it can be achieved with template stages as well.
It is done with a variable workaround as in the accepted answer, and with a reference to stagedependencies when no dependsOn exists. (Templates don't allow dependsOn). Somehow, this still works.
Example YAML using stage template (I have modified the code from the accepted answer):
stages:
- stage: CreateStageVarStage
displayName: 'Create StageVar Stage'
jobs:
- job: CreateStageVarJob
displayName: 'Create StageVar Job'
timeoutInMinutes: 5
pool:
name: 'Azure Pipelines'
vmImage: 'windows-2019'
steps:
- checkout: none
- pwsh: |
[string]$message = 'This is the value from the first stage'
Write-Host "Setting output variable 'VariableFromFirstStage' to '$message'"
Write-Output "##vso[task.setvariable variable=VariableFromFirstStage;isOutput=$true]$message"
name: SetValueStep
# stage template cannot use dependsOn, but is still allowed to refer to stagedependencies...
- template: templates/stage-showstagevars.yml
parameters:
ValueToOutput: $[ stagedependencies.CreateStageVarStage.CreateStageVarJob.outputs['SetValueStep.VariableFromFirstStage'] ]
Stage template:
parameters:
- name: ValueToOutput
type: string
stages:
- stage: ShowStageVarStage
variables:
- name: LocalVarOfValueToOutputParam
value: ${{ parameters.ValueToOutput }}
jobs:
- job: ShowStageVarJob
displayName: 'Show stage var'
pool:
name: 'Azure Pipelines'
vmImage: 'windows-2019'
steps:
- checkout: none
- pwsh: |
Write-Host "ValueToOutput parameter=${{ parameters.ValueToOutput }}"
Write-Host "LocalVarOfValueToOutputParam (pre-processor syntax)=${{ variables.LocalVarOfValueToOutputParam }}"
Write-Host "LocalVarOfValueToOutputParam (macro syntax)=$(LocalVarOfValueToOutputParam)"
displayName: 'Show StageVariable'
Solution
The answer isn't actually far away. The original expression just need to be passed through a variable in the template job. Basically, set a variable to be the value of the parameter and use the macro syntax to evaluate the variable.
parameters:
- name: ValueToOutput
type: string
...
variables:
- name: LocalVarOfValueToOutputParam
value: ${{ parameters.ValueToOutput }}
Using the macro syntax of $(LocalVarOfValueToOutputParam) will result in the value making its way into the template job correctly.
Example
If we have a yaml file for the build definition:
stages:
- stage: CreateStageVarStage
displayName: 'Create StageVar Stage'
jobs:
- job: CreateStageVarJob
displayName: 'Create StageVar Job'
timeoutInMinutes: 5
pool:
name: 'Azure Pipelines'
vmImage: 'windows-2019'
steps:
- checkout: none
- pwsh: |
[string]$message = 'This is the value from the first stage'
Write-Host "Setting output variable 'VariableFromFirstStage' to '$message'"
Write-Output "##vso[task.setvariable variable=VariableFromFirstStage;isOutput=$true]$message"
name: SetValueStep
- stage: UseJobTemplateStage
displayName: 'Use Job Template Stage'
dependsOn: CreateStageVarStage
jobs:
- template: templates/job-showstagevars.yml
parameters:
ValueToOutput: $[ stagedependencies.CreateStageVarStage.CreateStageVarJob.outputs['SetValueStep.VariableFromFirstStage'] ]
That uses the job template templates/job-showstagevars.yml
parameters:
- name: ValueToOutput
type: string
jobs:
- job: ShowStageVarJob
displayName: 'Show stage var'
timeoutInMinutes: 5
pool:
name: 'Azure Pipelines'
vmImage: 'windows-2019'
variables:
- name: LocalVarOfValueToOutputParam
value: ${{ parameters.ValueToOutput }}
steps:
- checkout: none
- pwsh: |
Write-Host "ValueToOutput parameter=${{ parameters.ValueToOutput }}"
Write-Host "LocalVarOfValueToOutputParam (pre-processor syntax)=${{ variables.LocalVarOfValueToOutputParam }}"
Write-Host "LocalVarOfValueToOutputParam (macro syntax)=$(LocalVarOfValueToOutputParam)"
displayName: 'Show StageVariable'
What we get in our output of the second stage is:
ValueToOutput parameter=$[ stagedependencies.CreateStageVarStage.CreateStageVarJob.outputs['SetValueStep.VariableFromFirstStage'] ]
LocalVarOfValueToOutputParam (pre-processor syntax)=$[ stagedependencies.CreateStageVarStage.CreateStageVarJob.outputs['SetValueStep.VariableFromFirstStage'] ]
LocalVarOfValueToOutputParam (macro syntax)=This is the value from the first stage

Azure DevOps YAML If condition

I am attempting to use the if condition to execute a block of operations but it doesn't seem to be working for me. I'm not entirely sure if I understand the if condition correctly, but this is how I have been using it - or at least a simplified version of it.
# Starter pipeline
# Start with a minimal pipeline that you can customize to build and deploy your code.
# Add steps that build, run tests, deploy, and more:
# https://aka.ms/yaml
name: MyTest-$(date:yyyyMMdd)$(rev:.r)
trigger: none
resources:
- repo: self
clean: false
fetchDepth: 2
variables:
- group: TestVG
- name: EnableSigning
value: true
- name: SignDebugBuilds
value: false
stages:
- stage: Stage_Build
displayName: Build Stage
dependsOn: []
jobs:
- job: Job_Build
displayName: Build Job
condition: succeeded()
dependsOn: []
pool:
vmImage: 'windows-latest'
variables:
BuildPlatform: AnyCPU
strategy:
matrix:
Debug:
BuildConfiguration: Debug
Retail:
BuildConfiguration: Retail
steps:
- template: param-test.yml
parameters:
Value: 'Build Solution 1'
#- script: echo 'Build Solution 1'
- ${{ if and(eq(variables['EnableSigning'], True), or(ne(variables['BuildConfiguration'], 'Debug'), eq(variables['SignDebugBuilds'], True))) }}:
#- script: echo 'Build Solution 2 IF check'
- template: param-test.yml
parameters:
Value: 'Build Solution 2 IF check'
- script: echo 'Build Solution 2 COND check'
condition: and(eq(variables['EnableSigning'], True), or(ne(variables['BuildConfiguration'], 'Debug'), eq(variables['SignDebugBuilds'], True)))
The param-test.yml file is a simple file to echo a line for logging/debugging purposes.
parameters:
Value: string
steps:
- script: echo '${{ parameters.Value }}'
My expectation was that for the debug build I would see the first log and the other 2 logs would get skipped, while all 3 logs would be printed for retail build.
However, what I see is that for the debug build, the first and second logs are printed, while only the 3rd log is skipped (the one with the explicit condition in the script step), while all 3 logs are printed for retail build.
To clarify a few items, I used the 3rd log as a control to check if the condition was correct or not. And this is a highly simplified version of the actual pipeline that I have in our repo. We have a primary pipeline YAML file where all the variables are declared including the EnableSigning and SignDebugBuilds. This calls a template file passing in the EnableSigning and SignDebugBuilds flags to the same. This happens a few times around with subtemplates, till one of the sub-templates, the one responsible for the build, uses these from the parameters in the if check. In the actual pipeline, I have the condition use parameters instead of variables, but the result is still the same.
I have looked at a few documentation but it wasn't very clear as to what we could expect from the if statement. And since template tags don't support an explicit condition that we can pass in, this seems to be the only option aside from maintaining 2 separate versions of the template files corresponding to the flags
In your current situation, the 'if' keyword cannot get the variables['BuildConfiguration'], because this variable is created when the job is running. And 'if' key word needs to use the runtime parameters.
So, every time the job is running, the result of the variables['BuildConfiguration'] under the 'if' keyword is NULL. Because when the job is init, this variable has not been created. This variable will be created when the job is running and then the pipeline will help you create other two jobs under your job.
At present the work around is to split the matrix to different two jobs and use the parameters to instead of the variables.
Here is the demo I create:
name: MyTest-$(date:yyyyMMdd)$(rev:.r)
trigger: none
resources:
- repo: self
clean: false
fetchDepth: 2
parameters:
- name: EnableSigning
type: boolean
displayName: 'EnableSigning'
default: true
- name: SignDebugBuilds
type: boolean
displayName: 'SignDebugBuilds'
default: false
variables:
# - group: TestVG
# - name: EnableSigning
# value: true
# - name: SignDebugBuilds
# value: false
- name: system.debug
value: true
stages:
- stage: Stage_Build
displayName: Build Stage
dependsOn: []
jobs:
- job: Build_Job_Debug
pool:
# vmImage: 'windows-latest'
name: default
variables:
BuildPlatform: AnyCPU
BuildConfiguration: Debug
steps:
- template: param-test.yml
parameters:
Value: 'Build Solution 1'
- ${{ if and(eq(parameters.EnableSigning, true), eq(parameters.SignDebugBuilds, true))}}:
- template: param-test.yml
parameters:
Value: 'Build Solution 2 IF check'
- script: echo 'Build Solution 2 COND check and BuildConfiguration is $(BuildConfiguration)'
name: 'CMD3_check'
condition: eq('${{ parameters.EnableSigning }}', true)
- job: Build_Job_Retail
pool:
# vmImage: 'windows-latest'
name: default
variables:
BuildPlatform: AnyCPU
BuildConfiguration: Retail
steps:
- template: param-test.yml
parameters:
Value: 'Build Solution 1'
- ${{ if or(eq(parameters.EnableSigning, true), eq(parameters.SignDebugBuilds, false))}}:
- template: param-test.yml
parameters:
Value: 'Build Solution 2 IF check'
- script: echo 'Build Solution 2 COND check and BuildConfiguration is $(BuildConfiguration)'
name: 'CMD3_check'
condition: eq('${{ parameters.EnableSigning }}', true)
Also, if you need to use the variable BuildConfiguration, you can also define it under the different job.