Azure Devops Return Variable from Template - azure-devops

Is there a way to return a variable from a template so I can use that as a parameter for other templates?
I have a yml file that looks something like this (actual names redacted).
- stage: A
jobs:
- template: a.yml
parameters:
environment: foo
group: bar
- template: b.yml
parameters:
dependsOn: a
environment: foo
group: bar
- template: c.yml
parameters:
dependsOn: a
environment: foo
group: bar
Each template is a deployment.
I want a.yml to set a variable, then for that to be returned so that I could use that variable as a parameter for the other 2 templates. How can I return a variable using 'a.yml'?

You can use depands to pass parameters between two templates just as passing parameters between stages. Here is an example. I use environment as the parameters to be passed between a.yml and b.yml.
a.yml:
parameters:
environment: foo
group: bar
jobs:
- job: deployA
variables:
env: ${{ parameters.environment }}
steps:
- pwsh: |
echo "##vso[task.setvariable variable=env;isOutput=true]$($env:ENV)"
name: outputVar
b.yml:
parameters:
dependsOn: a
environment: foo
group: bar
jobs:
- job: deployB
dependsOn: deployA
steps:
...
pipeline.yml:
trigger: none
pool:
vmImage: ubuntu-latest
stages:
- stage: A
jobs:
- template: a.yml
parameters:
environment: Test
- template: templateB.yml
parameters:
environment: $[dependencies.deployA.outputs['outputVar.env']]

Related

Azure yaml same key issue when checking out same repository in multiple jobs

I have reusable build template job. In my pipeline, I'm reusing this template every time I want to build new application. Inside this build template, it has inline checkout step. Problem occurs if I checkout same repository and branch.
Template structure
# **pipeline.yaml**
stages:
- template: templates/group-build.yaml
parameters:
branchRef: ${{ variables.branchRef}}
# -----------------------------------------
# **templates/group-build.yaml**
parameters:
- name: branchRef
type: string
stages:
- stage: 'Build'
displayName: 'Build'
variables:
jobs:
- template: template/functionapp-build-job.yaml
parameters:
branchRef: ${{ parameters.branchRef}}
- template: template/webapp-build-job.yaml
parameters:
branchRef: ${{ parameters.branchRef}}
# -----------------------------------------
# **template/functionapp-build-job.yaml**
parameters:
- name: branchRef
type: string
jobs:
- template: templates/function/build-job.yaml
parameters:
repository: repo
branchRef: ${{ parameters.branchRef}}
project: functionapp/function.csproj'
# -----------------------------------------
# **templates/function/build-job.yaml**
parameters:
- name: repository
type: string
- name: project
type: string
- name: componentName
type: string
- name: branchRef
type: string
jobs:
- job: JOB_${{ parameters.componentName}}_BUILD # <- Component is unique
displayName: Build ${{ parameters.componentName}} Job
steps:
// Checking out same repo is not allowed here
- checkout: git://project/${{ parameters.repository }}#${{ parameters.branchRef}}
- Other Task here...
# -----------------------------------------
# **template/webapp-build-job.yaml**
# Same structure as function app...
This is how it looks like structurally
Pipeline
Stages:
- Stage:
Jobs:
- Job: Web App Build
Steps:
- checkout: 'Here I call same repo and branch but different build steps'
- Job: Function App Build
Steps:
- checkout: 'Here I call same repo and branch but different build steps'
Validation issue image
Adding resources in pipeline is not my option because refs is being stored in variable group, which is not support to resolve due to limitation of azure yaml as stated here.
Any suggestion or idea to circumvent this issue is highly appreciated.
From your sample, the template webapp-build-job.yaml and functionapp-build-job.yaml are reference the same yaml:build-job.yaml.
In this case, the job names (- job: JOB_BUILD) are the same key and this can be the root cause of this issue.
To solve this issue, you can define job name as a parameter and assign value in webapp-build-job.yaml and functionapp-build-job.yaml.
For example:
build-job.yaml
parameters:
- name: repository
type: string
- name: project
type: string
- name: jobname
type: string
- name: branchRef
type: string
jobs:
- job: ${{ parameters.jobname }}
displayName: Build Job
steps:
- checkout: git://Artifacts/${{ parameters.repository }}#${{ parameters.branchRef}}
functionapp-build-job.yaml
parameters:
- name: branchRef
type: string
jobs:
- template: build-job.yaml
parameters:
repository: Repo11
branchRef: ${{ parameters.branchRef}}
jobname: test1
project: functionapp/function.csproj'
webapp-build-job.yaml
parameters:
- name: branchRef
type: string
jobs:
- template: build-job.yaml
parameters:
repository: Repo11
branchRef: ${{ parameters.branchRef}}
jobname: test2
project: functionapp/function.csproj'
group-build.yaml
parameters:
- name: branchRef
type: string
stages:
- stage: 'Build'
displayName: 'Build'
jobs:
- template: functionapp-build-job.yaml
parameters:
branchRef: ${{ parameters.branchRef}}
- template: webapp-build-job.yaml
parameters:
branchRef: ${{ parameters.branchRef}}
Pipeline.yaml
variables:
- group: test
stages:
- template: group-build.yaml
parameters:
branchRef: ${{ variables.branchRef}}

How do I describe jobs dynamically in Azure DevOps?

I have a number of jobs for different platforms I'd like to run in parallel. I'd like to build a different set of platforms for different situations (i.e. full build, smoke, pull request, etc.). How can I make a list of jobs dynamic based on variables?
For example, if this is one of the "hard-coded" implementations:
jobs:
- job: Platform1
pool: Pool1
steps:
- template: minimal_template.yml
parameters:
BuildTarget: Platform1
- job: Platform2
pool: Pool1
steps:
- template: minimal_template.yml
parameters:
BuildTarget: Platform2
- job: Platform3
pool: Pool2
steps:
- template: minimal_template.yml
parameters:
BuildTarget: Platform3
How could I instead extract out a collection of variable sets, i.e.
[[Platform1, Pool1], [Platform2, Pool1], [Platform3, Pool2]]
And execute that on a pipeline like:
jobs:
??(Foreach platform in platforms)??
- job: $(platform[0])
pool: $(platform[1])
steps:
- template: minimal_template.yml
parameters:
BuildTarget: $(platform[0])
You can define it in the parameters and loop it:
parameters:
- name: Platforms
type: object
default:
- name: 'Platform1'
pool: 'Platform1Pool'
- name: 'Platform2'
pool: 'Platform2Pool'
jobs:
- ${{ each platform in parameters.Platforms}}:
- job: ${{ platform.name }}
pool: ${{ platform.pool }}
steps:
- template: minimal_template.yml
You may alos use 'jobList' type for template parameters:
parameters:
- name: 'testsJobsList'
type: jobList
default: []
jobs:
- ${{ each job in parameters.testsJobsList }}: # Each job
- ${{ each pair in job }}: # Insert all properties other than "steps"
${{ if ne(pair.key, 'steps') }}:
${{ pair.key }}: ${{ pair.value }}
steps: # Wrap the steps
- ${{ job.steps }} # Users steps
And then:
trigger:
- none
pool:
vmImage: 'windows-latest'
jobs:
- template: deployment-template.yml
parameters:
testsJobsList:
- job: Platform1
pool: Platform1Pool
steps:
- template: minimal_template.yml
- job: Platform2
pool: Platform2Pool
steps:
- template: minimal_template.yml
You’re looking for conditions: https://learn.microsoft.com/en-us/azure/devops/pipelines/process/conditions?view=azure-devops&tabs=yaml
Stages, jobs, and steps can all have a condition defined.
For example, running a job if a variable is set to true:
condition: eq(variables['System.debug'], 'true')

Dynamic parameters/variables does not work for deployment job in checkout step in template but work directly in pipeline

The dynamic variables with the checkout step do not work with the template.
The same works without a template.
#- checkout: git://MY_PROJ/MY_REPO#refs/tags/${{parameters.tag_name}} ## DOES NOT WORK
Getting below error while running pipeline:
"ERROR: An item with the same key has already been added"
Working:
my_pipeline.yml without template works
parameters:
- name: RELEASE_TAG
displayName: Enter The Master Release Tag Name Example 1.0.0-RELEASE
default: 1.0.0-RELEASE
type: string
trigger:
- none
variables:
db_resource_path: '$(System.DefaultWorkingDirectory)/resources/db'
pipeline_environment_name: 'PROD_ENV'
db_env_name: 'prod'
stages:
- stage: "PROD_DB_DEPLOYMENT"
displayName: "PROD DB Deployment"
pool:
name: $(param.agent.pool.name)
variables:
- group: PROD_VG
jobs:
- job:
steps:
- script: |
echo param release tag: ${{parameters.RELEASE_TAG}}
- deployment: Deploy
environment: PROD_ENV
strategy:
runOnce:
deploy:
steps:
- checkout: git://MY_PROJ/MY_REPO#refs/tags/${{parameters.RELEASE_TAG}} ## WORKS
#- checkout: git://MY_PROJ/MY_REPO#refs/tags/${{variables.tag_name}} ## WORKS WITH VAR ALSO
Not Working:
Gives ERROR: An item with the same key has already been added.
my azure-pipelines.yml is:
parameters:
- name: RELEASE_TAG
displayName: Enter The Master Release Tag Name Example 1.0.0-RELEASE
default: 1.0.0-RELEASE
type: string
resources:
repositories:
- repository: templates
type: git
name: MY_PROJECT/MY_TEMPLATE_REPO
trigger:
- none
variables:
tagName: '${{ parameters.RELEASE_TAG }}'
stages:
- stage: "PROD_DB_DEPLOYMENT"
displayName: "PROD DB Deployment"
variables:
- group: PROD_VG
jobs:
- template: my_template.yml#templates
parameters:
tag_name: $(tagName)
db_env_name: 'prod'
agent_pool_name: $(param.agent.pool.name)
db_resource_path: $(System.DefaultWorkingDirectory)/resources/db
pipeline_environment_name: PROD_ENV
is_release: 'true'
My Template is:
parameters:
- name: 'agent_pool_name'
type: string
- name: 'db_resource_path'
type: string
- name: 'pipeline_environment_name'
type: string
- name: 'db_env_name'
type: string
- name: 'is_release'
type: string
default: 'false'
- name: 'tag_name'
type: string
default: '1.0.0-RELEASE'
jobs:
- job:
pool:
name: ${{parameters.agent_pool_name}}
steps:
- script: |
echo param tag_name: ${{parameters.tag_name}}
echo var tag_name: $(tagName)
- deployment: Deploy
pool:
name: ${{parameters.agent_pool_name}}
environment: ${{parameters.pipeline_environment_name}}
strategy:
runOnce:
deploy:
steps:
- checkout: self
- ${{ if eq(parameters.is_release, true) }}:
#- checkout: git://MY_PROJ/MY_REPO#refs/tags/1.0.0-RELEASE ## WORKS
#- checkout: git://MY_PROJ/MY_REPO#refs/tags/$(tag_name) ## DOES NOT WORK
#- checkout: git://MY_PROJ/MY_REPO#refs/tags/${{parameters.tag_name}} ## DOES NOT WORK
Tried below variable option also but get error:Unexpected value 'variables'
Any suggestion, please.
This is due to that you passed $(var) as parameters of the template.
You can pass ${{variables.tagName}} instead.
checked on my side screenshot
In a pipeline, template expression variables (${{ variables.var }}) get processed at compile time, before runtime starts. Macro syntax variables ($(var)) get processed during runtime before a task runs.
Because templates are expanded before the pipeline execution gets planned, so you cannot pass $(var) as parameter to the template.
Please check official doc for the variable syntax.
https://learn.microsoft.com/en-us/azure/devops/pipelines/process/variables?view=azure-devops&tabs=yaml%2Cbatch#understand-variable-syntax
Edit: for error "Unexpected value 'variables'":
Move variables to stage scope: my yaml here
My template here

Tags as variables are not supported in environment

I was trying to use a variable as tag while executing the deploy job using the environment template. But from https://developercommunity.visualstudio.com/content/problem/1015274/yaml-variable-names-not-expanded-for-tags-on-envir.html, it's evident that ADO doesn't support variables as tags in environment deployments. For example, this works
variables:
envName: 'EnvironmentName'
vmTag: 'vm-test-01'
environment:
name: "EnvironmentName"
resourceName: 'vm-test-01'
resourceType: "virtualMachine"
tags: "vm01"
strategy:
runOnce:
deploy:
steps:
- powershell: write-host "This is $(VMName)"
but this doesn't
name: "$(envName)"
resourceType: "virtualMachine"
tags: "$(vmTag)"
strategy:
runOnce:
deploy:
steps:
- powershell: write-host "This is $(VMName)"
Does anyone know of an alternate or a workaround for this?
Thanks
UPDATE: This is what I've tried
multistage-pipeline.yml
- stage: "deploytouservm"
displayName: "Stage - Deploy To User VM"
dependsOn: "build"
variables:
- name: envName
value: "environmentName"
- name: userVM
value: "vmName01" #this value is dynamically generated
- template: templates\jobs\deploy-template.yml
parameters:
envName: $(envName)
vmName: $(userVM)
deploy-template.yml
parameters:
- name: envName
- name: vmName
jobs:
- deployment: "deployJob"
environment: ${{ parameters['envName']}}.${{ parameters['vmName']}}
displayName: "Deploy - SCOM To User VM"
strategy:
runOnce:
deploy:
steps:
Try using parameters instead of variables:
parameters:
- name: envName
default: EnvironmentName
jobs:
- deployment: VMDeploy
displayName: Test_script
environment:
name: ${{ parameters.envName}}
resourceType: VirtualMachine
azure-pipelines.yml
stages:
- stage:
variables:
EnvironmentName: Prod
jobs:
- template: steps.yml
parameters:
myParameter: Test
....
steps:yml
parameters:
- name: myParameter
jobs:
- deployment: deployment
environment: ${{ parameters['myParameter']}}
strategy:
...

Is it possible to pass a template with a list of jobs to a jobList type param?

Currently in Azure pipelines, we can pass the list of jobs to be executed to a child template with a parameter of jobList type as shown in the doco.
https://learn.microsoft.com/en-us/azure/devops/pipelines/process/templates?view=azure-devops#iterative-insertion
Is there a way I can encapsulate these jobs from pipeline.yml that are being passed to the jobList param inside another job template and pass that template to a jobList param. I tried to structure my pipeline as follows:
pipeline.yml
deployment-template.yml
post-deploy-tests-dev.yml
post-deploy-smoke-tests-prod.yml
I would like to dynamically insert different tests' jobs to the end of the deployment template depending on the environment. I tried the jobList type parameter in the deployment-template.yml as the following but it throws an error saying mapping not expected.
#post-deploy-tests-dev.yml
jobs:
- job: Test1
steps:
- script: execute test1
#post-deploy-tests-smoke-tests-prod.yml
jobs:
- job: Test2
steps:
- script: execute test2
#pipeline.yml
...
- template: deployment-template.yml
parameters:
environment: dev
testsJobsList:
template: post-deploy-tests-dev.yml
- template: deployment-template.yml
parameters:
environment: prod
testsJobsList:
template: post-deploy-smoke-tests-prod.yml
#deployment-template.yml
parameters:
- name: testsJobsList
type: jobList
default: []
#All deployment jobs here
jobs:
...
...
#Tests as the end
- ${{ parameters.testsJobsList }}
Is there a way to dynamically pass the jobList ?
it throws an error saying mapping not expected.
Test with the YAML sample, the cause of this issue: you are missing - before the template field in pipeline.yml file (- template: post-deploy-tests-dev.yml and - template: post-deploy-smoke-tests-prod.yml). In this position, template is equivalent to a job, and you need to add -.
Here is my Sample:
pipeline.yml
trigger:
- none
pool:
vmImage: 'windows-latest'
jobs:
- template: deployment-template.yml
parameters:
testsJobsList:
- template: post-deploy-tests-dev.yml
- template: deployment-template.yml
parameters:
testsJobsList:
- template: post-deploy-smoke-tests-prod.yml
deployment-template.yml
parameters:
- name: 'testsJobsList'
type: jobList
default: []
jobs:
- ${{ each job in parameters.testsJobsList }}: # Each job
- ${{ each pair in job }}: # Insert all properties other than "steps"
${{ if ne(pair.key, 'steps') }}:
${{ pair.key }}: ${{ pair.value }}
steps: # Wrap the steps
- ${{ job.steps }} # Users steps
post-deploy-tests-dev.yml
jobs:
- job: Test1
steps:
- script: echo test1
post-deploy-smoke-tests-prod.yml
jobs:
- job: Test2
steps:
- script: echo test2
Result: