Optional job templates in YAML Pipelines - azure-devops

Is it possible to optionally include templates based on some kind of template expression? Specifically, I want my top-level definition in azure-pipelines.yml to call out which build job templates to use in an included stage template:
azure-pipelines.yml :
stages:
- template: generic-build-stage.yml # Template reference
parameters:
# Example of optional build templates to use
buildTypes: [SpecificBuildJobs1, SpecificBuildJobs3, SpecificBuildJobs4]
generic-build-stage.yml :
parameters:
buildTypes: ???
stages:
- stage: generic_build
jobs:
${{ }} # ???? What goes here to include the appropriate templates
- template: ???
The template expression above would ideally expand to this:
jobs:
- template: specific-build-jobs1.yml
- template: specific-build-jobs3.yml
- template: specific-build-jobs4.yml
Edit: The "Iterative insertion" example in the docs seems to suggest that some form of dynamic, parse-time insertion is possible.

The following method worked to allow a top-level pipeline definition to consume a variable number of job sets at a lower level.
azure-pipelines.yml :
stages:
- template: generic-build-stage.yml # Template reference
parameters:
# Example of optional build templates to use
buildTypes: [SpecificBuildJobs1, SpecificBuildJobs3, SpecificBuildJobs4]
generic-build-stage.yml :
parameters:
buildTypes: [MissingBuildType] # Use this if buildTypes is not provided
stages:
- stage: build_stage
jobs:
# Note: VS Code extension for Pipelines (1.1574.4) will
# say this is an "Unexpected property", but this works in ADO
- ${{ if containsValue(parameters.buildTypes, 'MissingBuildType') }}:
- template: build-stage-null.yml
- ${{ if containsValue(parameters.buildTypes, 'SpecificBuildJobs1') }}:
- template: specific-build-jobs1.yml
- ${{ if containsValue(parameters.buildTypes, 'SpecificBuildJobs2') }}:
- template: specific-build-jobs2.yml
- ${{ if containsValue(parameters.buildTypes, 'SpecificBuildJobs3') }}:
- template: specific-build-jobs3.yml
- ${{ if containsValue(parameters.buildTypes, 'SpecificBuildJobs4') }}:
- template: specific-build-jobs4.yml

It seems to be impossible, for the template reference is resolved at parse time.
You may have to set multiple templates for main pipeline, and set the value for buildTypes as the specific template name for job templates, and in generic-build-stage.yml use - template:${{parameters.buildTypes}}.yml to call corresponding job template;
Azure-pipelines.yml:
stages:
- template: generic-build-stage.yml
parameters:
buildTypes:specific-build-jobs1
- template: generic-build-stage.yml
parameters:
buildTypes:specific-build-jobs3
generic-build-stage.yml
parameters:
buildTypes: ""
stages:
- stage: generic_build
jobs:
- template: ${{parameters.buildTypes}}.yml

Related

Issues while passing variable groups parameter to template from azure pipeline.yml

I have declared a variable group Agile-Connections as below and the group do not have any restriction for any Pipeline:
I am using another template called vars.yml to store some other variables:
variables:
- group: Agile-Connections
- name: extensions_dir
value: /apps/agile/product/agile936/integration/sdk/extensions
- name: properties_dir
value: /apps/agile/product/Properties
- name: build_name
value: RestrictPreliminaryBOMPX.jar
- name: resource_name
value: RestrictPreliminaryBOMPX.properties
My Azure Pipeline looks like below, which is calling a deploy.yml template, and I am passing two parameters (connection, environment) from azure pipeline.yml to deploy.yml.
Below is my azure-pipeline.yml:
trigger:
- None
pool:
name: AgentBuildAgile
stages:
- template: templates/build.yml
- stage: DEV_Deployment
variables:
- template: templates/vars.yml
jobs:
- job:
steps:
- script:
echo $(Dev-mnode1)
- template: templates/deploy.yml
parameters:
connection: $(Dev-mnode1)
environment: 'DEV'
Below is my deploy.yml:
parameters:
- name: connection
- name: environment
jobs:
- deployment:
variables:
- template: vars.yml
environment: ${{ parameters.environment }}
displayName: Deploy to ${{ parameters.environment }}
strategy:
runOnce:
deploy:
steps:
- script:
echo Initiating Deployment ${{ parameters.connection }}
- template: copy-artifact.yml
parameters:
connection: ${{ parameters.connection }}
# - template: copy-resources.yml
# parameters:
# connection: ${{ parameters.connection }}
From my deploy.yml I am passing a parameter connection further to another template called copy-artifact.yml, which is below:
parameters:
- name: connection
jobs:
- job:
variables:
- template: vars.yml
displayName: 'Copy jar'
steps:
# - script:
# echo ${{ parameters.connection }}
- task: SSH#0
displayName: 'Task - Backup Existing jar file'
inputs:
sshEndpoint: ${{ parameters.connection }}
runOptions: inline
inline:
if [[ -f ${{ variables.extensions_dir }}/${{ variables.build_name }} ]]; then mv ${{ variables.extensions_dir }}/${{ variables.build_name }} ${{ variables.extensions_dir }}/${{ variables.build_name }}_"`date +"%d%m%Y%H%M%S"`"; echo "Successfully Backed up the existing jar"; fi
Now when I run my pipeline I am getting error message :
The pipeline is not valid. Job Job3: Step SSH input sshEndpoint references service connection $(Dev-mnode1) which could not be found. The service connection does not exist or has not been authorized for use. For authorization details, refer to https://aka.ms/yamlauthz.
When I print the value of $(Dev-mnode1) using the commented out lines in copy-artifacts.yml file, it prints fine (Dev11 Connection) but when I try to use that as service connection for my ssh task, it gives me the above error.
Also, there is a service connection Dev11 Connection in my project and all the pipelines are allowed to use that service connection.
The pipeline is not valid. Job Job3: Step SSH input sshEndpoint references service connection $(Dev-mnode1) which could not be found. The service connection does not exist or has not been authorized for use.
From the error message, the parameter not get the variable value when expand the template.
Test the same sample and can reproduce the issue.
When you define the variable template at stage level, the variables in template will expand at runtime and the paramter:connection will be expand at compile time. So the correct value cannot be passed to parameter.
To solve this issue, you can define the variable template at root level.
Refer to the sample:
trigger:
- None
pool:
name: AgentBuildAgile
variables:
- template: templates/vars.yml
stages:
- template: templates/build.yml
- stage: DEV_Deployment
jobs:
- job:
steps:
- script:
echo $(Dev-mnode1)
- template: templates/deploy.yml
parameters:
connection: $(Dev-mnode1)
environment: 'DEV'

Run template conditionally on azure pipeline yaml

I have two different yml pipelines. I always trigger pipeline & send variable values via postman body. Here is the body portion.
{
"definition": {
"id": 151
},
"parameters": "{\"Plan\":\"sand\"}"
}
Now i want to run a template with the condition that if the Plan is sand, then template will run. I have used many conditions but nothing is working. Providing some conditions that i tried,
- ${{ if eq(variables.Plan, 'sand') }}:
- template: Sandbox Tasks/sand.yml#templates
- ${{ if eq(parameters.Plan, 'sand') }}:
- template: Sandbox Tasks/sand.yml#templates
- template: Sandbox Tasks/sand.yml#templates
condition: eq(variables['Plan'], 'sand')
- template: Sandbox Tasks/sand.yml#templates
condition: eq(parameters['Plan'], 'sand')
- ${{ if eq(variables['Plan'], 'sand') }}:
- template: Sandbox Tasks/sand.yml#templates
- ${{ if eq(parameters['Plan'], 'sand') }}:
- template: Sandbox Tasks/sand.yml#templates
What would be the proper way to run a template with conditions by matching a variables value?
- ${{ if eq(parameters.is_python, true)}}:
- template: ../Python/install-python.yml
parameters:
variable_group: '${{parameters.variable_group}}'
- ${{ if eq(parameters.is_python, false)}}:
- template: cdk-installation.yml
parameters:
variable_group: '${{parameters.variable_group}}'

Is there any possibilities to call AzDo templated with parameters?

I would like to run a few templates based on the initial value chosen from the parameter and as soon as the value is chosen then a template will be issued which will further ask for more parameters only required for that template.
Let's say in main azure-pipelines.yml if a user chooses dev then simply a template will be called. However, if a user chooses test then template create-stack-tst-template.yml will be issued but along with that, it should prompt the parameters needed for this template. Is it possible?
If not, is there any possibility to club all the parameters only needed for dev and the same for test. So that when the individual templates are called then clubbed parameter will be passed which is necessary for that template to run but not for others.
Is there any kind of segregation exists?
trigger:
- none
parameters:
- name: DeployToEnvType
displayName: |
Select the env type to be deployed
type: string
values:
- dev
- test
stages:
- ${{ if eq(parameters['DeployToEnvType'], 'dev' ) }}:
- template: templates/create-stack-dev-template.yml
- ${{ if ne(parameters['DeployToEnvType'], 'test' ) }}:
- template: templates/create-stack-tst-template.yml
parameters:
- name: ProjectName
type: string
- name: ImageSource
type: string
it should prompt the parameters needed for this template. Is it possible?
This is not possible. You need to provide all parameters and pass further only those which are needed by particular template.
trigger:
- none
parameters:
- name: DeployToEnvType
displayName: |
Select the env type to be deployed
type: string
values:
- dev
- test
- name: ImageSource
type: string
stages:
- ${{ if eq(parameters['DeployToEnvType'], 'dev' ) }}:
- template: templates/create-stack-dev-template.yml
parameters:
ProjectName: projectA
ImageSource: ${{ parameters.ImageSource }}
- ${{ if ne(parameters['DeployToEnvType'], 'test' ) }}:
- template: templates/create-stack-tst-template.yml
parameters:
ProjectName: projectA
ImageSource: ${{ parameters.ImageSource }}
If you need control at runtime you need to make corresponding runtime parameter and pass it down. If you want to have some values fixed you can just put them inline.

Azure DevOps YAML stage template

I have two environments, preprod and prod and they are pretty much the same.
So I created an yaml file, InfurstructureTemplate.yaml
parameters:
xxxx
jobs:
- job: provision_job
I want to use this template for my two environments, here is what in mind:
stages:
- stage: PreProd Environment
- template: InfurstructureTemplate.yaml
- parameters:
xxxx
- stage: Prod Environment
- template: InfurstructureTemplate.yaml
- parameters:
xxxx
Is this the right way to use yaml template? When I googled this, seems template is on Stages level and you can't put it on stage.
Yes, you can do it very similar, put the template in the jobs:
stages:
- stage: PreProd_Environment
displayName: PreProd Environment
jobs:
- template: InfurstructureTemplate.yaml
parameters:
projectDir: xxx
- stage: Prod_Environment
displayName: Prod Environment
dependsOn: [ PreProd_Environment ]
jobs:
- template: InfurstructureTemplate.yaml
parameters:
projectDir: xxx
You may try this too as described here
stages:
- template: InfurstructureTemplate.yaml
parameters:
xxxx
- template: InfurstructureTemplate.yaml
parameters:
xxxx

Using variables in environment names on Azure Devops Yml Pipeline

I have a pipeline with a deployment job that has a environment
environment: "myname-$(variable1)"
but when i look at environments in Azure Devops, it has not replaced the variable and named my environment "myname-$(variable1)" and not "myname-helloworld".
Any way to use varaibles for the environment names?
Updated with example
stages.yml
stages:
- stage:
variables:
EnvironmentName: Prod
jobs:
- template: steps.yml
....
steps:yml
jobs:
- deployment: deployment
environment: ${{ EnvironmentName }}
strategy:
...
The solution is to use parameters instead of variables.
https://learn.microsoft.com/en-us/azure/devops/pipelines/process/templates?view=azure-devops#insert-a-template
stages.yml
stages:
- stage:
variables:
EnvironmentName: Prod
jobs:
- template: steps.yml
parameters:
myParameter: Test
....
steps:yml
parameters:
- name: myParameter
jobs:
- deployment: deployment
environment: ${{ parameters['myParameter']}}
strategy:
...
Refer to the expressions documentation. You need to use a runtime or compile time expression.
environment: "myname-${{ variable1 }}"
That expands the variable value when the YAML file is parsed. If variable is a variable that you've defined, it will be expanded accordingly.
If variable1 is defined at some point in the execution of the pipeline, then you need to use the runtime syntax: $[ variable1 ] instead.