I have a deployment pipeline in Azure DevOps which deploys database changes to a list of databases. Rather than having these run sequentially I would like to run them in parallel. The rolling deployment strategy supports running in parallel but I don't know how to pass variables in this configuration. For Jobs there is a Matrix option to pass variables to different executions. However I can't find an equivalent to use for deployment.
Here is the relevant pipeline portion
trigger:
- master
pool:
vmImage: 'windows-latest'
variables:
- group: LibraryData
parameters:
- name: 'qaDatabases'
type: object
default:
- databaseSet:
databases: ['DB1','DB1-A']
- databaseSet:
databases: ['DB2','DB2-A']
- databaseSet:
databases: ['DB3','DB3-A']
#Build details skipped, working fine
- stage: DeployToQA
jobs:
- deployment: DeployQA
environment:
name: QA
resourceType: VirtualMachine
tags: Database
strategy:
rolling:
maxParallel: 2
deploy:
steps:
- ${{ each databaseSet in parameters.qaDatabases }}:
- template: Pipeline-Templates/DBDeploy.yml
parameters:
DatabaseServer: "$(lib-QADBServer)"
DBName: ${{ databaseSet.databases[0] }}
DBaName: ${{ databaseSet.databases[1] }}
I am trying to make a pipeline template that takes a JobList a parameter and runs all the jobs, while ensuring that they run on the same agent every time. Basically the approach I've been taking is to try to adapt this answer into a genericized template format.
This is what I have so far, and I've tried a lot of slight tweaks of this with nothing passing the Validate test on the pipeline that calls it.
parameters:
- name: jobsToRun
type: jobList
- name: pool
type: string
default: Default
- name: demands
type: object
default: []
jobs:
- job:
steps:
- script: echo "##vso[task.setvariable variable=agentName;isOutput=true;]$(Agent.Name)"
pool:
name: ${{ parameters.pool }}
demands:
- ${{ each demand in parameters.demands }}:
${{ demand }}
- ${{ each j in parameters.jobsToRun }}:
${{ each pair in j }}:
${{ pair.key }} : ${{ pair.value }}
pool:
name: Default
demands:
- Agent.Name -equals $(agentName)
What am I doing wrong here? It seems like it should be possible if that answer I reference is correct, but it seems like I'm just a bit off.
Name missing on the job.., example below.
- job: 'test-Name'
Steps need a associated job and pool to run is declared inside
jobsToRun:
- job: sample_job1
displayName: "sample_job1"
pool:
name: "your_PoolName"
steps:
- script: |
echo "Hi"
On this bottom pool declaration...
pool:
name: Default
demands:
- Agent.Name -equals $(agentName)
i am not sure but i have tried this many times but i think this can't be included separate to the job since each individual job is passed as parameter. pool definition needs to be inside the job or inside the job template if you are using templates..
example:
jobsToRun:
- job: output_message_job1
displayName: "in pipe Output Message Job"
pool:
name: "your_PoolName"
steps:
- script: |
echo "Hi"
I have a YAML Azure DevOps pipeline that loops through series of configurations, copying artifacts to various places. What I want to do is, after the looping is done, to do something else (I'd like to send an email, but the question is more general than that).
But I can't insert anything after the looping part of the YAML, at least not with any of the experiments I've tried. Here's the YAML that calls the YAML template, with a comment for where I'd like to do another step. How might I do this?
parameters:
- name: configuration
type: object
default:
- Texas
- Japan
- Russia
- Spaghetti
- Philosophy
trigger:
- dev
- master
resources:
repositories:
- repository: templates
name: BuildTemplates
type: git
stages:
- ${{ each configuration in parameters.configuration }}:
- template: build.yml#templates
parameters:
configuration: ${{ configuration }}
appName: all
# Where I'd like to have another task or job or step or stage that can send an email or perhaps other things
Just define a new stage:
stages:
- ${{ each configuration in parameters.configuration }}:
- template: build.yml#templates
parameters:
configuration: ${{ configuration }}
appName: all
- stage: secondStage
jobs:
- job: jobOne
steps:
- task: PowerShell#2
I'm trying to create a deploy pipeline YAML template for all environments/stages. I've set up the Environments on Azure DevOps so that I can add checks and approvals on the Test and Prod environments before they get deployed. I've set up a library group for each stage and each one of them has a variable called 'env' which defines the current stage running in the pipeline. For some reason, the environment property under the deployment job (see code snippet below) doesn't read that variable.
Has anyone faced this issue before, or is there a reason why the variable won't be read for that specific property?
Note: I've tested the variables and they do work, for example, the stage property outputs as 'deploy-dev/test/prod' (depending on the environment)
- stage: deploy-$(env)
jobs:
- deployment: DeployWeb
displayName: deploy Web App
pool:
vmImage: 'Ubuntu-latest'
# creates an environment if it doesn't exist
environment: 'smarthotel-$(env)'
strategy:
runOnce:
deploy:
steps:
- script: echo Hello world
You can't do this because it have to be know at compilation phase.
But you can try this (lets name this file deploy.yml):
parameters:
- name: env
type: string
default: 'dev'
stages:
- stage: deploy-${{ parameters.env }}
jobs:
- deployment: DeployWeb
displayName: deploy Web App
pool:
vmImage: 'Ubuntu-latest'
# creates an environment if it doesn't exist
environment: 'smarthotel-${{ parameters.env }}'
strategy:
runOnce:
deploy:
steps:
- script: echo Hello world
and then you need to run as follows (in build.yml file):
stages:
- template: deploy.yml
parameters:
env: dev
- template: deploy.yml
parameters:
env: qa
- template: deploy.yml
parameters:
env: prod
I have an Azure Pipeline written in YAML which runs from a CI trigger whenever changes are made to the master branch. It can also be manually triggered from Pull Requests or by users against any branch.
Due to the use of a number of licensed components, the build from master needs to run on a specific agent. The other builds do not, and in fact I would rather they run on other agent(s).
So my question is, is there any way to specify a different agent/pool within the YAML pipeline based on what triggered the build, or what branch the build is building? I'd like this to be behaviour which is configured in the pipeline permanently, rather than requiring users to update the YAML on each branch they wish to build elsewhere.
I can't see anything obvious in the sections of the documentation on the pool/demands/condition keywords.
I solved this by putting the steps for the job into a template, and then creating a set of jobs in the pipeline with different condition entries, so that we can set demands based on those conditions.
A skeleton version looks like this:
- stage: Build
jobs:
- job: TopicBranchAndPullRequestBuild
condition: or(startsWith(variables['Build.SourceBranch'], 'refs/heads/topic'), startsWith(variables['Build.SourceBranch'], 'refs/pull'))
displayName: 'Build topic Branch or Pull Request'
pool:
name: the-one-and-only-pool
demands:
- HasLicensedComponents -equals false
steps:
- template: build-template.yml
- job: MasterAndReleaseBranchBuild
condition: or(eq(variables['Build.SourceBranch'], 'refs/heads/master'), startsWith(variables['Build.SourceBranch'], 'refs/heads/release'))
displayName: 'Build master or release Branch'
pool:
name: the-one-and-only-pool
demands:
- HasLicensedComponents -equals true
steps:
- template: build-template.yml
Obviously the values given here are for example only, but otherwise this is what I have working.
Could you try expressions? I've used this with success on variable groups so it might work on agent pools.
- ${{ if eq(variables['build.SourceBranchName'], 'prod') }}:
- pool: Host1
- ${{ if eq(variables['build.SourceBranchName'], 'staging') }}:
- pool: Host2
- ${{ if not(and(eq(variables['build.SourceBranchName'], 'staging'), eq(variables['build.SourceBranchName'], 'prod'))) }}:
- pool: Host3
Credit for original workaround for dynamically pulling variable groups here: https://github.com/MicrosoftDocs/vsts-docs/issues/3702#issuecomment-574278829
The pool keyword specifies which pool to use for a job of the pipeline. A pool specification also holds information about the job's strategy for running. You can specify a pool at the pipeline, stage, or job level. The pool specified at the lowest level of the hierarchy is used to run the job.
is there any way to specify a different agent/pool within the YAML
pipeline based on what triggered the build, or what branch the build
is building?
For this issue , you can create different yaml files for different branches, specify different agent pools in the corresponding yaml file.
You can switch to the yaml file of different branches in pipeline with the option shown below:
parameters:
- name: self_hosted_agent
displayName: Choose your pool agent
type: string
default: windows-latest
values:
- windows-latest
- self-hosted
- name: agent_name
displayName: Choose your agent
type: string
default: 'Laptop Clint'
values:
- 'Laptop Clint'
- 'Laptop Eastwood'
variables:
- name: pool_agent_key
${{ if eq(parameters.self_hosted_agent, 'windows-latest') }}:
value: vmImage
${{ if eq(parameters.self_hosted_agent, 'self-hosted') }}:
value: name
- name: pool_agent_value
${{ if eq(parameters.self_hosted_agent, 'windows-latest') }}:
value: windows-latest
${{ if eq(parameters.self_hosted_agent, 'self-hosted') }}:
value: Iot-Windows
- name: agent_name
value: ${{ parameters.agent_name }}
stages:
- stage: build
displayName: Build
pool:
${{ variables.pool_agent_key }}: ${{ variables.pool_agent_value }}
${{ if eq(parameters.self_hosted_agent, 'self-hosted') }}:
demands:
- agent.name -equals ${{ variables.agent_name }}