I have a deployment pipeline in Azure DevOps which deploys database changes to a list of databases. Rather than having these run sequentially I would like to run them in parallel. The rolling deployment strategy supports running in parallel but I don't know how to pass variables in this configuration. For Jobs there is a Matrix option to pass variables to different executions. However I can't find an equivalent to use for deployment.
Here is the relevant pipeline portion
trigger:
- master
pool:
vmImage: 'windows-latest'
variables:
- group: LibraryData
parameters:
- name: 'qaDatabases'
type: object
default:
- databaseSet:
databases: ['DB1','DB1-A']
- databaseSet:
databases: ['DB2','DB2-A']
- databaseSet:
databases: ['DB3','DB3-A']
#Build details skipped, working fine
- stage: DeployToQA
jobs:
- deployment: DeployQA
environment:
name: QA
resourceType: VirtualMachine
tags: Database
strategy:
rolling:
maxParallel: 2
deploy:
steps:
- ${{ each databaseSet in parameters.qaDatabases }}:
- template: Pipeline-Templates/DBDeploy.yml
parameters:
DatabaseServer: "$(lib-QADBServer)"
DBName: ${{ databaseSet.databases[0] }}
DBaName: ${{ databaseSet.databases[1] }}
Related
I have a YAML Azure DevOps pipeline that loops through series of configurations, copying artifacts to various places. What I want to do is, after the looping is done, to do something else (I'd like to send an email, but the question is more general than that).
But I can't insert anything after the looping part of the YAML, at least not with any of the experiments I've tried. Here's the YAML that calls the YAML template, with a comment for where I'd like to do another step. How might I do this?
parameters:
- name: configuration
type: object
default:
- Texas
- Japan
- Russia
- Spaghetti
- Philosophy
trigger:
- dev
- master
resources:
repositories:
- repository: templates
name: BuildTemplates
type: git
stages:
- ${{ each configuration in parameters.configuration }}:
- template: build.yml#templates
parameters:
configuration: ${{ configuration }}
appName: all
# Where I'd like to have another task or job or step or stage that can send an email or perhaps other things
Just define a new stage:
stages:
- ${{ each configuration in parameters.configuration }}:
- template: build.yml#templates
parameters:
configuration: ${{ configuration }}
appName: all
- stage: secondStage
jobs:
- job: jobOne
steps:
- task: PowerShell#2
I have a requirement for a couple of job templates but I can’t get the conditions and dependencies configured to run how I want. I have the basic yaml below:
parameters:
- name: environment
displayName: Environment
- name: action
values:
- deploy
- undeploy
- redeploy
default: redeploy
stages:
- stage: app
displayName: App
jobs:
- deployment: kickoff
environment:
name: ${{ parameters.environment }}
resourceType: virtualMachine
tags: app
strategy:
runOnce:
deploy:
steps:
- checkout: none
- powershell: Write-Host "Run kickoff tasks"
- template: undeploy.yml
parameters:
environment: ${{ parameters.environment }}
action: ${{ parameters.action }}
- template: deploy.yml
parameters:
environment: ${{ parameters.environment }}
action: ${{ parameters.action }}
The requirements I have are the below:
Undeploy.yml: if Action = ‘Deploy’, does not run. If Action = ‘Undeploy’, only run this template. Action = ‘Redeploy’, run both templates but undeploy.yml must run first.
Deploy.yml: If Action = ‘Deploy’, only run this template. If Action = ‘Undeploy’, does not run. If Action = ‘Redeploy’, run both templates but deploy.yml must run second.
Closest I can get to is setting the templates yaml as below:
undeploy.yml
parameters:
- name: environment
default: environmentToDeployTo
- name: action
values:
- deploy
- undeploy
- redeploy
default: redeploy
jobs:
- deployment: undeploy
dependsOn: kickoff
condition: ne ('${{parameters.action}}', 'deploy')
environment:
name: ${{ parameters.environment }}
resourceType: virtualMachine
tags: app
strategy:
runOnce:
deploy:
steps:
- checkout: none
#undeploy steps here#
deploy.yml
parameters:
- name: environment
default: environmentToDeployTo
- name: action
values:
- deploy
- undeploy
- redeploy
default: redeploy
jobs:
- deployment: dploy
dependsOn: undeploy
condition: ne ('${{parameters.action}}', 'undeploy')
environment:
name: ${{ parameters.environment }}
resourceType: virtualMachine
tags: app
strategy:
runOnce:
deploy:
steps:
- checkout: none
#dploy steps here#
This results only affects deploy.yml but it is not able to run in isolation - the undeploy.yml job template will have to run beforehand. However, without the "dependsOn", I can see no other way to ensure that undeploy.yml runs first.
Is there anyway to achieve this as per requirements outlined above?
Thanks in advance
dependsOn takes the name of a stage or job as input, not the name of a template file. Seeing more of your templates would be useful, but if you could use the name of the stage or job in dependsOn, you might get the result you're looking for.
Think of the expanded template as your guide here - you can refer to a stage or job that's declared in a different template, so long as it expands to the right place when the pipeline runs.
I'm trying to create a deploy pipeline YAML template for all environments/stages. I've set up the Environments on Azure DevOps so that I can add checks and approvals on the Test and Prod environments before they get deployed. I've set up a library group for each stage and each one of them has a variable called 'env' which defines the current stage running in the pipeline. For some reason, the environment property under the deployment job (see code snippet below) doesn't read that variable.
Has anyone faced this issue before, or is there a reason why the variable won't be read for that specific property?
Note: I've tested the variables and they do work, for example, the stage property outputs as 'deploy-dev/test/prod' (depending on the environment)
- stage: deploy-$(env)
jobs:
- deployment: DeployWeb
displayName: deploy Web App
pool:
vmImage: 'Ubuntu-latest'
# creates an environment if it doesn't exist
environment: 'smarthotel-$(env)'
strategy:
runOnce:
deploy:
steps:
- script: echo Hello world
You can't do this because it have to be know at compilation phase.
But you can try this (lets name this file deploy.yml):
parameters:
- name: env
type: string
default: 'dev'
stages:
- stage: deploy-${{ parameters.env }}
jobs:
- deployment: DeployWeb
displayName: deploy Web App
pool:
vmImage: 'Ubuntu-latest'
# creates an environment if it doesn't exist
environment: 'smarthotel-${{ parameters.env }}'
strategy:
runOnce:
deploy:
steps:
- script: echo Hello world
and then you need to run as follows (in build.yml file):
stages:
- template: deploy.yml
parameters:
env: dev
- template: deploy.yml
parameters:
env: qa
- template: deploy.yml
parameters:
env: prod
In my YAML pipeline I have a deployment job:
- stage: deployment_to_development
jobs:
- deployment: deployment_to_development
displayName: Deploy to Development
environment: Development
variables:
- template: migrate-vars.yml
strategy:
runOnce:
deploy:
steps:
- template: migrate-data.yml
The deployment template is a simple DbUp task:
steps:
- task: UpdateDatabaseWithDbUp#2
displayName: Migrate data
inputs:
ConnectionString: 'Data Source=$(DatabaseServer);Initial Catalog=$(DatabaseName);Integrated Security=SSPI'
ScriptPath: '$(Pipeline.Workspace)\data-migrations'
JournalSchemaName: dbo
JournalTableName: _SchemaVersions
LogScriptOutput: true
IncludeSubfolders: true
Order: FolderStructure
TransactionStrategy: SingleTransaction
The variables template defines the server and db name:
variables:
DatabaseServer: 'server'
DatabaseName: 'dbName'
Instances: '_1,_2'
This all works fine for a single database. However, I wish to repeat the task for each instance defined in the Instances variable, i.e for databases named dbName_1, dbName_2. This seemingly simple feat is proving difficult.
I have tried passing the instances as a parameter array and iterating them using
parameters:
param: []
steps:
- ${{each p in parameters.param}}:
but the 'p' variable isn't evaluated in the task.
There have been many more futile attempts. I must be missing something very obvious here. What is it?
I can't test UpdateDatabaseWithDbUp#2 but I have sth what explain how you can achieve your goal. First define template.yaml
parameters:
- name: 'instances'
type: object
default: {}
- name: 'server'
type: string
default: ''
steps:
- ${{ each instance in parameters.instances }}:
- script: echo ${{ parameters.server }}:${{ instance }}
then reuse this template in your build:
steps:
- template: template.yaml
parameters:
instances:
- test1
- test2
server: someServer
And here is the result:
For an Azure Pipelines yaml file I want to run a set of tasks once on every agent in a certain pool. When I looked at jobs strategy matrix, it looked as a good solution for this, but it currently cannot pick-up the variable I use for this.
The pipeline yaml file, relevant for this problem is this part:
resources:
- repo: self
trigger: none
jobs:
- job: RunOnEveryAgent
strategy:
maxParallel: 3
matrix:
agent_1:
agentName: Hosted Agent
agent_2:
agentName: Hosted VS2017 2
agent_3:
agentName: Hosted VS2017 3
pool:
name: Hosted VS2017
demands:
- msbuild
- visualstudio
- Agent.Name -equals $(agentName)
steps:
- (etc.)
With this script I tried to setup a matrix to run once on each of the three agents in the pool. However when I try to reference the agent on the list of demands it doesn't pick it up. The actual error message is as follows:
[Error 1] No agent found in pool Hosted VS2017 which satisfies the specified demands:
msbuild
visualstudio
Agent.Name -equals $(agentName)
Agent.Version -gtVersion 2.141.1
If I hardcode the agent name it does work:
demands:
- msbuild
- visualstudio
- Agent.Name Hosted VS2017 3
Is it supported to use these variables in the pool demands? Or should I use a different variable or expression?
parameters:
- name: agentNames
type: object
default: []
jobs:
- job: RunOnEveryAgent
strategy:
matrix:
${{ each agentName in parameters.agentNames }}:
${{ agentName }}:
agentName: ${{ agentName }}
pool:
name: Hosted VS2017
demands:
- msbuild
- visualstudio
- Agent.Name -equals $(agentName)
This would be a better solution in case you want to add more agents in future
Variables are not supported in some of these jobs, due to the order in which they get expanded.
What you can do however, is to use template include syntax (https://learn.microsoft.com/en-us/azure/devops/pipelines/process/templates?view=azure-devops) for your job strategy and pass the agentname in as a parameter.
So your build job in its own YAML file might look like:
parameters:
agentName1: ''
agentName2: ''
agentName3: ''
jobs:
- job: RunOnEveryAgent
strategy:
maxParallel: 3
matrix:
agent_1:
agentName: ${{ parameters.agentName1 }}
agent_2:
agentName: ${{ parameters.agentName2 }}
agent_3:
agentName: ${{ parameters.agentName3 }}
pool:
name: Hosted VS2017
demands:
- msbuild
- visualstudio
- Agent.Name -equals ${{ parameters.agentName3 }}
steps:
Your main azure-pipelines.yml then changes to look like this:
resources:
- repo: self
trigger: none
jobs:
- template: buildjob.yml
parameters:
agentName1: 'Hosted Agent'
agentName2: 'Hosted VS2017 2'
agentName3: 'Hosted VS2017 3'