I have a requirement for a couple of job templates but I can’t get the conditions and dependencies configured to run how I want. I have the basic yaml below:
parameters:
- name: environment
displayName: Environment
- name: action
values:
- deploy
- undeploy
- redeploy
default: redeploy
stages:
- stage: app
displayName: App
jobs:
- deployment: kickoff
environment:
name: ${{ parameters.environment }}
resourceType: virtualMachine
tags: app
strategy:
runOnce:
deploy:
steps:
- checkout: none
- powershell: Write-Host "Run kickoff tasks"
- template: undeploy.yml
parameters:
environment: ${{ parameters.environment }}
action: ${{ parameters.action }}
- template: deploy.yml
parameters:
environment: ${{ parameters.environment }}
action: ${{ parameters.action }}
The requirements I have are the below:
Undeploy.yml: if Action = ‘Deploy’, does not run. If Action = ‘Undeploy’, only run this template. Action = ‘Redeploy’, run both templates but undeploy.yml must run first.
Deploy.yml: If Action = ‘Deploy’, only run this template. If Action = ‘Undeploy’, does not run. If Action = ‘Redeploy’, run both templates but deploy.yml must run second.
Closest I can get to is setting the templates yaml as below:
undeploy.yml
parameters:
- name: environment
default: environmentToDeployTo
- name: action
values:
- deploy
- undeploy
- redeploy
default: redeploy
jobs:
- deployment: undeploy
dependsOn: kickoff
condition: ne ('${{parameters.action}}', 'deploy')
environment:
name: ${{ parameters.environment }}
resourceType: virtualMachine
tags: app
strategy:
runOnce:
deploy:
steps:
- checkout: none
#undeploy steps here#
deploy.yml
parameters:
- name: environment
default: environmentToDeployTo
- name: action
values:
- deploy
- undeploy
- redeploy
default: redeploy
jobs:
- deployment: dploy
dependsOn: undeploy
condition: ne ('${{parameters.action}}', 'undeploy')
environment:
name: ${{ parameters.environment }}
resourceType: virtualMachine
tags: app
strategy:
runOnce:
deploy:
steps:
- checkout: none
#dploy steps here#
This results only affects deploy.yml but it is not able to run in isolation - the undeploy.yml job template will have to run beforehand. However, without the "dependsOn", I can see no other way to ensure that undeploy.yml runs first.
Is there anyway to achieve this as per requirements outlined above?
Thanks in advance
dependsOn takes the name of a stage or job as input, not the name of a template file. Seeing more of your templates would be useful, but if you could use the name of the stage or job in dependsOn, you might get the result you're looking for.
Think of the expanded template as your guide here - you can refer to a stage or job that's declared in a different template, so long as it expands to the right place when the pipeline runs.
Related
I have a deployment pipeline in Azure DevOps which deploys database changes to a list of databases. Rather than having these run sequentially I would like to run them in parallel. The rolling deployment strategy supports running in parallel but I don't know how to pass variables in this configuration. For Jobs there is a Matrix option to pass variables to different executions. However I can't find an equivalent to use for deployment.
Here is the relevant pipeline portion
trigger:
- master
pool:
vmImage: 'windows-latest'
variables:
- group: LibraryData
parameters:
- name: 'qaDatabases'
type: object
default:
- databaseSet:
databases: ['DB1','DB1-A']
- databaseSet:
databases: ['DB2','DB2-A']
- databaseSet:
databases: ['DB3','DB3-A']
#Build details skipped, working fine
- stage: DeployToQA
jobs:
- deployment: DeployQA
environment:
name: QA
resourceType: VirtualMachine
tags: Database
strategy:
rolling:
maxParallel: 2
deploy:
steps:
- ${{ each databaseSet in parameters.qaDatabases }}:
- template: Pipeline-Templates/DBDeploy.yml
parameters:
DatabaseServer: "$(lib-QADBServer)"
DBName: ${{ databaseSet.databases[0] }}
DBaName: ${{ databaseSet.databases[1] }}
Lately I've been trying to make an Azure DevOps pipeline to deploy to 3 environments, with 2 different data centers and 2 different service connections. I've been trying to achieve this with using as little lines of YAML as possible.
After a lot of trial and error, I'm stuck on this message "An error occurred while loading the YAML build pipeline. An item with the same key has already been added."
deploy-env.yaml:
parameters:
- name: OPENSHIFT_NAMESPACE
displayName: 'OpenShift namespace'
type: object
values: []
- name: DCR
displayName: 'Data Center'
type: object
values: []
- name: OSC
displayName: 'Openshift service connection'
type: object
values: []
stages:
- ${{ each namespace in parameters.OPENSHIFT_NAMESPACE }}:
- ${{ each dcr in parameters.DCR }}:
- ${{ each osc in parameters.OSC }}:
- stage: deploy-${{ convertToJson(namespace) }}
jobs:
- deployment: deploy_to_dcr
environment: ${{ namespace }}
displayName: 'Deploy to DCR1'
strategy:
runOnce:
deploy:
steps:
- template: steps/deploy_to_cluster_with_helm_templating.yml#pipeline_templates
parameters:
DATA_CENTER: ${{ dcr }}
OPENSHIFT_NAMESPACE: ${{ namespace }}
OPENSHIFT_SERVICE_CONNECTION: '${{ osc }}'
HELM_VALUES:
- 'global.namespace=${{ namespace }}'
- 'global.data_center=${{ DCR }}'
azure-pipeline.yaml
resources:
repositories:
- repository: pipeline_templates
type: git
name: pipeline-templates
stages:
- template: deploy-env.yaml
parameters:
OPENSHIFT_NAMESPACE:
- development
- test
- acceptance
DCR:
- dcr1
- dcr2
OSC:
- OCP4DCR1
- OCP4DCR2
Does anyone knows why this error occurs? I've found other articles where stage/job names we're not unique, but that is not the case in this example.
Thanks in advance.
This line is getting repeated twelve times, with only four different values:
- stage: deploy-${{ convertToJson(namespace) }}
Stage names must be unique.
I have two Azure DevOps .yml-based templates, one referring to another. I'm finding that variables specified in the one being referred to do not expand when I need them to in a conditional in the template referring to that template. Here is how they look:
Parent template:
jobs:
- deployment:
displayName: Release
variables:
- template: variables.yml
environment:
name: QA
resourceType: VirtualMachine
strategy:
runOnce:
deploy:
...
#this copies to xxMaster
- task: CopyFiles#2
#condition: eq('${{ parameters.appName }}', 'all')
condition: and(eq(variables['Build.SourceBranchName'], 'master'), eq('${{ configuration }}', '$(productionConfiguration)'))
inputs:
SourceFolder: 'C:\DevOps\$(Build.BuildNumber)\Content\D_C\a\1\s\xxTexas\obj\Release\Package\PackageTmp'
Contents: '**\*.*'
OverWrite: true
TargetFolder: 'C:\Production\Web Sites\xxMaster'
Child Template:
variables:
- name: solution
value: '**/*.sln'
- name: buildPlatform
value: 'Any CPU'
- name: buildConfiguration
value: "${{ parameters.configuration }}"
- name: productionConfiguration
value: 'Horsie'
In this case, when the conditional should work and ${{ configuration }} equals 'Horsie', the YAML is evaluated and comes up with
and(eq(variables['Build.SourceBranchName'], 'master'), eq('Horsie', '$(productionConfiguration)'))
That is, $(productionConfiguration), the variable specified in the referred-to template, is never expanded into 'Horsie.' What should I do to make it so the variable from that sub-template is expanded and my conditional works?
From your YAML sample, the cause of this issue is the format of the variable:$(productionConfiguration) in condition.
You need to use the format: variables['productionConfiguration'] to call the pipeline variable.
Here is an example:
stages:
- stage: deploy
jobs:
- deployment: DeployWeb
displayName: deploy Web App
variables:
- template: template.yml
pool:
vmImage: 'Ubuntu-latest'
# creates an environment if it doesn't exist
environment: 'smarthotel-dev'
strategy:
runOnce:
deploy:
steps:
- script: echo Hello world
condition: and(eq(variables['Build.SourceBranchName'], 'main'), eq('${{ configuration }}', variables['productionConfiguration']))
I'm trying to create a deploy pipeline YAML template for all environments/stages. I've set up the Environments on Azure DevOps so that I can add checks and approvals on the Test and Prod environments before they get deployed. I've set up a library group for each stage and each one of them has a variable called 'env' which defines the current stage running in the pipeline. For some reason, the environment property under the deployment job (see code snippet below) doesn't read that variable.
Has anyone faced this issue before, or is there a reason why the variable won't be read for that specific property?
Note: I've tested the variables and they do work, for example, the stage property outputs as 'deploy-dev/test/prod' (depending on the environment)
- stage: deploy-$(env)
jobs:
- deployment: DeployWeb
displayName: deploy Web App
pool:
vmImage: 'Ubuntu-latest'
# creates an environment if it doesn't exist
environment: 'smarthotel-$(env)'
strategy:
runOnce:
deploy:
steps:
- script: echo Hello world
You can't do this because it have to be know at compilation phase.
But you can try this (lets name this file deploy.yml):
parameters:
- name: env
type: string
default: 'dev'
stages:
- stage: deploy-${{ parameters.env }}
jobs:
- deployment: DeployWeb
displayName: deploy Web App
pool:
vmImage: 'Ubuntu-latest'
# creates an environment if it doesn't exist
environment: 'smarthotel-${{ parameters.env }}'
strategy:
runOnce:
deploy:
steps:
- script: echo Hello world
and then you need to run as follows (in build.yml file):
stages:
- template: deploy.yml
parameters:
env: dev
- template: deploy.yml
parameters:
env: qa
- template: deploy.yml
parameters:
env: prod
In my YAML pipeline I have a deployment job:
- stage: deployment_to_development
jobs:
- deployment: deployment_to_development
displayName: Deploy to Development
environment: Development
variables:
- template: migrate-vars.yml
strategy:
runOnce:
deploy:
steps:
- template: migrate-data.yml
The deployment template is a simple DbUp task:
steps:
- task: UpdateDatabaseWithDbUp#2
displayName: Migrate data
inputs:
ConnectionString: 'Data Source=$(DatabaseServer);Initial Catalog=$(DatabaseName);Integrated Security=SSPI'
ScriptPath: '$(Pipeline.Workspace)\data-migrations'
JournalSchemaName: dbo
JournalTableName: _SchemaVersions
LogScriptOutput: true
IncludeSubfolders: true
Order: FolderStructure
TransactionStrategy: SingleTransaction
The variables template defines the server and db name:
variables:
DatabaseServer: 'server'
DatabaseName: 'dbName'
Instances: '_1,_2'
This all works fine for a single database. However, I wish to repeat the task for each instance defined in the Instances variable, i.e for databases named dbName_1, dbName_2. This seemingly simple feat is proving difficult.
I have tried passing the instances as a parameter array and iterating them using
parameters:
param: []
steps:
- ${{each p in parameters.param}}:
but the 'p' variable isn't evaluated in the task.
There have been many more futile attempts. I must be missing something very obvious here. What is it?
I can't test UpdateDatabaseWithDbUp#2 but I have sth what explain how you can achieve your goal. First define template.yaml
parameters:
- name: 'instances'
type: object
default: {}
- name: 'server'
type: string
default: ''
steps:
- ${{ each instance in parameters.instances }}:
- script: echo ${{ parameters.server }}:${{ instance }}
then reuse this template in your build:
steps:
- template: template.yaml
parameters:
instances:
- test1
- test2
server: someServer
And here is the result: