Why can't I use a variable to define the environment property in the Azure Pipeline YAML config file? - azure-devops

I'm trying to create a deploy pipeline YAML template for all environments/stages. I've set up the Environments on Azure DevOps so that I can add checks and approvals on the Test and Prod environments before they get deployed. I've set up a library group for each stage and each one of them has a variable called 'env' which defines the current stage running in the pipeline. For some reason, the environment property under the deployment job (see code snippet below) doesn't read that variable.
Has anyone faced this issue before, or is there a reason why the variable won't be read for that specific property?
Note: I've tested the variables and they do work, for example, the stage property outputs as 'deploy-dev/test/prod' (depending on the environment)
- stage: deploy-$(env)
jobs:
- deployment: DeployWeb
displayName: deploy Web App
pool:
vmImage: 'Ubuntu-latest'
# creates an environment if it doesn't exist
environment: 'smarthotel-$(env)'
strategy:
runOnce:
deploy:
steps:
- script: echo Hello world

You can't do this because it have to be know at compilation phase.
But you can try this (lets name this file deploy.yml):
parameters:
- name: env
type: string
default: 'dev'
stages:
- stage: deploy-${{ parameters.env }}
jobs:
- deployment: DeployWeb
displayName: deploy Web App
pool:
vmImage: 'Ubuntu-latest'
# creates an environment if it doesn't exist
environment: 'smarthotel-${{ parameters.env }}'
strategy:
runOnce:
deploy:
steps:
- script: echo Hello world
and then you need to run as follows (in build.yml file):
stages:
- template: deploy.yml
parameters:
env: dev
- template: deploy.yml
parameters:
env: qa
- template: deploy.yml
parameters:
env: prod

Related

Run Azure DevOps deployment pipeline in parallel

I have a deployment pipeline in Azure DevOps which deploys database changes to a list of databases. Rather than having these run sequentially I would like to run them in parallel. The rolling deployment strategy supports running in parallel but I don't know how to pass variables in this configuration. For Jobs there is a Matrix option to pass variables to different executions. However I can't find an equivalent to use for deployment.
Here is the relevant pipeline portion
trigger:
- master
pool:
vmImage: 'windows-latest'
variables:
- group: LibraryData
parameters:
- name: 'qaDatabases'
type: object
default:
- databaseSet:
databases: ['DB1','DB1-A']
- databaseSet:
databases: ['DB2','DB2-A']
- databaseSet:
databases: ['DB3','DB3-A']
#Build details skipped, working fine
- stage: DeployToQA
jobs:
- deployment: DeployQA
environment:
name: QA
resourceType: VirtualMachine
tags: Database
strategy:
rolling:
maxParallel: 2
deploy:
steps:
- ${{ each databaseSet in parameters.qaDatabases }}:
- template: Pipeline-Templates/DBDeploy.yml
parameters:
DatabaseServer: "$(lib-QADBServer)"
DBName: ${{ databaseSet.databases[0] }}
DBaName: ${{ databaseSet.databases[1] }}

Using service containers in Azure DevOps pipelines

I'm trying to use service containers within Azure DevOps pipelines
The agent is an ubuntu host
I would like to have the agent run a powershell container and a playwright container
The doc for this is not very verbose
So far I have this in my main 'azure-pipelines.yml'
trigger: none
pr: none
resources:
containers:
- container: playwright
image: mcr.microsoft.com/playwright:v1.29.0-focal
- container: pwsh
image: mcr.microsoft.com/powershell
pool:
vmImage: "ubuntu-latest"
services:
playwright: playwright
pwsh: pwsh
stages:
- stage: dev
displayName: dev
jobs:
- template: templates/test.yml
And this in my 'template/test.yml' file
- job: run_tests
displayName: Test
pool:
vmImage: ubuntu-latest
steps:
- powershell: |
Write-Host "This is powershell"
target:
container: pwsh
- script: yarn test:integration:ci
displayName: "Run tests"
env:
environment: dev
CI: true
target:
container: playwright
Azure pipelines does not like this. It is failing with:
/.azure/azure-pipelines.yml (Line: 18, Col: 1): Unexpected value 'stages'
when I try to run the pipeline. I thought stages: was the first part of a pipeline? (but I am very new to Azure pipelines so my understanding might be way off)
Could anyone help to clarify why/where I am screwing up at all please?
Thanks
Make the following changes to your yaml files.
azure-pipelines.yml
trigger: none
pr: none
pool:
vmImage: ubuntu-latest
resources:
containers:
- container: playwright
image: mcr.microsoft.com/playwright:v1.29.0-focal
- container: pwsh
image: mcr.microsoft.com/powershell
stages:
- stage: dev
displayName: dev
jobs:
- template: templates/test.yml
template.yml
jobs:
- job: run_tests
displayName: Test
pool:
vmImage: ubuntu-latest
services:
playwright: playwright
pwsh: pwsh
steps:
- powershell: |
Write-Host "This is powershell"
target:
container: pwsh
- script: yarn test:integration:ci
displayName: "Run tests"
env:
environment: dev
CI: true
target:
container: playwright
Reason why you were getting the error (/.azure/azure-pipelines.yml (Line: 18, Col: 1): Unexpected value 'stages') is because of the property services. According to the example in service containers documentation, the property services is defined in the root level of the yaml because the example did not use any stage or jobs.
Since you are using stages and jobs in your yaml pipeline, the services property should be nested within your job.
Hence, I have moved the services to the template.yml file. You can check which property is allowed under a stage or job using this YAML schema documentation
Reference: https://learn.microsoft.com/en-us/azure/devops/pipelines/yaml-schema/jobs-job-container?view=azure-pipelines

Azure DevOps: How do I expand variables in a template in the YML file referring to that template for use in a conditional?

I have two Azure DevOps .yml-based templates, one referring to another. I'm finding that variables specified in the one being referred to do not expand when I need them to in a conditional in the template referring to that template. Here is how they look:
Parent template:
jobs:
- deployment:
displayName: Release
variables:
- template: variables.yml
environment:
name: QA
resourceType: VirtualMachine
strategy:
runOnce:
deploy:
...
#this copies to xxMaster
- task: CopyFiles#2
#condition: eq('${{ parameters.appName }}', 'all')
condition: and(eq(variables['Build.SourceBranchName'], 'master'), eq('${{ configuration }}', '$(productionConfiguration)'))
inputs:
SourceFolder: 'C:\DevOps\$(Build.BuildNumber)\Content\D_C\a\1\s\xxTexas\obj\Release\Package\PackageTmp'
Contents: '**\*.*'
OverWrite: true
TargetFolder: 'C:\Production\Web Sites\xxMaster'
Child Template:
variables:
- name: solution
value: '**/*.sln'
- name: buildPlatform
value: 'Any CPU'
- name: buildConfiguration
value: "${{ parameters.configuration }}"
- name: productionConfiguration
value: 'Horsie'
In this case, when the conditional should work and ${{ configuration }} equals 'Horsie', the YAML is evaluated and comes up with
and(eq(variables['Build.SourceBranchName'], 'master'), eq('Horsie', '$(productionConfiguration)'))
That is, $(productionConfiguration), the variable specified in the referred-to template, is never expanded into 'Horsie.' What should I do to make it so the variable from that sub-template is expanded and my conditional works?
From your YAML sample, the cause of this issue is the format of the variable:$(productionConfiguration) in condition.
You need to use the format: variables['productionConfiguration'] to call the pipeline variable.
Here is an example:
stages:
- stage: deploy
jobs:
- deployment: DeployWeb
displayName: deploy Web App
variables:
- template: template.yml
pool:
vmImage: 'Ubuntu-latest'
# creates an environment if it doesn't exist
environment: 'smarthotel-dev'
strategy:
runOnce:
deploy:
steps:
- script: echo Hello world
condition: and(eq(variables['Build.SourceBranchName'], 'main'), eq('${{ configuration }}', variables['productionConfiguration']))

Azure Devops YML Template Conditions and Dependencies

I have a requirement for a couple of job templates but I can’t get the conditions and dependencies configured to run how I want. I have the basic yaml below:
parameters:
- name: environment
displayName: Environment
- name: action
values:
- deploy
- undeploy
- redeploy
default: redeploy
stages:
- stage: app
displayName: App
jobs:
- deployment: kickoff
environment:
name: ${{ parameters.environment }}
resourceType: virtualMachine
tags: app
strategy:
runOnce:
deploy:
steps:
- checkout: none
- powershell: Write-Host "Run kickoff tasks"
- template: undeploy.yml
parameters:
environment: ${{ parameters.environment }}
action: ${{ parameters.action }}
- template: deploy.yml
parameters:
environment: ${{ parameters.environment }}
action: ${{ parameters.action }}
The requirements I have are the below:
Undeploy.yml: if Action = ‘Deploy’, does not run. If Action = ‘Undeploy’, only run this template. Action = ‘Redeploy’, run both templates but undeploy.yml must run first.
Deploy.yml: If Action = ‘Deploy’, only run this template. If Action = ‘Undeploy’, does not run. If Action = ‘Redeploy’, run both templates but deploy.yml must run second.
Closest I can get to is setting the templates yaml as below:
undeploy.yml
parameters:
- name: environment
default: environmentToDeployTo
- name: action
values:
- deploy
- undeploy
- redeploy
default: redeploy
jobs:
- deployment: undeploy
dependsOn: kickoff
condition: ne ('${{parameters.action}}', 'deploy')
environment:
name: ${{ parameters.environment }}
resourceType: virtualMachine
tags: app
strategy:
runOnce:
deploy:
steps:
- checkout: none
#undeploy steps here#
deploy.yml
parameters:
- name: environment
default: environmentToDeployTo
- name: action
values:
- deploy
- undeploy
- redeploy
default: redeploy
jobs:
- deployment: dploy
dependsOn: undeploy
condition: ne ('${{parameters.action}}', 'undeploy')
environment:
name: ${{ parameters.environment }}
resourceType: virtualMachine
tags: app
strategy:
runOnce:
deploy:
steps:
- checkout: none
#dploy steps here#
This results only affects deploy.yml but it is not able to run in isolation - the undeploy.yml job template will have to run beforehand. However, without the "dependsOn", I can see no other way to ensure that undeploy.yml runs first.
Is there anyway to achieve this as per requirements outlined above?
Thanks in advance
dependsOn takes the name of a stage or job as input, not the name of a template file. Seeing more of your templates would be useful, but if you could use the name of the stage or job in dependsOn, you might get the result you're looking for.
Think of the expanded template as your guide here - you can refer to a stage or job that's declared in a different template, so long as it expands to the right place when the pipeline runs.

Azure DevOps Pipeline define variable in deployment and reuse in subsequent job

I'm using an Azure DevOps pipeline to deploy my code and now I'm in need of passing a variable value from a deployment job to a subsequent job that depends on it. I've read up on this example but it does not seem to work at all.
What I'm trying to do is run an Azure ARM Deployment that provisions a Key Vault. The name of the key vault is outputted from the ARM deployment job and I'm then trying to pass that name to another job which needs to add specific secrets. Access control is taken care of, but I still need to pass the name.
I've boiled the problem down to the basics of passing a variable from a deployment to a job. Here is my complete test pipeline (almost entirely copied from here):
trigger: none
stages:
- stage: X
jobs:
- deployment: A
pool:
vmImage: "ubuntu-16.04"
environment: test
strategy:
runOnce:
deploy:
steps:
- script: echo "##vso[task.setvariable variable=myOutputVar;isOutput=true]this is the deployment variable value"
name: setvarStep
- script: echo $(setvarStep.myOutputVar)
name: echovar
- job: B
dependsOn: A
pool:
vmImage: "ubuntu-16.04"
variables:
myVarFromDeploymentJob: $[ dependencies.A.outputs['deploy.setvarStep.myOutputVar'] ]
steps:
- script: "echo $(myVarFromDeploymentJob)"
name: echovar
Once I run this the echoed value is blank in job B, but defined in deployment A. Why is this? And is there a way to dum everything in dependencies.A.outputs so that I can see what I have to work with?
How can I pass a variable from a runOnce deployment job to a regular job?
I've solved it. The problem is that the documentation here specifies this schema for fetching the variable for a runOnce deployment:
$[dependencies.<job-name>.outputs['<lifecycle-hookname>.<step-name>.<variable-name>']]
This is in fact WRONG. The <lifecycle-hookname> parameter should be replaced with the name of the deployment like this:
$[dependencies.<job-name>.outputs['<job-name>.<step-name>.<variable-name>']]
The example from this documentation (scroll down a bit) is correct.
A full example pipeline that I've tested and works:
trigger: none
stages:
- stage: X
jobs:
- deployment: A # This name is important
pool:
vmImage: 'ubuntu-16.04'
environment: staging
strategy:
runOnce:
deploy:
steps:
- script: echo "##vso[task.setvariable variable=myOutputVar;isOutput=true]this is the deployment variable value"
name: setvarStep # This name is also important
- script: echo $(setvarStep.myOutputVar)
name: echovar
- job: B
dependsOn: A
pool:
vmImage: 'ubuntu-16.04'
variables:
myVarFromDeploymentJob: $[ dependencies.A.outputs['A.setvarStep.myOutputVar'] ]
steps:
- script: "echo $(myVarFromDeploymentJob)"
name: echovar