I have a environment with two VMs, tagges with "SQL" and "APP", I would like to have deployment job targeting SQL vm and having step, creating a output variable, another deployment job targeting "APP" VM should able to consume it in its step
Reference doc
Job consuming the variable should use syntax like -
$[dependencies.<job-name>.outputs['<lifecycle-hookname>_<resource-name>.<step-name>.<variable-name>']]
Consider a scenario you have environment with name "env-vm"
env-vm has two vms registered with it, vm1 with tag SQL and vm2 with tag APP. refer below sample for how to produce and consume the output variable from deployment job
jobs:
- deployment: producer
environment:
name: env-vm
resourceType: VirtualMachine
tags: SQL
strategy:
runOnce:
deploy:
steps:
- script: echo "##vso[task.setvariable variable=myOutputVar;isOutput=true]this is the deployment variable value"
name: setvarStep
- script: echo $(setvarStep.myOutputVar)
name: echovar
- deployment: consumer_deploy
dependsOn: producer
variables:
myVarFromDeploymentJob: $[ coalesce(dependencies.producer.outputs['deploy_Vm1.setvarStep.myOutputVar'], 'fubar') ]
environment:
name: env-vm
resourceType: VirtualMachine
tags: APP
strategy:
runOnce:
deploy:
steps:
- script: "echo $(myVarFromDeploymentJob)"
name: echovar
Related
I have a requirement for a couple of job templates but I can’t get the conditions and dependencies configured to run how I want. I have the basic yaml below:
parameters:
- name: environment
displayName: Environment
- name: action
values:
- deploy
- undeploy
- redeploy
default: redeploy
stages:
- stage: app
displayName: App
jobs:
- deployment: kickoff
environment:
name: ${{ parameters.environment }}
resourceType: virtualMachine
tags: app
strategy:
runOnce:
deploy:
steps:
- checkout: none
- powershell: Write-Host "Run kickoff tasks"
- template: undeploy.yml
parameters:
environment: ${{ parameters.environment }}
action: ${{ parameters.action }}
- template: deploy.yml
parameters:
environment: ${{ parameters.environment }}
action: ${{ parameters.action }}
The requirements I have are the below:
Undeploy.yml: if Action = ‘Deploy’, does not run. If Action = ‘Undeploy’, only run this template. Action = ‘Redeploy’, run both templates but undeploy.yml must run first.
Deploy.yml: If Action = ‘Deploy’, only run this template. If Action = ‘Undeploy’, does not run. If Action = ‘Redeploy’, run both templates but deploy.yml must run second.
Closest I can get to is setting the templates yaml as below:
undeploy.yml
parameters:
- name: environment
default: environmentToDeployTo
- name: action
values:
- deploy
- undeploy
- redeploy
default: redeploy
jobs:
- deployment: undeploy
dependsOn: kickoff
condition: ne ('${{parameters.action}}', 'deploy')
environment:
name: ${{ parameters.environment }}
resourceType: virtualMachine
tags: app
strategy:
runOnce:
deploy:
steps:
- checkout: none
#undeploy steps here#
deploy.yml
parameters:
- name: environment
default: environmentToDeployTo
- name: action
values:
- deploy
- undeploy
- redeploy
default: redeploy
jobs:
- deployment: dploy
dependsOn: undeploy
condition: ne ('${{parameters.action}}', 'undeploy')
environment:
name: ${{ parameters.environment }}
resourceType: virtualMachine
tags: app
strategy:
runOnce:
deploy:
steps:
- checkout: none
#dploy steps here#
This results only affects deploy.yml but it is not able to run in isolation - the undeploy.yml job template will have to run beforehand. However, without the "dependsOn", I can see no other way to ensure that undeploy.yml runs first.
Is there anyway to achieve this as per requirements outlined above?
Thanks in advance
dependsOn takes the name of a stage or job as input, not the name of a template file. Seeing more of your templates would be useful, but if you could use the name of the stage or job in dependsOn, you might get the result you're looking for.
Think of the expanded template as your guide here - you can refer to a stage or job that's declared in a different template, so long as it expands to the right place when the pipeline runs.
We have two deployment jobs that run in the same stage. The first job creates an output variable and the second job uses that output variable (code borrowed from here and implemented the same way in our pipeline).
jobs:
- deployment: producer
environment:
name: ${{ parameters.environment }}
resourceType: VirtualMachine
tags: ${{ parameters.tags }}
strategy:
runOnce:
deploy:
steps:
- script: echo "##vso[task.setvariable variable=myOutputVar;isOutput=true]this is the deployment variable value"
name: setvarStep
- script: echo $(setvarStep.myOutputVar)
name: echovar
- deployment: consumer_deploy
dependsOn: producer
variables:
myVarFromDeploymentJob: $[ dependencies.producer.outputs['deploy_Vm1.setvarStep.myOutputVar'] ]
environment:
name: ${{ parameters.environment }}
resourceType: VirtualMachine
tags: ${{ parameters.tags }}
strategy:
runOnce:
deploy:
steps:
- script: "echo $(myVarFromDeploymentJob)"
name: echovar
This works because we reference the virtual machine (hardcoded) that the producer deployment job runs on. However, not every stage will run on the same virtual machine.
I've tried regular variables ($(Agent.MachineName)), as well as expression syntax, passing the variable from a template file and changing the scope of the variable template, but none of them work and the 'myVarFromDeploymentJob' variable stays empty.
Is there a way to make the virtual machine name in the expression variable or more flexible? So going from this:
$[ dependencies.producer.outputs['deploy_Vm1.setvarStep.myOutputVar'] ]
To something like this:
$[ dependencies.producer.outputs['deploy_$(Agent.MachineName).setvarStep.myOutputVar'] ]
Adding a solution for others.
here missing link to docs: https://learn.microsoft.com/en-us/azure/devops/pipelines/process/deployment-jobs?view=azure-devops#support-for-output-variables
for runOnce deployment depends if you are addressing whole deployment (repeat Job name) or resource deployment (use deploy_resourceName):
variables:
myVarFromDeploymentJob: $[ dependencies.A2.outputs['A2.setvarStepTwo.myOutputVar'] ]
myOutputVarTwo: $[ dependencies.A2.outputs['Deploy_vmsfortesting.setvarStepTwo.myOutputVarTwo'] ]
I'm trying to create a deploy pipeline YAML template for all environments/stages. I've set up the Environments on Azure DevOps so that I can add checks and approvals on the Test and Prod environments before they get deployed. I've set up a library group for each stage and each one of them has a variable called 'env' which defines the current stage running in the pipeline. For some reason, the environment property under the deployment job (see code snippet below) doesn't read that variable.
Has anyone faced this issue before, or is there a reason why the variable won't be read for that specific property?
Note: I've tested the variables and they do work, for example, the stage property outputs as 'deploy-dev/test/prod' (depending on the environment)
- stage: deploy-$(env)
jobs:
- deployment: DeployWeb
displayName: deploy Web App
pool:
vmImage: 'Ubuntu-latest'
# creates an environment if it doesn't exist
environment: 'smarthotel-$(env)'
strategy:
runOnce:
deploy:
steps:
- script: echo Hello world
You can't do this because it have to be know at compilation phase.
But you can try this (lets name this file deploy.yml):
parameters:
- name: env
type: string
default: 'dev'
stages:
- stage: deploy-${{ parameters.env }}
jobs:
- deployment: DeployWeb
displayName: deploy Web App
pool:
vmImage: 'Ubuntu-latest'
# creates an environment if it doesn't exist
environment: 'smarthotel-${{ parameters.env }}'
strategy:
runOnce:
deploy:
steps:
- script: echo Hello world
and then you need to run as follows (in build.yml file):
stages:
- template: deploy.yml
parameters:
env: dev
- template: deploy.yml
parameters:
env: qa
- template: deploy.yml
parameters:
env: prod
I'm using an Azure DevOps pipeline to deploy my code and now I'm in need of passing a variable value from a deployment job to a subsequent job that depends on it. I've read up on this example but it does not seem to work at all.
What I'm trying to do is run an Azure ARM Deployment that provisions a Key Vault. The name of the key vault is outputted from the ARM deployment job and I'm then trying to pass that name to another job which needs to add specific secrets. Access control is taken care of, but I still need to pass the name.
I've boiled the problem down to the basics of passing a variable from a deployment to a job. Here is my complete test pipeline (almost entirely copied from here):
trigger: none
stages:
- stage: X
jobs:
- deployment: A
pool:
vmImage: "ubuntu-16.04"
environment: test
strategy:
runOnce:
deploy:
steps:
- script: echo "##vso[task.setvariable variable=myOutputVar;isOutput=true]this is the deployment variable value"
name: setvarStep
- script: echo $(setvarStep.myOutputVar)
name: echovar
- job: B
dependsOn: A
pool:
vmImage: "ubuntu-16.04"
variables:
myVarFromDeploymentJob: $[ dependencies.A.outputs['deploy.setvarStep.myOutputVar'] ]
steps:
- script: "echo $(myVarFromDeploymentJob)"
name: echovar
Once I run this the echoed value is blank in job B, but defined in deployment A. Why is this? And is there a way to dum everything in dependencies.A.outputs so that I can see what I have to work with?
How can I pass a variable from a runOnce deployment job to a regular job?
I've solved it. The problem is that the documentation here specifies this schema for fetching the variable for a runOnce deployment:
$[dependencies.<job-name>.outputs['<lifecycle-hookname>.<step-name>.<variable-name>']]
This is in fact WRONG. The <lifecycle-hookname> parameter should be replaced with the name of the deployment like this:
$[dependencies.<job-name>.outputs['<job-name>.<step-name>.<variable-name>']]
The example from this documentation (scroll down a bit) is correct.
A full example pipeline that I've tested and works:
trigger: none
stages:
- stage: X
jobs:
- deployment: A # This name is important
pool:
vmImage: 'ubuntu-16.04'
environment: staging
strategy:
runOnce:
deploy:
steps:
- script: echo "##vso[task.setvariable variable=myOutputVar;isOutput=true]this is the deployment variable value"
name: setvarStep # This name is also important
- script: echo $(setvarStep.myOutputVar)
name: echovar
- job: B
dependsOn: A
pool:
vmImage: 'ubuntu-16.04'
variables:
myVarFromDeploymentJob: $[ dependencies.A.outputs['A.setvarStep.myOutputVar'] ]
steps:
- script: "echo $(myVarFromDeploymentJob)"
name: echovar
I am trying to use Azure DevOps to configure a docker swarm cluster, with an environment containing 3 RHEL7 vm's with Docker installed and one of them being configured as the swarm host.
Here is the yaml file for the pipeline:
jobs:
- deployment: MasterNode
strategy:
runOnce:
deploy:
steps:
- bash: |
SWARMTOKEN=$(docker swarm join-token manager -q)
echo "##vso[task.setvariable variable=tokenswarm;isOutput=true]$SWARMTOKEN"
name: setvar
- script: echo $(setvar.tokenswarm)
name: echovar
environment:
name: swarm
resourceType: VirtualMachine
tags: SwarmNodeMaster
- deployment: SwarmNode
dependsOn: MasterNode
variables:
tokenvar: $[ dependencies.MasterNode.outputs['deploy.setvar.tokenswarm'] ]
strategy:
runOnce:
deploy:
steps:
- bash: |
echo $(tokenvar)
environment:
name: swarm
resourceType: VirtualMachine
tags: SwarmNode
I have logged this over at VSD https://developercommunity.visualstudio.com/content/problem/937451/unable-to-share-variable-from-one-deployment-job-t.html
I have tried setting the second deployment job to a normal job as per the syntax at https://learn.microsoft.com/en-us/azure/devops/release-notes/2020/sprint-164-update#support-for-output-variables-in-a-deployment-job
jobs:
- deployment: masternode
strategy:
runOnce:
deploy:
steps:
- script: |
SWARMTOKEN="Docker GUID"
echo "##vso[task.setvariable variable=token;isOutput=true]$SWARMTOKEN"
name: 'setvar'
- script: echo $(setvar.token)
name: echovar
environment:
name: swarm
resourceType: VirtualMachine
tags: SwarmNodeMaster
- job: 'swarmnode'
dependsOn: 'masternode'
variables:
tokenvar: $[ dependencies.masternode.outputs['deploy.setvar.token'] ]
steps:
- script: echo $(tokenvar)
name: echovar
It seems whatever I try I can't get the variable to expand in the next deployment step.
Thanks in advance for any help!
UPDATE:
#Kontekst thanks for sharing your working yaml, using that i've managed to track it down to defining the environment for the "master" node:
environment:
name: swarm
resourceType: VirtualMachine
tags: SwarmNodeMaster
The swarm master node has the tag "SwarmNodeMaster" and is part of the same environment as the 2 other VMs which have the tag "SwarmNode" which is used in the second step if I change the above to just environment: 'swarm' I can pass a variable to the next step, but again stops working if I try and target the master node...
Here is the yaml that successfully passes a variable to the next deployment job:
https://hastebin.com/gudelokufi.bash
My main problem is that I need to get the token from the swarm host to the swarm nodes, but when I try and pass a variable from the deployment step that targets the swarm host, to the deployment job that targets the swarm nodes, the variable cannot be expanded in the second deployment job...
This is what I want to work, but seems to break the output variable: https://hastebin.com/yocepowopu.bash
So after much trial and error I found that when you use environment tags as part of a deployment job, it appends the job name with the environment resource name, followed by the lifecycle hook. This is the working code:
stages:
- stage: MyStage
jobs:
- deployment: masterNode
displayName: AnyDeploy
pool:
vmImage: 'ubuntu-latest'
environment:
name: swarm
resourceType: virtualmachine
tags: swarmnodemaster
strategy:
runOnce:
deploy:
steps:
- task: Bash#3
displayName: 'Setting output variable'
inputs:
targetType: 'inline'
script: |
token=Some_Value
echo "##vso[task.setvariable variable=token;isOutput=true]$token"
echo $token
name: setvarStep
- task: Bash#3
inputs:
targetType: 'inline'
script: 'env | sort'
- deployment: swarmNode
dependsOn: masterNode
pool:
vmImage: 'ubuntu-latest'
environment:
name: swarm
resourceType: virtualmachine
tags: swarmnode
variables:
myToken: $[ dependencies.masterNode.outputs['masterNode_<resource name>_Deploy.setvarStep.token'] ]
strategy:
runOnce:
deploy:
steps:
- task: Bash#3
displayName: 'Checking if output variable was passed'
inputs:
targetType: 'inline'
script: |
echo My docker token = $(myToken)
I haven't found a way to reference the job name as a variable yet, as the value changes with each job, but the system variable is "SYSTEM_JOBNAME"