We have two deployment jobs that run in the same stage. The first job creates an output variable and the second job uses that output variable (code borrowed from here and implemented the same way in our pipeline).
jobs:
- deployment: producer
environment:
name: ${{ parameters.environment }}
resourceType: VirtualMachine
tags: ${{ parameters.tags }}
strategy:
runOnce:
deploy:
steps:
- script: echo "##vso[task.setvariable variable=myOutputVar;isOutput=true]this is the deployment variable value"
name: setvarStep
- script: echo $(setvarStep.myOutputVar)
name: echovar
- deployment: consumer_deploy
dependsOn: producer
variables:
myVarFromDeploymentJob: $[ dependencies.producer.outputs['deploy_Vm1.setvarStep.myOutputVar'] ]
environment:
name: ${{ parameters.environment }}
resourceType: VirtualMachine
tags: ${{ parameters.tags }}
strategy:
runOnce:
deploy:
steps:
- script: "echo $(myVarFromDeploymentJob)"
name: echovar
This works because we reference the virtual machine (hardcoded) that the producer deployment job runs on. However, not every stage will run on the same virtual machine.
I've tried regular variables ($(Agent.MachineName)), as well as expression syntax, passing the variable from a template file and changing the scope of the variable template, but none of them work and the 'myVarFromDeploymentJob' variable stays empty.
Is there a way to make the virtual machine name in the expression variable or more flexible? So going from this:
$[ dependencies.producer.outputs['deploy_Vm1.setvarStep.myOutputVar'] ]
To something like this:
$[ dependencies.producer.outputs['deploy_$(Agent.MachineName).setvarStep.myOutputVar'] ]
Adding a solution for others.
here missing link to docs: https://learn.microsoft.com/en-us/azure/devops/pipelines/process/deployment-jobs?view=azure-devops#support-for-output-variables
for runOnce deployment depends if you are addressing whole deployment (repeat Job name) or resource deployment (use deploy_resourceName):
variables:
myVarFromDeploymentJob: $[ dependencies.A2.outputs['A2.setvarStepTwo.myOutputVar'] ]
myOutputVarTwo: $[ dependencies.A2.outputs['Deploy_vmsfortesting.setvarStepTwo.myOutputVarTwo'] ]
Related
Here is a code snippet:
stages:
- stage: Apply
dependsOn: Plan
variables:
OVERRIDE_ADO_ENVIRONMENT: $[ dependencies.Plan.outputs['Plan.IsTerraformPlanEmpty.OVERRIDE_ADO_ENVIRONMENT'] ]
condition: and(succeeded(), ${{ parameters.terraform_apply }})
jobs:
- deployment: Apply
environment: ${{ coalesce(variables.OVERRIDE_ADO_ENVIRONMENT, parameters.ado_environment) }}
strategy:
runOnce:
deploy:
steps:
- template: start.yaml
- template: terraform_init.yaml
parameters:
I know the build variable OVERRIDE_ADO_ENVIRONMENT is declared correctly, because I can use it in the condition to skip the Apply stage completely.
However, this is incorrect. Even if the plan is empty, there could be a change in the terraform output variables. Therefore I must run the Apply logic always. However, there is no need for approvals in this case.
Therefore I would like to switch the environment to the one in the OVERRIDE_ADO_ENVIRONMENT build variable which is a special environment with no approvals.
However, trying to run this pipeline produces the following error message:
Job Apply: Environment $[ dependencies could not be found. The environment does not exist or has not been authorized for use.
From which I conclude we cannot use a build variable, albeit computed in a previous stage.
The question is - what is the least painful way to implement this logic? If at all possible.
Edit 1
I tried an approach where I create two stages with a condition that is affected by the output variable from the previous stage. However, I found out that:
The condition must be on the stage, not deployment job. Otherwise, the environment is applied even if the deployment job's condition disables it.
However, the condition on the stage does not see the build variables shared at the same level. Thus the condition is always false.
Here is my attempt to use this approach
parameters:
- name: terraform_apply
type: boolean
- name: ado_environment
- name: working_directory
- name: application
default: terraform
- name: apply_stages
type: object
default:
- name: ApplyNonEmptyPlan
displayName: Apply Non Empty Plan
tf_plan_tag: TF_NON_EMPTY_PLAN
- name: ApplyEmptyPlan
displayName: Apply Empty Plan
tf_plan_tag: TF_EMPTY_PLAN
ado_environment: Empty TF Plan
stages:
- ${{ each apply_stage in parameters.apply_stages }}:
- stage: ${{ apply_stage.name }}
displayName: ${{ apply_stage.displayName }}
dependsOn: Plan
variables:
TF_PLAN_TAG: $[ stageDependencies.Plan.Plan.outputs['IS_TERRAFORM_PLAN_EMPTY.TF_PLAN_TAG'] ]
condition: and(succeeded(), ${{ parameters.terraform_apply }}, eq(variables['TF_PLAN_TAG'], '${{ apply_stage.tf_plan_tag }}'))
jobs:
- deployment: ${{ apply_stage.name }}
environment: ${{ coalesce(apply_stage.ado_environment, parameters.ado_environment) }}
strategy:
runOnce:
deploy:
steps:
- template: start.yaml
Environment creation happens at compile time before run time. It doesn't support dynamic environment name. Hence, in the code snippet shared below, each element in
coalesce should be known(or hardcode) before you run the pipeline, it cannot depends on the output calculated from previous stage.
environment: ${{ coalesce(variables.OVERRIDE_ADO_ENVIRONMENT, parameters.ado_environment) }}
and
environment: ${{ coalesce(apply_stage.ado_environment, parameters.ado_environment) }}
I am trying to make a pipeline template that takes a JobList a parameter and runs all the jobs, while ensuring that they run on the same agent every time. Basically the approach I've been taking is to try to adapt this answer into a genericized template format.
This is what I have so far, and I've tried a lot of slight tweaks of this with nothing passing the Validate test on the pipeline that calls it.
parameters:
- name: jobsToRun
type: jobList
- name: pool
type: string
default: Default
- name: demands
type: object
default: []
jobs:
- job:
steps:
- script: echo "##vso[task.setvariable variable=agentName;isOutput=true;]$(Agent.Name)"
pool:
name: ${{ parameters.pool }}
demands:
- ${{ each demand in parameters.demands }}:
${{ demand }}
- ${{ each j in parameters.jobsToRun }}:
${{ each pair in j }}:
${{ pair.key }} : ${{ pair.value }}
pool:
name: Default
demands:
- Agent.Name -equals $(agentName)
What am I doing wrong here? It seems like it should be possible if that answer I reference is correct, but it seems like I'm just a bit off.
Name missing on the job.., example below.
- job: 'test-Name'
Steps need a associated job and pool to run is declared inside
jobsToRun:
- job: sample_job1
displayName: "sample_job1"
pool:
name: "your_PoolName"
steps:
- script: |
echo "Hi"
On this bottom pool declaration...
pool:
name: Default
demands:
- Agent.Name -equals $(agentName)
i am not sure but i have tried this many times but i think this can't be included separate to the job since each individual job is passed as parameter. pool definition needs to be inside the job or inside the job template if you are using templates..
example:
jobsToRun:
- job: output_message_job1
displayName: "in pipe Output Message Job"
pool:
name: "your_PoolName"
steps:
- script: |
echo "Hi"
I have a requirement for a couple of job templates but I can’t get the conditions and dependencies configured to run how I want. I have the basic yaml below:
parameters:
- name: environment
displayName: Environment
- name: action
values:
- deploy
- undeploy
- redeploy
default: redeploy
stages:
- stage: app
displayName: App
jobs:
- deployment: kickoff
environment:
name: ${{ parameters.environment }}
resourceType: virtualMachine
tags: app
strategy:
runOnce:
deploy:
steps:
- checkout: none
- powershell: Write-Host "Run kickoff tasks"
- template: undeploy.yml
parameters:
environment: ${{ parameters.environment }}
action: ${{ parameters.action }}
- template: deploy.yml
parameters:
environment: ${{ parameters.environment }}
action: ${{ parameters.action }}
The requirements I have are the below:
Undeploy.yml: if Action = ‘Deploy’, does not run. If Action = ‘Undeploy’, only run this template. Action = ‘Redeploy’, run both templates but undeploy.yml must run first.
Deploy.yml: If Action = ‘Deploy’, only run this template. If Action = ‘Undeploy’, does not run. If Action = ‘Redeploy’, run both templates but deploy.yml must run second.
Closest I can get to is setting the templates yaml as below:
undeploy.yml
parameters:
- name: environment
default: environmentToDeployTo
- name: action
values:
- deploy
- undeploy
- redeploy
default: redeploy
jobs:
- deployment: undeploy
dependsOn: kickoff
condition: ne ('${{parameters.action}}', 'deploy')
environment:
name: ${{ parameters.environment }}
resourceType: virtualMachine
tags: app
strategy:
runOnce:
deploy:
steps:
- checkout: none
#undeploy steps here#
deploy.yml
parameters:
- name: environment
default: environmentToDeployTo
- name: action
values:
- deploy
- undeploy
- redeploy
default: redeploy
jobs:
- deployment: dploy
dependsOn: undeploy
condition: ne ('${{parameters.action}}', 'undeploy')
environment:
name: ${{ parameters.environment }}
resourceType: virtualMachine
tags: app
strategy:
runOnce:
deploy:
steps:
- checkout: none
#dploy steps here#
This results only affects deploy.yml but it is not able to run in isolation - the undeploy.yml job template will have to run beforehand. However, without the "dependsOn", I can see no other way to ensure that undeploy.yml runs first.
Is there anyway to achieve this as per requirements outlined above?
Thanks in advance
dependsOn takes the name of a stage or job as input, not the name of a template file. Seeing more of your templates would be useful, but if you could use the name of the stage or job in dependsOn, you might get the result you're looking for.
Think of the expanded template as your guide here - you can refer to a stage or job that's declared in a different template, so long as it expands to the right place when the pipeline runs.
I have a environment with two VMs, tagges with "SQL" and "APP", I would like to have deployment job targeting SQL vm and having step, creating a output variable, another deployment job targeting "APP" VM should able to consume it in its step
Reference doc
Job consuming the variable should use syntax like -
$[dependencies.<job-name>.outputs['<lifecycle-hookname>_<resource-name>.<step-name>.<variable-name>']]
Consider a scenario you have environment with name "env-vm"
env-vm has two vms registered with it, vm1 with tag SQL and vm2 with tag APP. refer below sample for how to produce and consume the output variable from deployment job
jobs:
- deployment: producer
environment:
name: env-vm
resourceType: VirtualMachine
tags: SQL
strategy:
runOnce:
deploy:
steps:
- script: echo "##vso[task.setvariable variable=myOutputVar;isOutput=true]this is the deployment variable value"
name: setvarStep
- script: echo $(setvarStep.myOutputVar)
name: echovar
- deployment: consumer_deploy
dependsOn: producer
variables:
myVarFromDeploymentJob: $[ coalesce(dependencies.producer.outputs['deploy_Vm1.setvarStep.myOutputVar'], 'fubar') ]
environment:
name: env-vm
resourceType: VirtualMachine
tags: APP
strategy:
runOnce:
deploy:
steps:
- script: "echo $(myVarFromDeploymentJob)"
name: echovar
I'm using an Azure DevOps pipeline to deploy my code and now I'm in need of passing a variable value from a deployment job to a subsequent job that depends on it. I've read up on this example but it does not seem to work at all.
What I'm trying to do is run an Azure ARM Deployment that provisions a Key Vault. The name of the key vault is outputted from the ARM deployment job and I'm then trying to pass that name to another job which needs to add specific secrets. Access control is taken care of, but I still need to pass the name.
I've boiled the problem down to the basics of passing a variable from a deployment to a job. Here is my complete test pipeline (almost entirely copied from here):
trigger: none
stages:
- stage: X
jobs:
- deployment: A
pool:
vmImage: "ubuntu-16.04"
environment: test
strategy:
runOnce:
deploy:
steps:
- script: echo "##vso[task.setvariable variable=myOutputVar;isOutput=true]this is the deployment variable value"
name: setvarStep
- script: echo $(setvarStep.myOutputVar)
name: echovar
- job: B
dependsOn: A
pool:
vmImage: "ubuntu-16.04"
variables:
myVarFromDeploymentJob: $[ dependencies.A.outputs['deploy.setvarStep.myOutputVar'] ]
steps:
- script: "echo $(myVarFromDeploymentJob)"
name: echovar
Once I run this the echoed value is blank in job B, but defined in deployment A. Why is this? And is there a way to dum everything in dependencies.A.outputs so that I can see what I have to work with?
How can I pass a variable from a runOnce deployment job to a regular job?
I've solved it. The problem is that the documentation here specifies this schema for fetching the variable for a runOnce deployment:
$[dependencies.<job-name>.outputs['<lifecycle-hookname>.<step-name>.<variable-name>']]
This is in fact WRONG. The <lifecycle-hookname> parameter should be replaced with the name of the deployment like this:
$[dependencies.<job-name>.outputs['<job-name>.<step-name>.<variable-name>']]
The example from this documentation (scroll down a bit) is correct.
A full example pipeline that I've tested and works:
trigger: none
stages:
- stage: X
jobs:
- deployment: A # This name is important
pool:
vmImage: 'ubuntu-16.04'
environment: staging
strategy:
runOnce:
deploy:
steps:
- script: echo "##vso[task.setvariable variable=myOutputVar;isOutput=true]this is the deployment variable value"
name: setvarStep # This name is also important
- script: echo $(setvarStep.myOutputVar)
name: echovar
- job: B
dependsOn: A
pool:
vmImage: 'ubuntu-16.04'
variables:
myVarFromDeploymentJob: $[ dependencies.A.outputs['A.setvarStep.myOutputVar'] ]
steps:
- script: "echo $(myVarFromDeploymentJob)"
name: echovar