Trigger pipeline with jobs from another pipeline - azure-devops

I need to develop two azure devops pipelines. 'Starter.yml' - is trigger for second one. 'Processing.yml' - has some business logic.
Very important thing: both of them must have jobs.
I tried doing something like this:
Starter:
trigger: none
pool:
vmImage: 'windows-2019'
stages:
- stage: A
jobs:
- job: Triggering
steps:
- template: Processing.yml
Processing:
jobs:
- job: Processing
steps:
- task: PowerShell#2
inputs:
targetType: 'inline'
script: |
# Write your PowerShell commands here.
Write-Host "Hello World"
I get following result:
I saw a lot of examples but all of them dont work. Of course I have parallel jobs:
Is it generally possible to trigger pipeline from another pipeline when both of them have jobs?
Thank you for your answers.

Thanks everyone for their answers, I managed to do what I wanted:
In following example:
There are two pipelines ('Starter' and 'Processing')
'Starter' gets input parameters in variables
'Starter' triggers 'Processing' (- template: Processing.yml) and passes input parameters to it (parameters: firstName: $(FirsName))
Processing uses the parameters (${{parameters.firstName}})
Starter
trigger: none
pool:
vmImage: 'windows-2019'
stages:
- stage: A
jobs:
- template: Processing.yml
parameters:
firstName: $(FirsName)
lastName: $(LastName)
- job: Finishing
steps:
- task: PowerShell#2
inputs:
targetType: 'inline'
script: |
# Write your PowerShell commands here.
Write-Host "Finished"
Processing
parameters:
- name: firstName
default: ''
- name: lastName
default: ''
jobs:
- job: Processing
pool:
vmImage: 'windows-2019'
steps:
- task: PowerShell#2
inputs:
targetType: 'inline'
script: |
# Write your PowerShell commands here.
Write-Host "Hello from second pipeline ${{parameters.firstName}}"
For using parallel jobs you have to buy microsoft-hosted parallel jobs (settings of your project):

Related

Variable in stage not set

my goal is to define a variable in the parent script in Azure DevOps for my pipeline, that I can read it within the template script.
I tried this:
stages:
- stage: Download_Suppliers_Artifacts
variables:
buildVersion2: test2
jobs:
- template: /Pipelines/Templates/download-components-hcp5.yml
Within my script I tried to access it but it is empty:
It is not showed here:
- task: Bash#3
inputs:
targetType: 'inline'
script: 'env | sort'
Nor here:
- task: Bash#3
inputs:
targetType: 'inline'
script: '{{ variables.buildVersion2}}'
This is related to the Runtime expression syntax for variables: https://learn.microsoft.com/en-us/azure/devops/pipelines/process/variables?view=azure-devops&tabs=yaml%2Cbatch#runtime-expression-syntax
In your case, you could directly use macro syntax $(var) instead of template expression ${{ variables.var }}
Test sample (Parent script)
# Starter pipeline
# Start with a minimal pipeline that you can customize to build and deploy your code.
# Add steps that build, run tests, deploy, and more:
# https://aka.ms/yaml
trigger:
- none
pool:
vmImage: windows-latest
stages:
- stage: test
variables:
buildVersion2: test2
jobs:
- template: download-components-hcp5.yml
- job:
steps:
- script: echo $(buildVersion2)
Template:
# download-components-hcp5.yml
jobs:
- job: job1
steps:
- script: env | sort
- powershell: echo $(buildVersion2)

Pass value through jobs in Azure DevOps pipeline

I try to pass value through my jobs in Azure DevOps pipeline and I use for it this code:
trigger: none
pool:
vmImage: 'windows-2019'
stages:
- stage: Processing
jobs:
- job: A
steps:
- task: PowerShell#2
inputs:
targetType: 'inline'
script: |
$someValue = 1234
Write-Host ("##vso[task.setvariable variable=someValue; isOutput=true;]$someValue")
- job: B
dependsOn: ['A']
variables:
someValue: $[ dependencies.A.outputs['setVariable.someValue'] ]
steps:
- task: PowerShell#2
inputs:
targetType: 'inline'
script: |
Write-host "Hello there"
echo $(someValue)
As a result I get this:
What do I wrong? What code do I need for passing value?
Test your YAML sample and reproduce the same issue.
From your YAML code, it has the following issues.
1.When you set the variable in Powershell task, the command has an extra space character in the command.
Refer to the following sample to set variable:
Write-Host "##vso[task.setvariable variable=someValue;isOutput=true;]$someValue"
2.You need to define the name of the PowerShell task which is used to set the variable. Then you can use the name in next job to get the variable. name: taskname
Here is an full example:
pool:
vmImage: 'windows-2019'
stages:
- stage: Processing
jobs:
- job: A
steps:
- task: PowerShell#2
name: taskname
inputs:
targetType: 'inline'
script: |
$someValue = 1234
Write-Host "##vso[task.setvariable variable=someValue;isOutput=true;]$someValue"
- job: B
dependsOn: A
variables:
someValue: $[ dependencies.A.outputs['taskname.someValue'] ]
steps:
- task: PowerShell#2
inputs:
targetType: 'inline'
script: |
Write-host "Hello there"
echo $(someValue)
For more detailed info, you can refer to this doc: Share variables across pipelines

Pass keyvault variable to stage template azure devops pipeline

If I do the same thing but don't use a template so passing variables between 2 stages then it works and I can echo "$(varFromStageA)" but when I implement the same using a template the variable is blank.
azure-pipeline.yaml
stages:
- stage: A
jobs:
- job: JA
steps:
- task: AzureKeyVault#1
displayName: 'Get Secret'
inputs:
azureSubscription: "***********"
KeyVaultName: "*****"
SecretsFilter: '*'
RunAsPreJob: true
- script: |
echo "This is job Foo."
echo "##vso[task.setvariable variable=doThing;isOutput=true]$(MySecret)"
name: DetermineResult
- script: echo $(DetermineResult.doThing)
name: echovar
my-pipeline-template.yaml
parameters:
#source
sourceAccountEndpoint: '******'
sourceDatabaseName: '*****'
sourceAccount: '****'
sourceEnvironment: '******'
- stage: DownloadScript
displayName: migrata data
dependsOn: A
jobs:
- job: Pull
variables:
varFromStageA: $[stageDependencies.A.DetermineResult.outputs['DetermineResult.doThing'] ]
pool:
vmImage: windows-latest
displayName: migrate
steps:
- checkout: self
- task: Powershell#2
inputs:
targetType: 'inline'
script: |
echo "$(varFromStageA)"
Refer to this doc: Jobs can access output variables from previous stages
The format of referencing variables between stages is as follows:
stageDependencies.stageName.jobName.outputs['stepName.variableName']
From your YAML Sample, stageName: A, JobName : JA.
So you need to define the variable in template with the following format:
varFromStageA: $[stageDependencies.A.JA.outputs['DetermineResult.doThing']
Then the value of the variable can be passed.
Template example:
stages:
- stage: DownloadScript
displayName: migrata data
dependsOn: A
jobs:
- job: Pull
variables:
varFromStageA: $[stageDependencies.A.JA.outputs['DetermineResult.doThing'] ]
pool:
vmImage: windows-latest
displayName: migrate
steps:
- checkout: self
- task: Powershell#2
inputs:
targetType: 'inline'
script: |
echo "$(varFromStageA)"
Have a look at this:
https://learn.microsoft.com/en-us/azure/devops/pipelines/process/variables?view=azure-devops&tabs=yaml%2Cbatch
There's some documented examples - it might help. However, why not just set a global variable in the yaml file (i.e. above the stages)? It's not clear from your question why you can't do this.
If you can, something like this:
variables:
var1: newitem
stages:
- stage: one
etc
If that won't work, try these approaches:
https://arunksingh16.medium.com/azure-devops-share-variable-across-stages-9bca85abfe8a
https://stefanstranger.github.io/2019/06/26/PassingVariablesfromStagetoStage/

Using pipeline variables across stages with template jobs

Problem Description
I was having some problems trying to use variables created in one stage in another stage and managed to find various articles old and new describing how this can be done. The more recent articles/posts identifying the new syntax
$[stageDependencies.{stageName}.{jobName}.outputs['{stepName}.{variableName}']
Used like this:
variables:
myVariable: $[stagedependencies.CreateStageVarStage.CreateStageVarJob.outputs['SetValueStep.VariableFromFirstStage']]
This works great until you needed to use job templates.
None of the samples I found online covered the situation of templates. They just demonstrated how multiple stages in the same yaml file could obtain the value.
The syntax depends on being able to put the expression into a variable. Unfortunately, when you use a template for a job, it's not possible to declare variables and passing it as a parameter results in it being unevaluated.
- stage: UseJobTemplateStage
displayName: 'Use Job Template Stage'
dependsOn: CreateStageVarStage
jobs:
- template: templates/job-showstagevars.yml
parameters:
ValueToOutput: $[ stagedependencies.CreateStageVarStage.CreateStageVarJob.outputs['SetValueStep.VariableFromFirstStage'] ]
In this snippet, it comes through as-is. The value does not get substituted in.
Theoretically, you could set your job to have the expression present in the variables block but that sort of hard-coding undermines one of the main benefits of templates.
Related Articles
Share variables across stages in Azure DevOps Pipelines
Azure DevOps Release Notes - sprint 168
I know the question asks about template jobs, but for future reference I want to describe how it can be achieved with template stages as well.
It is done with a variable workaround as in the accepted answer, and with a reference to stagedependencies when no dependsOn exists. (Templates don't allow dependsOn). Somehow, this still works.
Example YAML using stage template (I have modified the code from the accepted answer):
stages:
- stage: CreateStageVarStage
displayName: 'Create StageVar Stage'
jobs:
- job: CreateStageVarJob
displayName: 'Create StageVar Job'
timeoutInMinutes: 5
pool:
name: 'Azure Pipelines'
vmImage: 'windows-2019'
steps:
- checkout: none
- pwsh: |
[string]$message = 'This is the value from the first stage'
Write-Host "Setting output variable 'VariableFromFirstStage' to '$message'"
Write-Output "##vso[task.setvariable variable=VariableFromFirstStage;isOutput=$true]$message"
name: SetValueStep
# stage template cannot use dependsOn, but is still allowed to refer to stagedependencies...
- template: templates/stage-showstagevars.yml
parameters:
ValueToOutput: $[ stagedependencies.CreateStageVarStage.CreateStageVarJob.outputs['SetValueStep.VariableFromFirstStage'] ]
Stage template:
parameters:
- name: ValueToOutput
type: string
stages:
- stage: ShowStageVarStage
variables:
- name: LocalVarOfValueToOutputParam
value: ${{ parameters.ValueToOutput }}
jobs:
- job: ShowStageVarJob
displayName: 'Show stage var'
pool:
name: 'Azure Pipelines'
vmImage: 'windows-2019'
steps:
- checkout: none
- pwsh: |
Write-Host "ValueToOutput parameter=${{ parameters.ValueToOutput }}"
Write-Host "LocalVarOfValueToOutputParam (pre-processor syntax)=${{ variables.LocalVarOfValueToOutputParam }}"
Write-Host "LocalVarOfValueToOutputParam (macro syntax)=$(LocalVarOfValueToOutputParam)"
displayName: 'Show StageVariable'
Solution
The answer isn't actually far away. The original expression just need to be passed through a variable in the template job. Basically, set a variable to be the value of the parameter and use the macro syntax to evaluate the variable.
parameters:
- name: ValueToOutput
type: string
...
variables:
- name: LocalVarOfValueToOutputParam
value: ${{ parameters.ValueToOutput }}
Using the macro syntax of $(LocalVarOfValueToOutputParam) will result in the value making its way into the template job correctly.
Example
If we have a yaml file for the build definition:
stages:
- stage: CreateStageVarStage
displayName: 'Create StageVar Stage'
jobs:
- job: CreateStageVarJob
displayName: 'Create StageVar Job'
timeoutInMinutes: 5
pool:
name: 'Azure Pipelines'
vmImage: 'windows-2019'
steps:
- checkout: none
- pwsh: |
[string]$message = 'This is the value from the first stage'
Write-Host "Setting output variable 'VariableFromFirstStage' to '$message'"
Write-Output "##vso[task.setvariable variable=VariableFromFirstStage;isOutput=$true]$message"
name: SetValueStep
- stage: UseJobTemplateStage
displayName: 'Use Job Template Stage'
dependsOn: CreateStageVarStage
jobs:
- template: templates/job-showstagevars.yml
parameters:
ValueToOutput: $[ stagedependencies.CreateStageVarStage.CreateStageVarJob.outputs['SetValueStep.VariableFromFirstStage'] ]
That uses the job template templates/job-showstagevars.yml
parameters:
- name: ValueToOutput
type: string
jobs:
- job: ShowStageVarJob
displayName: 'Show stage var'
timeoutInMinutes: 5
pool:
name: 'Azure Pipelines'
vmImage: 'windows-2019'
variables:
- name: LocalVarOfValueToOutputParam
value: ${{ parameters.ValueToOutput }}
steps:
- checkout: none
- pwsh: |
Write-Host "ValueToOutput parameter=${{ parameters.ValueToOutput }}"
Write-Host "LocalVarOfValueToOutputParam (pre-processor syntax)=${{ variables.LocalVarOfValueToOutputParam }}"
Write-Host "LocalVarOfValueToOutputParam (macro syntax)=$(LocalVarOfValueToOutputParam)"
displayName: 'Show StageVariable'
What we get in our output of the second stage is:
ValueToOutput parameter=$[ stagedependencies.CreateStageVarStage.CreateStageVarJob.outputs['SetValueStep.VariableFromFirstStage'] ]
LocalVarOfValueToOutputParam (pre-processor syntax)=$[ stagedependencies.CreateStageVarStage.CreateStageVarJob.outputs['SetValueStep.VariableFromFirstStage'] ]
LocalVarOfValueToOutputParam (macro syntax)=This is the value from the first stage

Azure DevOps Passing Variables

we are trying to create a DevOps pipeline where we generate a string in the first stage and we would like to store into a variable which can be used in subsequent stages, or in subsequent tasks within the same stage, is there a way to do this? For clarity the string is generated by querying an external api which returns a string value.
Hope that makes sense 🥳
Thanks in advance
Yes, you shoudl use logging command and mark is as output. Here you have an example
## azure-pipelines.yml
stages:
- stage: one
jobs:
- job: A
steps:
- task: Bash#3
inputs:
filePath: 'script-a.sh'
name: setvar
- bash: |
echo "##vso[task.setvariable variable=skipsubsequent;isOutput=true]true"
name: skipstep
- stage: two
jobs:
- job: B
variables:
- name: StageSauce
value: $[ stageDependencies.one.A.outputs['setvar.sauce'] ]
- name: skipMe
value: $[ stageDependencies.one.A.outputs['skipstep.skipsubsequent'] ]
steps:
- task: Bash#3
inputs:
filePath: 'script-b.sh'
name: fileversion
env:
StageSauce: $(StageSauce) # predefined in variables section
skipMe: $(skipMe) # predefined in variables section
- task: Bash#3
inputs:
targetType: 'inline'
script: |
echo 'Hello inline version'
echo $(skipMe)
echo $(StageSauce)