I have below main YML pipeline
And below is the template that is being called.
When trying to run the main pipeline is showing the error per below
Agree with Krzysztof Madej.
When you add condition to the stages level, you need to make sure that there is at least one stage that meets the condition at all times.
In addition to adding an empty stage, you could also add the negative condition in the Yaml sample:
For example:
stages:
- ${{if eq(variables['envName'],'sbx')}}:
- template: test.yml
parameters:
buildSteps: test
- ${{if ne(variables['envName'],'sbx')}}:
- stage: Test
jobs:
- job: TestJob
steps:
- script: echo Test
displayName: 'Test Stage'
In this case, if the envName = sbx, it will run the template, or it will run another stage.
It looks that your condition is not met. So envname is not sbx and thus your template is skipped, and you can't run pipeline without any stage. Please make sure you have always at least one stage regardless of the result of the condition, or sth like empty stage for non sbx env.
Related
Is it possible to run the first two stages parallelly in Azure DevOps pipelines? By default, Each stage starts only after the preceding stage is complete unless otherwise specified via the dependsOn property.
Current situation is:
I would like to run both the stages, iOS_Dev_Build and iOS_QA_Build parallelly. No dependsOn condition is added for iOS_QA_Build. But by default, it is waiting for the iOS_Dev_Build stage to complete before it starts
Please add dependsOn: [] in the iOS_QA_Build stage.
My example:
stages:
- stage: DEV
jobs:
- job: A
steps:
- bash: echo "A"
- stage: QA
dependsOn: []
jobs:
- job: A
steps:
- bash: echo "A"
When you define multiple stages in a pipeline, by default, they run sequentially in the order in which you define them in the YAML file. dependsOn: [] removes the implicit dependency on previous stage and causes this to run in parallel.
For more details, please refer Specify dependencies.
How to create a multi-stage pipeline depending on a stage/job-name derived from a parameter whereas stages run firstly in parallel and eventually one stage that waits for all previous stages?
Here's what I've tried so far:
A multi-stage pipeline runs for several stages depending on a tool parameter in parallel, whereas dependsOn is passed as parameter. Running it in parallel for each tool waiting for the previous stage for the said tool works smoothly.
Main template: all wait for for all
- ${{ each tool in parameters.Tools }}:
- template: ../stages/all-wait-for-all.yml
parameters:
Tool: ${{ tool }}
stages/all-wait-for-all.yml
parameters:
- name: Tool
type: string
stages:
- stage: ALL_WAIT_${{ parameters.Tool}}
dependsOn:
- PREPARE_STAGE
- OTHER_TEMPLATE_EXECUTED_FOR_ALL_TOOLS_${{ parameters.Tool }}
Now there should be one stage that should only run once and not per tool, but it should only run after the individual tool stages are done. It can't be hardcoded as there are various tools. So I hoped defining the individual wait-stages in a prepare job would work out:
Main template: prepare-stage
- script: |
toolJson=$(echo '${{ convertToJson(parameters.Tools) }}')
tools=$(echo "$toolJson" | jq '.[]' | xargs)
stage="ALL_WAIT"
for tool in $tools; do
stageName="${stage}_${tool }"
stageWaitArray+=($stageName)
done
echo "##vso[task.setvariable variable=WAIT_ON_STAGES]${stageWaitArray}"
echo "##vso[task.setvariable variable=WAIT_ON_STAGES;isOutput=true]${stageWaitArray}"
displayName: "Define wait stages"
name: WaitStage
stages/one-waits-for-all.yml
stages:
- stage: ONE_WAITS
dependsOn:
- $[ stageDependencies.PREPARE_STAGE.PREPARE_JOB.outputs['waitStage.WAIT_ON_STAGES'] ]
whereas below error is shown:
Stage ONE_WAITS depends on unknown stage $[ stageDependencies.PREPARE_STAGE.PREPARE_JOB.outputs['WaitStage.WAIT_ON_STAGES'] ].
As I understand depends on can not have dynamic $[] or macro $() expressions evaluated at runtime. You can use template expressions ${{}} which are evaluated at queue time.
Guess I was overthinking the solution as eventually it was pretty obvious.
So first template can be called within a loop from the main template whereas it's executed as many times as tools we got. Second template shall be called once waiting on previous stages for all tools, whereas the job/stage prefix is known, only the tool name as postfix was unknown. So just add them in a loop directly in dependsOn..
Here you go:
stages:
- stage: ONE_WAITS
dependsOn:
- PREPARE_STAGE
- ${{ each tool in parameters.Tools }}:
- OTHER_TEMPLATE_EXECUTED_FOR_ALL_TOOLS_${{ tool}}
- ALL_WAIT_${{ tool }}
I have an Azure Pipeline azure-pipelines.yml:
parameters:
- name: "stages"
type: object
default:
- stage1
- stage2
# possibly more...
pool:
vmImage: 'ubuntu-latest'
stages:
- stage: InitialStage
displayName: I must run first
jobs: ...
- ${{ each stage in parameters.stages }}:
- stage:
dependsOn: InitialStage
condition: succeeded()
jobs: ...
What I want to achieve is creating a pipeline where the first stage is InitialStage, then I want to generate the next stages to run after it sequentially. So the final pipeline should look like:
InitialStage ==> stage1 ==> stage2 ==> ...
One stage must run after the previous completed and the first stage in absolute must be InitialStage.
If I use the syntax above, I get the stages run in parallel like:
InitialStage ==> stage1
|
==> stage2
|
...
How can I achieve that?
Just need to remove the dependsOn keyword. Per Microsoft Docs:
When you define multiple stages in a pipeline, by default, they run sequentially in the order in which you define them in the YAML file. The exception to this is when you add dependencies. With dependencies, stages run in the order of the dependsOn requirements.
Can also remove the condition keyword:
You can specify the conditions under which each stage runs. By default, a stage runs if it does not depend on any other stage, or if all of the stages that it depends on have completed and succeeded. You can customize this behavior by forcing a stage to run even if a previous stage fails or by specifying a custom condition.
If you do need to map custom stage dependencies, I have done this before as well by creating a stage object. Something like:
parameters:
- name: environmentObjects
type: object
default:
- environmentName: 'dev'
dependsOn: ''
- environmentName: 'tst'
dependsOn: 'dev'
and then looped through and dynamically build the dependsOn. This would allow for scenarios with two test environmnts depending on dev but not each other.
I'm porting one of our release pipelines from ADO classic to YAML.
Neither classic nor YAML allow you to select jobs/tasks while creating the run.
At least in classic, once the release is created, you can disable a stage's jobs/tasks for that specific release before triggering the stage.
In YAML, however, only stages are selectable on the run creation screen. But after the run is started, I can't find a way to disable tasks and/or jobs like I can in classic.
Oddly enough, after a YAML stage runs there's an option to "Rerun failed jobs."
So my question is, is it possible to enable/disable jobs and/or tasks when creating a new YAML pipeline run (or after creating, but before triggering a stage)?
After creating the pipeline, I am afraid that there is no existing option can cancel a singal stage before triggering.
For a workaround, you can create Pipeline environment and add the Approval check. Then you can use the Environment in your stage.
In this case, before running this stage, you can stop the running of this stage by denying approval.
enable/disable jobs and/or tasks when creating a new YAML pipeline run
To achieve this requirement, you can use the If Expression and Runtime Parameters in Yaml Pipeline.
Here is an example:
parameters:
- name: test1
type: string
default: false
values:
- true
- false
pool:
vmImage: ubuntu-latest
stages:
- stage: testA
jobs:
- ${{ if eq(parameters['test1'], 'true' ) }}:
- job: test1
steps:
- script: echo 1
- job: test2
steps:
- script: echo 1
- stage: testB
jobs:
- job: test3
steps:
- script: echo 1
When you run the Pipeline, you can select the value to run the jobs/task.
Is it possible to use the build tags set on a multi-stage pipeline build, in the condition section of a later stage?
##### task in build stage #####
- task: YodLabs.VariableTasks.AddTag.AddTag#0
displayName: Adding environment tag to build
inputs:
tags: |
deploy
$(DEPLOY_ENV)
#### some later stage ####
- stage: deploy
displayName: deploy
condition: |
and(
succeeded(),
#Is there something I can put here to condition on tags
)
jobs:
Thanks
From what I know this is not possible with YAML yet, because there is no easy way to get tags availabe in YAML. What you can try is output variable
jobs:
- job: Foo
steps:
- script: |
echo "This is job Foo."
echo "##vso[task.setvariable variable=doThing;isOutput=true]Yes" #The variable doThing is set to true
name: DetermineResult
- job: Bar
dependsOn: Foo
condition: eq(dependencies.Foo.outputs['DetermineResult.doThing'], 'Yes') #map doThing and check if true
steps:
- script: echo "Job Foo ran and doThing is true."
You can also try with workaraund which is in this case:
Fetch tags using REST API in powershell script PUT https://dev.azure.com/{organization}/{project}/_apis/build/builds/{buildId}/tags/{tag}?api-version=5.1
Then assign your tags to output variables using logging commands
And finally use output variable in condition
EDIT
So it looks that this is not possible at the moment but should be available soon. Please check this GitHub issue.
Output variables may now be used across stages in a YAML-based
pipeline. This helps you pass useful information, such as a go/no-go
decision or the ID of a generated output, from one stage to the next.
The result (status) of a previous stage and its jobs is also
available.
Output variables are still produced by steps inside of jobs. Instead
of referring to dependencies.jobName.outputs['stepName.variableName'],
stages refer to
stageDependencies.stageName.jobName.outputs['stepName.variableName'].
Note: by default, each stage in a pipeline depends on the one just
before it in the YAML file. Therefore, each stage can use output
variables from the prior stage. You can alter the dependency graph,
which will also alter which output variables are available. For
instance, if stage 3 needs a variable from stage 1, it will need to
declare an explicit dependency on stage 1.
I tried it now:
stages:
- stage: A
jobs:
- job: JA
steps:
- script: |
echo "This is job Foo."
echo "##vso[task.setvariable variable=doThing;isOutput=true]Yes" #The variable doThing is set to true
name: DetermineResult
# stage B runs if A fails
- stage: B
condition: eq(stageDependencies.A.JA.outputs['DetermineResult.doThing'], 'Yes') #map doThing and check if true
jobs:
- job: JB
steps:
- bash: echo "Hello world stage B first job"
but I got this error:
An error occurred while loading the YAML build pipeline. Unrecognized
value: 'stageDependencies'. Located at position 4 within expression:
eq(stageDependencies.A.JA.outputs['DetermineResult.doThing'], 'Yes').
For more help, refer to https://go.microsoft.com/fwlink/?linkid=842996
However, this feature can be soon with us!