The multi stage pipeline looks like this.
A->B->C
Stage A consume artifact a
Stage B consume artifact b
Stage C consume artifact c
Artifact can be repository/pipeline...
How to trigger only Stage B when b artifact change ?
It looks that you need kind of trigger per stage which is not possible to achieve at the moment. Please check this topic on developer community
Sorry I should have clarified a bit more. I don't think that triggers will solve all use cases, but having triggers on stages would at least allow you to have the following:
Stage A has trigger /src/appA
Stage B has trigger /src/appB
If you commited (script, code, etc.) to /src/appB, it should use previous artifacts and only build appB and further if requested.
This is mentioned in the comment and left without feedback from MS, however it is not possible with existing Azure DevOps functionalities.
Related
I want to trigger a pipeline only when a new git tag is added, but being sure that the tag is on a specific branch.
As far as I understood the Azure DevOps predefined variables Build.SourceBranch and Build.SourceBranchName alternatively point to the current tag or the current branch depending on what have triggered the pipeline execution.
So I cannot use those variables to understand if the tag has been added within the same branch.
Do you think is it possible, during the pipeline execution, check if we are on a specific tag and branch together?
Does the thing make sense?
Their seems no direct solution for this. According to MS documentation, after specifying branch & tag filter together for trigger in a same pipeline, then the pipeline will trigger if either of the conditions are matched.
To achieve what you are looking for, you can create a custom pre-validation stage.
This stage could contain a PowerShell/Shell script:
to check IF the trigger happened from a specific branch that you need - by using the build.SourceBranch/build.BanchName variable
to check IF the tag is the required tag on that specific branch - tag name can be found by : startsWith(variables['Build.SourceBranch'], 'refs/tags/')
Based on results of the stage above, you can then decide to skip the downstream stages or execute them.
Hope this helps :)
I have a scenario where I have to execute a pipeline from different pipelines to log the validations.
So I'm planning to have a single pipeline for all the validations (like counts, duplicates, count drops etc..) and this pipeline should be trigger when a particular table execution completes.
for example: There are two pipelines P1 & P2 which both invokes this validation pipeline upon completion. so there is a chance that this validation pipeline may trigger twice at same time.
can we run a pipeline like this? is there any lock will applied automatically?
You can reuse a pipeline which acts as generic pipeline in other pipelines and call them parallelly and there is no lock aspect.
Just that make sure the generic pipeline is allowed parallel executions else it would be in queue
Is there a way to
get a particular pipeline's, say P1, status (failed / completed) in conditional component in pipeline P2?
Can we call a pipeline from conditional component?
Usecase:
I have functional pipelines F1, F2, F3 etc and audit pipelines as audit_success and audit_failure. If I can get F3's status in 1 single audit pipeline, I can have 2 branches in same pipeline thereby avoiding creation of 2 pipelines.
There is no conditional component that checks for another pipeline's status. However, you can achieve this through pipeline triggers, but as you mentioned it does require two different pipelines.
My problem is almost same as this question. However, I don't really get the comments or the solutions proposed there.
Say I have two artifacts: A1 and A2.
A1---> Dev stage
A2---> UAT stage.
What I want is that when A1 is released, only Dev stage is deployed. But at the moment, when A1 is rleased, Dev and UAT are triggered.
looking at the comments from previous question,
how can $(RELEASE.TRIGGERINGARTIFACT.ALIAS) be used as a trigger condition? Looking at filter, I can't see a place to put custom condition
If I create a third artifact as suggested, how can I set different tag base on if A1 or A2 is built?
If there other ways to solve this?
Say I have two artifacts: A1 and A2.
Open Dev stage ->click Agent job->expand Additional options and select Custom condition using variable expressions-> add the condition eq(variables['RELEASE.TRIGGERINGARTIFACT.ALIAS'], 'A1'). The Dev stage only runs when the value of RELEASE.TRIGGERINGARTIFACT.ALIAS is A1, otherwise this stage will be skipped.
The same configuration in UAT stage, the condition is eq(variables['RELEASE.TRIGGERINGARTIFACT.ALIAS'], 'A2')
In my release pipeline, I update the Artifact _test to trigger this release pipeline and the variable RELEASE_TRIGGERINGARTIFACT_ALIAS is _test, it runs Stage 1 and skip Stage 2
This might not be the exact answer, but it still can work as a solution to your problem.
You can tag your builds depending on the produced artifact name. If the build produces artifact A1, add a tag A1 to it. That way you can set up release stage conditions based on artifact filters like this:
For the Dev stage trigger, filter build tags by A1 and for the UAT trigger, filter by A2.
We have 100+ services/apps in a repository in Azure Devops. We have defined a single CI/CD YAML multistage pipeline for each (build and deployment). This limits blast radius and allows for auditability of each release of each project. We rely on templates for all the real pipeline work so this is easy to maintain; just a small root azure-pipelines.yml file for each project that includes the needed templates.
Now, we'd like to start using PR validation builds. And, as best as I can tell, we have two options:
Create a separate PR build for for every project and use the UI/API for policies to create 100+ policies
Create a single PR build that has stages for all 100+ projects.
I'm not a fan of the 1st option as now we'll have 200+ builds. The 2nd option is possible, but to avoid a 3 hour PR build, we'd need a way to only run needed stages (aka project builds).
Is there a 3rd option I'm missing? If the 2nd option is our best bet, how do we turn off stages for projects not changed in that PR (i.e. what condition would we use)?
(FYI, our policy is to change only one project per PR, but there are, on occasion exceptions to that.)
For personal suggestion, I also recommend the second method. Though the build script would be very large in one configure file, but much better than have hundreds build configuration files.
But the difficulty is these 100+ apps are all in one repository. This means all the normal method will not suitable for you, include using Build.Repository.Name value as the stage condition. Also, there's no more details which describing the source file path stored in the commit.
So, I suggest you and your team developers input the project name info into your commit message. Then, in the build pipeline you could use the variable Build.SourceVersionMessage to get its comment message. Since this is a environment variable which only work in step level(Not work for stage level and the job level), it needs you add one task in the first step and use the condition for it.
The logic of it is add one step as the first one in every stages. This step is only used to conditional judgment. If the Build.SourceVersionMessage matches the prefix or any key contents words, the jobs will be early-exit.
If use the condition like this:
condition: startsWith(variables['Build.SourceVersionMessage'], '[maven-plugin]')
It needs your commit message must follow a strict content writing format, starting with the specified project name.
Another condition can for you consider is:
condition: in(variables['Build.SourceVersionMessage'], 'maven-plugin')
This does not need the strict content writing format, but also need input the project name in the commit message. Thus it could be evaluated in the job condition with the above script.
Hope it could give you some help.