Is it possible to have a single file azure-pipelines.yaml that can :
Trigger a job A on a push from any branch BUT main
Trigger a job B on PR to main and all subsequent commit on that PR
Trigger a job C when main is merged
I have tried to play arround with trigger, pr keywords and even with condition(), variables['Build.Reason'], or System.PullRequest.TargetBranch but I didn't manage to reach the expected result.
I start thinking it cannot be done with a single file - am I wrong?
You can set conditions on your stages to run depending on a variable but I am not pretty sure this will work with your conditions. Maybe you could also combine some variable values.
For example source branch main and pr is created.
and(eq(variables['Build.Reason'], 'PullRequest'), eq(variables['Build.SourceBranch'], 'refs/heads/main'))
Azure documentation sample:
variables:
isMain: $[eq(variables['Build.SourceBranch'], 'refs/heads/main')]
stages:
- stage: A
jobs:
- job: A1
steps:
- script: echo Hello Stage A!
- stage: B
condition: and(succeeded(), eq(variables.isMain, 'true'))
jobs:
- job: B1
steps:
- script: echo Hello Stage B!
- script: echo $(isMain)
Keep in mind that triggers are appending resources. This means that if you specify a trigger like the below, it will be triggered whether the branch filter is triggered OR the pr is created.
trigger:
branches:
include:
- '*'
pr:
branches:
include:
- current
As you said this can be accomplished for sure with separate files for the pipelines.
https://learn.microsoft.com/en-us/azure/devops/pipelines/process/conditions?view=azure-devops&tabs=yaml
Your first question is possible by using trigger with include and exclude branches as below
trigger:
branches:
include:
- stage
- releases/*
exclude:
- master
Refer CI triggers in Azure Repos Git documentation to have more understanding.
Related
I have one pipeline created with two stages.
Lets say,
Stage A
Stage B
I would like to run Stage A when PR is merged into qa branch and run Stage B when release/* or hotfix/* branch is created.
Can this be achieved in one pipeline? If so how?
You can try to use CI trigger:
trigger:
- qa
- releases/*
- hotfix/*
Then detect them for each stage: Specify conditions
variables:
isQA: $[eq(variables['Build.SourceBranch'], 'refs/heads/qa')]
- stage: B
condition: and(succeeded(), eq(variables.isQA, 'true'))
jobs:
- job: B1
steps:
- script: echo Hello Stage B!
- script: echo $(isQA)
Check additional conditions: contains
I'm trying to set up a pipeline in Azure.
Actual behavior
On pull-request a build validation triggers the pipeline and all stages and jobs are triggered.
After the merge, all jobs are skipped.
Expected behavior
After the merge of the pull request, I would expect stage Bto be triggered
Question
What am I missing so pipeline triggers correctly on merge?
azure.pipelines.yml
trigger:
branches:
include:
- master
stages:
- template: 'main.yml'
main.yml
stages:
- stage: 'A'
condition: startsWith(variables['build.reason'], 'PullRequest')
jobs:
- job: A
steps:
- script: echo A
- stage: 'B'
jobs:
- job: 'B'
steps:
- script: echo B
The trigger feature only works for the whole pipeline and not for an individual stage/job in the pipeline.
Normally, we use the different trigger types (CI/PR, resources) and filters (branches, paths, tags) to define when the pipeline should be triggered.
In the pipeline, we generally specify conditions to a stage, job or step to define when the this stage, job or step should be run or not. The conditions will be verified after the pipeline has been triggered.
To specify conditions, you can use one of the following ways:
use the condition key.
use the if expression.
I found the issue and applied suggestions from #bright-ran-msft
Stage A and B are implicitly linked. Since stage A was not triggered on the merge, stage B would not start either.
Solution
Instead of using condition, it is required to use if
Example
stages:
- ${{ if eq( variables['build.reason'], 'PullRequest') }}:
- stage: 'A'
jobs:
- job: A
steps:
- script: echo A
- stage: 'B'
jobs:
- job: 'B'
steps:
- script: echo B
When a pipeline is triggered by a repository resource trigger I need to be able to determine metadata about the triggering repository (such as repo name and branch).
It sounds like these variables should be set during a pipeline run that was triggered by a repository resource; however, their values are blank when I do echo $(resources.triggeringAlias) or env | sort in a pipeline run that was triggered by a repository resource.
UPDATE: the predefined variables for Build.Repository.Name and Build.SourceBranchName now work as expected when used with a repository trigger. So, while I'm no longer in need of resources.triggeringAlias and resources.triggeringCategory, they still aren't working.
I need to be able to determine metadata about the triggering
repository (such as repo name and branch)
We can use $(Build.Repository.Name) and $(Build.Repository.Uri) to get repo name and repo uri.
And $(Build.SourceVersion) can be used to get CommitId, $(Build.SourceBranch) or $(Build.SourceBranchName) can be used to get branch info.
Just note we must also checkout the triggering repo to make above variables work to fetch the info of triggering repo, otherwise those variables will always represents the value of triggered repo:
steps:
- checkout: self
- checkout: TheTriggeringRepo
Some details:
I have a triggering repo PipelineA, and the triggered repo PipelineB. PipelineB's azure-pipelines.yml file:
resources:
repositories:
- repository: PipelineA
type: git
name: PipelineA
trigger:
- master
steps:
- checkout: self
- checkout: PipelineA
- task: Bash#3
inputs:
targetType: 'inline'
script: |
echo $(Build.Repository.Name)
echo $(Build.SourceBranch)
So this pipeline will be triggered by both PipelineA repo and PipelineB repo.
When PipelineB repo has changes:
When PipelineA repo has changes:
It's clear the $(Build.Repository.Name) variable can work well to output the real trigger repo if we checkout both these two repos. So just make sure you checkout those triggering repos, then my variables above would work for you.
resources:
pipelines:
- pipeline: client-qa-deployment ## Resources.TriggeringAlias will be 'client-qa-deployment' if this resource gets triggered.
source: benjose.dev-code
trigger:
stages:
- client_deploy_qa
- stage: qa_smoketest
displayName: Smoke Test on QA Environment
condition: eq(variables['Resources.TriggeringAlias'], 'client-qa-deployment')
pool:
vmImage: 'windows-latest'
jobs:
- job: TestQA
steps:
- script: echo Hello, $(Resources.TriggeringAlias)!
displayName: 'Running Smoke Test on QA Environment'
Hello, client-staging-deployment!
I believe the above snippet will help you to understand the syntax and usage of Resources.TriggeringAlias. This variable will be helpful if we have multiple resources in the same pipeline and to find out which resource got trigged.
Pushing in the dev branch triggers a build.
But I want only to create a build if the commit in the dev branch has a tag with a format of '?.??.??.?'.
trigger:
- dev
In the documentation I can only find how to trigger based on a tag OR an branch. How can I have two conditions the tag and the commit in the specific branch?
As a workaround you can use multiple jobs(if your pipeline has multiple jobs you can make use of multiple stages) in your pipeline to achieve above requirement.
Add an additional job(or stage)(ie. Job A) to get the tags associated to the commit. And then use script check if the tag has a certain format, if not then fail the job. Then and set the other job dependsOn this job. Check here for more information about dependenies and conditions
So that your job(ie.Job B) which builds your project will only get executed if the additional job succeeded. Please check below example yaml:
trigger:
branches:
include:
- dev
pool:
vmImage: "ubuntu-latest"
jobs:
- job: A
steps:
- powershell: |
#get the tags associated to the commit
$tags = git describe --exact-match $(Build.SourceVersion)
# check the tag format
if($($tags) -notmatch '.\...\...\..'){exit 1}
- job: B
dependsOn: A
condition: succeeded('A')
steps:
- powershell: |
$tags = git describe --exact-match $(Build.SourceVersion)
echo $($tags)
I am trying to set Azure DevOps to skip a stage on a multi-stage pipeline if a message does not start with a given text.
From the examples documentation, I think it is just
- stage: t1
condition: not(startsWith(variables['Build.SourceVersionMessage'], '[maven-release-plugin]'))
jobs:
- job: ReleasePrepare
displayName: Prepare release
pool:
vmImage: 'ubuntu-16.04'
steps:
- script: |
env | sort
However, this gets executed regardless. Here's an example of where I expect the t1 task to not be run based on the commit message https://dev.azure.com/trajano/experiments/_build/results?buildId=110&view=results
The output of env shows that the message was passed in correctly
Just in case it is a bug I reported it here as well https://developercommunity.visualstudio.com/content/problem/697290/startswith-buildsourceversionmessage-variable-not.html
It appears that Build.SourceVersionMessage at the time of this post is only resolvable on the steps.
Here's a working example that stores the value in a variable in one step and use it in the next job (which can be a deployment)
trigger:
batch: true
branches:
include:
- master
stages:
- stage: ci
displayName: Continuous Integration
jobs:
- job: Build
pool:
vmImage: 'ubuntu-16.04'
steps:
- script: |
env | sort
echo "$(Build.SourceVersionMessage)"
- stage: t1
displayName: Release
condition: eq(variables['Build.SourceBranch'],'refs/heads/master')
jobs:
- job: GetCommitMessage
displayName: Get commit message
steps:
- bash: |
echo "##vso[task.setvariable variable=commitMessage;isOutput=true]$(Build.SourceVersionMessage)"
echo "Message is '$(Build.SourceVersionMessage)''"
name: SetVarStep
displayName: Store commit message in variable
- job: ReleasePrepare
displayName: Prepare release
dependsOn: GetCommitMessage
pool:
vmImage: 'ubuntu-16.04'
condition: not(startsWith(dependencies.GetCommitMessage.outputs['SetVarStep.commitMessage'], '[maven-release-plugin]'))
steps:
- script: |
echo this would be a candidate for release
env | sort
displayName: Don't do it if maven release
- job: NotReleasePrepare
displayName: Don't Prepare Release
dependsOn: GetCommitMessage
pool:
vmImage: 'ubuntu-16.04'
condition: startsWith(dependencies.GetCommitMessage.outputs['SetVarStep.commitMessage'], '[maven-release-plugin]')
steps:
- script: |
echo this not be a candidate for release because it was created by the plugin
env | sort
condition: startsWith(variables.commitMessage, '[maven-release-plugin]')
displayName: Do it if maven release
The build can be found in https://dev.azure.com/trajano/experiments/_build/results?buildId=133&view=logs&s=6fc7e65a-555d-5fab-c78f-9502ae9436c4&j=b5187b8c-216e-5267-fcdb-c2c33d846e05
I am trying to set Azure DevOps to skip a stage if a message does not
start with a given text.
If I am not misunderstand, the condition you want is if the message match the start with maven-release-plugin, the current stage will be queued.
If this, the condition you write is not correct, I think you should specified it:
startsWith(variables['Build.SourceVersionMessage'], '[maven-release-plugin]')
As I tested on my pipeline:
And in fact, the value of this variable is Deleted 121321. Here is the result:
As you can see, that it is successful to skip the stage. My logic is, the value of Build.SourceVersionMessage should start with othermessage. But in fact, in my pipeline, it's value is Deleted 121321. Not match, so skip this stage.
(Delete 121321 is not only my PR name, I just set the commit message as the default PR name.)
Updated 2:
Though my test logic is not incorrect, but after I reproduced with YAML and many other method tested, such as use Build.SourceVersion which can only got after the source pulled.
Yes, you are right about that Build.SourceVersionMessage does not has value in Job level. As my tested, it indeed null in job level.
But, unfortunately, this is not a bug. This is as designed in fact.
We can think that the source repos be pulled locally only the stage job begin to execute, right? You can see the Checkout log, it record the process about pull source file down.
If the stage does not be executed, the source will not be pulled down. But also, if no source pulled, the server will also could not get the value of Build.SourceVersionMessage because of no source history. That's why I also tested with the variable Build.SourceVersion in the Job level.
We can not use these two variable at the agent job level because it hasn't pulled sources yet so Build.SourceVersionMessage is null. You'll need to copy it to each step in your pipeline. This is what confirmed by our Product Group team.
But, I still need to say sorry. Sorry about our doc is not too clear to announce that this could not used in agent job level.