Quite new to the CI/CD pipelines in AzDO. I was going through some official Azure Pipeline documents where I have some doubts on Schedule Triggers.
Below is a pipeline Snippet in main branch:
schedules:
- cron: "0 0 * * *"
displayName: Daily midnight build
branches:
include:
- main
- releases/*
exclude:
- releases/ancient/*
- cron: "0 12 * * 0"
displayName: Weekly Sunday build
branches:
include:
- releases/*
always: true
The documentation says the Pipeline will run for branches "main" and "releases" at midnight if there has been some changes to those branches since the last successful scheduled run and build the "releases" branch on sunday irrespective of changes to releases branch which seems understandable.
Which means we can control other branches (e.g. releases) to build from YAML file present in another branch (e.g. main).
Again, the documentation also states, for below YAML in a release branch
# YAML file in the release branch
schedules:
- cron: "0 0 * * *"
displayName: Daily midnight build
branches:
include:
- main
The pipeline won't build "release" branch since the branch name is missing under "branches" section. But will it build the "main" branch though since it's mentioned under "branches" section?
If no, then how can the first YAML in main branch make "releases" branches to build? Does the YAML file in main/default branch has some special capabilities?
If yes, does it really make sense to build "main" branch from release/non-main branches?
Thanks in advance.
If no, then how can the first YAML in main branch make "releases"
branches to build? Does the YAML file in main/default branch has some
special capabilities?
There's a setting for default branch in Azure Pipelines (Edit pipeline -> get sources -> Default branch for manual and scheduled builds) which tells the pipeline which branch to evaluate for scheduled runs. Yaml builds use the same scheduling that classic pipelines do and a changes to the yaml-file in that default branch are reflected in the scheduler.
Or to put it in another way, a pipeline in Azure DevOps is a separate thing from the yaml-file in repository. After creating a yaml-file in repo, you need to create a pipeline in Azure DevOps, via new pipeline -> select repo -> existing Azure Pipelines yaml file (or create a new yaml file). This step creates the actual pipeline, which has it's own configurations that are stored somewhere in Azure DevOps database. Those configurations point to the yaml file that is evaluated for pipeline runs and when the file is changed.
If you queue a pipeline manually, and select some other branch than the default, the yaml file in the branch that you selected is evaluated. Also, when evaluating branch triggers, pipeline evaluates the yaml file in the branch commit is made to. So, if you have this trigger in main branch:
trigger:
- main
- feature/*
And this in feature/foo -branch:
trigger:
- main
A commit to the feature/foo branch does not trigger the pipeline.
(Not sure if I answered the question clearly or just added some confusion, but there you go. The trigger implementation in Azure Pipelines is a bit confusing at first.)
Related
In Azure Devops, when i view a branch Repos->Branches->Select a branch i was able to click Set up build.
However, i am not able to choose an existing pipeline there.
When i select a pipeline Pipeline->Edit->Triggers i can add Branch filters aswell as path filters, however those do not take effect.
I tried to add filters for release/my-release aswell as release/* or a path-filter release.
I want to be able to start a pipeline from the Branch overview. What do i have to do?
If you're using YAML pipelines, e.g. azure-pipeline.yml, make sure this file is also available in your branch.
If the pipeline YAML was added to main after you've branched the release branches, the build will never trigger. Because the branch doesn't have the pipeline YAML .
Furthermore the triggers of the YAML should be specified in the YAML instead of Pipeline->Edit->Triggers.
Example:
trigger:
branches:
include:
- master
- release/*
Don't forget to turn off the "Override the YAML continuous integration trigger from here" Option:
I have an Azure DevOps system up and running, with triggers to run the pipeline due to PRs as well as a nightly schedule, like so:
trigger:
batch: true
branches:
include:
- main
- release/*
- pre-release/*
schedules:
- cron: "0 0 * * *"
displayName: Daily midnight build
branches:
include:
#- main
#- release/*
- pre-release/*
always: false
Is there a way to let the job know that's being called from a Schedule rather than a PR? I'd like to add another series of more rigorous tests, but just with nightlies, as part of the usual job.
Perhaps setting a variable that can be checked later in the pipeline.
You could use the Build.Reason predefined variable.
In your case checking if $(Build.Reason) equals Schedule should work.
Build.Reason. The event that caused the build to run.
Manual: A user manually queued the build.
IndividualCI: Continuous integration (CI) triggered by a Git push or a TFVC check-in.
BatchedCI: Continuous integration (CI) triggered by a Git push or a TFVC check-in, and the Batch changes was selected.
Schedule: Scheduled trigger.
ValidateShelvese: A user manually queued the build of a specific TFVC shelveset.
CheckInShelveset: Gated check-in trigger.
PullRequest: The build was triggered by a Git branch policy that requires a build.
ResourceTrigger: The build was triggered by a resource trigger or it was triggered by another build.
I have a YAML build pipeline which resides on the repo PipelineRepo, branch master.
It contains a reference to a second repo AppRepo with a trigger on branch dev.
When a commit is made on AppRepo/dev, the build triggers and produces an artifact. The Build.SourceBranch predefined variable available during the build is of course dev as this was the triggering branch on AppRepo. This is as per the docs here:
https://learn.microsoft.com/en-us/azure/devops/pipelines/repos/multi-repo-checkout?view=azure-devops#triggers
The specific part of the docs is this:
When an update to one of the repositories triggers a pipeline, then
the following variables are set based on triggering repository:
Build.Repository.ID
Build.Repository.Name
Build.Repository.Provider
Build.Repository.Uri
Build.SourceBranch
Build.SourceBranchName
Build.SourceVersion
Build.SourceVersionMessage
I consume the artifact in a Release pipeline, where I have a "Dev" stage with an artifact filter for branch dev:
I have a continuous deployment trigger on the artifact for branch dev:
Now we come to the problem
Every time a new build artifact is produced from the AppRepo/dev branch, the continuous deployment trigger doesn't fire because it thinks the build branch of the artifact was PipelineRepo/master. This is also true of the artifact filter on the stage - I have tested it by changing the continuous deployment trigger to master.
Note: you can see on the screenshot the build name contains the word "dev". This is because I use the Build.SourceBranch var in my custom build name. This proves the artifact is definitely produced by the AppRepo/dev triggering branch.
How can I get the DevOps Release Pipeline to pick up the triggering branch?
Based on your description, I could reproduce the similar issue in my organization.
When the pipeline is triggered by another repo branch, it still show the master branch in Release artifacts.
At the same time, the display of the trigger branch is also inconsistent in Pipelines.
For example:
Overview:
Detail view:
I suggest that you could create a feedback ticket in Our feedback Site.
For a workaround:
You could add a build tag in Pipeline to distinguish artifacts.
Yaml sample:
resources:
repositories:
- repository: test
type: git
name: 123/ARM
trigger:
- test
steps:
- checkout: test
- script: echo "##vso[build.addbuildtag]$(Build.SourceBranch)"
Release Pipeline:
You could set the master branch and add the build tag.
I have a single project but with two "master" branches.
master
virt/master
Each of them would have their own azure-pipeline.yml specific for their branch.
The first pipeline in master has the trigger set up as
trigger:
batch: true
branches:
include:
- refs/heads/master
The second one is in the virt/master branch.
trigger:
batch: true
branches:
include:
- refs/heads/virt/master
Here's the repository that I am experimenting on https://dev.azure.com/trajano/experiments/_git/multi-branch
master build https://dev.azure.com/trajano/experiments/_build?definitionId=11
virt/master build https://dev.azure.com/trajano/experiments/_build?definitionId=12
The problem I am having is when I push a change to the virt/master branch both pipelines get executed
Am I missing something in my configuration? Or is this a bug on Azure Devops?
I also tried to exclude but to no avail.
trigger:
batch: true
branches:
include:
- refs/heads/master
exclude:
- refs/heads/virt/master
If you want to have separate pipelines please create separate file definition for them. I think that your configuration is fine and the issue is that you share the same file as definition.
When I moved to separate file it works as expected:
To create different pipelines for different branches. You need to rename the azure-pipelines.yml file in virt/master branch or create a new yml file with the some contents and with a different name. And create pipeline multi-branch(virt) from this new yml file.
If both pipelines are created from the yaml file with the same name azure-pipeline.yml. And the azure-pipeline.yml file exists in both of the branches. Then they are identical pipelines(even though the azure-pipeline.yml file contents might be different).
You can see from above screen. Pipeline multi-branch and multi-branch(virt) were building the same virt/master branch(using the tasks in the azure-pipeline.yml of virt/master). If you push to master branch. You will see both pipelines will be triggered to build master branch(using the tasks in the azure-pipeline.yml of master). Pipeline multi-branch and multi-branch(virt) are one pipeline
See this thread for more information.
When a pipeline is created it must be specified its file path and branch. When this pipeline runs a branch is asked again. What is the purpose of this second branch? My initial thought was that the branch where my code is located and the branch where the Yaml file is located are two different things.
However, it seems, after some issues, that when I select a branch in the Run pipeline dialog both the Yaml file and the source code are checked out from this same branch. Is it correct?
Thanks for any help.
This is the expected behavior.
In Azure Devops, the location of the Yaml file is in the branch of the repo, so thehe executed yaml file and the source branch are synchronized.
The first operation of selecting a branch is used to create the pipeline, if you run directly for the first time, you don’t need to choose the execution branch.
When you run a DevOps pipeline(select execution branch), you choose a branch to execute the pipeline. The Yaml file in that branch is the one that will be executed by default and the pipeline will checkout the source code from the same branch.
The version that gets executed will be determined by which branch you're running the pipeline for.
If you want to checkout the source files of other branches in a yaml file, you need to add an additional repo source.
For example:
resources:
repositories:
- repository: TestRepo
ref: refs/heads/branchname
type: git
name: Projectname/RepoName
stages:
- stage: deploy
jobs:
- job: test
steps:
- checkout: TestRepo