Problem
Azure DevOps has a feature (documented here) to trigger a pipeline on completion from another pipeline.
This works fine in a test organization, but it won't work in our main organization.
There could be something on the organization, project, repository or even branching level, but I'm currently stuck and any help would be appreciated!
Pipelines
Pipeline Pipeline B should run automatically when pipeline Pipeline A completes.
File pipeline-a.yaml for Pipeline A:
pool:
vmImage: 'ubuntu-latest'
steps:
- script: echo Hello, world!
displayName: 'Do something'
File pipeline-b.yaml for Pipeline B:
trigger: none
pool:
vmImage: 'ubuntu-latest'
resources:
pipelines:
- pipeline: pipeline-a
source: 'Pipeline A'
branch: master
trigger:
branches:
- master
steps:
- script: echo Hello, world!
displayName: 'Do something'
Organizations
In my test organization the above pipelines run like a charm. This means that Pipeline A runs on a commit, and after completion, Pipeline B runs automatically.
Yet in our production organization, Pipeline B does not run automatically.
Discovery
Both pipelines run fine when started manually, in both organizations
All Preview features are equal on organization and personal level for both organizations, including the Multi-stage pipelines feature.
The production organization has branch policies on master, while the test organization does not have policies. I don't see a connection with pipeline triggers and did not investigate this.
Installing extensions to have them equal on test and production does not make a difference.
The test organization seems to be in the slow ring and was still on Sprint 161. EDIT: The issue persists after the organization was updated to Sprint 162.
It works when I use the classic editor and manually create a build completion trigger. But this overrides the YAML pipeline trigger and I don't want to do this (I want to generate the pipeline and it's triggers)
Deleting and re-adding the pipeline did the trick. So keep the YAML file but delete the pipeline and add it again.
The Azure DevOps backend seems to miss a relationship between pipelines now and then.
We troubleshot a similar problem today. With Pipeline-A defined as a resource that is meant to be consumed by Pipeline-B.
The consuming pipeline was never being triggered. Deleting and recreating the pipeline did not work for us. This work\pipeline was net new and on a feature branch. That ended up being important.
The ultimate fix was defining that feature branch as the Default branch for manual and scheduled builds in Pipeline-B. You can find that setting tucked away in Pipeline -> Edit -> triggers -> yaml-> Get Sources. Expect that as we promote this code to the main branch we will need to update the setting.
So it seems like the Default branch for manual and scheduled builds would be better named
Default branch for manual and scheduled builds and Pipeline Completion Triggers
In my case it was as simple as the source pipeline completing with an error. Testing with a very simple non erroring pipeline and the code worked fine.
Related
trigger:
branches:
include:
- master
tags:
include:
- Showcase
pr:
branches:
include:
- master
the code above is from my yaml azure pipeline with using bitbucket repo. Anytime I do a PR in my bitbucket repository I expect build pipeline to run automatically in azure devops but it's not the case. Currently, I have to run the pipeline manually to complete PR validation. creating pipeline with the same line of code under another repo works correctly. I'm not sure what the problem is.
I created anoter dummy pipeline with a new project, the pr triggered build automatically when pr to the the master. the webhohook seems to be working fine. the trigger option doesn't override the yaml pipeline. I can't think of anything else to check.
Like many before me I'm struggling hard with configuring pipeline triggers in Azure DevOps.
Background:
single project in the Organization
three branches: main, Infrastructure, Application
The branches are kind of independent of each other. They are never merged into main either.
I have a pipeline which deploys two App Services. The YAML file for this pipeline is in the Infrastructure branch. The Default branch for manual and scheduled builds is set to Infrastructure.
Then I have 2 pipelines, each to deploy a different App to the App Service. The YAMLs for those pipelines are in the Application branch. The Default branch for manual and scheduled builds is set to Application.
By themselves, the pipelines work perfectly fine. However what I am trying to achieve is to trigger the App pipelines after the App Service pipeline finishes. And no matter what combination of settings I try, I can't get it to work.
This is currently how it looks like in the n-th version of the YAML:
name: 'deploy-webapp-002'
pool:
vmImage: windows-latest
resources:
pipelines:
- pipeline: 'Deploy App Services' # Internal name of the source pipeline, used elsewhere within this YAML
# e.g. to reference published artifacts
source: deploy-appservices # Azure Pipelines name of the source pipeline referenced
project: HomeLab # Required only if the source pipeline is in another project
trigger:
branches:
include:
- Infrastructure
- Application
pr: none
trigger: none
Is it even possible to do what I'm trying to do?
If yes, what settings should be specified in the Resources/Pipelines section in the YAML, and how should the Default branch for manual and scheduled builds look like for each of those pipelines?
I can reproduce the same issue when I put the YAML files separately in two branches and set the default branch.
Refer to this doc: Define a pipelines resource
When you define a resource trigger, if its pipeline resource is from the same repository (say self) as the current pipeline, triggering follows the same branch and commit on which the event is raised. But, if the pipeline resource is from a different repository, the current pipeline triggers on the default branch of the self repository.
In your case， you are using the same repo. So triggering follows the same branch and commit on which the event is raised.
To solve this issue, you need to copy the YAML file in Application Branch to Infrastructure Branch.
On the other hand, you can also try to set Build completion trigger manually on UI.
For example:
I have some Azure YAML pipelines that I need to trigger in a chain. I am finding that the documented approach does not seem to work. I have tried various things recommended in other SO posts: Azure Pipeline to trigger Pipeline using YAML and Azure DevOps unable to trigger yaml pipeline off of completed build.
I also know they had open bugs for some of this so not sure if I am missing something or if the feature is broken now?
For sake of brevity, I have two pipelines named "MyProject.Common" and "MyProject.Foo". I want Foo to run after Common succeeds.
A snippet of my yaml is below. Note that here are some things I have already tried:
Not including the first "trigger: none" line.
Including "trigger: true" after the "source:" bit, instead of the "branches: include:... piece".
Tried is with "branches: include: - '*' "
And various other things. MyProject.Foo is just not being triggered.
I have also verified that there are no CI triggers under the Edit -> Triggers menu.
Have also verifies that MyProject.Common is publishing a build artifact.
Any ideas? If this is a broken feature, what would be a good workaround?
Snippet:
trigger: none
resources:
pipelines:
- pipeline: 'MyProject.Common'
source: 'MyProject.Common'
trigger:
branches:
include:
- main
- refs/head/main
Branch triggers and Pipeline resource triggers are very similar but have one very big difference - how and where the trigger is evaluated.
Branch triggers get evaluated for every branch.
Pipeline resource triggers get evaluated ONLY On the pipeline Default branch.
from the docs:
Pipeline completion triggers use the Default branch for manual and scheduled builds setting to determine which branch's version of a YAML pipeline's branch filters to evaluate when determining whether to run a pipeline as the result of another pipeline completing. By default this setting points to the default branch of the repository.
source
What does this mean?
Your pipeline resource trigger yaml changes need to be in default branch of the pipeline that you want to be triggered. You can do this by merging your changes into the default branch, or changing the default branch to your dev/feature branch by following these instructions.
I am not at all sure why, but I deleted the dependent pipeline and created a new one (referencing the original yaml file) and it then triggered properly.
I can only guess that Azure lost some type of reference behind the scenes as the YAML for the trigger was accurate.
Thanks for all the input. It was helpful even though it didn't solve the main issue.
I have a release pipeline with the usual stages
Dev -> Test -> Pre Production -> Production
I would like to know if there is a way to ensure that only the master branch can be promoted all the way to production to avoid releases from feature and develop branches ending on a production environment.
YAML Pipelines
If you're using a YAML-based pipeline, you can use the Checks + Approval Gates feature to add a branch policy check.
Create an Environment for each deployment environment
Structure your YAML pipeline to use a Deployment Job that specifies the Environment
stages:
- stage: build
- stage: dev
- stage: test
- stage: preprod
- stage: prod
You can put conditions on the stages to ensure the branch conditions are met.
- stage: production
dependsOn: preprod
condition: |
and(
succeeded('preprod'),
or(
eq(variables['Build.SourceBranch'], 'refs/heads/main'),
startsWith(variables['Build.SourceBranch'], 'refs/heads/release/'),
startsWith(variables['Build.SourceBranch'], 'refs/heads/hotfix/')
)
)
Ensure each stage that performs the deployment activities has a Deployment Job:
- deployment: deploy
displayName: 'Deploy'
environment: 'Production'
strategy:
runonce:
deploy:
steps:
- pwsh: Write-Host 'Deployment Steps go here!'
In your Production Environment, add the Branch Control check:
Classic Release Pipelines
If you're using a Classic Release pipeline, you can add artifact filters to prevent the pipeline from being queued, but these do not enforce that the Stage cannot be run manually or override these filters.
If you truly want to prevent the release, even if someone manually overrides the artifact filter, you could fail the build by checking that it is the correct branch in a script.
$currentBranch = "$(Release.Artifacts.MyAlias.SourceBranch)"
if ( $currentBranch -ne "refs/heads/main" )
{
Write-Host "##vso[task.logissue type=error]This release must originate from the main branch. Current branch: $currentBranch"
Exit 1
}
You can set a branch filter on the Continuous deployment Trigger, See below:
Click the highlighted trigger icon in the Artifacts section to open the trigger panel--> Enabled Continuous deployment Trigger--> Set the branch filter to only include master branch.
When you set your release pipeline Continuous deployment Trigger as above. Only the artifacts generated from master branch can trigger the release pipeline.
You can also set an artifact filter for each stage to make sure only artifacts come from a specific branch can be deployed to this stage. See below:
You can check out this document to learn more about classic CD pipeline.
#paddingtonMike,
This is a core feature of ADO Release Pipelines, you can complete this many ways; however, the most common is by modifying the source of your artifact.
I highly recommend that you invest some time in learning ADO Release Pipelines. Here is a great resource to get started with:
Learn - Create Release Pipeline
I don't know of a way to prevent a deployment from being created from a non-main branch. That said, Levi's answer will prevent a release from being created or deployed automatically. In addition to that, you can set up required reviewers on deployments which will guarantee no release can go out without the approval of a person who is authorized to deploy.
(Specific section of the link Levi shared) https://learn.microsoft.com/en-us/azure/devops/pipelines/release/define-multistage-release-process?view=azure-devops#add-pre-deployment-approvals
I'm setting up a pipeline using Azure Pipelines YAML format. I have created 3 stages: Build, Staging, and Production. As the names suggest, the Build stage builds the project and publishes the build artifacts. The Staging stage deploys to the Staging environment and the Production stage deploys to the Production environment.
In the Environments section of my project, I have added a check for the Production environment so that I can approve the deployment before going live.
The way that my pipeline works is that both Staging and Production stages are triggered automatically after the Build stage is finished. What I don't like about this is that when developers deploy their code to Staging, they need a couple of days to test it on Staging before pushing their code to Production. So, until then, my pipeline keeps running and waiting for my approval. The spinner at the top-left corner keeps spinning and the "Duration" field keeps passing.
Is there any ways that develpers manually trigger the Production stage whenever they are ready instead of the Build stage triggering it?
you can set the trigger to none to disable CI and only trigger it manual
trigger: none
Manual stages in yaml pipeline is not available currently. This feature request has been submitted to Microsoft. You can go and vote it up or submit a new one.
There are workarounds to achieve this.
You can move your staging and production stages to Classic Web UI Release Pipeline. Manually trigger a stage is available in Web UI Release pipeline. Please check here for more information.
Another way to achieve this is to separate your yaml pipeline into two yaml pipelines(stage pipeline and production pipeline). And disable CI build for production pipeline( in the pipeline edit page, click on the 3dots on the top right corner and choose triggers. Please refer to below pics).
So that you can manually run production pipeline after Developer done with their tests.
You can specify which stage you want to run.
When you click "Run pipeline", click on "Stages to run":
Now choose which staged will run:
I think there's a better way. You can add a pipeline variable which can can be overridden when starting the pipeline.
You have to add a new variable to your pipeline and chose 'Let users override this value when running this pipeline'.
In your pipeline add a condition to your stage such as:
condition: and(succeeded(), or(eq(variables['Build.SourceBranch'], 'refs/heads/master'), eq(variables['DEPLOY_PROD'], 'true')))
Now whenever you want a build to deploy to Production you start the build and then override the variable from here:
Set the value to 'true' and your build will trigger the stage you want.
I have a much cleaner solution that I've been using for quite a while. It's similar to my original solution posted here but instead of manually adding a variable to the pipeline I add a parameter and use it as a condition to trigger the deployment in a particular environment.
The azure-pipelines.yaml looks like this:
trigger:
- master
parameters:
- name: deployDEV
displayName: Deploy to DEV
type: boolean
default: false
stages:
- stage: Build
jobs:
- job: Build
steps:
- script: |
echo "Building something..."
- stage: Release_DEV
displayName: Release to DEV
condition: |
and(
succeeded('Build'),
eq(${{ parameters.deployDEV }}, true)
)
dependsOn: Build
jobs:
- job: Release DEV
steps:
- script: |
echo "Releasing to DEV..."
The beauty of this solution is that when you're starting a new instance you'll get the parameters as options in the UI like this:
Yes it can be done. We do not do it in the yaml directly. But instead we add environment in YAML. And on environment we add manual trigger.
environment: 'smarthotel-dev'
Environment and triggers are managed through UI.
https://learn.microsoft.com/en-us/azure/devops/pipelines/process/environments?view=azure-devops
One great workaround with YAML is using conditions and variables.
Just add condition: eq(variables['Build.Reason'], 'Manual') in the stage that needs manual intervention and it should be it.
There is a lot of useful information on https://ochzhen.com/blog/manual-trigger-in-yaml-azure-pipelines
Here is a link to view all values of Build.Reason: https://learn.microsoft.com/en-us/azure/devops/pipelines/build/variables?view=azure-devops&tabs=yaml