I have 3 Yaml Pipelines:
CI1 that should be build on any commit to services/* branches
CI2 that should be build on any commit to services-release/* branches
CD that should do the deployment of the artifacts created by CI2
CD is setup the following way:
YAML Settings
Triggers Settings
As you can see, I've tried different formats of the branches to branch filters. Even if I add non-wildcard filter, I still see the following behavior:
CD pipeline is triggered after CI2, triggered by commit into services-release/* branch (which is correct), but it releases latest build from a branch, specified in "default branch for manual and scheduled builds" dropdown - which is develop in my case.
What should I change to deploy the artifact that was generated by CI2 build from services-release/* branch?
Judging by the pictures, you are using yaml-pipelines but classic pipeline triggers. While this works for triggering the pipelines, you might want to consider implementing the triggers in yaml files for C1- and C2-pipelines.
https://learn.microsoft.com/en-us/azure/devops/pipelines/repos/azure-repos-git?view=azure-devops&tabs=yaml#ci-triggers
As for the yaml-based CD pipeline, in order for the pipeline to trigger upon completion of CI2 and for it to download artifacts from the triggering run (instead of latest from the default pipeline), you should reference the CI2-pipeline as resource:
https://learn.microsoft.com/en-us/azure/devops/pipelines/process/pipeline-triggers?view=azure-devops
So something like:
resources:
pipelines:
- pipeline: ci2_pipeline #this is used to reference this resource in CD pipeline
source: CI2 #Rename this to match your build pipeline name
trigger:
branches:
- services-release/*
For the artifacts, you want to ensure that you are using Pipeline Artifacts instead of classic build artifacts and use the Download Pipeline Artifacts -task in CD-pipeline (https://learn.microsoft.com/en-us/azure/devops/pipelines/tasks/utility/download-pipeline-artifact?view=azure-devops). So something like:
steps:
- download: none # Disable downloading default artifacts
- task: DownloadPipelineArtifact#2
inputs:
buildType: "specific"
project: "$(resources.pipeline.ci2_pipeline.projectID)"
definition: "$(resources.pipeline.ci2_pipeline.pipelineID)"
preferTriggeringPipeline: true
buildVersionToDownload: "latestFromBranch"
branchName: "$(Build.SourceBranch)"
targetPath: "$(Pipeline.Workspace)"
Related
I will try to explain in the best way.
First situation: I didn't find any task to insert my yml file which is in ruby. If I choose Ruby, I can't edit the yaml file. If I can edit that file, I should make this trigger fire. It actually fires, not with the yml file I want
If I use the other way I can't save because I don't have permission to run pr on the master branch. My pipeline doesn't run automatically just manually
enter image description here
enter image description here
My yml:
# Ruby
# Package your Ruby project.
# Add steps that install rails, analyze code, save build artifacts, deploy, and more:
# https://learn.microsoft.com/azure/devops/pipelines/languages/ruby
trigger:
branches:
include:
- homolog_gso
pool:
vmImage: 'ubuntu-latest'
steps:
- task: UseRubyVersion#0
inputs:
versionSpec: '>= 2.5'
- script: |
gem install bundler
bundle install --retry=3 --jobs=4
displayName: 'bundle install'
- script: bundle exec cucumber -t #incluir_setor_resp_em_branco
displayName: 'bundle exec cucumber'
- task: PublishTestResults#2
inputs:
testResultsFormat: 'NUnit'
testResultsFiles: '**/TEST-*.xml'
mergeTestResults: true
searchFolder: '$(Common.TestResultsDirectory)'
testRunTitle: 'Regression Test Geocall Gab'
When creating a new YAML pipeline and selecting the provided template to generate a initialized YAML file, by default this YAML file will be saved into the default branch of source repository (such as master, main).
If you do not have the permissions to directly edit files in the default branch, you are surely not able to edit the YAML file saved in the default branch.
you should copy / create the YAML to another branch where you have the permissions to edit files (e.g. homolog_gso). Then create a PR to merge the changes from this branch to the default branch.
About the PR trigger in your case, you can set the PR trigger like as below in the YAML file from the homolog_gso branch:
pr:
branches:
include:
- master
. . .
By this way, if you push some changes to the homolog_gso branch and then create a PR to merge the changes to the master branch, the PR trigger will fire to run the YAML pipeline.
We have a YAML based pipeline that Unit Tests and build an ASP NET Core website then if everything is OK it deploys to DEV, TEST and eventually Live Azure Resources.
Our source control is Git within Azure Dev Ops.
Our process has us working in a branch for each feature, once those branches are ready we merge them into a "release" branch for an integration test before being PR'ed to MAIN. An example of our release branch would be "release_3_1_5".
The start of the YAML pipeline looks like this
pool:
vmImage: 'windows-latest'
# Why would I want 'resources'
# resources:
# pipelines:
# - pipeline:
variables:
azureSubscriptionEndpoint: 'ARM Service Connection'
webAppKind: webApp
solution: '**/*.sln'
buildPlatform: 'Any CPU'
buildConfiguration: 'Release'
# Change this when adding functionality that is a breaking change
majorVersion: 3
# Change this when adding functionality that is backwards compatible
minorVersion: 1
# Change this when making fixes that are backwards compatible and not adding functionality
buildVersion: 0
# Concatenate version parts and buildId to get a full build version string
fullBuildVersionString: $(majorVersion).$(minorVersion).$(buildVersion).$(Build.BuildID)
name: $(MajorVersion).$(MinorVersion).$(buildVersion).$(Build.BuildID)
stages:
- stage: Build
jobs:
- job: Build_Job
steps:
- bash: |
echo $(fullBuildVersionString)
We don't specify any explicit triggers so the build runs everytime we push to a branch.
The "MAIN" branch has some branch policies set, those policies include "Build Validation" and currently the Build Validation build policy is configured to run the same YAML pipeline.
The CI pipeline works just fine when pushing changes to our branches, except when the branch in question is the subject\source of a PR to MAIN. In this situation the pipeline starts twice. Once for the push to the "release" branch and once by the branch policy because of the PR into MAIN.
Is there a better way to configure the pipeline so it does not kick off twice? I basically do not want the CI truigger to fire when the branch is the source of a PR to MAIN but that looks like an impossible condition
This is something we struggle with as well. We have just accepted the double builds for now. However, I am starting to consider not having a build trigger for feature/ branches and only trigger for PRs.
The only other option are double manifests. One manifest for branches that are not MAIN, and the other being a manifest that includes only PRs and the MAIN branch.
If you want builds to run for branches, you could consider pre-receive hooks that requires builds to run locally.
I have a YAML build pipeline which resides on the repo PipelineRepo, branch master.
It contains a reference to a second repo AppRepo with a trigger on branch dev.
When a commit is made on AppRepo/dev, the build triggers and produces an artifact. The Build.SourceBranch predefined variable available during the build is of course dev as this was the triggering branch on AppRepo. This is as per the docs here:
https://learn.microsoft.com/en-us/azure/devops/pipelines/repos/multi-repo-checkout?view=azure-devops#triggers
The specific part of the docs is this:
When an update to one of the repositories triggers a pipeline, then
the following variables are set based on triggering repository:
Build.Repository.ID
Build.Repository.Name
Build.Repository.Provider
Build.Repository.Uri
Build.SourceBranch
Build.SourceBranchName
Build.SourceVersion
Build.SourceVersionMessage
I consume the artifact in a Release pipeline, where I have a "Dev" stage with an artifact filter for branch dev:
I have a continuous deployment trigger on the artifact for branch dev:
Now we come to the problem
Every time a new build artifact is produced from the AppRepo/dev branch, the continuous deployment trigger doesn't fire because it thinks the build branch of the artifact was PipelineRepo/master. This is also true of the artifact filter on the stage - I have tested it by changing the continuous deployment trigger to master.
Note: you can see on the screenshot the build name contains the word "dev". This is because I use the Build.SourceBranch var in my custom build name. This proves the artifact is definitely produced by the AppRepo/dev triggering branch.
How can I get the DevOps Release Pipeline to pick up the triggering branch?
Based on your description, I could reproduce the similar issue in my organization.
When the pipeline is triggered by another repo branch, it still show the master branch in Release artifacts.
At the same time, the display of the trigger branch is also inconsistent in Pipelines.
For example:
Overview:
Detail view:
I suggest that you could create a feedback ticket in Our feedback Site.
For a workaround:
You could add a build tag in Pipeline to distinguish artifacts.
Yaml sample:
resources:
repositories:
- repository: test
type: git
name: 123/ARM
trigger:
- test
steps:
- checkout: test
- script: echo "##vso[build.addbuildtag]$(Build.SourceBranch)"
Release Pipeline:
You could set the master branch and add the build tag.
I have a release pipeline with the usual stages
Dev -> Test -> Pre Production -> Production
I would like to know if there is a way to ensure that only the master branch can be promoted all the way to production to avoid releases from feature and develop branches ending on a production environment.
YAML Pipelines
If you're using a YAML-based pipeline, you can use the Checks + Approval Gates feature to add a branch policy check.
Create an Environment for each deployment environment
Structure your YAML pipeline to use a Deployment Job that specifies the Environment
stages:
- stage: build
- stage: dev
- stage: test
- stage: preprod
- stage: prod
You can put conditions on the stages to ensure the branch conditions are met.
- stage: production
dependsOn: preprod
condition: |
and(
succeeded('preprod'),
or(
eq(variables['Build.SourceBranch'], 'refs/heads/main'),
startsWith(variables['Build.SourceBranch'], 'refs/heads/release/'),
startsWith(variables['Build.SourceBranch'], 'refs/heads/hotfix/')
)
)
Ensure each stage that performs the deployment activities has a Deployment Job:
- deployment: deploy
displayName: 'Deploy'
environment: 'Production'
strategy:
runonce:
deploy:
steps:
- pwsh: Write-Host 'Deployment Steps go here!'
In your Production Environment, add the Branch Control check:
Classic Release Pipelines
If you're using a Classic Release pipeline, you can add artifact filters to prevent the pipeline from being queued, but these do not enforce that the Stage cannot be run manually or override these filters.
If you truly want to prevent the release, even if someone manually overrides the artifact filter, you could fail the build by checking that it is the correct branch in a script.
$currentBranch = "$(Release.Artifacts.MyAlias.SourceBranch)"
if ( $currentBranch -ne "refs/heads/main" )
{
Write-Host "##vso[task.logissue type=error]This release must originate from the main branch. Current branch: $currentBranch"
Exit 1
}
You can set a branch filter on the Continuous deployment Trigger, See below:
Click the highlighted trigger icon in the Artifacts section to open the trigger panel--> Enabled Continuous deployment Trigger--> Set the branch filter to only include master branch.
When you set your release pipeline Continuous deployment Trigger as above. Only the artifacts generated from master branch can trigger the release pipeline.
You can also set an artifact filter for each stage to make sure only artifacts come from a specific branch can be deployed to this stage. See below:
You can check out this document to learn more about classic CD pipeline.
#paddingtonMike,
This is a core feature of ADO Release Pipelines, you can complete this many ways; however, the most common is by modifying the source of your artifact.
I highly recommend that you invest some time in learning ADO Release Pipelines. Here is a great resource to get started with:
Learn - Create Release Pipeline
I don't know of a way to prevent a deployment from being created from a non-main branch. That said, Levi's answer will prevent a release from being created or deployed automatically. In addition to that, you can set up required reviewers on deployments which will guarantee no release can go out without the approval of a person who is authorized to deploy.
(Specific section of the link Levi shared) https://learn.microsoft.com/en-us/azure/devops/pipelines/release/define-multistage-release-process?view=azure-devops#add-pre-deployment-approvals
I have a following structure:
Test-A and Test-B projects created on Azure Devops. Test-A project's CI build pipeline will produce an artifact.
Test-B's pipeline uses the artifact produced by Test-A's CI pipeline in its build. I am able to download the build artifact of Test-A project and use it.
The problem I am facing here is I am unable to do a CI automatic trigger with project Test-A dependency in project Test-B i.e when ever I make changes and push the changes on to Test-A github repo or whenever I do a new build on Test-A I want the build for Test-B to start automatically.
I have read the documentation on Azure devops but they are not working.
Link for pipeline trigger
Link for pipeline multi-trigger
Below is my .yml file.
trigger:
- master
pool:
vmImage: 'ubuntu-latest'
resources:
pipelines:
- pipeline: JustAName
project: Test-A
source: Test-A_CI
branch: master
trigger:
branches:
include:
- master
repositories:
- repository: justAnotherName
type: github
name: myGitRepo
endpoint: myGitServiceConnection
trigger:
branches:
include:
- master
steps:
- task: DownloadPipelineArtifact#2
inputs:
buildType: 'specific'
project: 'hashValue or Test-A'
definition: '1'
specificBuildWithTriggering: true
buildVersionToDownload: 'latest'
targetPath: '$(Agent.BuildDirectory)'
I am not sure where I am doing wrong or if it is a permission issue. I checked the logs to find any reference with the resources part in the yml but I had no luck.
Can someone suggest what is the best way to check what is the problem and resolve the issue.
Build completion option is disable in classic editor
The build completion option in the pipeline classic editor is limited to the pipelines within the same project. So it won't work for your scenario(the triggering pipeline and triggered pipeline reside in different project) even if it is enabled.
For issue build completion option is disabled. You can report this issue here. Click report a problem and select Azure Devops.
Resources pipeline trigger doesnot work properly sometimes. This similar issue has submitted to Microsoft by some other users. You can follow and vote on these cases or create a new one. Build Completion Triggers not working, Pipeline trigger not working as expressed in documentation
You can follow the workaround which using task TriggerPipeline given by #Hugh. You can also add a powershell task to call the rest api to queue another build pipeline. You can check this thread for example scripts
For your issue , there is a custom task in the Azure DevOps marketplace: Trigger Azure DevOps pipeline.
With this task you can trigger a build or release pipeline from another pipeline within the same project or organization but also in another project or organization.
To get started a PAT is needed with the appropriate rights to execute pipelines. Give the PAT the following rights depending on your scenario:
Triggering a Build: Build – Read & execute
When you have installed the extension, you can add this task into yaml. In the task setting ,you need to connect to a Azure DevOps Service connection. For detailed configuration, please refer to this.
steps:
- task: TriggerPipeline#1
inputs:
serviceConnection: 'triggerpipeline'
project: 'Test-B'
Pipeline: 'Build'
buildDefinition: 'xxx'
Branch: 'master'