So ive been building a build pipeline, that is triggered whenever a pull request is done to master, so we have a branch policy such that the only change to the master branch is through pull requests.
I want the build pipeline to checkout the source branch of the PR and do some commits to the source branch as part of the build pipeline. I thought i could just use the Build.SourceBranchName variable but when the pipeline is triggered the SourceBranchName is master. So I could not use it.
Are there any easy ways of doing this?
I want the build pipeline to checkout the source branch of the PR
To checkout the source branch of the PR, you could use the predefined system variables about PR:
System.PullRequest.SourceBranch and System.PullRequest.TargetBranch
To get the branch that is being reviewed in a pull request, we should select the variable System.PullRequest.SourceBranch.
now the issue becomes that because of a new commit to the PR it runs
the pipeline again, this should not happen since i have [skip ci] in
the commit message.
As we know, the [skip ci] or [ci skip] is used to skip running CI, like the option
Enable continuous integration on UI:
However, our current scenario is branch policy for build validation instead of CI. This is very different from CI, although they seem to be doing the same build task. Branch policy is to protect our branches from being corrupted by incorrect submit. This is a verified operation instead of continuous integration.
Check the document Skipping CI for individual commits for some more details.
So, this is two different scenarios, we could not apply the CI settings to the branch policy.
Second, Branch policy is used to protect our branches, any commit requires validation by branch pliocy, although sometimes we can know that our modifications don't require build validation, but we're not sure if there are any where we overlook that cause our target branch to be broken. Skip unnecessary verification will bring us some construction convenience, but with the risk measurement it brings, these conveniences are negligible, so we don't recommend skipping the verification of the branch office strategy.
If skipping Build Validation is your insistence, you can try LJ’s suggestion.
Related
We have a development branch that is fed from feature branches through pull requests. After merge a build occurs and the artifact created triggers the release pipeline with some deployments happening automatically and others happening after manual approval.
After the approval happens for production, and after successful deployment, we would like to merge the PR that was was merge to develop to our main branch. I was wondering if this could be achieved automatically from the pipeline.
Note that not all the PR are going to make its way to production and the latest may not be the one being deployed in production.
Assuming that you must complete the PR to trigger the CI build and release pipelines, you won't be able to re-use the existing PR to merge into another branch because PRs are specific to branches and once completed they can't be re-opened.
In theory, you could add logic to your release pipeline to create a new PR after successfully deploying into production but this is dangerous because PRs are based on branches and not individual commits. So if the develop branch changes while you're deploying into production the new PR would contain changes that haven't been deployed. If you have a small team and the number of PullRequests are low, this might not be a problem for you.
However, if you have a large team and dozens of pull-requests a day, you might benefit by adjusting your a gitflow branching strategy. In that model you would create a release branch and deploy that into other environments. Creating the pull request at the end of that pipeline flow makes total sense because you're deploying and merging a static branch.
For example, you could add a step in your release pipeline to create the PR using the Azure CLI:
$pr = az repos pr create `
--source-branch develop `
--target-branch main `
--title 'merge $(Build.BuildId) into main' |
ConvertFrom-Json
Write-Host 'Created PR $($pr.pullRequestId)'
There's another question where they're doing something similar.
Alternatively, look at the "Github flow" model. They use a trunk-based branching strategy where the PR is king and changes in the PR ultimately go to production or they don't get merged. The GitHub team had a model where they concentrated on a single PR at a time with a flow that looked like:
Designate a PR as a candidate for production
Automation would lock the target branch and create a temporary merge of the PR and the target branch
Automation would build this branch and deploy it to various environments.
Upon successful build into the environment, the automation would unlock the target branch and complete the PR.
It's worth pointing out that Azure DevOps does support triggering Releases from Pull Requests, and each PR-triggered build is a temporary merge of the target branch, so you could create a release and deploy it into your environments with your approval gates. You could add automation to your release to approve the PR:
az repos pr set-vote --id $pr.pullRequestId --vote approve
TL;DR
How do I avoid rebuilding artifacts on master when a feature is merged without creating multiple pipelines per project? Where do I access the information about which branch was merged?
More Info
I run Jenkins to build many projects stored in two different VCSs (Gitlab, Bitbucket). Auto-discovery for both VCSs work and create multi-branch pipelines for every project/branch/PR containing a Jenkinsfile (Gitlab Branch Source Plugin, Bitbucket Branch Source Plugin).
Build artifacts get produced and stored on every build (e.g. docker images pushed to registry).
As I follow a feature branch workflow, these features get eventually merged into master, master will then be deployed in irregular intervals.
When doing the merge, there is an artifact already built and stored for this code(see appendix:1). It was built for the feature branch the code originated from (e.g. container mysuperapp:feat-add-better-things-3). I would like to take this artifact and promote it as the new master artifact (e.g. mysuperapp:master), avoiding a rebuild (and unit + integration testing everything).
But, merging a feature branch just kicks off a new build pipeline on branch master without any information about the merged branch (see appendix:2). This is correct behavior concerning master (new commit(s) where pushed) but prevents me from reacting to the merged branch (e.g. the aforementioned promoting or even just deleting unused artifacts). Is there any way to get the information, which branch was merged?
I am aware, that I can create a new pipeline listening for PR webhooks from my VCSs, running a pipeline to do the promotion and ignore builds on master completely. But this moves visibility of this process to a different pipeline and requires additional pipelines for projects, e.g. reducing the advantage of auto-discovery to 50% (have to create these merge pipelines for each project).
How can I keep the advantages of auto-discovery and visibility of executed steps while also executing something on a merge?
Ideas: Tag artifacts differently, but how (needs to be able to clean up correctly)? Parameterize pipelines and setup a single merge pipeline which re-triggers the pipeline 'push on master' with parameters of the merged branch. But can this be done without having to setup the webhooks for every project? Ask the VCSs via REST about which branch belonged to a commit?
Greets and thanks for the help you all! This may be a complicated one, but it would be so cool to get this to work. It's the last barrier for me to enable continuos delivery for a lot of projects!
Appendix:
1: I am also aware, that to have consistent builds, I have to enforce --ff-only merges. This question is not about the pitfalls of git but rather about the way to go with Jenkins.
2: Git provides me with the parent commits, I can easily find out, which commit was merged. But, especially using "Delete branch after merge", leaves me without the branch ref in git. Tagging my docker images with commits instead of branches leaves me with backtracking the last commit on each build to delete the old, obsolete build.
I have a branch policy for build validation. As a final step i am wanting to update a file in the pull request and push it into the branch. it doesn't appear that [skip ci] works for these build triggers. is there another way to accomplish this quality gate?
deeper part of scenario. pull request is building a docker image, and tagging with git hash. i'm updating a yaml file with the new tag so i only have to build the image once. at some point i want to set up another pipeline that will deploy this image based off a successful run of this pipeline(build validation)
does skip ci work with build validation pipelines?
The answer is No.
First, I need to indicate that this is two different scenarios. As we know, the [skip ci] or [ci skip] is used to skip running CI, like the option
Enable continuous integration on UI:
However, our current scenario is branch policy for build validation instead of CI. This is very different from CI, although they seem to be doing the same build task. Branch policy is to protect our branches from being corrupted by incorrect submit. This is a verified operation instead of continuous integration.
Check the document Skipping CI for individual commits for some more details.
So, this is two different scenarios, we could not apply the CI settings to the branch policy.
Second, just like I said in the first point, Branch policy is used to protect our branches, any commit requires validation by branch pliocy, although sometimes we can know that our modifications don't require build validation, but we're not sure if there are any where we overlook that cause our target branch to be broken. Skip unnecessary verification will bring us some construction convenience, but with the risk measurement it brings, these conveniences are negligible, so we don't recommend skipping the verification of the branch office strategy.
Hope this helps.
I have set up a pull request release trigger in the following way.
I want to deploy Artifacts from VerifyApiTestEnvironment branch whenever pull request into that branch is successfully completed.
This is how my artifacts look.
This is how my CD trigger looks like.
This is how my pre-deployment conditions look like.
This is how my policy screen looks.
This is how my branch structure looks. I am always getting an error in the build saying source branch missing the changes from master when I complete the pull request targeting the VerifyApiTestEnvironment branch. What could be the reason behind this?
Every time I am committing my changes to a featureBranch and then I open a pull request for it to merge into VerifyApiTestEnvironment branch.
I expect to trigger a release and then a deployment every time I do this, but the release is not getting triggered.
Am I missing something related to configuring PR triggers?
The setting of yours has some problem. As Daniel said, it is used to set as deploy with a PR created. In addition, if you want to deploy just after PR is completed, the source of this release should be Repository instead of build pipeline.
So, first, you need to change your release source as Azure Repository:
And then, enable Continuous deployment trigger. While the PR completed, it means that code change are merged into the target branch of Repository. So, this need to be enabled, or the deploy will not be triggered while the PR is completed.
In addition, you need to set branch filter, or the deploy will be triggered no matter which branch is changed. Here I set just merge into master branch( PR target branch is master) can trigger this CD.
For this option, it just be use for the PR created trigger deploy. Since you just want deploy triggered by the PR completed, so you do not need enable it.
Now, if your master has policy that the code change applied successfully only after PR created and verified, the CD will be executed only after PR completed.
For some configuration of policy, build policy is used to set build trigger, and status policy used to trigger the release.
So, according to your policy setting, it's used to trigger the build first, and then the build will trigger the release. It's not directly trigger the deploy just after the PR completed.
I'm using a build policy on one of my Azure DevOps git branches.
Is there any way of requiring a manually triggered branch build policy be run again if the source branch is updated?
My branch policy is set to Manual trigger. We use Manual instead of Automatic to reduce the number of builds (we use self hosted build server) running on our CI server as we typically open PR's early to provide feedback and facilitate conversation around code.
If the trigger was set to automatic, then any time source branch is updated a new build is queued. This isn't case with Manual. I had situation where build policy passed but then a further commit was made to branch and the reviewer approved. But the build was not manually triggered again and the last commit & push introduced a bug. I'd like the build validation policy to either reset or fail every time the source branch is updated similar to how code reviewer votes are reset after a push.
Is this possible?
I do not think this is currently possible, it has been raised on the Developer community as a feature request / issue, so I would encourage you to up-vote this here.
A solution for our scenario (too many builds were being queued, because Code Reviews come back with changes) was to use Pull Request drafts.
Sprint 143 - Draft Pull Request
When creating a PR, you can click Create as Draft in the drop down beside the Create button.