Limit which branch is built by Jenkins pipeline? - github

I am currently configuring a Jenkins server hosted on a Docker container in AWS.
I am using BlueOcean to configure a repository.
Right now, the pipeline scans all branches on a repository to detect Jenkinsfiles and then will automatically build on that branch if it detects changes. I scan the repo every 5 minutes to detect changes.
However, I do not want to be running builds and jobs automatically if it is some random feature branch. I am trying to limit the automatically triggered builds to only changes in staging and master branches.
So my question is, how/where do you configure Jenkins GitHub pipeline to only build on certain branches rather than scanning all branches?

A Multibranch pipeline job is your friend.
Rather than trying to limit which branches Jenkins is polling firstly what I do in my Jenkinsfile is poll source control every minute:
triggers { pollSCM('* * * * *') }
This will poll every branch and create a job where it finds a Jenkinsfile in the location and name you specify in the Multibranch Pipeline job configuration.
Side Note
About the only configuration in a multibranch pipeline is:
Where's the SCM repo?
Workspace relative path and name of Jenkinsfile. (You can call it Bob if you want)
A multibranch pipeline job sets an additional environment variable: BRANCH_NAME which allows you to conditionally perform actions in pipeline like so:
script {
if( "${env.BRANCH_NAME}" == "integration" ) {
//Do something useful
}
}
Using this method you can also decide to do nothing in response to a poll event.

I assume you are using github plugin. I'd suggest configuring a webhook on your repository using Generic Webhook Trigger Plugin - https://wiki.jenkins.io/display/JENKINS/Generic+Webhook+Trigger+Plugin
This plugin is awesome and lets you easily extract the values in the incoming webhook and use those in your pipeline. For ex. you can extract the branch from where the webhook came from and only build if the branch is staging or master
In our setup we use a simple job 'webhook trigger processor' which reads the incoming webhook from all repositories and triggers downstream pipelines using values extracted from webhook.

Pipeline accepts input parameters. So you can create a parameter called branch.
Inside your pipeline you could use regex to match only required branches.

Related

Can I trigger pipelines in other repos when I create a PR?

We have 3 repos in Azure DevOps. We use Azure Pipelines to run our CI/CD chain.
Repo A is a toolset used by the other repos.
Repo B uses A as submodule, and C uses A as git submodule.
When creating a PR in repo A, we want to trigger test Pipelines in repo B and C and have the status in repo A's PR. B and C's pipelines/runs should use the PR commit when running their tests. This to avoid that changes in repo A breaks repo B and C upon release.
Our problem is that we want to make sure to use the right commit of repo A (the one in the PR) and to get the test results for repo B and C in to the PR.
This should be possible through build-validation policies in a branch policy, but it won't be straight forward.
Under the hood, when you specify a build in the build-validation policy, azure-devops will queue the pipeline on a temporary branch /refs/pull/<id>/merge. It queues the pipeline with several additional variables that represent pull request meta-data. When that build completes, it sends a status message back to pull-request to indicate whether the conditions were met.
Microsoft has a few articles on how to create a custom PR server or using a Function App to create custom branch policies. These could be used as an approach, but both articles suggest that they invoke based on a webhook when the pull request is created. Both articles mention how to send custom PR Status messages back to the PR.
If you want more control over manually re-queueing the build-validation, you could in theory also create a pipeline that triggers the other pipelines in the other repos.
Define a build-validation pipeline that can be used to trigger the other pipelines. This is equivalent to using a Webhook/Function app, but the main difference is the UI in the Pull Request provides options to retrigger the build.
This build-validation pipeline reads the queue-time variables to identify which PR is being executed. You can add logic to this pipeline to use the Azure DevOps REST API to queue the pipelines in the other repos/projects. When queuing these pipelines, pass the details of the pull request meta-data as queue-time variables in the body of the message.
When the external pipeline runs, read the queue time parameters to record which PR initiated the build. Add some custom logic to include checking out the /refs/pull/<id>/merge branch for your submodules then Build + Test.
When the external pipeline completes, post the PR Status back to the originating PR.
Modify the branch policy to indicate the Build Status from the other pipelines as Required.

Azure DevOps - How to easily switch branches to use for multiple environments

I have four environments that I deploy to.
I also have four different code branches that we use to deploy code from.
We constantly switch the branches we use to deploy on these environments.
One time I want to build and deploy a daily branch on my test environment.
Later I want to build and deploy a enhancements branch on the same test environment.
Next I want to build and deploy the daily branch on my test2 environment.
I think you get the picture
We are currently using a manual process to pull from the branch we want deployed, then zip it up and push it to AWS code deploy.
Using Azure DevOps pipeline and release what is the easiest method to allow me to switch to use different branchs on different environments.
I currently have a successful setup in Azure DevOps that performs a gradle build, creates the artifact and then lets me push it over to AWS CodeDeploy on one of my environments. I just can't seem to figure out a way to eastily swtich the branch without creating tons of Azure pipelines and releases.
Thanks all!
Where you manually trigger a build pipeline by clicking Queue or Run Pipeline, A new windows shown as below will be prompted which allows you to switch the branches.
If you want to automatically deploy different branch to different environment. You can push the build artifacts over to AWS CodeDeploy in a release pipeline and set the branch filters. Please refer to below steps:
1, set branch filter in the build pipeline as shown in below screenshot which will build the selected branched. Check here for more information about triggers.
2, create a release pipline to push build artifacts over to AWS CodeDeploy.
And Set the Artifact filters which will only allow the artifacts built from the specified branch to be deployed to this tage.
You could use a queue time variable to specify the branch name you would like to use on your build pipeline. You would need to:
Edit your build pipeline and create the variable on the "variables" tab. Make sure to mark the "Settable at queue time" check
variable creation
Update the source of your build pipeline, to specify the new variable under the "Default branch" option. It would look something like this:
pipeline source
RUN your pipeline. Before finally clicking on RUN, you will be able to specify the desire branch:
set variable value
Hope this works

Jenkins - How to use the same workspace as Multi-branch for Jenkins PR jobs

Backgroud:
I have setup a Multi-branch job that checks the repository and triggers through the Github webhook a build for every push on a branch.
Jenkinsfile is present on branches.
There are several stages present but the checkout scm is made using the same workspace location for all branches (repo > 1GB):
node("Node_Name") {
ws(workspacePath) {
echo "Checking out code on ${env.NODE_NAME}"
checkout scm
buildSolution()
}
}
Problem
When I set up to also trigger a job at a pull request, that particular job is triggered but it's ignoring the custom workspace set up in Jenkinsfile (present in both branches).
Any hints on how I could solve this? Checking out the entire repository (> 1GB) for every pull request takes up a lot of time, so I'd prefer avoiding this.

Automatically Build GitHub Branch on Commit

I am new to working with Jenkins pipeline. I am able to use the GitHub Plugin in Jenkins and Webhooks from GitHub to successfully build a specific branch of a repository for a free style job. I can't find documentation that documents how to setup the "Source Code Management" so that only specific branches are build based on the github webhook.
For now I can chain the pipeline job to a free style job so that I can build only specific branches. I would rather have the pipeline job configured specifically for the branch we are trying to build.
Thanks in advance for your help!
SCM Configuration for the pipeline job.
!https://i.stack.imgur.com/0NoOX.png
In order to accomplish this within a Jenkins Pipeline job, you must mark the Pipeline Definition as "Pipeline script from SCM". This will instruct Jenkins to base the pipeline execution based on a Jenkinsfile within the repository. Here, you can also instruct Jenkins which branches to build.
From there, you simply need to make sure that your GitHub pushes are triggering builds within Jenkins correctly, and that's all there is to it!
The goal of the pipeline job was to build and deploy specific branch automatically. The approach was to create a pipeline job and define the branch in the SCM configuration and enable webhooks so that the branch would automatically build when a new commit is pushed. Unfortunately the webhook SCM build for pipeline is broken or is not supported for webhooks.
We have decided to change our approach and use the multibranch pipeline job. This by default build ALL branches that have a jenkinsFile. We are filtering in the job for the specific branches we want automatically build.

Jenkins Pipeline pulls entire source code for jenkinsfile

I was playing Jenkins 2.0 with multibranch pipeline with one master server plus a couple of slave nodes.
The Jenkinsfile looks like:
node('slave') {
stage 'Checkout'
checkout scm
stage 'Build'
...
}
We see that the plugin pulls the entire source code on the master node in order to get Jenkinsfile and then pull the entire source tree again on the slave nodes. Is there a way to have the master only pull Jenkinsfile?
Thanks.
See JENKINS-33273. The current SCM plugins do not offer a way of retrieving a single file without doing a complete checkout.