Run Jenkins stage after pipeline success independently - github

So I have a groovy Jenkinsfile script that runs on my Jenkins server for pull requests. On my repos I have a protected master branch. I want my tests and stuff to run only on certain branches but what I want to happen is when the correct branch is detected it will attempt to merge the pull request automatically. The issue I'm getting is I need the pipeline job to report status success before this stage works. For example
pipeline {
agent none
options {...}
environment {...}
stages {
stage("1") {...}
stage("2") {...}
stage('Merge Pull Request') {
when {
allOf{
not { branch 'master' }
expression{env.CHANGE_TITLE.startsWith('branch title')}
}
}
agent {
docker {
label '...'
image '...'
}
}
steps {
touch "f.txt"
sh "echo \${GITHUB_TOKEN} > f.txt"
sh "gh auth --hostname <hostname> login --with-token < f.txt"
sh "gh pr merge -s ${env.CHANGE_URL}"
}
}
}
}
This works the way I want with the caveat that the merge occurs before the status is reported back so it fails because I'm trying to merge to a protected branch that hasn't received the success yet. Any idea on what I can do or how to trigger another job that is not downstream and waited on by this job to complete?

Presumably, you want the branch(es) to be protected so that manual merges cannot occur outside of the pipeline.
For this, I also presume you define a Required commit status check in your protected branch(es) settings.
By default, the pipeline will start the default 'pr-merge' status check at the beginning of the pipeline, and then mark it passed/failed at the very end. The PR Merge is denied because the required status check is not yet passed.
For your solution to work, The final stage ("Merge Pull Request") should first SET the desired status check to PASSED (via the GitHub API), and then make the PR Merge api call.
By the way, you can choose to use a different PR check than the one used by default in Jenkins, and name it what you want. This has the additional advantage than someone cannot circumvent your clever scheme by setting up a rogue Jenkins service with a no-op pipeline (that will mark the status check as "PASSED" automatically). It's not iron-clad, but it requires a bit more effort.

Related

Using env variable github.ref_name doesn't give me branch name

When I use in my workflow github.ref_name it doesn't provide me a branch name from where I triggered this workflow. I get 16/merge. Why? When I use github.ref_type I clearly see that my ref type is branch so why I don't get branch name?
Also it's showing when I use $GITHUB_REF or git symbolic-ref HEAD (and separating refs/heads/ off). Ah and I tried github.event.push.ref but it's not showing me anything.
How to get always branch name from where I triggered the workflow?
For following code:
Run echo running on branch ${GITHUB_REF##*/} ${GITHUB_REF}
When your workflow runs becuase of push event you will get:
running on branch main refs/heads/main
But for pulr request event it would be:
running on branch merge refs/pull/3/merge
Why's that?
If the repository is on GitHub and you have any Pull Requests that have been opened, you’ll get these references that are prefixed with refs/pull/. These are basically branches, but since they’re not under refs/heads/ you don’t get them normally when you clone or fetch from the server — the process of fetching ignores them normally.
You can also check this question here

How do you restrict which branches can be pulled into a target branch

I'm trying to set up policies on my Azure DevOps Branches.
I'm able to state that a branch must build and pass our unit tests before allowing a merge but is there a way to restrict which branch is allowed to merge into it.
I have two branches that this would impact.
I have my 'master' branch that I would like to restrict to only accept pull requests from a branch called 'UAT'.
I have a branch called 'UAT' that I would like to restrict to only accept pull requests coming from a branch called 'Dev'.
The closest workaround I could think of is to have a very simple pipeline that would run on pull requests and check System.PullRequest.SourceBranch and System.PullRequest.TargetBranch. If the values don't match your policy, then fail the pipeline, which in turn will block the PR.
Based on the answer by qbik i created this short yaml code. Replace the source and target as needed for your use case. The code below is only for testing in my pipeline, to create the desired failure.
- powershell: >
if ("$(System.PullRequest.SourceBranch)" -ne "refs/heads/acc" -And "$(System.PullRequest.TargetBranch)" -eq "refs/heads/test")
{
Write-Error "
=========================================================================================================
Branch check failed.
Illegal Pull Request from $(System.PullRequest.SourceBranch) into $(System.PullRequest.TargetBranch).
========================================================================================================="
}
displayName: Branch Check

How to trigger a GitHub Action workflow for every successful commit

I want to run a GitHub Actions workflow for every successful commit (e.g. after all check suites have ran successfully).
It seems like the check_suite events will be fired for each individual check suite, and contain no information about the other suites. I want to wait for all registered check suites.
The status event seems to be exactly what I want:
Statuses can include a context to indicate what service is providing that status. For example, you may have your continuous integration service push statuses with a context of ci, and a security audit tool push statuses with a context of security. You can then use the combined status endpoint to retrieve the whole status for a commit.
But the status lives completely unrelated to the check_suites events, and most CI services now uses the check_suites (including GitHub Actions itself), and not status.
How can I, with a GitHub Actions workflow, check that all check_suites finished and the commit has successfully passed all checks and tests?
You can use the GitHub API to list the check suites belonging to a ref: https://octokit.github.io/rest.js/#octokit-routes-checks-list-suites-for-ref.
So, within your action (triggered by check_suite), you can list all the suites for the current ref, check they're all successful and then run your code.
If you're building a javascript action it could look like this:
import * as github from "#actions/github";
const response = await client.checks.listSuitesForRef({
owner: github.context.repo.owner,
repo: github.context.repo.repo,
ref: sha
});
const passes = response.data.check_suites.every(...)
if (passes) { ... }
The major drawback of this approach is that this very action is itself a check suite belonging to the current commit, so it will appear in response.data.check_suites (and will never be successful at this point ... because it's still running!). I don't have a solution for this part yet...
It looks like you are now able to trigger actions based on commit status (reference).
on:
status
jobs:
if_error_or_failure:
runs-on: ubuntu-latest
if: >-
github.event.state == 'error' ||
github.event.state == 'failure'
steps:
- env:
DESCRIPTION: ${{ github.event.description }}
run: |
echo The status is error or failed: $DESCRIPTION

Can I skip an AWS CodePipeline build?

I am currently developing a personal project on master. Every time I push to origin master a build is triggered on CodePipeline. As I am the only developer working on this project and don't want to bother with branches at this stage it would be nice to skip unnecessary builds. I wouldn't mind pushing to another branch, but it's a small annoyance.
CodeShip allows you to skip a build by including --skip-ci in your commit message. Is something like this possible with CodePipeline?
None of my Google searches have yielded results. The CodePipeline documentation makes no mention of such a feature either.
A valid reason for not wanting to build a certain commit is when you use CodeBuild to generate a commit for you. For example, I have some code on the master branch which passes all the tests. I then want to update the changelog, package.json version and create a git tag on a new commit and push it back to the CodeCommit repo.
If I do this on Codebuild, the version-commit triggers another build! Given the contents of the commit does not materially change the behaviour of the code, there is no need to build & test this commit.
Besides all of this, Amazon should be looking at the features in the marketplace and attempting to provide at least feature-parity. Adding a RegEx check for "skip-ci" to the CodeBuild trigger-code would take a few hours to implement, at most.
By default codepipeline creates a cloudwatch event which triggers your pipeline on all changes of the specific branch.
What you can do is to set this cloudwatch event to trigger a lambda function. This function can check whether it is necessary to build this commit and start your CodePipeline.
Here is an example of how to achieve this:
https://aws.amazon.com/blogs/devops/adding-custom-logic-to-aws-codepipeline-with-aws-lambda-and-amazon-cloudwatch-events/
Here is a simple example for lambda function. It checks if the last commit has no [skip-CI] in its message and after that executes the pipeline.
Keep in mind, that this code checks only the last commit if your change was a series of commits you might want to check everything between oldCommitId and commitId.
const AWS = require('aws-sdk');
const codecommit = new AWS.CodeCommit();
const codepipeline = new AWS.CodePipeline();
exports.handler = async (event) => {
const { detail: { repositoryName, commitId, oldCommitId } } = event
const { commit } = await codecommit.getCommit({
commitId,
repositoryName
}).promise()
if(commit.message.search(/\[skip-CI\]/) === -1) {
const { pipelineExecutionId } = await codepipeline.startPipelineExecution({
name: 'your-pipeline-name'
}).promise()
console.log(`Pipeline have started. Execution id: ${pipelineExecutionId}!`)
} else {
console.log('Pipeline execution is not required')
}
return;
};
This isn't a feature offered by CodePipeline at this time.
I'd be curious to know why you view some builds as unnecessary. Do you push a sequence of commits and only want a build against the last commit? This may be more applicable in a team environment, but I would tend to want a build for every commit so I don't find myself in a situation where I push code and pick up and build someone else's broken code for the first time in my build.
I would suggest a manual review stage in your pipeline before it builds. Then you can just approve it when you are ready to build.

Prevent mercurial push during a jenkins build

I have a jenkins job that runs some tests on a mercurial repo, and if successful tags the local repo with a 'stable' tag and then pushes this back to the main repo. The issue I'm having is that if someone pushes changesets while the build is running, then I cannot push the 'stable' tag.
I was wondering if there was a way to set the remote repo to read-only while the build is running, then make it 'push-able' once the build finishes?
Thanks,
Vackar
Preventing the push is probably not what you want (and it's almost pretty much impossible). The promise of a DVCS like Mercurial or git is that there's no locking -- it's a step forward.
Have you considered having Jenkins just pull and update before it merges? You can still tag the proper revision. Something like this:
jenkins checks out the code and notes the revision id it's building
jenkins does the build, runs the tests, etc. and everything goes well
jenkins does a hg pull to get the latest from the server
jenkins does a hg tag -m "build number $BUILD_NUMBER" --revision X --force stable
jenkins does a hg push
Then there's (almost) no time between that final pull, tag, and push, but the tag still goes on the revision that was actually build -- because you saved that revision hash id from when you first pulled.
I've just been looking for something similar. In our case, Jenkins is performing a merge, running an extensive suite of tests and once they all pass, pushing the merged code back to the repository. So it takes ~1 hour and fails if a developer pushes while the job is executing (it can't do the final push).
I couldn't find a ready-made solution, so ended up writing a mercurial hook which checks whether the job is building (using the REST API) before allowing the push.
You'll need access to your remote mercurial repository, but other than that, it's not too complex.
Add the following to your-remote-repo/.hg/hgrc:
[hooks]
pretxnchangegroup.DisablePushDuringJenkinsBuild= python:.hg/disable_push_if_building_hook.py:check_jenkins
[jenkins]
url=http://path-to-jenkins
jobs=jenkins-job-name[,comma-separated, for-multiple, jobs]
And make sure this python script is in your-remote-repo/.hg/
import json, urllib2
from mercurial import util
TEN_SECONDS = 10
def check_jenkins(ui, repo, node, **kwargs):
jenkins_url = ui.config('jenkins', 'url', default=None, untrusted=False)
jenkins_jobs = ui.config('jenkins', 'jobs', default=None, untrusted=False)
if not jenkins_url:
raise util.Abort('Jenkins hook has not been configured correctly. Cannot find Jenkins url in .hg/hgrc.')
if not jenkins_jobs:
raise util.Abort('Jenkins hook has not been configured correctly. Cannot find Jenkins jobs in .hg/hgrc.')
jenkins_jobs = [x.strip() for x in jenkins_jobs.split(',')]
for job in jenkins_jobs:
job_url = jenkins_url + '/job/' + job + '/lastBuild/api/json'
ui.write('Checking if job is running at URL: %s\n' % job_url)
try:
job_metadata = json.load(urllib2.urlopen(job_url, timeout = TEN_SECONDS))
if 'building' in job_metadata and job_metadata['building']:
raise util.Abort('Jenkins build "%s" is in progress. Pushing is disabled until it completes.' % job_metadata['fullDisplayName'])
except urllib2.URLError, e:
raise util.Abort('Error while trying to poll Jenkins: "%s"' % e)
return False # Everything is OK, push can be accepted