I have a jenkins job that runs some tests on a mercurial repo, and if successful tags the local repo with a 'stable' tag and then pushes this back to the main repo. The issue I'm having is that if someone pushes changesets while the build is running, then I cannot push the 'stable' tag.
I was wondering if there was a way to set the remote repo to read-only while the build is running, then make it 'push-able' once the build finishes?
Thanks,
Vackar
Preventing the push is probably not what you want (and it's almost pretty much impossible). The promise of a DVCS like Mercurial or git is that there's no locking -- it's a step forward.
Have you considered having Jenkins just pull and update before it merges? You can still tag the proper revision. Something like this:
jenkins checks out the code and notes the revision id it's building
jenkins does the build, runs the tests, etc. and everything goes well
jenkins does a hg pull to get the latest from the server
jenkins does a hg tag -m "build number $BUILD_NUMBER" --revision X --force stable
jenkins does a hg push
Then there's (almost) no time between that final pull, tag, and push, but the tag still goes on the revision that was actually build -- because you saved that revision hash id from when you first pulled.
I've just been looking for something similar. In our case, Jenkins is performing a merge, running an extensive suite of tests and once they all pass, pushing the merged code back to the repository. So it takes ~1 hour and fails if a developer pushes while the job is executing (it can't do the final push).
I couldn't find a ready-made solution, so ended up writing a mercurial hook which checks whether the job is building (using the REST API) before allowing the push.
You'll need access to your remote mercurial repository, but other than that, it's not too complex.
Add the following to your-remote-repo/.hg/hgrc:
[hooks]
pretxnchangegroup.DisablePushDuringJenkinsBuild= python:.hg/disable_push_if_building_hook.py:check_jenkins
[jenkins]
url=http://path-to-jenkins
jobs=jenkins-job-name[,comma-separated, for-multiple, jobs]
And make sure this python script is in your-remote-repo/.hg/
import json, urllib2
from mercurial import util
TEN_SECONDS = 10
def check_jenkins(ui, repo, node, **kwargs):
jenkins_url = ui.config('jenkins', 'url', default=None, untrusted=False)
jenkins_jobs = ui.config('jenkins', 'jobs', default=None, untrusted=False)
if not jenkins_url:
raise util.Abort('Jenkins hook has not been configured correctly. Cannot find Jenkins url in .hg/hgrc.')
if not jenkins_jobs:
raise util.Abort('Jenkins hook has not been configured correctly. Cannot find Jenkins jobs in .hg/hgrc.')
jenkins_jobs = [x.strip() for x in jenkins_jobs.split(',')]
for job in jenkins_jobs:
job_url = jenkins_url + '/job/' + job + '/lastBuild/api/json'
ui.write('Checking if job is running at URL: %s\n' % job_url)
try:
job_metadata = json.load(urllib2.urlopen(job_url, timeout = TEN_SECONDS))
if 'building' in job_metadata and job_metadata['building']:
raise util.Abort('Jenkins build "%s" is in progress. Pushing is disabled until it completes.' % job_metadata['fullDisplayName'])
except urllib2.URLError, e:
raise util.Abort('Error while trying to poll Jenkins: "%s"' % e)
return False # Everything is OK, push can be accepted
Related
I have workflow that is triggered when a specific file is changed. Is it possible to ensure that this workflow is only triggered the first time that file is changed?
The use case is:
a repository is created from a template repository
to initialize the README and other things in the repo, some variables can be set in a JSON config file
once that file is committed, the workflow runs, creates the README from a template etc.
I have tried to do let the workflow delete itself before it commits and pushes the changed files. That doesn't work: fatal: not in a git directory and fatal: unsafe repository ('/github/workspace' is owned by someone else).
What might work is adding a topic like initialized to the repo at the end of the workflow and check for the presence of this topic at the beginning of the workflow. However, this is feels like a hack in that I'm abusing topics for something they're probably not meant to do.
So my question remains: is there a good way to only run the workflow once?
With the help of the comment by frennky I managed to do solve this by using the GitHub CLI to disable the workflow.
Here is the step at the end of the workflow that does this:
- name: Disable this workflow
shell: bash
run: |
gh workflow disable -R $GITHUB_REPOSITORY "${{ github.workflow }}"
env:
GITHUB_TOKEN: ${{secrets.GITHUB_TOKEN}}
When I use in my workflow github.ref_name it doesn't provide me a branch name from where I triggered this workflow. I get 16/merge. Why? When I use github.ref_type I clearly see that my ref type is branch so why I don't get branch name?
Also it's showing when I use $GITHUB_REF or git symbolic-ref HEAD (and separating refs/heads/ off). Ah and I tried github.event.push.ref but it's not showing me anything.
How to get always branch name from where I triggered the workflow?
For following code:
Run echo running on branch ${GITHUB_REF##*/} ${GITHUB_REF}
When your workflow runs becuase of push event you will get:
running on branch main refs/heads/main
But for pulr request event it would be:
running on branch merge refs/pull/3/merge
Why's that?
If the repository is on GitHub and you have any Pull Requests that have been opened, you’ll get these references that are prefixed with refs/pull/. These are basically branches, but since they’re not under refs/heads/ you don’t get them normally when you clone or fetch from the server — the process of fetching ignores them normally.
You can also check this question here
I have a set of GitHub Actions configured to block pull requests from being merged until the Actions complete successfully. However, every time a new commit is pushed to a PR, the Actions are run again, which can be very wasteful if the author is not yet ready to merge, and intends to make future changes. Is there any way to have a GitHub Action still block a PR being merged but also not run the Action automatically?
With this recent update you can now convert pull requests back to draft status. So you could do that when you need to make changes and disable the CI for drafts. Then convert the draft to a pull request after the changes are complete to rerun CI.
on: pull_request
jobs:
build:
if: github.event.pull_request.draft == 'false'
runs-on: ubuntu-latest
steps:
...
One workaroound would be for the Action to look for a specific comment (for example: "[TO MERGE]: this commit is about ...), and:
return immediately if '[TO MERGE]' is not found, minimizing the action overhead on each commit
go on with checks, if '[TO MERGE]' is found.
I am currently developing a personal project on master. Every time I push to origin master a build is triggered on CodePipeline. As I am the only developer working on this project and don't want to bother with branches at this stage it would be nice to skip unnecessary builds. I wouldn't mind pushing to another branch, but it's a small annoyance.
CodeShip allows you to skip a build by including --skip-ci in your commit message. Is something like this possible with CodePipeline?
None of my Google searches have yielded results. The CodePipeline documentation makes no mention of such a feature either.
A valid reason for not wanting to build a certain commit is when you use CodeBuild to generate a commit for you. For example, I have some code on the master branch which passes all the tests. I then want to update the changelog, package.json version and create a git tag on a new commit and push it back to the CodeCommit repo.
If I do this on Codebuild, the version-commit triggers another build! Given the contents of the commit does not materially change the behaviour of the code, there is no need to build & test this commit.
Besides all of this, Amazon should be looking at the features in the marketplace and attempting to provide at least feature-parity. Adding a RegEx check for "skip-ci" to the CodeBuild trigger-code would take a few hours to implement, at most.
By default codepipeline creates a cloudwatch event which triggers your pipeline on all changes of the specific branch.
What you can do is to set this cloudwatch event to trigger a lambda function. This function can check whether it is necessary to build this commit and start your CodePipeline.
Here is an example of how to achieve this:
https://aws.amazon.com/blogs/devops/adding-custom-logic-to-aws-codepipeline-with-aws-lambda-and-amazon-cloudwatch-events/
Here is a simple example for lambda function. It checks if the last commit has no [skip-CI] in its message and after that executes the pipeline.
Keep in mind, that this code checks only the last commit if your change was a series of commits you might want to check everything between oldCommitId and commitId.
const AWS = require('aws-sdk');
const codecommit = new AWS.CodeCommit();
const codepipeline = new AWS.CodePipeline();
exports.handler = async (event) => {
const { detail: { repositoryName, commitId, oldCommitId } } = event
const { commit } = await codecommit.getCommit({
commitId,
repositoryName
}).promise()
if(commit.message.search(/\[skip-CI\]/) === -1) {
const { pipelineExecutionId } = await codepipeline.startPipelineExecution({
name: 'your-pipeline-name'
}).promise()
console.log(`Pipeline have started. Execution id: ${pipelineExecutionId}!`)
} else {
console.log('Pipeline execution is not required')
}
return;
};
This isn't a feature offered by CodePipeline at this time.
I'd be curious to know why you view some builds as unnecessary. Do you push a sequence of commits and only want a build against the last commit? This may be more applicable in a team environment, but I would tend to want a build for every commit so I don't find myself in a situation where I push code and pick up and build someone else's broken code for the first time in my build.
I would suggest a manual review stage in your pipeline before it builds. Then you can just approve it when you are ready to build.
while checking out github pull heads through a * fetch configuration I retrieved a list of pull heads in the form of ref/pull/* or something similar.
Are these pointing to only open , closed or all pull requests ? and what about the pull requests that have been already merged and closed ?
Depends on which git version you have installed.
It was true prior to git v2.0 build was changed from match to simple
V2. release notes:
When "git push [$there]" does not say what to push, we have used the
traditional "matching" semantics so far (all your branches were sent
to the remote as long as there already are branches of the same name
over there). In Git 2.0, the default is now the "simple" semantics,
which pushes:
only the current branch to the branch with the same name, and only
when the current branch is set to integrate with that remote
branch, if you are pushing to the same remote as you fetch from; or
only the current branch to the branch with the same name, if you
are pushing to a remote that is not where you usually fetch from.
You can use the configuration variable "push.default" to change
this. If you are an old-timer who wants to keep using the
"matching" semantics, you can set the variable to "matching", for
example. Read the documentation for other possibilities.
How does the push configuration related to pull request?
Git hub store the pull request using refspec which means that its behave just like any other branch with the exception that instead of begin placed under /refs/heads/* its is stored under /refs/heads/<pulls (you can change this path- see below)>/*.
So to summarize it:
Github pull reqauests behave the same way as normal refs, depending on your configuration. If you did changed anything and you using git<2 you will get the full list of pull request, if you changed the settings or using git>2 you should only get partial of the pull requests.
Example:
[remote "origin"]
fetch = +refs/heads/*:refs/remotes/origin/*
url = git#github.com:xxx/xxxx.git
fetch = +refs/pull/*/head:refs/remotes/origin/pr/*
Fetch all the pull requests: (The pr folder is the one u defined above)
$ git fetch origin
From github.com:xxx/xxx
* [new ref] refs/pull/<id1>/head -> origin/pr/<id1>
* [new ref] refs/pull/<id2>/head -> origin/pr/<id2>
...