I have written a Jenkinsfile for use in a GitHub Organization object in Jenkins. Part of the Jenkinsfile is error correcting, and often makes changes to the project. At the end of the pipeline, Jenkins commits the changes to our git repository.
Now that I have all of this working, I have realized that upon pushing the corrections to the repo, the same job that just ran and corrected the errors will run again, since it is triggered by webhooks. It wouldn't be an infinite loop because there would be no errors to correct after running the first time, but it would run one more time than it needs to.
The fact that the changes made by the job end up triggering the job is problematic because the pipeline includes a deploy step. Redeploying the same code twice in a row feels like the result of a poorly written pipeline. That said, short of searching the last commit message for keywords, I cannot figure out a good way to make commits from the job not trigger the job. Is there a less hacky way to solve this problem?
edit: I am aware of the ci-skip plugin, but I can't documentation for making it work in a declarative pipeline. I would prefer an approach that lets me do something with the default scm checkout step at the start of the build.
edit 2: I ended up just running a grep of the last commit message everytime the pipeline starts to see if the message contained the text [ci skip]. Its dirty, but it works if its placed in a setup stage of the pipeline.
In the configuration to checkout a git repo, you can specify there to ignore certain commit messages (advanced checkout behavior) Rather than using code to do it. However, I'm not sure if if a github org job can do the same thing.
Another option would be to ignore certain directories if the changes are confined to certain directories, and those directories aren't normally changes otherwise. Again, not sure if github org supports this.
Don't use webhook triggeers, possibly, but this isn't a preferred option.
Related
I'm doing Visual Studio builds on a self hosted agent, which are currently being triggered by the Continuous Integration setting in an Azure Devops pipeline.
When a build completes, it shows: Triggered by Microsoft.VisualStudio.Services.TFS
It also shows the repository, branch and revision number.
However, it is expected it would show Triggered by , If not showing the correct Azure Devops user, at least showing the Subversion user name, that would be something.
There was an expectation it would be possible to send email notifications to the user of the commit. (Not fool proof that they caused the problem, but the most convenient way to give the responsibility to somebody to make sure any build error gets resolved)
Does anybody know if a solution exists?
In both Classic and yaml pipelines, you can specify a condition for a pipeline step. If you want it to run when the pipeline fails, it will be condition: failed() (in yaml), or Control Options -> Run this task -> Only when a previous task has failed (in Classic). Alternativel, you can check Agent.JobStatus variable.
there's no predefined variable for current committer, but you can easily determine the last commit's author by using svn command, then log it. (any other version control system will have its own CLI that should allow it).
In yaml, it could look like this (using git instead of svn):
steps:
... (your build)
- bash: |
author=`git log -1 --pretty=format:'%ae'` # get last commit author from git
echo "last commiter: $author"
# TODO: send email or other kind of notification
condition: failed()
In classic one:
You are using wrong tool for you task. CI build will be triggered after changes was committed to branch. In that case it is not possible to fix those changes. As a result you will have history where a lot of revisions are not stable.
It might be more suitable for you to use PR policy build. It is designed to validate incoming changes so target branch will be always stable and ready to some deploy. In this case, policy build will be triggered by PR creator so he will be informed about it's result. That can be configured in personal notification settings.
In the end I couldn't Continuous Integration triggers to reliably work. They would always stop working after a short time. I'm surprised I have ran into so many issues with this, but I guess it just isn't that well supported.
Instead, now, I am queue the build via an svn post-commit hook which uses the azure devops REST API.
the REST API has setting, requestedFor":{"id":""}, where you can add the user id (which I also needed a rest api command to find)
A lot of messing around to get to this point, for a feature I expected to 'just work', hopefully this keeps working
So, I've been using Jenkins for quite a while. I have set up numerous projects with the Github Pull Request Builder plugin to run tests whenever someone opens a pull request, and then trigger some other job (build, push, deploy, etc) whenever the pull request actually gets merged to master.
So, is there any way to set this up with a Jenkinsfile, or the organization folders, or the multibranch build deal?
The github-organization-folder plugin in combination with the multi-branch plugin plugin offers exactly this awesome feature: It scans a whole organization (optionally restricted to certain patterns in repo/branch names) for Jenkinsfiles and automatically adds jobs. This also happens for Pull Requests.
Once the PR is closed, it automatically removes the job.
To avoid arbitrary code execution, an organization member has to trigger building the job (same as for the GPRB plugin). The phrase can be configured in the Jenkins System settings.
EDIT: Under the Advanced section in Jenkins, you find options about what types of PR you want to build. If you build fork PRs, then there's afaik no way to prevent running code without prior inspecting it.
An example, how this looks like:
In a normal freestyle project, I configure the SCM plugin to point to the Git repo that I want to release, and I enable the "Poll SCM" option, which allows me to configure a Stash webhook to tell Jenkins whenever there has been a change to that repo. In this way, the job can be triggered whenever a change is pushed to the repo.
But when I use a workflow instead of a freestyle project, the SCM of the code that I need to build is specified programmatically in the groovy workflow script, which means that it is not listening for the Stash webhook. Instead, the SCM that is configured directly in the workflow is the SCM of the groovy script itself, which is different than the codebase that I am trying to build/release, so I don't want the trigger to be based on that.
node('docker_builder') {
git url: serviceRepo
releaseVersion = getVersion()
pipelineSpec = getPipelineSpec()
sh "./gradlew clean build pushDockerImage"
}
Any ideas about how to achieve SCM polling when using the workflow plugin?
I have resolved this question with lots of research and experimentation. This documentation got me on the right track: https://github.com/jenkinsci/workflow-scm-step-plugin/blob/master/README.md. It says:
Polling is supported across multiple SCMs (changes in one or more will trigger a new build), and again is done according to the SCMs used in the last build of the workflow."
This means that SCM polling is still supported with a Jenkins workflow, but unlike a normal freestyle project, you have to run it once manually before it starts listening for SCM changes. This makes sense because the SCM's are defined in Groovy code; they are not known until they run once.
One tricky element of this is that you can define many SCM's in your workflow. For example, I have three: one for the service itself, a deployment script, and the Groovy workflow DSL. By default, changes to any of those three SCM's would cause the "SCM poll" option to trigger a build, which may not be desirable. Luckily, setting the "poll: false" option on the "git" step in the Groovy code will disable polling on that repo. If you are reading your Groovy DSL from an SCM, then you can disable polling on that repo by clicking "additional behaviors" in the Jenkins UI and adding "Don't trigger a build on commit notifications".
Another tricky element is that the Stash web hook plugin by default includes the SHA1 hash code of the commit in the RESTful URL that it hits Jenkins with. Unfortunately, Jenkins makes the mistake of using that same commit code when it tries to pull any of the multiple SCM's that you may have defined. The hashcode is of course only relevant to one SCM, so it breaks. You can get around this by setting "Omit SHA1 Hash Code" in the Stash web hook plugin. Then Jenkins will just use the latest commit on whatever branch you build from in each of your SCM's.
I have created a build configuration in TeamCity (v8.0.1) to automatically build my Github pull requests by setting my VCS branch specification to +:refs/pull/(*/merge) (I followed this blog post).
TeamCity is picking up new pull requests and changes to existing pull requests just fine, however it is kicking off multiple builds per change when I would expect it to only perform a single build.
For example, I pushed up two commits to an existing pull request which resulted in 8 builds being executed by TeamCity. When I look at the changes for each of the builds the change graph is identical except for the commit hash of the pull request's build (at least I think that is what it is), e.g.
I have confirmed that Per-checkin Triggering is disabled in the Build Trigger configuration. I have also confirmed that we are not using any of Github's TeamCity hooks.
Any ideas what the problem might be?
I'm not sure what could cause this, but have you tried to turn on "Quiet Period"
http://confluence.jetbrains.com/display/TCD8/Configuring+VCS+Triggers
Currently, my team is using TeamCity to automatically build pull requests from GitHub.
We have a configuration to build all the pull requests. In the version control settings of the config, our branch specification is
+:refs/pull/*/merge
In the "Build Triggers" configuration setting, we have only one trigger with the following trigger rule:
+:root=Pull Requests on our Repository:\***/*\*
"Pull Requests on our Repository" is our VCS root name.
The issues:
When someone views a pull request on GitHub website without doing anything else, a build would be triggered in the TeamCity build agent. This is quite annoying, because from time to time, we have multiple build agents building the same pull requests (when multiple people view it).
When someone comments on a pull request, a build would also be triggered.
From my perspective, the only time I want TeamCity to start a build is when new commits are pushed to the pull requests.
Is there a way to do it?
Github's refs/pull/*/merge branches are updated every time mergeability of the branch is recalculated, i.e. on every commit to destination (most likely master) branch. They are also updated when pull request is closed and then reopened. Github's support says these branches are not intended for end users use. The only workaround at the moment is to run builds on refs/pull/*/head branches automatically and on refs/pull/*/merge branches manually.
Do you have TeamCity configured as per this blog post? I then activate the TeamCity service hook in GitHub which takes care of triggering a build in TeamCity whenever there is a push. This seems to do the right thing for me. Or am I missing something?
I know this is old but I wanted to post what we've found as alternatives:
Stop using VCS roots altogether as a mechanism for triggering pull requests. Instead, configure a GitHub webhook to notify a web app of yours whenever there is an update to a PR and only then trigger a build via the TeamCity REST API.
In your build config, add a step that checks what changed in the PR. If nothing changed (i.e. no new commits were added), or if the PR is closed, cancel the build. The problem with this is that the build queue will still be populated with builds that will then be cancelled. Also, you'd have to store somewhere the last commit that was built in order to do the check.
According to their TeamCity's issue tracker, the issue of the TeamCity.GitHub plugin causing an infinite loop of builds was fixed in v9.0