The default Github CircleCI webhook integration runs our tests for every push to github. We probably really only care about running our CCI tests for 1/50th of these pushes. But since we pay for every run on CCI, this is costing us a lot more money than we need to be spending.
How can I re-configure the webhook to run less often? Is there a way to run it manually or have some kind of keyword detection in the "last" commit message?
What's a good solution here?
CircleCI has two built-in options.
You can use the string [skip ci] in a commit message and CircleCI won't build it.
In Project Settings, you can turn on a setting that will only build PRs. With this feature on, regular commits before a PR is open will not run.
Keep in mind, just some advice, I get the desire to save money, but reducing how often you test reduces the value received from CI. Just keep that in mind.
Related
The scope of tis question is GitHub repositories.
Is there an easy option to enforce a few code quality rules before a commit can be done?
I have linting tools that I can run in a GitHub action runner. I hope I can run the linting scripts and prevent the commit if the lint program has an error output.
I think I am missing a pattern or special GitHub feature here. A link to documentation would be a great help.
If you're using github.com (that is, not a GitHub Enterprise Server instance), then you cannot run hooks on the server side. That's because hooks execute arbitrary code, which is of course a security concern.
The easiest way to do this is to run a CI job, such as you have in the GitHub Actions runner. You'd create the workflow file that runs the linting tools, such that the linting tool or your check script exits nonzero if the commit should be rejected. You can see the workflow file for Git LFS that runs script/cibuild to perform this kind of check.
Once you have your CI job working as you expect, you can protect the branch by going into the repository settings. Choose Branches → Branch Protection Rules, and create a check for whatever branch you want to require checks for. Usually this is your default branch. Choose “Require status checks to pass before merging” and then enable the relevant checks to make them required.
From that point on, in order to merge code into the protected branch, the checks must pass. If you want to require this for all branches in the repository, then you can use a wildcard, but you'll need to use a forking model since this will prevent people from pushing branches into the repository.
Note that this is much better than trying to use a pre-commit hook on the user system. As the Git FAQ explains, pre-commit hooks are easily bypassed, so while you can provide them for developers who want to use them, you can't require them. pre-commit hooks are also an impediment to certain workflows, so advanced developers may not care for them.
I have a branch policy for build validation. As a final step i am wanting to update a file in the pull request and push it into the branch. it doesn't appear that [skip ci] works for these build triggers. is there another way to accomplish this quality gate?
deeper part of scenario. pull request is building a docker image, and tagging with git hash. i'm updating a yaml file with the new tag so i only have to build the image once. at some point i want to set up another pipeline that will deploy this image based off a successful run of this pipeline(build validation)
does skip ci work with build validation pipelines?
The answer is No.
First, I need to indicate that this is two different scenarios. As we know, the [skip ci] or [ci skip] is used to skip running CI, like the option
Enable continuous integration on UI:
However, our current scenario is branch policy for build validation instead of CI. This is very different from CI, although they seem to be doing the same build task. Branch policy is to protect our branches from being corrupted by incorrect submit. This is a verified operation instead of continuous integration.
Check the document Skipping CI for individual commits for some more details.
So, this is two different scenarios, we could not apply the CI settings to the branch policy.
Second, just like I said in the first point, Branch policy is used to protect our branches, any commit requires validation by branch pliocy, although sometimes we can know that our modifications don't require build validation, but we're not sure if there are any where we overlook that cause our target branch to be broken. Skip unnecessary verification will bring us some construction convenience, but with the risk measurement it brings, these conveniences are negligible, so we don't recommend skipping the verification of the branch office strategy.
Hope this helps.
I have written a Jenkinsfile for use in a GitHub Organization object in Jenkins. Part of the Jenkinsfile is error correcting, and often makes changes to the project. At the end of the pipeline, Jenkins commits the changes to our git repository.
Now that I have all of this working, I have realized that upon pushing the corrections to the repo, the same job that just ran and corrected the errors will run again, since it is triggered by webhooks. It wouldn't be an infinite loop because there would be no errors to correct after running the first time, but it would run one more time than it needs to.
The fact that the changes made by the job end up triggering the job is problematic because the pipeline includes a deploy step. Redeploying the same code twice in a row feels like the result of a poorly written pipeline. That said, short of searching the last commit message for keywords, I cannot figure out a good way to make commits from the job not trigger the job. Is there a less hacky way to solve this problem?
edit: I am aware of the ci-skip plugin, but I can't documentation for making it work in a declarative pipeline. I would prefer an approach that lets me do something with the default scm checkout step at the start of the build.
edit 2: I ended up just running a grep of the last commit message everytime the pipeline starts to see if the message contained the text [ci skip]. Its dirty, but it works if its placed in a setup stage of the pipeline.
In the configuration to checkout a git repo, you can specify there to ignore certain commit messages (advanced checkout behavior) Rather than using code to do it. However, I'm not sure if if a github org job can do the same thing.
Another option would be to ignore certain directories if the changes are confined to certain directories, and those directories aren't normally changes otherwise. Again, not sure if github org supports this.
Don't use webhook triggeers, possibly, but this isn't a preferred option.
I am new to Jenkins, was curious to know how does the Jenkins poll feature works and how does it utlize system resources.
What could be best practice to use in CI system - Jenkins poll or github webhook and why?
This is a very easy answer, answered by Kohsuke, the creator of jenkins himself. Polling is less efficient, both for your computers, and for your people, since you have to wait--on the order of minutes--for your builds to happen, instead of them happening essentially instantly. We used to use polling before we switched to jenkins, and the biggest thing that people talk about is that "holy crap, jenkins builds fast." It's not jenkins, it's webhook pushing to trigger builds. Do that.
The only similar question I found is the following: Trigger travis ci builds if another git repository updates
Keep in mind that our E2E tests are full stack: we have a live server running, and our tests are run against the UI which hits the server. Nothing is stubbed or mocked.
Now, I can trigger in that way a travis rebuild, however problem arises when I have interdependency between branch and a different repository branch.
So let's say I have 3 repositories: backend, frontend and e2etests. If I create the new branch frontend/foo which requires also the backend/foo, there is no way e2e tests will ever pass because they will run one time with frontend/foo and backend/master and another time with frontend/master and backend/foo.
Did anyone face this problem previously? How did you deal with it?
After a lot of digging it was clear that there is no solution and everyone has their own built-in.
What worked for us is a git pre-push hook, environment variables pointing to our repositories and when you push it asks if you want to use a different branch for the UI project (if you are on the UI, asks if you want to use a different branch for the server project) and by default it uses the current branch for the project you are on. So it's quite fast on push, usually you'll just press ENTER an additional time, the two branches are basically the required dependencies and a travis build will be triggered with an API endpoint.
Works fine so far.