I am new to Jenkins, was curious to know how does the Jenkins poll feature works and how does it utlize system resources.
What could be best practice to use in CI system - Jenkins poll or github webhook and why?
This is a very easy answer, answered by Kohsuke, the creator of jenkins himself. Polling is less efficient, both for your computers, and for your people, since you have to wait--on the order of minutes--for your builds to happen, instead of them happening essentially instantly. We used to use polling before we switched to jenkins, and the biggest thing that people talk about is that "holy crap, jenkins builds fast." It's not jenkins, it's webhook pushing to trigger builds. Do that.
Related
The default Github CircleCI webhook integration runs our tests for every push to github. We probably really only care about running our CCI tests for 1/50th of these pushes. But since we pay for every run on CCI, this is costing us a lot more money than we need to be spending.
How can I re-configure the webhook to run less often? Is there a way to run it manually or have some kind of keyword detection in the "last" commit message?
What's a good solution here?
CircleCI has two built-in options.
You can use the string [skip ci] in a commit message and CircleCI won't build it.
In Project Settings, you can turn on a setting that will only build PRs. With this feature on, regular commits before a PR is open will not run.
Keep in mind, just some advice, I get the desire to save money, but reducing how often you test reduces the value received from CI. Just keep that in mind.
I started studying GitLab for my boss's order.
I found GitLab supports built-in CI/CD features.
But I don't want CI feature. Only to deliver code manually.
...and the concept of CI/CD is too difficult to me.
Of course I am going to search this topic continuously,
but also worried that it is impossible only to deliver code from GitLab Web repository to remote server.
Anyone who know this question?
Please help me.
Gitlab is one of the most flexible platforms to code versioning and CI/CD
almost everything is possible, but of course some things needs more expertise than other
In your case, delivering code manually will work without any problem.
But you need to understand more about git concepts
look at this article https://thepilcrow.net/explaining-basic-concepts-git-and-github/
forget the differences between github and gitlab now.... focus on understanding git (gitlab is just an inferface)
After that, when you start to study ci/cd you will discover the power of gitlab :D
GitHub announced an upcoming feature, GitHub Actions.
I'm positive on the benefits of CI tools like Jenkins for automatic building or testing, which GitHub Actions is aimed to be used for in the future.
Having a repository on GitHub and using an external CI tool has the huge benefit of allowing to move the repository to another Git repository platform (or even local) without rewriting of the whole CI process. With GitHub Actions, you're more or less tied to the GitHub ecosystem.
I assume the integration of GitHub's Actions will be more fluent in the native environment, but are there any other advantages or disadvantages besides that?
I've been working with GitHub actions full time for a couple of months now.
It's still early days (June 2019), but here's my list:
Advantages:
GitHub actions are just consecutive docker runs. Very easy to reason about and debug.
Reproducing the build environment for container-based Travis is
possible,
but more difficult.
On GitHub actions it's just a docker build docker run away.
The individual actions in a workflow are isolated by default.
You can use a completely different computing environment for, say, compilation and testing.
Travis CI (and I think other "traditional" CI) would run all "stages" (~ actions) in the same computing environment.
Again, GitHub actions are much easier to reason about and debug.
The main.workflow spec (a subset of the HCL and really just a directed acyclic graph) is open source.
The whole thing is a pretty thin wrapper around Docker anyway, so platform lock-in is arguably minimal.
There are already open-source reimplementations of GitHub actions, such as act for local testing.
You have ready access to the GitHub API with (somewhat limited) authentication out of the box.
There might be a vibrant community (marketplace?) where people can share actions.
For example, I'm reusing deploy actions build by different people in different ecosystems.
A directed acyclic graph (DAG) and the visual editor for main.workflows is perhaps a good way to model CI/CD in particular and workflows in general.
Takes some getting used to, but generalises well.
GitHub actions can do a whole lot more than just CI! You've got basically the whole API at your fingertips as inputs and outputs.
Disadvantages:
GitHub actions (still?) has sometimes surprisingly foundational limitations at this point (june 2019).
No native caching.
You get image and layer caching (it's complicated), but nothing else.
For build artefacts, you have to roll your own cache (via AWS, Azure, etc. ...), which can be a lot of work. (You can see a hacky setup here.
Surprisingly, no support for pull requests from forks.
It's again a bit complicated, and understandable from a security standpoint, but it's currently not possible to run actions a) against the secrets of the receiving repo of a fork PR (base), and/or b) against the would-be merge result of a fork PR (that's what travis does).
For a workflow that involves forks, that makes GitHub actions largely unuseable as CI/CD tool.
Single platform, it's just whatever you can run inside docker, so some Linux distro. That seems unlikely to change, but might be an acceptable limitation.
You can always add an action to call other cross-platfrom CI/CD services.
The documentation is still pretty sparse.
Not much in the way of best practices or scaffolding.
The quality and breadth of published GitHub actions (at least on the marketplace) is still pretty low / limited.
We'll see whether that takes off.
No great way to unit-test actions. (I hacked something together, but I'm not too sure about it).
Having a repository on GitHub and using an external CI tool has the huge benefit of allowing to move the repository to another Git repository platform (or even local) without rewriting of the whole CI process.
With GitHub Actions, you're more or less tied to the GitHub ecosystem.
Yes, and starting November 2019, slightly less so:
See Joe Bourne's annoucement "Self-hosted runners for GitHub Actions is now in beta".
You can have self-hosted runners, which means:
Your environment, your tools,
Any size machine or configuration,
Secure access and networking,
Large workload support.
To support using self-hosted runners in your workflows, we’ve expanded the experience of using the runs-on key.
When registering your self-hosted runners, they’re each given a read-only label self-hosted which you can use with runs-on.
Here’s an example:
# Use Any available Self-hosted runners connected to repo
runs-on: self-hosted
See the documentation at "Hosting your own runners".
I've set up my own GitLab CE server with CI in it. I can already create a webhook to deploy my code to a server just by pushing it. (Many thanks to Matt Jones and his little invention).
However, I still have one issue there. I don't really find a very good description about GitlAbs new feature about Webhook for Build Events. I think I'll need this, since I wish to deploy my code ONLY AFTER a build was succesfull. If I set the webhook for push event, then it has no problem, inmediatly deploys the code. Can any of you provide me with a proper instruction, what do I have to make, to achieve my goal?
Thanks a lot in advance!
I use jenkins for these purposes, you can also set your preferences about when to deploy your code and run pre and post build steps. It's really useful and it has Gitlab integration with the Web hooks.
I followed this answer to set up a Jenkins job, and it's working fine.
I have scheduled a Job on github master commit push as
Poll SCM : * * * * *
But, it continuously starts a build Job each minute.
How can I restrict this so that it runs only once per commit push?
There are several options. The two I've used with most success are:
Use a git commit hook to call the Jenkins rest API to start the job. The simple approach is to call the job's build API call directly (something like http://jenkinsmachine:8080/job/your-jobs-name/build), but that hardcodes the job name and branch into the git hook script. A more flexible approach is to use the Git plugin's own rest mini-API, as described by Kohsuke in his blog.
Use something like the Throttle Concurrent Builds plugin or a creative use of slaves nodes and executors to limit the number of cuncurrent builds of that job.
The first option is much preferred, but there are times when rest access to the jenkins machine from the git machine is not available, so the second option can be used in those circumstances.
This approach is actually polling. That means Jenkins is scanning every minute if there aren't any changes in GitHub repository.
If you want true Push from Github to your Jenkins You need to Integrate Github WebHooks with Jenkins. I wrote a blog post on this subject. Scroll to section 2: "Jenkins - Github Integration"
If you are just playing around or using it for your personal open source project, you may want to look into Jenkins alternatives like Drone.io or Codeship.io. These services are generally free for open source and can configure Github Webhooks in few clicks. But they are not suitable for complicated enterprise builds.