Are there Distributed Schedulers available for free commercial use?
I saw Quartz Job Scheduler but its distributed version comes at a premium.
Thanks
You can use hinemos http://hinemos-en.atomitech.jp/ .You can control scheduling from GUI .Though spring schedular,quartz schedular provides scheduling but they provide from API level .You have to write a code for scheduling .But hinemos provides a GUI to control yours scheduling.
http://hinemos-en.atomitech.jp/ You can configure for different cluster and provide a scheduling command via GUI.
What are the available alternatives to Quartz?
There are no known competing open source projects (there are a few other open source schedulers, but they are basically just Cron replacements written in Java).
Commercially, you will want to look at the well-regarded Flux scheduler.
Related
We are on the way of replacing Control M with JS7 scheduler which is an open source for scheduling jobs in our EBS application. How do we test the high availability feature with minimal efforts in this scheduler?
We are on the way of migrating jobs to JS7 in test environment.
The high availability feature of JS7 is only available with the commercial license. If you have a commercial license you should be able to ask the people at JS7.
We have a kind of evaluation job which consists of several thousand invocations of a legacy binary with various inputs, each of which running like a minute. The individual runs are perfectly parallelizable (one instance per core).
What is the state of the art to do this in a hybrid cloud scenario?
Kubernetes itself does not seem to provide an interface for prioritizing or managing waiting jobs. Jenkins would be good at these points, but feels like a hack. Of course, we could hack something ourselves, but the problem should be sufficiently generic to already have an out-of-the box solution.
There are a lot of frameworks that helps managing jobs in Kubernetes cluster. The most popular are:
Argo for orchestrating parallel jobs on Kubernetes. Workflows is implemented as a Kubernetes CRD (Custom Resource Definition).
Airflow - has a modular architecture and uses a message queue to orchestrate an arbitrary number of workers. Also take a look for kubernetes-executor.
I recommend you to look for this video which describe each of framework and help you decide which is better for you.
You may be interested in following aricles about using Mesos for Hybrid Cloud
Xue, Noha & Haugerud, Hårek & Yazidi, Anis. (2017). Towards a Hybrid Cloud Platform Using Apache Mesos. 143-148. 10.1007/978-3-319-60774-0_12.
Hybrid cloud technology is becoming increasingly popular as it merges private and public clouds to bring the best of two worlds together. However, due to the heterogeneous cloud installation, facilitating a hybrid cloud setup is not simple. Despite the availability of some commercial solutions to build a hybrid cloud, an open source implementation is still unavailable. In this paper, we try to bridge the gap by providing an open source implementation by leveraging the power of Apache Mesos. We build a hybrid cloud on the top of multiple cloud platforms, private and public.
Apache Mesos For All Your Hybrid Cloud Needs
Choosing the Best Approach to Hybrid Cloud
Is there any possibility of applying DevOps concepts to build and Deploy Dell Boomi Processes.
Dell Boomi recommends usage of Atmosphere for change management activities, it also says its a UI based tool and we cannot export the code to a source control but have the following questions
Do we have any option to automate deployments across environments?
What kind of testing can be performed and at what stages?
As far as I know there is a possibility to create a cross reference table with many columns including names of the process execution id, dates, action and so on and use it as a tool to deploy many processes from one place.
For the moment i was reading about it there was an issue because many developers who had an access to deployment could override values which made a lot of chaos.
Instead you can use 3rd party tool like jenkins.
https://www.youtube.com/watch?v=mf5MsQtEa5o
above example Boomi with Jenkins
You can use the AtomSphere API to automate deployments. In effect, you'll use a series of API calls to tell AtomSphere to automate the deployment of a given version of a package to an environment. This step is safe to repeat in many environments, and this way you can deploy to an arbitrary number of environments.
At a strategy level, testing Boomi is like any other integration tool. You should have unit tests, integration tests and systems tests. There are a range of techniques for each. Dell have some wiki pages that talk through unit testing approaches, specifically.
both have very complementary features and it would be great to be able to have the schedule flexibility of Quartz, with the distributed computing of Apache Mesos.
I couldn't find anything on google.
Has anybody tried this?
Does anyone know of a paid service that allows you to load up queue of url based tasks that are then run in realtime? I'm looking at setting up a local task queue, but I'm willing to pay for it.
Beanstalkd is easy enough to install on Linux, or there is a SAAS compatible version - http://www.iron.io/products/mq
Beyond that, Amazon SQS has various libraries that make it quite easy to use, though it uses HTTP to get/set items in the queue, so it has an additional overhead.