My entire application's state depends on the quartz scheduler.
I am running a daily job which runs every 24 hours and updates the application state based on the logic written in job's execution function.
While testing, I found that sometimes when the system gets heavily loaded, scheduler do not work and the state does not change.
When I was running scheduler for every second and keeps system heavily loaded I was able to reproduce this case frequently.
Is there any way to ensure that quartz scheduler is running always?
Related
I'm wanting to setup an ECS task to schedule various other application tasks.
The "tasks" this task will schedule will mostly involve calling restful endpoints in another load balanced service.
I know there are other ways to do this, using cloudwatch to trigger a lambda etc. However this seems overly complex for what I need.
I was planning to just make a very simple, light-weight apline based image with a crontab to do the triggering of the restful calls.
This all seems easy enough. The only concern I have is that I would want to prevent, as far as possible, having multiple instances of this task running, even if only for a short period of time.
If my CI/CD pipeline triggers an update to this cron task, then there may be a short period of time, where the old and new task will be running simultaneously.
There may therefore be a small chance that a cron task could be triggered twice.
What I would like to do, is to have ECS stop the currently running task completely, before attempting to start the new one.
This seems to be contrary to the normal way it wants to work, where it will ensure the new task is up, and healthy before stopping the old one.
Is this possible, and if so, how do I configure it?
It's not a problem if my crons don't run for a period of time, but it could be a problem if any get triggered more than once.
Instead of using ECS Service (which makes sure a particular number of tasks is always running and deploys via rolling or B/G deploy strategy which is not you desire) - how about using StopTask and RunTask api to control when a task is stopped and started - gives you complete control.
Instead of using scheduled tasks, you could create an ECS service and use scheduled scaling to scale the desired service count to 1 and back down to zero.
I am using quartz to schedule jobs to be executed daily as a part of a much larger web application. However, after a couple of days, the administrator would like to stop the execution of a particular job (maybe because it is no longer needed). How do I go about doing this? I read the api docs for the Scheduler and it has a method called interrupt(JobKey jobkey) but that method would work only with the same instance of the scheduler that was used to schedule the job.
interrupt(JobKey jobKey)
Request the interruption, within this Scheduler instance, of all
currently executing instances of the identified Job, which must be an
implementor of the InterruptableJob interface.
Is there anyway of getting the instance of an existing scheduler? Or maybe use singletons?
Should definitely use a singleton instance of your scheduler. I recommend the use of an IoC container to manage this in a clean and efficient way.
I'm looking into using Quartz Scheduler, and I was wondering if it was possible to schedule jobs not by time, but when another job finishes. So, when Job A is done, it starts Job B. When that's done, it starts Job C, etc.
Job A -> Job B -> Job C -> Job A... continuously.
Is this the right tool for the job? Or should I be looking into something else?
Check out JobChainingJobListener, built-in to Quartz (bold mine):
Keeps a collection of mappings of which Job to trigger after the completion of a given job. If this listener is notified of a job completing that has a mapping, then it will then attempt to trigger the follow-up job. This achieves "job chaining", or a "poor man's workflow".
That's right, you are looking for a process or workflow engine. Have a look at activiti or jbpm.
You may want to check the QuartzDesk project I have been involved in. QuartzDesk is a management and monitoring platform for Quartz-based apps and in version 2.0 we have added a new job chaining engine to the platform.
The engine allows you to orchestrate the execution of your jobs and there is no need to modify your application code in any way. Job chains can be dynamically updated through the QuartzDesk GUI without any disruption to your application.
I have a building block which sets up a Quartz job to send out emails every morning. The job is fired three times every morning instead of once. We have a hosted instance of Blackboard, which I am told runs on three virtual servers. I am guessing this is what is causing the problem, as the building block was previously working fine on a single server installation.
Does anyone have Quartz experience, or could suggest how one might prevent the job from firing multiple times?
Thanks,
You didn't describe in detail how your Quartz instance(s) are being instantiated and started, but be aware that undefined behavior will result if you run multiple Quartz instances against the same job store database at the same time, unless you enable clustering (see http://www.quartz-scheduler.org/docs/configuration/ConfigJDBCJobStoreClustering.html).
I guess I'm a little late responding to this, but we have a similar sort of scenario with our application. We have 4 servers running jobs, some of which can run on multiple servers concurrently, and some should only be run once. As Will's response said, you can look into the clustering features of Quartz.
Our approach was a bit different, as we had a home-grown solution in place before we switched to Quartz. Our jobs utilize a database table that store the cron triggers and other job information, and then "lock" the entry for a job so that none of the other servers can execute it. This keeps jobs from running multiple-times on the servers, and has been fairly effective so far.
Hope that helps.
I had the same issue before but I discovered that I was calling scheduler.scheduleJob(job, trigger); to update the job data while the job is running which is randomly triggered the job 5-6 times each run. I had to use the following to update the job data without updating the trigger scheduler.addJob(job, true);
I have several web-servers and need them to use Quartz. The clustering feature of Quartz would be ideal, but it requires that the servers clocks are in complete sync. They have a very scary warning about this:
Never run clustering on separate machines, unless their clocks are synchronized using some form of time-sync service (daemon) that runs very regularly (the clocks must be within a second of each other).
I cannot guarantee complete clock synchronization, so instead of using the clustering feature I was thinking to have a single Quartz instance (with a stand-by for fail-over). Having a single instance executing jobs is not a problem, but I still need all of the web servers to be able to schedule jobs.
Can I directly add jobs into the JDBCJobStore from the web servers, and will they be picked up by the (non-clustered) Quartz server? I would be doing this by creating schedule instances in the web servers to add jobs. These instances would never be started, just used to access the JobStore.
Wrote a test program that creates a "non-clustered" Quartz scheduler using the same JobStore as the "real" scheduler (also non-clustered), and schedules jobs. After a few seconds, these jobs do get executed, so it seems to work.
Update: I cross-posted this question to the Quartz forum, and got the answer that this should work. In a related question they state that
The jobs can be inserted into that database by another process by:
1- using the rmi features of quartz from another process, and using the quartz API
2- instantiating a scheduler within another process (e.g. webapp), also pointing it to the same database, but NOT start()ing that scheduler instace, and the using the quartz api to schedule jobs.