I'm looking into using Quartz Scheduler, and I was wondering if it was possible to schedule jobs not by time, but when another job finishes. So, when Job A is done, it starts Job B. When that's done, it starts Job C, etc.
Job A -> Job B -> Job C -> Job A... continuously.
Is this the right tool for the job? Or should I be looking into something else?
Check out JobChainingJobListener, built-in to Quartz (bold mine):
Keeps a collection of mappings of which Job to trigger after the completion of a given job. If this listener is notified of a job completing that has a mapping, then it will then attempt to trigger the follow-up job. This achieves "job chaining", or a "poor man's workflow".
That's right, you are looking for a process or workflow engine. Have a look at activiti or jbpm.
You may want to check the QuartzDesk project I have been involved in. QuartzDesk is a management and monitoring platform for Quartz-based apps and in version 2.0 we have added a new job chaining engine to the platform.
The engine allows you to orchestrate the execution of your jobs and there is no need to modify your application code in any way. Job chains can be dynamically updated through the QuartzDesk GUI without any disruption to your application.
Related
I'm wanting to setup an ECS task to schedule various other application tasks.
The "tasks" this task will schedule will mostly involve calling restful endpoints in another load balanced service.
I know there are other ways to do this, using cloudwatch to trigger a lambda etc. However this seems overly complex for what I need.
I was planning to just make a very simple, light-weight apline based image with a crontab to do the triggering of the restful calls.
This all seems easy enough. The only concern I have is that I would want to prevent, as far as possible, having multiple instances of this task running, even if only for a short period of time.
If my CI/CD pipeline triggers an update to this cron task, then there may be a short period of time, where the old and new task will be running simultaneously.
There may therefore be a small chance that a cron task could be triggered twice.
What I would like to do, is to have ECS stop the currently running task completely, before attempting to start the new one.
This seems to be contrary to the normal way it wants to work, where it will ensure the new task is up, and healthy before stopping the old one.
Is this possible, and if so, how do I configure it?
It's not a problem if my crons don't run for a period of time, but it could be a problem if any get triggered more than once.
Instead of using ECS Service (which makes sure a particular number of tasks is always running and deploys via rolling or B/G deploy strategy which is not you desire) - how about using StopTask and RunTask api to control when a task is stopped and started - gives you complete control.
Instead of using scheduled tasks, you could create an ECS service and use scheduled scaling to scale the desired service count to 1 and back down to zero.
I am using quartz to schedule jobs to be executed daily as a part of a much larger web application. However, after a couple of days, the administrator would like to stop the execution of a particular job (maybe because it is no longer needed). How do I go about doing this? I read the api docs for the Scheduler and it has a method called interrupt(JobKey jobkey) but that method would work only with the same instance of the scheduler that was used to schedule the job.
interrupt(JobKey jobKey)
Request the interruption, within this Scheduler instance, of all
currently executing instances of the identified Job, which must be an
implementor of the InterruptableJob interface.
Is there anyway of getting the instance of an existing scheduler? Or maybe use singletons?
Should definitely use a singleton instance of your scheduler. I recommend the use of an IoC container to manage this in a clean and efficient way.
I'm looking for a lightweight system that will let me queue up a one-off (non-recurring) task and have it execute at a specific time in the future.
This is for the backend of a game where the user does tasks that are time-based. I need the server to check the status of the user's "job" at the completion time and perform the necessary housekeeping on their game state.
I'm somewhat familiar with Redis, Celery, Beanstalkd, ZeroMQ, et al., but I haven't found any info on scheduling a single unit of work to be executed in the future. (or pop off the queue at a set time) Celerybeat has a scheduler for cron-type recurring tasks, but I didn't see anything for one-off.
I've also seen the "at" command in *nix, but I'm not aware of any frontend for it that can help me manage the jobs.
I realize there are some easy solutions such as ordering keys in Redis and doing a blocking pop, but I'd like to not have to continuously poll a queue to see if the next job is ready.
The closest I've found is the deferred library on GAE, but I was hoping for something that runs on my own Linux box along with my other components.
I'd appreciate any suggestions!
Celery allows you to specify a countdown or an ETA at the call of a task to be executed.
The documentation says it best:
http://docs.celeryproject.org/en/latest/userguide/calling.html#eta-and-countdown
I have several web-servers and need them to use Quartz. The clustering feature of Quartz would be ideal, but it requires that the servers clocks are in complete sync. They have a very scary warning about this:
Never run clustering on separate machines, unless their clocks are synchronized using some form of time-sync service (daemon) that runs very regularly (the clocks must be within a second of each other).
I cannot guarantee complete clock synchronization, so instead of using the clustering feature I was thinking to have a single Quartz instance (with a stand-by for fail-over). Having a single instance executing jobs is not a problem, but I still need all of the web servers to be able to schedule jobs.
Can I directly add jobs into the JDBCJobStore from the web servers, and will they be picked up by the (non-clustered) Quartz server? I would be doing this by creating schedule instances in the web servers to add jobs. These instances would never be started, just used to access the JobStore.
Wrote a test program that creates a "non-clustered" Quartz scheduler using the same JobStore as the "real" scheduler (also non-clustered), and schedules jobs. After a few seconds, these jobs do get executed, so it seems to work.
Update: I cross-posted this question to the Quartz forum, and got the answer that this should work. In a related question they state that
The jobs can be inserted into that database by another process by:
1- using the rmi features of quartz from another process, and using the quartz API
2- instantiating a scheduler within another process (e.g. webapp), also pointing it to the same database, but NOT start()ing that scheduler instace, and the using the quartz api to schedule jobs.
I have to move an SAP background job (ABAP report for A/P) into Cronacle and can't figure out how to stop the job in SAP so I can start running it with the Cronacle schedule.
The job runs in SAP from user SAPSYS every morning at 7:15am, but if you look it up with sm37 there is no time scheduled for it and it's not triggered by an event; also, it's status is SCHEDULED.
I had our Cronacle team search by job number but they couldn't find any scripts pointing to that job. If you look at the finished job it shows that it's scheduled daily for 7:15am. Also, there is no predecessor or successor jobs listed. Is it possible it's being started from another job? How do I find out without deleting this one?
Some suggestion.
If you don't want to delete the scheduled job. try to rename it, and see if it still runs.
Make sure that the users you are using for sm37 has full authorization for the backround administration.
A previous job can schedule and release and create and whatever a new job. Look at what is running before the problematic job.
Look deeply at the dev traces. They somtimes hints about what is going on in the system.
In addtion to a previous job creating the new job explicitly it is also possible that the job is created manually by an ABAP program that is scheduled in another job. Doing a where-used on the function module OPEN_JOB and looking for Z* or Y* programs may give you a hint.
Another thing: Is this scheduled job ever actually excecuted (i.e. are there any previous "FINISHED" jobs with the same name). A Scheduled job will not run unless it is first released. So if it never runs it may be obsolete.
Thanks for the responses! It turned out to be a case of "newbie ignorance." When using SM37 to view the job I neglected to extend the search date to the next day. I don't know why it doesn't show the released job for the current day, but extending it to the next day showed it. That's a lesson I won't forget!