I am writing a Quartz.net application using AdoJobStore to allow automated report scheduling.
In my scenario, users will define custom reports to be scheduled in one application which will add the required jobs and triggers to the database (using the AdoJobStore routines).
A separate Quartz.net application then reads these settings from the database (also using the AdoJobStore routines) and emails the reports as necessary.
Is there a way to get the quartz scheduler to automatically start scheduling new jobs and triggers that have been added to the database after the scheduler last started, or will I need to write a routine that periodically checks for database changes, and if found restart the Quartz scheduler instance?
You can handle all of this directly with Quartz.Net. Here's one way to do it:
Set up a Quartz.Net server as a windows service. The distribution comes with a Windows Service implementation, or you can build your own. Enable remoting on the quartz server.
From the application where users will configure their reports and schedules, connect to the Quartz.Net server using the Quartz.Net library and directly schedule the jobs and triggers as necessary.
You'll probably want to store the user's report configuration elsewhere in case the user wants to look at it later or change/copy it. Store this data somewhere else other than Quartz.Net. If the user changes the stored report configuration, connect again to the Quartz.Net server and update/reschedule the jobs using the Quartz.Net library. Alternatively, you could create a job that runs on the Quartz.Net server and periodically checks whether there have been any report configuration changes.
You'll have to create the actual jobs that will generate your reports in a generic enough fashion so that any report can be built by passing in data to job via the JobDataMap, instead of having to create a job for each report.
Related
We have some spring-batch jobs are triggered by autosys with shell scripts as short lived processes.
Right now there's no way to view what is going on in the spring-batch process so I was exploring ways to view the status & manage(stop) the jobs.
Spring Cloud Data Flow is one of the options that I was exploring - but it seems that may not work when jobs are scheduled with Autosys.
What are the other options that I can explore in this regard and what is the recommended approach to manage spring-batch jobs now?
To stop a job, you first need to get the ID of the job execution to stop. This can be done using the JobExplorer API that allows you to explore meta-data that Spring Batch is aware of in the job repository. Once you get the job execution ID, you can stop it by calling the JobOperator#stop method, please refer to the Stopping a job section of the reference documentation.
This is independent of any method you used to launch the job (either manually, or via a scheduler or a graphical tool) and allows you to gracefully stop a job and leave the repository in a consistent state (ready for a restart if needed).
We're trying to create a UI screen which will be able to trigger spring batch jobs and use our existing database with job execution records.
I was able to get all existing jobNames via jobExplorer but now I get an error on jobRegistry.getJob(jobName). It seems the jobs are not registered in jobRegistry.
The actual configuration of the jobs is placed in another application. I try to trigger the job from another application (solely for batch related functions - runJob, stopJob, view executions, etc).
EDIT:
Is it possible to be able to register the jobs to JobRegistry from existing batch metadata tables? - What I mean by this is that the jobs and step beans would be recreated from existing metadata tables.
What we did for a workaround is that execution records can be retrieved using the metadata tables but the runJob, stopJob functions would need to be redirected to exposed endpoints of the batch job applications.
I want to consolidate a couple of historically grown scripts (Python, Bash and Powershell) which purpose is to sync data between a lot of different database backends (mostly postgres, but also oracle and sqlserver) and on different sites. There isn't really a master, its more like a loose couple of partner companies working on the same domain specific use cases, everyone with its own data silo and its my job to hold all this together as good as I can.
Currently those scripts I mentioned are cron scheduled and need to run on the origin server where a dataset gets initially written, to sync it to every partner over night.
I am also familiar with and use Apache Airflow in another project. So my idea was to use an workflow management tool like Airflow to streamline the sync process and get it more centralized. But also with Airflow there is only a time interval scheduler available to trigger a DAG.
As most writes come in over postgres databases, I'd like to make use of the NOTIFY/LISTEN feature and already have a python daemon based on this listening to any database change (via triggers) and calling an event handler then.
The last missing piece is how its probably best done to trigger an airflow DAG with this handler and how to keep all this running reliably?
Perhaps there is a better solution?
I am working to migrate from Quartz 1.6 to 2.1 and use a JDBCJobStore. Previously, the the jobs were loaded via an xml file when the webapp started. The scheduler is now running using the JDBCJobStore but I don't understand how to add the jobs to the database which need to run on an ongoing basis (not one-off jobs).
My first thought is to create a servlet which runs on startup which adds the jobs to the database. But my concern is that this will be executed every time I need to restart the app and the jobs will get duplicated.
Thanks,
steve
The Jobs wont disappear from the database when you do a restart. So within your servlet, when it starts up before adding any jobs check to see if they already exist. When you create your jobs you can give them identities. Using the identities and some quartz methods you check if they already exist.
It sounds like the memory based scheduler is a better fit for these fixed jobs. You can create more than one scheduler, one memory, one JDBC if that makes sense for your application.
I am using Quartz to schedule cron jobs in my web application. i am using a oracle Databse to store jobs and related info. When i add the jobs in the Database, i need to re-start the server/application (tomcat server) for these new jobs to get scheduled. How can i add jobs in the database and make them work without restarting the server.
I assume you mean you are using JDBCJobStore? In that case it is not ideal to make direct changes in the database tables storing the job data. However, I suppose you could set up a separate job that runs every X minutes / hours, checks whether there are new jobs in the database (that need to be scheduled), and schedule them as usual.
Add jobs via the Scheduler API.
http://www.quartz-scheduler.org/docs/best_practices.html