Scheduling using Quartz - quartz-scheduler

Question

We have a .Net project that checks the existence of a file on a remote computer.
We need to execute this against multiple remote computers (thousands) within a department, each at predefined time, everyday. The execution time is specified in database which keeps changing frequently and the execution time for each remote computer will be different (or some of them could be the same).
To achieve this, we plan to use Quartz scheduler. Since we are new to Quartz, we would like to know how to achieve this. At a high level, we need these -
Scheduler should start off at a specific time everyday - Will Quartz do this?
Once started, it should fetch execution time for each remote computer from DB
Prepare an xml with list of all remote computers and their execution times
Schedule execution for each of the remote computers
Execute the .Net project at the scheduled time for each remote computer
What type of projects/components will be needed to achieve the above? We already have a .net class lib project that checks the remote computers. How do we integrate with Quartz?
UPDATE
Thanks a lot granadaCoder
As per my understanding, there's a main job that runs once a day (based on quartz.config) and schedules other jobs by fetching details from DB. In this case the console app neeeds to be running all the time...
What is your opinion on writing a console app and scheduling it using task scheduler to run at 12am daily?
Within console app, we'll prepare a (custom) xml containing the list of jobs (with details such as trigger time and data needed by job class) and pass it on to a scheduler module (a class lib project) that'll start scheduler & queue up all jobs in xml.
After scheduling all these jobs, we'll wait (inside scheduler module) for job completion notification from all the jobs and then shutdown scheduler and exit the console app. This may take long depending on the trigger time of the last job.
Let me know what you think about this approach.
In addition, we have multiple departments (4 in total), so I'm thinking of writing 4 console apps - one for each dept. And schedule all of them using task scheduler (at the same time as differnt timings may not help because each dept may have jobs with trigger time spanning entire day).
Alternatively, I'm also wondering if it is possible to specify 4 jobs in quartz.config file with the same trigger time? (not sure how this'll work, will it create 4 dept-specific scheduler instances and we can queue up department-wise jobs to each scheduler instance?)

Answer 1

Write a job defined in .xml that is the job that does your job cleanup and (re)scheduling. ("ScheduleOtherJobsJob" in the code below)
ScheduleOtherJobsJob will clear out any old entries. It read some datastore and figure out the list of new jobs it has to perform. It will add these jobs to the scheduler.
I've written a basic example. I don't have fancy when-to-run-based-on-exact-date logic....I just push into the future WithIntervalInSeconds.
I'm using GROUP_NAME to figure out which jobs to remove from the scheduler.....before re-adding them.
I show the .xml for the ScheduleOtherJobsJob. You could add it to a AdoStore, or whereever. The .xml is just simpler to put for an example.
Keep in mind that you'll need a process that keeps the Scheduler "alive". Aka, you just can't add new jobs to the IScheduler and then kill the hosting process. In a Console.App, you would write a "Console.Readline()"...so the program doesn't stop running.
The key points are......one job to schedule other jobs. Clearing out the older jobs based on some filter (GROUP_NAME here). Re-add the jobs based on some datastore......and making sure the host-process stays running so all the newly scheduled jobs will run.
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading;
using System.Threading.Tasks;
using Quartz;
using Quartz.Impl.Matchers;
namespace MyNamespace
{
public class ScheduleOtherJobsJob : IJob
{
private const string GROUP_NAME = "MySpecialGroupName";
/// <summary>
/// Called by the <see cref="IScheduler" /> when a
/// <see cref="ITrigger" /> fires that is associated with
/// the <see cref="IJob" />.
/// </summary>
public virtual void Execute(IJobExecutionContext context)
{
JobKey key = context.JobDetail.Key;
JobDataMap jbDataMap = context.JobDetail.JobDataMap;
// string jobSays = jbDataMap.GetString("KeyOne");
JobDataMap trgDataMap = context.Trigger.JobDataMap;
string triggerParameter001 = trgDataMap.GetString("TriggerFileName");
JobDataMap mergedMap = context.MergedJobDataMap;
string whoWins = mergedMap.GetString("DefinedInJobDetailAndTriggerKey");
string msg = string.Format("HasParametersJob : JobKey='{0}', jobSays='{1}', jobDetailParameter001='{2}', triggerParameter001='{3}', triggerParameter002='{4}' , whoWins='{5}' at '{6}'", key, jobSays, jobDetailParameter001, triggerParameter001, triggerParameter002, whoWins, DateTime.Now.ToLongTimeString());
Console.WriteLine(msg);
/* */
context.Scheduler.UnscheduleJobs(GetAllJobTriggerKeys(context.Scheduler));
/* Schedule Your Jobs */
List<OtherJobInfo> infos = new OtherJobInfoData().GetOtherJobs();
foreach (OtherJobInfo info in infos)
{
ScheduleAHasParametersJob(context.Scheduler, info);
}
}
private IList<TriggerKey> GetAllJobTriggerKeys(IScheduler scheduler)
{
/* Find all current jobs.....filter here if need be */
IList<TriggerKey> returnItems = new List<TriggerKey>();
IList<string> jobGroups = scheduler.GetJobGroupNames();
//IList<string> triggerGroups = scheduler.GetTriggerGroupNames();
IList<string> filteredJobGroups = jobGroups.Where(g => g.Equals(GROUP_NAME)).ToList();
foreach (string group in filteredJobGroups)
{
var groupMatcher = GroupMatcher<JobKey>.GroupContains(group);
var jobKeys = scheduler.GetJobKeys(groupMatcher);
foreach (var jobKey in jobKeys)
{
var detail = scheduler.GetJobDetail(jobKey);
var triggers = scheduler.GetTriggersOfJob(jobKey);
foreach (ITrigger trigger in triggers)
{
returnItems.Add(trigger.Key);
Console.WriteLine(group);
Console.WriteLine(jobKey.Name);
Console.WriteLine(detail.Description);
Console.WriteLine(trigger.Key.Name);
Console.WriteLine(trigger.Key.Group);
Console.WriteLine(trigger.GetType().Name);
Console.WriteLine(scheduler.GetTriggerState(trigger.Key));
DateTimeOffset? nextFireTime = trigger.GetNextFireTimeUtc();
if (nextFireTime.HasValue)
{
Console.WriteLine(nextFireTime.Value.LocalDateTime.ToString());
}
DateTimeOffset? previousFireTime = trigger.GetPreviousFireTimeUtc();
if (previousFireTime.HasValue)
{
Console.WriteLine(previousFireTime.Value.LocalDateTime.ToString());
}
}
}
}
return returnItems;
}
private static void ScheduleAHasParametersJob(IScheduler sched, OtherJobInfo info)
{
IJobDetail hasParametersJobDetail = JobBuilder.Create<FileNameDoSomethingJob>()
.WithIdentity(info.UniqueIdentifier + "IJobDetailWithIdentity", GROUP_NAME)
//.UsingJobData("JobDetailFileName", info.FileName)
.Build();
ITrigger hasParametersJobTrigger001 = TriggerBuilder.Create()
.WithIdentity(info.UniqueIdentifier + "ITriggerWithIdentity", GROUP_NAME)
.UsingJobData("TriggerFileName", info.FileName)
.StartNow()
.WithSimpleSchedule(x => x
.WithIntervalInSeconds(info.WithIntervalInSeconds) /* You'll have to do something fancier here with scheduling if you want an exact time */
.WithRepeatCount(0))
.Build();
sched.ScheduleJob(hasParametersJobDetail, hasParametersJobTrigger001);
}
}
public class FileNameDoSomethingJob : IJob
{
public virtual void Execute(IJobExecutionContext context)
{
JobKey key = context.JobDetail.Key;
JobDataMap jbDataMap = context.JobDetail.JobDataMap;
JobDataMap trgDataMap = context.Trigger.JobDataMap;
string triggerFileNameParameter = trgDataMap.GetString("TriggerFileName");
JobDataMap mergedMap = context.MergedJobDataMap;
string msg = string.Format("HasParametersJob : JobKey='{0}', triggerFileNameParameter='{1}' at '{2}'", key, triggerFileNameParameter, DateTime.Now.ToLongTimeString());
Console.WriteLine(msg);
}
}
public class OtherJobInfoData
{
public List<OtherJobInfo> GetOtherJobs()
{
List<OtherJobInfo> returnItems = new List<OtherJobInfo>();
OtherJobInfo oji1 = new OtherJobInfo() { UniqueIdentifier = "ABC123", WithIntervalInSeconds = 5, FileName = #"C:\file1.xml"};
OtherJobInfo oji2 = new OtherJobInfo() { UniqueIdentifier = "DEF234", WithIntervalInSeconds = 5, FileName = #"C:\file2.xml" };
OtherJobInfo oji3 = new OtherJobInfo() { UniqueIdentifier = "GHI345", WithIntervalInSeconds = 5, FileName = #"C:\file3.xml" };
returnItems.Add(oji1);
returnItems.Add(oji2);
returnItems.Add(oji3);
return returnItems;
}
}
public class OtherJobInfo
{
public string UniqueIdentifier { get; set; }
public int WithIntervalInSeconds { get; set; }
public string FileName { get; set; }
}
}
And some xml to run the ScheduleOtherJobsJob
<?xml version="1.0" encoding="UTF-8"?>
<job-scheduling-data xmlns="http://quartznet.sourceforge.net/JobSchedulingData"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" version="2.0">
<!-- This value wipes out existing jobs...be very careful with it being "true" -->
<processing-directives>
<overwrite-existing-data>true</overwrite-existing-data>
</processing-directives>
<schedule>
<job>
<name>ScheduleOtherJobsJobName</name>
<group>ScheduleOtherJobsJobGroupName</group>
<description>My Description</description>
<job-type>MyNamespace.ScheduleOtherJobsJob, MyAssembly</job-type>
<durable>true</durable>
<recover>false</recover>
<job-data-map>
</job-data-map>
</job>
<trigger>
<simple>
<name>ScheduleOtherJobsJobTriggerName</name>
<group>ScheduleOtherJobsJobTriggerGroup</group>
<description>My ScheduleOtherJobsJobTriggerName Description</description>
<job-name>ScheduleOtherJobsJobName</job-name>
<job-group>ScheduleOtherJobsJobGroupName</job-group>
<job-data-map>
</job-data-map>
<!--<start-time>1982-06-28T18:15:00.0Z</start-time>-->
<!--<end-time>2020-05-04T18:13:51.0Z</end-time>-->
<misfire-instruction>SmartPolicy</misfire-instruction>
<!-- repeat indefinitely every 5 seconds -->
<repeat-count>-1</repeat-count>
<repeat-interval>5000</repeat-interval>
</simple>
</trigger>
</schedule>
</job-scheduling-data>
EDIT : APPEND
Make sure your app.config (or web.config) is pointing to a quartz-configuration file. This is the top of mine......note, it should be used as a guide.....
<?xml version="1.0" encoding="utf-8"?>
<configuration>
<configSections>
<section name="quartz" type="System.Configuration.NameValueSectionHandler, System, Version=1.0.5000.0,Culture=neutral, PublicKeyToken=b77a5c561934e089" />
</configSections>
<quartz configSource="MyQuartzConfiguration.config" />
and then MyQuartzConfiguration.config file needs some stuff, the most important being "Quartz_Jobs_001.xml" (the name isn't important, but this file contains the info to your jobs and triggers.
<add key="quartz.plugin.jobInitializer.type" value="Quartz.Plugin.Xml.XMLSchedulingDataProcessorPlugin" />
<add key="quartz.scheduler.instanceName" value="DefaultQuartzScheduler" />
<add key="quartz.threadPool.type" value="Quartz.Simpl.SimpleThreadPool, Quartz" />
<add key="quartz.threadPool.threadCount" value="10" />
<add key="quartz.threadPool.threadPriority" value="2" />
<add key="quartz.jobStore.misfireThreshold" value="60000" />
<add key="quartz.jobStore.type" value="Quartz.Simpl.RAMJobStore, Quartz" />
<add key="quartz.plugin.jobInitializer.fileNames" value="Quartz_Jobs_001.xml" />
<add key="quartz.plugin.jobInitializer.failOnFileNotFound" value="true" />
<add key="quartz.plugin.jobInitializer.scanInterval" value="120" />