I'm using Optaplanner to develop a system, it similars with the example - MeetingScheduling. Assigns some tasks to some machines and determine the start time. I create a class - TaskAssignment as the planning entity, the fields - "machine" and "startTimeGrain" as the planning variables.
But in my use case, there is a constraint doesn't exist in MeetingScheduling, I don't know how to achieve. In some cases, possibly there is a preparation time on the front of the task. It means, TaskA and TaskB is the contiguous tasks on the same machine, TaskB is not going to start until TaskA finished (TaskA is the previous task of TaskB), and possibly there is the preparation time between those tasks, means that after TaskA finished, TaskA have to wait for a while to start, but how long to wait is not fixed, it depends on its previous task.
Possibly like following:
TaskA -> TaskB: TaskB's preparation time is 5 mins.
TaskC -> TaskB: TaskB's preparation time is 15 mins.
TaskC -> TaskA: TaskA's preparation time is 0 min.
So. I get the preparation time for the task base on its previous task (read it from a list) and calculate the interval between two tasks. if the interval is less than preparation time, interval minus preparation time as the punish score.
When I run planning, the rule through a Score Corruption exception. I found that the reason is that both the interval and preparation time are uncertain.
For the interval, it depends on the previous task's end time and its own task's start time, the start time is the planning variable, so it's uncertain.
For the preparation time, there is a preparation time list in each task, which preparation time is available depends on this previous task, due to the start time is keep changing during planning, the preparation time keeps changing too. so preparation time is uncertain too.
in this case, is any way to achieve?
Many thanks
Here is my rule, but score corruption exception appear.
rule "Make sure interval large than preparation time"
salience 1
when
$currentTA : TaskAssignment(
$machine: machine != null,
startingTimeGrain != null,
$lack : getIntervalLack() < 0L // in getIntervalLack(), interval minus preparation time
)
then
scoreHolder.addHardConstraintMatch(kcontext, $lack);
end
The exception message:
Exception in thread "main" java.lang.IllegalStateException: Score corruption: the workingScore (-17hard/0medium/0soft) is not the uncorruptedScore (-20hard/0medium/0soft) after completedAction ([TaskAssignment-5 {Machine-1([023]) -> Machine-1([023])}, TaskAssignment-5 {TimeGrain-2 -> TimeGrain-2}]):
The corrupted scoreDirector has no ConstraintMatch(s) which are in excess.
The corrupted scoreDirector has 1 ConstraintMatch(s) which are missing:
com.esquel.configuration/Make sure interval large than preparation time/[TaskAssignment-4]=-3hard/0medium/0soft
Check your score constraints.
at org.optaplanner.core.impl.score.director.AbstractScoreDirector.assertWorkingScoreFromScratch(AbstractScoreDirector.java:496)
at org.optaplanner.core.impl.solver.scope.DefaultSolverScope.assertWorkingScoreFromScratch(DefaultSolverScope.java:132)
at org.optaplanner.core.impl.phase.scope.AbstractPhaseScope.assertWorkingScoreFromScratch(AbstractPhaseScope.java:167)
at org.optaplanner.core.impl.constructionheuristic.decider.ConstructionHeuristicDecider.processMove(ConstructionHeuristicDecider.java:140)
at org.optaplanner.core.impl.constructionheuristic.decider.ConstructionHeuristicDecider.doMove(ConstructionHeuristicDecider.java:126)
at org.optaplanner.core.impl.constructionheuristic.decider.ConstructionHeuristicDecider.decideNextStep(ConstructionHeuristicDecider.java:99)
at org.optaplanner.core.impl.constructionheuristic.DefaultConstructionHeuristicPhase.solve(DefaultConstructionHeuristicPhase.java:74)
at org.optaplanner.core.impl.solver.AbstractSolver.runPhases(AbstractSolver.java:87)
at org.optaplanner.core.impl.solver.DefaultSolver.solve(DefaultSolver.java:167)
at com.esquel.main.App.startPlan(App.java:94)
at com.esquel.main.App.main(App.java:43)
If it's a hard constraint, I 'd make it build in and do it with a shadow variable:
I'd probably pre-calculate the task dependencies, so taskB has a reference to it's potentioalPrecedingTasks (taskA and taskC in your example). Then I 'd use the "chained through time" pattern (see docs) to determine the order in which the tasks get executed. Based on that order, the starting time is a shadow variable that is actualPrecedingTask.endingTime + lookUpPreperationTime(precedingTask, thisTask). See arrivalTime listener in VRP, same principle.
If it's a soft constraint, I 'd still have that same shadow variable, but call it desiredStartingTime and add a soft constraint to check if the real startingTime is equal or higher than the desiredStartingTime.
Background summary: Some orders are sent to the production workshop, an order is split into multiple tasks in a sequence by the process routing. The tasks of an order must be executed in the sequence. Each task can only be executed on a particular machine. There is maybe preparation time before starting a task, whether the preparation time exists or not, and it is long or short, depends on what task at the front of it on the same machine.
The following are the main constraints what hard to implement:
Hard constraints:
1. A task must be executed by the particular machine.
2. Tasks in an order must be executed by a particular sequence (tasks of an order, come from the processes of the order, usually they need to be executed by different machine).
3. The first task of an order has the earliest start time. It means when the order arrived the production workshop.
4. Some tasks in an order maybe have a request start time, means if the previous task finish, the next task has to start in a period. For example, TaskA is the previous task of TaskB in the same order, TaskB has to start in 16 hours after TaskA finish.
5. A task probably has a preparation time, depends on its previous task of in the same machine(usually, the same process from different order were assigned to the same machine). If there is the preparation time on the task, the task has to start after the preparation time. In other words, there is an interval between these tasks.
Soft constraints:
1. All task should be executed as soon as possible.
2. Minimize the preparation time, due to the different location of the tasks lead to the different relationship of the tasks, then the preparation times are different.
So, here are two "chains" in the solution during planning. Optaplanner generated a chain for the tasks in the same machine. The another "chain" comes from the order, and in this "chain", tasks will be assigned to the different machine. Two "Chains"
were hang together.
I named that the chain in a machine (generated by Optaplanner) as "Machine chain", and the "chain" in an order as "Order chain".
Now, you can see, due to two "Chains" were hang together, a task as the node both in the Machine chain and Order chain.
I had tried the "chained through time" pattern, the undoMove corruption appeared. I think the reason is when I updated a task in Machine chain, the following task in the same Machine chain will be updated too, these tasks are the nodes of Order chains, a chain reaction broke out.
I think my case looks like the example - Project Job Scheduling. But the difference is two "Chains" in this example never hang together.
So, I try the simple patter, but I can't escape the Score Corruption exception.
Related
I am writing a school project about real-time systems and their scheduling algorithms. Particularly I am trying to compare several of these algorithms, namely RMS, EDF, and LLF under overloaded systems. But I am confused about how these systems deal with deadlines that are missed.
For example, consider the following set of tasks. I assume all tasks are periodic and their deadlines are equal to their periods.
Task 1:
Execution time: 2
Period: 5
Task 2:
Execution time: 2
Period: 6
Task 3:
Execution time: 2
Period: 7
Task 4:
Execution time: 2
Period: 8
It is not possible to make a feasible schedule with these tasks because the CPU Utilization is over 100%, which means some deadlines will miss and more importantly some tasks will not be completed at all. For the sake of comparison, I want to calculate a penalty (or cost) for each of the tasks which increases as more and more deadlines are missed. Here's where the questions and confusion start.
Now I understand, for example, that in RMS, the first task will never miss since it has the highest priority, and the second task also never misses. On the other hand, the third task does miss a bunch of deadlines. Here is the first question:
Do we consider a task to be dropped in RMS if it misses its deadline and a new task is dispatched?
1.a) If we do consider it dropped how would I reflect this in my penalty calculations? Since the task is never completed it would seem redundant to calculate the time it took to complete the task after its deadline passed.
1.b) If we do not consider it to be dropped and the execution of the task continues even after its deadline passes by a whole period, what happens to the new task that is dispatched? Do we drop that task instead of the one we started already, or does it just domino onto the next one and the next one and etc.? If that is the case this means that when a schedule with a length of the LCM of the tasks' periods are made, there are some task 3 dispatches that are not completed at all.
Another confusion is of the same nature but with EDF. EDF fails after a certain time on several tasks. I understand that in the case of EDF I must continue with the execution of the tasks even if I pass their deadlines which means all of the tasks will be completed even though they will not fit their deadlines completely, hence the domino effect. Then the question becomes
Do we drop any tasks at all? What happens to the tasks which are dispatched because the period resets but they cannot be executed because the same task is being executed since it missed its deadline on the period before?
I know it is a long post but any help is appreciated. Thank you. If you cannot understand any of the questions I may clarify them at your request.
I've come up with a task scheduling and resource allocation problem in which resources can start running a new task with a complex condition.
A resource can start a new task on an even time unit if at least 2n time units has passed from starting the previous task started on an even time unit.
The same holds for odd time units.
Below is a valid scheduling on a single resource. Each number represents that a new task has been started at that time.
0, 1, 2n, 2n+1, 4n, 4n+1, ...
I've got a lot of tasks with precedence relations in between (I know how to cope with the precedence relations) and several resources of this kind. I carried the scheduling out the following way which does not yields an optimal result:
Although a task can start on an odd or an even time unit, I've constrained half of the tasks to start on even time units and the other half on odd time units using "forbidStart" and "stepFunction".
Per resource s, I've considered two "cumulFunction"s s_even and s_odd.
Tasks that are forbidden to start on even (odd) time units need the s_odd (s_even) resource. I defined this constrained using "cumulFunction" and "pulse".
Although the above procedure produces a valid scheduling, it is not enough since I'm seeking for an optimal solution. Does anybody have any idea how to carry out this problem in CPLEX?
As said at https://www.ibm.com/developerworks/community/forums/html/topic?id=ac7a4fa1-f304-420c-8302-18501b4b7602&ps=25 by Philippe Laborie
just consider an additional interval variable 'task' of length 2n that represents the task and have an alternative on two optional tasks 'taskEven' and 'taskOdd'. These two intervals are the ones you already have in your model (with the adequate forbidStart constraints, and with a contribution to the adequate resource).
I need to create schedulers to execute jobs(class files) at specified intervals..For Now, I'm using Quartz Scheduler which triggers the jobs at defined intervals from the time of triggering of it.
For Eg: Consider I'm giving a cron expression to run for every one hour starting at morning 9.My first run will be at 9 and my second run will be at 10 and so on.
If my job is taking 20 minutes to execute then in that case this method is not that much efficient.
What I need to do is to schedule a job for every one hour from the completion time of the previously ran job
For Eg: Consider my job to run every one hour is triggered at 9 and for the first run it took 20 minutes to run, so for the next time the job should trigger only at 10:20 instead of 10 (ie., one hour from the completion of previous ran job)
I need to know whether there are any methods in Quartz Scheduling to achieve this or any other logic I need to do.
If anyone could help me out on this,it would be very helpful for me.
You can easily achieve this by job-chaining your job executions. There are various approaches you can choose from:
(1) Implement a Quartz JobListener and in its jobWasExecuted method, that is invoked by Quartz whenever a job finishes executing, re-fire your job.
(2) Look at the Quartz JobChainingJobListener that you can use to implement simple job chaining scenarios. Please note that the functionality of this listener is very limited as it does not allow you to insert delays between job executions, there is no support for conditions that must be met before target jobs are executed etc. But you can use it as a good starting point to implement (1).
(3) Use QuartzDesk (our commercial product) or any other product that allows you to create job chains while externalizing and managing all job dependencies outside of your application. A job chain can have multiple target jobs that can be executed immediately, with a fixed delay or at arbitrary time in the future produced by a JavaScript expression. It also allows you to implement somewhat more sophisticated works flows, such as firing a target job when multiple source jobs complete their execution etc. I am attaching screenshots showing you what a simple job chain that re-executes Job1 with a 1 minute delay upon Job1's completion (with any job execution status) looks like:
I realize that when I execute a SCOM Task on demand from a Powershell script, there are 2 columns in Task Status view called Schedule Time and Start Time. It seems that there is an interval these two fields of around 15 seconds. I'm wondering if there is a way to minimize this time so I could have a response time shorter when I execute an SCOM task on demand.
This is not generally something that users can control. The "ScheduledTime" correlates to the time when the SDK received the request to execute the task. The "StartTime" represents the time that the agent healthservice actually began executing the task workflow locally.
In between those times, things are moving as fast as they can. The request needs to propagate to the database, and a server healthservice needs to be notified that a task is being triggered. The servers then need to determine the correct route for the task message to take, then the healthservices need to actually send and receive the message. Finally, it gets to the actual agent where the task will execute. All of these messages go through the same queues as other monitoring data.
That sequence can be very quick (when running a task against the local server), or fairly slow (in a big Management Group, or when there is lots of load, or if machines/network are slow). Besides upgrading your hardware, you can't really do anything to make the process run quicker.
I'm looking for recommended solution to work around celerybeat being a single point of failure for celery/rabbitmq deployment. I didn't find anything that made sense so far, by searching the web.
In my case, once a day timed scheduler kicks off a series of jobs that could run for half a day or longer. Since there can only be one celerybeat instance, if something happens to it or the server that it's running on, critical jobs will not be run.
I'm hoping there is already a working solution for this, as I can't be the only one who needs reliable (clustered or the like) scheduler. I don't want to resort to some sort of database-backed scheduler, if I don't have to.
There is an open issue in celery github repo about this. Don't know if they are working on it though.
As a workaround you could add a lock for tasks so that only 1 instance of specific PeriodicTask will run at a time.
Something like:
if not cache.add('My-unique-lock-name', True, timeout=lock_timeout):
return
Figuring out lock timeout is well, tricky. We're using 0.9 * task run_every seconds if different celerybeats will try to run them at different times.
0.9 just to leave some margin (e.g. when celery is a little behind schedule once, then it is on schedule which would cause lock to still be active).
Then you can use celerybeat instance on all machines. Each task will be queued for every celerybeat instance but only one task of them will finish the run.
Tasks will still respect run_every this way - worst case scenario: tasks will run at 0.9*run_every speed.
One issue with this case: if tasks were queued but not processed at scheduled time (for example because queue processors was unavailable) - then lock may be placed at wrong time causing possibly 1 next task to simply not run. To go around this you would need some kind of detection mechanism whether task is more or less on time.
Still, this shouldn't be a common situation when using in production.
Another solution is to subclass celerybeat Scheduler and override its tick method. Then for every tick add a lock before processing tasks. This makes sure that only celerybeats with same periodic tasks won't queue same tasks multiple times. Only one celerybeat for each tick (one who wins the race condition) will queue tasks. In one celerybeat goes down, with next tick another one will win the race.
This of course can be used in combination with the first solution.
Of course for this to work cache backend needs to be replicated and/or shared for all of servers.
It's an old question but I hope it helps anyone.