I have number of monitoring functions that I want to execute with dynamic frequencies, depending on project. The range of frequencies are from once every second, to once a day. TaskScheduler or ScheduledJob are not precise enough for this purpose.
Is any function scheduler such as this available?
Spinning up a powershell instance is not a trivial event. If you're wanting monitoring on a frequency less than a few minutes you'll be much better off to write a script that stays resident and runs in an endless process-sleep-process loop.
I created script for this purpose that you can find on github
Related
By following your advice I’m constructing small models to learn how to use AnyLogic and build my simulation.
I need discrete events diagram interacting with agent based, where the agent based will represent a “service process” based in a previous recommendation it was straight forward to trigger the agent based activity, but I cannot stop or suspend or delay the “delay” block, I tryed to use the “until stopDelay is called” function but I could not make it work, I decided to test with and cyclic event inside the discrete event agent and but was not possible. I am considering that maybe my approach is not correct, and I need to use a different strategy to stop the discrete events process while the agent-based process is running, however since agent based is attempting to simulate some human behaviour, I’m interested in the time variations this could cause to the discrete events process. So my question is how to stop or suspend the “service delay or the delay blocks and restart them from the agent based diagram?
If you just need to store an entity somewhere until Agent process is done then I would recommend using using a 'wait' block instead of a 'delay'. The whole point of a delay is to have a timed exit so suspending it doesn't align with the intended use-case. You can read more about 'wait' block here.
I found the the Job Shop model example, some blocks using stopDelayForAll(), with a "if" code block, so I noticed that it was using a parameter, so I made some changes and the code I'm using and worked is this:
if ( Inqueue >= queCap )
delay.stopDelayForAll();
"Inqueue" is a variable capturing data from the delay block and queCap is a parameter telling the queue block capacity.
The Area Creation process can take up to 24 hours. If something happens during that time which causes the process to stop, will it resume when I run it again or does it start back over from the beginning?
We can assume for this question that the files in $DB_DIR remain in place throughout the running/stopping/starting process.
It will start over from the beginning, assuming you're using areas.osm3s to define the area creation rules. This file contains a number of queries which are being executed to generate the areas. If you restart the process, it will execute those very same queries again from the beginning.
For performance reasons, we use areas_delta.osm3s and the accompanying rules_delta_loop.sh script on the production servers. This way, we can limit the workload to those areas, which have been changed since the last area creation run.
I am building an application that will need to run hundreds of short running tasks every minute. These functions are not doing anything special other than making calls to an HTTP endpoint. I need a reliable mechanism for scheduling these invocations every minute indefinitely. Failures to run at the scheduled time cannot be tolerated. I have considered the following options for the scheduler:
AWS Lambda
Mesosphere Chronos
Cron
Python Celery
Obviously there is a trade off between cost, maintainability (I will need to update the logic of these functions every once in a while), and reliability.
My question is, which of these options would be the most appropriate if I am most concerned about consistency/reliability? Are there options I'm missing that I should consider?
As you already mentioned, there are multiple technologies that could help you do this, I would say that the trick is more to find the logic flow/model to use.
For example, If the number of tasks are not fixed, a publish/subscribe pattern could apply, for this something like rabbitMQ or AWS SQS could be used.
There are multiple ways about how to submit a task to the queue and also how to de-queue, you could have multiple workers reading/waiting for events in where they could read one by one or by chunks (based on the num of cores per server) all this bound to the speed and precision you may want.
Scaling I would say is easier since if need more speed (precision to do all tasks every minute) just need to add more workers.
For more ideas check this article Using AWS Lambda with Amazon DynamoDB it covers a stream-based model / event-sourcing.
I'm planning a mechanism whose usage scenarios would be like cron's. It's a clock-ish mechanism that attempts task execution at prespecified times. Cron doesn't seem suitable, because these tasks trigger Scala method calls and the queue stored on a cloud database.
I imagine it like this: every x minutes, tasks' execution dates are retrieved from the database, and compared against current time, if the task is over-due it is executed and removed from queue.
My question is: how do I run the aforementioned check every x minutes on a distributed environment?
All advice encouraged.
I think the Akka scheduler might be what you are looking for. Here's a link to the Akka documentation and here's another link describing how to use Akka in Play.
Update: as Viktor Klang points out Akka is not a scheduler, however it does allow you to run a task periodically. I've used it in this mode quite successfully.
The best known library for this is Quartz Scheduler.
I realize that when I execute a SCOM Task on demand from a Powershell script, there are 2 columns in Task Status view called Schedule Time and Start Time. It seems that there is an interval these two fields of around 15 seconds. I'm wondering if there is a way to minimize this time so I could have a response time shorter when I execute an SCOM task on demand.
This is not generally something that users can control. The "ScheduledTime" correlates to the time when the SDK received the request to execute the task. The "StartTime" represents the time that the agent healthservice actually began executing the task workflow locally.
In between those times, things are moving as fast as they can. The request needs to propagate to the database, and a server healthservice needs to be notified that a task is being triggered. The servers then need to determine the correct route for the task message to take, then the healthservices need to actually send and receive the message. Finally, it gets to the actual agent where the task will execute. All of these messages go through the same queues as other monitoring data.
That sequence can be very quick (when running a task against the local server), or fairly slow (in a big Management Group, or when there is lots of load, or if machines/network are slow). Besides upgrading your hardware, you can't really do anything to make the process run quicker.