Reactive systems - Reacting to time passing - reactive-programming

Let's say we have a reactive sales forecasting system.
Every time we make a sale we re-calculate our Forecast for future sales.
This works beautifully if there are lots of sales triggering our re-forecasting.
What happens however if sales go from 100 events per second, to 0. And stay 0 for a long time?
The forecast we published back when sales were good stays being the most up to date forecast.
How would you model in this situation an event that represents 'No sales happening' without falling back to some batch hourly/minutely/arbitrary time segment event that says 'X time has passed'.
This is a specific case of a generic question - How do you model time passing with nothing happening in an event based system - without using a ticking clock style event which would wake everyone up to reconsider their current values [an implementation which would not scale].
The only option I have considered that makes sense:
Every time we take a sale, we also schedule a deferred event 2 hours in the future that asks us to reconsider our assessment of that sale.
In handling that deferred event we may then choose to schedule further deferred events for re-considering.

Considering this is a very generic scenario, you've made a rather large assumption that it's not possible to come up with a design for re-evaluating past sales in a scalable way unless it's done one sale at a time.
There are many different scale related numbers in the scenario, and you're only looking at the one whereby a single scheduled forecast updater may attempt to process a very large number of past sales at the same time.
Other scalability issues I can think of:
Reevaluating the forecast for every single new sale doesn't sound great if you're expecting 100s of sales per second. If you're talking about a financial forecasting model for accounting, it's unlikely it needs to be updated every single time the organisation makes a sale, if the organisation is making hundreds of sales a second.
If you're talking about a short term predictive engine to be used for financial markets (ie predicting how much cash you'll need in the next 10 seconds, or energy, or other resources), then it sounds like you have constant volatility and you're not really likely to have a situation where nothing happens for hours. And if you do need forecasts updated very frequently, waiting a couple of hours before triggering a re-update is not likely to get you the kind of information you need in the way you need it.
With your approach, you will end up with one future scheduled event per product (which could be large), and every time you make a sale, you'll be dropping the old scheduled event and scheduling a new one. So for frequently selling products, you'll be doing repetitive work to constantly kick the can down the road a bit further, when you're not likely to ever get there.
What constitutes a good design is going to be based on the real scenario. The generic case is interesting to think about, but good designs need to be shaped to their circumstances.
Here are a few ideas I have that might be appropriate:
If you want an updated forecast per product when that product has a sale, but some products can sell very frequently, then a good approach may be to throttle or buffer the sales on a per product basis. If a product is selling 50 times a second, you can probably afford to wait 1 second, 10 seconds, 2 hours, whatever and evaluate all those sales at once, rather than re-forecasting 50 times a second. Especially if your forecasting process is heavy, doing it for every sale is likely to cause high load for low value, as the information will be outdated almost straight away by the next sale.
You could also have a generic timer that updates forecasts for all products that haven't sold in the last window, but handle the products in buffers. For example, every hour you could pick the 10 products with the oldest forecasts and update them. This prevents the single timer from taking on re-forecasting the entire product set in one hit.
You could use only the single timer approach above and forget the forecast updates on every sale if you want something dead simple.
If you're worried about load from batch forecasting, this kind of work should be done on different hardware from the ones handling sales.

Related

Increasing model simulation speed with big number of agents/transporters Anylogic

I've a question. In my model I want to model patients/employees in a hospital environment as agents. Due to the big amount of patients/employees in the hospital (>4000), my model runs very slow (which is logical, I know). Because I want the patients employees to block eachother and AGVs, I let them move by a transporterfleet (which has >5000 capacity), by doing this method I can make it possible to let them block eachother.
Now what I want is that when the patient or employees arrive to the location departments they disappear for hours/minutes as long as there appointment or workschedule is saying to them. The most perfect is to just delete them for example 4 hours of runtime, so that I will only have the moving agents in my model, is there a way to do this? So that when a patient/employee arrives at an appointment, the agent and transport disappears for 4 hours and returns after this delay, keeping in mind that it should speed up the run-speed of my simulation (otherwise I just can use a delay block, but than my model keeps very slow). The transporters are making the model really slow.
Or perhaps another method will be better to model the patients and employees with the feature that they are blocking AGVs? An option which I should think of is that an moving agent is constantly checking if there is an agent close to him every second, but I do not know if this will be speed up the runtime or just will slow it down even more with >4000agents.
Or another option is that I only let the AGVs check if there are agents around them every second just by coding(this will only be around 40 agents).
Also you see that it is only using 5% of my memory data..
Thanks.

Different Pseudo Clock for groups of Facts

I am new to drools / fusion (7.x) and am not sure how to solve this requirement. Assume I have event objects as Event{long: timestamp, id: string} where id identifies a physical asset (like tractor) and timestamp represents the time the event fired relative to the asset. In my scenario these Events do not arrive in my system in 'real-time', meaning they can be seconds, minutes or even days late. And my rules system needs to monitor multiple assets. Given this, when rules are evaluate the clock needs to be relative to the asset being monitored, it can't be a clock that spans assets.
I'm aware of Pseudo Clock, is there a way to assign Pseudo clocks per Asset?
My assumption is that a clock must always progress forward or temporal functions will not work properly. Take for the example the following scenario:
Fact A for Asset 1 arrive at 1:00 it is inserted into memory and rules fired. Then Fact B arrives for same Asset 1 at 2:00. It too is inserted and rules fired. Now Fact Z arrives for Asset 2 at 1:30 (- 30 minutes from clock). I'm assuming I shouldn't simply progress the clock backwards and evaluate, furthermore I'd want to set the clock back to 2:00 since that was the "latest" data I received. Now assume I am monitoring thousands of assets, all sending data at different times...
The best way I can think to address this is to keep a clock per asset and then save the engine state when each assets data is evaluated. Can individual KieSession's have different clocks, or is it at a container level?
Sample rule: When Fact 1 arrives after Fact 2 for the same Asset.
You're approaching the problem incorrectly. Regardless of whether you're using a realtime or psuedo clock, you're using a clock. You can't say "Fact #1 use clock A, and Fact #2 use clock B."
Instead you should be leveraging the metadata tags for events, specifically the #timestamp tag. This tag indicates to Drools that a specific field inside of the event is actually the timestamp for the Event, rather than the actual time the fact enters working memory.
For example:
import com.example.SampleEvent
declare SampleEvent
#role( event )
// this field is actually in the object, it's not the time the fact was inserted
#timestamp( createdDateTime )
end
Not knowing anything about what your rules are actually doing, the major issue I can foresee here is that if your rules rely on the temporal operators or define an expiry (#expires), they're not going to work and you'll need to redesign them. Especially for expirations: once an event expires, it is removed from working memory; when your out-of-band events come in any previously expired events are already gone and can't be worked against.
Of course that concern would be true regardless of whether you use #timestamp or your original "different psuedo clock" plan. Either way you're going to have to manage the fact that events cannot live forever in working memory -- you will eventually run out of resources and your system will crash. Events must be evicted at some point, so you'll need to design around that in both your models and your rules.

Is this the best way to set CoreData entity variables to 0 every day at 24:00?

I currently have a coreData entity called CalorieProgress, which I would like to reset all variables (calorieProgress, fatProgress) to 0, every day.
I am still quite new to SwiftUI, and the only method I thought of as of now, is to add a Date Created variable to this entity called created, and when the user opens the app, to check if that date was yesterday. If so set all values to 0 etc.
Is there a more efficient way to do this?
Thanks
Your design is good and simple, and a reasonable choice if you're getting started.
It can have trouble, however, when people move between time zones. It is even possible for people to move to previous days this way (most dramatically when they cross the date line). There is no single answer to that question. Your app has to decide what it means by "today" when strange clock events happen. (Users also sometimes change their clock, and you want to behave "reasonably" in those cases, even if it means the data is wrong.)
Having built several of these, my suggestion is to just store raw, immutable, data records, and work out things like resetting values when you're running queries. For example, to work out how many calories someone has burned "today" doesn't require that you set any value to zero. You can just perform a query for records that occur after some time and sum their calories (you can even do this with aggregate queries directly on Core Data).
Core Data can be very fast, but if these queries become too slow, you can store daily aggregation records in Core Data. But keeping the original raw data means that those are really just caches and you can throw them away and recompute any time you need to.
Assuming that a new day starts as midnight (I've worked on apps where days started "when the user wakes up in the morning" which is much more complicated...) you should also be aware of significantTimeChangeNotification which is posted at midnight (and a few other times). You can't use this to launch your app or do processing in the background, but it's very nice for updating your UI if the user has the app open.

How to stop timeout in service block

I am modeling ticket system with various SLA. The model must contain several service blocks with different reaction time ( from 2 to 32 hours). In the service block only working hours should be taken into account. So in the service block timeout should stop when non-workong hours and on the weekend. Could you please kindly tell me how i can realize it?
Thank you very much in advance!
I can think of two answers, one simplified but works in many cases, the other more advanced and probably more accurate:
Simplified approach: I would set the model in hours and keep everything running as is without any stop. So, at the end of the simulation, if the total time is 100 hours and you know that you have 8 hours/day with 5 days/week, then you'd know the total duration is 2.5 weeks. Of course, this might have limitations or might become more complex later on if you want day-specific actions (e.g. you want to differentiate between Monday, Tuesday, etc.)
Advanced more accurate approach: Create resources whose capacities are defined by schedule and assigned them to your services. Create a schedule and specify the working hours in that schedule. Check the below link to learn more about schedules. I call this the more advanced approach because you need to make sure the schedule is defined correctly and make sure all elements in the model are properly controlled (e.g. non-service blocks such as source, delays, etc.).
https://help.anylogic.com/topic/com.anylogic.help/html/data/schedule.html?resultof=%22%73%63%68%65%64%75%6c%65%73%22%20%22%73%63%68%65%64%75%6c%22%20
I personally would use the first approach if the model is rather simple and modeling working hours is enough for analysis. Otherwise, I'd go for option 2.
Finally, another option I'd like to highlight is the "suspend/resume" functions. I am only adding this because you asked "how to stop timeout". So these functions specifically stop and resume timeout. But you'll need to define the times at which they are executed (through an event for example).

Event Sourcing and Retroactive Events

I have a need to incorporate retroactive events in my event stream and I'm not sure of the best way to implement it.
We need to keep the original event stream unchanged for audit and all of the other standard benefits. The event stream is also temporal in nature, giving us the ability to see the values for any point in history. i.e. The value of x was 10.00 at 5 pm June 1st. Occasionally we find out on June 5 the value of x as was actually 12.00 at 5pm June 1st. In this scenario we refer to 10.00 as the 'as-at' value and '12.00' as the as-of value and we track both of these in the event stream.
Rebuilding the state for the as-at value is a straight forward query from the most recent snap before 5 pm June 1st and all of the events though June 1st.
Where I am hesitant is in rebuilding the as-of state. If there is an as-of correction to the model then then it should be used by default rather than the as-at, but I can't see any way to determine if there is an as-of correction without reading the entire event stream from the point in time until the present (this can be large) and most of the changes will not matter as they will be related to future changes and not the point in time in question.
Is there a different approach I should be looking at here?
Thanks,
Chris
I think what you're referring to is a bitemporal data model. That is, you can answer not only "who won the US Presidential election in 2000", but "who did we think won the US Presidential election in the evening of election day in 2000".
In general, your event stream is not necessarily built to answer all your queries and bitemporal queries efficiently. It is simply a history of the facts you learned. If you learn today a fact about last year, it still belongs at the end of your event stream, but marked with the relevant dates.
The best way to query this data depends on what kinds of questions you want to answer. There are several nice papers on how to construct temporal and bitemporal database schemas, which would be populated by projectors feeding off your event stream.