This is a simplified example scenario.
I collect interval temperature data from a room's heating, let's say every minute.
Additionally, there is a light switch, which sends its status (on/off) when someone switches the light on or off. All events are stored in crate with their timestamps.
I'd like to query all temperature readings while the light switch is in the "on"-state.
My design is to denormalize the data. So, each temperature event gets a new field, which contains the light switch status. My query breaks down to a simple filter then. However, I have only the events if someone presses the switch, but no continuous readings. So, I need to read out all light switch data, resemble the switch's state over time and update all temperature data accordingly.
Using crate, is there a way doing everything within crate using crate's SQL only, i.e. 'in-crate' data updates? I do not want to setup and maintain external client programs for such operations.
In more complex scenarios, I may also face the problem of reading a huge amount of data first via a single client program in order to update afterwards other data stored within the same data store. This "bottleneck" design approach worries me. Any ideas?
Thanks,
nodot
Related
In my source block I want to be the amount of agents based on two different factors namely the amount of beds and visitors per bed. The visitors per bed is just a variable (e.g. visitors=3) and the amount of beds is loaded from the database table which is an excel file (see first image). Now I want to code this in the code block as shown in the example in image 2, but I do not know the correct code and do not know if it is even possible.
Simplest solution is just to do the pre-calcs in the input file and have in the dbase.
The more complex solution is to set the Source arrivals as:
Now, you read your dbase code at the start of the model using SQL (i.e. the query constructor). Make the necessary computations and create a Dynamic Event for each arrival when you want it to happen, relative to the model start. Each dynamic event then calls the source.inject(1) method.
Better still is to not use Source at all but a simple Enter block. The dynamic event creates the agent with all relevant properties from your dbase and pushes it into the Enter block using enter.take(myNewAgent)
But as I said: this is not trivial
I am new to drools / fusion (7.x) and am not sure how to solve this requirement. Assume I have event objects as Event{long: timestamp, id: string} where id identifies a physical asset (like tractor) and timestamp represents the time the event fired relative to the asset. In my scenario these Events do not arrive in my system in 'real-time', meaning they can be seconds, minutes or even days late. And my rules system needs to monitor multiple assets. Given this, when rules are evaluate the clock needs to be relative to the asset being monitored, it can't be a clock that spans assets.
I'm aware of Pseudo Clock, is there a way to assign Pseudo clocks per Asset?
My assumption is that a clock must always progress forward or temporal functions will not work properly. Take for the example the following scenario:
Fact A for Asset 1 arrive at 1:00 it is inserted into memory and rules fired. Then Fact B arrives for same Asset 1 at 2:00. It too is inserted and rules fired. Now Fact Z arrives for Asset 2 at 1:30 (- 30 minutes from clock). I'm assuming I shouldn't simply progress the clock backwards and evaluate, furthermore I'd want to set the clock back to 2:00 since that was the "latest" data I received. Now assume I am monitoring thousands of assets, all sending data at different times...
The best way I can think to address this is to keep a clock per asset and then save the engine state when each assets data is evaluated. Can individual KieSession's have different clocks, or is it at a container level?
Sample rule: When Fact 1 arrives after Fact 2 for the same Asset.
You're approaching the problem incorrectly. Regardless of whether you're using a realtime or psuedo clock, you're using a clock. You can't say "Fact #1 use clock A, and Fact #2 use clock B."
Instead you should be leveraging the metadata tags for events, specifically the #timestamp tag. This tag indicates to Drools that a specific field inside of the event is actually the timestamp for the Event, rather than the actual time the fact enters working memory.
For example:
import com.example.SampleEvent
declare SampleEvent
#role( event )
// this field is actually in the object, it's not the time the fact was inserted
#timestamp( createdDateTime )
end
Not knowing anything about what your rules are actually doing, the major issue I can foresee here is that if your rules rely on the temporal operators or define an expiry (#expires), they're not going to work and you'll need to redesign them. Especially for expirations: once an event expires, it is removed from working memory; when your out-of-band events come in any previously expired events are already gone and can't be worked against.
Of course that concern would be true regardless of whether you use #timestamp or your original "different psuedo clock" plan. Either way you're going to have to manage the fact that events cannot live forever in working memory -- you will eventually run out of resources and your system will crash. Events must be evicted at some point, so you'll need to design around that in both your models and your rules.
I currently have a coreData entity called CalorieProgress, which I would like to reset all variables (calorieProgress, fatProgress) to 0, every day.
I am still quite new to SwiftUI, and the only method I thought of as of now, is to add a Date Created variable to this entity called created, and when the user opens the app, to check if that date was yesterday. If so set all values to 0 etc.
Is there a more efficient way to do this?
Thanks
Your design is good and simple, and a reasonable choice if you're getting started.
It can have trouble, however, when people move between time zones. It is even possible for people to move to previous days this way (most dramatically when they cross the date line). There is no single answer to that question. Your app has to decide what it means by "today" when strange clock events happen. (Users also sometimes change their clock, and you want to behave "reasonably" in those cases, even if it means the data is wrong.)
Having built several of these, my suggestion is to just store raw, immutable, data records, and work out things like resetting values when you're running queries. For example, to work out how many calories someone has burned "today" doesn't require that you set any value to zero. You can just perform a query for records that occur after some time and sum their calories (you can even do this with aggregate queries directly on Core Data).
Core Data can be very fast, but if these queries become too slow, you can store daily aggregation records in Core Data. But keeping the original raw data means that those are really just caches and you can throw them away and recompute any time you need to.
Assuming that a new day starts as midnight (I've worked on apps where days started "when the user wakes up in the morning" which is much more complicated...) you should also be aware of significantTimeChangeNotification which is posted at midnight (and a few other times). You can't use this to launch your app or do processing in the background, but it's very nice for updating your UI if the user has the app open.
Good day
I'm a new user trying to find my with Anylogic.
Any help with the following question will be appreciated.
Is it possible to start a model with initial values/quantities given to certain blocks/sections in a model? In other words not have the model start from 0 but from the values given.
You can run a "warmup" period manually and save that as a model snapshot. In future runs, you can start off from that snapshot by loading it. See the help on model snapshots
This is the general problem of model initialisation (e.g., if you're modelling a manufacturing facility, you may want the run to start with the facility at the state it would be at on 9am next Monday morning). There is no generic answer: what initialisation you need is 100% model-dependent (as is how easy/hard this is).
In particular, process models make this difficult because entities (agents) are expected to have flowed through the process up to the point they 'start' in. You can use things like extra initialisation-only Source/Enter blocks to 'inject' agents into the appropriate process points, but in most models it's not this easy: you will have all kinds of model state that needs to be made consistent with this (e.g., the agents flowing through the process might have attributes which have changed based on what's happened to them so far, so this would have to be made consistent).
That's why warm-up periods (letting the model run 'from empty' for a period until its state is qualitatively what you want as your starting point) is a common approach. Model snapshots can help you here (see Ben's answer) but they're not the only way of doing it. (You can also just 'reset' all your metrics/output gathering at the point when you determine the warm-up period has ended --- i.e., you are effectively establishing a new 'time zero' --- but, again, exactly what you need to do is 100% model dependent.)
I am currently stuck in a problem and hope you can help me with some ideas or best practices here...
Let's say I have an application using Event Sourcing and CQRS. I have
A green pot containing a number
A red pot containing a number
A setting where I define, the number from which pot should be displayed in the UI.
A calculated result that contains the number to be displayed
The current state of my application is
Red pot: 10
Green pot 20
Setting: Red
Result: 10 (value from red pot)
I have a Calculation service that subscribes to Red Pot service, Green Pot service and Settings service. I have a View Updater service that additionally subscribes to the Calculation service and updates the read model on any changes.
Now the following events are dropping in:
Green Pot: 25
Setting: Green Pot
The View Updater service is a bit busy today and has some delay in updating the view model.
The Calculation service handles the Green Pot event. It fetches the setting from read model (which is still set to Red) and does nothing.
After that the Calculation service handles Settings event. It fetches the Green value (which is still 20) from read read model and sends a new event (Result: 20)
After that the View Updater handles both events and updates the read model.
In this case, my application is not consistent - not even eventually.
Do have any thoughts how to handle something like this? Thanks for any ideas :-)
First thought is that it isn't clear that you are sharing the common understanding of eventual consistency. Martin Kleppmann's talk emphasized three ideas
Eventual Delivery
Convergence
No data loss
Second thought is that you seem to have introduced a race condition into your Calculation service design. If RedPot, GreenPot, and Setting are separate aggregates/streams, then there really isn't any time binding between them. The arrival of events from those sources is inherently racy.
Udi Dahan wrote Race Conditions Don't Exist
A microsecond difference in timing shouldn’t make a difference to core business behaviors.
This is where convergence comes into play: you need to design your solutions so that they reach the same results even though the timing of messages is different. This often means that you need to include in your model some concept of a clock, or time, and some way of defining the interval in which some result is true.
As you've defined the problem, the Results produced by the calculation service are more about caching a snapshot than maintaining a history. So a different way to think about the problem you are having is to consider that the calculation service should not accept any arbitrary data from the read model, but rather one that aligns with the events that it is consuming.
Calculation service: "Hi, I can haz green pot as of event #40?"
Read model: "Sorry, no can haz, retry-after: 10 minutes"
Calculation service: "OK, I'll try again later."
The Calculation service should subscribe to and receive all events (GreenPotUpdated, RedPotUpdated and SettingsChanged).
It is not OK for the Calculation service to depend on an eventually consistent read-model. Instead it should maintain its own private state, ensuring that the events are received in the correct order.