I want to monitor if an event does NOT arrive within 10 minutes of arrival of event with the same id.
This is the EPL I am currently using:
SELECT * FROM pattern[ every s=Order_Status(status="placed") -> (timer:interval(600 sec) and not e=Order_Status(status="delivered", id=s.id))]
Usually placed event arrives before delivered, but sometime because of some lag in our systems, delivered event happens to come before placed for some id.
Cases
time: 8:00 event: Order_Status{id=167, status="placed"}
time: 8:07 event: Order_Status{id=167, status="delivered"}
< No alert > (delivered within 10 minutes)
time: 8:00 event: Order_Status{id=189, status="placed"}
time: 8:17 event: Order_Status{id=189, status="delivered"}
< Alert> (delivered after 10 minutes)
time: 8:00 event: Order_Status{id=2637, status="delivered"}
time: 8:08 event: Order_Status{id=2637, status="placed"}
< Alert > (but shouldn't alert, the problem is delivered event for this id has arrived before placed)
As stated, I would get a false alert as the EPL pattern starts the window after the placed event and waits for delivered event which has already arrived.
How do I handle this scenario of out of order events ?
Note:
(Basically I want to check for every id if the time difference between placed and delivered is above a certain threshold.
I also have the timestamp fields inside each event)
You have a "not" in your pattern that is used to detect the absence of events. Your requirement doesn't search for absence so the "not" isn't right.
There is a requirement questions as well. You don't state what happens when there are many A events and just one B event. Are there many matches or just the match for the last A event or the first A event or something else?
Sample pattern:
pattern [A -> (timer:interval(10 minutes) and B)]
Or this is a join that would seem to match what you want:
select * from B unidirectional, A#window(10 minutes)#lastevent
The "A#window(10 minutes)#lastevent" keeps the last event for up to 10 minutes.
Related
I have the following use case, which I'm trying to write in EPL, without success. I'm generating analytics events of different types, generated in different intervals (1min, 5min, 10min, ...). In special kind of analytics, I need to collect 4 specific
Analytics events (from which I will count another analytic event) of different types, returned every interval (1min, 5min, 10min, ...). The condition there is, that on every whole interval, e.g., every whole minute 00:01:00, 00:02:00 I want to have returned either 4 events or nothing if the events don't arrive in some slack period after (e.g., 2s).
case 1: events A,B,C,D arrive at times 00:01:00.500, 00:01:00.600, 00:01:00.700, 00:01:00.800 - right after fourth event arrives to esper, the aggregated event with all 4 events is returned
case 2: slack period is 2seconds, events A,B,C,D arrives at 00:01:00.500, 00:01:00.600, 00:01:00.700, 00:01:02.200 - nothing is arrived, as the last event is out of the slack period
You could create a trigger event every minute like this:
insert into TriggerEvent select * from pattern[timer:schedule(date:'1970-01-01T00:00:00.0Z', period: 1 minute, repetitions: -1)]
The trigger that arrives every minute can kick off a pattern or context. A pattern would seem to be good enough. Here is something like that:
select * from pattern [every TriggerEvent -> (a=A -> b=B -> c=C -> d=D) where timer:within(2 seconds)]
I am planning to use the historical event traces (stored in JSON with my own event time stamp recorded for each event) to debug the Siddhi stream queries that I have just created. My stream starts with:
from MyInputEventStream#window.externalTime(my_own_timestamp, 10 min)
select some_fields
insert into MyOutpuStream;
and I will input my events from traces, one by one.
Supposed event 1 arrives at the specified my_own_timestamp = 1528905600000, which is 9 am PST time, June 13. and event 2 arrives at 11 minutes later, my_own_timestamp = 1528906260000. I believe that I will get the output at MyOutpuStream at 9:10 am, as time_stamp(e2) - time_stamp(e1) > 10 min, and e2 will trigger the system after the windows passes.
Now supposed event 1 arrives at my_own_timestamp = 1528905600000, that is, 9:00 am. But no events will arrive in the next 2 hours. Do I still get the output at 9:10 am, as in reality, the window time should expire at 9:10 am, independent of when the next event should arrive? But it seems that in this case, the internal timing system of Siddhi will have to incorporate my event input's time stamp, and then set the expiration time of the events based on the clock system of the process on which the Siddhi is running. Is this correct? could you help clarify it.
You won't get an output at 9:10 am. Because if you use externalTime, the event expiration logic will entirely base on the timestamp that you defined. And it will wait for a timestamp that satisfies the time difference which is greater than or equal to expire the previous event.
What internally happens is;
def array previousEvents;
foreach currentEvent in currentEvents (events that are coming in):
def currentTime = currentEvent.timestamp;
foreach previousEvent in previousEvents:
def previousTime = previousEvent.timestamp;
def timeDiff = previousTime - currentTime + windowLength;
if (timeDiff <= 0) {
remove previousEvent from previousEvents;
set expired timestamp of previousEvent to currentTime;
expire previousEvent;
}
previousEvents.add(currentEvent);
I have a host that I monitor in 08:00-20:00 and trigger which acts when no ping.
Sometimes a host can be switched off in period of 20:00-08:00. I want in this situation dont't to trigger that act till 10:00 next day. Is it possible?
You can use the time() trigger function. Appending this to the existing function should work (replace ... with the reference to the same item as already in the expression):
and time(...) > 080000 and time(...) < 200000
Note that this will make the trigger ignore the time period outside of the bounds, so it will fire at 08:00 only if the problem condition still persists. Not sure how 10:00 fits in your original description.
On newer versions of Zabbix you can set Working hours for items, so the collect just will happens in a specific range..
https://www.zabbix.com/documentation/2.0/manual/appendix/time_period
I am developing a social calendar application and want to make a feature for recuring events. I decided to use the following schhema as this might be the least memory intense for transfering (using meteorJS)
{
title: "Some Title",
/* ... some other fields for the event, e.g. location, name of venue, decription etc, a lot of data */
date: new Date(), // JS Date the event takes place first time or date for single event
repeatMethod: { //Every 2nd Monday in month, is used for generating repeatingEvents array
type: "monthly"
day: "Monday"
value: "2"
},
repeatingEvents: [
{
date: /*date for the first month */,
additionalStuff....
},
{
date: /* date for the second month */,
},
{
date: /*Date for the third month */,
canceled: true
}
]
}
I have several of these events (Around 100 Recuring events that repeat every week and around 50 events that repeat every month) and some single events that have no recurence.
Problem: I need to be able to list all events in a list like this:
Event 1 Occurence 1
Event 2 Occurence 1
Event 3 Single Event
Event 1 Occurence 2
Event 2 Occurence 2
Event 3 Occurence 1
...
but I want also to be able to limit the start and enddate of this list.
My first approach was to use single datasets per event but that blows up the database extremely. It also would be needed to store a complete event just for marking it canceled.
What way can I effectively find all events in a special timespan and list them ordered by date?
You're worrying about space when you should be worrying about seek times & user experience. Space is cheap, computations (and definitely user retention) aren't.
I suggest 2 collections, Events, Occurrences.
Store the eventId on the Occurrence document & index it.
So an event doc has _id, venue, title, etc.
An occurrence doc has _id, eventId, datetime, venue, title, etc.
Note:
Occurrence repeats venue because what happens if the regular venue isn't available a particular week? As a user, if changing the venue for 1 week changes the venue for all weeks, I'm gonna be grumpy.
We also repeat title (and all other fields, except the repetition info) so we don't need to access 2 collections for our queries. Denormalization is healthy, especially in a NoSQL environment.
This enables the user to change a field (like title) for a particular week (occurrence). If they want to change that field for ALL upcoming weeks, or maybe change the repeat structure, then they edit the event itself & that change to the event doc is propagated to all future occurrences.
A side effect of this structure is a really easy query for what you want: Occurrences.find({eventId: event, datetime: {$gt: date, $lt: date}).
I have one iphone application in which recurring events happens.the duration are like every day,every 2nd day,every 3rd day..,every week on mon,every week on tues,...,every second week on mon,..,every month on 1sr,every month on 2nd...,every second month on 1st,every second month on 2nd,... For some events there is end date but some events occurs forever.so how can i insert them automatically in sqlite3 database.eg.If an event repeats every 3rd day.how can i store it automatically after 3 days.or should i store all the events at the time of creation.If i store all the evnets at time of creation then the events that repeats forever.upto what duration i should store the value of them in database.
For this i have thought 2 approaches.one is storing just one occurance with repeated duration like every day.but in my application edit and delete functionality is also there.suppose event has one field event description then it can be different for different dates if user edit the events.events are displayed datewise on screen for a particular month and user can navigate to any previous and next month for current ,next and previous years.So if i use only single occurance then where should those edited or deleted events should be stored.
And if i take second approach store each occurance in database.Then upto what duration i should store the events which has no enddate.or is there a way that insert is automatically performed after specified duration.
Two ways, one an easy hack, one more difficult but correct :)
(1) Store each individual event for the next 10 years or so (or however long you want to!)
Good :
Easy and quick to implement
Bad :
Your db will get big very quickly.
What if the user wants to edit the details - how do you edit all of them at once?
What if the user wants to cancel / move the event- you have to delete all of them!
(2) Store some sort of 'event recurrence' information in the database with each event i.e. 'Every tuesday'
Good :
Only one event in the database, regardless of how many times it repeats.
Easy for the user to edit / delete the event - there's only one row in the database.
Bad:
More complicated event object - each event must have a list of 'when this event happens' information.
Makes very complicated event timings hard - i.e. 'last friday of every month'
Takes longer to implement
EDIT : My personal choice
I would choose option (2) - it takes longer to implement but I think option (1) will et you into trouble in the future.
I would have a data model like
Event has many Occurances
where your Event is the thing that the user has created with a description, start date etc and an Occurance is some sort of object that will say 'every friday' or 'not on the 4th'.
As part of creating an event, the user will say 'occurs once on Friday 13th' or 'occurs every Wednesday'. That information is used to create an array of Occurance objects.
Occurance would be a protocol that simply has the method occursOn: so you can have lots of different types of occurance (and you can add new types as your app gets more complicated).
The Event object would have a method something like occursOn: where you give it an NSDate and it returns if it occurs on that day. It does this by asking each of it's occurances in turn to see if they apply to that day.
To deal with deleted events, just add an Occurance to the end of the Event's array that overrides the others - i.e. 'not on Friday 13th'.
For example :
(1)
A user creates an event called 'My Event' starting on 1st Jan, occurring every Friday.
Your app would store
Event
description : 'My Event',
start date : 1st Jan 2011
occurances :
WeeklyOccurance
day : Friday
where WeeklyOccurance implements the <Occurance> protocol
(2)
The user asks to show the week's events, starting on Sunday the 8th Jan 2011
The app would :
For each day in the week
For each event in the database
if occursOn: this day
show the event on the ui
and for our event 'My Event', it would implement occursOn: like
- (BOOL)occursOn:(NSDate *)date
is this date before this event starts
if it is, return NO
set remembered = NO
for each occurance
does this occurance say 'yes','no' or '?' for this date?
if 'yes' set remembered YES
if 'no' return NO
if '?' continue the loop
return remembered
Because WeeklyOccurace only knows that it occurs on Fridays, it would return 'yes' for Fridays and '?' for all other days so the ui would show 'My Event' on Friday and not on any other days.
To add different types of occurance, just implement the <Occurance> protocol in different ways.
(3)
The user says actually it should be on every Friday apart from the 22nd
The app would create another Occurance, this time a NotOnThisDayOccurance and add it to the end of the Event's array i.e.
Event
description : 'My Event',
start date : 1st Jan 2011
occurances :
WeeklyOccurance
day : Friday
NotOnThisDayOccurance
day: 22nd Jan 2011
Now, if the users asks to display the weekly events, 'My Event' would look do this :
Ask the WeeklyOccurance if it's valid for friday the 22nd - this would return yes.
Ask the NotOnThisDayOccurance if it's valid for friday the 22nd - this would override the previous result and say NO
Therefore, the event would not show up on the 22nd but would show up on all the other fridays.