Esper EPL to match 5 pairs of events of distinct field values within 1 minute - complex-event-processing

Question

I have a stream of events defined by:
create schema Event(id string, username string, <additionalFields>)
where the <additionalFields> are used to construct contexts but don't directly participate in the rest of the pattern matching EPL (all the EPL statements will be executed within the context).
The desired match behavior is to match:
On five pairs of events for distinct usernames within one minute or less.
If more than two events occur for a given username within the one minute window then they are to be ignored for the purposes of the match.
If duplicate events (events with the same id field value) occur they should be ignored for the purposes of the match.
Ideally the match would consume the events such that the same events can't participate in later matches, however if the EPL is much simpler to understand then we can do post processing to eliminate these overlaps if needed.
Example input events:
Event={id='e1', username='user1'}
t=t.plus(5 seconds)
Event={id='e2', username='user2'}
t=t.plus(5 seconds)
Event={id='e3', username='user4'}
t=t.plus(5 seconds)
Event={id='e4', username='user3'}
t=t.plus(5 seconds)
Event={id='e5', username='user1'}
t=t.plus(5 seconds)
Event={id='e6', username='user1'}
t=t.plus(5 seconds)
Event={id='e7', username='user1'}
t=t.plus(5 seconds)
Event={id='e8', username='user5'}
t=t.plus(5 seconds)
Event={id='e9', username='user5'}
t=t.plus(5 seconds)
Event={id='e10', username='user4'}
t=t.plus(5 seconds)
Event={id='e11', username='user2'}
t=t.plus(5 seconds)
Event={id='e12', username='user3'}
Ideal output events:
Event={id='e1', username='user1'}
Event={id='e2', username='user2'}
Event={id='e3', username='user4'}
Event={id='e4', username='user3'}
Event={id='e5', username='user1'}
Event={id='e8', username='user5'}
Event={id='e9', username='user5'}
Event={id='e10', username='user4'}
Event={id='e11', username='user2'}
Event={id='e12', username='user3'}
The following would also be acceptable for output events:
Event={id='e1', username='user1'}
Event={id='e5', username='user1'}
Event={id='e8', username='user5'}
Event={id='e9', username='user5'}
Event={id='e3', username='user4'}
Event={id='e10', username='user4'}
Event={id='e2', username='user2'}
Event={id='e11', username='user2'}
Event={id='e4', username='user3'}
Event={id='e12', username='user3'}
I've tried using a named window:
create window AtMostTwoEventsPerUsername#time(1 minute) as Event;
on Event as e merge AtMostTwoEventsPerUsername as w where w.id = e.id or (select count(*) from AtMostTwoEventsPerUsername where username = e.username) > 1 when not matched then insert select *;
on Event insert into FivePairsOfTwoEventsPerUsername select w.* from AtMostTwoEventsPerUsername as w where w.username in (select username from AtMostTwoEventsPerUsername group by username having count(*) = 2) having count(*) = 10;
on FivePairsOfTwoEventsPerUsername as m delete from AtMostTwoEventsPerUsername as w where w.id = m.id;
#Name("Out") select * from FivePairsOfTwoEventsPerUsername#time(1 minute)#length_batch(10);
and it seems to be close, however it requires an extra event after the matching events that is undesireable:
Event={id='e1', username='user1'}
t=t.plus(5 seconds)
Event={id='e2', username='user2'}
t=t.plus(5 seconds)
Event={id='e3', username='user4'}
t=t.plus(5 seconds)
Event={id='e4', username='user3'}
t=t.plus(5 seconds)
Event={id='e5', username='user1'}
t=t.plus(5 seconds)
Event={id='e6', username='user1'}
t=t.plus(5 seconds)
Event={id='e7', username='user1'}
t=t.plus(5 seconds)
Event={id='e8', username='user5'}
t=t.plus(5 seconds)
Event={id='e9', username='user5'}
t=t.plus(5 seconds)
Event={id='e10', username='user4'}
t=t.plus(5 seconds)
Event={id='e11', username='user2'}
t=t.plus(5 seconds)
Event={id='e12', username='user3'}
t=t.plus(1 seconds)
Event={id='e13', username='user999'} // this shouldn't be needed to trigger a match
results in the desired output events:
FivePairsOfTwoEventsPerUsername={id='e1', username='user1'}
FivePairsOfTwoEventsPerUsername={id='e2', username='user2'}
FivePairsOfTwoEventsPerUsername={id='e3', username='user4'}
FivePairsOfTwoEventsPerUsername={id='e4', username='user3'}
FivePairsOfTwoEventsPerUsername={id='e5', username='user1'}
FivePairsOfTwoEventsPerUsername={id='e8', username='user5'}
FivePairsOfTwoEventsPerUsername={id='e9', username='user5'}
FivePairsOfTwoEventsPerUsername={id='e10', username='user4'}
FivePairsOfTwoEventsPerUsername={id='e11', username='user2'}
FivePairsOfTwoEventsPerUsername={id='e12', username='user3'}
If the last event (Event={id='e13', username='user999'}) is removed from the input events stream then the "Out" stream unexpectedly has no matching events.
I'd like to understand why the extra event at the end is needed to trigger the match and if there is a simpler set of EPL statements to achieve the desired pattern matching.

Answer 1

To ignore all events with the same id field value that would mean remembering all id field values that ever occurred... is that right?
I would approach this by making an intermediate stream that has a flag that indicates whether the username is "entering" or "leaving" the set of user names that are currently distinct. I would use that intermediate stream to add and remove usernames (and their additional info) from a named window. Select the final output with an "insert into resultstream select window(*) from namedwindow having count > x". Then use "resultstream" as a trigger to delete the named window contents so that these usernames disappear avoiding overlaps (namedwindow having the merge of the intermediate stream).
This way your solution becomes a two step design. The first step produces the intermediate stream. The second retains the intermediate stream for output when 5 are found.