I want to detect a missing event in a data stream ( e.g. detect a customer request that has not been responded within 1 hour of its reception. )
Here, I want to detect the "Response" event and make an alert.
I tried using tick tuple by setting TOPOLOGY_TICK_TUPLE_FREQ_SECS but it is configured at a bolt level and might come after 15th minute of getting a customer request.
#Override public Map getComponentConfiguration() {
Config conf = new Config();
conf.put(Config.TOPOLOGY_TICK_TUPLE_FREQ_SECS, 1800);
return conf; }
^ this doesn't work.
Let me know in comments if any other information is required. Thanks in advance for the help.
This might help http://storm.apache.org/releases/1.0.3/Windowing.html
you can define 5 mins windows and check the status of last window events and alert based on what is received
or create an intermediate bolt which maintains these windows and sends the normal alert tuples(instead of tick tuple) in case of timeouts
Related
I have a pretty simple Anylogic DE model where POs are launched regularly, and a certain amount of material gets to the incoming Queue in one shot (See Sample Picture below). Then the Manufacturing process starts using that material at a regular rate, but I want to check if the material in the queue gets outdated, so I'm using the TimeOut option of that queue, in order to scrap the outdated material (older than 40wks).
The problem is that every time that some material gets scrapped through this Timeout exit, the downstream Manufacturing process "stops" pulling more material, instead of continuing, and it does not get restarted until a new batch of material gets received into the Queue.
What am I doing wrong here? Thanks a lot in advance!!
Kindest regards
Your situation is interesting because there doesn't seem to be anything wrong with what you're doing. So even though what you are doing seems to be correct, I will provide you with a workaround. Instead of the Queue block, use a Wait block. You can assign a timeout and link the timeout port just like you did for the queue (seem image at the end of the answer).
In the On Enter field of the wait block (which I will assume is named Fridge), write the following code:
if( MFG.size() < MFG.capacity ) {
self.free(agent);
}
In the On Enter of MFG block write the following:
if( self.size() < self.capacity && Fridge.size() > 0 ) {
Fridge.free(Fridge.get(0));
}
And finally, in the On Exit of your MFG block write the following:
if( Fridge.size() > 0 ) {
Fridge.free(Fridge.get(0));
}
What we are doing in the above, is we are manually pushing the agents. Each time an agent is processed, the model checks if there is capacity to send more, if yes, a new agent is sent.
I know this is an unpleasant workaround, but it provides you with a solution until AnyLogic support can figure it out.
So I have a vehicle which is going to pick up exact quantity (Wait for) the first 50 minutes in the simulation. After those 50 minutes have gone by, I want the same vehicle to pick up quantity (if available). How do I go about this?
Alternative approach (avoiding complex Java coding) is to use 2 Pickup blocks, each with a different setup. Put a SelectOutput block before them and route agents into the respective block using time()>50*minute() in the SelectOutput condition
Set it up for the first setup by default.
Create an event to trigger after 50 mins and make it execute this code:
myPickupObject.set_pickupType(PickupType.QUANTITY);
Here is a way to allow container entity to wait in a Pickup for some time t and then leave with whatever entities have been picked up. The example model looks like this:
There are two key components:
ReleaseOnTimeout Dynamic event which has a parameter called '_agent' of type Agent and following code:
for (Object o : pickup.getEmbeddedObjects()) {
// find the Delay object inside Pickup
if (Delay.class == o.getClass().getSuperclass()) {
// remove the container from the Delay
Agent a = ((Delay)o).remove(_agent);
if (a != null) {
// send the removed container into Enter
enter.take(a);
}
}
}
In pickup on enter action contains following code: `create_ReleaseOnTimeout(10, container);
How this works:
pickup is configured to have Exact quantity (wait for) behaviour
a container object enters Pickup block pickup
on entry a dynamic event ReleaseOnTimeout is scheduled in 10 units to check up on container
if sufficient number of entities were available then container picks them up and leaves
alternatively to (4), if by the time 10 units elapsed the container is still stuck in pickup then it will be removed and put into enter
I'm trying to create an agent statechart where a transition should happen every day at 4 pm (except weekends).
I have already tried:
1. a conditional transition (condition: getHourOfDay() == 16)
2: A timeout transition that will "reinsert" my agent into the chart every 1 s and check if time = 16.
My code is still not running, does anyone have any idea how to solve it?
This is my statechart view. Customer is a single resource that is supposed to "get" the products out of my stock everyday at 4pm. It is supposed to happen in the "Active" state.
I have set a timeout transition (from Active-Active) that runs every 1s.
Inside my "Active" state in the "entrance action" i wrote my code to check if it is 4 pm and run my action if so.
I thought since i set a timeout transition it would check my condition every 1s, but apparently it is not working.
Your agent does not enter the Active state via an internal transition.
Redraw the transition to actually go out of the Active state and then enter it again as below:
Don't use condition-based transitions, for performance reasons. In your case, it also never triggers because it is only evaluated when something happens in the model. Incidentally, that is not the case at 4pm.
re your timeout approach: Why would you "reinsert" your agent into its own statechart? Not sure I understand...
Why not set up a schedule or event with your recurrence requirement and make it send a message to the statechart: stateChart.fireEvent("trigger!");. In your statechart, add a message-based transition that waits for this message. This will work.
Be careful to understand the difference between the Statechart.fireEvent() and the Statechart.receiveMessage() functions, though.
PS: and agree with Felipe: please start using SOF the way it is meant, i.e. also mark replies as solved. It helps us but also future users to quickly find solutions :-) cheers
I have a streaming application that continuously takes in a stream of coordinates along with some custom metadata that also includes a bitstring. This stream is produced onto a kafka topic using producer API. Now another application needs to process this stream [Streams API] and store the specific bit from the bit string and generate alerts when this bit changes
Below is the continuous stream of messages that need to be processed
{"device_id":"1","status_bit":"0"}
{"device_id":"2","status_bit":"1"}
{"device_id":"1","status_bit":"0"}
{"device_id":"3","status_bit":"1"}
{"device_id":"1","status_bit":"1"} // need to generate alert with change: 0->1
{"device_id":"3","status_bits":"1"}
{"device_id":"2","status_bit":"1"}
{"device_id":"3","status_bits":"0"} // need to generate alert with change 1->0
Now I would like to write these alerts to another kafka topic like
{"device_id":1,"init":0,"final":1,"timestamp":"somets"}
{"device_id":3,"init":1,"final":0,"timestamp":"somets"}
I can save the current bit in the state store using something like
streamsBuilder
.stream("my-topic")
.mapValues((key, value) -> value.getStatusBit())
.groupByKey()
.windowedBy(TimeWindows.of(Duration.ofMinutes(1)))
.reduce((oldAggValue, newMessageValue) -> newMessageValue, Materialized.as("bit-temp-store"));
but I am unable to understand how can I detect this change from the existing bit. Do I need to query the state store somehow inside the processor topology? If yes? How? If no? What else could be done?
Any suggestions/ideas that I can try(maybe completely different from what I am thinking) are also appreciated. I am new to Kafka and thinking in terms of event driven streams is eluding me.
Thanks in advance.
I am not sure this is the best approach, but in the similar task I used an intermediate entity to capture the state change. In your case it will be something like
streamsBuilder.stream("my-topic").groupByKey()
.aggregate(DeviceState::new, new Aggregator<String, Device, DeviceState>() {
public DeviceState apply(String key, Device newValue, DeviceState state) {
if(!newValue.getStatusBit().equals(state.getStatusBit())){
state.setChanged(true);
}
state.setStatusBit(newValue.getStatusBit());
state.setDeviceId(newValue.getDeviceId());
state.setKey(key);
return state;
}
}, TimeWindows.of(…) …).filter((s, t) -> (t.changed())).toStream();
In the resulting topic you will have the changes. You can also add some attributes to DeviceState to initialise it first, depending whether you want to send the event, when the first device record arrives, etc.
I am trying to register to a "Device added/ Device removed" event using WMI. When I say device - I mean something in the lines of a Disk-On-Key or any other device that has files on it which I can access...
I am registering to the event, and the event is raised, but the EventType propery is different from the one I am expecting to see.
The documentation (MSDN) states : 1- config change, 2- Device added, 3-Device removed 4- Docking. For some reason I always get a value of 1.
Any ideas ?
Here's sample code :
public class WMIReceiveEvent
{
public WMIReceiveEvent()
{
try
{
WqlEventQuery query = new WqlEventQuery(
"SELECT * FROM Win32_DeviceChangeEvent");
ManagementEventWatcher watcher = new ManagementEventWatcher(query);
Console.WriteLine("Waiting for an event...");
watcher.EventArrived +=
new EventArrivedEventHandler(
HandleEvent);
// Start listening for events
watcher.Start();
// Do something while waiting for events
System.Threading.Thread.Sleep(10000);
// Stop listening for events
watcher.Stop();
return;
}
catch(ManagementException err)
{
MessageBox.Show("An error occurred while trying to receive an event: " + err.Message);
}
}
private void HandleEvent(object sender,
EventArrivedEventArgs e)
{
Console.WriteLine(e.NewEvent.GetPropertyValue["EventType"]);
}
public static void Main()
{
WMIReceiveEvent receiveEvent = new WMIReceiveEvent();
return;
}
}
Well, I couldn't find the code. Tried on my old RAC account, nothing. Nothing in my old backups. Go figure. But I tried to work out how I did it, and I think this is the correct sequence (I based a lot of it on this article):
Get all drive letters and cache
them.
Wait for the WM_DEVICECHANGE
message, and start a timer with a
timeout of 1 second (this is done to
avoid a lot of spurious
WM_DEVICECHANGE messages that start
as start as soon as you insert the
USB key/other device and only end
when the drive is "settled").
Compare the drive letters with the
old cache and detect the new ones.
Get device information for those.
I know there are other methods, but that proved to be the only one that would work consistently in different versions of windows, and we needed that as my client used the ActiveX control on a webpage that uploaded images from any kind of device you inserted (I think they produced some kind of printing kiosk).
Oh! Yup, I've been through that, but using the raw Windows API calls some time ago, while developing an ActiveX control that detected the insertion of any kind of media. I'll try to unearth the code from my backups and see if I can tell you how I solved it. I'll subscribe to the RSS just in case somebody gets there first.
Well,
u can try win32_logical disk class and bind it to the __Instancecreationevent.
You can easily get the required info
I tried this on my system and I eventually get the right code. It just takes a while. I get a dozen or so events, and one of them is the device connect code.