I'm approaching Lagom + CQRS/Event Sourcing for the first time and I would like to implement a behaviour like:
A service call is executed ( for example via a REST API call )
A command is run and triggers an event that mutates the state ( for example starts some kind of timer ).
After a pre-defined interval, the timer should expire so a new event should be triggered ( without other external commands ) to mutate the state in order to invalidate the timer.
The first two steps are straightforward, but once I trigger the TimerStartedEvent and mutate the state, how do I "schedule" an event after a fixed amount of time? How do I implement the third step?
I found a possibile implementation ( actually on the online-auction-scala sample code itself ).
Because Lagom it's built on top of Akka, it injects the ActorSystem, so you can use the system.scheduler.schedule call to schedule something in the future.
To answer the CQRS part of the question, the sample code does something like:
system.scheduler.scheduler(offset, delay) {
checkFinishBidding()
}
checkFinishBidding() = {
registry.refFor[Entity](id).ask(SomeCommand)
}
So when the time fires, you can dequeue the entity ref from the registry and run a command just like a normal service call.
Related
I am working on OpenCL implementation wherein the host side particular function has to call every time the clEnqueueReadBuffer is done executing.
I am calling the kernels in a loop. It will look like below in an ordered queue.
clEnqueueNDRangeKernel() -> clEnqueueReadBuffer(&Event) ->
clEnqueueNDRangeKernel() -> clEnqueueReadBuffer(&Event) .......
I have used clSetEventCall() to register Events in each read command to execute a callback function. I have observed that, though the command queue is an in-order queue, the order of the callback function does not execute in-order.
Also, in OpenCL 1.2, it has a mention as below.
The order in which the registered user callback functions are called
is undefined. There is no guarantee that the callback functions
registered for various execution status values for an event will be
called in the exact order that the execution status of a command
changes.
Can anyone give me a solution? I want to execute the callback function in order.
A simple solution could be to subscribe the same callback function to both events. In the callback code, you can check the status of each relevant event and perform the operation you want accordingly.
Note that on some implementations, the driver will batch multiple commands for execution.
The immediate effect is that that multiple events will be signaled "at once" even though the associated commands complete at a different time.
// event1 & event2 are likely to be signaled at once:
clEnqueueNDRangeKernel();
clEnqueueReadBuffer(&event1);
clEnqueueNDRangeKernel();
clEnqueueReadBuffer(&event2);
Wheres:
// event1 is likely to be signaled before event2:
clEnqueueNDRangeKernel();
clEnqueueReadBuffer(&event1);
clflush(queue);
clEnqueueNDRangeKernel();
clEnqueueReadBuffer(&event2);
clflush(queue);
I would also check on which exact thread the callbacks are invoked.
Is it the same thread each time? or a different one? If the implementation opens a new thread for this task, it might be wiser to open a single thread yourself and wait for events in the order that you wish.
I'm developing an Apache Spark job to run and I plan to deploy it as one stage in an AWS Step Function. Unfortunately the particular way that I wish to deploy it isn't directly supported by Step Functions at this time; however, Step Functions has an API for a generic Task that I can make use of. Essentially, once the task is started, it needs to periodically make a call to sendTaskHeartbeat and then on completion it needs to call sendTaskSuccess.
My Spark job is written in Scala, and I'm wondering what the best approach for running something on a timer is within the context of an Apache Spark job. I see from other answers that I could make use of java.util.concurrent or perhaps java.util.Timer, but I'm not sure how the threading would work specifically in a Spark context. Since Spark is already doing a lot to distribute my code across each node I'm not sure if there are some hidden considerations I need to be weary of (i.e. I don't really want more than one instance of my timer, I want to make sure it stops when the sparky parts of my code complete, etc.
Is it safe to use a regular Timer in a Spark job? If I did something like this:
val timer = new Timer()
val task = new TimerTask {
/* sendTaskHeartbeat */
}
timer.schedule(task, 1000L, 1000L)
val myRDD = spark.read.parquet(pathToParquetFiles)
val transformedRDD = myRDD.map( /* business logic */ )
transformedRDD.saveAsHadoopDataset(config) andThen task.cancel
Would that be sufficient? Or is there a risk that this code would lose track of the task and timer objects by the time it reaches the andThen, due to the distribution across nodes?
I believe your implement is sufficient. The timer task will only run in the driver node. (as long as you do not include them in the RDD transformation)
Only thing need to be careful is error handling. Make sure the timer task getting terminated when the transformation throws an error. otherwise your job could stuck because of timer thread is still alive.
I ended up making use of a combination of a java.util.Timer and a SparkListener. I instantiate the Timer on the onJobStart event (and only once, so if (TIMER == null) { /* instantiate */ }, because the onJobStart event seemingly can fire multiple times). And then I handle the completion activity on the onApplicationEnd event (which does only fire once). The reason I didn't use onApplicationStart was because it seemed like by the time I hooked in my listener to the Spark context, this event had already fired.
I am working with a library (ScalaJS and react specifically) where I have an interesting situation that I assume is pretty routine for an experienced reactive-programmer. I have a Component with State and a callback shouldComponentUpdate(State). The basic idea here is that certainly if the callback is triggered but the State has not changed from the last render, returnfalse. Otherwise, perhaps return true if the State change matters.
I am using a library monix but it seems identical to other reactive libraries so I would imagine this is a fairly context-independent question.
I would like to do something like: have some state that reflects the deltas of State since the last render. On each render, clear the buffer. Or, have a renderedState subject that reflects all rendered states as a sequence, a receivedState subject that reflects all received State updates, and a needsUpdate subject that reflects whether the latest receivedState matches the latest renderedState. I am having trouble actually executing these ideas, though. Here is where I am stuck at:
Here is what I've done for other callbacks:
lazy val channel_componentWillUpdate = channel_create[ComponentWillUpdate[Props, State, ResponsiveLayoutContainerBackend, TopNode]]
def componentWillUpdate(cwupd: ComponentWillUpdate[Props, State, ResponsiveLayoutContainerBackend, TopNode]) =
Callback {
channel_componentWillUpdate.onNext(cwupd)
}
So when then componentWillUpdate callback is triggered, the handler fires onNext on the channel (subject).
The shouldComponentUpdate is different though. It returns a value, so it needs to be structured differently. I am having trouble thinking of the right adjustment.
To summarize a bit:
react has callbacks at different stages of the view lifecycle, like componentDidMount, componentDidUpdate, etc.
I am handling all but one stage the same way - the shape of the callback is State -> Callback<Void> so alls I have to do is use a Subject for each type of lifecycle event and submit its onNext when the callback is triggered.
But one type of event has shape either State -> Boolean or State -> Callback<Boolean>.
I feel like I should be able to model this with a subject representing the delta between the last state rendered/received.
However, I don't know how this fits into the reactive style.
I'm having a difficult time trying to understand how the control flow in protractor work in relation to how JS event loop works. Here is what I know so far:
Protractor control flow stores commands that return promises in a queue. The first command will be at the front of the queue and the last command will be at the back. No command will be executed until the command in front of it has its promise resolved.
JS event loop stores asynchronous task (callbacks to be specific). Callbacks are not executed until all functions in the stack have completed and the stack is empty. Before running each callback, there is a check on whether the stack is empty or not.
so lets take this code for example. The code is basically clicking a search button and a api request is made. Then after data is returned, it checks whether the field that stores the returned data exists.
elem('#searchButton').click(); //will execute a api call to retrieve data
browser.wait(ExpectedConditions.presenceOf(elem('#resultDataField'),3000));
expect(elem('#resultDataField').isPresent()).toBeTruthy();
So with this code, I'm able to get it to work. But I don't know how it does it. How is the event loop applied in this scenario?
The core of the ControlFlow implementation is in runEventLoop_ (in Selenium's promise.js implementation).
As I understand it, the ControlFlow registers a call to runEventLoop_ with the JS event loop (e.g., with a 0-second timeout or somesuch). The call to runEventLoop_ can be thought of as a single iteration of a normal event loop. It registers code to actually run a scheduled task (i.e., actually do the work you queued up during your it). Once that task completes or fails (e.g., by hooking its async promise callbacks) the next iteration of runEventLoop_ is scheduled (see the calls to scheduleEventLoop in runEventLoop_).
There is some complexity when a callback ends up registering new promises (those need to be "inserted" before the old next event, this is accomplished by creating a "nested" control flow. Mostly you should never have to know this.)
I need to add an Action to a Schedule object that is being created through the API. There are documented interfaces to set almost all the options except the Action. How are Actions attached to these Objects?
When I attempt to programmatically add a new event, read from a separate configuration file, to a Schedule object I get errors stating that the Schedule has already been initialized and that I must construct a new object and add its configuration manually. I can do most of that using the available Schedule API. I can set up everything about the Schedule except the Action code.
The Schedule is used in a Process Model. Looking at the model in the Java editor, I see the code I'm trying to replicate via the API in a function that looks like this:
#Override
#AnyLogicInternalCodegenAPI
public void executeActionOf( EventTimeout _e ) {
if ( _e == _fuelDeliverySchedule_Action_xjal ) {
Schedule<Integer> self = this.fuelDeliverySchedule;
Integer value = fuelDeliverySchedule.getValue();
logger.info("{} received {} pounds of fuel", this.getName(), this.fuelDeliverySchedule.getValue());
this.fuelAvailablePounds += fuelDeliverySchedule.getValue();
;
_fuelDeliverySchedule_Action_xjal.restartTo( fuelDeliverySchedule.getTimeOfNextValue() );
return;
}
super.executeActionOf( _e );
}
Maybe I can use something like this to create my own action function, but I'm not sure how to make the Scheduled event use it.
Thanks,
Thom
[Edited (expanded/rewrote) 03.11.2014 after more user detail on the context.]
You clarified the context with
When I attempt to programatically add "a thing that happens", read
from a separate configuration file, to a Schedule object I get errors
stating that the Schedule has already been initialized and that I must
construct a new object and add its configuration manually. I can do
most of that using the available Schedule API. I can set up everything
about the Schedule except the Action code.
(You might want to edit that into the question... In general, it's always good to explain the context for why you're trying to do the thing.)
I think I understand now. I presume that your config file contains scheduling details and, when you say you were trying to "add a thing that happens" (which errored), you meant that you were trying to change the scheduling 'pattern' in the Schedule. So your problem is that, since you couldn't adjust a pre-existing schedule, you had to instantiate (create) your own programmatically, but the Schedule API doesn't allow you to set the action code (as seen on the GUI schedule element).
This is a fairly involved solution so bear with me. I give a brief 'tl;dr'
summary before diving into the detail.
Summary
You can't programmatically code an AnyLogic action (for any element) because that would amount to
dynamically creating a Java class. Solving your problem requires recognising
that the schedule GUI element creates both a Schedule instance and a
timeout event (EventTimeout) instance to trigger the action. You can therefore create these two elements explicitly yourself (the former dynamically). The trick is to reset the timeout event when you replace the Schedule instance (to trigger at the next 'flip' point of the new Schedule).
[Actually, from your wording, I suspect that the action is always the same but, for generality, I show how you could handle it if your config file details might want to change the nature of the action as well as those of the scheduling pattern.]
Detail
The issue is that the GUI element (confusingly) isn't just a Schedule instance
in terms of the code it generates. There is one (with the same name as that of
the GUI element), which just contains the schedule 'pattern' and, as in the API,
has methods to determine when the next on/off period (for an on/off schedule) occurs. (So
it is kind of fancy calendar functionality.) But AnyLogic also generates a
timeout event to actually perform the action; if you look further in the code
generated, you'll see stuff similar to the below (assuming your GUI schedule is called
fuelSchedule, with Java comments added by
me):
// Definition of the timeout event
#AnyLogicInternalCodegenAPI
public EventTimeout _fuelSchedule_Action_xjal = new EventTimeout(this);
// First occurrence time of the event as the next schedule on/off change
// time
#Override
#AnyLogicInternalCodegenAPI
public double getFirstOccurrenceTime( EventTimeout _e ) {
if ( _e == _fuelSchedule_Action_xjal ) return fuelSchedule.getTimeOfValue() == time() ? time() : fuelSchedule.getTimeOfNextValue();
return super.getFirstOccurrenceTime( _e );
}
// After your user action code, the event is rescheduled for the next
// schedule on/off change time
_fuelSchedule_Action_xjal.restartTo( fuelSchedule.getTimeOfNextValue() );
i.e., this creates an event which triggers each time the schedule 'flips', and performs the action specified in the GUI schedule element.
So there is no action to change on the Schedule instance; it's actually related to the EventTimeout instance. However, you can't programmatically change it there (or create a new one dynamically) for the same reason that you can't change the action of any AnyLogic element:
this would effectively be programmatically
creating a Java class definition, which isn't possible without very specialised
Java code. (You can create Java source code in a string and
dynamically run a Java compiler on it to generate a class. However, this is very
'advanced' Java, has lots of potential pitfalls, and I would definitely not
recommend going that route. You would also have to be creating source for a user subclass
of EventTimeout, since you don't know the correct source code for AnyLogic's proprietary EventTimeout class, and this might change per release in any case.)
But you shouldn't need to: there should be a strict set of possible actions that your config file can contain. (They can't be arbitrary Java code snippets, since they have to 'fit in' with the simulation.) So you can do what you want by programmatically creating the Schedule but with a GUI-created timeout event that you adjust accordingly(assuming an off/on schedule here and that there is
only one schedule active at once; obviously tweak this skeleton to your needs
and I haven't completely tested this in AnyLogic):
1. Have an AnyLogic variable activeAction which specifies the current active
action. (I take this as an int here for simplicity, but it's better to use a
Java enum which is the same as an AnyLogic 7 Option List, and can just be
created in raw Java in AnyLogic 6.)
2. Create a variable in the GUI, say called fuelSchedule, of type Schedule but with initial value null. Create a separate timeout event, say called fuelScheduleTrigger, in User Control mode, with action as:
// Perform the appropriate action (dependent on activeAction)
doAppropriateScheduleAction();
// Set the event to retrigger at the next schedule on/off switch time
fuelScheduleTrigger.restartTo(fuelSchedule.getTimeOfNextValue());
(Being in User Control mode, this event isn't yet triggered to initially fire, which is what we want.)
3. Code a set of functions for each of the different action alternatives; let's say
there are only 2 (fuelAction1 and fuelAction2) here as an example. Code
doAppropriateScheduleAction as:
if (activeAction == 1) {
fuelAction1();
}
else if (activeAction == 2) {
fuelAction2();
}
4. In your code which reads the config file and gets updated schedule info.
(presumably run from a cyclic timeout event or similar), have this replace
fuelSchedule with a new instance with the revised schedule pattern (as you've
been doing), set activeAction appropriately, and then reset the timeout event to
the new fuelSchedule.getTimeOfValue() time:
[set up fuelSchedule and activeAction]
// Reset schedule action to match revised schedule
fuelScheduleTrigger.restartTo(fuelSchedule.getTimeOfNextValue());
I think this works OK in the edge case when the new Schedule had its next 'flip' at the time
you set it up. (If you restart an event to the current time, I think it schedules an event OK at the current time which will occur next if there are no other events also scheduled for the current time; actually, it will definitely occur next if you are using a LIFO simultaneous-time-scheduling regime---see my blog post.)
Alternative & AnyLogic Enhancement
An alternative is to create a 'full' schedule in the GUI with action as earlier. Your config file reading code can replace the underlying Schedule instance and then reset the internal AnyLogic-generated timeout event. However, this is less preferable because you are relying on an internally-named AnyLogic event (which might also change in future AnyLogic releases, breaking your code).
AnyLogic could help this situation by adding a method to the Schedule API that gets the related timeout event; e.g., getActionTriggeringEventTimeout(). Then you would be able to 'properly' restart it and the Schedule API would make much clearer that the Schedule was always associated with an EventTimeout that did the triggering for the action.
Of course, AnyLogic could also go further by changing Schedule to allow scheduling details to be changed dynamically (and internally handling the required updates to the timeout event if it continued to be designed like that), but that's a lot more work and there may be deeper technical reasons why they wanted the schedule pattern to be fixed once the Schedule is initialised.
Any AnyLogic support staff reading?