How can I retrieve the first-completed Actor in a group of Actors in Scala? - scala

I have a moderate number of long-running Actors and I wish to write a synchronous function that returns the first one of these that completes. I can do it with a spin-wait on futures (e.g.,:
while (! fs.exists(f => f.isSet) ) {
Thread.sleep(100)
}
val completeds = fs.filter(f => f.isSet)
completeds.head()
), but that seems very "un-Actor-y"
The scala.actors.Futures class has two methods awaitAll() and awaitEither() that seem awfully close; if there were an awaitAny() I'd jump on it. Am I missing a simple way to do this or is there a common pattern that is applicable?

A more "actorish" way of waiting for completion is creating an actor in charge of handling completed result (lets call it ResultHandler)
Instead of replying, workers send their answer to ResultHandler in fire-and-forget manner. The latter will continue processing the result while other workers complete their job.

The key for me was the discovery that every (?) Scala object is, implicitly, an Actor, so you can use Actor.react{ } to block. Here is my source code:
import scala.actors._
import scala.actors.Actor._
//Top-level class that wants to return the first-completed result from some long-running actors
class ConcurrentQuerier() {
//Synchronous function; perhaps fulfilling some legacy interface
def synchronousQuery : String = {
//Instantiate and start the monitoring Actor
val progressReporter = new ProgressReporter(self) //All (?) objects are Actors
progressReporter.start()
//Instantiate the long-running Actors, giving each a handle to the monitor
val lrfs = List (
new LongRunningFunction(0, 2000, progressReporter), new LongRunningFunction(1, 2500, progressReporter), new LongRunningFunction(3, 1500, progressReporter),
new LongRunningFunction(4, 1495, progressReporter), new LongRunningFunction(5, 1500, progressReporter), new LongRunningFunction(6, 5000, progressReporter) )
//Start 'em
lrfs.map{ lrf =>
lrf.start()
}
println("All actors started...")
val start = System.currentTimeMillis()
/*
This blocks until it receives a String in the Inbox.
Who sends the string? A: the progressReporter, which is monitoring the LongRunningFunctions
*/
val s = receive {
case s:String => s
}
println("Received " + s + " after " + (System.currentTimeMillis() - start) + " ms")
s
}
}
/*
An Actor that reacts to a message that is a tuple ("COMPLETED", someResult) and sends the
result to this Actor's owner. Not strictly necessary (the LongRunningFunctions could post
directly to the owner's mailbox), but I like the idea that monitoring is important enough
to deserve its own object
*/
class ProgressReporter(val owner : Actor) extends Actor {
def act() = {
println("progressReporter awaiting news...")
react {
case ("COMPLETED", s) =>
println("progressReporter received a completed signal " + s);
owner ! s
case s =>
println("Unexpected message: " + s ); act()
}
}
}
/*
Some long running function
*/
class LongRunningFunction(val id : Int, val timeout : Int, val supervisor : Actor) extends Actor {
def act() = {
//Do the long-running query
val s = longRunningQuery()
println(id.toString + " finished, sending results")
//Send the results back to the monitoring Actor (the progressReporter)
supervisor ! ("COMPLETED", s)
}
def longRunningQuery() : String = {
println("Starting Agent " + id + " with timeout " + timeout)
Thread.sleep(timeout)
"Query result from agent " + id
}
}
val cq = new ConcurrentQuerier()
//I don't think the Actor semantics guarantee that the result is absolutely, positively the first to have posted the "COMPLETED" message
println("Among the first to finish was : " + cq.synchronousQuery)
Typical results look like:
scala ActorsNoSpin.scala
progressReporter awaiting news...
All actors started...
Starting Agent 1 with timeout 2500
Starting Agent 5 with timeout 1500
Starting Agent 3 with timeout 1500
Starting Agent 4 with timeout 1495
Starting Agent 6 with timeout 5000
Starting Agent 0 with timeout 2000
4 finished, sending results
progressReporter received a completed signal Query result from agent 4
Received Query result from agent 4 after 1499 ms
Among the first to finish was : Query result from agent 4
5 finished, sending results
3 finished, sending results
0 finished, sending results
1 finished, sending results
6 finished, sending results

Related

RxJava/RxScala backpressure using request

I am having an issue using RxJava backpressure. Basically, I have one producer that produces more items than the consumer can handle and want to have some buffer queue to handle only items that I can deal with, and request when I complete some of them, like in this example:
object Tester extends App {
Observable[Int] { subscriber =>
(1 to 100).foreach { e =>
subscriber.onNext(e)
Thread.sleep(100)
println("produced " + e + "(" + Thread.currentThread().getName + Thread.currentThread().getId + ")")
}
}
.subscribeOn(NewThreadScheduler())
.observeOn(ComputationScheduler())
.subscribe(
new Subscriber[Int]() {
override def onStart(): Unit = {
request(2)
}
override def onNext(value: Int): Unit = {
Thread.sleep(1000)
println("consumed " + value + "(" + Thread.currentThread().getName + Thread.currentThread().getId + ")")
request(1)
}
override def onCompleted(): Unit = {
println("finished ")
}
})
Thread.sleep(100000)
I expect to get output like
produced 1(RxNewThreadScheduler-113)
consumed 1(RxComputationThreadPool-312)
produced 2(RxNewThreadScheduler-113)
consumed 2(RxComputationThreadPool-312)
produced 3(RxNewThreadScheduler-113)
consumed 3(RxComputationThreadPool-312)
......
but instead, I get
produced 1(RxNewThreadScheduler-113)
produced 2(RxNewThreadScheduler-113)
produced 3(RxNewThreadScheduler-113)
produced 4(RxNewThreadScheduler-113)
produced 5(RxNewThreadScheduler-113)
produced 6(RxNewThreadScheduler-113)
produced 7(RxNewThreadScheduler-113)
produced 8(RxNewThreadScheduler-113)
produced 9(RxNewThreadScheduler-113)
consumed 1(RxComputationThreadPool-312)
produced 10(RxNewThreadScheduler-113)
produced 11(RxNewThreadScheduler-113)
produced 12(RxNewThreadScheduler-113)
produced 13(RxNewThreadScheduler-113)
.....
When you implement your Observable using Observable.create it is up to you to manage backpressure (which is not a simple task). Here your observable simply ignores reactive pull requests (you just iterate, not waiting for a request to call the iterator's next() method).
If possible, try to use Observable factory methods like range, etc... and composing using map/flatMap to obtain the desired source Observable, as those will respect backpressure.
Otherwise, have a look at the experimental utility classes introduced recently for correctly managing backpressure in a OnSubscribe implementation: AsyncOnSubscribe and SyncOnSubscribe.
Here is a quite naïve example:
Observable<Integer> backpressuredObservable =
Observable.create(SyncOnSubscribe.createStateful(
() -> 0, //starts the state at 0
(state, obs) -> {
int i = state++; //first i is 1 as desired
obs.next(i);
if (i == 100) { //maximum is 100, stop there
obs.onCompleted();
}
return i; //update the state
}));

Omitting all Scala Actor messages except the last

I want omit all the same type of messages except the last one:
def receive = {
case Message(type:MessageType, data:Int) =>
// remove previous and get only last message of passed MessageType
}
for example when I send:
actor ! Message(MessageType.RUN, 1)
actor ! Message(MessageType.RUN, 2)
actor ! Message(MessageType.FLY, 1)
then I want to recevie only:
Message(MessageType.RUN, 2)
Message(MessageType.FLY, 1)
Of course if they will be send very fast, or on high CPU load
You could wait a very short amount of time, storing the most recent messages that arrive, and then process only those most recent ones. This can be accomplished by sending messages to yourself, and scheduleOnce. See the second example under the Akka HowTo: Common Patterns, Scheduling Periodic Messages. Instead of scheduling ticks whenever the last tick ends, you can wait until new messages arrive. Here's an example of something like that:
case class ProcessThis(msg: Message)
case object ProcessNow
var onHold = Map.empty[MessageType, Message]
var timer: Option[Cancellable] = None
def receive = {
case msg # Message(t, _) =>
onHold += t -> msg
if (timer.isEmpty) {
import context.dispatcher
timer = Some(context.system.scheduler.scheduleOnce(1 millis, self, ProcessNow))
}
case ProcessNow =>
timer foreach { _.cancel() }
timer = None
for (m <- onHold.values) self ! ProcessThis(m)
onHold = Map.empty
case ProcessThis(Message(t, data)) =>
// really process the message
}
Incoming Messages are not actually processed right away, but are stored in a Map that keeps only the last of each MessageType. On the ProcessNow tick message, they are really processed.
You can change the length of time you wait (in my example set to 1 millisecond) to strike a balance between responsivity (length of time from a message arriving to response) and efficiency (CPU or other resources used or held up).
type is not a good name for a field, so let's use messageType instead. This code should do what you want:
var lastMessage: Option[Message] = None
def receive = {
case m => {
if (lastMessage.fold(false)(_.messageType != m.messageType)) {
// do something with lastMessage.get
}
lastMessage = Some(m)
}
}

Akka actors' state not monitored properly

Monitoring Akka actors' state is said to be possible by using .underlyingActor.
In the example below, there is an ActorWithState using an Integer counter. The process of counter's incrementation and decrementation is tested by an ActorWithStateTest. The incrementation step always passes. However, counter's decrementation does not seem to work in that the 2nd assert always fails. The error message is the following. What is wrong?
[Error message of this test]
[info] ActorWithStateTest
[info] - should validate counter incrementation and decrementation *** FAILED ***
[info] 1 did not equal 0, Expected counter to be 0 after 'Terminated' message
[Actor with counter]
object ActorWithState {
case class Inc;
}
class ActorWithState(snooper: ActorRef) extends Actor with ActorLogging {
var counter = 0
def receive = {
case Inc => counter += 1
case Terminated(ref) => counter -= 1
}
}
[ActorTest shall test counter's behaviour]
class ActorWithStateTest extends TestKit(ActorSystem("SimpleTestSpec")) {
"Test" should {
"validate counter incrementation and decrementation" in {
val aws = TestActorRef(Props(classOf[ActorWithState], testActor), name = "aws")
val awsA: ActorWithState = aws.underlyingActor
// tell aws to increment its counter
aws ! ActorWithState.Inc
// this assert always passes
assert(awsA.counter == 1, ", Expected counter to be 1 after 'Inc' message")
// triggers a 'Terminated' message being sent to aws
val tp = TestProbe()
aws watch tp.ref
system stop tp.ref
// [EDIT] the following assert passes if some - time consuming - processing is added here
// the following assert does NOT pass, WHY?
assert(awsA.counter == 0, ", Expected counter to be 0 after 'Terminated' message")
}
}
[akka-actor: 2.3.2, akka-testkit: 2.3.2, scalatest: 2.0]
As mentioned in my comment, your issue stems from the stopping of an actor being async. Your assertion is happening before the Terminated message hits your actor that you are testing. One quick and dirty way to fix this is to introduce another probe that also listens for the terminated event and perform an assertion on that first. That will wait until the terminated is received by the second watcher which should mean that your actor has also received the terminated event and you can perform your assertion. This worked for me:
val tp = TestProbe()
val tp2 = TestProbe()
aws watch tp.ref
tp2 watch tp.ref
system stop tp.ref
tp2.expectTerminated(tp.ref)
assert(awsA.counter == 0, ", Expected counter to be 0 after 'Terminated' message")

Why scheduleOnce runs everytime on app shutdown?

I have Play 2 app.
And code something like that.
So, I run app in start mode and run mode.
val calendar = Calendar.getInstance()
calendar.set(Calendar.HOUR_OF_DAY, 19)
calendar.set(Calendar.MINUTE, 0)
calendar.set(Calendar.SECOND, 0)
calendar.set(Calendar.MILLISECOND, 0)
val now = new Date()
val timeDifference = calendar.getTime.getTime - now.getTime
if (timeDifference >= 0) {
val initialDelay = Duration.create(timeDifference, duration.MILLISECONDS)
Akka.system.scheduler.scheduleOnce(initialDelay, new Runnable {
override def run() {
LOGGER.error(System.lineSeparator() +
"=================================================" + System.lineSeparator() +
"| Forming and send report |" + System.lineSeparator() +
"=================================================" + System.lineSeparator()
)
}
})
}
Why scheduleOnce runs on shutdown?
The behavior you are seeing here actually effects all tasks scheduled in the default scheduler (LightArrayResolverScheduler) whether they were scheduled with schedule or scheduleOnce. If you look at the close method on LightArrayResolverScheduler, you will see this logic:
override def close(): Unit = Await.result(stop(), getShutdownTimeout) foreach {
task =>
try task.run() catch {
case e: InterruptedException => throw e
case _: SchedulerException => // ignore terminated actors
case NonFatal(e) => log.error(e, "exception while executing timer task")
}
}
So basically, it looks like during a shutdown of the timer that it will collect all outstanding tasks and execute them serially. I suppose this would not be a big deal if your scheduling was just sending messages to actors (as opposed to using a Runnable) as those actors would hopefully be terminated by this point and result in nothing happening (dead letters).
If you want to avoid this behavior, you could store the result of the call to scheduleOnce and explicitly cancel it first before you initiate your shutdown process.

How to set a socket read timeout in IOManager

In Akka, IOManager "is the recommended entry point to creating sockets for performing IO." I'm looking at the API and was wondering how to set a read timeout? Of course, I can just schedule an actor to fire a message in n seconds to close the socket, but it may have already received all the read in that time and is now working on processing the read-in data. So, it's not truly a read timeout. Any ideas how to do this? Or must I somehow introduce some state to my actor?
Ok, Derek Williams gave me a hint on akka-user. Here's the code, just in case anyone else needs to do something similar.
When we accept a new client, we set a timer for 5 seconds to close the connection.
def receive = {
case IO.NewClient(server) =>
val socket = server.accept()
val readTimeout = context.system.scheduler.scheduleOnce(5 seconds, self, Timeout(socket))
state(socket) flatMap (_ => MyServer.processRequest(socket, readTimeout))
case IO.Read(socket, bytes) =>
state(socket)(IO Chunk bytes)
case IO.Closed(socket, cause) =>
state(socket)(IO EOF None)
state -= socket
case Timeout(socket) =>
socket.close()
}
And to cancel the timeout after we've read, we call cancel() on the Cancellable schedule.
object MyServer {
def processRequest(socket: IO.SocketHandle, readTimeout: Cancellable): IO.Iteratee[Unit] =
for {
request <- readRequest
} yield {
readTimeout.cancel()
request match {
val response = ...
socket write ByteString(response).compact
socket.close()
}
}
}