I start two remote actors on one host which just echo whatever is sent to them. I then create another actor which sends some number of messages (using !! ) to both actors and keep a List of Future objects holding the replies from these actors. Then I loop over this List fetching the result of each Future. The problem is that most of the time some futures never return, even thought the actor claims it has sent the reply. The problem happens randomly, sometimes it will get through the whole list, but most of the time it gets stuck at some point and hangs indefinitely.
Here is some code which produces the problem on my machine:
Sink.scala:
import scala.actors.Actor
import scala.actors.Actor._
import scala.actors.Exit
import scala.actors.remote.RemoteActor
import scala.actors.remote.RemoteActor._
object Sink {
def main(args: Array[String]): Unit = {
new RemoteSink("node03-0",43001).start()
new RemoteSink("node03-1",43001).start()
}
}
class RemoteSink(name: String, port: Int) extends Actor
{
def act() {
println(name+" starts")
trapExit=true
alive(port)
register(Symbol(name),self)
loop {
react {
case Exit(from,reason) =>{
exit()
}
case msg => reply{
println(name+" sending reply to: "+msg)
msg+" back at you from "+name
}
}
}
}
}
Source.scala:
import scala.actors.Actor
import scala.actors.Actor._
import scala.actors.remote.Node;
import scala.actors.remote.RemoteActor
import scala.actors.remote.RemoteActor._
object Source {
def main(args: Array[String]):Unit = {
val peer = Node("127.0.0.1", 43001)
val source = new RemoteSource(peer)
source.start()
}
}
class RemoteSource(peer: Node) extends Actor
{
def act() {
trapExit=true
alive(43001)
register(Symbol("source"),self)
val sinks = List(select(peer,Symbol("node03-0"))
,select(peer,Symbol("node03-1"))
)
sinks.foreach(link)
val futures = for(sink <- sinks; i <- 0 to 20) yield sink !! "hello "+i
futures.foreach( f => println(f()))
exit()
}
}
What am I doing wrong?
I'm guessing your problem is due to this line:
futures.foreach( f => println(f()))
in which you loop through all your futures and block on each in turn, waiting for its result. Blocking on futures is generally a bad idea and should be avoided. What you want to do instead is specify an action to carry out when the future's result is available. Try this:
futures.foreach(f => f.foreach(r => println(r)))
Here's an alternate way to say that with a for comprehension:
for (future <- futures; result <- future) { println(result) }
This blog entry is an excellent primer on the problem of blocking on futures and how monadic futures overcome it.
I have seen a similar case as well. When the code inside the thread throws certain types of exception and exits, the corresponding future.get never returns. One can try with raising an exception of java.lang.Error vs java.lang.NoSuchMethodError. The latter's corresponding future would never return.
Related
I'm accessing data from a REST endpoint :
"https://api-public.sandbox.pro.coinbase.com/products/BTC-EUR/ticker"
To access the data once per second I use an infinite loop while(true) { to invoke a message send to the actor once per second which begins the process of invoking the REST request:
The actor to access the data is:
object ProductTickerRestActor {
case class StringData(data: String)
}
class ProductTickerRestActor extends Actor {
override def receive: PartialFunction[Any, Unit] = {
case ProductTickerRestActor.StringData(data) =>
try {
println("in ProductTickerRestActor")
val rData = scala.io.Source.fromURL("https://api-public.sandbox.pro.coinbase.com/products/BTC-EUR/ticker").mkString
println("rData : "+rData)
}
catch {
case e: Exception =>
println("Exception thrown in ProductTickerRestActor: " + e.getMessage)
}
case msg => println(s"I cannot understand ${msg.toString}")
}
}
I start the application using:
object ExchangeModelDataApplication {
def main(args: Array[String]): Unit = {
val actorSystem = ActorSystemConfig.getActorSystem
val priceDataActor = actorSystem.actorOf(Props[ProductTickerRestActor], "ProductTickerRestActor")
val throttler = Throttlers.getThrottler(priceDataActor)
while(true) {
throttler ! ProductTickerRestActor.StringData("test")
Thread.sleep(1000)
}
}
Throttler:
object Throttlers {
implicit val materializer = ActorMaterializer.create(ActorSystemConfig.getActorSystem)
def getThrottler(priceDataActor: ActorRef) = Source.actorRef(bufferSize = 1000, OverflowStrategy.dropNew)
.throttle(1, 1.second)
.to(Sink.actorRef(priceDataActor, NotUsed))
.run()
}
How to run the following code asynchronously instead of blocking using an infinite loop? :
throttler ! ProductTickerRestActor.StringData("test")
Thread.sleep(1000)
Also, the throttler, in this case, maybe redundant as I'm throttling requests within the loop regardless.
I would just use Akka Streams for this along with Akka HTTP. Using Akka 2.6.x, something along these lines would be sufficient for 1 request/second
import akka.actor.ActorSystem
import akka.http.scaladsl.Http
import akka.http.scaladsl.model._
import akka.stream.scaladsl._
import scala.concurrent.duration._
object HTTPRepeatedly {
implicit val system = ActorSystem()
import system.dispatcher
val sourceFromHttp: Source[String, NotUsed] =
Source.repeated("test") // Not sure what "test" is actually used for here...
.throttle(1, 1.second)
.map { str =>
HttpRequest(uri = "https://api-public.sandbox.pro.coinbase.com/products/BTC-EUR/ticker")
}.mapAsync(1) { req =>
Http().singleRequest(req)
}.mapAsync(1)(_.entity.toStrict(1.minute))
.map(_.data.decodeString(java.nio.charset.StandardCharsets.UTF_8))
}
Then you could, for instance (for simplicity, put this in a main within HTTPRepeatedly so the implicits are in scope etc.)
val done: Future[Done] =
sourceFromHttp
.take(10) // stop after 10 requests
.runWith(Sink.foreach { rData => println(s"rData: $rData") })
scala.concurrent.Await.result(done, 11.minute)
system.terminate()
Sending a request every second is not a good idea. If, for some reason, the request is delayed, you are going to get a lot of requests piled up. Instead, send the next request one second after the previous requests completes.
Because this code uses a synchronous GET request, you can just send the next request one second after mkString returns.
But using synchronous requests is not a good way to consume a RESTful API in Akka. It blocks the actor receive method until the request is complete, which can eventually block the whole ActorSystem.
Instead, use Akka Http and singleRequest to perform an asynchronous request.
Http().singleRequest(HttpRequest(uri = "https://api-public.sandbox.pro.coinbase.com/products/BTC-EUR/ticker"))
This returns a Future. Make the new request one second after the request completes (e.g. using onComplete on the Future).
Not only is this safer and more asynchronous, it also provides much more control over the REST API call than fromUrl
I'm working currently on a Play application with a background job which should send mails periodically for which I want to use Akka. I have to add that I'm really new to Scala/Play/Akka.
Currently I have the following setup:
// JobModule.scala
bind(classOf[MailJobScheduler]).asEagerSingleton()
This should start up the following piece of code which does work every second
// MailJobScheduler.scala
val mailActor = actorSystem.actorOf(MailActor.props, "mail-actor")
actorSystem.scheduler.schedule(0 seconds, 1 seconds) {
// check how many mails have to be sent and sent messages to the mailActor
}
It might be that each second multiple new mails should be send. I am wondering: in case I send every second 10 messages to the mailActor will it be really only one actor having to do all the work or will there be multiple actors doing the work concurrently?
If it is one actor how can I have multiple actors for which I can assign the work and how many can/should I have?
How about using Akka streams instead?
import akka.Done
import akka.stream.{KillSwitch, KillSwitches, OverflowStrategy}
import akka.stream.scaladsl.{Keep, Sink, Source}
import scala.concurrent.duration._
import scala.concurrent.Future
object BatchEmailSender {
sealed trait Msg
case object Tick extends Msg
case class Email(toAddress: String, body: String) extends Msg
def apply(sendEmail: Email => Future[Done], sendInterval: FiniteDuration = 10.seconds)(implicit mat: ActorMaterializer)
: (Email => Unit, KillSwitch) = {
val emailQueue = scala.collection.mutable.Queue[Email]()
val (emailCmdQueue, killSwitch) = Source.queue[Msg](0, OverflowStrategy.backpressure)
.merge(Source.tick(0.seconds, sendInterval, Tick))
.viaMat(KillSwitches.single)(Keep.both)
.toMat(Sink.foreach {
case newEmail: Email =>
emailQueue.enqueue(newEmail)
case Tick =>
emailQueue.dequeueAll(_ => true).foreach { email =>
sendEmail(email).onFailure { case e =>
println(s"Error sending email to ${email.toAddress}: $e")
}
}
})(Keep.left)
.run()
(emailCmdQueue.offer(_), killSwitch)
}
}
You need a sendEmail function, and then it would work like this:
import scala.concurrent.ExecutionContext.Implicits.global // TODO: remove me
object TestApp extends App {
import BatchEmailSender._
implicit val system = ActorSystem()
implicit val materializer = ActorMaterializer()
def sendEmail(email: Email): Future[Done] ={
println(s"Sending email $email") // TODO: insert real email sender code here
Future.successful(Done)
}
val (sendEmailEvery10s, killSwitch) = BatchEmailSender(sendEmail)
sendEmailEvery10s(Email("foo#bar.com", "Email will arrive in 10s"))
sendEmailEvery10s(Email("foo#bar.com", "Email will arrive in same batch"))
Thread.sleep(11000)
sendEmailEvery10s(Email("foo#bar.com", "Email will arrive after another 10s"))
Thread.sleep(11000)
killSwitch.shutdown()
}
I may have just complicated your life, but Akka streams lets you do these things without worrying about which actor does what, has back-pressure and typically is much more robust code.
I would have used 1 actor if Akka streams did not exist. Accumulate all messages in the actor and then send a tick to itself periodically.
Use the scheduler as you are doing in your example, but I don't see how the mailActor helps you here.
actorSystem.scheduler.schedule(0 seconds, 1 seconds) {
// just call the code the the checks for email
}
Don't assume that there will be one thread. i.e. be extra careful to not close over unstable references
There are many questions on SO that combine Futures with Timeout. To be honest, I haven't completely understood how to use them. But it seems I have stumbled upon a problem where I will have to (or maybe not).
I want to throw a TimeoutException if a statement takes more than say 1 minute.To be more clear, currently, this statement tries to get a response from a server but does not throw if the server is not setup. It currently looks like this:
//proper import of exceptions
case class ServerException(exception: Throwable) extends Exception(exception)
//Code that instantiates client and post
val response = try {
client.execute(post)
} catch {
case e#(_: IOException | _: ClientProtocolException) => throw new ServerException(e)
}
To mitigate this problem, I want to introduce a timeout. How do I introduce timeout to this statement such that it throws if no response is got within one minute, else it instantiates response and the program continues as it is?
It's not available in scala Futures. You can switch to scalaz Task - it's a bit different abstraction for async/delayed computations. You can read awesome documentation for it here: http://timperrett.com/2014/07/20/scalaz-task-the-missing-documentation/
import java.util.concurrent.Executors
import scalaz.concurrent.Task
import scala.concurrent.duration._
implicit val scheduledThreadPool =
Executors.newScheduledThreadPool(5)
def executeRequest(req: Request): Task[Response] = ???
val withTimeOut: Task[Response] =
executeRequest(req).timed(1.minute)
Update
Btw you can easily transform your Future to Task, for example it Future is coming from 3rd party lib
object Future2Task {
implicit class Transformer[+T](fut: => Future[T]) {
def toTask(implicit ec: scala.concurrent.ExecutionContext): Task[T] = {
import scala.util.{Failure, Success}
import scalaz.syntax.either._
Task.async {
register =>
fut.onComplete {
case Success(v) => register(v.right)
case Failure(ex) => register(ex.left)
}
}
}
}
}
Timeouts are usually implemented by having an asynchronous timer act as the timeout signal and completing the future in question whenever it or the timer completes.
I believe Akka has a such a timer, but it's pretty simple to roll your own:
object ConcurrencyUtil {
// creates a Future that will complete after a specified duration
object Delay {
def apply(d: Duration): Future[Unit] = {
val p = Promise[Unit]()
val t = new Timer
t.schedule(new TimerTask {
override def run(): Unit = p.success()
}, d.toMillis)
p.future
}
}
implicit class FutureExtensions[T](future: Future[T]) {
def timeout(timeout: Duration) = Future.firstCompletedOf(Seq(
Delay(timeout).map(_ => throw new TimeoutException()),
future
))
}
}
Now you can compose timeout with your future like this:
import ConcurrencyUtil._
val f = someTaskReturningAFuture.timeout(1.minute)
What is now if the task has not completed within 1 minute, the delay will fire, get mapped to throwing a TimeoutException and complete the future f as failed.
Note: This does not address cancellation, i.e. the other future, while no longer being listened for will continue to exist and if it's executing something, continue to execute.
I'm trying to make two external calls (to a Redis database) inside an Actor's receive method. Both calls return a Future and I need the result of the first Future inside the second.
I'm wrapping both calls inside a Redis transaction to avoid anyone else from modifying the value in the database while I'm reading it.
The internal state of the actor is updated based on the value of the second Future.
Here is what my current code looks like which I is incorrect because I'm updating the internal state of the actor inside a Future.onComplete callback.
I cannot use the PipeTo pattern because I need both both Future have to be in a transaction.
If I use Await for the first Future then my receive method will block.
Any idea how to fix this ?
My second question is related to how I'm using Futures. Is this usage of Futures below correct? Is there a better way of dealing with multiple Futures in general? Imagine if there were 3 or 4 Future each depending on the previous one.
import akka.actor.{Props, ActorLogging, Actor}
import akka.util.ByteString
import redis.RedisClient
import scala.concurrent.Future
import scala.util.{Failure, Success}
object GetSubscriptionsDemo extends App {
val akkaSystem = akka.actor.ActorSystem("redis-demo")
val actor = akkaSystem.actorOf(Props(new SimpleRedisActor("localhost", "dummyzset")), name = "simpleactor")
actor ! UpdateState
}
case object UpdateState
class SimpleRedisActor(ip: String, key: String) extends Actor with ActorLogging {
//mutable state that is updated on a periodic basis
var mutableState: Set[String] = Set.empty
//required by Future
implicit val ctx = context dispatcher
var rClient = RedisClient(ip)(context.system)
def receive = {
case UpdateState => {
log.info("Start of UpdateState ...")
val tran = rClient.transaction()
val zf: Future[Long] = tran.zcard(key) //FIRST Future
zf.onComplete {
case Success(z) => {
//SECOND Future, depends on result of FIRST Future
val rf: Future[Seq[ByteString]] = tran.zrange(key, z - 1, z)
rf.onComplete {
case Success(x) => {
//convert ByteString to UTF8 String
val v = x.map(_.utf8String)
log.info(s"Updating state with $v ")
//update actor's internal state inside callback for a Future
//IS THIS CORRECT ?
mutableState ++ v
}
case Failure(e) => {
log.warning("ZRANGE future failed ...", e)
}
}
}
case Failure(f) => log.warning("ZCARD future failed ...", f)
}
tran.exec()
}
}
}
The compiles but when I run it gets struck.
2014-08-07 INFO [redis-demo-akka.actor.default-dispatcher-3] a.e.s.Slf4jLogger - Slf4jLogger started
2014-08-07 04:38:35.106UTC INFO [redis-demo-akka.actor.default-dispatcher-3] e.c.s.e.a.g.SimpleRedisActor - Start of UpdateState ...
2014-08-07 04:38:35.134UTC INFO [redis-demo-akka.actor.default-dispatcher-8] r.a.RedisClientActor - Connect to localhost/127.0.0.1:6379
2014-08-07 04:38:35.172UTC INFO [redis-demo-akka.actor.default-dispatcher-4] r.a.RedisClientActor - Connected to localhost/127.0.0.1:6379
UPDATE 1
In order to use pipeTo pattern I'll need access to the tran and the FIRST Future (zf) in the actor where I'm piping the Future to because the SECOND Future depends on the value (z) of FIRST.
//SECOND Future, depends on result of FIRST Future
val rf: Future[Seq[ByteString]] = tran.zrange(key, z - 1, z)
Without knowing too much about the redis client you are using, I can offer an alternate solution that should be cleaner and won't have issues with closing over mutable state. The idea is to use a master/worker kind of situation, where the master (the SimpleRedisActor) receives the request to do the work and then delegates off to a worker that performs the work and responds with the state to update. That solution would look something like this:
object SimpleRedisActor{
case object UpdateState
def props(ip:String, key:String) = Props(classOf[SimpleRedisActor], ip, key)
}
class SimpleRedisActor(ip: String, key: String) extends Actor with ActorLogging {
import SimpleRedisActor._
import SimpleRedisWorker._
//mutable state that is updated on a periodic basis
var mutableState: Set[String] = Set.empty
val rClient = RedisClient(ip)(context.system)
def receive = {
case UpdateState =>
log.info("Start of UpdateState ...")
val worker = context.actorOf(SimpleRedisWorker.props)
worker ! DoWork(rClient, key)
case WorkResult(result) =>
mutableState ++ result
case FailedWorkResult(ex) =>
log.error("Worker got failed work result", ex)
}
}
object SimpleRedisWorker{
case class DoWork(client:RedisClient, key:String)
case class WorkResult(result:Seq[String])
case class FailedWorkResult(ex:Throwable)
def props = Props[SimpleRedisWorker]
}
class SimpleRedisWorker extends Actor{
import SimpleRedisWorker._
import akka.pattern.pipe
import context._
def receive = {
case DoWork(client, key) =>
val trans = client.transaction()
trans.zcard(key) pipeTo self
become(waitingForZCard(sender, trans, key) orElse failureHandler(sender, trans))
}
def waitingForZCard(orig:ActorRef, trans:RedisTransaction, key:String):Receive = {
case l:Long =>
trans.zrange(key, l -1, l) pipeTo self
become(waitingForZRange(orig, trans) orElse failureHandler(orig, trans))
}
def waitingForZRange(orig:ActorRef, trans:RedisTransaction):Receive = {
case s:Seq[ByteString] =>
orig ! WorkResult(s.map(_.utf8String))
finishAndStop(trans)
}
def failureHandler(orig:ActorRef, trans:RedisTransaction):Receive = {
case Status.Failure(ex) =>
orig ! FailedWorkResult(ex)
finishAndStop(trans)
}
def finishAndStop(trans:RedisTransaction) {
trans.exec()
context stop self
}
}
The worker starts the transaction and then makes calls into redis and ultimately completes the transaction before stopping itself. When it calls redis, it gets the future and pipes back to itself for the continuation of the processing, changing the receive method between as a mechanism of showing progressing through its states. In a model like this (which I suppose is somewhat similar to the error kernal pattern), the master owns and protects the state, delegating the "risky" work off to a child who can figure out what the change for the state should be, but the changing is still owned by the master.
Now again, I have no idea about the capabilities of the redis client you are using and if it is safe enough to even do this kind of stuff, but that's not really the point. The point was to show a safer structure for doing something like this that involves futures and state that needs to be changed safely.
Using the callback to mutate internal state is not a good idea, excerpt from the akka docs:
When using future callbacks, such as onComplete, onSuccess, and onFailure, inside actors you need to carefully avoid closing over the containing actor’s reference, i.e. do not call methods or access mutable state on the enclosing actor from within the callback.
Why do you worry about pipeTo and transactions?
Not sure how redis transactions work, but I would guess that the transaction does not encompass the onComplete callback on the second future anyways.
I would put the state into a separate actor which you pipe the future too. This way you have a separate mailbox, and the ordering there will be the same as the ordering of the messages that came in to modify the state. Also if any read requests come in, they will also be put in the correct order.
Edit to respond to edited question: Ok, so you don't want to pipe the first future, that makes sense, and should be no problem as the first callback is harmless. The callback of the second future is the problem, as it manipulates the state. But this future can be pipe without the need for access to the transaction.
So basically my suggestion is:
val firstFuture = tran.zcard
firstFuture.onComplete {
val secondFuture = tran.zrange
secondFuture pipeTo stateActor
}
With stateActor containing the mutable state.
I'm trying to grok Futures and ask pattern in akka.
So, I make two actors, and one asking another to send him back a message. Well, according to akka's Futures documentation, actor should ask(?) for message and it shall give him a Future instanse. Then actor should block (using Await) to get Future results.
Well, I never get my future done. Why is that?
Code is:
package head_thrash
import akka.actor._
import akka.util.Timeout
import scala.concurrent.Await
import scala.concurrent.duration._
object Main extends App {
val system = ActorSystem("actors")
val actor1 = system.actorOf(Props[MyActor], "node_1")
val actor2 = system.actorOf(Props[MyActor], "node_2")
actor2 ! "ping_other"
system.awaitTermination()
Console.println("Bye!")
}
class MyActor extends Actor with ActorLogging {
import akka.pattern.ask
implicit val timeout = Timeout(100.days)
def receive = {
case "ping_other" => {
val selection = context.actorSelection("../node_1")
log.info("Sending ping to node_1")
val result = Await.result(selection ? "ping", Duration.Inf) // <-- Blocks here forever!
log.info("Got result " + result)
}
case "ping" => {
log.info("Sending back pong!")
sender ! "pong"
}
}
}
If I change Duration.Inf to 5.seconds, then actor waits 5 seconds, tells that my future is Timeouted (by throwing TimeoutException), and then other actor finally replies with needed message. So, no async happens. Why? :-(
How should I properly implement that pattern? Thanks.
The official Akka documentation says that Await.result will cause the current thread to block and wait for the Actor to 'complete' the Future with it's reply.
It is strange that your code blocks there forever, do you have only one thread for all your application?
Anyway I guess a more "idiomatic" way to code it would be to use a callback on the future success.
def receive = {
case "ping_other" => {
val selection = context.actorSelection("../node_1")
log.info("Sending ping to node_1")
val future: Future[String] = ask(selection, "ping").mapTo[String]
future.onSuccess {
case result : String ⇒ log.info("Got result " + result)
}
}
...
Two reasons why that doesn't work.
First, "node_1" asks itself and the "ping" will not be processed because it is blocking in waiting for the ask.
Also, there is a shortcoming of actorSelection for relative paths ("../node_1"). It is processed with message passing, and since your actor is blocking it cannot process any other message. This has been improved in upcoming 2.3 version of Akka, but you should avoid blocking anyway.