I want to have some action completed ONCE, when two promises are success. So I created such a piece of code:
val promiseForPersistence = Promise[Unit]()
val promiseForReplication = Promise[Unit]()
promiseForPersistence.future.onComplete {
case scala.util.Success(_) => {
if (promiseForReplication.isCompleted) {
println(s"Acking from persistence promise (replication one already completed)" )
}
}
case _ =>
}
promiseForReplication.future.onComplete {
case scala.util.Success(_) => {
if (promiseForPersistence.isCompleted) {
println(s"Acking from replication promise (persistence one already completed)" )
}
}
case _ =>
}
promiseForPersistence.success()
promiseForReplication.success()
The problem is, both log messages are printed. I want the action to be fired only once. Why it is fired twice, and how can I have it fire only once?
You should not use isCompleted method on Future for this purpose. You should combine 2 futures in a single future using flatMap like this:
for {
_ <- promiseForPersistence.future
_ <- promiseForReplication.future
} println("Both are completed" )
Callback in onComplete could be evaluated after both Promises are completed. So you'll get isCompleted == true in both callbacks and 2 println messages.
Valid way is to add callback in callback:
promiseForPersistence.future.onSuccess { _ =>
promiseForReplication.future.onSuccess { _ =>
println(s"Acking from replication promise (persistence one already completed)" )
}
}
You'll get exactly the same behavior with flatMap (for-comprehension), but with less verbose syntax.
Related
Because I do some "complex" operations, I think my Actor became asyncronous. The main problem I think is that I use Await.result inside the method which return responses.
actor:
def process(subscribers: Set[ActorRef]): Receive = {
case Join(ref) => context become process(subscribers + ref)
case Leave(ref) => context become process(subscribers - ref)
case Push(request) =>
val filteredSubscribers = (subscribers - sender())
.filter(s => exists(s, request)) // just some actor filters
filteredSubscribers.foreach { subscriber =>
// here I have a Map with each actor requests
val actorOptions = getActorOptions(subscriber)
subscriber ? getResponse(actorOptions, request)
}
}
The problem is inside getResponse (I think).
getResponse(actorOptions: JsValue, request: SocketRequest): JsValue = {
(actorOptions \ "dashboardId").asOpt[Int] match {
case Some(id) => {
val response = widgetsService.getByDashboadId(id) map { widgets =>
val widgetsResponse: List[Future[String]] = widgets.map(w => {
widgetsService.getDataById(w.id) map {
data => s"""{ "widgetId": ${w.id}, "data": $data }"""
}
})
var responses: List[String] = List.empty
widgetsResponse.foreach(f => {
f.onComplete {
case Success(value) => responses = value :: responses
case Failure(e) => println(s"Something happened: ${e.getMessage}")
}
})
// first time when I use Await.result
// used to populate the responses list with data from all futures
Await.result(Future.sequence(widgetsResponse), Duration.Inf)
Json.parse(s"""{
"dashboardId": $id,
"widgets": [${response.mkString(", ")}]
}""".stripMargin)
}
// second time when I use Await.result
// used to return a JsValue instead of a Future[JsValue]
Await.result(response, Duration.Inf)
}
case None => buildDefaultJson // return default json value, unimportant for this example
}
}
Due of that, In frontend, if I have 2 sockets clients, the response for the second will be send only after first.
I found that I can obtain a "fake" increase of performance if I embrance the getResponse in a future inside of my Actor.
filteredSubscribers.foreach { subscriber =>
val actorOptions = getActorOptions(subscriber)
Future(subscriber ? getResponse(actorOptions, request))
}
So, for both subscribers the action will be started in same time, but when the first will reach the Await.result, the second will be locked until first is done.
I need to avoid using Await.result there, but I don't know how to get the results of a list of futures, without using for-comprehension (because is a dynamically list) for first time where I use it.
Because Akka ask operator (?) return a Future[Any], I tried that my getResponse method to return directly a JsValue to be mapped then in Future[JsValue]. If I remove the second Await.result and my method will return Future[JsValue], then the actor will return a Future[Future[JsValue]] which I don't think is too right.
After some more researches and solutions found on so, my code become:
Future.sequence(widgetsResponse) map { responses =>
Json.parse(
s"""
|{
|"dashboardId": $id,
|"tableSourceId": $tableSourceId,
|"widgets": [ ${responses.mkString(", ")}]
|}""".stripMargin
)
}
getResponse returns a Future[JsValue] now, removing both Await.result cases, and actor case become:
filteredSubscribers.foreach { subscriber =>
val actorOptions = getActorOptions(subscriber)
getResponse(actorOptions, request) map { data =>
subscriber ? data
}
}
I don't know why, still have a synchronous behavior. Damn, this can be due of my subscribers type: Set[ActorRef]? I tried to use parallel foreach and this looks like solving my problem:
filteredSubscribers.par.foreach { subscriber =>
val actorOptions = getActorOptions(subscriber)
getResponse(actorOptions, request) map { data =>
subscriber ? data
}
}
I need to execute a Future method on some elements I have in a list simultaneously. My current implementation works sequentially, which is not optimal for saving time. I did this by mapping my list and calling the method on each element and processing the data this way.
My manager shared a link with me showing how to execute Futures simultaneously using for-comprehension but I cannot see/understand how I can implement this with my List.
The link he shared with me is https://alvinalexander.com/scala/how-use-multiple-scala-futures-in-for-comprehension-loop/
Here is my current code:
private def method1(id: String): Tuple2[Boolean, List[MyObject]] = {
val workers = List.concat(idleWorkers, activeWorkers.keys.toList)
var ready = true;
val workerStatus = workers.map{ worker =>
val option = Await.result(method2(worker), 1 seconds)
var status = if (option.isDefined) {
if (option.get._2 == id) {
option.get._1.toString
} else {
"INVALID"
}
} else "FAILED"
val status = s"$worker: $status"
if (option.get._1) {
ready = false
}
MyObject(worker.toString, status)
}.toList.filterNot(s => s. status.contains("INVALID"))
(ready, workerStatus)
}
private def method2(worker: ActorRef): Future[Option[(Boolean, String)]] = Future{
implicit val timeout: Timeout = 1 seconds;
Try(Await.result(worker ? GetStatus, 1 seconds)) match {
case Success(extractedVal) => extractedVal match {
case res: (Boolean, String) => Some(res)
case _ => None
}
case Failure(_) => { None }
case _ => { None }
}
}
If someone could suggest how to implement for-comprehension in this scenario, I would be grateful. Thanks
For method2 there is no need for the Future/Await mix. Just map the Future:
def method2(worker: ActorRef): Future[Option[(Boolean, String)]] =
(worker ? GetStatus).map{
case res: (Boolean, String) => Some(res)
case _ => None
}
For method1 you likewise need to map the result of method2 and do the processing inside the map. This will make workerStatus a List[Future[MyObject]] and means that everything runs in parallel.
Then use Future.sequence(workerStatus) to turn the List[Future[MyObject]] into a Future[List[MyObject]]. You can then use map again to do the filtering/ checking on that List[MyObject]. This will happen when all the individual Futures have completed.
Ideally you would then return a Future from method1 to keep everything asynchronous. You could, if absolutely necessary, use Await.result at this point which would wait for all the asynchronous operations to complete (or fail).
In my backend controller function I have the following code:
def getContentComponents = Action.async {
contentComponentDTO.list().map { contentComponentsFuture =>
contentComponentsFuture.foreach(contentComponentFuture =>
contentComponentFuture.typeOf match {
case 5 => contentComponentDTO.getContentComponentText(contentComponentFuture.id.get).map(text => {
contentComponentFuture.text = text.text
println(contentComponentFuture.text)
})
}
)
Ok(Json.toJson(contentComponentsFuture))
}
}
The problem is, that OK() is called before the stuff above is finished. Is there a smart way to wait until the foreach is finished?
thanks
That have been two differen questions with different problems! So thats the reason for two questions looking similar
There are two approaches that you can take, but before going to that let's recap what you are trying to do.
You have a list of items contentComponentsFuture, I am assuming retrieving from database, which is why you are getting a future.
Now your contentComponentsFuture is a mutable variable (which I strongly suggest not to use, stick to immutable data) has a text field which you need to update.
Now all that code block before the Ok() will return a future as it is working on a future list. So the easiest solution is to do a map over the future and return the result. The map over a future is just a onComplete function which triggers once the future is resolved. So the code will look like:
def getContentComponents = Action.async {
val futureResult = contentComponentDTO.list().map { contentComponentsFuture =>
contentComponentsFuture.map(contentComponentFuture =>
contentComponentFuture.typeOf match {
case 5 => contentComponentDTO.getContentComponentText(contentComponentFuture.id.get).map(text => {
contentComponentFuture.text = text.text
contentComponentFuture
})
}
)
}
futureResult.map(result => {
Future.sequence(result).map(t => Ok(Json.toJson(t))
}))
}
The other option will be to use scala asycn library: https://github.com/scala/scala-async
which gives a handy wrapper so that you do not need to map over future explicitly, the same code above with scala async library will look like below:
def getContentComponents = Action.async {
Async.async {
val result = Async.await(contentComponentDTO.list().map { contentComponentsFuture =>
contentComponentsFuture.map(contentComponentFuture =>
contentComponentFuture.typeOf match {
case 5 => contentComponentDTO.getContentComponentText(contentComponentFuture.id.get).map(text => {
contentComponentFuture.text = text.text
contentComponentFuture
})
}
)
})
Ok(Json.toJson(result))
}
}
I am wondering how can I immediately terminate the for comprehension without calling Future.failed and yield the Try[Failure] result instead.
Example:
for {
resultOne <- Future { Successful("Data") }
resultTwo <- Future {
// Something went wrong here...
// Instead of calling Future.failed();
// I want to terminate this for-comprehension and return a
// Failure() object and bypass the next line
}
resultThree <- Future {
// Something went wrong here...
// Instead of calling Future.failed();
// I want to terminate this for-comprehension and return a
// Failure() object and bypass the next line
}
resultFour <- Future {
// ... Some database call which retuns a Try[]
}
} yield resultFour
Another example
You're right in saying that my example is not running in parallel. I guess I don't have other options but to make it like this?
for {
resultOne <- Future { Successful("Data") }
resultFour <- {
// ... Some async function which returns a Try
}
.flatMap {
case Success(resultTwo) => // ... Some async function which returns a Try
case Failure(ex) => Future.success(Failure(ex))
}
.flatMap {
case Success(resultTwo) => // ... Some database call which retuns a Try[]
case Failure(ex) => Future.success(Failure(ex))
}
}
} yield resultFour
What you have there has this fail-fast behaviour but is not running in parallel! You need to create the Futures outside the comprehension in order to run them in parallel.
There is no comprehension behind the scenes, it is syntactic sugar over map/flatMap. So it is not a "thing" you can cancel as a whole.
I assume what you want is to stop other futures if one fails (e.g. to save resources or return as soon as anything fails). Scala's Futures don't support this. Monix is a scala library that provides CanceleableFuture that supports exactly this use case.
I have an Akka actor responsible of handling http calls. I use scala dispatch to send multiple HTTP requests over an API:
urls.foreach { u
val service = url(u)
val promise = Http(service OK as.String).either
for(p <- promise)
{
p match
{
case Left(error) =>
faultHandler(error)
case Right(result) =>
resultHandler(result)
}
}
In the resultHandlerfunction, I increment an instance variable nbOfResults and compare to the number of calls I have done.
def resultHandler(result:String)
{
this.nbOfResults++
...
if(nbOfResults == nbOfCalls)
// Do something
}
Is it safe ? May the nbOfResultsvaraible be accessed at the same time if two calls return their results simultaneously ?
For now, I believed that the actor is more or less equivalent to a thread and therefore the callback functions are not executed concurrently. Is it correct ?
Here is a variant of Alexey Romanov response using only dispatch :
//Promises will be of type Array[Promise[Either[Throwable, String]]]
val promises = urls.map { u =>
val service = url(u)
Http(service OK as.String).either
}
//Http.promise.all transform an Iterable[Promise[A]] into Promise[Iterable[A]]
//So listPromise is now of type Promise[Array[Either[Throwable, String]]]
val listPromise = Http.promise.all(promises)
for (results <- listPromise) {
//Here results is of type Array[Either[Throwable, String]]
results foreach { result =>
result match {
Left(error) => //Handle error
Right(response) => //Handle response
}
}
}
There is a far better way:
val promises = urls.map {u =>
val service = url(u)
val promise = Http(service OK as.String).either
}
val listPromise = Future.sequence(promises)
listPromise.onComplete { whatever }
I agree with Alexey Romanov on his answer. Whatever way you choose to synchronize your http requests beware of the way your are processing the promises completion. Your intuition is correct in that concurrent access may appear on the state of the actor. The better way to handle this would be to do something like this:
def resultHandler(result: String) {
//on completion we are sending the result to the actor who triggered the call
//as a message
self ! HttpComplete(result)
}
and in the actor's receive function:
def receive = {
//PROCESS OTHER MESSAGES HERE
case HttpComplete(result) => //do something with the result
}
This way, you make sure that processing the http results won't violate the actor's state from the exterior, but from the actor's receive loop which is the proper way to do it
val nbOfResults = new java.util.concurrent.atomic.AtomicInteger(nbOfCalls)
// After particular call was ended
if (nbOfResults.decrementAndGet <= 0) {
// Do something
}
[EDIT] Removed old answer with AtomicReference CAS - while(true), compareAndSet, etc