I am new to scala and I have to make an async call ( Elastic indexing using elastic4s ) while iterating through a for each loop. What is the best way to do that in scala.
val data = List("1","2","3","4")
data.foreach(element=>{
asyncCall(element)
})
How to do some action after all the asyncCall is completed. I don'nt want to do anything with the response. Just print success if everything success and print fail if any of the call fails. asyncCall returns Future[T].
You should use Future.sequence. It changes List[Future[T]] to Future[List[T]].
So in your example:
import scala.concurrent.Future
import scala.concurrent.ExecutionContext.Implicits.global
import scala.util.{Failure, Success}
val f = Future.sequence(data.map(asyncCall)) // Future[List[String]]
//you can now do something when future finishes:
f.onComplete{
case Success(s) => println(s"Evethings fine: $s")
case Failure(e) => println(s"Something went wrong: $e")
}
Related
I am currently building a client-side WebSockets consumer using Akka-HTTP. Instead of trying to do the parsing in the Sink, I wanted to wrap the code in a function which emits the outcome from the Sink, and then use this output (from the function) later for further processing (more parsing...etc.).
I am currently able to print every message from the Sink; however, the return type of the function remains to be Unit. My objective is to Emit a String from the function, for each item that lands in the sink, and then use the returned string to do further parsing. I have the code I have so far (Note: it's mostly boiler plate).
import java.util.concurrent.atomic.AtomicInteger
import akka.Done
import akka.actor.ActorSystem
import akka.http.scaladsl.Http
import akka.http.scaladsl.model.StatusCodes
import akka.http.scaladsl.model.ws.{Message, TextMessage, WebSocketRequest, WebSocketUpgradeResponse}
import akka.http.scaladsl.settings.ClientConnectionSettings
import akka.stream.Materializer
import akka.stream.scaladsl.{Flow, Keep, Sink, Source}
import akka.util.ByteString
import scala.concurrent.ExecutionContext.Implicits.global
import scala.concurrent.Future
import scala.util.{Failure, Success, Try}
object client extends App {
def parseData(uri: String)(implicit system: ActorSystem, materializer: Materializer): Unit = {
val defaultSettings = ClientConnectionSettings(system)
val pingCounter = new AtomicInteger()
val customWebsocketSettings = defaultSettings.websocketSettings.withPeriodicKeepAliveData(
() => ByteString(s"debug-${pingCounter.incrementAndGet()}")
)
val customSettings = defaultSettings.withWebsocketSettings(customWebsocketSettings)
val outgoing = Source.maybe[Message]
val sink: Sink[Message, Future[Done]] = Sink.foreach[Message] {
case message: TextMessage.Strict => message.text // I Want to emit/stream this message as a String from the function (or get a handle on it from the outside)
case _ => println("Other")
}
val webSocketFlow: Flow[Message, Message, Future[WebSocketUpgradeResponse]] =
Http().webSocketClientFlow(WebSocketRequest(uri), settings = customSettings)
val (upgradeResponse, closed) =
outgoing
.viaMat(webSocketFlow)(Keep.right)
.toMat(sink)(Keep.both)
.run()
val connected = upgradeResponse.flatMap { upgrade =>
if (upgrade.response.status == StatusCodes.SwitchingProtocols) {
Future.successful(Done)
} else {
throw new RuntimeException(
s"Connection failed: ${upgrade.response.status}"
)
}
}
connected.onComplete {
case Success(value) => value
case Failure(exception) => throw exception
}
closed.onComplete { _ =>
println("Retrying...")
parseData(uri)
}
upgradeResponse.onComplete {
case Success(value) => println(value)
case Failure(exception) => throw exception
}
}
}
And in a seperate object, I would like to do the parsing, so something like:
import akka.actor.ActorSystem
import akka.stream.Materializer
import api.client.parseData
object Application extends App {
implicit val system: ActorSystem = ActorSystem()
implicit val materializer: Materializer = Materializer(system)
val uri = "ws://localhost:8080/foobar"
val res = parseData(uri) // I want to handle the function output here
// parse(res)
println(res)
Is there a way I can get a handle on the Sink from outside the function, or do I need to do any parsing in the Sink. I am mainly trying to not overcomplicate the Sink.
Update: I am also considering if adding another Flow element to the stream (which handles the parsing) is a better practice than getting values outside of the stream.
Adding a flow element seems to solve your problem while being totally idiomatic.
What you have to keep in mind is that the sinks semantic is meant to describe how to "terminate" the stream, so while it can describe very complex computations, it will always return a single value which is returned only once the stream ends.
Said differently, a sink does not return a value per stream element, it returns a value per whole stream.
I have an Rest API provided by akka-http. In some cases I need to get data from an external database (Apache HBase), and I would like the query to fail if the database takes too long to deliver the data.
One naïve way is to wrap the call inside a Future and then block it with an Await.result with the needed duration.
import scala.concurrent.duration._
import scala.concurrent.{Await, Future}
object AsyncTest1 extends App {
val future = Future {
getMyDataFromDB()
}
val myData = Await.result(future, 100.millis)
}
The seems to be inefficient as this implementation needs two threads. Is There an efficient way to do this ?
I have another use case where I want to send multiple queries in parallel and then aggregates the results, with the same delay limitation.
val future1 = Future {
getMyDataFromDB1()
}
val future2 = Future {
getMyDataFromDB2()
}
val foldedFuture = Future.fold(
Seq(future1, future2))(MyAggregatedData)(myAggregateFunction)
)
val myData = Await.result(foldedFuture, 100.millis)
Same question here, what is the most efficient way to implement this ?
Thanks for your help
One solution would be to use Akka's after function which will let you pass a duration, after which the future throws an exception or whatever you want.
Take a look here. It demonstrates how to implement this.
EDIT:
I guess I'll post the code here in case the link gets broken in future:
import scala.concurrent._
import scala.concurrent.duration._
import ExecutionContext.Implicits.global
import scala.util.{Failure, Success}
import akka.actor.ActorSystem
import akka.pattern.after
val system = ActorSystem("theSystem")
lazy val f = future { Thread.sleep(2000); true }
lazy val t = after(duration = 1 second, using = system.scheduler)(Future.failed(new TimeoutException("Future timed out!")))
val fWithTimeout = Future firstCompletedOf Seq(f, t)
fWithTimeout.onComplete {
case Success(x) => println(x)
case Failure(error) => println(error)
}
I expect that the following code will generate value 5 as I have written recover to handle the exception and return 5. But I always see the exception in IntelliJ. What is the mistake here?
import scala.concurrent._
import scala.concurrent.ExecutionContext.Implicits.global
import scala.util.{Failure, Success, Try}
import scala.concurrent.duration._
import scala.language.postfixOps
object ConcurrencyExample extends App {
val time = System.currentTimeMillis()
val futures1: Future[Int] = Future {
println("in Future.. sleeping")
Thread.sleep(10)
println("awakein future")
throw new Exception("future failed")
1
}
val result = futures1.map(x=>x+3)
result recover {
case e:Exception => 5
}
result onComplete{
case Success(v)=>println("result is "+v)
case Failure(e) =>println("failed result:"+e)
}
Await.result(result,100 millis)
}
result
in Future.. sleeping
awakein future
Exception in thread "main" LegacyException: future failed
It is not working because the future that would do the recovering is not the one you are calling Await.result() on.
From the Future's scaladoc, we learn that recover creates a new future that will handle any matching throwable that this future might contain. If there is no match, or if this future contains a valid result then the new future will contain the same.
Thus, it is this new future you are now interested in.
Try this:
val result = futures1.map(x=>x+3)
.recover {
case e:Exception => 5
}
Await.result(result,100 millis)
Given the following code:
import spray.http._
import spray.client.pipelining._
import scala.concurrent.Future
implicit val system = ActorSystem()
import system.dispatcher // execution context for futures
val pipeline: HttpRequest => Future[HttpResponse] = sendReceive
val response: Future[HttpResponse] = pipeline(Get("http://spray.io/"))
The following pseudo-code function waits 10 seconds, returning "GOOD" if the HttpResponse returned, or "BAD" on an Await#result exception (see docs.
import scala.concurrent.Await
import scala.concurrent.duration._
def f(fut: Future[HttpResponse]): String = {
try {
val result = Await.result(fut, 10.seconds)
"GOOD"
}
catch e # (_: InterruptedException | _: IllegalArgumentException
| _: TimeoutException ) => "BAD"
}
In my catch, is it only necessary to catch exception thrown by Await#result? In other words, am I not catching any possible exceptions here?
The Await.result itself can throw the exceptions you caught, however, if the future it awaits does not complete successfully, it forwards the exception contained by the future. You might want to read the Blocking section from here: Futures and Promises.
So yes, there may be exceptions you aren't catching, anything that can result from the failed computation of a HttpResponse.
Blocking in real code is usually bad and should be done only for testing purposes, but if you really need to, I would recommend to wrap the Await in a scala.util.Try so you could manipulate it elegantly later and also keep the information of when and why it failed.
I want to do something apparently simple: calling a webservice and saving the result in the database.
I'm inside an Akka Actor code and what I do is to call an object method:
object Service {
def run {
val response = WS.url("http://api.server.com/rest/")
.withAuth("test", "test", com.ning.http.client.Realm.AuthScheme.BASIC)
.get.value.get.get.body
}
}
How do I parse the body? I tried to print it on the console but I got NotSuchElement exception.
Any idea, thought? How do I parse arrays, attributes, elements of the XML ?
I'm in the play version 2.1.0
Things have changed a bit since the previous version. Play 2.1.0 depends on the scala.concurrent package instead of their own classes:
A Play Promise is now a Scala Future
A Play Redeemable is now a Scala Promise
I didn't have time to test it, but from the documentation I gathered it should be something like this:
import play.api.libs.ws.WS
import play.api.libs.concurrent.Execution.Implicits._
import scala.concurrent.Await
import scala.concurrent.duration._
import scala.language.postfixOps
object WebserviceCallParseXML {
val responseFuture = WS.url("http://api.server.com/rest/")
.withAuth("test", "test", com.ning.http.client.Realm.AuthScheme.BASIC)
.get()
val resultFuture = responseFuture map { response =>
response.status match {
case 200 => Some(response.xml)
case _ => None
}
}
val result = Await.result(resultFuture, 5 seconds)
println(if (result.isDefined) result.get else "No result found" )
}
The documentation about Future.value:
If the future is not completed the returned value will be None. If the future is completed the value will be Some(Success(t)) if it contains a valid result, or Some(Failure(error)) if it contains an exception.