I've a scala project with http4s 0.15.16a and slick 3.2.1 with these steps:
Receive a ID by rest call
passing ID to MySlickDAO that responds with a Future
Call Await.result(res, Duration.Inf) on Future returned by MySlickDAO
Create the json
The problem is that I use a Await.result and this is bad practices
is there a better solution ?
Here the code:
val service = HttpService {
//http://localhost:8080/rest/id/9008E75A-F112-396B-E050-A8C08D26075F
case GET -> Root / "rest" / "id" / id =>
val res = MySlickDAO.load(id)
Await.result(res, Duration.Inf)
val ll = res.value.get.get
ll match {
case Failure(x) =>
InternalServerError(x)
case Success(record) =>
val r = record.map(x => MyEntity(x._1, x._2, x._3))
jsonOK(r.asJson)
}
case ....
}
Instead of awaiting, you can chain the result of one Future into another:
val resFut = MySlickDAO.load(id)
resFut.map { record =>
val r = record.map(x => MyEntity(x._1, x._2, x._3))
jsonOK(r.asJson)
} recover { x =>
InternalServerError(x)
}
The result of this will be a Future of a common supertype of jsonOK and InternalServerError (not familiar with the libraries you're using; so I may have the type of load wrong: it's not a Future[Try[_]] is it?).
BTW: your original code has a very problematic line:
val ll = res.value.get.get
res.value is an Option[Try[T]]. Calling get on an Option or a Try is generally a bad idea (even though in this case because of the Await, the Option should never be None, so the get is technically safe) because it can throw an exception. You're much better off using map, flatMap, and friends.
The issue is that http4s 0.15 uses the Scalaz concurrency constructs, while Slick uses the native Scala ones, and the two aren't designed to work with each other. My understanding is that http4s 0.17+ has switched from Scalaz to Cats, which might entail using native Scala Futures, so if you can upgrade that might be worth a shot. If not, you can handle the conversion by manually creating a Task that wraps your future:
def scalaFutureRes = MySlickDAO.load(id)
val scalazTaskRes = Task.async { register =>
scalaFutureRes.onComplete {
case Success(success) => register(success.right)
case Failure(ex) => register(ex.left)
}
}
At this point you've got a Task[ResultType] from the Future[ResultType] which you can map/flatMap with the rest of your logic like in Levi's answer.
You can also use the delorean library for this which has this logic and the opposite direction defined on the classes in question via implicit conversions, so that you can just call .toTask on a Future to get it in a compatible form. Their readme also has a lot of useful information on the conversion and what pitfalls there are.
Related
I'm writing an HTTPS service for a chat bot and find myself dealing with a lot of Futures and Options. Usually if an Option returns None or a Future fails I want to log the exception and reset the user back to the start. Here's a toy example of how I accomplish this:
(for {
user <- userService.retrieve(userId)
userPet <- Future(user.userPet.get)
_ <- sendTextAsJson(s"You're holding a $userPet!")
} yield {}).recover {
case ex: Exception =>
log.error(ex.toString)
fail
}
This works fine but it feels a little weird to wrap things in Future just so their exceptions are swallowed and dealt with in the recover block. It also feels weird to include an empty yield block. Is there a better way?
What you basically do is using onSuccess or onFailure to retrieve the futures result. What you also might try is Try.
There is an example of the underlying functionality.
http://www.scala-lang.org/api/2.9.3/scala/util/Try.html
I might suggest you this article: http://docs.scala-lang.org/overviews/core/futures.html I can't summarize whats stated there in a few sentences. But if you look to the table of contents on the right side, the point Futures explains what happens and how to handle it is stated under Excepetions. Thats the idiomatic way.
I don't think it's too bad tbh, assuming userService.retrieve() returns a Future in the first place. I'd personnally rather use map in this case to make things a bit more explicit :
val futureMsg = userService.retrieve(userId)
.map(user => sendTextAsJson(s"You're holding a ${user.userPet.get}!")
.recover {
case NonFatal(ex) => //Look it up ;)
log.error(ex.toString)
fail
}
You now have Future[Unit] to do whatever you want with.
I agree with you that that's an awkward use of a for-comprehension. This is how I would write this:
import scala.util.control.NonFatal
userService.retrieve(userId)
.map(_.userPet.get)
.map(userPet => s"You're holding a $userPet!")
.flatMap(sendTextAsJson)
.recover {
case NonFatal(ex) =>
log.error(ex.toString)
fail
}
Looking at your sendTextAsJson and fail functions you seem to want to side-effect when the future completes. I would not use map after you perform the Option.get and instead look at Futures onComplete method to either handle the success of the future or handle the exception throw. I am not sure how this way and recover are different though so double check that. I did like #sascha10000 link to the scala doc futures. Its been a long time since I read that but its a good resource from what I remember
Example implementation of your code with onComplete:
import scala.concurrent.Future
import scala.util.{Failure, Success}
import scala.concurrent.ExecutionContext.Implicits.global
object UserFuture extends App {
def fail = println("failed")
def sendTextAsJson(message: String): Unit = println(message)
case class User(userPet: Option[String])
object userService {
def userPet = Some("dog")
val someUser = User(userPet)
def retrieve(userId: Int): Future[User] = Future {
someUser
}
}
def getPet(userId: Int): Unit = {
userService.retrieve(userId)
.map(user => user.userPet.get)
.onComplete {
case Success(userPet) => sendTextAsJson(s"You're holding a $userPet!")
case Failure(ex) => println(ex.toString); fail
}
}
getPet(1)
Thread.sleep(10000) // I forgot how to use Await. This is just here to be able to make sure we see some printouts in the console.
}
In scala concurrent package, there is onComplete method which returns nothing while in akka-http directives, it returns the Directive1.
Is that the only difference? I sometimes got confused on those two methods.
These methods are different in what they aim to accomplish, but they are related in that the akka http onComplete is meant to work with the completion of a Future in order to complete the routing of a request.
The concurrent onComplete is a method on a Future that is meant to perform some sort of side effecting when the Future has completed. There are a lot of other methods on a Future like map, flatMap and filter that will produce another Future as you go through a series of transforms, but onComplete returns nothing, and is sort of a "dead end" in the processing of that Future.
An example of using onComplete while also showing how something like map can be used to change the Future is as follows:
val fut1 = Future{"1"} // A Future[String]
val fut2 = fut1.map(_.toInt) // Now a Future[Int]
fut2.onComplete{
case util.Success(i) => println(s"success: $i")
case util.Failure(ex) => ex.printStackTrace()
}
val badFut1 = Future{"A"}
val badFut2 = badFut.map(_.toInt) //Will be failed
badFut2.onComplete{
case util.Success(i) => println(s"success: $i")
case util.Failure(ex) => ex.printStackTrace()
}
In the first case, we will hit the success case as "1" can be mapped into an Int. In the second example, we will hit the error case as "A" cannot be mapped to an Int. Either way, you can see that something like map can be used to create a new transformed future while onComplete can be used to do something based on the final result of that Future.
In Akka Http, you sometimes need to create Route definitions that need to do something asynchronously. In that case, you need to complete the route when that asynchronous task completed. A Route itself needs to be fully defined in order to compile, so you can't leverage the existing onComplete on a Future because that returns nothing and that would render the definition of that Route as incomplete. Therefore, the guys at Akka defined an onComplete directive that will snap into a routing tree and work with a Future, returning a Directive that will fully define the tree for that Route as oppose to leaving it open ended as the Future.onComplete would. An example:
val myRoute =
get{
val fut = myActor ? "foo"
onComplete(foo){
case util.Success(_) => complete(StatusCodes.OK)
case util.Failure(ex) => complete(StatusCodes.InternalServerError)
}
}
So here, I'm making a request to an Actor using ask (?) and I get a Future back. I still need to produce something that completes the Route I am defining. If I did something like this:
val myRoute =
get{
val fut = myActor ? "foo"
fut.onComplete{
case util.Success(_) => complete(StatusCodes.OK)
case util.Failure(ex) => complete(StatusCodes.InternalServerError)
}
}
It worn't work because the onComplete here is the one from the Future and not the directive, and that returns Unit and my Route won't be full defined and won't compile.
So the onComplete directive is there to allow you to define a Route based on the completion of a Future which you can't do properly with the pre-existing onComplete from Future itself.
I still struggle sometimes to map Futures using Play with Scala...
I am trying to pass to a view my entire Supply DB table (i.e. all the supplies in the DB).
I have tried two distinct ways but both have failed...
Below are the methods I've tried and the errors I get.
Can someone please help me solve this, and also explain me why both of these have failed?
Thank you in advance!
Note: Calling supplyService.all returns a Future[Seq[Supply]].
First attempt
def index = SecuredAction.async { implicit request =>
supplyService.all map { supplies =>
Future.successful(Ok(views.html.supplies.index(request.identity, SupplyForm.form, supplies)))
}
}
Second attempt
def index = SecuredAction.async { implicit request =>
val supplies = supplyService.all
Future.successful(Ok(views.html.supplies.index(request.identity, SupplyForm.form, supplies)))
}
First variant without Future.succesfull
supplyService.all.map( supplies => Ok(views.html.supplies.index(request.identity, SupplyForm.form, supplies)) )
Since you have can construct function Seq[Supply] => Result you can easily map
Future[Seq[Supply]] to Future[Result] via functor interface.
Future is also a monad , so you can use Seq[Supply] => Future[Result] with flatMap method.
But Future.successfull is monad unit , and as for many monads for Future its true that
mx.flatMap(f andThen unit) = mx.map(f)
so your
ms.flatMap(supplies => Future.successfull(f(supplies)) =
ms.flatMap(f andThen Future.successfull) =
ms.map(f)
I'm new to Scala, being slightly confused at the various ways to handle exceptions and looking for best-practice advice on the topic. I'm writing a simple method to retrieve a Customer using an existing blocking SDK. The possible outcomes are:
Customer is found
Customer is not found (comes back as a NotFoundException from the SDK)
Error while talking to remote server (SDK throws some other exception)
So I want my method to have a return type Future[Option[Customer]], and return for each case above:
Successful Future: Some(customer)
Successful Future: None
Failed Future
Here is what I wrote using try/catch:
private def findCustomer(userId: Long): Future[Option[Customer]] = future {
try {
Some(gateway.find(userId))
} catch {
case _: NotFoundException => None
}
}
This works fine and seems clean to me, but doesn't seem to be really the "Scala way" to do it - I've been told to avoid using try/catch. So I've been looking for a way to rewrite it using a Try instead.
Here are 2 variants that (I think) behave exactly the same, using a Try.
Variant A:
private def findCustomer(userId: Long): Future[Option[Customer]] = future {
Try(
Some(gateway.find(userId))
).recover {
case _: NotFoundException => None
}.get
}
Variant B:
private def findCustomer(userId: Long): Future[Option[Customer]] = future {
Try(
Some(gateway.find(userId))
).recover {
case _: NotFoundException => None
}
} flatMap {
case Success(s) => Future.successful(s)
case Failure(f) => Future.failed(f)
}
I'm not a huge fan of A (even though it's more concise than B) since the .get seems a bit treacherous. B is definitely the most explicit but mapping the Try cases to corresponding Future outcomes seems boring.
How would an experienced Scala programmer write this?
I think your initial version using try/catch is perfectly fine, because it's a wrapper around an existing SDK.
Alternatively, you can use the recover method on Future:
def findCustomer(userId: Long): Future[Option[Customer]] =
Future(Option(gateway.find(userId))).recover {
case e: NotFoundException => None
}
One option is to avoid chaining Try and Future. In a sense, a future is an asynchronous Try.
You could use Future[Customer] directly and consider NotFoundException as something to recover from rather than a None value. Normally, you would chain operation on the future itself, with no need to deal with the failure cases (by mapping it, etc). It all depends on how you are going to use it (how your logic branches when the future is completed). In other words, maybe it seems convoluted to you because it is, and you are forcing it to be by forcing the Future[Option[Customer]] return type.
The ability to do chain multiple operations and carry on with computing if and only if all if going all right is the nice Scala feature here (especially on for comprehensions, etc).
You might be looking for:
Future.fromTry(myTry)
So for your example you might do:
Future.fromTry {
Try(Some(gateway.find(userId))).recover {
case _: NotFoundException => None
}
}
To only catch the NotFoundException, as in your solution. The only problem here is that the method won't be executed asynchronously. If that is necessary, consider using Future.recover, as suggested by Ionut.
Another idiomatic option worth mentioning here is to use Either[F, S], where S is the type of the successful return, and F can hold the errors (which you might want to propagate). So you could use Either[ Exception, Option[String]] or Either[String, Option[String]] where the first String is an error message.
If you're ready to adopt some scalaz. disjunction from scalaz is very useful and natural in handling erronous scenarios. It's like scala Either but scalaz disjunction \/ is right biased. you'll have success value on the right and exception on left. wrapping your code block with \/.fromTryCatch returns exception on the left side of disjunction. right will always have success value. mapping over disjunction is easiser than Scala Either since disjunction is right biased and easily gives you value from right.
import scalaz.\/
private def findCustomer(userId: Long): Future[\/[Throwable,Option[Customer]] ]= future {
\/.fromTryCatch {
Some(gateway.find(userId))
}
}
Given rowParser of type RowParser[Photo], this is how you would parse a list of rows coming from a table photo, according to the code samples I have seen so far:
def getPhotos(album: Album): List[Photo] = DB.withConnection { implicit c =>
SQL("select * from photo where album = {album}").on(
'album -> album.id
).as(rowParser *)
}
Where the * operator creates a parser of type ResultSetParser[List[Photo]]. Now, I was wondering if it was equally possible to get a parser that yields a Stream (thinking that being more lazy is always better), but I only came up with this:
def getPhotos(album: Album): Stream[Photo] = DB.withConnection { implicit c =>
SQL("select * from photo where album = {album}").on(
'album -> album.id
)() collect (rowParser(_) match { case Success(photo) => photo })
}
It works, but it seems overly complicated. I could of course just call toStream on the List I get from the first function, but my goal was to only apply rowParser on rows that are actually read. Is there an easier way to achieve this?
EDIT: I know that limit should be used in the query, if the number of rows of interest is known beforehand. I am also aware that, in many cases, you are going to use the whole result anyway, so being lazy will not improve performance. But there might be a case where you save a few cycles, e.g. if for some reason, you have search criteria that you cannot or do not want to express in SQL. So I thought it was odd that, given the fact that anorm provides a way to obtain a Stream of SqlRow, I didn't find a straightforward way to apply a RowParser on that.
I ended up creating my own stream method which corresponds to the list method:
def stream[A](p: RowParser[A]) = new ResultSetParser[Stream[A]] {
def apply(rows: SqlParser.ResultSet): SqlResult[Stream[A]] = rows.headOption.map(p(_)) match {
case None => Success(Stream.empty[A])
case Some(Success(a)) => {
val s: Stream[A] = a #:: rows.tail.flatMap(r => p(r) match {
case Success(r) => Some(r)
case _ => None
})
Success(s)
}
case Some(Error(msg)) => Error(msg)
}
}
Note that the Play SqlResult can only be either Success/Error while each row can also be Success/Error. I handle this for the first row only, assuming the rest will be the same. This may or may not work for you.
You're better off making smaller (paged) queries using limit and offset.
Anorm would need some modification if you're going to keep your (large) result around in memory and stream it from there. Then the other concern would be the new memory requirements for your JVM. And how would you deal with caching on the service level? See, previously you could easily cache something like photos?page=1&size=10, but now you just have photos, and the caching technology would have no idea what to do with the stream.
Even worse, and possibly on a JDBC-level, wrapping Stream around limited and offset-ed execute statements and just making multiple calls to the database behind the scenes, but this sounds like it would need a fair bit of work to port the Stream code that Scala generates to Java land (to work with Groovy, jRuby, etc), then get it on the approved for the JDBC 5 or 6 roadmap. This idea will probably be shunned as being too complicated, which it is.
You could wrap Stream around your entire DAO (where the limit and offset trickery would happen), but this almost sounds like more trouble than it's worth :-)
I ran into a similar situation but ran into a Call Stack Overflow exception when the built-in anorm function to convert to Streams attempted to parse the result set.
In order to get around this I elected to abandon the anorm ResultSetParser paradigm, and fall back to the java.sql.ResultSet object.
I wanted to use anorm's internal classes for the parsing result set rows, but, ever since version 2.4, they have made all of the pertinent classes and methods private to their package, and have deprecated several other methods that would have been more straight-forward to use.
I used a combination of Promises and Futures to work around the ManagedResource that anorm now returns. I avoided all deprecated functions.
import anorm._
import java.sql.ResultSet
import scala.concurrent._
def SqlStream[T](sql:SqlQuery)(parse:ResultSet => T)(implicit ec:ExecutionContext):Future[Stream[T]] = {
val conn = db.getConnection()
val mr = sql.preparedStatement(conn, false)
val p = Promise[Unit]()
val p2 = Promise[ResultSet]()
Future {
mr.map({ stmt =>
p2.success(stmt.executeQuery)
Await.ready(p.future, duration.Duration.Inf)
}).acquireAndGet(identity).andThen { case _ => conn.close() }
}
def _stream(rs:ResultSet):Stream[T] = {
if (rs.next()) parse(rs) #:: _stream(rs)
else {
p.success(())
Stream.empty
}
}
p2.future.map { rs =>
rs.beforeFirst()
_stream(rs)
}
}
A rather trivial usage of this function would be something like this:
def getText(implicit ec:ExecutionContext):Future[Stream[String]] = {
SqlStream(SQL("select FIELD from TABLE")) { rs => rs.getString("FIELD") }
}
There are, of course, drawbacks to this approach, however, this got around my problem and did not require inclusion of any other libraries.