this thread gave me an idea how to structure my code: Scala-way to handle conditions in for-comprehensions?
The part in question:
// First create the JSON
val resultFuture: Future[Either[Failure, JsResult]] = for {
userRes <- userDao.findUser(userId)
user <- userRes.withFailure(UserNotFound).right
authRes <- userDao.authenticate(user)
auth <- authRes.withFailure(NotAuthenticated).right
goodRes <- goodDao.findGood(goodId)
good <- goodRes.withFailure(GoodNotFound).right
checkedGood <- checkGood(user, good).right
} yield renderJson(Map("success" -> true)))
This are the lines I do not understand:
user <- userRes.withFailure(UserNotFound).right
authRes <- userDao.authenticate(user)
The userRes.withFailure(UserNotFound).right is mapped to userDao.authenticate(user). This will create a new Either with a Future on its right, correct?
How can
val resultFuture: Future[Either[Failure, JsResult]]
be of its type. I think instead of a JsResult there should be another future.
Can anyone explain this to me?
EDIT: Since cmbaxter and Arne Claassen confirmed this, the new question is: How should I write this code, so it does not look ugly, but clean and structured?
I believe the answer you received needlessly mixed Either's into the mix when Future's are already perfectly capable of communicating failure. The main thing you were missing was a way to get from an Option to the option's value without explicitly throwing exceptions.
I would suggest that you change the Failures object to the following:
object Failures {
sealed trait Failure extends Exception
// Four types of possible failures here
case object UserNotFound extends Failure
case object NotAuthenticated extends Failure
case object GoodNotFound extends Failure
case object NoOwnership extends Failure
// Put other errors here...
// Converts options into Futures
implicit class opt2future[A](opt: Option[A]) {
def withFailure(f: Failure) = opt match {
case None => Future.failed(f)
case Some(x) => Future.successful(x)
}
}
}
Now you can map a Future[Option[A]] to a Future[A] and specify the failure condition, resulting in a for comprehension like this:
def checkGood(user: User, good: Good) =
if (checkOwnership(user, good))
Future.successful(good)
else
Future.failed(NoOwnership)
val resultFuture: Future[JsResult] = for {
userOpt <- userDao.findUser(userId)
user <- userOpt.withFailure(UserNotFound)
authOpt <- userDao.authenticate(user)
auth <- authOpt.withFailure(NotAuthenticated)
goodOpt <- goodRes.withFailure(GoodNotFound)
checkedGood <- checkGood(user, good)
} yield renderJson(Map("success" -> true))))
Now that you have a Future[JsResult] you can map the failed scenarios to your desired output and the success scenario is just the JsResult. Hopefully you are using this in an asynchronous framework which expects you to feed it a future and has its own failed future to error response mapping (such as Play!).
Related
Currently my error handling is not working as I want to, this is what I am trying to do:
UserApi.insert fails, return its error and don't continue
WorkApi.insert fails, return its error after calling UserApi.delete
WorkApi.assign fails, return its error after calling WorkApi.delete and UserApi.delete
So in summary, UserApi.insert is called, if it is successfull, continue to #2. If WorkApi.insert is successfull, continue. And so on, if the current step fails, you have to reverse the previous one.
Also it is important to return the most relevant error for the Api call that failed.
If all calls were successful, I want to return the first calls value which is a User.
import scala.concurrent.Future
import scala.concurrent.ExecutionContext.Implicits.global
import scala.util.{Success, Failure}
val u1 = User("u1")
val w1 = Work("w1")
val resp = for {
insertResp <- UserApi.insert(u1)
workInsertResp <- WorkApi.insert(w1)
workAssignResp <- WorkApi.assign(w1)
} yield insertResp
println("ending...")
resp onComplete {
case Success(r) => println(r)
case Failure(t) => println(t)
}
case class User(name: String)
case class Work(name: String)
case class MyError(name: String)
object UserApi {
def insert(user: User): Future[Either[MyError, User]] =
if (user.name == "u1") Future(Right(user))
else Future(Left(MyError("UserApi.insert")))
def delete(user: User): Future[Either[MyError, String]] =
Future(Right("UserApi.delete"))
}
object WorkApi {
def insert(work: Work): Future[Either[MyError, Work]] =
if (work.name == "w1") Future(Right(work))
else Future(Left(MyError("WorkApi.insert")))
def delete(work: Work): Future[Either[MyError, Work]] = Future(Right(work))
def assign(work: Work): Future[Either[MyError, Work]] =
if (work.name == "w1") Future(Right(work))
else Future(Left(MyError("WorkApi.assign")))
}
Currently I am not sure how to bubble the correct error up.
Note: I am using scala 2.13.x, and I am not using other frameworks just plain Scala.
https://scastie.scala-lang.org/OV4Ax58qQ1S3R3fFUikSbw
I believe this does what you've described.
import scala.concurrent.Future
import scala.concurrent.ExecutionContext.Implicits.global
val resp: Future[Either[MyError,User]] =
UserApi.insert(u1).flatMap{_.fold(
{err => Future.successful(Left(err))}
,usr => WorkApi.insert(w1).flatMap{_.fold(
{err => UserApi.delete(u1); Future.successful(Left(err))}
, _ => WorkApi.assign(w1).map{_.fold(
{err => WorkApi.delete(w1); UserApi.delete(u1); Left(err)}
, _ => Right(usr)
)}
)}
)}
. . . //and the rest of your code
testing:
import scala.concurrent.duration.DurationInt
concurrent.Await.result(resp, 9999.millis)
//res0: Either[MyError,User] = Right(User(u1))
As you can see, your current code design is not well suited for the task you've laid out.
First of all, I would recommend against mixing Future and Either this way. Future has its own way of representing failures, and wrapping an Either in a Future means you will need to handle both the failed Future case and the Left case of Either, which can lead to some confusing code.
In the code provided in the question, there's no asynchronous execution, so using Future is redundant, and you could use Either types directly. However, I assume you want to replace these methods with ones that make actual (asynchronous) API calls, in which case you'll want to use Future without Either. Future requires that failure values extend Throwable, so this would require a change to MyError:
case class MyError(name: String) extends Exception(name)
Second, it's not a good practice to use Future.apply for non-blocking construction as in Future(Right(user)) or Future(Left(MyError("UserApi.insert"))). It isn't obvious, but this actually causes Right(user) to be scheduled as a task on the implicit execution context, rather than being computed synchronously on the current thread. It's better to use Future.successful or Future.failed to create a completed Future when the result is trivial.
With these changes, the new method implementations are:
object UserApi {
def insert(user: User): Future[User] =
if (user.name == "u1") Future.successful(user)
else Future.failed(MyError("UserApi.insert"))
def delete(user: User): Future[String] =
Future.successful("UserApi.delete")
}
object WorkApi {
def insert(work: Work): Future[Work] =
if (work.name == "w1") Future.successful(work)
else Future.failed(MyError("WorkApi.insert"))
def delete(work: Work): Future[Work] =
Future.successful(work)
def assign(work: Work): Future[Work] =
if (work.name == "w1") Future.successful(work)
else Future.failed(MyError("WorkApi.assign"))
}
I have the following Slick code that given an id returns a customer (if exists). If there's a problem (such as connectivity lost) a Failure clause will throw an exception:
def read (id: Int): Future[Option[Customer]] = {
val db = // ....
val customers = TableQuery[CustomerDB]
val action = customers.filter(_.id === id).result
val future = db.run(action.asTry)
future.map{
case Success(s) =>
if (s.length>0)
Some(s(0))
else
None
case Failure(f) => throw new Exception (f.getMessage)
}
}
Now, my understanding is that instead of using try/catch/finally of exceptions, in Scala one should use Try. In addition, no exceptions should be thrown. But if the exception is not thrown, how to notify the upper layer that a problem occurred?
Future itself does already have Try inside. So, I would say that you need to just flatten (also you code a bit complicated, I simplified):
future.flatMap {
case Success(s) => Future.successful(s.headOption)
case Failure(f) => Future.failed(f)
}
Result Future when in failed state notifies caller that execution failed (with wrapped original exception). Otherwise, successful.
The right way to do report errors is by using Either.
trait Error
case class NotFound(id: Int) extends Error
case class QueryFailed(msg: String) extends Error
def read (id: Int): Future[Either[Error, Customer]] = {
val db = // ....
val customers = TableQuery[CustomerDB]
val action = customers.filter(_.id === id).result
val future = db.run(action.asTry)
future.map{
case Success(s) =>
if (s.length>0)
Right(s(0))
else
Left(NotFound(id))
case Failure(f) => Left(QueryFailed(f.getMessage))
}
}
Ok so, in general you can use Future.successful or Future.failed(msg: String) to "signal" the upper level (aka calling method) you got the value or not.
Better approach
A good approach is however to use .recoverWith{} on a Future in case of failure.
For example:
def getUserFromCloud (userId: String): Future[String] = Future{
cloudProviderApi.getUsername(userId)
}.recoverWith{
Future.failed(s"$userId does not exist.")
}
What about the calling method?
Well you just map the success with and underscode and deal with the error by using recover:
getUserFromCloud("test").map(_ => {
//In case of success
}).recover{
//In case of failure, like return BadRequest.
}
More on recover and recoverWith in case you are interested: Scala recover or recoverWith
Most of the time my Future[T] operations are dependent to previous future in the chain. I am using flatMap function with pattern matching most of the time. Such as;
findUser(userId).flatMap {
case None => Future.successful(NotFound("No user with given id"))
case Some(user) => findAddress(user.addressId).flatMap {
case None => Future.successful(NotFound("No address with given id"))
case Some(address) => findCity(address.cityId).flatMap {
case None => Future.successful(NotFound("No city with given id"))
case Some => Future.successful(Ok)
}
}
}
with this way i am able to return an object related with the problem, all branchings are being handled. But downside of this approach in my opinion (and my code reading pleasure) it is getting nested a lot. Also if the lines are too long it is impossible to track which case statement is which even with a proper formatting. So that goes to the right-bottom side of the editor.
The other way one would suggest might using for comprehension. Below is kind of a equivalent of the code above. But the difference is for-comp one is throwing an exception if the if-guard is not satisfied. Also it returns an option to use which wherever i want to use i need to call get method (which i don't want to do);
val items = for {
user <- findUser(userId) if user.isDefined
address <- findAddress(user.addressId) if address.isDefined
city <- findCity(address.cityId) if address.isDefined
} yield (user.get, address.get, city.get)
Again one may suggest catching the exception but as i read from many sources catching exceptions are considered not good. Also the exception wouldn't provide which case statement didn't satisfy the condition.
Same thing applies for return statements as well. As myself coming from java and .net based languages, i am inclined to use the style below.
val user = Await.result(findUser(userId), timeout)
if (user.isEmpty) {
return Future.successful(NotFound("No user with given id"))
}
val address = Await.result(findAddress(user.get.addressId), timeout)
if (address.isEmpty) {
return Future.successful(NotFound("No address with given id"))
}
val city = Await.result(findUser(address.get.cityId), timeout)
if(city.isEmpty) {
return Future.successful(NotFound("No city with given id"))
}
Future.successful(Ok)
which is definitely out of question in my understanding. First of all it makes the code-block blocking, secondly again it forces me to use get values and uses return blocks which are similar with the throwing exceptions in the matter of cutting the execution short.
Haven't been able to find an elegant solution to this. I am currently going with the nested approach which makes it harder to read
Thanks
You should use .failed futures rather than successful to communicate exceptional conditions:
sealed trait NotFoundErr
class NoUser extends Exception("No user with given id") with NotFoundErr
class NoAddress extends Exception("No address with given id") with NotFoundErr
class NoCity extends Exception("No city with given id") with NotFoundErr
def getOrElse[T](ifNot: Exception)(what: => Future[Option[T]]) = what
.map(_.getOrElse(throw ifNot))
val items = for {
user <- getOrElse(new NoUser)(findUser(userId))
address <- getOrElse(new NoAddress)(findAddress(user.addressId))
city <- getOrElse(new NoCity)(findCity(address.cityId))
} yield (user, address, city)
items
.map(_ => Ok)
.recover { case e: Exception with NotFoundErr => NotFound(e.getMessage) }
You can make it look even fancier with an implicit:
object RichFuture {
implicit class Pimped[T](val f: Future[Option[T]]) extends AnyVal {
def orElse(what: => T) = f.map(_.getOrElse(what))
}
}
Now, you can write the for-comprehension like:
for {
user <- findUser(userId) orElse throw(new NoUser)
address <- findAddress(user.addressId) orElse throw(new NoAddress)
city <- findCity(address.cityId) orElse throw(new NoCity)
} yield (user, address, city)
The elegant solution to this problem is to use an appropriate data type to wrap the different failure cases.
I'd suggest you look into
Cats Validated or
Scalaz Validation
Those types collects the operation outcome and compose well in comprehensions and possibly with futures
Consider the following code inside a Play Framework controller:
val firstFuture = function1(id)
val secondFuture = function2(id)
val resultFuture = for {
first <- firstFuture
second <- secondFuture(_.get)
result <- function3(first, second)
} yield Ok(s"Processed $id")
resultFuture.map(result => result).recover { case t => InternalServerError(s"Error organizing files: $t.getMessage")}
Here are some details about the functions:
function1 returns Future[List]
function2 returns Future[Option[Person]]
function1 and function2 can run in parallel, but function3 needs the results for both.
Given this information, I have some questions:
Although the application is such that this code is very unlikely to be called with an improper id, I would like to handle this possibility. Basically, I would like to return NotFound if function2 returns None, but I can't figure out how to do that.
Will the recover call handle an Exception thrown any step of the way?
Is there a more elegant or idiomatic way to write this code?
Perhaps using collect, and then you can recover the NoSuchElementException--which yes, will recover a failure from any step of the way. resultFuture will either be successful with the mapped Result, or failed with the first exception that was thrown.
val firstFuture = function1(id)
val secondFuture = function2(id)
val resultFuture = for {
first <- firstFuture
second <- secondFuture.collect(case Some(x) => x)
result <- function3(first, second)
} yield Ok(s"Processed $id")
resultFuture.map(result => result)
.recover { case java.util.NoSuchElementException => NotFound }
.recover { case t => InternalServerError(s"Error organizing files: $t.getMessage")}
I would go with Scalaz OptionT. Maybe when you have only one function Future[Optipn[T]] it's overkill, but when you'll start adding more functions it will become super useful
import scala.concurrent.ExecutionContext.Implicits.global
import scalaz.OptionT
import scalaz.OptionT._
import scalaz.std.scalaFuture._
// Wrap 'some' result into OptionT
private def someOptionT[T](t: Future[T]): OptionT[Future, T] =
optionT[Future](t.map(Some.apply))
val firstFuture = function1(id)
val secondFuture = function2(id)
val action = for {
list <- someOptionT(firstFuture)
person <- optionT(secondFuture)
result = function3(list, person)
} yield result
action.run.map {
case None => NotFound
case Some(result) => Ok(s"Processed $id")
} recover {
case NonFatal(err) => InternalServerError(s"Error organizing files: ${err.getMessage}")
}
I am trying to create a neat construction with for-comprehension for business logic built on futures. Here is a sample which contains a working example based on Exception handling:
(for {
// find the user by id, findUser(id) returns Future[Option[User]]
userOpt <- userDao.findUser(userId)
_ = if (!userOpt.isDefined) throw new EntityNotFoundException(classOf[User], userId)
user = userOpt.get
// authenticate it, authenticate(user) returns Future[AuthResult]
authResult <- userDao.authenticate(user)
_ = if (!authResult.ok) throw new AuthFailedException(userId)
// find the good owned by the user, findGood(id) returns Future[Option[Good]]
goodOpt <- goodDao.findGood(goodId)
_ = if (!good.isDefined) throw new EntityNotFoundException(classOf[Good], goodId)
good = goodOpt.get
// check ownership for the user, checkOwnership(user, good) returns Future[Boolean]
ownership <- goodDao.checkOwnership(user, good)
if (!ownership) throw new OwnershipException(user, good)
_ <- goodDao.remove(good)
} yield {
renderJson(Map(
"success" -> true
))
})
.recover {
case ex: EntityNotFoundException =>
/// ... handle error cases ...
renderJson(Map(
"success" -> false,
"error" -> "Your blahblahblah was not found in our database"
))
case ex: AuthFailedException =>
/// ... handle error cases ...
case ex: OwnershipException =>
/// ... handle error cases ...
}
However this might be seen as a non-functional or non-Scala way to handle the things. Is there a better way to do this?
Note that these errors come from different sources - some are at the business level ('checking ownership') and some are at controller level ('authorization') and some are at db level ('entity not found'). So approaches when you derive them from a single common error type might not work.
Don't use exceptions for expected behaviour.
It's not nice in Java, and it's really not nice in Scala. Please see this question for more information about why you should avoid using exceptions for regular control flow. Scala is very well equipped to avoid using exceptions: you can use Eithers.
The trick is to define some failures you might encounter, and convert your Options into Eithers that wrap these failures.
// Failures.scala
object Failures {
sealed trait Failure
// Four types of possible failures here
case object UserNotFound extends Failure
case object NotAuthenticated extends Failure
case object GoodNotFound extends Failure
case object NoOwnership extends Failure
// Put other errors here...
// Converts options into Eithers for you
implicit class opt2either[A](opt: Option[A]) {
def withFailure(f: Failure) = opt.fold(Left(f))(a => Right(a))
}
}
Using these helpers, you can make your for comprehension readable and exception free:
import Failures._
// Helper function to make ownership checking more readable in the for comprehension
def checkGood(user: User, good: Good) = {
if(checkOwnership(user, good))
Right(good)
else
Left(NoOwnership)
}
// First create the JSON
val resultFuture: Future[Either[Failure, JsResult]] = for {
userRes <- userDao.findUser(userId)
user <- userRes.withFailure(UserNotFound).right
authRes <- userDao.authenticate(user)
auth <- authRes.withFailure(NotAuthenticated).right
goodRes <- goodDao.findGood(goodId)
good <- goodRes.withFailure(GoodNotFound).right
checkedGood <- checkGood(user, good).right
} yield renderJson(Map("success" -> true)))
// Check result and handle any failures
resultFuture.map { result =>
result match {
case Right(json) => json // serve json
case Left(failure) => failure match {
case UserNotFound => // Handle errors
case NotAuthenticated =>
case GoodNotFound =>
case NoOwnership =>
case _ =>
}
}
}
You could clean up the for comprehension a little to look like this:
for {
user <- findUser(userId)
authResult <- authUser(user)
good <- findGood(goodId)
_ <- checkOwnership(user, good)
_ <- goodDao.remove(good)
} yield {
renderJson(Map(
"success" -> true
))
}
Assuming these methods:
def findUser(id:Long) = find(id, userDao.findUser)
def findGood(id:Long) = find(id, goodDao.findGood)
def find[T:ClassTag](id:Long, f:Long => Future[Option[T]]) = {
f(id).flatMap{
case None => Future.failed(new EntityNotFoundException(implicitly[ClassTag[T]].runtimeClass, id))
case Some(entity) => Future.successful(entity)
}
}
def authUser(user:User) = {
userDao.authenticate(user).flatMap{
case result if result.ok => Future.failed(new AuthFailedException(userId))
case result => Future.successful(result)
}
}
def checkOwnership(user:User, good:Good):Future[Boolean] = {
val someCondition = true //real logic for ownership check goes here
if (someCondition) Future.successful(true)
else Future.failed(new OwnershipException(user, good))
}
The idea here is to use flatMap to turn things like Options that are returned wrapped in Futures into failed Futures when they are None. There are going to be a lot of ways to do clean up that for comp and this is one possible way to do it.
The central challenge is that for-comprehensions can only work on one monad at a time, in this case it being the Future monad and the only way to short-circuit a sequence of future calls is for the future to fail. This works because the subsequent calls in the for-comprehension are just map and flatmap calls, and the behavior of a map/flatmap on a failed Future is to return that future and not execute the provided body (i.e. the function being called).
What you are trying to achieve is the short-cicuiting of a workflow based on some conditions and not do it by failing the future. This can be done by wrapping the result in another container, let's call it Result[A], which gives the comprehension a type of Future[Result[A]]. Result would either contain a result value, or be a terminating result. The challenge is how to:
provide subsequent function calls the value contained by a prior non-terminating Result
prevent the subsequent function call from being evaluated if the Result is terminating
map/flatmap seem like the candidates for doing these types of compositions, except we will have to call them manually, since the only map/flatmap that the for-comprehension can evaluate is one that results in a Future[Result[A]].
Result could be defined as:
trait Result[+A] {
// the intermediate Result
def value: A
// convert this result into a final result based on another result
def given[B](other: Result[B]): Result[A] = other match {
case x: Terminator => x
case v => this
}
// replace the value of this result with the provided one
def apply[B](v: B): Result[B]
// replace the current result with one based on function call
def flatMap[A2 >: A, B](f: A2 => Future[Result[B]]): Future[Result[B]]
// create a new result using the value of both
def combine[B](other: Result[B]): Result[(A, B)] = other match {
case x: Terminator => x
case b => Successful((value, b.value))
}
}
For each call, the action is really a potential action, as calling it on or with a terminating result, will simply maintain the terminating result. Note that Terminator is a Result[Nothing] since it will never contain a value and any Result[+A] can be a Result[Nothing].
The terminating result is defined as:
sealed trait Terminator extends Result[Nothing] {
val value = throw new IllegalStateException()
// The terminator will always short-circuit and return itself as
// the success rather than execute the provided block, thus
// propagating the terminating result
def flatMap[A2 >: Nothing, B](f: A2 => Future[Result[B]]): Future[Result[B]] =
Future.successful(this)
// if we apply just a value to a Terminator the result is always the Terminator
def apply[B](v: B): Result[B] = this
// this apply is a convenience function for returning this terminator
// or a successful value if the input has some value
def apply[A](opt: Option[A]) = opt match {
case None => this
case Some(v) => Successful[A](v)
}
// this apply is a convenience function for returning this terminator or
// a UnitResult
def apply(bool: Boolean): Result[Unit] = if (bool) UnitResult else this
}
The terminating result makes it possible to to short-circuit calls to functions that require a value [A] when we've already met our terminating condition.
The non-terminating result is defined as:
trait SuccessfulResult[+A] extends Result[A] {
def apply[B](v: B): Result[B] = Successful(v)
def flatMap[A2 >: A, B](f: A2 => Future[Result[B]]): Future[Result[B]] = f(value)
}
case class Successful[+A](value: A) extends SuccessfulResult[A]
case object UnitResult extends SuccessfulResult[Unit] {
val value = {}
}
The non-teminating result makes it possible to provide the contained value [A] to functions. For good measure, I've also predefined a UnitResult for functions that are purely side-effecting, like goodDao.removeGood.
Now let's define your good, but terminating conditions:
case object UserNotFound extends Terminator
case object NotAuthenticated extends Terminator
case object GoodNotFound extends Terminator
case object NoOwnership extends Terminator
Now we have the tools to create the the workflow you were looking for. Each for comprehention wants a function that returns a Future[Result[A]] on the right-hand side, producing a Result[A] on the left-hand side. The flatMap on Result[A] makes it possible to call (or short-circuit) a function that requires an [A] as input and we can then map its result to a new Result:
def renderJson(data: Map[Any, Any]): JsResult = ???
def renderError(message: String): JsResult = ???
val resultFuture = for {
// apply UserNotFound to the Option to conver it into Result[User] or UserNotFound
userResult <- userDao.findUser(userId).map(UserNotFound(_))
// apply NotAuthenticated to AuthResult.ok to create a UnitResult or NotAuthenticated
authResult <- userResult.flatMap(user => userDao.authenticate(user).map(x => NotAuthenticated(x.ok)))
goodResult <- authResult.flatMap(_ => goodDao.findGood(goodId).map(GoodNotFound(_)))
// combine user and good, so we can feed it into checkOwnership
comboResult = userResult.combine(goodResult)
ownershipResult <- goodResult.flatMap { case (user, good) => goodDao.checkOwnership(user, good).map(NoOwnership(_))}
// in order to call removeGood with a good value, we take the original
// good result and potentially convert it to a Terminator based on
// ownershipResult via .given
_ <- goodResult.given(ownershipResult).flatMap(good => goodDao.removeGood(good).map(x => UnitResult))
} yield {
// ownership was the last result we cared about, so we apply the output
// to it to create a Future[Result[JsResult]] or some Terminator
ownershipResult(renderJson(Map(
"success" -> true
)))
}
// now we can map Result into its value or some other value based on the Terminator
val jsFuture = resultFuture.map {
case UserNotFound => renderError("User not found")
case NotAuthenticated => renderError("User not authenticated")
case GoodNotFound => renderError("Good not found")
case NoOwnership => renderError("No ownership")
case x => x.value
}
I know that's a whole lot of setup, but at least the Result type can be used for any Future for-comprehension that has terminating conditions.