I have an use case where I need to use for comprehension with ADT in Scala. I could write the same code using flatMaps but it seems a bit unreadable. Below is the piece of code.
case class MovieRecord(movie: Movie,
screenId: String,
availableSeats: Int,
reservedSeats: Option[Int] = None) {
def movieInfo = new MovieInfoResponse(movie.imdbId, screenId, movie.title, availableSeats, reservedSeats.getOrElse(0))
}
sealed trait MovieBookingInformation
case class MovieBookingInformationFetched(bookMovie: MovieRecord) extends MovieBookingInformation
case object MovieBookingInformationFetchError extends MovieBookingInformation
def modifyBooking(reserveMovie: MovieSelected): Future[String] = {
fetchRecordByImdbAndScreenId(reserveMovie.imdbId, reserveMovie.screenId) flatMap {
case MovieBookingInformationFetched(m) if (m.availableSeats > 0) =>
updateSeatReservationByImdbAndScreenId(m.copy(availableSeats = m.availableSeats - 1, reservedSeats = Some(m.reservedSeats.getOrElse(0) + 1))) flatMap {
case MovieBookingUpdated(updatedMovieBooking) =>
Future.successful(s"One seat reserved at Screen - ${updatedMovieBooking.screenId}")
case MovieBookingUpdateFailed =>
Future.successful(s"Movie seat reservation failed at screen ${reserveMovie.screenId}")
}
case MovieBookingInformationFetched(m) =>
Future.successful(s"Sorry! No more seats available for ${m.movie.title} at Screen - ${m.screenId}")
case MovieBookingInformationFetchError => Future.successful(s"No movie with IMDB ID ${reserveMovie.imdbId} found at ${reserveMovie.screenId}")
}
}
In the above code, the next method is invoked on a resultant ADT content and result of if statement. How do I include the if statement in the for-comprehension to achieve the same.
Thanks in advance.
You can put pattern matching and if statements in a for comprehension on Future:
for {
MovieBookingInformationFetched(m) <- future1
if m.availableSeats > 0
MovieBookingUpdated(updatedMovieBooking) <- future2(m)
} yield updatedMovieBooking
However, this will be translated into Future.filter, so if the predicate is not satisfied, or the pattern cannot be matched, you'll end up with
Future.failed(new NoSuchElementException("Future.filter predicate was not satisfied")
You can then catch this failure in a recover statement, after the for-comp. The problem is that you want to catch three different "errors": the fact that there are no more available seats, the fact that fetchRecordByImdbAndScreenId can return a MovieBookingInformationFetchError, and the fact that updateSeatReservationByImdbAndScreenId can return a MovieBookingUpdateFailed.
You won't be able to dissociate between these three using only for comprehension, unless you define custom exceptions, instead of custom result types (and recover these exceptions afterwards).
Related
In my code, I very often need to process a list by performing operations on an internal model. For each processed element, the model is returned and then a 'new' model is used for the next element of the list.
Usually, I implement this by using a tail recursive method:
def createCar(myModel: Model, record: Record[Any]): Either[CarError, Model] = {
record match {
case c: Car =>
// Do car stuff...
val newModel: Model = myModel.createCar(record)
Right(newModel)
case _ => Left(CarError())
}
}
#tailrec
def processCars(myModel: Model, records: List[Record[Any]]): Either[CarError, Model] =
records match {
case x :: xs =>
createCar(myModel, x) match {
case Right(m) => processCars(m, xs)
case e#Left(_) => e
}
case Nil => Right(myModel)
}
Since I keep repeating this kind of pattern, I am searching for ways to make it more concise and more functional (i.e., the Scala way).
I have looked into foldLeft, but cannot get it to work with Either:
recordsList.foldLeft(myModel) { (m, r) =>
// Do car stuff...
Right(m)
}
Is foldLeft a proper replacement? How can I get it to work?
Following up on my earlier comment, here's how to unfold() to get your result. [Note: Scala 2.13.x]
def processCars(myModel: Model
,records: List[Record[_]]
): Either[CarError, Model] =
LazyList.unfold((myModel,records)) { case (mdl,recs) =>
recs.headOption.map{
createCar(mdl, _).fold(Left(_) -> (mdl,Nil)
,m => Right(m) -> (m,recs.tail))
}
}.last
The advantage here is:
early termination - Iterating through the records stops after the 1st Left is returned or after all the records have been processed, whichever comes first.
memory efficient - Since we're building a LazyList, and nothing is holding on to the head of the resulting list, every element except the last should be immediately released for garbage collection.
You can do that using fold like that:
def processCars(myModel: Model, records: List[Record[Any]]): Either[CarError, Model] = {
records.foldLeft[Either[CarError, Model]](Right(myModel))((m, r) => {
m.fold(Left.apply, { model =>
createCar(model, r).fold(Left.apply, Right.apply)
})
})
}
I have to get a list of issues for each file of a given list from a REST API with Scala. I want to do the requests in parallel, and use the Dispatch library for this. My method is called from a Java framework and I have to wait at the end of this method for the result of all the futures to yield the overall result back to the framework. Here's my code:
def fetchResourceAsJson(filePath: String): dispatch.Future[json4s.JValue]
def extractLookupId(json: org.json4s.JValue): Option[String]
def findLookupId(filePath: String): Future[Option[String]] =
for (json <- fetchResourceAsJson(filePath))
yield extractLookupId(json)
def searchIssuesJson(lookupId: String): Future[json4s.JValue]
def extractIssues(json: org.json4s.JValue): Seq[Issue]
def findIssues(lookupId: String): Future[Seq[Issue]] =
for (json <- searchIssuesJson(componentId))
yield extractIssues(json)
def getFilePathsToProcess: List[String]
def thisIsCalledByJavaFramework(): java.util.Map[String, java.util.List[Issue]] = {
val finalResultPromise = Promise[Map[String, Seq[Issue]]]()
// (1) inferred type of issuesByFile not as expected, cannot get
// the type system happy, would like to have Seq[Future[(String, Seq[Issue])]]
val issuesByFile = getFilePathsToProcess map { f =>
findLookupId(f).flatMap { lookupId =>
(f, findIssues(lookupId)) // I want to yield a tuple (String, Seq[Issue]) here
}
}
Future.sequence(issuesByFile) onComplete {
case Success(x) => finalResultPromise.success(x) // (2) how to return x here?
case Failure(x) => // (3) how to return null from here?
}
//TODO transform finalResultPromise to Java Map
}
This code snippet has several issues. First, I'm not getting the type I would expect for issuesByFile (1). I would like to just ignore the result of findLookUpId if it is not able to find the lookUp ID (i.e., None). I've read in various tutorials that Future[Option[X]] is not easy to handle in function compositions and for expressions in Scala. So I'm also curious what the best practices are to handle these properly.
Second, I somehow have to wait for all futures to finish, but don't know how to return the result to the calling Java framework (2). Can I use a promise here to achieve this? If yes, how can I do it?
And last but not least, in case of any errors, I would just like to return null from thisIsCalledByJavaFramework but don't know how (3).
Any help is much appreciated.
Thanks,
Michael
Several points:
The first problem at (1) is that you don't handle the case where findLookupId returns None. You need to decide what to do in this case. Fail the whole process? Exclude that file from the list?
The second problem at (1) is that findIssues will itself return a Future, which you need to map before you can build the result tuple
There's a shortcut for map and then Future.sequence: Future.traverse
If you cannot change the result type of the method because the Java interface is fixed and cannot be changed to support Futures itself you must wait for the Future to be completed. Use Await.ready or Await.result to do that.
Taking all that into account and choosing to ignore files for which no id could be found results in this code:
// `None` in an entry for a file means that no id could be found
def entryForFile(file: String): Future[(String, Option[Seq[Issue]])] =
findLookupId(file).flatMap {
// the need for this kind of pattern match shows
// the difficulty of working with `Future[Option[T]]`
case Some(id) ⇒ findIssues(id).map(issues ⇒ file -> Some(issues))
case None ⇒ Future.successful(file -> None)
}
def thisIsCalledByJavaFramework(): java.util.Map[String, java.util.List[Issue]] = {
val issuesByFile: Future[Seq[(String, Option[Seq[Issue]])]] =
Future.traverse(getFilePathsToProcess)(entryForFile)
import scala.collection.JavaConverters._
try
Await.result(issuesByFile, 10.seconds)
.collect {
// here we choose to ignore entries where no id could be found
case (f, Some(issues)) ⇒ f -> issues
}
.toMap.mapValues(_.asJava).asJava
catch {
case NonFatal(_) ⇒ null
}
}
effective scala style states the following:
this is both succinct and correct, but nearly every reader will have a
difficult time recovering the original intent of the author. A
strategy that often serves to clarify is to name intermediate results
and parameters:
val votesByLang = votes groupBy { case (lang, _) => lang }
val sumByLang = votesByLang map { case (lang, counts) =>
val countsOnly = counts map { case (_, count) => count }
(lang, countsOnly.sum)
}
val orderedVotes = sumByLang.toSeq
.sortBy { case (_, count) => count }
.reverse
what I fail to understand is in this style if i have errors while evaluating the value for votesByLang, sumByLang, ... how can I have a single recover at the end after all if i just had .map.map ... I could have a single recover at the end. So is it possible to have a single recover also with this style? (and not only possible but with good style...)
Or in other words what is the succinct and correct error handling in this succinct and correct functional pipelining code?
you see with the non-concise version i'm able to code this:
object MyRealMainObj extends App {
println(
Try(1)
.map(doOne)
.map(doTwo)
.recover { // non succinct version can catch any prior error how to achieve the same in succinct version
case e: Throwable => println("recovering from: " + e.getMessage)
}
)
def doOne(i: Int): Int = { i + 1; throw new RuntimeException("failed in one") }
def doTwo(i: Int): Int = { i + 2; throw new RuntimeException("failed in two") }
}
a single recover will catch any previous error in the .map but with the concise-succinct how can I achieve the same in concise and succinct way?
It sound like you expect some sort of scope to be magically created by .recover. This is not the case. With some minor simplifications, Try(expr) is
try {
val result = expr
Success(result)
catch {
e => Failure(e)
}
tryResult.map(f) is :
tryResult match {
case Success(x) => Try(f(x))
case f: Failure => f
}
tryResult.recover(f) is
tryResult match {
case Failure(e) if f.isDefinedAt(e) => Try(f(e))
case other => other
}
There is no special magic. At every stage, you have values, not code in the scope of some try/catch. Putting those values in val will not change a thing. If you "debug by hand", you will notice that you get a Success object, then at some point, one of your map may fail and return a Failure, the following ones will keep being failures, and then when you pass the last one to recover, it may go back to Success. Settings vals in between cannot make any difference.
I'm developing a method that is supposed to persist an object, if it passes a list of conditions.
If any (or many) condition fail (or any other kind of error appears), a list with the errors should be returned, if everything goes well, a saved entity should be returned.
I was thinking about something like this (it's pseudocode, of course):
request.body.asJson.map { json =>
json.asOpt[Wine].map { wine =>
wine.save.map { wine =>
Ok(toJson(wine.update).toString)
}.getOrElse { errors => BadRequest(toJson(errors))}
}.getOrElse { BadRequest(toJson(Error("Invalid Wine entity")))}
}.getOrElse { BadRequest(toJson(Error("Expecting JSON data")))}
That is, I'd like to treat it like an Option[T], that if any validation fails, instead of returning None it gives me the list of errors...
The idea is to return an array of JSON errors...
So the question would be, is this the right way to handle these kind of situation? And what would be the way to accomplish it in Scala?
--
Oops, just posted the question and discovered Either
http://www.scala-lang.org/api/current/scala/Either.html
Anyway, I'd like to know what you think about the chosen approach, and if there's any other better alternative to handle it.
Using scalaz you have Validation[E, A], which is like Either[E, A] but has the property that if E is a semigroup (meaning things that can be concatenated, like lists) than multiple validated results can be combined in a way that keeps all the errors that occured.
Using Scala 2.10-M6 and Scalaz 7.0.0-M2 for example, where Scalaz has a custom Either[L, R] named \/[L, R] which is right-biased by default:
import scalaz._, Scalaz._
implicit class EitherPimp[E, A](val e: E \/ A) extends AnyVal {
def vnel: ValidationNEL[E, A] = e.validation.toValidationNEL
}
def parseInt(userInput: String): Throwable \/ Int = ???
def fetchTemperature: Throwable \/ Int = ???
def fetchTweets(count: Int): Throwable \/ List[String] = ???
val res = (fetchTemperature.vnel |#| fetchTweets(5).vnel) { case (temp, tweets) =>
s"In $temp degrees people tweet ${tweets.size}"
}
Here result is a Validation[NonEmptyList[Throwable], String], either containing all the errors occured (temp sensor error and/or twitter error or none) or the successful message. You can then switch back to \/ for convenience.
Note: The difference between Either and Validation is mainly that with Validation you can accumulate errors, but cannot flatMap to lose the accumulated errors, while with Either you can't (easily) accumulate but can flatMap (or in a for-comprehension) and possibly lose all but the first error message.
About error hierarchies
I think this might be of interest for you. Regardless of using scalaz/Either/\//Validation, I experienced that getting started was easy but going forward needs some additional work. The problem is, how do you collect errors from multiple erring functions in a meaningful way? Sure, you can just use Throwable or List[String] everywhere and have an easy time, but doesn't sound too much usable or interpretable. Imagine getting a list of errors like "child age missing" :: "IO error reading file" :: "division by zero".
So my choice is to create error hierarchies (using ADT-s), just like as one would wrap checked exceptions of Java into hierarchies. For example:
object errors {
object gamestart {
sealed trait Error
case class ResourceError(e: errors.resource.Error) extends Error
case class WordSourceError(e: errors.wordsource.Error) extends Error
}
object resource {
case class Error(e: GdxRuntimeException)
}
object wordsource {
case class Error(e: /*Ugly*/ Any)
}
}
Then when using result of erring functions with different error types, I join them under a relevant parent error type.
for {
wordSource <-
errors.gamestart.WordSourceError <-:
errors.wordsource.Error <-:
wordSourceCreator.doCreateWordSource(mtRandom).catchLeft.unsafePerformIO.toEither
resources <-
errors.gamestart.ResourceError <-:
GameViewResources(layout)
} yield ...
Here f <-: e maps the function f on the left of e: \/ since \/ is a Bifunctor. For se: scala.Either you might have se.left.map(f).
This may be further improved by providing shapeless HListIsos to be able to draw nice error trees.
Revisions
Updated: (e: \/).vnel lifts the failure side into a NonEmptyList so if we have a failure we have at least one error (was: or none).
If you have Option values, and you want to turn them into success/failure values, you can turn an Option into an Either using the toLeft or toRight method.
Usually a Right represents success, so use o.toRight("error message") to turn Some(value) into Right(value) and None into Left("error message").
Unfortunately Scala doesn't recognise this right-bias by default, so you have to jump through a hoop (by calling the .right method) in order to neatly compose your Eithers in a for-comprehension.
def requestBodyAsJson: Option[String] = Some("""{"foo":"bar"}""")
def jsonToWine(json: String): Option[Wine] = sys.error("TODO")
val wineOrError: Either[String, Wine] = for {
body <- requestBodyAsJson.toRight("Expecting JSON Data").right
wine <- jsonToWine(body).toRight("Invalid Wine entity").right
} yield wine
If you need an empty value, instead of using Either[A,Option[B]] you can use lift Box, which can have three values:
Full (there is a valid result)
Empty (no result, but no error either)
Failure (an error happened)
Box are more flexible than Either thanks to a rich API. Of course, although they were created for Lift, you can use them in any other framework.
well, this is my attemp using Either
def save() = CORSAction { request =>
request.body.asJson.map { json =>
json.asOpt[Wine].map { wine =>
wine.save.fold(
errors => JsonBadRequest(errors),
wine => Ok(toJson(wine).toString)
)
}.getOrElse (JsonBadRequest("Invalid Wine entity"))
}.getOrElse (JsonBadRequest("Expecting JSON data"))
}
And wine.save is like the following:
def save(wine: Wine): Either[List[Error],Wine] = {
val errors = validate(wine)
if (errors.length > 0) {
Left(errors)
} else {
DB.withConnection { implicit connection =>
val newId = SQL("""
insert into wine (
name, year, grapes, country, region, description, picture
) values (
{name}, {year}, {grapes}, {country}, {region}, {description}, {picture}
)"""
).on(
'name -> wine.name, 'year -> wine.year, 'grapes -> wine.grapes,
'country -> wine.country, 'region -> wine.region, 'description -> wine.description,
'picture -> wine.picture
).executeInsert()
val newWine = for {
id <- newId;
wine <- findById(id)
} yield wine
newWine.map { wine =>
Right(wine)
}.getOrElse {
Left(List(ValidationError("Could not create wine")))
}
}
}
}
Validate checks several preconditions. I still have to add a try/catch to catch any db error
I'm still looking for a way to improve the whole thing, it feels much to verbose to my taste...
Option monad is a great expressive way to deal with something-or-nothing things in Scala. But what if one needs to log a message when "nothing" occurs? According to the Scala API documentation,
The Either type is often used as an
alternative to scala.Option where Left
represents failure (by convention) and
Right is akin to Some.
However, I had no luck to find best practices using Either or good real-world examples involving Either for processing failures. Finally I've come up with the following code for my own project:
def logs: Array[String] = {
def props: Option[Map[String, Any]] = configAdmin.map{ ca =>
val config = ca.getConfiguration(PID, null)
config.properties getOrElse immutable.Map.empty
}
def checkType(any: Any): Option[Array[String]] = any match {
case a: Array[String] => Some(a)
case _ => None
}
def lookup: Either[(Symbol, String), Array[String]] =
for {val properties <- props.toRight('warning -> "ConfigurationAdmin service not bound").right
val logsParam <- properties.get("logs").toRight('debug -> "'logs' not defined in the configuration").right
val array <- checkType(logsParam).toRight('warning -> "unknown type of 'logs' confguration parameter").right}
yield array
lookup.fold(failure => { failure match {
case ('warning, msg) => log(LogService.WARNING, msg)
case ('debug, msg) => log(LogService.DEBUG, msg)
case _ =>
}; new Array[String](0) }, success => success)
}
(Please note this is a snippet from a real project, so it will not compile on its own)
I'd be grateful to know how you are using Either in your code and/or better ideas on refactoring the above code.
Either is used to return one of possible two meaningful results, unlike Option which is used to return a single meaningful result or nothing.
An easy to understand example is given below (circulated on the Scala mailing list a while back):
def throwableToLeft[T](block: => T): Either[java.lang.Throwable, T] =
try {
Right(block)
} catch {
case ex => Left(ex)
}
As the function name implies, if the execution of "block" is successful, it will return "Right(<result>)". Otherwise, if a Throwable is thrown, it will return "Left(<throwable>)". Use pattern matching to process the result:
var s = "hello"
throwableToLeft { s.toUpperCase } match {
case Right(s) => println(s)
case Left(e) => e.printStackTrace
}
// prints "HELLO"
s = null
throwableToLeft { s.toUpperCase } match {
case Right(s) => println(s)
case Left(e) => e.printStackTrace
}
// prints NullPointerException stack trace
Hope that helps.
Scalaz library has something alike Either named Validation. It is more idiomatic than Either for use as "get either a valid result or a failure".
Validation also allows to accumulate errors.
Edit: "alike" Either is complettly false, because Validation is an applicative functor, and scalaz Either, named \/ (pronounced "disjonction" or "either"), is a monad.
The fact that Validation can accumalate errors is because of that nature. On the other hand, / has a "stop early" nature, stopping at the first -\/ (read it "left", or "error") it encounters. There is a perfect explanation here: http://typelevel.org/blog/2014/02/21/error-handling.html
See: http://scalaz.googlecode.com/svn/continuous/latest/browse.sxr/scalaz/example/ExampleValidation.scala.html
As requested by the comment, copy/paste of the above link (some lines removed):
// Extracting success or failure values
val s: Validation[String, Int] = 1.success
val f: Validation[String, Int] = "error".fail
// It is recommended to use fold rather than pattern matching:
val result: String = s.fold(e => "got error: " + e, s => "got success: " + s.toString)
s match {
case Success(a) => "success"
case Failure(e) => "fail"
}
// Validation is a Monad, and can be used in for comprehensions.
val k1 = for {
i <- s
j <- s
} yield i + j
k1.toOption assert_≟ Some(2)
// The first failing sub-computation fails the entire computation.
val k2 = for {
i <- f
j <- f
} yield i + j
k2.fail.toOption assert_≟ Some("error")
// Validation is also an Applicative Functor, if the type of the error side of the validation is a Semigroup.
// A number of computations are tried. If the all success, a function can combine them into a Success. If any
// of them fails, the individual errors are accumulated.
// Use the NonEmptyList semigroup to accumulate errors using the Validation Applicative Functor.
val k4 = (fNel <**> fNel){ _ + _ }
k4.fail.toOption assert_≟ some(nel1("error", "error"))
The snippet you posted seems very contrived. You use Either in a situation where:
It's not enough to just know the data isn't available.
You need to return one of two distinct types.
Turning an exception into a Left is, indeed, a common use case. Over try/catch, it has the advantage of keeping the code together, which makes sense if the exception is an expected result. The most common way of handling Either is pattern matching:
result match {
case Right(res) => ...
case Left(res) => ...
}
Another interesting way of handling Either is when it appears in a collection. When doing a map over a collection, throwing an exception might not be viable, and you may want to return some information other than "not possible". Using an Either enables you to do that without overburdening the algorithm:
val list = (
library
\\ "books"
map (book =>
if (book \ "author" isEmpty)
Left(book)
else
Right((book \ "author" toList) map (_ text))
)
)
Here we get a list of all authors in the library, plus a list of books without an author. So we can then further process it accordingly:
val authorCount = (
(Map[String,Int]() /: (list filter (_ isRight) map (_.right.get)))
((map, author) => map + (author -> (map.getOrElse(author, 0) + 1)))
toList
)
val problemBooks = list flatMap (_.left.toSeq) // thanks to Azarov for this variation
So, basic Either usage goes like that. It's not a particularly useful class, but if it were you'd have seen it before. On the other hand, it's not useless either.
Cats has a nice way to create an Either from exception-throwing code:
val either: Either[NumberFormatException, Int] =
Either.catchOnly[NumberFormatException]("abc".toInt)
// either: Either[NumberFormatException,Int] = Left(java.lang.NumberFormatException: For input string: "abc")
in https://typelevel.org/cats/datatypes/either.html#working-with-exception-y-code