I have an expensive case statement which needs to hit the database to determine a complete match. If there is a match, the result from the aforementioned call must be used to perform further operations:
def intent = {
case request # GET(Path(Seg(database :: Nil))) if recordsFrom(database) != Nil =>
renderOutput(recordsFrom(database))
case ...
}
I would like to call recordsFrom(database) only once. In the above example, it is called twice. It seems like I should be able to apply some alias to the statement?
Lawrence, from what I'm seeing you're using Unfiltered to handle a RESTful request but you've also combined a database lookup with that response filtering. I would advise you not to do that. Instead I'd arrange things as following:
val dbReqCommand = new DBRequestCommand(myDbConPool)
def intent ={
case req # GET(Path(Seq(database :: Nil))) => dbReqCommand(req, database)
}
Wherein you've encapsulated the db requests in an object that you could substitute out for testing purposes (think integration tests without a DB backend.) Within the request handler you might then put in the response:
Option(recordsFrom(database)) match{
case Some(value) => OK ~> renderOpupt(value)
case None => //an error response or Pass
}
That way you might have something along the lines of:
trait DBReqPlan{
def dbReqCommand: RequestCommand[String]
def intent ={
case req # GET(Path(Seq(database :: Nil))) => dbReqCommand(req, database)
}
}
which is easier to test against and work with.
What's wrong with:
def intent = {
case request # GET(Path(Seg(database :: Nil))) =>
val records = recordsFrom(database)
if(!records.isEmpty){
renderOutput(records)
} else {
...
}
case ...
You can move the body of the first case to a different function if you want to avoid having too many nested blocks.
Related
I'm slowly wrapping my brain around Futures in Scala, and have a bit of a layer cake going on that I'm trying to unravel.
The specific use case is a DeferredResolver in sangria-graphql + akka. I've stolen their demo code, which looks like this
Future.fromTry(Try(
friendIds map (id => CharacterRepo.humans.find(_.id == id) orElse CharacterRepo.droids.find(_.id == id))))
and added my own modification to it. Theirs does an in-memory lookup, whereas mine asks something of another actor:
Future.fromTry(Try(
accountIds match {
case h :: _ =>
val f = sender ? TargetedMessage(h)
val resp = Await.result(f, timeout.duration).asInstanceOf[TargetedMessage]
marshallAccount(resp.body)
case _ => throw new Exception("Not found")
}
))
The pertinent piece here is that I pick the first element in the list, send it to an ActorRef that I got elsewhere and wait for the result. This works. What I'd like to do, however, is not have to wait for the result here, but return the whole thing as a Future
Future.fromTry(Try(
accountIds match {
case h :: _ =>
sender ? TargetedMessage(h) map {
case resp:TargetedMessage => marshallAccount(resp.body)
}
case _ => throw new Exception("Not found")
}
))
This doesn't work. When this is consumed, instead of being of type Account (the return type of function marshallAccount, it's of type Promise. If I understand correctly, it's because instead of having a return type of Future[Account], this has a type of Future[Future[Account]]
How do I flatten this?
You are looking at the wrong API method. Future.fromTry is used to create an immediately resolved Future, meaning the call is not actually asynchronous. Dive into the implementation of Future.fromTry which will take you to:
def fromTry[T](result: Try[T]): Promise[T] = new impl.Promise.KeptPromise[T](result)
A promise kept is basically something that has already happened, so just like Future.successful this is just used to ensure the right return type or similar, it's not actually a way to make something async.
The reason why the return type is Future[Future[Something]] is because you are trying to wrap something that already returns a future into another future.
The ask pattern, namely sender ? TargetMessage(h) is a way to ask something of an actor and await for a result, which will return a future.
The correct way to approach this:
val future: Future[Account] = accountIds match {
case h :: _ => sender ? TargetedMessage(h) map (marshallAccount(_.body)
case _ => Future.failed(throw new Exception("Not found"))
}
Basically you need to use Future.failed to return a failed future from an exception if you want to keep the return type consistent. It's worth reviewing this tutorial to learn a bit more about Futures and how to write application logic with them.
I am trying to incorporate a database into my http-microservice.
The microservice has a function getValueFromInternet(val: Foo): Future[Value] which was being called by my microservice on a GET request. Now, I want it to happen such that, a function getValue(val: Foo): Future[Value] would first query a db and if the database returns no results, call getValueFromInternet. The database query returns a Future[Seq[Value2]] where I can convert Value2 to Value using a function. And if no entry is found corresponding to that value, an empty Vector is returned.
This is what I have tried so far:
def getValue(val: Foo): Future[Value] = {
val resultFuture = db.getValue(val)
// 1st attempt. Clearly wrong
resultFuture onComplete {
case Success(Vector()) => getValueFromInternet(val)
case Success(vec) => convertValue2to1(vec.head)
}
// 2nd attempt. This is also wrong
resultFuture match {
case Future(Success(Vector())) => getValueFromInternet(val)
case Future(Success(vec)) => convertValue2to1(vec.head)
}
}
I would be grateful for any help suggesting how I can do this.
I have implemented the database and microservice independently and you can find them here and here
You have to use flatMap, since the thing you want to do if the first operation does not return a result also returns a future.
This is as close to your code as possible while still compiling. Note that you can't have identifiers called val in scala, since that is a keyword.
def getValue(v: Foo)(implicit ec: ExecutionContext): Future[Value] = {
val resultFuture: Future[Seq[Value2]] = db.getValue(v)
resultFuture.flatMap { vec =>
if(vec.isEmpty)
getValueFromInternet(v)
else
Future.successful(convertValue2to1(vec.head))
}
}
I have to get a list of issues for each file of a given list from a REST API with Scala. I want to do the requests in parallel, and use the Dispatch library for this. My method is called from a Java framework and I have to wait at the end of this method for the result of all the futures to yield the overall result back to the framework. Here's my code:
def fetchResourceAsJson(filePath: String): dispatch.Future[json4s.JValue]
def extractLookupId(json: org.json4s.JValue): Option[String]
def findLookupId(filePath: String): Future[Option[String]] =
for (json <- fetchResourceAsJson(filePath))
yield extractLookupId(json)
def searchIssuesJson(lookupId: String): Future[json4s.JValue]
def extractIssues(json: org.json4s.JValue): Seq[Issue]
def findIssues(lookupId: String): Future[Seq[Issue]] =
for (json <- searchIssuesJson(componentId))
yield extractIssues(json)
def getFilePathsToProcess: List[String]
def thisIsCalledByJavaFramework(): java.util.Map[String, java.util.List[Issue]] = {
val finalResultPromise = Promise[Map[String, Seq[Issue]]]()
// (1) inferred type of issuesByFile not as expected, cannot get
// the type system happy, would like to have Seq[Future[(String, Seq[Issue])]]
val issuesByFile = getFilePathsToProcess map { f =>
findLookupId(f).flatMap { lookupId =>
(f, findIssues(lookupId)) // I want to yield a tuple (String, Seq[Issue]) here
}
}
Future.sequence(issuesByFile) onComplete {
case Success(x) => finalResultPromise.success(x) // (2) how to return x here?
case Failure(x) => // (3) how to return null from here?
}
//TODO transform finalResultPromise to Java Map
}
This code snippet has several issues. First, I'm not getting the type I would expect for issuesByFile (1). I would like to just ignore the result of findLookUpId if it is not able to find the lookUp ID (i.e., None). I've read in various tutorials that Future[Option[X]] is not easy to handle in function compositions and for expressions in Scala. So I'm also curious what the best practices are to handle these properly.
Second, I somehow have to wait for all futures to finish, but don't know how to return the result to the calling Java framework (2). Can I use a promise here to achieve this? If yes, how can I do it?
And last but not least, in case of any errors, I would just like to return null from thisIsCalledByJavaFramework but don't know how (3).
Any help is much appreciated.
Thanks,
Michael
Several points:
The first problem at (1) is that you don't handle the case where findLookupId returns None. You need to decide what to do in this case. Fail the whole process? Exclude that file from the list?
The second problem at (1) is that findIssues will itself return a Future, which you need to map before you can build the result tuple
There's a shortcut for map and then Future.sequence: Future.traverse
If you cannot change the result type of the method because the Java interface is fixed and cannot be changed to support Futures itself you must wait for the Future to be completed. Use Await.ready or Await.result to do that.
Taking all that into account and choosing to ignore files for which no id could be found results in this code:
// `None` in an entry for a file means that no id could be found
def entryForFile(file: String): Future[(String, Option[Seq[Issue]])] =
findLookupId(file).flatMap {
// the need for this kind of pattern match shows
// the difficulty of working with `Future[Option[T]]`
case Some(id) ⇒ findIssues(id).map(issues ⇒ file -> Some(issues))
case None ⇒ Future.successful(file -> None)
}
def thisIsCalledByJavaFramework(): java.util.Map[String, java.util.List[Issue]] = {
val issuesByFile: Future[Seq[(String, Option[Seq[Issue]])]] =
Future.traverse(getFilePathsToProcess)(entryForFile)
import scala.collection.JavaConverters._
try
Await.result(issuesByFile, 10.seconds)
.collect {
// here we choose to ignore entries where no id could be found
case (f, Some(issues)) ⇒ f -> issues
}
.toMap.mapValues(_.asJava).asJava
catch {
case NonFatal(_) ⇒ null
}
}
I'm developing a method that is supposed to persist an object, if it passes a list of conditions.
If any (or many) condition fail (or any other kind of error appears), a list with the errors should be returned, if everything goes well, a saved entity should be returned.
I was thinking about something like this (it's pseudocode, of course):
request.body.asJson.map { json =>
json.asOpt[Wine].map { wine =>
wine.save.map { wine =>
Ok(toJson(wine.update).toString)
}.getOrElse { errors => BadRequest(toJson(errors))}
}.getOrElse { BadRequest(toJson(Error("Invalid Wine entity")))}
}.getOrElse { BadRequest(toJson(Error("Expecting JSON data")))}
That is, I'd like to treat it like an Option[T], that if any validation fails, instead of returning None it gives me the list of errors...
The idea is to return an array of JSON errors...
So the question would be, is this the right way to handle these kind of situation? And what would be the way to accomplish it in Scala?
--
Oops, just posted the question and discovered Either
http://www.scala-lang.org/api/current/scala/Either.html
Anyway, I'd like to know what you think about the chosen approach, and if there's any other better alternative to handle it.
Using scalaz you have Validation[E, A], which is like Either[E, A] but has the property that if E is a semigroup (meaning things that can be concatenated, like lists) than multiple validated results can be combined in a way that keeps all the errors that occured.
Using Scala 2.10-M6 and Scalaz 7.0.0-M2 for example, where Scalaz has a custom Either[L, R] named \/[L, R] which is right-biased by default:
import scalaz._, Scalaz._
implicit class EitherPimp[E, A](val e: E \/ A) extends AnyVal {
def vnel: ValidationNEL[E, A] = e.validation.toValidationNEL
}
def parseInt(userInput: String): Throwable \/ Int = ???
def fetchTemperature: Throwable \/ Int = ???
def fetchTweets(count: Int): Throwable \/ List[String] = ???
val res = (fetchTemperature.vnel |#| fetchTweets(5).vnel) { case (temp, tweets) =>
s"In $temp degrees people tweet ${tweets.size}"
}
Here result is a Validation[NonEmptyList[Throwable], String], either containing all the errors occured (temp sensor error and/or twitter error or none) or the successful message. You can then switch back to \/ for convenience.
Note: The difference between Either and Validation is mainly that with Validation you can accumulate errors, but cannot flatMap to lose the accumulated errors, while with Either you can't (easily) accumulate but can flatMap (or in a for-comprehension) and possibly lose all but the first error message.
About error hierarchies
I think this might be of interest for you. Regardless of using scalaz/Either/\//Validation, I experienced that getting started was easy but going forward needs some additional work. The problem is, how do you collect errors from multiple erring functions in a meaningful way? Sure, you can just use Throwable or List[String] everywhere and have an easy time, but doesn't sound too much usable or interpretable. Imagine getting a list of errors like "child age missing" :: "IO error reading file" :: "division by zero".
So my choice is to create error hierarchies (using ADT-s), just like as one would wrap checked exceptions of Java into hierarchies. For example:
object errors {
object gamestart {
sealed trait Error
case class ResourceError(e: errors.resource.Error) extends Error
case class WordSourceError(e: errors.wordsource.Error) extends Error
}
object resource {
case class Error(e: GdxRuntimeException)
}
object wordsource {
case class Error(e: /*Ugly*/ Any)
}
}
Then when using result of erring functions with different error types, I join them under a relevant parent error type.
for {
wordSource <-
errors.gamestart.WordSourceError <-:
errors.wordsource.Error <-:
wordSourceCreator.doCreateWordSource(mtRandom).catchLeft.unsafePerformIO.toEither
resources <-
errors.gamestart.ResourceError <-:
GameViewResources(layout)
} yield ...
Here f <-: e maps the function f on the left of e: \/ since \/ is a Bifunctor. For se: scala.Either you might have se.left.map(f).
This may be further improved by providing shapeless HListIsos to be able to draw nice error trees.
Revisions
Updated: (e: \/).vnel lifts the failure side into a NonEmptyList so if we have a failure we have at least one error (was: or none).
If you have Option values, and you want to turn them into success/failure values, you can turn an Option into an Either using the toLeft or toRight method.
Usually a Right represents success, so use o.toRight("error message") to turn Some(value) into Right(value) and None into Left("error message").
Unfortunately Scala doesn't recognise this right-bias by default, so you have to jump through a hoop (by calling the .right method) in order to neatly compose your Eithers in a for-comprehension.
def requestBodyAsJson: Option[String] = Some("""{"foo":"bar"}""")
def jsonToWine(json: String): Option[Wine] = sys.error("TODO")
val wineOrError: Either[String, Wine] = for {
body <- requestBodyAsJson.toRight("Expecting JSON Data").right
wine <- jsonToWine(body).toRight("Invalid Wine entity").right
} yield wine
If you need an empty value, instead of using Either[A,Option[B]] you can use lift Box, which can have three values:
Full (there is a valid result)
Empty (no result, but no error either)
Failure (an error happened)
Box are more flexible than Either thanks to a rich API. Of course, although they were created for Lift, you can use them in any other framework.
well, this is my attemp using Either
def save() = CORSAction { request =>
request.body.asJson.map { json =>
json.asOpt[Wine].map { wine =>
wine.save.fold(
errors => JsonBadRequest(errors),
wine => Ok(toJson(wine).toString)
)
}.getOrElse (JsonBadRequest("Invalid Wine entity"))
}.getOrElse (JsonBadRequest("Expecting JSON data"))
}
And wine.save is like the following:
def save(wine: Wine): Either[List[Error],Wine] = {
val errors = validate(wine)
if (errors.length > 0) {
Left(errors)
} else {
DB.withConnection { implicit connection =>
val newId = SQL("""
insert into wine (
name, year, grapes, country, region, description, picture
) values (
{name}, {year}, {grapes}, {country}, {region}, {description}, {picture}
)"""
).on(
'name -> wine.name, 'year -> wine.year, 'grapes -> wine.grapes,
'country -> wine.country, 'region -> wine.region, 'description -> wine.description,
'picture -> wine.picture
).executeInsert()
val newWine = for {
id <- newId;
wine <- findById(id)
} yield wine
newWine.map { wine =>
Right(wine)
}.getOrElse {
Left(List(ValidationError("Could not create wine")))
}
}
}
}
Validate checks several preconditions. I still have to add a try/catch to catch any db error
I'm still looking for a way to improve the whole thing, it feels much to verbose to my taste...
Option monad is a great expressive way to deal with something-or-nothing things in Scala. But what if one needs to log a message when "nothing" occurs? According to the Scala API documentation,
The Either type is often used as an
alternative to scala.Option where Left
represents failure (by convention) and
Right is akin to Some.
However, I had no luck to find best practices using Either or good real-world examples involving Either for processing failures. Finally I've come up with the following code for my own project:
def logs: Array[String] = {
def props: Option[Map[String, Any]] = configAdmin.map{ ca =>
val config = ca.getConfiguration(PID, null)
config.properties getOrElse immutable.Map.empty
}
def checkType(any: Any): Option[Array[String]] = any match {
case a: Array[String] => Some(a)
case _ => None
}
def lookup: Either[(Symbol, String), Array[String]] =
for {val properties <- props.toRight('warning -> "ConfigurationAdmin service not bound").right
val logsParam <- properties.get("logs").toRight('debug -> "'logs' not defined in the configuration").right
val array <- checkType(logsParam).toRight('warning -> "unknown type of 'logs' confguration parameter").right}
yield array
lookup.fold(failure => { failure match {
case ('warning, msg) => log(LogService.WARNING, msg)
case ('debug, msg) => log(LogService.DEBUG, msg)
case _ =>
}; new Array[String](0) }, success => success)
}
(Please note this is a snippet from a real project, so it will not compile on its own)
I'd be grateful to know how you are using Either in your code and/or better ideas on refactoring the above code.
Either is used to return one of possible two meaningful results, unlike Option which is used to return a single meaningful result or nothing.
An easy to understand example is given below (circulated on the Scala mailing list a while back):
def throwableToLeft[T](block: => T): Either[java.lang.Throwable, T] =
try {
Right(block)
} catch {
case ex => Left(ex)
}
As the function name implies, if the execution of "block" is successful, it will return "Right(<result>)". Otherwise, if a Throwable is thrown, it will return "Left(<throwable>)". Use pattern matching to process the result:
var s = "hello"
throwableToLeft { s.toUpperCase } match {
case Right(s) => println(s)
case Left(e) => e.printStackTrace
}
// prints "HELLO"
s = null
throwableToLeft { s.toUpperCase } match {
case Right(s) => println(s)
case Left(e) => e.printStackTrace
}
// prints NullPointerException stack trace
Hope that helps.
Scalaz library has something alike Either named Validation. It is more idiomatic than Either for use as "get either a valid result or a failure".
Validation also allows to accumulate errors.
Edit: "alike" Either is complettly false, because Validation is an applicative functor, and scalaz Either, named \/ (pronounced "disjonction" or "either"), is a monad.
The fact that Validation can accumalate errors is because of that nature. On the other hand, / has a "stop early" nature, stopping at the first -\/ (read it "left", or "error") it encounters. There is a perfect explanation here: http://typelevel.org/blog/2014/02/21/error-handling.html
See: http://scalaz.googlecode.com/svn/continuous/latest/browse.sxr/scalaz/example/ExampleValidation.scala.html
As requested by the comment, copy/paste of the above link (some lines removed):
// Extracting success or failure values
val s: Validation[String, Int] = 1.success
val f: Validation[String, Int] = "error".fail
// It is recommended to use fold rather than pattern matching:
val result: String = s.fold(e => "got error: " + e, s => "got success: " + s.toString)
s match {
case Success(a) => "success"
case Failure(e) => "fail"
}
// Validation is a Monad, and can be used in for comprehensions.
val k1 = for {
i <- s
j <- s
} yield i + j
k1.toOption assert_≟ Some(2)
// The first failing sub-computation fails the entire computation.
val k2 = for {
i <- f
j <- f
} yield i + j
k2.fail.toOption assert_≟ Some("error")
// Validation is also an Applicative Functor, if the type of the error side of the validation is a Semigroup.
// A number of computations are tried. If the all success, a function can combine them into a Success. If any
// of them fails, the individual errors are accumulated.
// Use the NonEmptyList semigroup to accumulate errors using the Validation Applicative Functor.
val k4 = (fNel <**> fNel){ _ + _ }
k4.fail.toOption assert_≟ some(nel1("error", "error"))
The snippet you posted seems very contrived. You use Either in a situation where:
It's not enough to just know the data isn't available.
You need to return one of two distinct types.
Turning an exception into a Left is, indeed, a common use case. Over try/catch, it has the advantage of keeping the code together, which makes sense if the exception is an expected result. The most common way of handling Either is pattern matching:
result match {
case Right(res) => ...
case Left(res) => ...
}
Another interesting way of handling Either is when it appears in a collection. When doing a map over a collection, throwing an exception might not be viable, and you may want to return some information other than "not possible". Using an Either enables you to do that without overburdening the algorithm:
val list = (
library
\\ "books"
map (book =>
if (book \ "author" isEmpty)
Left(book)
else
Right((book \ "author" toList) map (_ text))
)
)
Here we get a list of all authors in the library, plus a list of books without an author. So we can then further process it accordingly:
val authorCount = (
(Map[String,Int]() /: (list filter (_ isRight) map (_.right.get)))
((map, author) => map + (author -> (map.getOrElse(author, 0) + 1)))
toList
)
val problemBooks = list flatMap (_.left.toSeq) // thanks to Azarov for this variation
So, basic Either usage goes like that. It's not a particularly useful class, but if it were you'd have seen it before. On the other hand, it's not useless either.
Cats has a nice way to create an Either from exception-throwing code:
val either: Either[NumberFormatException, Int] =
Either.catchOnly[NumberFormatException]("abc".toInt)
// either: Either[NumberFormatException,Int] = Left(java.lang.NumberFormatException: For input string: "abc")
in https://typelevel.org/cats/datatypes/either.html#working-with-exception-y-code