I'm getting an error because inside the for-comprehension I have EitherTs instead of Eithers. I've looked for ways to convert and EitherT to an Either but Scala searches yield very few good answers.
val res: Either[ClientError, Response] = stuff.proc match {
case Client1 => client1.doTheThing(sid) // returns Either[ClientError, Response]
case Client2 =>
for {
x <- getIt(a) // returns EitherT[Future, ClientError, Response]
y <- getTheOtherThing(b) // returns EitherT[Future, ClientError, Response]
} yield {
log.info(s"$x and $y")
makeItHappen(x, y) // returns Either[ClientError, Response]
}
}
Scala searches yield very few good answers.
This is a strong hint taken approach should be reconsidered.
Thechnically you could call value on EitherT[Future, ClientError, Response] to get back to Future[Either[ClientError, Response] but now you have to block to get out Either[ClientError, Response], so usually we do not do that. More idiomatic approach is to continue working within appropriate monadic context, however the problem is you cannot mix and match them. Instead you will likely have to lift all the other functions to the “highest” monadic context with something like EitherT(Future.successful(anEither)) so that all monads align.
Related
In my current project, I used Either[Result, HandbookModule] (Result is an HTTP Statuscode) as a return type so that I can create the correct status when something goes wrong. I have now refactored my database access to be non-blocking.
This change requires that my return type for database access functions changed to Future[Either[Result, HandbookModule]].
Now I am not sure on how to glue this function together with another function which returns Either[Result, Long].
So to better illustrate what I mean:
def moduleDao.getHandbooks(offset, limit): Future[Either[Result, List[Module]] = Future(Right(List(Module(1))))
def nextOffset(offset, limit, results): Either[_, Long] = Right(1)
def getHandbooks(
offset: Long,
limit: Long): Future[Either[Result, (List[HandbookModule], Long)]] = {
for {
results <- moduleDao.getHandbooks(offset, limit)
offset <- nextOffset(offset, limit, results)
} yield (results, offset)
}
Before the change, this was obviously no problem but I don't know what would be the best approach.
Or is there a way to convert a Future[Either[A, B]] to an Either[A, Future[B]]?
In order to unwrap your method from the Future, you'd have to block it and wait for the result. You could do it using Await.result.
But blocking the future is usually not considered to be the best practice. More on this here.
So you should tackle that problem differently. What you're facing is actually common issue with nested monad stacks and it can be handled with monad transformers.
Scala's functional programming library cats provides an implementation of EitherT monad transformer.
In your case, you could use EitherT.apply to transform Future[Either[Result, List[Module]] into EitherT[Future, Result, List[Module]] and EitherT.fromEither to lift Either[_, Long].
It could look like this:
import cats.data.EitherT
import cats.implicits._
def getHandbooks(
offset: Long,
limit: Long
): Future[Either[String, (List[String], Long)]] = {
val result: EitherT[Future, String, (List[String], Long)] = for {
results <- EitherT(moduleDao.getHandbooks(offset, limit))
offset <- EitherT.fromEither[Future](nextOffset(offset, limit, results))
} yield (results, offset)
result.value //unwrap result from EitherT
}
I had to make a number of assumptions and adjustments/corrections to the posted code in order to make it usable. (You don't make it easy for those who want to help you.)
If you can tolerate a default Long value, when nextOffset() returns Left instead of Right[Long], then this appears to type-check and compile.
def getHandbooks(offset: Long
,limit : Long
): Future[Either[Result, (List[Module], Long)]] =
moduleDao.getHandbooks(offset,limit).map(_.map(ms =>
(ms, nextOffset(offset,limit,ms).getOrElse(0L))))
If I understand correctly, the Left side of you Either represents the error state, correct?
If so, I think you should refactor your API to not use Either, and simply use a failed Future to represent the error state. Something along the lines of:
// Custom exception that wraps existing Result
case class MyCustomException(result: Result) extends Exception
class ModuleDao {
...
def getHandbooks(offset, limit): Future[List[Module] = {
// You'd probably want to do this asynchronously
// But for demonstration purposes
val origRetVal: Either[Result, List[Module] = ??? // current code returning your Either
origRetVal match {
case Right(modules: List[Module]) =>
Future.successful(modules)
case Left(result: Result) =>
// Failed future wrapping custom exception
Future.failed(MyCustomException(result))
}
}
...
}
I have a sequence of parameters. For each parameter I have to perform DB query, which may or may not return a result. Simply speaking, I need to stop after the first result is non-empty. Of course, I would like to avoid doing unnecessary calls. The caveat is - I need to have this operation(s) contained as a another Future - or any "most reactive" approach.
Speaking of code:
//that what I have
def dbQuery(p:Param): Future[Option[Result]] = {}
//my list of params
val input = Seq(p1,p2,p3)
//that what I need to implements
def getFirstNonEmpty(params:Seq[Param]): Future[Option[Result]]
I know I can possibly just wrap entire function in yet another Future and execute code sequentially (Await? Brrr...), but that not the cleanest solution.
Can I somehow create lazy initialized collection of futures, like
params.map ( p => FutureWhichWontStartUnlessAskedWhichWrapsOtherFuture { dbQuery(p) }).findFirst(!_.isEmpty())
I believe it's possible!
What do you think about something like this?
def getFirstNonEmpty(params: Seq[Param]): Future[Option[Result]] = {
params.foldLeft(Future.successful(Option.empty[Result])) { (accuFtrOpt, param) =>
accuFtrOpt.flatMap {
case None => dbQuery(param)
case result => Future.successful(result)
}
}
}
This might be overkill, but if you are open to using scalaz we can do this using OptionT and foldMap.
With OptionT we sort of combine Future and Option into one structure. We can get the first of two Futures with a non-empty result using OptionT.orElse.
import scalaz._, Scalaz._
import scala.concurrent.Future
import scala.concurrent.ExecutionContext.Implicits.global
val someF: Future[Option[Int]] = Future.successful(Some(1))
val noneF: Future[Option[Int]] = Future.successful(None)
val first = OptionT(noneF) orElse OptionT(someF)
first.run // Future[Option[Int]] = Success(Some(1))
We could now get the first non-empty Future from a List with reduce from the standard library (this will however run all the Futures) :
List(noneF, noneF, someF).map(OptionT.apply).reduce(_ orElse _).run
But with a List (or other collection) we can't be sure that there is at least one element, so we need to use fold and pass a start value. Scalaz can do this work for us by using a Monoid. The Monoid[OptionT[Future, Int]] we will use will supply the start value and combine the Futures with the orElse used above.
type Param = Int
type Result = Int
type FutureO[x] = OptionT[Future, x]
def query(p: Param): Future[Option[Result]] =
Future.successful{ println(p); if (p > 2) Some(p) else None }
def getFirstNonEmpty(params: List[Param]): Future[Option[Result]] = {
implicit val monoid = PlusEmpty[FutureO].monoid[Result]
params.foldMap(p => OptionT(query(p))).run
}
val result = getFirstNonEmpty(List(1,2,3,4))
// prints 1, 2, 3
result.foreach(println) // Some(3)
This is an old question, but if someone comes looking for an answer, here is my take. I solved it for a use case that required me to loop through a limited number of futures sequentially and stop when the first of them returned a result.
I did not need a library for my use-case, a light-weight combination of recursion and pattern matching was sufficient. Although the question here does not have the same problem as a sequence of futures, looping through a sequence of parameters would be similar.
Here would be the pseudo-code based on recursion.
I have not compiled this, fix the types being matched/returned.
def getFirstNonEmpty(params: Seq[Param]): Future[Option[Result]] = {
if (params.isEmpty) {
Future.successful(None)
} else {
val head = params.head
dbQuery(head) match {
case Some(v) => Future.successful(Some(v))
case None => getFirstNonEmpty(params.tail)
}
}
}
I have the following snippet in Scala using Scalaz and Task/EitherT:
def myMethod(request: MyRequest) : EitherT[Task, Failure, Success] =
EitherT[Task, Failure, Success](
for {
aamOpt <- async.findUser(request)
aamEith <- Task.now(aamOpt.\/>(NoUserFound()))
result <- aamEith.fold(err => Task.now(-\/(err)), aam => process(aam)).run)
} yield result)
Where async.findUser returns a Task[Option[MyUser]] and process(aam) returns EitherT[Task, Failure, Success]. These return types are needed as these methods interact (down the line) with external services.
Is there any way to simplify the last line of the for comprehension into something nicer? I mean this one:
result <- aamEith.fold(err => Task.now(-\/(err)), aam => process(aam)).run)
I would probably immediately lift the result from async.findUser into the relevant EitherT monad and then just flatMap:
def myMethod(request: MyRequest): EitherT[Task, Failure, Success] =
EitherT.eitherT[Task, Failure, MyUser](
async.findUser(request).map(_.\/>(NoUserFound()))
).flatMap(process)
This is untested but something like it should more or less work.
I have to get a list of issues for each file of a given list from a REST API with Scala. I want to do the requests in parallel, and use the Dispatch library for this. My method is called from a Java framework and I have to wait at the end of this method for the result of all the futures to yield the overall result back to the framework. Here's my code:
def fetchResourceAsJson(filePath: String): dispatch.Future[json4s.JValue]
def extractLookupId(json: org.json4s.JValue): Option[String]
def findLookupId(filePath: String): Future[Option[String]] =
for (json <- fetchResourceAsJson(filePath))
yield extractLookupId(json)
def searchIssuesJson(lookupId: String): Future[json4s.JValue]
def extractIssues(json: org.json4s.JValue): Seq[Issue]
def findIssues(lookupId: String): Future[Seq[Issue]] =
for (json <- searchIssuesJson(componentId))
yield extractIssues(json)
def getFilePathsToProcess: List[String]
def thisIsCalledByJavaFramework(): java.util.Map[String, java.util.List[Issue]] = {
val finalResultPromise = Promise[Map[String, Seq[Issue]]]()
// (1) inferred type of issuesByFile not as expected, cannot get
// the type system happy, would like to have Seq[Future[(String, Seq[Issue])]]
val issuesByFile = getFilePathsToProcess map { f =>
findLookupId(f).flatMap { lookupId =>
(f, findIssues(lookupId)) // I want to yield a tuple (String, Seq[Issue]) here
}
}
Future.sequence(issuesByFile) onComplete {
case Success(x) => finalResultPromise.success(x) // (2) how to return x here?
case Failure(x) => // (3) how to return null from here?
}
//TODO transform finalResultPromise to Java Map
}
This code snippet has several issues. First, I'm not getting the type I would expect for issuesByFile (1). I would like to just ignore the result of findLookUpId if it is not able to find the lookUp ID (i.e., None). I've read in various tutorials that Future[Option[X]] is not easy to handle in function compositions and for expressions in Scala. So I'm also curious what the best practices are to handle these properly.
Second, I somehow have to wait for all futures to finish, but don't know how to return the result to the calling Java framework (2). Can I use a promise here to achieve this? If yes, how can I do it?
And last but not least, in case of any errors, I would just like to return null from thisIsCalledByJavaFramework but don't know how (3).
Any help is much appreciated.
Thanks,
Michael
Several points:
The first problem at (1) is that you don't handle the case where findLookupId returns None. You need to decide what to do in this case. Fail the whole process? Exclude that file from the list?
The second problem at (1) is that findIssues will itself return a Future, which you need to map before you can build the result tuple
There's a shortcut for map and then Future.sequence: Future.traverse
If you cannot change the result type of the method because the Java interface is fixed and cannot be changed to support Futures itself you must wait for the Future to be completed. Use Await.ready or Await.result to do that.
Taking all that into account and choosing to ignore files for which no id could be found results in this code:
// `None` in an entry for a file means that no id could be found
def entryForFile(file: String): Future[(String, Option[Seq[Issue]])] =
findLookupId(file).flatMap {
// the need for this kind of pattern match shows
// the difficulty of working with `Future[Option[T]]`
case Some(id) ⇒ findIssues(id).map(issues ⇒ file -> Some(issues))
case None ⇒ Future.successful(file -> None)
}
def thisIsCalledByJavaFramework(): java.util.Map[String, java.util.List[Issue]] = {
val issuesByFile: Future[Seq[(String, Option[Seq[Issue]])]] =
Future.traverse(getFilePathsToProcess)(entryForFile)
import scala.collection.JavaConverters._
try
Await.result(issuesByFile, 10.seconds)
.collect {
// here we choose to ignore entries where no id could be found
case (f, Some(issues)) ⇒ f -> issues
}
.toMap.mapValues(_.asJava).asJava
catch {
case NonFatal(_) ⇒ null
}
}
I have posted quite several questions about failure handling in Scala and I really thank you all for your answers.
I understand my options when dealing with Either and Scalaz or a for comprehension, and I have another (last?) question:
How to do a fail-fast sequence of operations when the operations are dealing with the outside non-functional world, like a DB?
I mean I have a method like that:
def insertItem(item: Item): Either[Error,Item]
Thanks to Either and these answers, I know how to do it with Either: Chaining method calls with Either and
Method parameters validation in Scala, with for comprehension and monads
But my Item case class is immutable and it doesn't really make sense to return it as a Right since the caller already has the value.
Thus how can I do the same kind of thing with:
def insertItem(item: Item): Option[Error]
In my application, when an user is created, we also create some items for him.
If an item fails to create, then the whole process should stop.
When I use directly Option[Error] in a for comprehension, I don't think I'll get the result I expect.
I guess it makes sense to do something like that:
for {
_ <- insertItem(item1).toLeft("???").right
_ <- insertItem(item2).toLeft("???").right
_ <- insertItem(item3).toLeft("???").right
}
But as the values "???" I put in my Right are never useful, I guess I'm missing the elegant solution which do not involve creating Rights that will never be used.
I think I'm looking for something that will continue the for comprehension only when the result is None, which is kind of weird because I just want to continue to the next operation, and not do a real map operation.
By the way, if possible, I would like both non-Scalaz and Scalaz solutions.
I'm not sure Scalaz handle such things, because it seems more focused on real functional programming and perhaps do not provide code for side-effect behaviors like my use case?
I don't see a principled reason not to use Either[Error, Unit] in a case like this (or at least I've done it, and I don't feel guilty about it). Say we have the following:
def launch(thing: String): Either[String, Unit] = Either.cond(
thing.nonEmpty,
println("Launching the " + thing),
"Need something to launch!"
)
We can show that the right projection monad is appropriately lazy:
scala> for {
| _ <- launch("flowers").right
| _ <- launch("").right
| r <- launch("MISSILES").right
| } yield r
Launching the flowers
res1: Either[String,Unit] = Left(Need something to launch!)
No missiles get launched, as desired.
It's worth noting that if you use Option instead of Either, the operation you're describing is just the sum given the "First" monoid instance for Option (where the addition operation is just orElse). For example, we can write the following with Scalaz 7:
import scalaz._, Scalaz._
def fst[A](x: Option[A]): Option[A] ## Tags.First = Tag(x)
def launch(thing: String): Option[String] = if (thing.isEmpty) Some(
"Need something to launch!"
) else {
println("Launching the " + thing)
None
}
And now:
scala> val r: Option[String] = fst(launch("flowers")) |+| fst(
| launch("")) |+| fst(launch("MISSILES"))
Launching the flowers
r: Option[String] = Some(Need something to launch!)
Once again, no missiles.
We do something similar where I work with scalaz to fail fast with message sending, though this is not idiomatic scalaz:
def insertItem(item: Item): Validation[Error, Unit]
val result = for {
_ <- insertItem(item1)
_ <- insertItem(item2)
_ <- insertItem(item3)
} yield Unit
If you want to chain together methods that contain Option[Error] and only execute the next step if the Option is None, you can use orElse for that.
If you are only interested in the first failure, than an Iterator can be your choice:
case class Item(i: Int)
case class Error(i: Item)
val items = Iterator(1,3,7,-5,-2,0) map Item
def insertItem(item: Item): Option[Error] =
if (item.i < 0) Some(Error(item)) else None
scala> val error = (items map insertItem find (_.isDefined)).flatten
error: Option[Error] = Some(Error(Item(-5)))
If insertItem is not your only method it is possible to call them separated with your values:
val items = Iterator(
() => insertItem(i1),
() => deleteItem(i2),
() => insertItem(i3),
() => updateItem(i4))