There're map/flatMap methods, there're also recover/recoverWith methods in the Scala Future standard API.
Why there's no collectWith ?
The code of the collect method is pretty simple :
def collect[S](pf: PartialFunction[T, S])(implicit executor: ExecutionContext): Future[S] =
map {
r => pf.applyOrElse(r, (t: T) => throw new NoSuchElementException("Future.collect partial function is not defined at: " + t))
}
The code of the collectWith method is then easy to imagine :
def collectWith[S](pf: PartialFunction[T, Future[S]])(implicit executor: ExecutionContext): Future[S] =
flatMap {
r => pf.applyOrElse(r, (t: T) => throw new NoSuchElementException("Future.collect partial function is not defined at: " + t))
}
I know that I can implement it and "extend" the Future standard API easily thanks to this article : http://debasishg.blogspot.fr/2008/02/why-i-like-scalas-lexically-scoped-open.html
I done that in my project :
class RichFuture[T](future: Future[T]) {
def collectWith[S](pf: PartialFunction[T, Future[S]])(implicit executor: ExecutionContext): Future[S] =
future.flatMap {
r => pf.applyOrElse(r, (t: T) => throw new NoSuchElementException("Future.collect partial function is not defined at: " + t))
}
}
trait WithRichFuture {
implicit def enrichFuture[T](person: Future[T]): RichFuture[T] = new RichFuture(person)
}
Maybe my needs for that does not justify to implement it in the standard API ?
Here is why I need this method in my Play2 project :
def createCar(key: String, eligibleCars: List[Car]): Future[Car] = {
def handleResponse: PartialFunction[WSResponse, Future[Car]] = {
case response: WSResponse if response.status == Status.CREATED => Future.successful(response.json.as[Car])
case response: WSResponse
if response.status == Status.BAD_REQUEST && response.json.as[Error].error == "not_the_good_one" =>
createCar(key, eligibleCars.tail)
}
// The "carApiClient.createCar" method just returns the result of the WS API call.
carApiClient.createCar(key, eligibleCars.head).collectWith(handleResponse)
}
I don't know how to do that without my collectWith method.
Maybe it's not the right way to do something like this ?
Do you know a better way ?
EDIT:
I have maybe a better solution for the createCar method that does not requires the collectWith method :
def createCar(key: String, eligibleCars: List[Car]): Future[Car] = {
for {
mayCar: Option[Car] <- Future.successful(eligibleCars.headOption)
r: WSResponse <- carApiClient.createCar(key, mayCar.get) if mayCar.nonEmpty
createdCar: Car <- Future.successful(r.json.as[Car]) if r.status == Status.CREATED
createdCar: Car <- createCar(key, eligibleCars.tail) if r.status == Status.BAD_REQUEST && r.json.as[Error].error == "not_the_good_one"
} yield createdCar
}
What do you think about this second solution ?
Second edit:
Just for information, here is my final solution thanks to #Dylan answer :
def createCar(key: String, eligibleCars: List[Car]): Future[Car] = {
def doCall(head: Car, tail: List[Car]): Future[Car] = {
carApiClient
.createCar(key, head)
.flatMap( response =>
response.status match {
case Status.CREATED => Future.successful(response.json.as[Car])
case Status.BAD_REQUEST if response.json.as[Error].error == "not_the_good_one" =>
createCar(key, tail)
}
)
}
eligibleCars match {
case head :: tail => doCall(head, tail)
case Nil => Future.failed(new RuntimeException)
}
}
Jules
How about:
def createCar(key: String, eligibleCars: List[Car]): Future[Car] = {
def handleResponse(response: WSResponse): Future[Car] = response.status match {
case Status.Created =>
Future.successful(response.json.as[Car])
case Status.BAD_REQUEST if response.json.as[Error].error == "not_the_good_one" =>
createCar(key, eligibleCars.tail)
case _ =>
// just fallback to a failed future rather than having a `collectWith`
Future.failed(new NoSuchElementException("your error here"))
}
// using flatMap since `handleResponse` is now a regular function
carApiClient.createCar(key, eligibleCars.head).flatMap(handleResponse)
}
Two changes:
handleResponse is no longer a partial function. The case _ returns a failed future, which is essentially what you were doing in your custom collectWith implementation.
use flatMap instead of collectWith, since handleResponse now suits that method signature
edit for extra info
If you really need to support the PartialFunction approach, you could always convert a PartialFunction[A, Future[B]] to a Function[A, Future[B]] by calling orElse on the partial function, e.g.
val handleResponsePF: PartialFunction[WSResponse, Future[Car]] = {
case ....
}
val handleResponse: Function[WSResponse, Future[Car]] = handleResponsePF orElse {
case _ => Future.failed(new NoSucheElementException("default case"))
}
Doing so would allow you to adapt an existing partial function to fit into a flatMap call.
(okay, technically, it already does, but you'd be throwing MatchErrors rather than your own custom exceptions)
Related
I'm not sure whether I chose the right title for my question..
I'm interested as to why the collection in the companion object is defined. Am I mistaken that this collection will have only one f in it? What I am seeing is a collection with exactly one element.
Here's the Future I'm dealing with:
trait Future[+T] { self =>
def onComplete(callback: Try[T] => Unit): Unit
def map[U](f: T => U) = new Future[U] {
def onComplete(callback: Try[U] => Unit) =
self onComplete (t => callback(t.map(f)))
}
def flatMap[U](f: T => Future[U]) = new Future[U] {
def onComplete(callback: Try[U] => Unit) =
self onComplete { _.map(f) match {
case Success(fu) => fu.onComplete(callback)
case Failure(e) => callback(Failure(e))
} }
}
def filter(p: T => Boolean) =
map { t => if (!p(t)) throw new NoSuchElementException; t }
}
Its companion object:
object Future {
def apply[T](f: => T) = {
val handlers = collection.mutable.Buffer.empty[Try[T] => Unit]
var result: Option[Try[T]] = None
val runnable = new Runnable {
def run = {
val r = Try(f)
handlers.synchronized {
result = Some(r)
handlers.foreach(_(r))
}
}
}
(new Thread(runnable)).start()
new Future[T] {
def onComplete(f: Try[T] => Unit) = handlers.synchronized {
result match {
case None => handlers += f
case Some(r) => f(r)
}
}
}
}
}
In my head I was imagining something like the following instead of the above companion object (notice how I replaced the above val handlers .. with var handler ..):
object Future {
def apply[T](f: => T) = {
var handler: Option[Try[T] => Unit] = None
var result: Option[Try[T]] = None
val runnable = new Runnable {
val execute_when_ready: Try[T] => Unit = r => handler match {
case None => execute_when_ready(r)
case Some(f) => f(r)
}
def run = {
val r = Try(f)
handler.synchronized {
result = Some(r)
execute_when_ready(r)
}
}
}
(new Thread(runnable)).start()
new Future[T] {
def onComplete(f: Try[T] => Unit) = handler.synchronized {
result match {
case None => handler = Some(f)
case Some(r) => f(r)
}
}
}
}
}
So why does the function execute_when_ready leads to stackoverflow, but that's not the case with handlers.foreach? what is the collection is offering me which I can't do without it? And is it possible to replace the collection with something else in the companion object?
The collection is not in the companion object, it is in the apply method, so there is a new instance for each Future. It is there because there can be multiple pending onComplete handlers on the same Future.
Your implementation only allows a single handler and silently removes any existing handler in onComplete which is a bad idea because the caller has no idea if a previous function has added an onComplete handler or not.
As noted in the comments, the stack overflow is because execute_when_ready calls itself if handler is None with no mechanism to stop the recursion.
Can different monads be used in for-comprehensions? Here's the code that uses map
case class Post(id: Int, text: String)
object PostOps {
def find(id: Int) : Option[Post] = if (id == 1) Some(Post(1, "text")) else None
def permitted(post: Post, userId: Int) : Try[Post] = if (userId == 1) Success(post) else Failure(new UnsupportedOperationException)
def edit(id: Int, userId : Int, text: String) = find(id).map(permitted(_, userId).map(_.copy(text = text))) match {
case None => println("Not found")
case Some(Success(p)) => println("Success")
case Some(Failure(_)) => println("Not authorized")
}
}
The straightforward version of for-comprehension doesn't work for obvious reasons, but is it possible to make it work with some additional code? I know it's possible in C# so it would be weird if it is not in Scala.
You can only use one type of monad in a for comprehension, since it is just syntactic sugar for flatMap and map.
If you have a stack of monads (eg Future[Option[A]]) you could use a monad transformer, but that does not apply here.
A solution for your case could be to use one monad : go from Option to Try or go from both Option and Try to Either[String, A].
def tryToEither[L, R](t: Try[R])(left: Throwable => L): Either[L, R] =
t.transform(r => Success(Right(r)), th => Success(Left(left(th)))).get
def edit(id: Int, userId: Int, text: String) = {
val updatedPost = for {
p1 <- find(id).toRight("Not found").right
p2 <- tryToEither(permitted(p1, userId))(_ => "Not Authorized").right
} yield p2.copy(text = text)
updatedPost match {
case Left(msg) => println(msg)
case Right(_) => println("success")
}
}
You could define an error type instead of using String, this way you can use Either[Error, A].
sealed trait Error extends Exception
case class PostNotFound(userId: Int) extends Error
case object NotAuthorized extends Error
I assume you mean the fact that you now have an Option[Try[Post]]
find(id).map(permitted(_, userId).map(_.copy(text = text))) match {
case None => println("Not found")
case Some(Success(p)) => println("Success")
case Some(Failure(_)) => println("Not authorized")
}
Could be done as a for a few ways.
Nesting fors:
for {
post <- find(id)
} yield {
for {
tryOfPost <- permitted(post, userId)
} yield {
tryOfPost.copy(text = text)
}
}
Convert Option to a Try so you're using a single type, this has the disadvantage of losing the difference between an error in the Try and a None from the Option. credit here for how to go from Option to Try.
for {
post <- find(id).fold[Try[Post]](Failure[Post](new OtherException))(Success(_))
permittedPost <- permitted(post, userId)
} yield {
permittedPost.copy(text = text)
}
You could also look into the OptionT monad transformer in scalaz to create a type which is an OptionTTry.
Fundamentally, though, Monads don't compose this way, at least not generically.
In Scala we have a by-name-parameters where we can write
def foo[T](f: => T):T = {
f // invokes f
}
// use as:
foo(println("hello"))
I now want to do the same with an array of methods, that is I want to use them as:
def foo[T](f:Array[ => T]):T = { // does not work
f(0) // invokes f(0) // does not work
}
foo(println("hi"), println("hello")) // does not work
Is there any way to do what I want? The best I have come up with is:
def foo[T](f:() => T *):T = {
f(0)() // invokes f(0)
}
// use as:
foo(() => println("hi"), () => println("hello"))
or
def foo[T](f:Array[() => T]):T = {
f(0)() // invokes f(0)
}
// use as:
foo(Array(() => println("hi"), () => println("hello")))
EDIT: The proposed SIP-24 is not very useful as pointed out by Seth Tisue in a comment to this answer.
An example where this will be problematic is the following code of a utility function trycatch:
type unitToT[T] = ()=>T
def trycatch[T](list:unitToT[T] *):T = list.size match {
case i if i > 1 =>
try list.head()
catch { case t:Any => trycatch(list.tail: _*) }
case 1 => list(0)()
case _ => throw new Exception("call list must be non-empty")
}
Here trycatch takes a list of methods of type ()=>T and applies each element successively until it succeeds or the end is reached.
Now suppose I have two methods:
def getYahooRate(currencyA:String, currencyB:String):Double = ???
and
def getGoogleRate(currencyA:String, currencyB:String):Double = ???
that convert one unit of currencyA to currencyB and output Double.
I use trycatch as:
val usdEuroRate = trycatch(() => getYahooRate("USD", "EUR"),
() => getGoogleRate("USD", "EUR"))
I would have preferred:
val usdEuroRate = trycatch(getYahooRate("USD", "EUR"),
getGoogleRate("USD", "EUR")) // does not work
In the example above, I would like getGoogleRate("USD", "EUR") to be invoked only if getYahooRate("USD", "EUR") throws an exception. This is not the intended behavior of SIP-24.
Here is a solution, although with a few restrictions compared to direct call-by-name:
import scala.util.control.NonFatal
object Main extends App {
implicit class Attempt[+A](f: => A) {
def apply(): A = f
}
def tryCatch[T](attempts: Attempt[T]*): T = attempts.toList match {
case a :: b :: rest =>
try a()
catch {
case NonFatal(e) =>
tryCatch(b :: rest: _*)
}
case a :: Nil =>
a()
case Nil => throw new Exception("call list must be non-empty")
}
def a = println("Hi")
def b: Unit = sys.error("one")
def c = println("bye")
tryCatch(a, b, c)
def d: Int = sys.error("two")
def e = { println("here"); 45 }
def f = println("not here")
val result = tryCatch(d, e, f)
println("Result is " + result)
}
The restrictions are:
Using a block as an argument won't work; only the last expression of the block will be wrapped in an Attempt.
If the expression is of type Nothing (e.g., if b and d weren't annotated), the conversion to Attempt is not inserted since Nothing is a subtype of every type, including Attempt. Presumably the same would apply for an expression of type Null.
As of Scala 2.11.7, the answer is no. However, there is SIP-24, so in some future version your f: => T* version may be possible.
I have this code that I use in a spray handler
get {
def callService = {
val p = Promise[Option[DocumentRef]]
val fut = p.future
archiveService.getByHash(ZeroHash, {
result => p success result
})
fut
}
onComplete(OnCompleteFutureMagnet(callService)){
case Success(docRef) => {
val doc = docRef map {
x => x.title
} getOrElse "nothing"
complete("Done with " + doc)
}
case Failure(ex) => complete("error ${ex.getMessage}")
}
}
so I had the bright idea of writing the following function to encapsulate the work done to create a future out of a promise:
def callback2Future[T](funToCall: (T => Unit) => Any): Future[T] = {
val p = Promise[T]
val resultFuture = p future
def callbacklistener(arg: T): Unit = {
arg: T => p success arg
}
funToCall(callbacklistener)
resultFuture
}
And restructure the onComplete as:
onComplete(OnCompleteFutureMagnet(callback2Future(archiveService.getByHash(ZeroHash, _: Option[DocumentRef] => Unit)))) {
case Success(docRef) => {
...
}
In the original implementation with callservice, it works great (with great throughput too), with the callback2Future implementation I get a forever wait and it eventually times out. They seem the same to me, can anyone spot the error?
I believe that your problem is due to the infamous auto-Unit feature of Scala. Your function:
def callbacklistener(arg: T): Unit = {
arg: T => p success arg
}
will probably be interpreted as:
def callbacklistener(arg: T): Unit = {
{ arg: T => p success arg }
()
}
What you really want is probably:
def callbacklistener(arg: T): Unit = p success arg
To be clear, in your implementation you are defining a function callbackListener with return type Unit; in the body of this function you have an expression, { arg: T => p success arg }, whose value is of type T => Unit and is discarded; the Scala compiler will then put a free () in your code as the return type of the callbackListener is supposed to be Unit.
Is there a way to turn a Seq[Future[X]] into an Enumerator[X] ? The use case is that I want to get resources by crawling the web. This is going to return a Sequence of Futures, and I'd like to return an Enumerator that will push the futures in the order in which they are first finished on to the Iteratee.
It looks like Victor Klang's Future select gist could be used to do this - though it looks pretty inefficient.
Note: The Iteratees and Enumerator's in question are those given by the play framework version 2.x, ie with the following imports: import play.api.libs.iteratee._
Using Victor Klang's select method:
/**
* "Select" off the first future to be satisfied. Return this as a
* result, with the remainder of the Futures as a sequence.
*
* #param fs a scala.collection.Seq
*/
def select[A](fs: Seq[Future[A]])(implicit ec: ExecutionContext):
Future[(Try[A], Seq[Future[A]])] = {
#scala.annotation.tailrec
def stripe(p: Promise[(Try[A], Seq[Future[A]])],
heads: Seq[Future[A]],
elem: Future[A],
tail: Seq[Future[A]]): Future[(Try[A], Seq[Future[A]])] = {
elem onComplete { res => if (!p.isCompleted) p.trySuccess((res, heads ++ tail)) }
if (tail.isEmpty) p.future
else stripe(p, heads :+ elem, tail.head, tail.tail)
}
if (fs.isEmpty) Future.failed(new IllegalArgumentException("empty future list!"))
else stripe(Promise(), fs.genericBuilder[Future[A]].result, fs.head, fs.tail)
}
}
I can then get what I need with
Enumerator.unfoldM(initialSeqOfFutureAs){ seqOfFutureAs =>
if (seqOfFutureAs.isEmpty) {
Future(None)
} else {
FutureUtil.select(seqOfFutureAs).map {
case (t, seqFuture) => t.toOption.map {
a => (seqFuture, a)
}
}
}
}
A better, shorter and I think more efficient answer is:
def toEnumerator(seqFutureX: Seq[Future[X]]) = new Enumerator[X] {
def apply[A](i: Iteratee[X, A]): Future[Iteratee[X, A]] = {
Future.sequence(seqFutureX).flatMap { seqX: Seq[X] =>
seqX.foldLeft(Future.successful(i)) {
case (i, x) => i.flatMap(_.feed(Input.El(x)))
}
}
}
}
I do realise that the question is a bit old already, but based on Santhosh's answer and the built-in Enumterator.enumerate() implementation I came up with the following:
def enumerateM[E](traversable: TraversableOnce[Future[E]])(implicit ec: ExecutionContext): Enumerator[E] = {
val it = traversable.toIterator
Enumerator.generateM {
if (it.hasNext) {
val next: Future[E] = it.next()
next map {
e => Some(e)
}
} else {
Future.successful[Option[E]] {
None
}
}
}
}
Note that unlike the first Viktor-select-based-solution this one preserves the order, but you can still start off all computations asynchronously before. So, for example, you can do the following:
// For lack of a better name
def mapEachM[E, NE](eventuallyList: Future[List[E]])(f: E => Future[NE])(implicit ec: ExecutionContext): Enumerator[NE] =
Enumerator.flatten(
eventuallyList map { list =>
enumerateM(list map f)
}
)
This latter method was in fact what I was looking for when I stumbled on this thread. Hope it helps someone! :)
You could construct one using the Java Executor Completeion Service (JavaDoc). The idea is to use create a sequence of new futures, each using ExecutorCompletionService.take() to wait for the next result. Each future will start, when the previous future has its result.
But please b e aware, that this might be not that efficient, because a lot of synchronisation is happening behind the scenes. It might be more efficient, to use some parallel map reduce for calculation (e.g. using Scala's ParSeq) and let the Enumerator wait for the complete result.
WARNING: Not compiled before answering
What about something like this:
def toEnumerator(seqFutureX: Seq[Future[X]]) = new Enumerator[X] {
def apply[A](i: Iteratee[X, A]): Future[Iteratee[X, A]] =
Future.fold(seqFutureX)(i){ case (i, x) => i.flatMap(_.feed(Input.El(x)))) }
}
Here is something I found handy,
def unfold[A,B](xs:Seq[A])(proc:A => Future[B])(implicit errorHandler:Throwable => B):Enumerator[B] = {
Enumerator.unfoldM (xs) { xs =>
if (xs.isEmpty) Future(None)
else proc(xs.head) map (b => Some(xs.tail,b)) recover {
case e => Some((xs.tail,errorHandler(e)))
}
}
}
def unfold[A,B](fxs:Future[Seq[A]])(proc:A => Future[B]) (implicit errorHandler1:Throwable => Seq[A], errorHandler:Throwable => B) :Enumerator[B] = {
(unfold(Seq(fxs))(fxs => fxs)(errorHandler1)).flatMap(unfold(_)(proc)(errorHandler))
}
def unfoldFutures[A,B](xsfxs:Seq[Future[Seq[A]]])(proc:A => Future[B]) (implicit errorHandler1:Throwable => Seq[A], errorHandler:Throwable => B) :Enumerator[B] = {
xsfxs.map(unfold(_)(proc)).reduceLeft((a,b) => a.andThen(b))
}
I would like to propose the use of a Broadcast
def seqToEnumerator[A](futuresA: Seq[Future[A]])(defaultValue: A, errorHandler: Throwable => A): Enumerator[A] ={
val (enumerator, channel) = Concurrent.broadcast[A]
futuresA.foreach(f => f.onComplete({
case Success(Some(a: A)) => channel.push(a)
case Success(None) => channel.push(defaultValue)
case Failure(exception) => channel.push(errorHandler(exception))
}))
enumerator
}
I added errorHandling and defaultValues but you can skip those by using onSuccess or onFailure, instead of onComplete