I have been looking at this How are Scala Futures chained together with flatMap? and the corresponding article as well on translating for comprehension. I am slowly adding stuff to my for comprehension and am stuck as I guess the code I thought would translate to is not correct.
Here I have a runProgram and runProgram2 which I thought would be equivalent and are not because runProgram2 does not compile. Can someone explain the equiavalent of this for comprehension...
NOTE: yes I know that future.flatMap is typically for collapsing Future[Future[String]] but this is a trimmed down version of my file(perhaps I trimmed it down too far).
def main(args: Array[String]) = {
val future1: Future[String] = runMyProgram()
//val future2: Future[String] = runMyProgram2()
}
def runMyProgram() : Future[String] = {
val future = serviceCall()
future.flatMap(processAllReturnCodes)
}
// def runMyProgram2() : Future[String] = {
// val future = serviceCall()
// for {
// result <- future
// } yield processAllReturnCodes(result)
// }
def processAllReturnCodes(count: Int) : Future[String] = {
val promise = Promise.successful("done")
promise.future
}
def serviceCall() : Future[Int] = {
val promise = Promise.successful(5)
promise.future
}
def serviceCall2() : Future[String] = {
val promise = Promise.successful("hithere")
promise.future
}
This for comprehension:
for {
result <- future
} yield processAllReturnCodes(result)
Is being translated to this:
val t: Future[Future[String]] = future.map(result => processAllReturnCodes(result))
Comprehension is really only syntactic sugar for map and flatMap, with flatMap you can flatten the future nesting:
val u: Future[String] = future.flatMap(result => processAllReturnCodes(result))
The difference lies in the signatures:
def map[S](f: T => S)(implicit executor: ExecutionContext): Future[S]
So map takes a function form T to S and wraps the S in a future, the problem is that here your S is a Future[String] which is wrapped in another future giving Future[Future[String]], flatMap instead:
def flatMap[S](f: T => Future[S])(implicit executor: ExecutionContext): Future[S]
Takes a function from T to Future[S] and returns that future, in your case your method already returns a future and it's a valid parameter for flatMap.
Related
I'm writing tests for function bar:
def bar(fut1: Future[Int],
fut2: Future[Int],
fut3: Future[Int]): Future[Result] = ???
bar returns Result like this:
case class Result(
x: Int, // fut1 value
oy: Option[Int], // if fut2 is complete then Some of fut2 value else None
oz: Option[Int] // if fut3 is complete then Some of fut3 value else None
)
I want to write tests for all test cases:
fut1 completed, fut2 and fut3 did not complete
fut1 completed, fut2 completed, fut3 did not complete
etc.
So I am writing a fake implementation of functions foo1, foo2, and foo3 for these tests.
def foo1(x: Int): Future[Int] = ???
def foo2(x: Int): Future[Int] = ???
def foo3(x: Int): Future[Int] = ???
Test #1 invokes all these functions, checks if fut1 completes first, and invokes bar
val fut1 = foo1(0)
val fut2 = foo2(0)
val fut3 = foo3(0)
// make sure `fut1` completes first
Test #2 invokes all these functions, makes sure that fut2 completes first, and invokes bar.
Test #3 invokes all these functions, makes sure that fut3 completes first, and invokes bar.
My question is how to implement the functions foo1, foo2, and foo3 and the tests.
You can try attach completeness timestamp to each future via map, like:
import scala.concurrent.ExecutionContext.Implicits.global
import scala.concurrent._
import scala.concurrent.duration._
import scala.language.postfixOps
def foo1(x: Int): Future[Int] = Future {Thread.sleep(200); 1}
def foo2(x: Int): Future[Int] = Future {Thread.sleep(500); 2}
def foo3(x: Int): Future[Int] = Future {Thread.sleep(500); 3}
def completeTs[T](future: Future[T]): Future[(T, Long)] = future.map(v => v -> System.currentTimeMillis())
val resutls = Await.result(Future.sequence(
List(completeTs(foo1(1)), completeTs(foo2(1)), completeTs(foo3(1)))
), 2 seconds)
val firstCompleteTs = resutls.map(_._2).min
val firstCompleteIndex = resutls.indexWhere(_._2 == firstCompleteTs)
assert(firstCompleteIndex == 0)
Scatie: https://scastie.scala-lang.org/L9g78DSNQIm2K1jGlQzXBg
You could repurpose firstCompletedOf to verify whether the future of a given index in the futures list is the first completed one:
import java.util.concurrent.atomic.AtomicReference
import scala.concurrent.{ExecutionContext, Future, Promise}
import scala.util.Try
def isFirstCompleted[T](idx: Int)(futures: List[Future[T]])(
implicit ec: ExecutionContext): Future[Boolean] = {
val promise = Promise[(T, Int)]()
val pRef = new AtomicReference[Promise[(T, Int)]](promise)
futures.zipWithIndex foreach { case (f, i) => f onComplete { case tt: Try[T] =>
val p = pRef.getAndSet(null)
if (p != null) p tryComplete tt.map((_, i))
}
}
promise.future.map{ case (t, i) => i == idx }
}
Test running:
import scala.concurrent.ExecutionContext.Implicits.global
val futures = List(
Future{Thread.sleep(100); 1},
Future{Thread.sleep(150); throw new Exception("oops!")},
Future{Thread.sleep(50); 3}
)
isFirstCompleted(0)(futures) // Future(Success(false))
isFirstCompleted(2)(futures) // Future(Success(true))
For writing test cases, consider using ScalaTest AsyncFlatSpec.
It is unclear what it is exactly you are trying to test.
If you simply use futures that have already completed, you will get the behavior you describe:
def f1 = Future.successful(1)
def f2 = Future.successful(2)
def f3 = Future.successful(3)
eventually {
Future.firstCompletedOf(Seq(f1, f2, f3)).value shouldBe Some(1)
}
(note, that you cannot compare directly with fut1 like you did in the question, that'll always be false, because .firstCompletedOf returns a new future).
You can also make only one future complete, and leave the others alone:
val promise = Promise[Int].future
def f1 = promise.future // or just Future.successful(1) ... or Future(1)
def f2 = Future.never
def f3 = Future.never
result = Future.firstCompletedOf(Seq(f1, f2, f3))
promise.complete(Success(1))
eventually {
result.value shouldBe 1
}
Etc ... Can make the other futures be backed by their own promise too for example, if you want them all to complete eventually (not sure what it'll gain you, but then again, I am not sure what you are testing here to begin with).
Another possibility is make them depend on each other:
val promise = Promise[Int]
def f1 = promise.future
def f2 = promise.future.map(_ + 1)
def f3 = promise.future.map(_ + 2)
...
promise.complete(Success(1))
I´m learning the monad IO of scalaZ and I cannot understand how catchAll and catchSome operators works.
I was expecting so see a behave like the onError or onErrorrResumeNext of RxJava, but instead is not catching the throwable, and it´s just breaking the test and throwing the NullPointerException..
Here my two examples
#Test
def catchAllOperator(): Unit = {
val errorSentence: IO[Throwable, String] =
IO.point[Throwable, String](null)
.map(value => value.toUpperCase())
.catchAll(error => IO.fail(error))
println(unsafePerformIO(errorSentence))
}
And catchSome example
#Test
def catchSomeOperator(): Unit = {
val errorFunction = new PartialFunction[Throwable /*Entry type*/ , IO[Throwable, String] /*Output type*/ ] {
override def isDefinedAt(x: Throwable): Boolean = x.isInstanceOf[NullPointerException]
override def apply(v1: Throwable): IO[Throwable, String] = IO.point("Default value")
}
val errorSentence = IO.point[Throwable, String](null)
.map(value => value.toUpperCase())
.catchSome(errorFunction)
println(unsafePerformIO(errorSentence))
}
Any idea what I´m doing wrong?.
Regards
I'm trying to figure out how to write this piece of code in an elegant pure-functional style using scalaz7 IO and monad transformers but just can't get my head around it.
Just imagine I have this simple API:
def findUuid(request: Request): Option[String] = ???
def findProfile(uuid: String): Future[Option[Profile]] = redisClient.get[Profile](uuid)
Using this API I can easily write impure function with OptionT transformer like this:
val profileT = for {
uuid <- OptionT(Future.successful(findUuid(request)))
profile <- OptionT(findProfile(uuid))
} yield profile
val profile: Future[Option[Profile]] = profileT.run
As you have noticed - this function contains findProfile() with a side-effect. I want to isolate this effect inside of the IO monad and interpret outside of the pure function but don't know how to combine it all together.
def findProfileIO(uuid: String): IO[Future[Option[Profile]]] = IO(findProfile(uuid))
val profileT = for {
uuid <- OptionT(Future.successful(findUuid(request)))
profile <- OptionT(findProfileIO(uuid)) //??? how to put Option inside of the IO[Future[Option]]
} yield profile
val profile = profileT.run //how to run transformer and interpret IO with the unsafePerformIO()???
Any peaces of advice on how it might be done?
IO is meant more for synchronous effects. Task is more what you want!
See this question and answer: What's the difference between Task and IO in Scalaz?
You can convert your Future to Task and then have an API like this:
def findUuid(request: Request): Option[String] = ???
def findProfile(uuid: String): Task[Option[Profile]] = ???
This works because Task can represent both synchronous and asynchronous operations, so findUuid can also be wrapped in Task instead of IO.
Then you can wrap these in OptionT:
val profileT = for {
uuid <- OptionT(Task.now(findUuid(request)))
profile <- OptionT(findProfileIO(uuid))
} yield profile
Then at the end somewhere you can run it:
profileT.run.attemptRun
Check out this link for converting Futures to Tasks and vice versa: Scalaz Task <-> Future
End up with this piece of code, thought it might be useful for someone (Play 2.6).
Controller's method is a pure function since Task evaluation takes place outside of the controller inside of PureAction ActionBuilder. Thanks to Luka's answer!
Still struggling with new paradigm of Action composition in Play 2.6 though, but this is another story.
FrontendController.scala:
def index = PureAction.pure { request =>
val profileOpt = (for {
uuid <- OptionT(Task.now(request.cookies.get("uuid").map(t => uuidKey(t.value))))
profile <- OptionT(redis.get[Profile](uuid).asTask)
} yield profile).run
profileOpt.map { profileOpt =>
Logger.info(profileOpt.map(p => s"User logged in - $p").getOrElse("New user, suggesting login"))
Ok(views.html.index(profileOpt))
}
}
Actions.scala
Convenient action with Task resolution at the end
class PureAction #Inject()(parser: BodyParsers.Default)(implicit ec: ExecutionContext) extends ActionBuilderImpl(parser) {
self =>
def pure(block: Request[AnyContent] => Task[Result]): Action[AnyContent] = composeAction(new Action[AnyContent] {
override def parser: BodyParser[AnyContent] = self.parser
override def executionContext: ExecutionContext = self.ec
override def apply(request: Request[AnyContent]): Future[Result] = {
val taskResult = block(request)
taskResult.asFuture //End of the world lives here
}
})
}
Converters.scala
Task->Future and Future->Task implicit converters
implicit class FuturePimped[+T](root: => Future[T]) {
import scalaz.Scalaz._
def asTask(implicit ec: ExecutionContext): Task[T] = {
Task.async { register =>
root.onComplete {
case Success(v) => register(v.right)
case Failure(ex) => register(ex.left)
}
}
}
}
implicit class TaskPimped[T](root: => Task[T]) {
import scalaz._
val p: Promise[T] = Promise()
def asFuture: Future[T] = {
root.unsafePerformAsync {
case -\/(ex) => p.failure(ex); ()
case \/-(r) => p.success(r); ()
}
p.future
}
}
In the code below I have to return the result of a future that is invoked after another future. I'm getting the following error in the future2.map line:
type mismatch; found : scala.concurrent.Future[play.api.mvc.Result]
required: play.api.mvc.Result
How to make this work?
def method1 = Action.async { request =>
val future1 = f1
future1.map { result1 =>
val future2 = f2
future2.map { result2 =>
Ok(result1+result2+"")
}
}
}
def f1 = Future { 1 }
def f2 = Future { 2 }
You could do this in many ways. But first, you need to understand how map and flatMap work with Future:
def map[S](f: (T) ⇒ S): Future[S]
def map[S](f: (T) ⇒ Future[S]): Future[Future[S]]
def flatMap[S](f: (T) ⇒ Future[S]): Future[S]
Note that,in above signatures, you are calling map and flatMap with a value that already is a future i.e Future[<some-value>].map(...) or Future[<some-value>].flatMap(...).
Approach 1:
def method1 = Action.async { request =>
val future1 = f1
future1.flatMap { result1 => //replaced map with flatMap
val future2 = f2
future2.map { result2 =>
Ok(result1+result2+"")
}
}
}
def f1 = Future { 1 }
def f2 = Future { 2 }
Approach 2:
def method1 = Action.async { request =>
val future1 = f1
future1.flatMap { result1 => //replaced map with flatMap
val future2 = f2
future2.flatMap { result2 => //replaced map with flatMap
Future.successful{Ok(result1+result2+"")} // used Future.successful{} to generate a Future of Result
}
}
}
def f1 = Future { 1 }
def f2 = Future { 2 }
Changing future1.map to future1.flatMap should do the trick. Mapping over a Future returns another Future and changes the value inside. In this case, you're returning a Future that contains another Future that contains a Result. By using flatMap, it essentially flattens the nested Future[Future[Result]] into a Future[Result].
I have a collection of methods that return different types:
Either[ErrorResponse, X]
Future[Either[ErrorResponse, X]]
Option[ErrorResponse]
These methods need the result from a previous method to perform their computation. The methods:
type Parameters = Map[String, String]
// allows me to flatmap on an either
implicit def toRightProjection[Failure, Success](e: Either[Failure, Success]) =
e.right
// converts anything to a future
implicit def toFuture[T](t: T) =
Future.successful(t)
// retrieves the request paramters from the given request
def requestParameters(request: RequestHeader): Either[ErrorResponse, Parameters] = ???
// retrieves the response type from the given parameters
def responseType(p: Parameters): Either[ErrorResponse, String] = ???
// retrieves the client id from the given parameters
def clientId(p: Parameters): Either[ErrorResponse, String] = ???
// retrieves the client using the given client id
def client(clientId: String): Future[Either[ErrorResponse, Client]] = ???
// validates the response type of the client
def validateResponseType(client: Client, responseType: String): Option[ErrorResponse] = ???
I can the wire them together with the following for comprehension (note that I wrote down some types to clarify the contents of specific parts of the computation).
val result: Either[ErrorResponse, Future[Either[ErrorResponse, Client]]] =
for {
parameters <- requestParameters(request)
clientId <- clientId(parameters)
responseType <- responseType(parameters)
} yield {
val result: Future[Either[ErrorResponse, Either[ErrorResponse, Client]]] =
for {
errorOrClient <- client(clientId)
client <- errorOrClient
} yield validateResponseType(client, responseType).toLeft(client)
result.map(_.joinRight)
}
val wantedResult: Future[Either[ErrorResponse, Client]] =
result.left.map(Future successful Left(_)).merge
The above code is quite messy and I feel this can be done differently. I read about monads and monad transformers. The concept of those is very new to me and I can not get my head around it.
Most of the examples only deal with two types of results: Either[X, Y] and Future[Either[X, Y]]. I still find it very hard to bend my mind around it.
How can I write a nice and clean for comprehension that replaces the above one?
Something like this would be awesome (I am not sure if that is even possible):
val result: Future[Either[ErrorResponse, Client]] =
for {
parameters <- requestParameters(request)
clientId <- clientId(parameters)
responseType <- responseType(parameters)
client <- client(clientId)
_ <- validateResponseType(client, responseType)
}
OK, here is my attempt at this:
import scalaz._, Scalaz._
implicit val futureMonad = new Monad[Future] {
override def point[A](a: ⇒ A): Future[A] = future(a)
override def bind[A, B](fa: Future[A])(f: A ⇒ Future[B]): Future[B] =
fa.flatMap(f)
}
import EitherT._
val result: EitherT[Future, ErrorResponse, Client] =
for {
parameters <- fromEither(Future(requestParameters(request)))
clientId <- fromEither(Future(clientId(parameters)))
responseType <- fromEither(Future(responseType(parameters)))
client <- fromEither(client(clientId))
response <- fromEither[Future, ErrorResponse, Client](Future(validateResponseType(client, responseType).toLeft(client)))
} yield response
val x: Future[\/[ErrorResponse, Client]] = result.run
scala.util.Either is not a Monad, but the scalaz library has a great implementation.
object Test extends ToIdOps {
import scalaz.{ Monad, Functor, EitherT, \/, -\/, \/- }
import scalaz.syntax.ToIdOps
implicit val FutureFunctor = new Functor[Future] {
def map[A, B](a: Future[A])(f: A => B): Future[B] = a map f
}
implicit val FutureMonad = new Monad[Future] {
def point[A](a: => A): Future[A] = Future(a)
def bind[A, B](fa: Future[A])(f: (A) => Future[B]): Future[B] = fa flatMap f
}
def someMethod: Future[\/[InvalidData, ValidData]] = {
// things went well
ValidData.right // this comes from ToIdOps
// or something went wrong
InvalidData.left
}
def someOtherMethod: Future[\/[InvalidData, ValidData]] // same as above
val seq = for {
d <- EitherT(someMethod)
y <- EitherT(someOtherMethod)
} yield { // whatever}
// you can now Await.result(seq.run, duration)
// you can map or match etc with \/- and -\/
val result = seq.run map {
case -\/(left) => // invalid data
case \/-(right) => // game on
}
}
There is no really clean way to do comprehensions over multiple monad types. In ScalaZ there is OptionT that might help, worth checking out. You could also transform your Eithers to Options or the other way around and be able to have a little bit less of a mess. A third option might be to create your own kind of wrapper that combines Future[Either|Option] into the same monad and then comprehend over that.
For reference I asked aboutish the same question on the play framework mailing list recently and got some good links in the replies: https://groups.google.com/d/topic/play-framework/JmCsXNDvAns/discussion