Unexpected Future.map() execution order - scala

I have a following Scala program:
object FutureMapTest extends App {
println("start")
val f: Future[Long] = Future {
Thread.sleep(2000)
val x = 1
println(s"started with ${x}")
x
}
f.map { i =>
println(s"mapped to ${i*2}")
}
f.map {
val nothing = "nothing"
println(s"mapped to ${nothing}")
_ * 2
}
Thread.sleep(3000)
println("end")
}
What I'd expect it to print on the console is
start
started with 1
followed by (in any order):
mapped to 2
mapped to nothing
followed by
end
What it actually prints is:
start
mapped to nothing
started with 1
mapped to 2
end
So, it seems like the second "map" block gets executed immediately, without waiting for the original future to complete. How is that possible?
You can even remove Thread.sleep() from the original future block, the result would still be the same.

There are a couple sources of confusion here.
This:
f.map {
val nothing = "nothing"
println(s"mapped to ${nothing}")
_ * 2
}
Expands to:
f.map {
val nothing = "nothing"
println(s"mapped to ${nothing}")
i => i * 2
}
What does this mean? Future#map expects a function argument of a A => B for some Future[A]. The expression:
val nothing = "nothing"
println(s"mapped to ${nothing}")
i => i * 2
..Evaluates to Long => Long, but the val assignment and println are evaluated first because they are part of the expression that returns the function. i => i * 2 isn't executed until f completes. This is similar to (Scala puzzler 001):
scala> List(1, 2, 3) map {
| val a = 1 // this only happens once, not three times
| i => a + i + 1
| }
res0: List[Int] = List(3, 4, 5)
Changing it to this will exhibit the behavior you expect (now that val assignment and println are part of the function body):
f.map { i =>
val nothing = "nothing"
println(s"mapped to ${nothing}")
i * 2
}
Here's another way to look at it:
f.map {
println("evaluated immediately")
i => { println("evaluated after f"); i * 2 }
}

Related

Scala Yeild returning Try[Either[]] rather then Either

I am trying to do some handson with scala basic operations and got stuck here in the following sample code
def insuranceRateQuote(a: Int, tickets:Int) : Either[Exception, Double] = {
// ... something
Right(Double)
}
def parseInsuranceQuoteFromWebForm(age: String, numOfTickets: String) : Either[Exception, Double]= {
try{
val a = Try(age.toInt)
val tickets = Try(numOfTickets.toInt)
for{
aa <- a
t <- tickets
} yield insuranceRateQuote(aa,t) // ERROR HERE
} catch {
case _ => Left(new Exception)}
}
The Error I am getting is that it says found Try[Either[Exception,Double]]
I am not getting why it is wrapper under Try of Either
PS - This must not be the perfect way to do in scala so feel free to post your sample code :)
The key to understand is that for-comprehensions might transform what is inside the wrapper but will not change the wrapper itself. The reason is because for-comprehension de-sugar to map/flatMap calls on the wrapper determined in the first step of the chain. For example consider the following snippet
val result: Try[Int] = Try(41).map(v => v + 1)
// result: scala.util.Try[Int] = Success(42)
Note how we transformed the value inside the Try wrapper from 41 to 42 however the wrapper remained unchanged. Alternatively we could express the same thing using a for-comprehension
val result: Try[Int] = for { v <- Try(41) } yield v + 1
// result: scala.util.Try[Int] = Success(42)
Note how the effect is exactly the same. Now consider the following for comprehension which chains multiple steps
val result: Try[Int] =
for {
a <- Try(41) // first step determines the wrapper for all the other steps
b <- Try(1)
} yield a + b
// result: scala.util.Try[Int] = Success(42)
This expands to
val result: Try[Int] =
Try(41).flatMap { (a: Int) =>
Try(1).map { (b: Int) => a + b }
}
// result: scala.util.Try[Int] = Success(42)
where again we see the result is the same, namely, a value transformed inside the wrapper but wrapper remained untransformed.
Finally consider
val result: Try[Either[Exception, Int]] =
for {
a <- Try(41) // first step still determines the top-level wrapper
b <- Try(1)
} yield Right(a + b) // here we wrap inside `Either`
// result: scala.util.Try[Either[Exception,Int]] = Success(Right(42))
The principle remains the same - we did wrap a + b inside Either however this does not affect the top-level outer wrapper which is still Try.
Mario Galic's answer already explains the problem with your code, but I'd fix it differently.
Two points:
Either[Exception, A] (or rather, Either[Throwable, A]) is kind of equivalent to Try[A], with Left taking the role of Failure and Right the role of Success.
The outer try/catch is not useful because the exceptions should be captured by working in Try.
So you probably want something like
def insuranceRateQuote(a: Int, tickets:Int) : Try[Double] = {
// ... something
Success(someDouble)
}
def parseInsuranceQuoteFromWebForm(age: String, numOfTickets: String): Try[Double] = {
val a = Try(age.toInt)
val tickets = Try(numOfTickets.toInt)
for{
aa <- a
t <- tickets
q <- insuranceRateQuote(aa,t)
} yield q
}
A bit unfortunately, this does a useless map(q => q) if you figure out what the comprehension does, so you can write it more directly as
a.flatMap(aa => tickets.flatMap(t => insuranceRateQuote(aa,t)))

Functional way of interrupting lazy iteration depedning on timeout and comparisson between previous and next, while, LazyList vs Stream

Background
I have the following scenario. I want to execute the method of a class from an external library, repeatedly, and I want to do so until a certain timeout condition and result condition (compared to the previous result) is met. Furthermore I want to collect the return values, even on the "failed" run (the run with the "failing" result condition that should interrupt further execution).
Thus far I have accomplished this with initializing an empty var result: Result, a var stop: Boolean and using a while loop that runs while the conditions are true and modifying the outer state. I would like to get rid of this and use a functional approach.
Some context. Each run is expected to run from 0 to 60 minutes and the total time of iteration is capped at 60 minutes. Theoretically, there's no bound to how many times it executes in this period but in practice, it's generally 2-60 times.
The problem is, the runs take a long time so I need to stop the execution. My idea is to use some kind of lazy Iterator or Stream coupled with scanLeft and Option.
Code
Boiler plate
This code isn't particularly relevant but used in my approach samples and provide identical but somewhat random pseudo runtime results.
import scala.collection.mutable.ListBuffer
import scala.util.Random
val r = Random
r.setSeed(1)
val sleepingTimes: Seq[Int] = (1 to 601)
.map(x => Math.pow(2, x).toInt * r.nextInt(100))
.toList
.filter(_ > 0)
.sorted
val randomRes = r.shuffle((0 to 600).map(x => r.nextInt(10)).toList)
case class Result(val a: Int, val slept: Int)
class Lib() {
def run(i: Int) = {
println(s"running ${i}")
Thread.sleep(sleepingTimes(i))
Result(randomRes(i), sleepingTimes(i))
}
}
case class Baz(i: Int, result: Result)
val lib = new Lib()
val timeout = 10 * 1000
While approach
val iteratorStart = System.currentTimeMillis()
val iterator = for {
i <- (0 to 600).iterator
if System.currentTimeMillis() < iteratorStart + timeout
f = Baz(i, lib.run(i))
} yield f
val iteratorBuffer = ListBuffer[Baz]()
if (iterator.hasNext) iteratorBuffer.append(iterator.next())
var run = true
while (run && iterator.hasNext) {
val next = iterator.next()
run = iteratorBuffer.last.result.a < next.result.a
iteratorBuffer.append(next)
}
Stream approach (Scala.2.12)
Full example
val streamStart = System.currentTimeMillis()
val stream = for {
i <- (0 to 600).toStream
if System.currentTimeMillis() < streamStart + timeout
} yield Baz(i, lib.run(i))
var last: Option[Baz] = None
val head = stream.headOption
val tail = if (stream.nonEmpty) stream.tail else stream
val streamVersion = (tail
.scanLeft((head, true))((x, y) => {
if (x._1.exists(_.result.a > y.result.a)) (Some(y), false)
else (Some(y), true)
})
.takeWhile {
case (baz, continue) =>
if (!baz.eq(head)) last = baz
continue
}
.map(_._1)
.toList :+ last).flatten
LazyList approach (Scala 2.13)
Full example
val lazyListStart = System.currentTimeMillis()
val lazyList = for {
i <- (0 to 600).to(LazyList)
if System.currentTimeMillis() < lazyListStart + timeout
} yield Baz(i, lib.run(i))
var last: Option[Baz] = None
val head = lazyList.headOption
val tail = if (lazyList.nonEmpty) lazyList.tail else lazyList
val lazyListVersion = (tail
.scanLeft((head, true))((x, y) => {
if (x._1.exists(_.result.a > y.result.a)) (Some(y), false)
else (Some(y), true)
})
.takeWhile {
case (baz, continue) =>
if (!baz.eq(head)) last = baz
continue
}
.map(_._1)
.toList :+ last).flatten
Result
Both approaches appear to yield the correct end result:
List(Baz(0,Result(4,170)), Baz(1,Result(5,208)))
and they interrupt execution as desired.
Edit: The desired outcome is to not execute the next iteration but still return the result of the iteration that caused the interruption. Thus the desired result is
List(Baz(0,Result(4,170)), Baz(1,Result(5,208)), Baz(2,Result(2,256))
and lib.run(i) should only run 3 times.
This is achieved by the while approach, as well as the LazyList approach but not the Stream approach which executes lib.run 4 times (Bad!).
Question
Is there another stateless approach, which is hopefully more elegant?
Edit
I realized my examples were faulty and not returning the "failing" result, which it should, and that they kept executing beyond the stop condition. I rewrote the code and examples but I believe the spirit of the question is the same.
I would use something higher level, like fs2.
(or any other high-level streaming library, like: monix observables, akka streams or zio zstreams)
def runUntilOrTimeout[F[_]: Concurrent: Timer, A](work: F[A], timeout: FiniteDuration)
(stop: (A, A) => Boolean): Stream[F, A] = {
val interrupt =
Stream.sleep_(timeout)
val run =
Stream
.repeatEval(work)
.zipWithPrevious
.takeThrough {
case (Some(p), c) if stop(p, c) => false
case _ => true
} map {
case (_, c) => c
}
run mergeHaltBoth interrupt
}
You can see it working here.

Wait for future to end before printing a variable

In this code I need to print variable seq, but since it's printed before the futures are processed it is printed empty. How to wait for variable seq to be populated before the statement println(seq) is executed?
object TestFutures5 extends App {
def future (i:Int) = Future { i * 10 }
val seq = Seq[Int]()
for ( x <- 1 to 10 ) {
val future2 = future(x)
future2.map { y =>
println(y)
seq :+ y
}
}
println(seq) // <-- this always prints List()
Thread.sleep(5000)
}
The print statement must be executed after all the futures completed, which means that you need to store a reference to each created future. Your sequence is also immutable so you can not add elements to it. If you want to do this in without mutating variables your loop should be refactored like this:
val futureResult = (1 to 10).map {
x =>
future(x)
}
Then simply use Future.sequence to group the futures and do the print:
Future.sequence(futureResult).map(res => println(res))

Cats Writer Vector is empty

I wrote this simple program in my attempt to learn how Cats Writer works
import cats.data.Writer
import cats.syntax.applicative._
import cats.syntax.writer._
import cats.instances.vector._
object WriterTest extends App {
type Logged2[A] = Writer[Vector[String], A]
Vector("started the program").tell
val output1 = calculate1(10)
val foo = new Foo()
val output2 = foo.calculate2(20)
val (log, sum) = (output1 + output2).pure[Logged2].run
println(log)
println(sum)
def calculate1(x : Int) : Int = {
Vector("came inside calculate1").tell
val output = 10 + x
Vector(s"Calculated value ${output}").tell
output
}
}
class Foo {
def calculate2(x: Int) : Int = {
Vector("came inside calculate 2").tell
val output = 10 + x
Vector(s"calculated ${output}").tell
output
}
}
The program works and the output is
> run-main WriterTest
[info] Compiling 1 Scala source to /Users/Cats/target/scala-2.11/classes...
[info] Running WriterTest
Vector()
50
[success] Total time: 1 s, completed Jan 21, 2017 8:14:19 AM
But why is the vector empty? Shouldn't it contain all the strings on which I used the "tell" method?
When you call tell on your Vectors, each time you create a Writer[Vector[String], Unit]. However, you never actually do anything with your Writers, you just discard them. Further, you call pure to create your final Writer, which simply creates a Writer with an empty Vector. You have to combine the writers together in a chain that carries your value and message around.
type Logged[A] = Writer[Vector[String], A]
val (log, sum) = (for {
_ <- Vector("started the program").tell
output1 <- calculate1(10)
foo = new Foo()
output2 <- foo.calculate2(20)
} yield output1 + output2).run
def calculate1(x: Int): Logged[Int] = for {
_ <- Vector("came inside calculate1").tell
output = 10 + x
_ <- Vector(s"Calculated value ${output}").tell
} yield output
class Foo {
def calculate2(x: Int): Logged[Int] = for {
_ <- Vector("came inside calculate2").tell
output = 10 + x
_ <- Vector(s"calculated ${output}").tell
} yield output
}
Note the use of for notation. The definition of calculate1 is really
def calculate1(x: Int): Logged[Int] = Vector("came inside calculate1").tell.flatMap { _ =>
val output = 10 + x
Vector(s"calculated ${output}").tell.map { _ => output }
}
flatMap is the monadic bind operation, which means it understands how to take two monadic values (in this case Writer) and join them together to get a new one. In this case, it makes a Writer containing the concatenation of the logs and the value of the one on the right.
Note how there are no side effects. There is no global state by which Writer can remember all your calls to tell. You instead make many Writers and join them together with flatMap to get one big one at the end.
The problem with your example code is that you're not using the result of the tell method.
If you take a look at its signature, you'll see this:
final class WriterIdSyntax[A](val a: A) extends AnyVal {
def tell: Writer[A, Unit] = Writer(a, ())
}
it is clear that tell returns a Writer[A, Unit] result which is immediately discarded because you didn't assign it to a value.
The proper way to use a Writer (and any monad in Scala) is through its flatMap method. It would look similar to this:
println(
Vector("started the program").tell.flatMap { _ =>
15.pure[Logged2].flatMap { i =>
Writer(Vector("ended program"), i)
}
}
)
The code above, when executed will give you this:
WriterT((Vector(started the program, ended program),15))
As you can see, both messages and the int are stored in the result.
Now this is a bit ugly, and Scala actually provides a better way to do this: for-comprehensions. For-comprehension are a bit of syntactic sugar that allows us to write the same code in this way:
println(
for {
_ <- Vector("started the program").tell
i <- 15.pure[Logged2]
_ <- Vector("ended program").tell
} yield i
)
Now going back to your example, what I would recommend is for you to change the return type of compute1 and compute2 to be Writer[Vector[String], Int] and then try to make your application compile using what I wrote above.

Running futures sequentially

The objective of the code below is to execute Future f3 or f4 depending on a condition. Note that the condition depends on the result of Future f1 or f2, so it has to wait. This seems to work, however since f1 and f2 are futures this code shouldn't run sequentially. Is this code correct?
object TestFutures extends App {
val f1 = Future { 1 }
val f2 = Future { 2 }
val f3 = Future { 3 }
val f4 = Future { 4 }
val y = 1
for {
condition <- if (y>0) f1 else f2
_ <- if (condition==1) f3.map {a => println("333")} else f4.map {b => println("444")}
} yield ()
Thread.sleep(5000)
}
No it is not correct. When you create a Future like you do it, it starts the computations immediately. Before reaching for comprehension, all of your 4 futures are running already. You need to create them later, depending on the conditions.
val y = 1
for {
condition <- if (y > 0) Future { 1 } else Future { 2 }
_ <- if (condition == 1)
Future { 3 }.map(a => println("333"))
else
Future { 4 }.map(b => println("444"))
} yield ()
It is probably good to extract creating each of those to a method, that you will just call, for sake of readability.
It should be obvious they start running when they are created because you can just say
Future(1).map(x => println(x))
and it works without any sort of triggering. Anyway try to run the following code
import scala.concurrent.Future
import scala.concurrent.ExecutionContext.Implicits.global
def printWhenCompleted[A](f: Future[A]): Future[A] = f.map { x =>
println(x)
x
}
val f1 = printWhenCompleted(Future { 1 })
val f2 = printWhenCompleted(Future { 2 })
val f3 = printWhenCompleted(Future { 3 })
for {
r3 <- f3
r2 <- f2
r1 <- f1
} yield r1 + r2 + r3
it should give you those numbers in random order, instead of sequential 3, 2, 1
Edit
Here is implementation of the first code (without println) using flatMap
val futureCondition = if (y > 0) Future(1) else Future(2)
futureCondition.flatMap(condition => if (condition == 1) Future(3) else Future(4))