I'm still pretty new to Scala, and I'm trying to understand a section of code I came across:
for {
_ <- func1()
result <- func2()
} yield result
func1 and func2 return different types. I'm trying to understand the first line _ <- func1()
It looks to me like it just calls func1() and then drops the result? Does func1() act as a gating mechanism for func2()? That is, if func1() throws an exception, then func2() does not get run? But otherwise, any return value from func1() is thrown away?
Related
I'm fairly new to cats-effect, but I think I am getting a handle on it. But I have come to a situation where I want to memoize the result of an IO, and it's not doing what I expect.
The function I want to memoize transforms String => String, but the transformation requires a network call, so it is implemented as a function String => IO[String]. In a non-IO world, I'd simply save the result of the call, but the defining function doesn't actually have access to it, as it doesn't execute until later. And if I save the constructed IO[String], it won't actually help, as that IO would repeat the network call every time it's used. So instead, I try to use Async.memoize, which has the following documentation:
Lazily memoizes f. For every time the returned F[F[A]] is bound, the
effect f will be performed at most once (when the inner F[A] is bound
the first time).
What I expect from memoize is a function that only ever executes once for a given input, AND where the contents of the returned IO are only ever evaluated once; in other words, I expect the resulting IO to act as if it were IO.pure(result), except the first time. But that's not what seems to be happening. Instead, I find that while the called function itself only executes once, the contents of the IO are still evaluated every time - exactly as would occur if I tried to naively save and reuse the IO.
I constructed an example to show the problem:
def plus1(num: Int): IO[Int] = {
println("foo")
IO(println("bar")) *> IO(num + 1)
}
var fooMap = Map[Int, IO[IO[Int]]]()
def mplus1(num: Int): IO[Int] = {
val check = fooMap.get(num)
val res = check.getOrElse {
val plus = Async.memoize(plus1(num))
fooMap = fooMap + ((num, plus))
plus
}
res.flatten
}
println("start")
val call1 = mplus1(2)
val call2 = mplus1(2)
val result = (call1 *> call2).unsafeRunSync()
println(result)
println(fooMap.toString)
println("finish")
The output of this program is:
start
foo
bar
bar
3
Map(2 -> <function1>)
finish
Although the plus1 function itself only executes once (one "foo" printed), the output "bar" contained within the IO is printed twice, when I expect it to also print only once. (I have also tried flattening the IO returned by Async.memoize before storing it in the map, but that doesn't do much).
Consider following examples
Given the following helper methods
def plus1(num: Int): IO[IO[Int]] = {
IO(IO(println("plus1")) *> IO(num + 1))
}
def mPlus1(num: Int): IO[IO[Int]] = {
Async.memoize(plus1(num).flatten)
}
Let's build a program that evaluates plus1(1) twice.
val program1 = for {
io <- plus1(1)
_ <- io
_ <- io
} yield {}
program1.unsafeRunSync()
This produces the expected output of printing plus1 twice.
If you do the same but instead using the mPlus1 method
val program2 = for {
io <- mPlus1(1)
_ <- io
_ <- io
} yield {}
program2.unsafeRunSync()
It will print plus1 just once confirming that memoization is working.
The trick with the memoization is that it should be evaluated only once to have the desired effect. Consider now the following program that highlights it.
val memIo = mPlus1(1)
val program3 = for {
io1 <- memIo
io2 <- memIo
_ <- io1
_ <- io2
} yield {}
program3.unsafeRunSync()
And it outputs plus1 twice as io1 and io2 are memoized separately.
As for your example, the foo is printed once because you're using a map and update the value when it's not found and this happens only once. The bar is printed every time when IO is evaluated as you lose the memoization effect by calling res.flatten.
I'm sporadically getting util.NoSuchElementException: Future.filter predicate is not satisfied exceptions.
My understanding from other questions is that this appears with if guards in for comprehensions. I extensively use for comprehensions but I can't find any conditions. I do however make some assignments.
for {
foo <- Future{0}
bar = foo + 1
} yield bar
However, my understanding is that this should be fine.
I'm struggling to find the source of the runtime exception. Are there other cases where Future.filter would be called, besides if guards? Any other ideas of what I should be looking for?
Edit:
Could this be caused by pattern matching, like
for {
foo <- Future{0 -> 1}
(bar, _) <- foo
} yield bar + 1
Edit2:
I'm guessing the above is fine but this might cause a RTE?
for {
(bar, _) <- Future{0 -> 1}
} yield bar + 1
.filter and .withFilter (lazy version preferred by for-comprehension) remove from your F[_] values that don't fulfill the predicate:
with List it removed values from the List and returns a List with those values removed
with Option it turns Some to None if predicate is not fulfilled
with Try - it's similar to Option but Success is treated like Some, and Failure like None - however Failure cannot be empty, and that's why it puts Exception there
Future is asynchronous Try.
For that reason .withFilter(condition) is basically lazy version of
.flatMap { value =>
if (condition(value)) => Future.successful(value)
else Future.failed(new util.NoSuchElementException("Future.filter predicate is not satisfied"))
}
You will get that for every case that call .withFilter on Future underneath. With for comprehension that would be:
direct call of .withFilter
usage of if clause
usage of pattern matching
for {
f <- future.withFilter(cond1) // this might raise exception
g <- makeFuture(f) if cond2 // this might raise
(head, tail) <- makeFutureList(g) // this might raise
} yield
To avoid it (in Future or other IO monads)
don't call withFilter manually
instead of if use if-else:
for {
value <- future
success <- if (cond(value)) Future.successful(value)
else Future.failed(yourOwnError) // handle however you like
} yield success
You can also use better monadic for to avoid using .withFilter on pattern matching which cannot fail (though if they can fail you would still get exception, so it's not a solution, but still nice a way of getting rid of unwanted .withFilter).
I've got what seems like a fairly simple situation here:
I create a sequence of futures with yield of signature Future[(String, Date)]
I use Future.sequence to turn the IndexedSeq[Future[(String, Date)]] into a Future[IndexedSeq[(String, Date)]]
I call onComplete on that future to do something with each future after all of them have completed.
I'm doing this because the first wave are database inserts, I can't do the updateCalculatedField call until all inserts are completed, because updateCalculatedField involves aggregate functions and I don't want to introduce a race condition. Here's the code:
val initialInserts = for(i <- 0 until responseData.numValues()) yield { ... }
val res = Future.sequence(initialInserts)
res.onComplete{
case Success(seq) =>
for((i, d) <- seq)
updateCalculatedField(i, d)
case Failure(e) => e.printStackTrace()
}
What exactly am I doing wrong here? The code within the onComplete call never executes, and the program exits with no complaints. I've checked, and res.size is equal to the amount of rows in my database after the program exits... so the callback should have been executed. I'm at a loss here.
Thanks.
If I have some computation that takes a while I might place it in a scala.concurrent.Future:
val f = Future { someLongRunningFunction() }
and let's say I want to do something else asynchronously once that computation is completed:
f.flatMap{ _ => anotherLongRunningFunction() }
In the event that f's initial block fails, how do I "idiomatically" handle this when using flatMap or other combinators? Is it merely a case of using recover or onFailure before the flatMap?
I like the elegance and simplicity of using a flatMap but it seems failure scenarios get in the way of it.
Edit: the second future is reliant on the first, hence the flatMap. I'm looking for a solution that'll elegantly let me chain like I would with flatMap but also handle failures of the first.
To quote the scaladoc for flatMap:
Creates a new future by applying a function to the successful result
of this future, and returns the result of the function as the new
future. If this future is completed with an exception then the new
future will also contain this exception.
Notice the bold, meaning that whatever you pass to flatMap will only be executed if the initial future completes successfully.
So you should only handle the result of the entire execution:
val result = future1.flatMap {
result => functionReturningFuture2(result)
}
and then:
result.onFailure // or
result.onSuccess // or
result.recover
If you have several futures you can put them in a for comprehension.
val futureResult = for {
result1 <- future1
result2 <- future2
...
} yield {
//computation with results
}
You can add a recover at the end in case you want to process any exception you may find:
futureResult.recover{
case exceptionResult: Throwable => // Process exception
}
I think this is more clean that using flatMap.
I'm only starting with Future, but here are some ideas.
If you really want to use flatMap, you have to turn the failure into a success.
for{ a <- f recover r
b <- another(a)
} yield b
This works if the return type of r is :> the result type of f.
Or you can pass the problem of what to do with the failure on to the next process
for{ a <- f map (x => Success(x)) recover (ex => Failure(ex))
b <- another(a)
} yield b
Here the argument type of another would be Try[T] where the type of f is Future[T].
I am struggling to understand how this code 'strips out' the Future.
getFutureResult() returns a Future[String]. So how come the return from the yield is only a String?
def getFutureResult:Future[String] = { ... }
def getMyResult:String = {
for {
myFutureResult <- getFutureResult()
} yield {
myFutureResult
}
}
It is translated according to Scala for-comprehension translation rules:
for (x <- <expr>) yield f(x)
becomes:
<expr>.map(x => f(x))
This is a desugaring done by the compiler irregardless if <expr> has the type of a collection, a future or something else.
Future in Scala has a method map, so it uses the myFutureResult String from above to construct another Future[String].
You never know if the resulting future getMyResult is completed -- you should install an onComplete callback that will be called by the resulting future once it completes:
getMyResult onComplete {
case Success(x) => println("Yes! " + x)
case Failure(t) => println("An error: " + t)
}
The code for the onComplete method is executed asynchronously -- it might happen on a different thread, much later or simultaneously.
If you really need to know if the Future has completed, use the:
Await(getMyResult, 0 nanos)
pattern to block the calling thread until the getMyResult future is completed. In general, you should avoid this and compose your Future code using for-comprehensions and callback.