Why am I getting stackoverflow error with ZIO Ref - scala

Below is a memoized version of Fibonacci function implemented in ZIO.
If the highlighted line _ <- memo.get is commented out the function fib results in a Stackoverflow error. If it is un-commented the code works as expected.
Why does this get call on the memo Ref is needed to make this function work? and why without it we are running into a stackoverflow error?
object Main extends ZIOAppDefault {
private def compute(num: Int, memo: Ref[Map[Int, Int]]): UIO[Int] = for {
_ <- memo.get // commenting this line will result in Stackoverflow Error
a1 <- fib(num - 1, memo)
a2 <- fib(num - 2, memo)
} yield a1 + a2
private def fib(num: Int, memo: Ref[Map[Int, Int]]): UIO[Int] = {
if(num <= 1)
ZIO.succeed(1)
else
for {
map <- memo.get
res <- map.get(num).map(x => ZIO.succeed(x)).getOrElse(compute(num, memo))
} yield res
}
private def calculate(num: Int) = for {
map <- Ref.make[Map[Int, Int]](Map())
res <- fib(num, map)
} yield res
override def run: ZIO[Any with ZIOAppArgs with Scope, Any, Any] = calculate(9).debug
}

Related

Getting compilation error when using for and yield

What is wrong with the following code snippet?
val loginInfoFuture: Future[LoginInfo] = credentialsProvider.authenticate(credentials)
for{loginInfo <- loginInfoFuture}{
println("in loginInfo future")
} yield Future{Ok(Json.toJson(JsonResultError("Invalid Body Type. Need Json")))}
I am seeing error in IDE - Error:(239, 17) ';' expected but 'yield' found.
} yield Ok(Json.toJson(JsonResultError("Invalid Body Type. Need Json")))
I tried a similar piece of code on REPL and that seem to work fine.
scala> import scala.concurrent.Future
import scala.concurrent.Future
scala> import scala.concurrent.ExecutionContext.Implicits.global
import scala.concurrent.ExecutionContext.Implicits.global
scala> val f:Future[Int] = Future{1}
f: scala.concurrent.Future[Int] = Future(Success(1))
scala> for(f1 <- f) yield f1
res0: scala.concurrent.Future[Int] = Future(<not completed>)
scala>
For reference, below is the full function
def signInUser = silhouette.UserAwareAction.async { implicit request => {
val body: AnyContent = request.body
val jsonBody: Option[JsValue] = body.asJson
jsonBody match {
case Some(json) => {
val userSignin: Option[UserSignin] = json.asOpt[UserSignin] //check if json conforms with UserProfile structure
userSignin match {
case Some(signinInfo) => { //format of JSON is correct
//Get signin info from JSON (email and password)
val credentials: Credentials = Credentials(signinInfo.signinInfo.email, signinInfo.signinInfo.password)
val authInfoRepository = new DelegableAuthInfoRepository(userRepo.passwordRepo)
val passwordHasherRegistory = new PasswordHasherRegistry(userRepo.passwordHasher)
val credentialsProvider = new CredentialsProvider(authInfoRepository, passwordHasherRegistory)
for{loginInfo <- loginInfoFuture}{ //for returns unit. Should use yield
println("in loginInfo future")
} yield Future{Ok(Json.toJson(JsonResultError("Invalid Body Type. Need Json")))}
}
case None => { //No signin info found
Future {
Ok(Json.toJson(JsonResultError("Invalid user. No Login info found")))
}
}
}
}
case None => {//NO Body
Future {
Ok(Json.toJson(JsonResultError("Invalid Body Type. Need Json")))
}
}
} //jsonBody match
}//async
}//def signin
for{loginInfo <- loginInfoFuture}{ //for returns unit. Should use yield
println("in loginInfo future")
} yield Future{Ok(Json.toJson(JsonResultError("Invalid Body Type. Need Json")))}
This is invalid. Your for/yield needs to be in the format:
for {
y <- z
x <- y
//etc
} yield {
//whatever
}
The println after the for but before the yield is throwing you. To get the result of the println inside the for/yield, you could to assign it to a value:
for {
y <- z
a = println(y) // will print out every y
x <- y
//etc
} yield {
//whatever
}
for/yield blocks are stupid like that. At least there are work-arounds though!
The following section is from scala's Future documentation:
def foo(): Unit = {
val f = Future { 5 }
val g = Future { 3 }
val h = for {
x: Int <- f // returns Future(5)
y: Int <- g // returns Future(3)
} yield x + y
}
You on the other hand try to do this:
def foo(): Unit = {
val f = Future { 5 }
val g = Future { 3 }
val h = for {
x: Int <- f // returns Future(5)
y: Int <- g // returns Future(3)
} {
println("whatever") // <<<<<<<<<
} yield x + y
}
The extra block of code that I point is what causing the compilation error which you did not add in your scala repl example.
This is how you can print within a Future:
def foo(): Unit = {
val f = Future {
println("5")
5
}
val g = Future {
println("3")
3
}
val h = for {
x: Int <- f // returns Future(5)
y: Int <- g // returns Future(3)
} yield x + y
}
Error:(239, 17) ';' expected but 'yield' found.
simply means that the for loop definition is wrong
So either with yield
for{loginInfo <- loginInfoFuture
//other conditions and statements
} yield //value to be returned
or without yield
for(loginInfo <- loginInfoFuture){
//value updated
}
are correct for loop definitions

Scala Lazy Dynamic Programming

So I'm following http://jelv.is/blog/Lazy-Dynamic-Programming/ and implementing the Fibonacci example in Scala. Here is my implementation:
class Lazy[T] (expr : => T) {
lazy val value = expr
def apply(): T = value
}
object Lazy{ def apply[T](expr : => T) = new Lazy({expr}) }
def fib(n: Int): Int = {
def doFib(i: Int): Lazy[Int] = Lazy {
if (i <= 2) 1
else fibs(i - 1)() + fibs(i - 2)()
}
lazy val fibs = Array.tabulate[Lazy[Int]](n)(doFib)
doFib(n).value
}
fib(5)
In this case, fib(5) correctly returns result 5.
Then I want to see if Lazy[T] can be made into a monad by trying the following code, which results in StackOverflow runtime error:
class Lazy[T] (expr : => T) {
lazy val value = expr
def apply(): T = value
def flatMap[A](f: T => Lazy[A]): Lazy[A] = Lazy { f(value).value }
def map[A](f: T => A): Lazy[A] = Lazy { f(value) }
}
object Lazy{ def apply[T](expr : => T) = new Lazy({expr}) }
def fib(n: Int): Int = {
def doFib(i: Int): Lazy[Int] =
if (i <= 2) Lazy(1)
else for {
a <- fibs(i - 1)
b <- fibs(i - 2)
} yield a + b
lazy val fibs = Array.tabulate[Lazy[Int]](n)(doFib)
doFib(n).value
}
fib(5)
It appears that fibs(i - 1) is calculated too early, which results in infinite recursion. I wonder if there is a for comprehension syntax that's equivalent to the first code snippet?
You are right, "fibs(i - 1) is calculated too early". It is evaluated immediately when you call doFib, because the doFib(i) needs fibs(i - 1) in order to be able to return anything, which in turn needs the return value of doFib(i - 1) and so on, so that the recursion unfolds completely while you are constructing the array of lazy ints (before you invoke doFib(n).value).
If you want it lazy, then return a Lazy that does not require immediate evaluation of fibs(i - 1):
class Lazy[T] (expr : => T) {
lazy val value = expr
def apply(): T = value
def flatMap[A](f: T => Lazy[A]): Lazy[A] = Lazy { f(value).value }
def map[A](f: T => A): Lazy[A] = Lazy { f(value) }
}
object Lazy{ def apply[T](expr : => T) = new Lazy({expr}) }
def fib(n: Int): Int = {
def doFib(i: Int): Lazy[Int] =
if (i <= 2) Lazy(1)
else Lazy{ (for {
a <- fibs(i - 1)
b <- fibs(i - 2)
} yield a + b).value
}
lazy val fibs = Array.tabulate[Lazy[Int]](n)(doFib)
doFib(n).value
}
println(fib(40)) // 102334155
Alternatively, you can wrap the whole if-else in a Lazy:
def doFib(i: Int): Lazy[Int] = Lazy {
if (i <= 2) 1
else (for {
a <- fibs(i - 1)
b <- fibs(i - 2)
} yield a + b).value
}
This produces the same expected result.

Running sequential Futures with similar shapes using for comprehension

def func1(list : List[T]) : Future[\/[Throwable,Unit] ]
def func2(list : List[T]) : Future[List[\/[Throwable,Unit]]]
where T is just a specific type and that type will be same for both the functions. Now func2 is dependent on success of first func 's future. so func2 should run sequentially only after func completed successfully. I want a for comprehension something in a similar line as below (following isn't valid compilable code) and return Future[\/[Throwable,Unit] ]
def func3 combiner(list) : Future[\/[Throwable,Unit] ] = for{
u <- func1(list)
us <- u
d <- func2(list)
}yield
Any pointers how to go about this?
Because futures either complete with a value or an exception you won't need the Either (or do you have other reasons for using it?).
Running this code should help you (and reading the documentation on futures):
import scala.concurrent._
import ExecutionContext.Implicits.global
def f1(l: List[Int]): Future[Int] = future { println("f1"); l head }
def f2(l: List[Int]): Future[Int] = future { println("f2"); throw new Exception("bang") }
def f3(l: List[Int]): Future[Int] = future { println("f3"); l last }
val result1 = for {
x1 <- f2(List(1, 2))
x2 <- f1(List(1, 2)) // f1 is not run
} yield x2
val result2 = for {
x1 <- f1(List(1, 2))
x3 <- f3(List(1, 2))
} yield x3
result1.onComplete(res => println("result1 = " + res))
result2.onComplete(res => println("result2 = " + res))

Using for-comprehension, Try and sequences in Scala

Let's say you've got a bunch of methods:
def foo() : Try[Seq[String]]
def bar(s:String) : Try[String]
and you want to make a for-comprhension:
for {
list <- foo
item <- list
result <- bar(item)
} yield result
of course this won't compile since Seq cannot be used with Try in this context.
Anyone has a nice solution how to write this clean without breaking it into separate two for's?
I've came across this syntax problem for the thirds time and thought that it's about time to ask about this.
IMHO: Try and Seq is more than what you need to define a monad transformer:
Code for library:
case class trySeq[R](run : Try[Seq[R]]) {
def map[B](f : R => B): trySeq[B] = trySeq(run map { _ map f })
def flatMap[B](f : R => trySeq[B]): trySeq[B] = trySeq {
run match {
case Success(s) => sequence(s map f map { _.run }).map { _.flatten }
case Failure(e) => Failure(e)
}
}
def sequence[R](seq : Seq[Try[R]]): Try[Seq[R]] = {
seq match {
case Success(h) :: tail =>
tail.foldLeft(Try(h :: Nil)) {
case (Success(acc), Success(elem)) => Success(elem :: acc)
case (e : Failure[R], _) => e
case (_, Failure(e)) => Failure(e)
}
case Failure(e) :: _ => Failure(e)
case Nil => Try { Nil }
}
}
}
object trySeq {
def withTry[R](run : Seq[R]): trySeq[R] = new trySeq(Try { run })
def withSeq[R](run : Try[R]): trySeq[R] = new trySeq(run map (_ :: Nil))
implicit def toTrySeqT[R](run : Try[Seq[R]]) = trySeq(run)
implicit def fromTrySeqT[R](trySeqT : trySeq[R]) = trySeqT.run
}
and after you can use for-comrehension (just import your library):
def foo : Try[Seq[String]] = Try { List("hello", "world") }
def bar(s : String) : Try[String] = Try { s + "! " }
val x = for {
item1 <- trySeq { foo }
item2 <- trySeq { foo }
result <- trySeq.withSeq { bar(item2) }
} yield item1 + result
println(x.run)
and it works for:
def foo() = Try { List("hello", throw new IllegalArgumentException()) }
// x = Failure(java.lang.IllegalArgumentException)
You can take advantage of the fact that Try can be converted to Option, and Option to Seq:
for {
list <- foo.toOption.toSeq // toSeq needed here, as otherwise Option.flatMap will be used, rather than Seq.flatMap
item <- list
result <- bar(item).toOption // toSeq not needed here (but allowed), as it is implicitly converted
} yield result
This will return a (possibly empty, if the Trys failed) Seq.
If you want to keep all the exception detail, you'll need a Try[Seq[Try[String]]]. This can't be done with a single for comprehension, so you're best sticking with plain map:
foo map {_ map bar}
If you want to mingle your Trys and Seqs in a different way, things get fiddlier, as there's no natural way to flatten a Try[Seq[Try[String]]]. #Yury's answer demonstrates the sort of thing you'd have to do.
Or, if you're only interested in the side effects of your code, you can just do:
for {
list <- foo
item <- list
result <- bar(item)
} result
This works because foreach has a less restrictive type signature.
A Try can be converted to an Option, which you can than use in a for-comprehension. E.g.
scala> def testIt() = {
| val dividend = Try(Console.readLine("Enter an Int that you'd like to divide:\n").toInt)
| dividend.toOption
| }
testIt: ()Option[Int]
scala> for (x <- testIt()) println (x * x)
Enter an Int that you'd like to divide:
scala> for (x <- testIt()) println (x * x)
Enter an Int that you'd like to divide:
1522756
First time I entered "w", then second time 1234.

Value assignment inside for-loop in Scala

Is there any difference between this code:
for(term <- term_array) {
val list = hashmap.get(term)
...
}
and:
for(term <- term_array; val list = hashmap.get(term)) {
...
}
Inside the loop I'm changing the hashmap with something like this
hashmap.put(term, string :: list)
While checking for the head of list it seems to be outdated somehow when using the second code snippet.
The difference between the two is, that the first one is a definition which is created by pattern matching and the second one is a value inside a function literal. See Programming in Scala, Section 23.1 For Expressions:
for {
p <- persons // a generator
n = p.name // a definition
if (n startsWith "To") // a filter
} yield n
You see the real difference when you compile sources with scalac -Xprint:typer <filename>.scala:
object X {
val x1 = for (i <- (1 to 5); x = i*2) yield x
val x2 = for (i <- (1 to 5)) yield { val x = i*2; x }
}
After code transforming by the compiler you will get something like this:
private[this] val x1: scala.collection.immutable.IndexedSeq[Int] =
scala.this.Predef.intWrapper(1).to(5).map[(Int, Int), scala.collection.immutable.IndexedSeq[(Int, Int)]](((i: Int) => {
val x: Int = i.*(2);
scala.Tuple2.apply[Int, Int](i, x)
}))(immutable.this.IndexedSeq.canBuildFrom[(Int, Int)]).map[Int, scala.collection.immutable.IndexedSeq[Int]]((
(x$1: (Int, Int)) => (x$1: (Int, Int) #unchecked) match {
case (_1: Int, _2: Int)(Int, Int)((i # _), (x # _)) => x
}))(immutable.this.IndexedSeq.canBuildFrom[Int]);
private[this] val x2: scala.collection.immutable.IndexedSeq[Int] =
scala.this.Predef.intWrapper(1).to(5).map[Int, scala.collection.immutable.IndexedSeq[Int]](((i: Int) => {
val x: Int = i.*(2);
x
}))(immutable.this.IndexedSeq.canBuildFrom[Int]);
This can be simplified to:
val x1 = (1 to 5).map {i =>
val x: Int = i * 2
(i, x)
}.map {
case (i, x) => x
}
val x2 = (1 to 5).map {i =>
val x = i * 2
x
}
Instantiating variables inside for loops makes sense if you want to use that variable the for statement, like:
for (i <- is; a = something; if (a)) {
...
}
And the reason why your list is outdated, is that this translates to a foreach call, such as:
term_array.foreach {
term => val list= hashmap.get(term)
} foreach {
...
}
So when you reach ..., your hashmap has already been changed. The other example translates to:
term_array.foreach {
term => val list= hashmap.get(term)
...
}