I was just playing around with a implementation of foldlLeft in Scala
def foldLeft[A,B] (as: List[A], z: B) (f: (B, A) => B): B = as match {
case Nil => z
case Cons(x, xs) => foldLeft(xs, f(z,x)) (f)
}
In this implementation, f(z,x) is in the recursive call given as parameter for z, but I was wondering how this actually works?
When the recursive call happens, does foldLeft() receive the value of the execution of f(z,b) or does it receive the function call the way it is written, and then execute when needed?
Example:
if we call foldLeft() with the following values
def foldLeft[A,B] ([1,2,3], 0) (f: (x, y) => x + y): B = as match {
case Nil => z
case Cons(x, xs) => foldLeft([2,3], f(0,1)) (f)
}
Will the next execution of foldLeft() like this, where z is equal to the value of f()?
def foldLeft[A,B] ([2,3], 1) (f: (x, y) => x + y): B = as match {
case Nil => z
case Cons(x, xs) => foldLeft([2,3], f(1,2)) (f)
}
or does it work like this, where foldLeft() receives the call itself?
def foldLeft[A,B] ([2,3], f(0,1)) (f: (x, y) => x + y): B = as match {
case Nil => z
case Cons(x, xs) => foldLeft([3], f(f(1,0),2)) (f)
}
The question is essentially about when the values that a tail recursive function are evaluated?
Scala, like almost every mainstream language, is a strict language with an eager evaluation strategy and pass-by-value argument passing semantics.
This means that all arguments to a method or function call will be fully evaluated before being passed into the method or function.
However, what I wrote is actually not quite true: that is only the default.
There are two ways to deviate from the default:
The lazy modifier makes a value, well, lazy.
by-name parameters are passed by name, not by value.
#1 is not applicable here. #2 needs to be explicitly declared in the parameter list, which is not the case here.
Therefore, we can conclude that, yes, f(z, x) will be fully evaluated before the recursive call to foldLeft.
Related
To find prime factors of a number I was using this piece of code :
def primeFactors(num: Long): List[Long] = {
val exists = (2L to math.sqrt(num).toLong).find(num % _ == 0)
exists match {
case Some(d) => d :: primeFactors(num/d)
case None => List(num)
}
}
but this I found a cool and more functional approach to solve this using this code:
def factors(n: Long): List[Long] = (2 to math.sqrt(n).toInt)
.find(n % _ == 0).fold(List(n)) ( i => i.toLong :: factors(n / i))
Earlier I was using foldLeft or fold simply to get sum of a list or other simple calculations, but here I can't seem to understand how fold is working and how this is breaking out of the recursive function.Can somebody plz explain how fold functionality is working here.
Option's fold
If you look at the signature of Option's fold function, it takes two parameters:
def fold[B](ifEmpty: => B)(f: A => B): B
What it does is, it applies f on the value of Option if it is not empty. If Option is empty, it simply returns output of ifEmpty (this is termination condition for recursion).
So in your case, i => i.toLong :: factors(n / i) represents f which will be evaluated if Option is not empty. While List(n) is termination condition.
fold used for collection / iterators
The other fold that you are taking about for getting sum of collection, comes from TraversableOnce and it has signature like:
def foldLeft[B](z: B)(op: (B, A) => B): B
Here, z is starting value (suppose incase of sum it's 0) and op is associative binary operator which is applied on z and each value of collection from left to right.
So both folds differ in their implementation.
This is a implementation of the Y-combinator in Scala:
scala> def Y[T](func: (T => T) => (T => T)): (T => T) = func(Y(func))(_:T)
Y: [T](func: (T => T) => (T => T))T => T
scala> def fact = Y {
| f: (Int => Int) =>
| n: Int =>
| if(n <= 0) 1
| else n * f(n - 1)}
fact: Int => Int
scala> println(fact(5))
120
Q1: How does the result 120 come out, step by step? Because the Y(func) is defined as func(Y(func)), the Y should become more and more,Where is the Y gone lost and how is the 120 come out in the peform process?
Q2: What is the difference between
def Y[T](func: (T => T) => (T => T)): (T => T) = func(Y(func))(_:T)
and
def Y[T](func: (T => T) => (T => T)): (T => T) = func(Y(func))
They are the same type in the scala REPL, but the second one can not print the result 120?
scala> def Y[T](func: (T => T) => (T => T)): (T => T) = func(Y(func))
Y: [T](func: (T => T) => (T => T))T => T
scala> def fact = Y {
| f: (Int => Int) =>
| n: Int =>
| if(n <= 0) 1
| else n * f(n - 1)}
fact: Int => Int
scala> println(fact(5))
java.lang.StackOverflowError
at .Y(<console>:11)
at .Y(<console>:11)
at .Y(<console>:11)
at .Y(<console>:11)
at .Y(<console>:11)
First of all, note that this is not a Y-combinator, since the lambda version of the function uses the free variable Y. It is the correct expression for Y though, just not a combinator.
So, let’s first put the part which computes the factorial into a separate function. We can call it comp:
def comp(f: Int => Int) =
(n: Int) => {
if (n <= 0) 1
else n * f(n - 1)
}
The factorial function can now be constructed like this:
def fact = Y(comp)
Q1:
Y is defined as func(Y(func)). We invoke fact(5) which is actually Y(comp)(5), and Y(comp) evaluates to comp(Y(comp)). This is the key point: we stop here because comp takes a function and it doesn’t evaluate it until needed. So, the runtime sees comp(Y(comp)) as comp(???) because the Y(comp) part is a function and will be evaluated only when (if) needed.
Do you know about call-by-value and call-by-name parameters in Scala? If you declare your parameter as someFunction(x: Int), it will be evaluated as soon as someFunction is invoked. But if you declare it as someFunction(x: => Int), then x will not be evaluated right away, but at the point where it is used. Second call is “call by name” and it is basically defining your x as a “function that takes nothing and returns an Int”. So if you pass in 5, you are actually passing in a function that returns 5. This way we achieve lazy evaluation of function parameters, because functions are evaluated at the point they are used.
So, parameter f in comp is a function, hence it is only evaluated when needed, which is in the else branch. That’s why the whole thing works - Y can create an infinite chain of func(func(func(func(…)))) but the chain is lazy. Each new link is computed only if needed.
So when you invoke fact(5), it will run through the body into the else branch and only at that point f will be evaluated. Not before. Since your Y passed in comp() as parameter f, we will dive into comp() again. In the recursive call of comp() we will be calculating the factorial of 4. We will then again go into the else branch of the comp function, thus effectively diving into another level of recursion (calculating factorial of 3). Note that in each function call your Y provided a comp as an argument to comp, but it is only evaluated in the else branch. Once we get to the level which calculates factorial of 0, the if branch will be triggered and we will stop diving further down.
Q2:
This
func(Y(func))(_:T)
is syntax sugar for this
x => func(Y(func))(x)
which means we wrapped the whole thing into a function. We didn’t lose anything by doing this, only gained.
What did we gain? Well, it’s the same trick as in the answer to a previous question; this way we achieve that func(Y(func)) will be evaluated only if needed since it’s wrapped in a function. This way we will avoid an infinite loop. Expanding a (single-paramter) function f into a function x => f(x) is called eta-expansion (you can read more about it here).
Here’s another simple example of eta-expansion: let’s say we have a method getSquare() which returns a simple square() function (that is, a function that calculates the square of a number). Instead of returning square(x) directly, we can return a function that takes x and returns square(x):
def square(x: Int) = x * x
val getSquare: Int => Int = square
val getSquare2: Int => Int = (x: Int) => square(x)
println(square(5)) // 25
println(getSquare(5)) // 25
println(getSquare2(5)) // 25
Hope this helps.
Complementing the accepted answer,
First of all, note that this is not a Y-combinator, since the lambda version of the function uses the free variable Y. It is the correct expression for Y though, just not a combinator.
A combinator isn't allowed to be explicitly recursive; it has to be a lambda expression with no free variables, which means that it can't refer to its own name in its definition. In the lambda calculus it is not possible to refer to the definition of a function in a function body. Recursion may only be achieved by passing in a function as a parameter.
Given this, I've copied the following implementation from rosetta code that uses some type trickery to implement Y combinator without explicit recursion. See here
def Y[A, B](f: (A => B) => (A => B)): A => B = {
case class W(wf: W => A => B) {
def get: A => B =
wf(this)
}
val g: W => A => B = w => a => f(w.get)(a)
g(W(g))
}
Hope this helps with the understanding.
I don't know the answer, but will try to guess. Since you have def Y[T](f: ...) = f(...) compiler can try to substitute Y(f) with simply f. This will create an infinite sequence of f(f(f(...))). Partially applying f you create a new object, and such substitution becomes impossible.
I got a following code:
trait Stream[+A] {
def uncons: Option[(A, Stream[A])]
def foldRight[B](z: => B)(f: (A, => B) => B): B = {
uncons.map(t => {
f(t._1, t._2.foldRight(z)(f))
}).getOrElse(z)
}
def exists(p: A => Boolean) =
foldRight(false)((x, acc) => acc || p(x))
def forAll(p: A => Boolean) =
foldRight(true)((x, acc) => p(x) && acc)
}
object Stream {
def cons[A](h: => A, t: => Stream[A]): Stream[A] =
new Stream[A] {
lazy val uncons = Some((h, t))
}
}
Then I create a Stream in a lazy manner and invoke exists method to check what stream elements were evaluated:
println(Stream.cons({println("5"); 1}, Stream.cons({println("6"); 2}, Stream.cons({println("7"); 3}, Stream.cons({println("8"); 4}, Stream.empty)))).exists(_ == 1))
And what I see is:
5
6
7
8
true
So all the elements were evaluated in spite of only first one would be enough. I seem to understand why exists acts the way it does.
Then I run the following code:
println(Stream.cons({println("13"); 1}, Stream.cons({println("14"); 2}, Stream.cons({println("15"); 3}, Stream.cons({println("16"); 4}, Stream.empty)))).forAll(_ < 2))
and see the following:
13
14
false
So as far as forAll comes across a non-satisfying value it terminates the traversal.
But why forAll acts that way? What's the crucial difference between it and exists?
There are two things to consider :
the type of acc
the order of p(x) in the boolean expression.
Laziness
If you change the type of acc to B, you won't be able to fail-fast (or short-circuit) in either of your methods. You must know it since your code extensively uses laziness, but a variable of type => B will get evaluated only when its value is required i.e. used in some expression. In this case, acc is the future of the result computed over the stream. This future will happen only if you try looking at it. Thus, to prevent the whole stream to be evaluated, you must prevent this future to be looked at.
Short-circuiting in boolean expressions
This is where the order of p(x) matters. In the expression a && b, if a is false then we know the whole conjunction is also false, thus Scala won't try evaluating b because it's pointless.
Combining the two
Now what happens if one of your operands is a lazy expression ? Well, if you have lazyA || b, Scala will read the expression from left to right and evaluate lazyA. In your case, lazyA represents the accumulation of the next element and the rest of the stream. Thus, lazyA expands to a0 :: lazyA1, which expands to a0 :: a1 :: lazyA2. You will therefore end up computing the whole stream just for computing the left part of your boolean binop.
Now, if you have a && lazyB, this expands to a && (b0 :: b1 :: lazyB2). As you see here, as soon as a or bi is false, this will return without evaluating the right part of the statement. This is what happens in your forAll.
How to fix it
The good news is that the fix is very easy : just swap the order of p(x) and acc : as soon as p(x) is true, the disjunction will return without evaluating acc, stopping the computation.
def exists(p: A => Boolean) = foldRight(false)((x, acc) => p(x) || acc)
Output :
5
true
I've a question regarding this pattern matching in scala:
val div: (Double, Double) => Double = {
case (x, y) if y != 0 => x / y
}
I've understand how pattern matching works and its syntaxis in scala, but this expression drives me crazy. How does complier knows that x and y is an arguments of the function and pattern match on them?
The rules for this are defined in section 8.5 "Pattern Matching Anonymous Functions" of the Scala Language Specification. If using an anonymous function with pattern matching, the type must be partially provided. You do that by saying the type is (Double, Double) => Double, which is shorthand for Function2[Double, Double, Double].
Now:
If the expected type is scala.Function k [S1,…,Sk, R], the expression is taken to be equivalent to the anonymous function:
(x1:S1,…,xk:Sk) => (x1,…,xk) match {
case p1 => b1 … case pn => bn
}
So no matter what the arity of your function, the pattern match is passed a tuple of the function's arguments, hence you can use the regular tuple extractor syntax.
So your example is short for
val div: (Double, Double) => Double = (a, b) => (a, b) match {
case (x, y) if y != 0 => x / y
}
or
val div = (a: Double, b: Double) => (a, b) match {
case (x, y) if y != 0 => x / y
}
The naming of the extractor parameters x and y is up to your imagination. You decide how to call the resulting elements of the extractor, you could as well write case (foo, bar) => ...
How do you use Map.foldLeft? According to the docs it looks like
foldLeft [B] (z: B)(op: (B, (A, B)) ⇒ B) : B
But I'm having difficulty:
Map("first"->1,"second"->2).foldLeft(0)((a,(k,v)) => a+v )
error: not a legal formal parameter
The error points to the open bracket in front of k.
If you want to use the (a, (k, v)) syntax, you need to advise the compiler to use pattern matching.
Map("first"->1, "second"->2).foldLeft(0){ case (a, (k, v)) => a+v }
Note that a case statement requires curly braces.
I think, you can't do the pattern match on tuples as you expect:
Map("first"->1,"second"->2).foldLeft(0)((a, t) => a + t._2)
Actually, using values and sum is simpler.
Map("first"->1,"second"->2).values.sum
The trick is to use a partial function as the code block, in other words you add a case statement that matches on the arguments:
Map("first" -> 1, "second" -> 2).foldLeft(0) { case (a, (k, v)) => a + v }
This is not really an answer to your question but I found it useful when starting out with folds, so I'll say it anyway! Note that the /: method "alias" for foldLeft can be clearer for two reasons:
xs.foldLeft(y) { (yy, x) => /* ... */ }
(y /: xs) { (yy, x) => /* ... */ }
Note that in the second line:
it's more clear that the value y is being folded into the collection xs
you can easily remember the ordering of the Tuple2 argument is the same as the ordering of the method "call"