I'm working on a Functional Programming in Scala exercise to implement foldRight on a Stream.
Before getting to that exercise, let me show how I've implemented foldLeft.
def foldLeft[A, B](as: Stream[A])(z: B)(f: (B, A) => B): B = {
def go(bs: Stream[A], acc: B) : B = bs match {
case x #:: xs => go(xs, f(acc, x))
case Stream() => acc
}
}
My understanding is that, via #::, I'm executing the fold in a tail-recursive and stream-y fashion, i.e. the tail isn't fully evaluated.
However, when I thought of how to implement foldRight, I figured I could simply do:
stream.reverse.foldLeft(monoid.zero)(monoid.op)
However, calling reverse will result in full evaluation of the stream:
scala> val x = Stream(1,2,3)
x: scala.collection.immutable.Stream[Int] = Stream(1, ?)
scala> x.reverse
res15: scala.collection.immutable.Stream[Int] = Stream(3, 2, 1)
How can I stream-ily implement foldRight?
Related
I am having trouble understanding the solution to the functional programming exercise:
Implement flatMap using only foldRight, Nil and :: (cons).
The solution is as follows:
def flatMap[A, B](xs: List[A])(f: A => List[B]): List[B] =
xs.foldRight(List[B]())((outCurr, outAcc) =>
f(outCurr).foldRight(outAcc)((inCurr, inAcc) => inCurr :: inAcc))
I have tried to factor out anonymous functions into function definitions to rewrite the solution to no luck. I cannot understand what is happening or think of a way to break it down so it's less complicated. So, any help or explanation regarding the solution would be appreciated.
Thanks!
First, just ignore the constraints and think about the flatMap function in this case. You have a List[A] and a function f: A => List[B]. Normally, if you just do a map on the list and apply the f function, you'll get back a List[List[B]], right? So to get a List[B], what would you do? You would foldRight on the List[List[B]] to get back a List[B] by just appending all elements in the List[List[B]]. So the code will look somewhat like this:
def flatMap[A, B](xs: List[A])(f: A => List[B]): List[B] = {
val tmp = xs.map(f) // List[List[B]]
tmp.foldRight(List[B]())((outCurr, outAcc) => outCurr ++ outAcc)
}
To verify what we have so far, running the code in REPL and verify the result against built-in flatMap method:
scala> def flatMap[A, B](xs: List[A])(f: A => List[B]): List[B] = {
| val tmp = xs.map(f) // List[List[B]]
| tmp.foldRight(List[B]())((outCurr, outAcc) => outCurr ++ outAcc)
| }
flatMap: [A, B](xs: List[A])(f: A => List[B])List[B]
scala> flatMap(List(1, 2, 3))(i => List(i, 2*i, 3*i))
res0: List[Int] = List(1, 2, 3, 2, 4, 6, 3, 6, 9)
scala> List(1,2,3).flatMap(i => List(i, 2*i, 3*i))
res1: List[Int] = List(1, 2, 3, 2, 4, 6, 3, 6, 9)
OK, so now, look at our constraints, we are not allowed to use map here. But we don't really need to, because the map here is just for iterating through the list xs. We can then use foldRight for this same purpose. So let's rewrite the map part using foldRight:
def flatMap[A, B](xs: List[A])(f: A => List[B]): List[B] = {
val tmp = xs.foldRight(List[List[B]]())((curr, acc) => f(curr) :: acc) // List[List[B]]
tmp.foldRight(List[B]())((outCurr, outAcc) => outCurr ++ outAcc)
}
OK, let's verify the new code:
scala> def flatMap[A, B](xs: List[A])(f: A => List[B]): List[B] = {
| val tmp = xs.foldRight(List[List[B]]())((curr, acc) => f(curr) :: acc) // List[List[B]]
| tmp.foldRight(List[B]())((outCurr, outAcc) => outCurr ++ outAcc)
| }
flatMap: [A, B](xs: List[A])(f: A => List[B])List[B]
scala> flatMap(List(1, 2, 3))(i => List(i, 2*i, 3*i))
res3: List[Int] = List(1, 2, 3, 2, 4, 6, 3, 6, 9)
OK, so far so good. So let's optimize the code a bit, which is instead of having two foldRight in sequential, we'll combine them into just one foldRight. That shouldn't be too hard:
def flatMap[A, B](xs: List[A])(f: A => List[B]): List[B] = {
xs.foldRight(List[B]()) { (curr, acc) => // Note: acc is List[B]
val tmp2 = f(curr) // List[B]
tmp2 ++ acc
}
}
Verify again:
scala> def flatMap[A, B](xs: List[A])(f: A => List[B]): List[B] = {
| xs.foldRight(List[B]()) { (curr, acc) => // Note: acc is List[B]
| val tmp2 = f(curr) // List[B]
| tmp2 ++ acc
| }
| }
flatMap: [A, B](xs: List[A])(f: A => List[B])List[B]
scala> flatMap(List(1, 2, 3))(i => List(i, 2*i, 3*i))
res4: List[Int] = List(1, 2, 3, 2, 4, 6, 3, 6, 9)
OK, so let's look at our constraints, it looks like we can't use ++ operation. Well, ++ is just a way to append the two List[B] together, so we can certainly achieve the same thing using foldRight method, like this:
def flatMap[A, B](xs: List[A])(f: A => List[B]): List[B] = {
xs.foldRight(List[B]()) { (curr, acc) => // Note: acc is List[B]
val tmp2 = f(curr) // List[B]
tmp2.foldRight(acc)((inCurr, inAcc) => inCurr :: inAcc)
}
}
And then, we can combine them all into one line by:
def flatMap[A, B](xs: List[A])(f: A => List[B]): List[B] =
xs.foldRight(List[B]())((curr, acc) =>
f(curr).foldRight(acc)((inCurr, inAcc) => inCurr :: inAcc))
Isn't it the given answer :)
Sometimes it is easier to understand with a simple example:
Suppose we have a val xs = List[Int](1,2,3) and a function f: Int => List[Int], f(x) = List(x,x) (lambda x => List(x,x))
Appling f to each element of xs List(f(1),f(2), f(3)) will result in List(List(1,1),List(2,2),List(3,3)) so we need to flatten out this List[List[Int]]. The final result should be List(1, 1, 2, 2, 3, 3). Given Cons (::) the constructor of nonempty List, this should be Cons(1, Cons(1, Cons(2, Cons(2, Cons(3, Cons(3, Nil)))))). Observe foldRight operation in the result, which applies the constructor Cons (::) to the result of f applied to each element of the list xs. So a first implementation of flatMap would be
def flatMap[A,B](xs: List[A])(f: A => List[B]): List[B] = xs match {
case Cons(head, tail) => foldRight(f(head), Nil)((a,b) => Cons(a,b))
}
In this form, flatMap(List(1,2,3)) will return List(1,1) or Cons(1,1,Nil) (by substitution). So we need to continue down with a recursive call to flatMap on the tail (reducing the problem by 1 from 3(elements) to 2(elements)) and adding the base case for the case of an empty List as Nil (Nil is the "zero" element for Cons operation)
def flatMap[A,B](xs: List[A])(f: A => List[B]): List[B] = xs match {
case Nil => Nil
case Cons(head, tail) => foldRight(f(head), flatMap(tail)(f))((a,b) => Cons(a,b))
}
which is the final implementation.
In Scala, I wrote a Higher Order Function that iterates a list, and calls back with the head and list, for each element, that is:
def headAndSelf[A, B](li: List[A], f: (A, List[A]) => B): List[B] = {
if (li == Nil) Nil
else f(li.head, li) :: headAndSelf(li.tail, f)
}
Given val a = List(1,2,3)
scala> headAndSelf(a, (x: Int, y: List[Int]) => (x,y))
res4: List[(Int, List[Int])] = List((1,List(1, 2, 3)), (2,List(2, 3)), (3,List(3)))
It then occurred to me I could get the head from the list, so it was simpler to write:
def self[A, B](li: List[A], f: (List[A]) => B): List[B] = {
if (li == Nil) Nil
else f(li) :: self(li.tail, f)
}
(even though it makes the lambda a bit less elegant)
scala> self(a, (x: List[Int]) => x)
res7: List[List[Int]] = List(List(1, 2, 3), List(2, 3), List(3))
Then I thought, surely there's an even easier, idiomatic way to turn a List into a List of Lists. So what is it?
Bonus Question: why can't Scala infer the type of the lambda? a is a List[Int], so x should be a List[Int] too, no?:
scala> self(a, x => x)
<console>:13: error: missing parameter type
self(a, x => x)
^
tails will do most of the job.
tails will have an empty list as its last element, and you need to filter that out, which is what collect does below:
def headAndSelf[A,B](l: List[A])(f: (A, List[A]) => B) =
l.tails.collect{case list # head :: _ => f(head, list)}.toList
Having two parameters lists as demonstrated here will allow proper type inference. You need two list at call site too, but it is often convenient :
headAndSelf(yourList){(h,l) => whatever}
Here is a recursive version of append for two lists:
def append[A](x: List[A], y: List[A]): List[A] = x match {
case Nil => y
case h :: t => h :: append(t, y)
}
How to convert it to tail-recursive version?
Try this, though x is prepended in reverse order. is that intended?
import scala.annotation.tailrec
#tailrec
def append[A](x: List[A], y: List[A]): List[A] = x match {
case Nil => y
case h :: t => append(t, h :: y)
}
If you want to prepend x in the order it comes with, you'ld have to do something like this:
import scala.annotation.tailrec
def append[A](x: List[A], y: List[A]): List[A] = {
#tailrec
def innerAppend[A](x: List[A], y: List[A]): List[A] = x match {
case Nil => y
case h :: t => innerAppend(t, h :: y)
}
innerAppend(x.reverse, y)
}
The Scala Standard Library's foldLeft method for sequences is a tail-recursive method that allows for the creation of many core sequence operations which apply a binary operation between their elements.
Using foldLeft, here's how you'd implement your append method:
def append[A](x: List[A], y: List[A]): List[A] =
x.reverse.foldLeft(y)((t, h) => h :: t) //Note the swapped order for h and t
which will return an appended list in the same order as given with the inclusion of reverse:
append(List(1,2,3), List(4,5,6))
res0: List[Int] = List(1, 2, 3, 4, 5, 6)
The lack of #tailrec annotation is not meant to suggest that this is not tail-recursive; rather, #tailrec just won't detect properly when you are implementing a method or function through a tail-recursive method like foldLeft.
Unlike foldLeft, foldRight from the Standard Library will not create a tail-recursive method, though its syntax is easier to understand than having to reverse the input list and swap the parameters. That's why I prefer implementing a foldRightViaFoldLeft method, as it has a more user-friendly syntax, and keeps the tail-recursive properties:
def foldRVL[A,B](l: List[A], z: B)(f: (A,B) => B): B = //"RVL" == "Right Via Left"
l.reverse.foldLeft(z)((b, a) => f(a, b)) //Note again that we are swapping a and b
def append[A](x: List[A], y: List[A]): List[A] =
foldRVL(x, y)(_ :: _) //the swap in foldRVL means that we don't have swap when using it
This is still tail-recursive, but is now easier to read. There are many, many uses for foldRVL, and other fold methods. Use them wherever you can. For more information about folds, and a better explanation about their recursive properties, I recommend reading this post: Scala Code Review: foldLeft and foldRight.
I'd also highly recommend the book, Functional Programming in Scala, from which I learned how to properly fold things.
As part of an exercise from FP in Scala, I'm working on the implementation of foldLeft on an IndexedSeq.
I wrote 2 functions:
def foldLeft[A, B](as: IndexedSeq[A])(z: B)(f: (B, A) => B): B = {
def go(bs: IndexedSeq[A], acc: B): B = {
if (bs.isEmpty) acc
else go(bs.tail, f(acc, bs.head))
}
go(as, z)
}
And, then the pattern match way:
def foldLeftPM[A, B](as: IndexedSeq[A])(z: B)(f: (B, A) => B): B = {
def go(bs: IndexedSeq[A], acc: B): B = bs match {
case x +: xs => go(xs, f(acc, x))
case _ => acc
}
go(as, z)
}
EDIT Note that I got the +: operator from dhgs's answer. It appears to be a member of IndexedSeq's class or its parent since it's available without defining per the linked post.
Is either way better (from a performance or idiomatic Scala point of view)?
The pattern match is definitely more idiomatic.
For performance, they should be about the same, since they are exactly equivalent.
Though only benchmarking would decide, and that includes a lot of assumptions.
I feel insertSortRight is less efficient than insertSortLeft because insertSortRight needs to call List.last (which is O(n)) as one of the arguments to insert(), where insertSortLeft calls List.head (which is O(1)) as one of the arguments to insert().
Is this understanding correct? Thanks.
def insertSortRight(unsorted: List[Int]) : List[Int] = {
(unsorted :\ List[Int]()) ((a, b) => insert(a, b))
}
def insertSortLeft(unsorted: List[Int]) : List[Int] = {
(List[Int]() /: unsorted) ((a, b) => insert(b, a))
}
def insert(a: Int, list: List[Int]) : List[Int] = list match {
case List() => List(a)
case y::ys => if (a > y) y::insert(a, ys) else a::y::ys
}
DHG answered "always prefer left folding". But, Programming in Scala has an example the other way.
def flattenLeft[T](xss: List[List[T]]) = (List[T]() /: xss) (_ ::: )
def flattenRight[T](xss: List[List[T]]) = (xss :~List[T]()) ( ::: _)
I guess that is because flattenRight in this case is achieved by just one function call, while flattenLeft is achieved by n function call?
So, for a List, since head operations are desired, foldLeft is the natural choice. That way you work through the list from left to right, always taking the head. As you can see, its implementation (on LinearSeqOptimized) simply uses a while-loop and traverses once.
override /*TraversableLike*/
def foldLeft[B](z: B)(f: (B, A) => B): B = {
var acc = z
var these = this
while (!these.isEmpty) {
acc = f(acc, these.head)
these = these.tail
}
acc
}
It seems like 'foldRight' would be O(n^2) since, in order to take the last element, you have to traverse the n elements of the List n times, but the library actually optimizes this for you. Behind the scenes, foldRight is implemented like this (also on LinearSeqOptimized):
def foldRight[B](z: B)(f: (A, B) => B): B =
if (this.isEmpty) z
else f(head, tail.foldRight(z)(f))
As you can see, this function is constructed by recursively calling foldRight on the tail, holding each head on the stack, and applying the function to each head in reverse order after reaching the last element.