Tail-recursive Implementation of Scala's List partition method - scala

For exercise purposes I've been trying to implement a couple of Scala's List methods in a functional manner, one of them being partition. Assume the following signature:
def partition[T](l: List[T], f: T => Boolean): (List[T], List[T])
It returns a tuple consisting of two lists - the first one contains all the elements from l that fulfill the passed predicate f and another one which contains all the other elements.
I came up with the following recursive solution which is unfortunately not tail-recursive:
def partition[T](l: List[T], f: T => Boolean): (List[T], List[T]) = {
l match {
case Nil => (Nil, Nil)
case head :: rest => {
val (left, right) = partition(rest, f)
if (f(head))
(head :: left, right)
else
(left, head :: right)
}
}
}
In this stack overflow question (Can all recursive functions be re-written as tail-recursions?) it is made clear that an accumulator could be used in some cases. In the given one I would claim that this is not possible since it would return the final lists in a reversed manner.
Could you please give me a tail-recursive solution? Maybe even with continuation passing (I haven't really understood how it works and how it could be applied)?

You can keep a tuple as the accumulator, and make sure to reverse the lists before returning them:
def partition[T](l: List[T])(f: T => Boolean): (List[T], List[T]) = {
#tailrec
def partitionInternal(l: List[T])(acc: (List[T], List[T])): (List[T], List[T]) = {
l match {
case Nil => acc
case head :: tail =>
if (f(head)) partitionInternal(tail)(head :: acc._1, acc._2)
else partitionInternal(tail)(acc._1, head :: acc._2)
}
}
val (lf, r) = partitionInternal(l)((List.empty[T], List.empty[T]))
(lf.reverse, r.reverse)
}
An alternative solution would be to append (:+) instead of prepending (::), but then you pay the price of O(n) for every entry.
Another idea would be to use a ListBuffer[T] instead of List[T] for the internal recursive implementation which has constant time append. All you need to do is call .toList at the end:
def partition[T](l: List[T])(f: T => Boolean): (List[T], List[T]) = {
#tailrec
def partitionInternal(l: List[T])(acc: (ListBuffer[T], ListBuffer[T])): (ListBuffer[T], ListBuffer[T]) = {
l match {
case Nil => acc
case head :: tail =>
val (leftAcc, rightAcc) = acc
if (f(head)) partitionInternal(tail)((leftAcc += head, rightAcc))
else partitionInternal(tail)((leftAcc, rightAcc += head))
}
}
val (lf, r) = partitionInternal(l)((ListBuffer.empty[T], ListBuffer.empty[T]))
(lf.toList, r.toList)
}
Additionaly, notice that I create a separate argument list for the List[T] and the function from T => Boolean. That is in order to help the compiler infer the right type argument when applying the method since type inference flows between parameter lists.

You need to keep two accumulators, one for left and one for right. When you're done going through the input list, simply return both accumulators (reversing them to get back to the original order):
def partition[T](l: List[T], f: T => Boolean): (List[T], List[T]) = {
#annotation.tailrec
def aux(tl: List[T], left: List[T], right: List[T]): (List[T], List[T]) = tl match {
case Nil => (left.reverse, right.reverse)
case head :: rest => {
if (f(head))
aux(rest, head :: left, right)
else
aux(rest, left, head :: right)
}
}
aux(l, List(), List())
}
Using it:
scala> def partition[T](l: List[T], f: T => Boolean): (List[T], List[T]) = {
| #annotation.tailrec
| def aux(tl: List[T], left: List[T], right: List[T]): (List[T], List[T]) = tl match {
| case Nil => (left.reverse, right.reverse)
| case head :: rest => {
| if (f(head))
| aux(rest, head :: left, right)
| else
| aux(rest, left, head :: right)
| }
| }
|
| aux(l, List(), List())
| }
partition: [T](l: List[T], f: T => Boolean)(List[T], List[T])
scala> partition(List(1, 2, 3, 4, 5), (i: Int) => i%2 == 0)
res1: (List[Int], List[Int]) = (List(2, 4),List(1, 3, 5))

Related

Scala Tail Recursion From a Flatmap

I have a recursive call as defined below:
def getElems[A](a: A)(f: A => List[A]): List[A] = {
f(a)
}
def parse[A](depth: Int, elems: List[A], f: A => List[A]): List[A] = {
elems.flatMap(elem => {
if (depth > 0) {
parse(depth - 1, getElems(elem)(f), f)
} else elems
})
}
As it can be seen that for every elem in the elems, I run a function that in turn gives me back another List. I do this until I reach the depth 0. So for example., I start with a certain elems and a certain depth like:
parse(depth = 2, elems = List("1", "2"), someFnThatGivesBackAListOfString)
What I'm doing with my code above is that for each element in elems, I check the depth value and if the depth is > 0, I run the function for that elem and go over the same process until I hit a depth of 0. This works as expected, but as it can be seen that it is not stack safe, I'm thiking of getting a tail recursive implementation. To my understanding tail recursion is about reduction, but here it is not the case. So how do I make it stack safe or how can I do a tail recursive logic here?
I started with something like this, but this is not quite right:
def firstAttempt[A](ls: List[A], depthOrig: Int)(f: (A => List[A])): List[A] = {
#annotation.tailrec
def helper(acc: List[A], ls: List[A], depth: Int): List[A] =
ls match {
case Nil => acc
case sublist # (head :: tail) =>
// Check if the entry is available in the bloom filter
if (depth > 0)
helper(acc ::: f(head), tail, depth - 1)
else
helper(acc.appended(head), tail, depthOrig)
}
helper(Nil, ls, depthOrig)
}
I got this to work by attaching the current depth to each element.
def parse[A](depth:Int, elems:List[A], f:A => List[A]): List[A] = {
#annotation.tailrec
def loop(todo:List[(A,Int)], acc:List[A]): List[A] = todo match {
case Nil => acc
case (_,dpth)::_ if dpth < 1 =>
val (zs, td) = todo.span(_._2 < 1)
loop(td, acc ++ zs.flatMap(_ => zs.map(_._1)))
case (elm,dpth)::tl =>
loop(f(elm).map(_ -> (dpth-1)) ++ tl, acc)
}
loop(elems.map(_ -> depth), Nil)
}

Scala cats trampoline

The test("ok") is copied from book "scala with cats" by Noel Welsh and Dave Gurnell pag.254 ("D.4 Safer Folding using Eval
"), the code run fine, it's the trampolined foldRight
import cats.Eval
test("ok") {
val list = (1 to 100000).toList
def foldRightEval[A, B](as: List[A], acc: Eval[B])(fn: (A, Eval[B]) => Eval[B]): Eval[B] =
as match {
case head :: tail =>
Eval.defer(fn(head, foldRightEval(tail, acc)(fn)))
case Nil =>
acc
}
def foldRight[A, B](as: List[A], acc: B)(fn: (A, B) => B): B =
foldRightEval(as, Eval.now(acc)) { (a, b) =>
b.map(fn(a, _))
}.value
val res = foldRight(list, 0L)(_ + _)
assert(res == 5000050000l)
}
The test("ko") returns same values of test("ok") for small list but for long list the value is different. Why?
test("ko") {
val list = (1 to 100000).toList
def foldRightSafer[A, B](as: List[A], acc: B)(fn: (A, B) => B): Eval[B] = as match {
case head :: tail =>
Eval.defer(foldRightSafer(tail, acc)(fn)).map(fn(head, _))
case Nil => Eval.now(acc)
}
val res = foldRightSafer(list, 0)((a, b) => a + b).value
assert(res == 5000050000l)
}
This is #OlegPyzhcov's comment, converted into a community wiki answer
You forgot the L in 0L passed as second argument to foldRightSafer.
Because of that, the inferred generic types of the invocation are
foldRightSafer[Int, Int]((list : List[Int]), (0: Int))((_: Int) + (_: Int))
and so your addition overflows and gives you something smaller than 2000000000 (9 zeroes, Int.MaxValue = 2147483647).

scala n-arity tree tail recursive evaluation

I have a Tree structure, which is more general than a binary tree structure
sealed trait Tree[+A]
case class Leaf[A](value: Terminal[A]) extends Tree[A]
case class Node[A](op: Function[A], branches: Tree[A]*) extends Tree[A]
As you see, it can have a arbitrary number of branches.
I'm trying to make an evaluation method to be tail recursive but i'm not being able to do it.
def evaluateTree[A](tree: Tree[A]): A = tree match {
case Leaf(terminal) => terminal.value
case Node(op: Function[A], args # _*) => op.operator((for (i <- args) yield evaluateTree(i)))
}
How can i save the stack manually?
If each Node can hold a different op then, no, I don't think tail recursion is possible.
If, on the other hand, you can feed all the Leaf.values to a single op then it might be possible.
def evaluateTree[A](tree: Tree[A]): A = {
#tailrec
def allValues(branches: Seq[Tree[A]], acc: Seq[A] = Seq()): Seq[A] =
if (branches.length < 1) acc
else branches.head match {
case Leaf(term) => allValues(branches.tail, term.value +: acc)
case Node(_, args: Seq[Tree[A]]) => allValues(branches.tail ++ args, acc)
}
tree match {
case Leaf(terminal) => terminal.value
case Node(op: Function[A], args: Seq[Tree[A]]) => op.operator(allValues(args))
}
}
I can't compile this as I don't have definitions for Terminal and Function, but it should be a reasonable outline of one approach to the problem.
Actually it was possible, using deep first search.
def evaluateTree[A](tree: Tree[A]): A = {
#tailrec
def evaluateWhile[C](l: List[Function[C]], arguments: List[List[C]], n_args: List[Int], f: Int => Boolean, acc: C): (List[Function[C]], List[List[C]], List[Int]) =
n_args match {
case h :: t if f(h) =>
evaluateWhile(l.tail, arguments.tail, n_args.tail, f, l.head.operator(arguments.head ::: List(acc)))
case h :: t =>
(l, (List(acc) ::: arguments.head) :: arguments.tail, List(n_args.head - 1) ::: n_args.tail)
case _ =>
(l, List(acc) :: arguments, n_args)
}
#tailrec
def DFS(toVisit: List[Tree[A]], visited: List[String] = Nil, operators: List[Function[A]] = Nil, arguments: List[List[A]] = Nil, n_args: List[Int] = Nil, debug: Int = 0): A = toVisit match {
case Leaf(id, terminal) :: tail if !visited.contains(id) => {
val (operators_to_pass, args_to_pass, n_args_to_pass) =
evaluateWhile[A](operators, arguments, n_args, x => x == 1, terminal.value)
DFS(toVisit.tail, visited ::: List(id), operators_to_pass, args_to_pass, n_args_to_pass, debug + 1)
}
case Node(id, op, args #_*) :: tail if !visited.contains(id) => {
DFS(args.toList ::: toVisit.tail, visited ::: List(id), op :: operators, List(Nil) ::: arguments, List(args.length ) ::: n_args, debug + 1)
}
case _ => arguments.flatten.head
}
DFS(List(tree))
}

Tail recursive functions for BinaryTree

I am stuck with implementing tail recursive foreach, reduce, map and toList functions for a very simple implementation of binary tree.
sealed trait Tree[+A]
case object EmptyTree extends Tree[Nothing]
case class Node[A](value: A, left: Tree[A], right: Tree[A]) extends Tree[A]
object Tree {
def apply[A]: Tree[A] = EmptyTree
def apply[A](value: A): Tree[A] = Node(value, EmptyTree, EmptyTree)
def apply[A](value: A, left: Tree[A], right: Tree[A]): Tree[A] = Node(value, left, right)
def foreach[A](tree: Tree[A], f: (A) => Unit): Unit = {
//#tailrec
def iter[A](tree: Tree[A], f: (A) => Unit): Unit = tree match {
case EmptyTree =>
case Node(v, l, r) =>
iter(l, f)
f(v)
iter(r, f)
}
iter(tree, f)
}
def reduce[A](tree: Tree[A], value: A, f: (A, A) => A): A = {
//#tailrec
def loop(tree: Tree[A], value: A): A = tree match {
case Node(v, l, r) => loop(l, f(loop(r, value), v))
case EmptyTree => value
}
loop(tree, value)
}
def map[A, B](tree: Tree[A], f: A => B): Tree[B] = {
//#tailrec
def iter[A](tree: Tree[A], f: A => B): Tree[B] = tree match {
case Node(v, l, r) => Node(f(v), iter(l, f), iter(r, f))
case EmptyTree => EmptyTree
}
iter(tree, f)
}
def toList[A](t: Tree[A]): List[A] = {
//#tailrec
def iter[A](t: Tree[A]): List[A] = t match {
case Node(v, l, r) => v :: iter(l) ::: iter(r)
case EmptyTree => List.empty
}
iter(t)
}
}
Code for testing:
val tree = Tree(1, Tree(2, Tree(3), Tree(4)), Tree(5, Tree(6), Tree(7)))
Tree.foreach(tree, (x: Int) => println(x))
Tree.reduce(tree, 0, (x: Int, y: Int) => x + y)
Tree.map(tree, (x: Int) => x + 1)
Tree.toList(tree)
I cant use #tailrec attribute because as you can see, recursive calls are not the last calls in a function, and I do not know how to rewrite it because there are several calls in one function, for example
v :: iter(l) ::: iter(r)
I know that I can use accumulator for inner recursive functions but how I should use it in case of several calls ?
Thanks in advance.
Updated:
def toListRec[A](tree: Tree[A]): List[A] = {
#tailrec
def iter(result: List[A], todo: List[Tree[A]]): List[A] = todo match {
case x :: tail => x match {
case Node(v, l, r) => iter(v :: result, l :: r :: tail)
case EmptyTree => iter(result, tail)
}
case Nil => result.reverse
}
iter(List.empty, List(tree))
}
Without tail recursion, a(/the) stack is used to keep track of calling functions. If you want to use tail recursion, you'll have to find a way to keep track of this information elsewhere. In simpler "linear" cases, such as factorial, this information is pretty limited and can often easily be taken care of by using an accumulator.
In your case, the problem is that the recursion isn't linear. After one recursive call, the function doesn't just compute the result, but it makes another recursive call before being able to get to the result.
In order to apply tail recursion in this case, you will have to explicitly keep track of the remaining recursive calls that have to be made. An easy way is to simply keep a "to-do" list. For example:
def toList[A](t: Tree[A]): List[A] = {
#tailrec def iter[A](todo: List[Tree[A]], r: List[A]): List[A] =
todo match {
case t :: rest => t match {
case Node(v, l, r) => iter(l :: r :: rest, v :: r)
case EmptyTree => iter(rest, r)
}
case List.empty => reverse(r)
}
iter(List(t), List.empty)
}
Disclaimer: I know nothing about scala. :)
The solution that mweerden suggests would work, however, there is another way of solving the problem, which I think is much more elegant. Here is the code which traverses a tree to list
def toList[T](t: Tree[T]): List[T] = {
def tailRecursive(tree: Tree[T], acc: List[T]): List[T] = tree match {
case EmptyTree => acc
case Node(value, right, left) =>
tailRecursive(left, value :: tailRecursive(right, acc))
}
tailRecursive(t, List())
}
The solution implies that the tree is a binary search tree, and the list produced will be in ascending order (if the ascending order is not required, 6th line can be changed, putting the value in front of first recursive call or straightly into the accumulator would be possible).

How can I make this Scala function (a "flatMap" variant) tail recursive?

I'm having a look at the following code
http://aperiodic.net/phil/scala/s-99/p26.scala
Specifically
def flatMapSublists[A,B](ls: List[A])(f: (List[A]) => List[B]): List[B] =
ls match {
case Nil => Nil
case sublist#(_ :: tail) => f(sublist) ::: flatMapSublists(tail)(f)
}
I'm getting a StackOverflowError for large values presumably because the function is not tail recursive. Is there a way to transform the function to accommodate large numbers?
It is definitely not tail recursive. The f(sublist) ::: is modifying the results of the recursive call, making it a plain-old-stack-blowing recursion instead of a tail recursion.
One way to ensure that your functions are tail recursive is to put the #annotation.tailrec on any function that you expect to be tail recursive. The compiler will report an error if it fails to perform the tail call optimization.
For this, I would add a small helper function that's actually tail recursive:
def flatMapSublistsTR[A,B](ls: List[A])(f: (List[A]) => List[B]): List[B] = {
#annotation.tailrec
def helper(r: List[B], ls: List[A]): List[B] = {
ls match {
case Nil => r
case sublist#(_ :: tail) => helper(r ::: f(sublist), tail)
}
}
helper(Nil, ls)
}
For reasons not immediately obvious to me, the results come out in a different order than the original function. But, it looks like it works :-) Fixed.
Here is another way to implement the function:
scala> def flatMapSublists[A,B](ls: List[A])(f: (List[A]) => List[B]): List[B] =
| List.iterate(ls, ls.size)(_.tail).flatMap(f)
flatMapSublists: [A, B](ls: List[A])(f: List[A] => List[B])List[B]
A simply comparison between dave's flatMapSublistsTR and mine:
scala> def time(count: Int)(call : => Unit):Long = {
| val start = System.currentTimeMillis
| var cnt = count
| while(cnt > 0) {
| cnt -= 1
| call
| }
| System.currentTimeMillis - start
| }
time: (count: Int)(call: => Unit)Long
scala> val xs = List.range(0,100)
scala> val fn = identity[List[Int]] _
fn: List[Int] => List[Int] = <function1>
scala> time(10000){ flatMapSublists(xs)(fn) }
res1: Long = 5732
scala> time(10000){ flatMapSublistsTR(xs)(fn) }
res2: Long = 347232
Where the method flatMapSublistsTR is implemented as:
def flatMapSublistsTR[A,B](ls: List[A])(f: (List[A]) => List[B]): List[B] = {
#annotation.tailrec
def helper(r: List[B], ls: List[A]): List[B] = {
ls match {
case Nil => r
case sublist#(_ :: tail) => helper(r ::: f(sublist), tail)
}
}
helper(Nil, ls)
}
def flatMapSublists2[A,B](ls: List[A], result: List[B] = Nil)(f: (List[A]) => List[B]): List[B] =
ls match {
case Nil => result
case sublist#(_ :: tail) => flatMapSublists2(tail, result ++ f(sublist))(f)
}
You generally just need to add a result result parameter to carry from one iteration to the next, and spit out the result at the end instead of adding the end to the list.
Also that confusting sublist# thing can be simplified to
case _ :: tail => flatMapSublists2(tail, result ++ f(ls))(f)
Off-topic: here's how I solved problem 26, without the need for helper methods like the one above. If you can make this tail-recursive, have a gold star.
def combinations[A](n: Int, lst: List[A]): List[List[A]] = n match {
case 1 => lst.map(List(_))
case _ => lst.flatMap(i => combinations (n - 1, lst.dropWhile(_ != i).tail) map (i :: _))
}