I have a node tree:
val root = Node("a", List(Node("w", Nil), Node("b", List(Node("c", List(Node("d", Nil))), Node("m", List(Node("n", Nil)))))))
and list of intermediate nodes:
val ks = List("b", "m")
that leads me to where which I have to add a node:
val child = Node("t", Nil)
that results in:
val res = Node("a", List(Node("w", Nil), Node("b", List(Node("c", List(Node("d", Nil))), Node("m", List(Node("n", Nil), Node("t", Nil)))))))
Tree in Scala is modeled as:
sealed trait Tree[A]
case class Empty[A]() extends Tree[A]
case class Node[A](rootNode: A, subtree: List[Tree[A]]) extends Tree[A]
I have made an attempt to traverse the tree and add the node like so,
#tailrec
def traverseAndAdd(root: Node[String], subNode: Node[String], child: Node[String], ls: List[String]): Node[String] = ls match {
case Nil =>
root
case x :: Nil =>
val k = subNode.subtree.filter(l => l.asInstanceOf[Node[String]].rootNode == x).head.asInstanceOf[Node[String]].subtree :+ child
println(k)
traverseAndAdd(root, subNode, child, Nil)
case x :: xs =>
traverseAndAdd(root, subNode.subtree.filter(l => l.asInstanceOf[Node[String]].rootNode == x).head.asInstanceOf[Node[String]], child, xs)
}
val root = Node("a", List(Node("w", Nil), Node("b", List(Node("c", List(Node("d", Nil))), Node("m", List(Node("n", Nil)))))))
val child = Node("t", Nil)
val ks = List("b", "m")
println("res: " + traverseAndAdd(root, root, child, ks))
I understand that here the list is immutable and even the code doesn't make any attempt to build the node ground up. I am asking for a suggestion on how can I achieve such a thing in Scala. Please help.
Also, I have already attempted the problem to which this problem is a sub problem using normal (not tail) recursion but I ended up with stackoverflow for the data is that huge. So thought of this approach!
I noticed a few shortcomings in your code.
class Empty is defined but never used.
val k is populated but never used.
Your data structure does not require uniqueness so a given Node path is not guaranteed to be unique.
Here's what I was able to construct. It's not tail recursive, nor does it address the uniqueness issue, but it appears to work.
case class Node[A](elem: A, subNodes: List[Node[A]])
def addNode[T](node: Node[T], path: List[T]): Node[T] = {
if (path.isEmpty)
node
else if (path.head == node.elem)
addNode(node, path.tail)
else if (node.subNodes.map(_.elem).contains(path.head))
Node(node.elem, node.subNodes.map{n =>
if (n.elem == path.head) addNode(n,path.tail)
else n
})
else
Node(node.elem
,node.subNodes :+ addNode(Node(path.head, List[Node[T]]()), path.tail))
}
val root = Node("a", List(Node("w", Nil)
,Node("b", List(Node("c", List(Node("d", Nil)))
,Node("m", List(Node("n", Nil)))))))
addNode(root,List("a","b","m","n","t"))
Related
Suppose I have a tree data structure like this:
trait Node { val name: String }
case class BranchNode(name: String, children: List[Node]) extends Node
case class LeafNode(name: String) extends Node
Suppose also I've got a function to map over leaves:
def mapLeaves(root: Node, f: LeafNode => LeafNode): Node = root match {
case ln: LeafNode => f(ln)
case bn: BranchNode => BranchNode(bn.name, bn.children.map(ch => mapLeaves(ch, f)))
}
Now I am trying to make this function tail-recursive but having a hard time to figure out how to do it. I've read this answer but still don't know to make that binary tree solution work for a multiway tree.
How would you rewrite mapLeaves to make it tail-recursive?
"Call stack" and "recursion" are merely popular design patterns that later got incorporated into most programming languages (and thus became mostly "invisible"). There is nothing that prevents you from reimplementing both with heap data structures. So, here is "the obvious" 1960's TAOCP retro-style solution:
trait Node { val name: String }
case class BranchNode(name: String, children: List[Node]) extends Node
case class LeafNode(name: String) extends Node
def mapLeaves(root: Node, f: LeafNode => LeafNode): Node = {
case class Frame(name: String, mapped: List[Node], todos: List[Node])
#annotation.tailrec
def step(stack: List[Frame]): Node = stack match {
// "return / pop a stack-frame"
case Frame(name, done, Nil) :: tail => {
val ret = BranchNode(name, done.reverse)
tail match {
case Nil => ret
case Frame(tn, td, tt) :: more => {
step(Frame(tn, ret :: td, tt) :: more)
}
}
}
case Frame(name, done, x :: xs) :: tail => x match {
// "recursion base"
case l # LeafNode(_) => step(Frame(name, f(l) :: done, xs) :: tail)
// "recursive call"
case BranchNode(n, cs) => step(Frame(n, Nil, cs) :: Frame(name, done, xs) :: tail)
}
case Nil => throw new Error("shouldn't happen")
}
root match {
case l # LeafNode(_) => f(l)
case b # BranchNode(n, cs) => step(List(Frame(n, Nil, cs)))
}
}
The tail-recursive step function takes a reified stack with "stack frames". A "stack frame" stores the name of the branch node that is currently being processed, a list of child nodes that have already been processed, and the list of the remaining nodes that still must be processed later. This roughly corresponds to an actual stack frame of your recursive mapLeaves function.
With this data structure,
returning from recursive calls corresponds to deconstructing a Frame object, and either returning the final result, or at least making the stack one frame shorter.
recursive calls correspond to a step that prepends a Frame to the stack
base case (invoking f on leaves) does not create or remove any frames
Once one understands how the usually invisible stack frames are represented explicitly, the translation is straightforward and mostly mechanical.
Example:
val example = BranchNode("x", List(
BranchNode("y", List(
LeafNode("a"),
LeafNode("b")
)),
BranchNode("z", List(
LeafNode("c"),
BranchNode("v", List(
LeafNode("d"),
LeafNode("e")
))
))
))
println(mapLeaves(example, { case LeafNode(n) => LeafNode(n.toUpperCase) }))
Output (indented):
BranchNode(x,List(
BranchNode(y,List(
LeafNode(A),
LeafNode(B)
)),
BranchNode(z, List(
LeafNode(C),
BranchNode(v,List(
LeafNode(D),
LeafNode(E)
))
))
))
It might be easier to implement it using a technique called trampoline.
If you use it, you'd be able to use two functions calling itself doing mutual recursion (with tailrec, you are limited to one function). Similarly to tailrec this recursion will be transformed to plain loop.
Trampolines are implemented in scala standard library in scala.util.control.TailCalls.
import scala.util.control.TailCalls.{TailRec, done, tailcall}
def mapLeaves(root: Node, f: LeafNode => LeafNode): Node = {
//two inner functions doing mutual recursion
//iterates recursively over children of node
def iterate(nodes: List[Node]): TailRec[List[Node]] = {
nodes match {
case x :: xs => tailcall(deepMap(x)) //it calls with mutual recursion deepMap which maps over children of node
.flatMap(node => iterate(xs).map(node :: _)) //you can flat map over TailRec
case Nil => done(Nil)
}
}
//recursively visits all branches
def deepMap(node: Node): TailRec[Node] = {
node match {
case ln: LeafNode => done(f(ln))
case bn: BranchNode => tailcall(iterate(bn.children))
.map(BranchNode(bn.name, _)) //calls mutually iterate
}
}
deepMap(root).result //unwrap result to plain node
}
Instead of TailCalls you could also use Eval from Cats or Trampoline from scalaz.
With that implementation function worked without problems:
def build(counter: Int): Node = {
if (counter > 0) {
BranchNode("branch", List(build(counter-1)))
} else {
LeafNode("leaf")
}
}
val root = build(4000)
mapLeaves(root, x => x.copy(name = x.name.reverse)) // no problems
When I ran that example with your implementation it caused java.lang.StackOverflowError as expected.
Given the following data model, where elements record their relations to ancestors by a delimited string path:
case class Entity(id:String, ancestorPath:Option[String] = None)
val a = Entity("a")
val c = Entity("c")
val b = Entity("b", Some("a"))
val d = Entity("d", Some("a/b"))
val test = a :: c :: b :: d :: Nil
What would be a good way to process the relationship into a nested structure such as:
case class Entity2(id:String, children:List[Entity])
The desired function would output a list of Entity2s that nest their children. The elements themselves would therefore be the root nodes. We can assume the input list is sorted lexigraphically by the value of parentId and a None is considered earlier than a list, exactly as test is above.
Example desired output:
List(
Entity2("a", List(
Entity2("b", List(
Entity2("d", Nil)
),
),
Entity2("c", Nil)
)
I've had a few tries at it but what's tripping me up is finding a good way to invert the relationship as you go... recursing the Entity classes gives you the backward (my parent/ancestors is/are) reference whereas the desired output records the forward (my children are) reference. Thanks!
One straight-forward solution looks like:
case class Entity(id:String, ancestorPath:Option[String] = None)
case class Entity2(id:String, children:List[Entity2])
object Main {
def main(args: Array[String]) {
val a = Entity("a")
val c = Entity("c")
val b = Entity("b", Some("a"))
val d = Entity("d", Some("a/b"))
val test = a :: c :: b :: d :: Nil
println(buildTree(test))
}
def immediateParent(path: String) = {
val pos = path.lastIndexOf('/')
if (pos == -1) path
else path.substring(pos+1)
}
def buildTree(all: List[Entity]): List[Entity2] = {
val childEntitiesByParentId = all.groupBy(_.ancestorPath.map(immediateParent _))
val roots = childEntitiesByParentId.getOrElse(None, Nil)
roots.map({ root => buildTreeHelper(root, childEntitiesByParentId) })
}
def buildTreeHelper(
parent: Entity,
childEntitiesByParentId: Map[Option[String], List[Entity]]): Entity2 = {
val children = childEntitiesByParentId.getOrElse(Some(parent.id), Nil).map({ child =>
buildTreeHelper(child, childEntitiesByParentId)
})
Entity2(parent.id, children)
}
}
If your trees are very deep you will blow the stack - trampolines are a good solution:
import scala.util.control.TailCalls
def buildTree(all: List[Entity]): List[Entity2] = {
val childEntitiesByParentId = all.groupBy(_.ancestorPath.map(immediateParent _))
val roots = childEntitiesByParentId.getOrElse(None, Nil)
buildTreeHelper(roots, childEntitiesByParentId).result
}
def buildTreeHelper(
parents: List[Entity],
childEntitiesByParentId: Map[Option[String], List[Entity]]): TailCalls.TailRec[List[Entity2]] = {
parents match {
case Nil => TailCalls.done(Nil)
case parent :: tail =>
val childEntities = childEntitiesByParentId.getOrElse(Some(parent.id), Nil)
for {
children <- TailCalls.tailcall(buildTreeHelper(childEntities, childEntitiesByParentId))
siblings <- buildTreeHelper(tail, childEntitiesByParentId)
} yield Entity2(parent.id, children) :: siblings
}
}
Start with an empty list, and incrementally build this up by adding one entity at a time to it. Each time you add an entity, you have to inspect its ancestor path, and then traverse the corresponding path in the new list to insert the entity in the correct location. The destination list is really a tree structure, since you have nested components; you just need to find the correct location in the tree to insert into.
It will be more efficient if you use maps instead of lists, but should be possible either way. You may find it easier to build the results if you use mutable structures, but should be possible either way as well.
My list looks like the following: List(Person,Invite,Invite,Person,Invite,Person...). I am trying to match based on a inviteCountRequired, meaning that the Invite objects following the Person object in the list is variable. What is the best way of doing this? My match code so far looks like this:
aList match {
case List(Person(_,_,_),Invitee(_,_,_),_*) => ...
case _ => ...
}
First stack question, please go easy on me.
Let
val aList = List(Person(1), Invite(2), Invite(3), Person(2), Invite(4), Person(3), Invite(6), Invite(7))
Then index each location in the list and select Person instances,
val persons = (aList zip Stream.from(0)).filter {_._1.isInstanceOf[Person]}
namely, List((Person(1),0), (Person(2),3), (Person(3),5)) . Define then sublists where the lower bound corresponds to a Person instance,
val intervals = persons.map{_._2}.sliding(2,1).toArray
res31: Array[List[Int]] = Array(List(0, 3), List(3, 5))
Construct sublists,
val latest = aList.drop(intervals.last.last) // last Person and Invitees not paired
val associations = intervals.map { case List(pa,pb,_*) => b.slice(pa,pb) } ++ latest
Hence the result looks like
Array(List(Person(1), Invite(2), Invite(3)), List(Person(2), Invite(4)), List(Person(3), Invite(6), Invite(7)))
Now,
associations.map { a =>
val person = a.take(1)
val invitees = a.drop(1)
// ...
}
This approach may be seen as a variable size sliding.
Thanks for your tips. I ended up creating another case class:
case class BallotInvites(person:Person,invites:List[Any])
Then, I populated it from the original list:
def constructBallotList(ballots:List[Any]):List[BallotInvites] ={
ballots.zipWithIndex.collect {
case (iv:Ballot,i) =>{
BallotInvites(iv,
ballots.distinct.takeRight(ballots.distinct.length-(i+1)).takeWhile({
case y:Invitee => true
case y:Person =>true
case y:Ballot => false})
)}
}}
val l = Ballot.constructBallotList(ballots)
Then to count based on inviteCountRequired, I did the following:
val count = l.count(b=>if ((b.invites.count(x => x.isInstanceOf[Person]) / contest.inviteCountRequired)>0) true else false )
I am not sure I understand the domain but you should only need to iterate once to construct a list of person + invites tuple.
sealed trait PorI
case class P(i: Int) extends PorI
case class I(i: Int) extends PorI
val l: List[PorI] = List(P(1), I(1), I(1), P(2), I(2), P(3), P(4), I(4))
val res = l.foldLeft(List.empty[(P, List[I])])({ case (res, t) =>
t match {
case p # P(_) => (p, List.empty[I]) :: res
case i # I(_) => {
val head :: tail = res
(head._1, i :: head._2) :: tail
}
}
})
res // List((P(4),List(I(4))), (P(3),List()), (P(2),List(I(2))), (P(1),List(I(1), I(1))))
I am attempting to return a list of widgets from an N-tree data structure. In my unit test, if i have roughly about 2000 widgets each with a single dependency, i'll encounter a stack overflow. What I think is happening is the for loop is causing my tree traversal to not be tail recursive. what's a better way of writing this in scala? Here's my function:
protected def getWidgetTree(key: String) : ListBuffer[Widget] = {
def traverseTree(accumulator: ListBuffer[Widget], current: Widget) : ListBuffer[Widget] = {
accumulator.append(current)
if (!current.hasDependencies) {
accumulator
} else {
for (dependencyKey <- current.dependencies) {
if (accumulator.findIndexOf(_.name == dependencyKey) == -1) {
traverseTree(accumulator, getWidget(dependencyKey))
}
}
accumulator
}
}
traverseTree(ListBuffer[Widget](), getWidget(key))
}
The reason it's not tail-recursive is that you are making multiple recursive calls inside your function. To be tail-recursive, a recursive call can only be the last expression in the function body. After all, the whole point is that it works like a while-loop (and, thus, can be transformed into a loop). A loop can't call itself multiple times within a single iteration.
To do a tree traversal like this, you can use a queue to carry forward the nodes that need to be visited.
Assume we have this tree:
// 1
// / \
// 2 5
// / \
// 3 4
Represented with this simple data structure:
case class Widget(name: String, dependencies: List[String]) {
def hasDependencies = dependencies.nonEmpty
}
And we have this map pointing to each node:
val getWidget = List(
Widget("1", List("2", "5")),
Widget("2", List("3", "4")),
Widget("3", List()),
Widget("4", List()),
Widget("5", List()))
.map { w => w.name -> w }.toMap
Now we can rewrite your method to be tail-recursive:
def getWidgetTree(key: String): List[Widget] = {
#tailrec
def traverseTree(queue: List[String], accumulator: List[Widget]): List[Widget] = {
queue match {
case currentKey :: queueTail => // the queue is not empty
val current = getWidget(currentKey) // get the element at the front
val newQueueItems = // filter out the dependencies already known
current.dependencies.filterNot(dependencyKey =>
accumulator.exists(_.name == dependencyKey) && !queue.contains(dependencyKey))
traverseTree(newQueueItems ::: queueTail, current :: accumulator) //
case Nil => // the queue is empty
accumulator.reverse // we're done
}
}
traverseTree(key :: Nil, List[Widget]())
}
And test it out:
for (k <- 1 to 5)
println(getWidgetTree(k.toString).map(_.name))
prints:
ListBuffer(1, 2, 3, 4, 5)
ListBuffer(2, 3, 4)
ListBuffer(3)
ListBuffer(4)
ListBuffer(5)
For the same example as in #dhg's answer, an equivalent tail recursive function with no mutable state (the ListBuffer) would be:
case class Widget(name: String, dependencies: List[String])
val getWidget = List(
Widget("1", List("2", "5")),
Widget("2", List("3", "4")),
Widget("3", List()),
Widget("4", List()),
Widget("5", List())).map { w => w.name -> w }.toMap
def getWidgetTree(key: String): List[Widget] = {
def addIfNotAlreadyContained(widgetList: List[Widget], widgetNameToAdd: String): List[Widget] = {
if (widgetList.find(_.name == widgetNameToAdd).isDefined) widgetList
else widgetList :+ getWidget(widgetNameToAdd)
}
#tailrec
def traverseTree(currentWidgets: List[Widget], acc: List[Widget]): List[Widget] = currentWidgets match {
case Nil => {
// If there are no more widgets in this branch return what we've traversed so far
acc
}
case Widget(name, Nil) :: rest => {
// If the first widget is a leaf traverse the rest and add the leaf to the list of traversed
traverseTree(rest, addIfNotAlreadyContained(acc, name))
}
case Widget(name, dependencies) :: rest => {
// If the first widget is a parent, traverse it's children and the rest and add it to the list of traversed
traverseTree(dependencies.map(getWidget) ++ rest, addIfNotAlreadyContained(acc, name))
}
}
val root = getWidget(key)
traverseTree(root.dependencies.map(getWidget) :+ root, List[Widget]())
}
For the same test case
for (k <- 1 to 5)
println(getWidgetTree(k.toString).map(_.name).toList.sorted)
Gives you:
List(2, 3, 4, 5, 1)
List(3, 4, 2)
List(3)
List(4)
List(5)
Note that this is postorder not preorder traversal.
Awesome! thanks. I didn't know about the #tailrec annotation. that's a pretty cool little gem there. I had to tweak the solution just a little bit because a widget with a self reference was resulting in in an endless loop. also newQueueItems was an Iterable when the call to traverseTree was expecting a List, so i had to toList that bit.
def getWidgetTree(key: String): List[Widget] = {
#tailrec
def traverseTree(queue: List[String], accumulator: List[Widget]): List[Widget] = {
queue match {
case currentKey :: queueTail => // the queue is not empty
val current = getWidget(currentKey) // get the element at the front
val newQueueItems = // filter out the dependencies already known
current.dependencies.filter(dependencyKey =>
!accumulator.exists(_.name == dependencyKey) && !queue.contains(dependencyKey)).toList
traverseTree(newQueueItems ::: queueTail, current :: accumulator) //
case Nil => // the queue is empty
accumulator.reverse // we're done
}
}
traverseTree(key :: Nil, List[Widget]())
}
I'm trying to refactor a component that currently produces a Seq[X] using a fairly expensive recursive algorithm so that it produces a Stream[X] instead, so X's can be loaded/calculated on-demand, and the producer doesn't have to try to guess beforehand how much digging it'll have to do to satisfy the consumer.
From what I've read, this is an ideal use for an "unfold", so that's the route I've been trying to take.
Here's my unfold function, derived from David Pollak's example, which has been vetted by a certain Mr. Morris:
def unfold[T,R](init: T)(f: T => Option[(R,T)]): Stream[R] = f(init) match {
case None => Stream[R]()
case Some((r,v)) => r #:: unfold(v)(f)
}
And here's a little tree to try our luck with:
case class Node[A](data: A, children: List[Node[A]]) {
override def toString = "Node(" + data + ", children=(" +
children.map(_.data).mkString(",") +
"))"
}
val tree = Node("root", List(
Node("/a", List(
Node("/a/1", Nil),
Node("/a/2", Nil)
)),
Node("/b", List(
Node("/b/1", List(
Node("/b/1/x", Nil),
Node("/b/1/y", Nil)
)),
Node("/b/2", List(
Node("/b/2/x", Nil),
Node("/b/2/y", Nil),
Node("/b/2/z", Nil)
))
))
))
And finally, here's my failed attempt at a breadth-first traversal that uses unfold:
val initial = List(tree)
val traversed = ScalaUtils.unfold(initial) {
case node :: Nil =>
Some((node, node.children))
case node :: nodes =>
Some((node, nodes))
case x =>
None
}
assertEquals(12, traversed.size) // Fails, 8 elements found
/*
traversed foreach println =>
Node(root, children=(/a,/b))
Node(/a, children=(/a/1,/a/2))
Node(/b, children=(/b/1,/b/2))
Node(/b/1, children=(/b/1/x,/b/1/y))
Node(/b/2, children=(/b/2/x,/b/2/y,/b/2/z))
Node(/b/2/x, children=())
Node(/b/2/y, children=())
Node(/b/2/z, children=())
*/
Can anyone give me some hints as to how to fix (or rewrite) my traversal logic so that all the nodes are returned? Thanks!
You just forgot to include the inner nodes' children during the traversal of the tree:
val traversed = unfold(initial) {
case node :: Nil =>
Some((node, node.children))
case node :: nodes =>
// breadth-first
Some((node, nodes ::: node.children))
// or depth-first: Some((node, node.children ::: nodes))
case x =>
None
}
Here is a complete version of Moritz' answer, with a corrected partial function (the last case never matched anything in the original problem):
case class CNode[A](data: A, children: List[CNode[A]]=Nil) {
override def toString: String = if (children.isEmpty) s"node($data)" else
s"node($data, children=(${ children.map(_.data).mkString(",") }))"
}
object Main extends App {
def unfold[T, R](init: T)(f: T => Option[(R, T)]): Stream[R] = f(init) match {
case None => Stream[R]()
case Some((r, v)) => r #:: unfold(v)(f)
}
val tree = List(
CNode("root", List(
CNode("/a", List(
CNode("/a/1", Nil),
CNode("/a/2", Nil)
)),
CNode("/b", List(
CNode("/b/1", List(
CNode("/b/1/x", Nil),
CNode("/b/1/y", Nil)
)),
CNode("/b/2", List(
CNode("/b/2/x", Nil),
CNode("/b/2/y", Nil),
CNode("/b/2/z", Nil)
))
))
))
)
val traversed = unfold(tree) {
case node :: Nil =>
Some((node, node.children))
case node :: nodes =>
// breadth-first
Some((node, nodes ::: node.children))
// or depth-first: Some((node, node.children ::: nodes))
case Nil =>
None
}
println(traversed.force.mkString("\n"))
}