I'm in a Coursera's course and I'm trying to understand the logic of the scanLeft on trees.
we have the following code:
Here we have as an input a tree (without intermediate values, only with values in Leafs) and returns a tree with intermediate values (with values in nodes)
def upsweep[A](t: Tree[A], f: (A,A) => A): TreeRes[A] = t match {
case Leaf(v) => LeafRes(v)
case Node(l, r) => {
val (tL, tR) = parallel(upsweep(l, f), upsweep(r, f))
NodeRes(tL, f(tL.res, tR.res), tR)
}
}
And the following code that given a tree with intermediate values (with values in nodes) returns a tree without intermediate values (a0 is the reduce of all elements left of the tree t).
def downsweep[A](t: TreeRes[A], a0: A, f : (A,A) => A): Tree[A] = t match {
case LeafRes(a) => Leaf(f(a0, a))
case NodeRes(l, _, r) => {
val (tL, tR) = parallel(downsweep[A](l, a0, f),
downsweep[A](r, f(a0, l.res), f))
Node(tL, tR) }
}
And finally the scanLeft code:
def scanLeft[A](t: Tree[A], a0: A, f: (A,A) => A): Tree[A] = {
val tRes = upsweep(t, f)
val scan1 = downsweep(tRes, a0, f)
prepend(a0, scan1)
}
And my question is, why is necesary to use the upsweep method before downsweep?
With upsweep we generate the intermediate values and later with downsweep we "remove" (we dont need to use) them.
Thanks in advance.
Actually look more closely at this part
case NodeRes(l, _, r) => {
val (tL, tR) = parallel(downsweep[A](l, a0, f),
downsweep[A](r, f(a0, l.res), f))
what is l.res? why it is so necessary to have it?(it is created at upsweep) I recommend you to draw on a piece of paper step by step what exactly is being done by this algorithm with easy function like (_ + _). Also it Is very good technique if you do not understand smth to do it, just go easy step by step and resolve it by yourself.
Related
Consider an example
val a= List(1,2,3)
val b= List(4,5,6)
merge reduce function taking two lists and two function where one function acts as merge and another function to reduce it to Integer more of a general form.
merge by multiplying the head of two lists and then reduce using add
merge using max then get the min of the generated list
mergeReduce(a,b,product,add) = 32
mergeReduce(a,b,max,min) = 4
This can be achieved using inbuilt functions but is there a better way to do without the use of those functions in a recursive manner.
Here is your mergeReduce() (as I understand it).
def mergeReduce(a :List[Int], b :List[Int]
,f :(Int,Int)=>Int, g :(Int,Int)=>Int) :Int =
a.zip(b).map(f.tupled).reduce(g)
val a= List(1,2,3)
val b= List(4,5,6)
mergeReduce(a,b,_*_,_+_) // 32
mergeReduce(a,b,math.max,math.min) // 4
So, what are the "inbuilt" functions you want to replace? And why do you want to replace them?
Here then is a version without map, reduce, zip, and tupled.
def mergeReduce(lsta :List[Int], lstb :List[Int]
,f :(Int,Int)=>Int, g :(Int,Int)=>Int) :Int = {
def merg(x :List[Int], y :List[Int], acc :List[Int] = Nil) :List[Int] =
if (x.isEmpty || y.isEmpty) acc.reverse
else merg(x.tail, y.tail, f(x.head,y.head) :: acc)
def reduc(z: List[Int]) :Int = z match {
case Nil => -1 //error
case i :: Nil => i
case a::b::c => reduc(g(a,b) :: c)
}
reduc(merg(lsta, lstb))
}
This uses .isEmpty, .reverse, .head, .tail, and .unapply (the method by which pattern matching is accomplished). Still too much "inbuilt"?
I think this is what you are looking for. It performs merge and reduce in a single pass, using only the basic List operations:
def mergeReduce[T](a: List[T], b: List[T], merge: (T, T) => T, reduce: (T, T) => T): T = {
#tailrec
def loop(a: List[T], b: List[T], res: T): T =
(a, b) match {
case (a :: at, b :: bt) => loop(at, bt, reduce(res, merge(a, b)))
case _ => res
}
loop(a.tail, b.tail, merge(a.head, b.head))
}
This will fail if either list is Nil and will silently discard the values from the longer list if the lengths are not the same.
I understand map and flatten operations can be combined into flatMap, and filter and map into collect in Scala.
Is there anyway I can combine zip/zipwithIndex with map operation?
There is no single operation in the standard library, as far as I know, but there is an extension method on various tuples, called zipped. This method returns an object which provides methods like map and flatMap, which would perform zipping in step with mapping:
(xs, ys).zipped.map((x, y) => x * y)
This object also is implicitly convertible to Traversable, so you can call more complex methods like mkString or foldLeft.
If, for some reason, you really wanted a combined version you could write one yourself.
implicit class SeqOps[A](s: Seq[A]) {
def zipWithIndex2[A1 >: A, B >: Int, That](f: (A, Int) => (A1, B))(implicit bf: CanBuildFrom[Seq[A], (A1, B), That]): That = {
val b = bf(s)
var i = 0
for (x <- s) {
b += f(x, i)
i += 1
}
b.result()
}
}
Call it like:
s.zipWithIndex2 {
case (a, b) => (a + "2", b + 2)
}
I'd really think about this twice though and most likely go with any of the other approaches that have been suggested.
I am trying to find a tail recursive fold function for a binary tree. Given the following definitions:
// From the book "Functional Programming in Scala", page 45
sealed trait Tree[+A]
case class Leaf[A](value: A) extends Tree[A]
case class Branch[A](left: Tree[A], right: Tree[A]) extends Tree[A]
Implementing a non tail recursive function is quite straightforward:
def fold[A, B](t: Tree[A])(map: A => B)(red: (B, B) => B): B =
t match {
case Leaf(v) => map(v)
case Branch(l, r) =>
red(fold(l)(map)(red), fold(r)(map)(red))
}
But now I am struggling to find a tail recursive fold function so that the annotation #annotation.tailrec can be used.
During my research I have found several examples where tail recursive functions on a tree can e.g. compute the sum of all leafs using an own stack which is then basically a List[Tree[Int]]. But as far as I understand in this case it only works for the additions because it is not important whether you first evaluate the left or the right hand side of the operator. But for a generalised fold it is quite relevant. To show my intension here are some example trees:
val leafs = Branch(Leaf(1), Leaf(2))
val left = Branch(Branch(Leaf(1), Leaf(2)), Leaf(3))
val right = Branch(Leaf(1), Branch(Leaf(2), Leaf(3)))
val bal = Branch(Branch(Leaf(1), Leaf(2)), Branch(Leaf(3), Leaf(4)))
val cmb = Branch(right, Branch(bal, Branch(leafs, left)))
val trees = List(leafs, left, right, bal, cmb)
Based on those trees I want to create a deep copy with the given fold method like:
val oldNewPairs =
trees.map(t => (t, fold(t)(Leaf(_): Tree[Int])(Branch(_, _))))
And then proof that the condition of equality holds for all created copies:
val conditionHolds = oldNewPairs.forall(p => {
if (p._1 == p._2) true
else {
println(s"Original:\n${p._1}\nNew:\n${p._2}")
false
}
})
println("Condition holds: " + conditionHolds)
Could someone give me some pointers, please?
You can find the code used in this question at ScalaFiddle: https://scalafiddle.io/sf/eSKJyp2/15
You could reach a tail recursive solution if you stop using the function call stack and start using a stack managed by your code and an accumulator:
def fold[A, B](t: Tree[A])(map: A => B)(red: (B, B) => B): B = {
case object BranchStub extends Tree[Nothing]
#tailrec
def foldImp(toVisit: List[Tree[A]], acc: Vector[B]): Vector[B] =
if(toVisit.isEmpty) acc
else {
toVisit.head match {
case Leaf(v) =>
val leafRes = map(v)
foldImp(
toVisit.tail,
acc :+ leafRes
)
case Branch(l, r) =>
foldImp(l :: r :: BranchStub :: toVisit.tail, acc)
case BranchStub =>
foldImp(toVisit.tail, acc.dropRight(2) ++ Vector(acc.takeRight(2).reduce(red)))
}
}
foldImp(t::Nil, Vector.empty).head
}
The idea is to accumulate values from left to right, keep track of the parenthood relation by the introduction of a stub node and reduce the result using your red function using the last two elements of the accumulator whenever a stub node is found in the exploration.
This solution could be optimized but it is already a tail recursive function implementation.
EDIT:
It can be slightly simplified by changing the accumulator data structure to a list seen as a stack:
def fold[A, B](t: Tree[A])(map: A => B)(red: (B, B) => B): B = {
case object BranchStub extends Tree[Nothing]
#tailrec
def foldImp(toVisit: List[Tree[A]], acc: List[B]): List[B] =
if(toVisit.isEmpty) acc
else {
toVisit.head match {
case Leaf(v) =>
foldImp(
toVisit.tail,
map(v)::acc
)
case Branch(l, r) =>
foldImp(r :: l :: BranchStub :: toVisit.tail, acc)
case BranchStub =>
foldImp(toVisit.tail, acc.take(2).reduce(red) :: acc.drop(2))
}
}
foldImp(t::Nil, Nil).head
}
The following code is adapted from a paper (R. O. Bjarnason, Stackless Scala With Free Monads).
The title of the paper points to the purpose of the proposed data structures in general - that is to afford recursive processing in constant stack space, and to let the user express recursion in a clear way.
Specificly, my goal is to have a monadic structure that affords structural rewriting of an immutable Tree of Pairs (binary tree) or of Lists (n-ary-tree) based on simple pattern matching in constant stack space when ascending.
sealed trait Free[S[+_], +A]{
private case class FlatMap[S[+_], A, +B](
a: Free[S, A],
f: A => Free[S, B]
) extends Free[S, B]
def map[B](f: A => B): Free[S, B] = this.flatMap((a:A) => Done[S, B](f(a)))
def flatMap[B](f: A => Free[S, B]): Free[S, B] = this match {
case FlatMap(a, g) => FlatMap(a, (x: Any) => g(x).flatMap(f))
case x => FlatMap(x, f)
}
#tailrec
final def resume(implicit S: Functor[S]): Either[S[Free[S, A]], A] = {
this match {
case Done(a) => Right(a)
case More(k) => Left(k)
case FlatMap(a, f) => a match {
case Done(a) => f(a).resume
case More(k) => Left(S.map(k)((x)=>x.flatMap(f)))
case FlatMap(b, g) => b.flatMap((x: Any) => g(x).flatMap(f)).resume
}
}
}
}
case class Done[S[+_], +A](a: A) extends Free[S, A]
case class More[S[+_], +A](k: S[Free[S, A]]) extends Free[S,A]
trait Functor[F[+_]] {
def map[A, B](m: F[A])(f: A => B): F[B]
}
type RoseTree[+A] = Free[List, A]
implicit object listFunctor extends Functor[List] {
def map[A, B](a: List[A])(f: A => B) = a.map(f)
}
var tree : Free[List, Int]= More(List(More(List(More(List(Done(1), Done(2))), More(List(Done(3), Done(4))))), More(List(More(List(Done(5), Done(6))), More(List(Done(7), Done(8)))))))
How is the rewriting achieved using Free?
Where is a hook for the pattern matcher? - The pattern matcher has to be exposed to each entire subtree when ascending!
Can this be done within a for block?
[The question was edited.]
Update: the answer below addresses an earlier version of the question, but is mostly still relevant.
First of all, your code isn't going to work as it is. You can either make everything invariant, or go with the variance annotations in the original paper. For the sake of simplicity I'll take the invariant route (see here for a complete example), but I've also just confirmed that the version in the paper will work on 2.10.2.
To answer your first question first: your binary tree type is isomorphic to BinTree[Int]. Before we can show this, though, we need a functor for our pair type:
implicit object pairFunctor extends Functor[Pair] {
def map[A, B](a: Pair[A])(f: A => B) = (f(a._1), f(a._2))
}
Now we can use resume, which we'll need for the conversion from BinTree back to T:
def from(tree: T): BinTree[Int] = tree match {
case L(i) => Done(i)
case F((l, r)) => More[Pair, Int]((from(l), from(r)))
}
def to(tree: BinTree[Int]): T = tree.resume match {
case Left((l, r)) => F((to(l), to(r)))
case Right(i) => L(i)
}
Now we can define your example tree:
var z = 0
def f(i: Int): T = if (i > 0) F((f(i - 1), f(i - 1))) else { z = z + 1; L(z) }
val tree = f(3)
Let's demonstrate our isomorphism and the monad for BinTree by replacing every leaf value with the tree containing its predecessor and successor:
val newTree = to(
from(tree).flatMap(i => More[Pair, Int]((Done(i - 1), Done(i + 1))))
)
After some reformatting, the result will look like this:
F((
F((
F((
F((L(0), L(2))),
F((L(1), L(3)))
)),
F((
F((L(2), L(4))),
F((L(3), L(5)))
)),
...
And so on, as expected.
For your second question: if you want to do the same kind of thing for a rose tree, you'd just replace the pair with a list (or a stream). You'll need to provide a functor instance for lists, as we did above for pairs, and then you've got a tree with Done(x) representing leaves and More(xs) for branches.
Your type needs map for the for-comprehension syntax to work. Fortunately you can write map in terms of flatMap and Done—just add the following to the definition of Free:
def map[B](f: A => B): Free[S, B] = this.flatMap(f andThen Done.apply)
Now the following is exactly the same as the newTree above:
val newTree = to(
for {
i <- from(tree)
m <- More[Pair, Int]((Done(i - 1), Done(i + 1)))
} yield m
)
The same thing will work with the Free[List, _] rose tree.
This is a followup to: Expand a Set of Sets of Strings into Cartesian Product in Scala
The idea is you want to take:
val sets = Set(Set("a","b","c"), Set("1","2"), Set("S","T"))
and get back:
Set("a&1&S", "a&1&T", "a&2&S", ..., "c&2&T")
A general solution is:
def combine[A](f:(A, A) => A)(xs:Iterable[Iterable[A]]) =
xs.reduceLeft { (x, y) => x.view.flatMap {a => y.map(f(a, _)) } }
used as follows:
val expanded = combine{(x:String, y:String) => x + "&" + y}(sets).toSet
Theoretically, there should be a way to take input of type Set[Set[A]] and get back a Set[B]. That is, to convert the type while combining the elements.
An example usage would be to take in sets of strings (as above) and output the lengths of their concatenation. The f function in combine would something of the form:
(a:Int, b:String) => a + b.length
I was not able to come up with an implementation. Does anyone have an answer?
If you really want your combiner function to do the mapping, you can use a fold but as Craig pointed out you'll have to provide a seed value:
def combine[A, B](f: B => A => B, zero: B)(xs: Iterable[Iterable[A]]) =
xs.foldLeft(Iterable(zero)) {
(x, y) => x.view flatMap { y map f(_) }
}
The fact that you need such a seed value follows from the combiner/mapper function type (B, A) => B (or, as a curried function, B => A => B). Clearly, to map the first A you encounter, you're going to need to supply a B.
You can make it somewhat simpler for callers by using a Zero type class:
trait Zero[T] {
def zero: T
}
object Zero {
implicit object IntHasZero extends Zero[Int] {
val zero = 0
}
// ... etc ...
}
Then the combine method can be defined as:
def combine[A, B : Zero](f: B => A => B)(xs: Iterable[Iterable[A]]) =
xs.foldLeft(Iterable(implicitly[Zero[B]].zero)) {
(x, y) => x.view flatMap { y map f(_) }
}
Usage:
combine((b: Int) => (a: String) => b + a.length)(sets)
Scalaz provides a Zero type class, along with a lot of other goodies for functional programming.
The problem that you're running into is that reduce(Left|Right) takes a function (A, A) => A which doesn't allow you to change the type. You want something more like foldLeft which takes (B, A) ⇒ B, allowing you to accumulate an output of a different type. folds need a seed value though, which can't be an empty collection here. You'd need to take xs apart into a head and tail, map the head iterable to be Iterable[B], and then call foldLeft with the mapped head, the tail, and some function (B, A) => B. That seems like more trouble than it's worth though, so I'd just do all the mapping up front.
def combine[A, B](f: (B, B) => B)(g: (A) => B)(xs:Iterable[Iterable[A]]) =
xs.map(_.map(g)).reduceLeft { (x, y) => x.view.flatMap {a => y.map(f(a, _)) } }
val sets = Set(Set(1, 2, 3), Set(3, 4), Set(5, 6, 7))
val expanded = combine{(x: String, y: String) => x + "&" + y}{(i: Int) => i.toString}(sets).toSet