flatMap() in Scala during recursion - scala

I was going through a Scala-99 problem to reduce a complex nested list into a flat list. Code given below:
def flatten(l: List[Any]): List[Any] = l flatMap {
case ms:List[_] => flatten(ms)
case l => List(l)
}
val L = List(List(1, 1), 2, List(3, List(5, 8)))
val flattenedList = flatten(L)
For the given input above L, I understood this problem by drawing a tree (given below)
List(List(1, 1), 2, List(3, List(5, 8))) (1)
| \ \
List(1, 1) List(2) List(3, List(5, 8)) (2)
| \ | \
List(1) List(1) List(3) List(5, 8) (3)
| \
List(5) List(8) (4)
What I've understood is that, the program results in the leaf nodes being added in a list maintained by Scala internally, like:
li = List(List(1), List(1), List(2), List(3), List(5), List(8))
and then the result is passed to the flatten method which results in the final answer:
List(1, 1, 2, 3, 5, 8)
Is my understanding correct?
EDIT: I'm sorry, I forgot to add this:
I wanted to ask that if my understanding is correct then why does replacing flatMap with map in the flatten's definition above produces this list:
List(List(List(1), List(1)), List(2), List(List(3), List(List(5), List(8))))
I mean isn't flatMap just map then flatten. Shouldn't I be getting like the one I mentioned above:
li = List(List(1), List(1), List(2), List(3), List(5), List(8))

You're right that flatMapis just map and flatten but note that this flatten is not the same flatten you define, for list it only concatenate inner lists at 1 level.
One very useful way to unpack these is to use substitution model, just like maths
if I define it like this, (calling it f to avoid confusion with flatten here and flatten in std library)
def f(l: List[Any]): List[Any] = l map {
case ms:List[_] => f(ms)
case l => List(l)
}
then
f(List( List(1, 1), 2))
= List(f(List(1, 1)), f(2)) // apply f to element of the outer most list
= List(List(f(1), f(1)), f(2)) // apply f to element of the inner list
= List(List(List(1), List(1)), List(2))) // no more recursion
Notice map doesn't change the structure of your list, it only applies the function to each element. This should explains how you have the result if you replace flatMap with map
Now if you have flatMap instead of map, then the flatten step is simply concatenating
def f(l: List[Any]): List[Any] = l flatMap {
case ms:List[_] => f(ms)
case l => List(l)
}
then
f(List(List(1,1), 2))
= f(List(1,1)) ++ f(2) // apply f to each element and concatenate
= (f(1) ++ f(1)) ++ f(2)
= (List(1) ++ List(1)) ++ List(2)
= List( 1,1) ++ List(2)
= List(1,2,3)
or in another way, using flatten instead of ++
f( List( List(1,1), 2))
= flatten(List( f( List( 1, 1)) , f(2))) // map and flatten
= flatten(List( flatten(List(f(1), f(1))), f(2))) // again map and flatten
= flatten(List( flatten(List(List(1), List(1))), List(2))))
now you can see that flatten is called multiple times, at every level where you recursively apply f which will collapse your tree 1 level at a time into just 1 big list.
To answer your comment: why is List(1,1) is turned into flatten(List(List(1), List(1)). It's because this is the simple case, but consider List(1, List(2)), then f will be applied for 1 and List(2). Because the next step is to 'flatten' (in stdlib) then both 1 & List(2) must be turned into a List so that it is in the right shape

Related

Scala Iteratively build lists

As a Scala beginner I am still struggling working with immutable lists. All I am trying to do append elements to my list. Here's an example of what I am trying to do.
val list = Seq()::Nil
val listOfInts = List(1,2,3)
listOfInts.foreach {case x=>
list::List(x)
}
expecting that I would end up with a list of lists: List(List(1),List(2),List(3))
Coming from java I am used to just using list.add(new ArrayList(i)) to get the same result. Am I way off here?
Since the List is immutable you can not modify the List in place.
To construct a List of 1 item Lists from a List, you can map over the List. The difference between forEach and map is that forEach returns nothing, i.e. Unit, while map returns a List from the returns of some function.
scala> def makeSingleList(j:Int):List[Int] = List(j)
makeSingleList: (j: Int)List[Int]
scala> listOfInts.map(makeSingleList)
res1: List[List[Int]] = List(List(1), List(2), List(3))
Below is copy and pasted from the Scala REPL with added print statement to see what is happening:
scala> val list = Seq()::Nil
list: List[Seq[Nothing]] = List(List())
scala> val listOfInts = List(1,2,3)
listOfInts: List[Int] = List(1, 2, 3)
scala> listOfInts.foreach { case x=>
| println(list::List(x))
| }
List(List(List()), 1)
List(List(List()), 2)
List(List(List()), 3)
During the first iteration of the foreach loop, you are actually taking the first element of listOfInts (which is 1), putting that in a new list (which is List(1)), and then adding the new element list (which is List(List()) ) to the beginning of List(1). This is why it prints out List(List(List()), 1).
Since your list and listOfInts are both immutable, you can't change them. All you can do is perform something on them, and then return a new list with the change. In your case list::List(x) inside the loop actually doesnt do anything you can see unless you print it out.
There are tutorials on the documentation page.
There is a blurb for ListBuffer, if you swing that way.
Otherwise,
scala> var xs = List.empty[List[Int]]
xs: List[List[Int]] = List()
scala> (1 to 10) foreach (i => xs = xs :+ List(i))
scala> xs
res9: List[List[Int]] = List(List(1), List(2), List(3), List(4), List(5), List(6), List(7), List(8), List(9), List(10))
You have a choice of using a mutable builder like ListBuffer or a local var and returning the collection you build.
In the functional world, you often build by prepending and then reverse:
scala> var xs = List.empty[List[Int]]
xs: List[List[Int]] = List()
scala> (1 to 10) foreach (i => xs = List(i) :: xs)
scala> xs.reverse
res11: List[List[Int]] = List(List(1), List(2), List(3), List(4), List(5), List(6), List(7), List(8), List(9), List(10))
Given val listOfInts = List(1,2,3), and you want the final result as List(List(1),List(2),List(3)).
Another nice trick I can think of is sliding(Groups elements in fixed size blocks by passing a "sliding window" over them)
scala> val listOfInts = List(1,2,3)
listOfInts: List[Int] = List(1, 2, 3)
scala> listOfInts.sliding(1)
res6: Iterator[List[Int]] = non-empty iterator
scala> listOfInts.sliding(1).toList
res7: List[List[Int]] = List(List(1), List(2), List(3))
// If pass 2 in sliding, it will be like
scala> listOfInts.sliding(2).toList
res8: List[List[Int]] = List(List(1, 2), List(2, 3))
For more about the sliding, you can have a read about sliding in scala.collection.IterableLike.
You can simply map over this list to create a List of Lists.
It maintains Immutability and functional approach.
scala> List(1,2,3).map(List(_))
res0: List[List[Int]] = List(List(1), List(2), List(3))
Or you, can also use Tail Recursion :
#annotation.tailrec
def f(l:List[Int],res:List[List[Int]]=Nil) :List[List[Int]] = {
if(l.isEmpty) res else f(l.tail,res :+ List(l.head))
}
scala> f(List(1,2,3))
res1: List[List[Int]] = List(List(1), List(2), List(3))
In scala you have two (three, as #som-snytt has shown) options -- opt for a mutable collection (like Buffer):
scala> val xs = collection.mutable.Buffer(1)
// xs: scala.collection.mutable.Buffer[Int] = ArrayBuffer(1)
scala> xs += 2
// res10: xs.type = ArrayBuffer(1, 2)
scala> xs += 3
// res11: xs.type = ArrayBuffer(1, 2, 3)
As you can see, it works just like you would work with lists in Java. The other option you have, and in fact it's highly encouraged, is to opt to processing list functionally, that's it, you take some function and apply it to each and every element of collection:
scala> val ys = List(1,2,3,4).map(x => x + 1)
// ys: List[Int] = List(2, 3, 4, 5)
scala> def isEven(x: Int) = x % 2 == 0
// isEven: (x: Int)Boolean
scala> val zs = List(1,2,3,4).map(x => x * 10).filter(isEven)
// zs: List[Int] = List(10, 20, 30, 40)
// input: List(1,2,3)
// expected output: List(List(1), List(2), List(3))
val myList: List[Int] = List(1,2,3)
val currentResult = List()
def buildIteratively(input: List[Int], currentOutput: List[List[Int]]): List[List[Int]] = input match {
case Nil => currentOutput
case x::xs => buildIteratively(xs, List(x) :: currentOutput)
}
val result = buildIteratively(myList, currentResult).reverse
You say in your question that the list is immutable, so you do are aware that you cannot mutate it ! All operations on Scala lists return a new list. By the way, even in Java using a foreach to populate a collection is considered a bad practice. The Scala idiom for your use-case is :
list ::: listOfInts
Shorter, clearer, more functional, more idiomatic and easier to reason about (mutability make things more "complicated" especially when writing lambda expressions because it breaks the semantic of a pure function). There is no good reason to give you a different answer.
If you want mutability, probably for performance purposes, use a mutable collection such as ArrayBuffer.

How to sum adjacent elements in scala

I want to sum adjacent elements in scala and I'm not sure how to deal with the last element.
So I have a list:
val x = List(1,2,3,4)
And I want to sum adjacent elements using indices and map:
val size = x.indices.size
val y = x.indices.map(i =>
if (i < size - 1)
x(i) + x(i+1))
The problem is that this approach creates an AnyVal elemnt at the end:
res1: scala.collection.immutable.IndexedSeq[AnyVal] = Vector(3, 5, 7, ())
and if I try to sum the elements or another numeric method of the collection, it doesn't work:
error: could not find implicit value for parameter num: Numeric[AnyVal]
I tried to filter out the element using:
y diff List(Unit) or y diff List(AnyVal)
but it doesn't work.
Is there a better approach in scala to do this type of adjacent sum without using a foor loop?
For a more functional solution, you can use sliding to group the elements together in twos (or any number of them), then map to their sum.
scala> List(1, 2, 3, 4).sliding(2).map(_.sum).toList
res80: List[Int] = List(3, 5, 7)
What sliding(2) will do is create an intermediate iterator of lists like this:
Iterator(
List(1, 2),
List(2, 3),
List(3, 4)
)
So when we chain map(_.sum), we will map each inner List to it's own sum. toList will convert the Iterator back into a List.
You can try pattern matching and tail recursion also.
import scala.annotation.tailrec
#tailrec
def f(l:List[Int],r :List[Int]=Nil):List[Int] = {
l match {
case x :: xs :: xss =>
f(l.tail, r :+ (x + xs))
case _ => r
}
}
scala> f(List(1,2,3,4))
res4: List[Int] = List(3, 5, 7)
With a for comprehension by zipping two lists, the second with the first item dropped,
for ( (a,b) <- x zip x.drop(1) ) yield a+b
which results in
List(3, 5, 7)

Why do we need intermediate newBuilder in inits method of Scala List

Scala has inits method defined for Lists which does the following:
List(1, 2, 3).inits.toList // List(List(1, 2, 3), List(1, 2), List(1), List())
It is implemented this way:
def inits: Iterator[Repr] = iterateUntilEmpty(_.init)
private def iterateUntilEmpty(f: Traversable[A #uV] => Traversable[A #uV]): Iterator[Repr] = {
val it = Iterator.iterate(thisCollection)(f) takeWhile (x => !x.isEmpty)
it ++ Iterator(Nil) map (x => (newBuilder ++= x).result)
}
I whipped up a less generic version of it (just to play with):
val L = List(1,2,3)
val it = Iterator.iterate(L)(_.init) takeWhile (x => !x.isEmpty)
it ++ Iterator(Nil) map (x => (new ListBuffer ++= x).result) toList
// List(List(1, 2, 3), List(1, 2), List(1), List())
But then it seemed to me we can do without ListBuffer part:
val L = List(1,2,3)
val it = Iterator.iterate(L)(_.init) takeWhile (x => !x.isEmpty)
it ++ Iterator(Nil) toList
// List(List(1, 2, 3), List(1, 2), List(1), List())
The result seems to be the same.
So why does the library implementation use newBuilder?
And how does it affect to performance?
Hopefully, Rex Kerr #rex-kerr will kick off a long-running job of some sort and check SO for questions, but until then, you're asking why doesn't List specialize TraversableLike.inits.
I think the two answers are that specializing for every concrete collection is a maintenance problem, and that overriding the method competes with the JVM's strategy to compile it.
I think I read in the sources that the JVM will tolerate two such dispatches to overrides before performance is affected. I'm not an expert, but the intuition is that assuming a method is effectively final is more efficient.
But wait a sec, your .toList is not the same as .map(_.toList).
Your question reduces to, why not leave off the builder entirely for List, since you get Lists from the iterator.
Well, maybe it doesn't work out for other types:
scala> val is = (1 to 10).to[Vector]
is: Vector[Int] = Vector(1, 2, 3, 4, 5, 6, 7, 8, 9, 10)
scala> val is2 = Iterator.iterate(is)(_.init) takeWhile (x => !x.isEmpty)
is2: Iterator[Vector[Int]] = non-empty iterator
scala> is2 ++ Iterator(Nil)
res9: Iterator[scala.collection.immutable.Seq[Int] with scala.collection.AbstractSeq[Int] with Serializable] = non-empty iterator

What is the difference between ::: and :::.() in scala

In the scala list class it is shown that:
List(1, 2) ::: List(3, 4) = List(3, 4).:::(List(1, 2)) = List(1, 2, 3, 4).
Shouldn't:
List(1,2) ::: List(3,4) = List(1,2).:::(List(3,4)) = List(3,4,1,2)
( method ::: prefixes the list)
From the docs:
def :::(prefix: List[A]): List[A]
[use case]
Adds the elements of a given list in front of this list.
Example:
List(1, 2) ::: List(3, 4) = List(3, 4).:::(List(1, 2)) = List(1, 2, 3, 4)
In Scala, operators that end in : are right associative and the invoking object appears on the right side of the operator. Because it is right associative, the prefix method will be called on the object "to the right", which is List(3, 4).
Then it will do what its name says, it will prefix List(3, 4) with List(1, 2), and that's why you get List(1, 2, 3 ,4). Not the most intuitive thing in the world but it's like this:
a ::: b
// ending in :, so flip the argument order, call the method on b.
b .:: a // :: = prefix b with a
result = a(concatenated) with b
Now let's look at the right hand side:
List(3, 4).:::(List(1, 2))
The . dot is performing an inversion of the ::: prefixed invocation, so it will simply invert the two List objects before performing the ::: prefix operation.
List(3, 4).:::(List(1,2)) = List(1, 2) ::: List(3, 4) // exactly as the above.
// and now you it's the same story.
The . dot is a simply way to invert associative operators. a ::: b is the same as b .::: a
I think that what you are doing with
List(1,2).:::(List(3,4)) is changing the order of things.
Ending with : makes scala want to call the method on the object to the right.
So
a :: b
is really
b.::(a)
When being explicit with the dots and parentheses, you change the order.
Not sure if this example makes it clearer:
scala> class X() {
def `a:`(s: String): Unit = {
println(s)
}}
scala> var x = new X()
scala> x.`a:`("X")
X
scala> x `a:` "X"
<console>:10: error: value a: is not a member of java.lang.String
x `a:` "X"
^
You see that scala wants to call the a: method on the string object on the right.

Scala - convert List of Lists into a single List: List[List[A]] to List[A]

What's the best way to convert a List of Lists in scala (2.9)?
I have a list:
List[List[A]]
which I want to convert into
List[A]
How can that be achieved recursively? Or is there any other better way?
List has the flatten method. Why not use it?
List(List(1,2), List(3,4)).flatten
> List(1,2,3,4)
.flatten is obviously the easiest way, but for completeness you should also know about flatMap
val l = List(List(1, 2), List(3, 4))
println(l.flatMap(identity))
and the for-comprehension equivalent
println(for (list <- l; x <- list) yield x)
flatten is obviously a special case of flatMap, which can do so much more.
Given the above example, I'm not sure you need recursion. Looks like you want List.flatten instead.
e.g.
scala> List(1,2,3)
res0: List[Int] = List(1, 2, 3)
scala> List(4,5,6)
res1: List[Int] = List(4, 5, 6)
scala> List(res0,res1)
res2: List[List[Int]] = List(List(1, 2, 3), List(4, 5, 6))
scala> res2.flatten
res3: List[Int] = List(1, 2, 3, 4, 5, 6)
If your structure can be further nested, like:
List(List(1, 2, 3, 4, List(5, 6, List(7, 8))))
This function should give you the desire result:
def f[U](l: List[U]): List[U] = l match {
case Nil => Nil
case (x: List[U]) :: tail => f(x) ::: f(tail)
case x :: tail => x :: f(tail)
}
You don't need recursion but you can use it if you want:
def flatten[A](list: List[List[A]]):List[A] =
if (list.length==0) List[A]()
else list.head ++ flatten(list.tail)
This works like flatten method build into List. Example:
scala> flatten(List(List(1,2), List(3,4)))
res0: List[Int] = List(1, 2, 3, 4)
If you want to use flatmap, here is the the way
Suppose that you have a List of List[Int] named ll, and you want to flat it to List,
many people already gives you the answers, such as flatten, that's the easy way. I assume that you are asking for using flatmap method. If it is the case, here is the way
ll.flatMap(_.map(o=>o))