Scala has inits method defined for Lists which does the following:
List(1, 2, 3).inits.toList // List(List(1, 2, 3), List(1, 2), List(1), List())
It is implemented this way:
def inits: Iterator[Repr] = iterateUntilEmpty(_.init)
private def iterateUntilEmpty(f: Traversable[A #uV] => Traversable[A #uV]): Iterator[Repr] = {
val it = Iterator.iterate(thisCollection)(f) takeWhile (x => !x.isEmpty)
it ++ Iterator(Nil) map (x => (newBuilder ++= x).result)
}
I whipped up a less generic version of it (just to play with):
val L = List(1,2,3)
val it = Iterator.iterate(L)(_.init) takeWhile (x => !x.isEmpty)
it ++ Iterator(Nil) map (x => (new ListBuffer ++= x).result) toList
// List(List(1, 2, 3), List(1, 2), List(1), List())
But then it seemed to me we can do without ListBuffer part:
val L = List(1,2,3)
val it = Iterator.iterate(L)(_.init) takeWhile (x => !x.isEmpty)
it ++ Iterator(Nil) toList
// List(List(1, 2, 3), List(1, 2), List(1), List())
The result seems to be the same.
So why does the library implementation use newBuilder?
And how does it affect to performance?
Hopefully, Rex Kerr #rex-kerr will kick off a long-running job of some sort and check SO for questions, but until then, you're asking why doesn't List specialize TraversableLike.inits.
I think the two answers are that specializing for every concrete collection is a maintenance problem, and that overriding the method competes with the JVM's strategy to compile it.
I think I read in the sources that the JVM will tolerate two such dispatches to overrides before performance is affected. I'm not an expert, but the intuition is that assuming a method is effectively final is more efficient.
But wait a sec, your .toList is not the same as .map(_.toList).
Your question reduces to, why not leave off the builder entirely for List, since you get Lists from the iterator.
Well, maybe it doesn't work out for other types:
scala> val is = (1 to 10).to[Vector]
is: Vector[Int] = Vector(1, 2, 3, 4, 5, 6, 7, 8, 9, 10)
scala> val is2 = Iterator.iterate(is)(_.init) takeWhile (x => !x.isEmpty)
is2: Iterator[Vector[Int]] = non-empty iterator
scala> is2 ++ Iterator(Nil)
res9: Iterator[scala.collection.immutable.Seq[Int] with scala.collection.AbstractSeq[Int] with Serializable] = non-empty iterator
Related
I was going through a Scala-99 problem to reduce a complex nested list into a flat list. Code given below:
def flatten(l: List[Any]): List[Any] = l flatMap {
case ms:List[_] => flatten(ms)
case l => List(l)
}
val L = List(List(1, 1), 2, List(3, List(5, 8)))
val flattenedList = flatten(L)
For the given input above L, I understood this problem by drawing a tree (given below)
List(List(1, 1), 2, List(3, List(5, 8))) (1)
| \ \
List(1, 1) List(2) List(3, List(5, 8)) (2)
| \ | \
List(1) List(1) List(3) List(5, 8) (3)
| \
List(5) List(8) (4)
What I've understood is that, the program results in the leaf nodes being added in a list maintained by Scala internally, like:
li = List(List(1), List(1), List(2), List(3), List(5), List(8))
and then the result is passed to the flatten method which results in the final answer:
List(1, 1, 2, 3, 5, 8)
Is my understanding correct?
EDIT: I'm sorry, I forgot to add this:
I wanted to ask that if my understanding is correct then why does replacing flatMap with map in the flatten's definition above produces this list:
List(List(List(1), List(1)), List(2), List(List(3), List(List(5), List(8))))
I mean isn't flatMap just map then flatten. Shouldn't I be getting like the one I mentioned above:
li = List(List(1), List(1), List(2), List(3), List(5), List(8))
You're right that flatMapis just map and flatten but note that this flatten is not the same flatten you define, for list it only concatenate inner lists at 1 level.
One very useful way to unpack these is to use substitution model, just like maths
if I define it like this, (calling it f to avoid confusion with flatten here and flatten in std library)
def f(l: List[Any]): List[Any] = l map {
case ms:List[_] => f(ms)
case l => List(l)
}
then
f(List( List(1, 1), 2))
= List(f(List(1, 1)), f(2)) // apply f to element of the outer most list
= List(List(f(1), f(1)), f(2)) // apply f to element of the inner list
= List(List(List(1), List(1)), List(2))) // no more recursion
Notice map doesn't change the structure of your list, it only applies the function to each element. This should explains how you have the result if you replace flatMap with map
Now if you have flatMap instead of map, then the flatten step is simply concatenating
def f(l: List[Any]): List[Any] = l flatMap {
case ms:List[_] => f(ms)
case l => List(l)
}
then
f(List(List(1,1), 2))
= f(List(1,1)) ++ f(2) // apply f to each element and concatenate
= (f(1) ++ f(1)) ++ f(2)
= (List(1) ++ List(1)) ++ List(2)
= List( 1,1) ++ List(2)
= List(1,2,3)
or in another way, using flatten instead of ++
f( List( List(1,1), 2))
= flatten(List( f( List( 1, 1)) , f(2))) // map and flatten
= flatten(List( flatten(List(f(1), f(1))), f(2))) // again map and flatten
= flatten(List( flatten(List(List(1), List(1))), List(2))))
now you can see that flatten is called multiple times, at every level where you recursively apply f which will collapse your tree 1 level at a time into just 1 big list.
To answer your comment: why is List(1,1) is turned into flatten(List(List(1), List(1)). It's because this is the simple case, but consider List(1, List(2)), then f will be applied for 1 and List(2). Because the next step is to 'flatten' (in stdlib) then both 1 & List(2) must be turned into a List so that it is in the right shape
Want to merge val A = Option(Seq(1,2)) and val B = Option(Seq(3,4)) to yield a new option sequence
val C = Option(Seq(1,2,3,4))
This
val C = Option(A.getOrElse(Nil) ++ B.getOrElse(Nil)),
seems faster and more idiomatic than
val C = Option(A.toList.flatten ++ B.toList.flatten)
But is there a better way? And am I right that getOrElse is faster and lighter than toList.flatten?
What about a neat for comprehension:
val Empty = Some(Nil)
val C = for {
a <- A orElse Empty
b <- B orElse Empty
} yield a ++ b
Creates less intermediate options.
Or, you could just do a somewhat cumbersome pattern matching:
(A, B) match {
case (None, None) => Nil
case (None, sb#Some(b)) => sb
case (sa#Some(a), None) => sa
case (Some(a), Some(b)) => Some(a ++ b)
}
I think this at least creates less intermediate collections than the double flatten.
Your first case:
// In this case getOrElse is not needed as the option is clearly not `None`.
// So, you can replace the following:
val C = Option(A.getOrElse(Nil) ++ B.getOrElse(Nil))
// By this:
val C = Option(A.get ++ B.get) // A simple concatenation of two sequences.
C: Option[Seq[Int]] = Some(List(1, 2, 3, 4))
Your second case/option is wrong for multiple reasons.
val C = Option(A.toList.flatten ++ B.toList.flatten)
Option[List[Int]] = Some(List(1, 2, 3, 4))
It returns the incorrect type Option[List[Int]] instead of Option[Seq[Int]]
It needlessly invokes toList on A & B. You could simply add the options and invoke flatten on them.
It is not DRY and redundantly calls flatten on both A.toList & B.toList whereas it could call flatten on (A ++ B)
Instead of this, you could do this more efficiently:
val E = Option((A ++ B).flatten.toSeq)
E: Option[Seq[Int]] = Some(List(1, 2, 3, 4))
Using foldLeft
Seq(Some(List(1, 2)), None).foldLeft(List.empty[Int])(_ ++ _.getOrElse(List.empty[Int]))
result: List[Int] = List(1, 2)
Using flatten twice
Seq(Some(Seq(1, 2, 3)), Some(4, 5, 6), None).flatten.flatten
result: Seq(1, 2, 3, 4, 5, 6)
Scala REPL
scala> val a = Some(Seq(1, 2, 3))
a: Some[Seq[Int]] = Some(List(1, 2, 3))
scala> val b = Some(Seq(4, 5, 6))
b: Some[Seq[Int]] = Some(List(4, 5, 6))
scala> val c = None
c: None.type = None
scala> val d = Seq(a, b, c).flatten.flatten
d: Seq[Int] = List(1, 2, 3, 4, 5, 6)
As a Scala beginner I am still struggling working with immutable lists. All I am trying to do append elements to my list. Here's an example of what I am trying to do.
val list = Seq()::Nil
val listOfInts = List(1,2,3)
listOfInts.foreach {case x=>
list::List(x)
}
expecting that I would end up with a list of lists: List(List(1),List(2),List(3))
Coming from java I am used to just using list.add(new ArrayList(i)) to get the same result. Am I way off here?
Since the List is immutable you can not modify the List in place.
To construct a List of 1 item Lists from a List, you can map over the List. The difference between forEach and map is that forEach returns nothing, i.e. Unit, while map returns a List from the returns of some function.
scala> def makeSingleList(j:Int):List[Int] = List(j)
makeSingleList: (j: Int)List[Int]
scala> listOfInts.map(makeSingleList)
res1: List[List[Int]] = List(List(1), List(2), List(3))
Below is copy and pasted from the Scala REPL with added print statement to see what is happening:
scala> val list = Seq()::Nil
list: List[Seq[Nothing]] = List(List())
scala> val listOfInts = List(1,2,3)
listOfInts: List[Int] = List(1, 2, 3)
scala> listOfInts.foreach { case x=>
| println(list::List(x))
| }
List(List(List()), 1)
List(List(List()), 2)
List(List(List()), 3)
During the first iteration of the foreach loop, you are actually taking the first element of listOfInts (which is 1), putting that in a new list (which is List(1)), and then adding the new element list (which is List(List()) ) to the beginning of List(1). This is why it prints out List(List(List()), 1).
Since your list and listOfInts are both immutable, you can't change them. All you can do is perform something on them, and then return a new list with the change. In your case list::List(x) inside the loop actually doesnt do anything you can see unless you print it out.
There are tutorials on the documentation page.
There is a blurb for ListBuffer, if you swing that way.
Otherwise,
scala> var xs = List.empty[List[Int]]
xs: List[List[Int]] = List()
scala> (1 to 10) foreach (i => xs = xs :+ List(i))
scala> xs
res9: List[List[Int]] = List(List(1), List(2), List(3), List(4), List(5), List(6), List(7), List(8), List(9), List(10))
You have a choice of using a mutable builder like ListBuffer or a local var and returning the collection you build.
In the functional world, you often build by prepending and then reverse:
scala> var xs = List.empty[List[Int]]
xs: List[List[Int]] = List()
scala> (1 to 10) foreach (i => xs = List(i) :: xs)
scala> xs.reverse
res11: List[List[Int]] = List(List(1), List(2), List(3), List(4), List(5), List(6), List(7), List(8), List(9), List(10))
Given val listOfInts = List(1,2,3), and you want the final result as List(List(1),List(2),List(3)).
Another nice trick I can think of is sliding(Groups elements in fixed size blocks by passing a "sliding window" over them)
scala> val listOfInts = List(1,2,3)
listOfInts: List[Int] = List(1, 2, 3)
scala> listOfInts.sliding(1)
res6: Iterator[List[Int]] = non-empty iterator
scala> listOfInts.sliding(1).toList
res7: List[List[Int]] = List(List(1), List(2), List(3))
// If pass 2 in sliding, it will be like
scala> listOfInts.sliding(2).toList
res8: List[List[Int]] = List(List(1, 2), List(2, 3))
For more about the sliding, you can have a read about sliding in scala.collection.IterableLike.
You can simply map over this list to create a List of Lists.
It maintains Immutability and functional approach.
scala> List(1,2,3).map(List(_))
res0: List[List[Int]] = List(List(1), List(2), List(3))
Or you, can also use Tail Recursion :
#annotation.tailrec
def f(l:List[Int],res:List[List[Int]]=Nil) :List[List[Int]] = {
if(l.isEmpty) res else f(l.tail,res :+ List(l.head))
}
scala> f(List(1,2,3))
res1: List[List[Int]] = List(List(1), List(2), List(3))
In scala you have two (three, as #som-snytt has shown) options -- opt for a mutable collection (like Buffer):
scala> val xs = collection.mutable.Buffer(1)
// xs: scala.collection.mutable.Buffer[Int] = ArrayBuffer(1)
scala> xs += 2
// res10: xs.type = ArrayBuffer(1, 2)
scala> xs += 3
// res11: xs.type = ArrayBuffer(1, 2, 3)
As you can see, it works just like you would work with lists in Java. The other option you have, and in fact it's highly encouraged, is to opt to processing list functionally, that's it, you take some function and apply it to each and every element of collection:
scala> val ys = List(1,2,3,4).map(x => x + 1)
// ys: List[Int] = List(2, 3, 4, 5)
scala> def isEven(x: Int) = x % 2 == 0
// isEven: (x: Int)Boolean
scala> val zs = List(1,2,3,4).map(x => x * 10).filter(isEven)
// zs: List[Int] = List(10, 20, 30, 40)
// input: List(1,2,3)
// expected output: List(List(1), List(2), List(3))
val myList: List[Int] = List(1,2,3)
val currentResult = List()
def buildIteratively(input: List[Int], currentOutput: List[List[Int]]): List[List[Int]] = input match {
case Nil => currentOutput
case x::xs => buildIteratively(xs, List(x) :: currentOutput)
}
val result = buildIteratively(myList, currentResult).reverse
You say in your question that the list is immutable, so you do are aware that you cannot mutate it ! All operations on Scala lists return a new list. By the way, even in Java using a foreach to populate a collection is considered a bad practice. The Scala idiom for your use-case is :
list ::: listOfInts
Shorter, clearer, more functional, more idiomatic and easier to reason about (mutability make things more "complicated" especially when writing lambda expressions because it breaks the semantic of a pure function). There is no good reason to give you a different answer.
If you want mutability, probably for performance purposes, use a mutable collection such as ArrayBuffer.
What's the best way to convert a List of Lists in scala (2.9)?
I have a list:
List[List[A]]
which I want to convert into
List[A]
How can that be achieved recursively? Or is there any other better way?
List has the flatten method. Why not use it?
List(List(1,2), List(3,4)).flatten
> List(1,2,3,4)
.flatten is obviously the easiest way, but for completeness you should also know about flatMap
val l = List(List(1, 2), List(3, 4))
println(l.flatMap(identity))
and the for-comprehension equivalent
println(for (list <- l; x <- list) yield x)
flatten is obviously a special case of flatMap, which can do so much more.
Given the above example, I'm not sure you need recursion. Looks like you want List.flatten instead.
e.g.
scala> List(1,2,3)
res0: List[Int] = List(1, 2, 3)
scala> List(4,5,6)
res1: List[Int] = List(4, 5, 6)
scala> List(res0,res1)
res2: List[List[Int]] = List(List(1, 2, 3), List(4, 5, 6))
scala> res2.flatten
res3: List[Int] = List(1, 2, 3, 4, 5, 6)
If your structure can be further nested, like:
List(List(1, 2, 3, 4, List(5, 6, List(7, 8))))
This function should give you the desire result:
def f[U](l: List[U]): List[U] = l match {
case Nil => Nil
case (x: List[U]) :: tail => f(x) ::: f(tail)
case x :: tail => x :: f(tail)
}
You don't need recursion but you can use it if you want:
def flatten[A](list: List[List[A]]):List[A] =
if (list.length==0) List[A]()
else list.head ++ flatten(list.tail)
This works like flatten method build into List. Example:
scala> flatten(List(List(1,2), List(3,4)))
res0: List[Int] = List(1, 2, 3, 4)
If you want to use flatmap, here is the the way
Suppose that you have a List of List[Int] named ll, and you want to flat it to List,
many people already gives you the answers, such as flatten, that's the easy way. I assume that you are asking for using flatmap method. If it is the case, here is the way
ll.flatMap(_.map(o=>o))
I was wondering how you would write a method in Scala that takes a function f and a list of arguments args where each arg is a range. Suppose I have three arguments (Range(0,2), Range(0,10), and Range(1, 5)). Then I want to iterate over f with all the possibilities of those three arguments.
var sum = 0.0
for (a <- arg(0)) {
for (b <- arg(1)) {
for (c <- arg(2)) {
sum += f(a, b, c)
}
}
}
However, I want this method to work for functions with a variable number of arguments. Is this possible?
Edit: is there any way to do this when the function does not take a list, but rather takes a standard parameter list or is curried?
That's a really good question!
You want to run flatMap in sequence over a list of elements of arbitrary size. When you don't know how long your list is, you can process it with recursion, or equivalently, with a fold.
scala> def sequence[A](lss: List[List[A]]) = lss.foldRight(List(List[A]())) {
| (m, n) => for (x <- m; xs <- n) yield x :: xs
| }
scala> sequence(List(List(1, 2), List(4, 5), List(7)))
res2: List[List[Int]] = List(List(1, 4, 7), List(1, 5, 7), List(2, 4, 7), List(2
, 5, 7))
(If you can't figure out the code, don't worry, learn how to use Hoogle and steal it from Haskell)
You can do this with Scalaz (in general it starts with a F[G[X]] and returns a G[F[X]], given that the type constructors G and F have the Traverse and Applicative capabilities respectively.
scala> import scalaz._
import scalaz._
scala> import Scalaz._
import Scalaz._
scala> List(List(1, 2), List(4, 5), List(7)).sequence
res3: List[List[Int]] = List(List(1, 4, 7), List(1, 5, 7), List(2, 4, 7), List(2
, 5, 7))
scala> Seq(some(1), some(2)).sequence
res4: Option[Seq[Int]] = Some(List(1, 2))
scala> Seq(some(1), none[Int]).sequence
res5: Option[Seq[Int]] = None
That would more or less do the job (without applying f, which you can do separately)
def crossProduct[A](xxs: Seq[A]*) : Seq[Seq[A]]
= xxs.foldLeft(Vector(Vector[A]())){(res, xs) =>
for(r <- res; x <- xs) yield r :+ x
}
You can then just map your function on that. I'm not sure it's a very efficient implementation though.
That's the answer from recursive perspective. Unfortunately, not so short as others.
def foo(f: List[Int] => Int, args: Range*) = {
var sum = 0.0
def rec(ranges: List[Range], ints: List[Int]): Unit = {
if (ranges.length > 0)
for (i <- ranges.head)
rec(ranges.tail, i :: ints)
else
sum += f(ints)
}
rec(args.toList, List[Int]())
sum
}
Have a look at this answer. I use this code for exactly this purpose. It's slightly optimized. I think I could produce a faster version if you need one.