Explanation of List[_] in pattern matching used in flatten function - scala

Question

I am new to scala and I can not understand the following function
val L = List(List(1, 1), 2, List(3, List(5, 8)))
def flatten(l: List[Any]): List[Any] = l flatMap {
case ms:List[_] => flatten(ms)
case l => List(l)
}
flatten(L) // res2: List[Any] = List(1, 1, 2, 3, 5, 8)
In particular I do not understand the combination of flatMap and pattern matching and also the meaning of the first case ms:List[_]
Can someone explain that and maybe provide a simpler example to clarify the concept?

Answer 1

map and flatMap
First of all flatMap is a higher-order function.
map is also a higher order function which will convert a List to a new List by applying a function on all elements of that List. For example if you have
val l = List(1,2,3)
you can map it to a new list by using
val doubled = l.map(_ * 2) // List(2,4,6)
So what is flatMap? flatMap is useful when your function gets an Int but returns a List[Int]. In this case if you pass this function to map what you get will be a List[List[Int]] but sometimes you want to get a List[Int] instead.
What you can do is to use flatMap instead. Mathematically it is equivalent to first map and then flatten the result but in action it could be the flatten function which is defined based on flatMap not the other way around.
Technical details aside, flatten means that if you have a List[List[List...[Int]]], if you flatten it, you will get a List[Int].
When we look at
def flatten(l: List[Any]): List[Any] = l flatMap {
case ms:List[_] => flatten(ms)
case l => List(l)
}
we first need to know what does this mean. We already know that we have to pass a function to flatMap. In this case the function we are passing is
{
case ms:List[_] => flatten(ms)
case l => List(l)
}
which at first does not seem like a function but it is! It is actually a partial function which you can consider as a function with some nuances!
What is a partial function?
We could achieve (almost) the same result with:
def flatten(l: List[Any]): List[Any] = l flatMap { _ match {
case ms:List[_] => flatten(ms)
case l => List(l)
}
}
The difference between an ordinary function and a partial function is that partial functions may be undefined for some specific inputs. So the compiler will automatically generate a partial function from a body which starts with case keyword (It's not that simple but for the sake of simplicity I skip the details here).
Pattern matching by Type
A pattern like ms:List[_] is called a type pattern. It means that the type of ms will be checked in runtime against the List[_]. _ is a wild card and means any type so ms:List[_] literally means a List of any type (Due to type erasure you can not use a pattern like ms:List[Int], refer to this thread for more info).
So the whole pattern match on this code snippets means that
If ms is a list the result will be flatten(ms), other wise the result will be List(l).
The flatten function defined like this is recursive function which unwraps the lists and do this until there is no more list, then it will wrap the result again in a List! This way List[List[List.......[_]]] will be converted to List[_].
An additional example for type-patterns:
def hello(o: Any) = o match {
case _:Int => "Hi, I am an Int and I'm proud of it!"
case _:List[_] => "I have many things to tell you! I am a list after all!"
case b:Boolean => s"I'm simply a flag and my value is $b"
case _ => "I'm everything else you can imagine :D"
}
Disambiguation
By the way partial function is totally different from partially applied function and partial application!
The concept of partial function in Scala is the same as partial function in mathematics: A function which may be undefined for some of the values of its domain.