scala function - how does it work? - scala

Question

I'm new to scala/java and I'm trying to understand this code below which returns a list of files in a directory. Function source
It takes an argument called dir - is the type of dir a File or File object?
It returns an array of type file.
It calls the method listFiles on dir.
What does the last line do?
def getRecursiveListOfFiles(dir: File): Array[File] = {
val these = dir.listFiles
these ++ these.filter(_.isDirectory).flatMap(getRecursiveListOfFiles)
}

Answer 1

This code does a breadth-first search, using recursion.
A File can either be a file, or a directory.
The code dir.listFiles list all files in directory. Remember that this will be a list of files and directories!
We can then break down the last line into 3 things. These could easily be separate lines.
these.filter(_.isDirectory) will return a list of directories that need searching in. It filters out the files.
flatMap(getRecursiveListOfFiles) takes this list of directories, and calls getRecursiveListOfFiles for every single directory. It then adds flattens these results into one list.
++ adds together two arrays. We add these to the result of the recursive call.
flatMap is key here. Read up about it, and how it differs from the map function to fully understand what's going on.

Answer 2

In short:
these ++ these.filter(_.isDirectory).flatMap(getRecursiveListOfFiles)
is fundamentally:
val allSubDirectories:Array[Files] = these.filter(_.isDirectory)
allSubDirectories.flatMap(getRecursiveListOfFiles)
//i.e. for each sub-directory, again find all files in sub-directory
these ++ (files of all sub-directories)
//ultimately add files of sub-directory to the actual list
Another alternative way to understand would be:
def getAllFiles(dir: File): List[File] = {
val these = dir.listFiles.toList
these ::: these.filter(_.isDirectory).map(x => getAllFiles(x)).flatten
}
Fundamentally same control flow, that for each sub-directory, you get a list of all the files and then to the same list, you add files of sub-directory.

Answer 3

I'm going to try to walk you through how I would think of this as someone trying to read a piece of unknown code.
It should be pretty clear by context that File can either be a regular file or directory. It doesn't contain the file contents, but represents any entry in the file system, and you can open the file with other library commands. For the purposes of this function, File is just something that happens to contain more Files in the case that it's a directory, and these can be accessed via the method listFiles: List[Files]. Presumably, it also provides other info, so that the original caller of getRecursiveListOfFiles could do something with the resulting list.
Also by context, these is pretty clearly the entries in the current directory.
The last line is the most subtle. But to break it down, it augments these with the Files found in those entries in these which happen to be directories.
To explain this step, the signature of flatMap on a List[File] can be thought of as flatMap[B](f: File => List[B]): List[B], where B is a type variable. In this case, because the same function getRecursiveListOfFiles, which is of type File => List[File], is being passed recursively, B is just File, so we can think of this particular call as flatMap(f: File => List[File]): List[File].
Roughly speaking, flatMap applies a function f to each item in a container, where f is required to return the same type of container. The "flat" part is simply the fact that these individual containers get combined, instead of being nested, which is what map would do. This is what allows the function to recursively add all the files found in its subdirectories. Pretty slick.

Answer 4

What's the distinction you're drawing? The type of dir is File.
An array of Files, yes.
It's probably implicitly converting these to a scala List or Buffer, which is the confusing part (e.g. it may import scala.collection.JavaConversions._ - it's better to use JavaConverters which makes the conversions explicit calls to .asScala or asJava). You can find the definitions of ++, filter and flatMap in the scaladoc, which will hopefully be enough to understand what's happening.
(note: my 4. is coming out as a 3. because of markdown :()