Why did Scala use a while and not recursion for `find` - scala

Question

Just curious why the Scala authors didn't use recursion or even pattern matching when implementing find on Lists?
Their implementation looks like this:
override final def find(p: A => Boolean): Option[A] = {
var these: List[A] = this
while (!these.isEmpty) {
if (p(these.head)) return Some(these.head)
these = these.tail
}
None
}
Using a while and head and tail. They could have done something for "scala-esq" with recursion no?
#tailrec
def find(p: A => Boolean): Option[A] = {
this match {
case Nil => None
case head :: tail if p(head) => Some(head)
case elements => find(p, elements.tail)
}
}
It can't be because of tail-call optimisation can it? Is it somehow more efficient and I'm missing it? Could it be just author preference and style?! Something inflexible about it when A could be anything? hmmm

Answer 1

A quick experiment (using Scala 2.13.2). The three candidate implementations are:
while-loop
tail-recursive, but keeping the same logic as the while version
tail-recursive with a pattern match
I've modified the logic where appropriate to depend less on compiler optimizations (nonEmpty vs. !isEmpty and explicitly saving these.head so it's not called twice).
import scala.annotation.tailrec
object ListFindComparison {
def whileFind[A](lst: List[A])(p: A => Boolean): Option[A] = {
var these: List[A] = lst
while (these.nonEmpty) {
val h = these.head
if (p(h)) return Some(h)
else these = these.tail
}
None
}
def tailrecFind[A](lst: List[A])(p: A => Boolean): Option[A] = {
#tailrec
def iter(these: List[A]): Option[A] =
if (these.nonEmpty) {
val h = these.head
if (p(h)) Some(h)
else iter(these.tail)
} else None
iter(lst)
}
def tailRecPM[A](lst: List[A])(p: A => Boolean): Option[A] = {
#tailrec
def iter(these: List[A]): Option[A] =
these match {
case Nil => None
case head :: tail if p(head) => Some(head)
case _ => iter(these.tail)
}
iter(lst)
}
}
When inspecting the bytecode (using :javap ListFindComparison$), we see
For whileFind, the emitted code is straightforward
Code:
0: aload_1
1: astore_3
2: aload_3
3: invokevirtual #25 // Method scala/collection/immutable/List.nonEmpty:()Z
6: ifeq 50
9: aload_3
10: invokevirtual #29 // Method scala/collection/immutable/List.head:()Ljava/lang/Object;
13: astore 4
15: aload_2
16: aload 4
18: invokeinterface #35, 2 // InterfaceMethod scala/Function1.apply:(Ljava/lang/Object;)Ljava/lang/Object;
23: invokestatic #41 // Method scala/runtime/BoxesRunTime.unboxToBoolean:(Ljava/lang/Object;)Z
26: ifeq 39
29: new #43 // class scala/Some
32: dup
33: aload 4
35: invokespecial #46 // Method scala/Some."<init>":(Ljava/lang/Object;)V
38: areturn
39: aload_3
40: invokevirtual #49 // Method scala/collection/immutable/List.tail:()Ljava/lang/Object;
43: checkcast #21 // class scala/collection/immutable/List
46: astore_3
47: goto 2
50: getstatic #54 // Field scala/None$.MODULE$:Lscala/None$;
53: areturn
The tail-recursive finds are basically the same:
aload_0
aload_1
aload_2
invokespecial // call the appropriate (private) iter methods
areturn
The iter in tailrecFind is
Code:
0: aload_1
1: invokevirtual #25 // Method scala/collection/immutable/List.nonEmpty:()Z
4: ifeq 53
7: aload_1
8: invokevirtual #29 // Method scala/collection/immutable/List.head:()Ljava/lang/Object;
11: astore 4
13: aload_2
14: aload 4
16: invokeinterface #35, 2 // InterfaceMethod scala/Function1.apply:(Ljava/lang/Object;)Ljava/lang/Object;
21: invokestatic #41 // Method scala/runtime/BoxesRunTime.unboxToBoolean:(Ljava/lang/Object;)Z
24: ifeq 39
27: new #43 // class scala/Some
30: dup
31: aload 4
33: invokespecial #46 // Method scala/Some."<init>":(Ljava/lang/Object;)V
36: goto 50
39: aload_1
40: invokevirtual #49 // Method scala/collection/immutable/List.tail:()Ljava/lang/Object;
43: checkcast #21 // class scala/collection/immutable/List
46: astore_1
47: goto 0
50: goto 56
53: getstatic #54 // Field scala/None$.MODULE$:Lscala/None$;
56: areturn
There's no major difference in the core of the while and this iter: it's quite likely that JIT will bring these to the same machine code after enough invocations. tailrecFind has slightly greater constant overhead for getting into iter than whileFind has for getting into the loop. There's not likely to be a meaningful performance difference here (and in fact, since while is leaving the language definition in dotty, the future of while is as a library function which tail-recursively calls a block as long as a predicate passes).
The iter with pattern-matching is very different:
Code:
0: aload_1
1: astore 5
3: getstatic #77 // Field scala/collection/immutable/Nil$.MODULE$:Lscala/collection/immutable/Nil$;
6: aload 5
8: invokevirtual #80 // Method java/lang/Object.equals:(Ljava/lang/Object;)Z
11: ifeq 22
14: getstatic #54 // Field scala/None$.MODULE$:Lscala/None$;
17: astore 4
19: goto 92
22: goto 25
25: aload 5
27: instanceof #82 // class scala/collection/immutable/$colon$colon
30: ifeq 78
33: aload 5
35: checkcast #82 // class scala/collection/immutable/$colon$colon
38: astore 6
40: aload 6
42: invokevirtual #83 // Method scala/collection/immutable/$colon$colon.head:()Ljava/lang/Object;
45: astore 7
47: aload_2
48: aload 7
50: invokeinterface #35, 2 // InterfaceMethod scala/Function1.apply:(Ljava/lang/Object;)Ljava/lang/Object;
55: invokestatic #41 // Method scala/runtime/BoxesRunTime.unboxToBoolean:(Ljava/lang/Object;)Z
58: ifeq 75
61: new #43 // class scala/Some
64: dup
65: aload 7
67: invokespecial #46 // Method scala/Some."<init>":(Ljava/lang/Object;)V
70: astore 4
72: goto 92
75: goto 81
78: goto 81
81: aload_1
82: invokevirtual #49 // Method scala/collection/immutable/List.tail:()Ljava/lang/Object;
85: checkcast #21 // class scala/collection/immutable/List
88: astore_1
89: goto 0
92: aload 4
94: areturn
This is unlikely to be anywhere near as performant as the versions without pattern-matching (though to be fair, the branches will in practice be really easy for a predictor: not-taken (not-Nil), not-taken (::), not-taken (predicate fails), except for the very last run).
It's a little interesting to me that we get a call to equals when checking for Nil: it's probably still faster than isEmpty/nonEmpty, but it would be even faster without pattern-matching and with an explicit eq/ne against Nil.
I also note that pattern-matching against this is a bit of antipattern IMO: at that point, you're almost certainly better off using virtual method dispatch since you're basically implementing a slow vtable (it does have the advantage of potentially being pre-JIT'd if you put the common case first).
If you really care about performance, I'd try to avoid pattern-matching.
PS: I haven't analyzed the simple foldLeft solution:
lst.foldLeft(None) { (acc, v) =>
acc.orElse {
if (p(v)) Some(v)
else None
}
}
But since that doesn't short-circuit, I suspect it won't consistently beat any of the candidates, and even in the cases where there's no match before the last element, it might not even beat the pattern-match version then.