I'm a bit confused by this Scala notation:
List(1, 2, 3).foldLeft(0)((x, acc) => acc+x)
Both "0" and the function are arguments for foldLeft, why are they passed in two adjacent brackets groups? I'd aspect this to work:
List(1, 2, 3).foldLeft(0, ((x, acc) => acc+x))
But it doesn't. Can anyone explain this to me? Also, how and why to declare such a type of function? Thanks
Scala allows you to have multiple arguments list:
def foo(a: Int)(b: String) = ???
def bar(a: Int)(b: String)(c: Long) = ???
The reason for using such syntax for foldLeft is the way compiler does type inference: already inferred types in the previous group of arguments used to infer types in consecutive arguments group. In case of foldLeft it allows you to drop type ascription next to the (x, acc), so instead of:
List(1, 2, 3).foldLeft(0)((x: Int, acc: Int) => acc+x)
you can write just
List(1, 2, 3).foldLeft(0)((x, acc) => acc+x)
This is an example of multiple parameter lists in Scala. They're really just syntactic sugar for a normal method call (if you look at the class file's method signatures with javap you'll see that when compiled to Java bytecode they're all combined into a single argument list). The reason for supporting multiple parameter lists are twofold:
Passing functions as arguments: Scala will allow you to replace a parameter list that takes a single argument with a function literal in curly braces {}. For example, your code could be re-written as List(1, 2, 3).foldLeft(0) { (x, acc) => acc+x }, which might be considered more readable. (Then again, I'd just use List(1, 2, 3).foldLeft(0)(_+_) in this case...) Being able to use curly braces like this makes it possible for the user to declare new functions that look more like native syntax. A good example of this is the react function for Actors.
Type inference: There are some details of the type inference process (which I admit I don't fully understand) that make it easier to infer the types used in a later list based on the types in an earlier list. For example, the initial z value passed to foldLeft is used to infer the result type (and left argument type) of the function parameter.
Because in Scala you can define function arguments in multiple groups separated by ()
def test(a: String)(b: String)(implicit ev: Something) { }
The most practical scenario is where a context bound or currying is required, e.g. a specific implicit definition available in scope.
For instance, Future will expect an implicit executor. Look here.
If you look at the definition of the foldLeft method, you will see the first argument is an accumulator and the second a function that will be used for currying.
def foldLeft[B](z: B)(op: (B, A) ⇒ B): B
The parentheses thing is a very useful separation of concerns.
Also, once you define a method with:
def test(a: String)(b: String)
You can't call it with: test("a", "b");
Related
Lets say we have a function def fun(x: X): X => Y and we pass this function as a parameter to another function using fun _ instead of just fun. I understand that fun _ is actually a function value, while fun refers to a function definition.
For example let:
val a = List(1,2,3,4)
def fun(x: Int) = {println(x); x + 1}
Then Running:
//This line works
a.map(fun _)
//This one also works, even though "fun" is not a function value
a.map(fun)
They have the same output:
1
2
3
4
resX: List[Int] = List(2, 3, 4, 5)
For the most part they seem to work the same, are there any examples in which the function value is not equivalent to the function definition?
In the signature of map, you can see that it's expecting a
"function" to apply to each element
But in your code, fun is a regular method in a class. So when you do:
a.map(fun _)
you are explicitly asking for eta-expansion. When you do:
a.map(fun)
you are implicitly asking for eta-expansion.
Because fun is a "method", and is being used in a place where a Function type is expected, it's automagically converted to that type. Basically to something like:
new Function1[Int, Int] {
def apply(x: Int): Int = fun(x)
}
This transformation converting the name fun to a Function is called eta-expansion. See documentation for details.
Unfortunately, there are various ways of doing what you're doing - a.map(fun), a.map(fun _), a.map(fun(_)) and a.map(x => fun(x)). This is one of those frequent scenarios in Scala where you can explicitly do something yourself, or explicitly ask the compiler to do it for you, or just let the compiler do it implicitly. They can have different behaviors because of implicits and it can be a major source of confusion. Also, _ is heavily overloaded in the language, only adding to the confusion. So I use implicit behavior sparingly in general.
As others have pointed out in the comments, you need to use the fun _ syntax (which performs an eta expansion) when you need a value (methods have no value by themselves). Within the context of a map (or other functional contexts) an eta expansion is performed on the method implicitly. There are some cases where the eta expansion must be triggered manually.
As a concrete example of where an explicit eta expansion is needed, consider this valid snippet:
def f1(x: Int): Int = 2*x
def f2(x: Int): Int = 3*x
val fList1 = List(f1 _, f2 _)
fList1.map(_(2)) // List(4, 6)
as opposed to this invalid snippet.
val fList2 = List(f1, f2)
I'm learning Functional Programming, by following the book Functional Programming in Scala by Paul Chiusano and Rúnar Bjarnason. I'm specifically on chapter 3, where I am implementing some companion functions to a class representing a singly-linked list, that the authors provided.
package fpinscala.datastructures
sealed trait List[+A]
case object Nil extends List[Nothing]
case class Cons[+A](head: A, tail: List[A]) extends List[A]
object List {
def sum(ints: List[Int]): Int = ints match {
case Nil => 0
case Cons(x,xs) => x + sum(xs)
}
def product(ds: List[Double]): Double = ds match {
case Nil => 1.0
case Cons(0.0, _) => 0.0
case Cons(x,xs) => x * product(xs)
}
def apply[A](as: A*): List[A] =
if (as.isEmpty) Nil
else Cons(as.head, apply(as.tail: _*))
def tail[A](ls: List[A]): List[A] = ls match {
case Nil => Nil
case Cons(x,xs) => xs
}
... (more functions)
}
The functions I am implementing go inside the object List, being companion functions.
While implementing dropWhile, whose method signature is:
def dropWhile[A](l: List[A])(f: A => Boolean): List[A]
I came across some questions regarding partial function application:
In the book, the authors say that the predicate, f, is passed in a separate argument group to help the scala compiler with type inference because if we do this, Scala can determine the type of f without any annotation, based on what it knows about the type of the List , which makes the function more convenient to use.
So, if we passed f in the same argument group, scala would force the call to become something like this: val total = List.dropWhile(example, (x:Int) => 6%x==0 ) where we define the type of x explicitly and we would "lose" the possibility of partial function application, am I right?
However, why is partial function application useful in this case? Only to allow for type inference? Does it make sense to "partially apply" a function like dropWhile without applying the predicate f to it? Because it seems to me that the computation becomes "halted" before being useful if we don't apply f...
So... why is partial function application useful? And is this how it's always done or is it only something specific to Scala? I know Haskell has something called "complete inference" but I don't know exactly its implications...
Thanks in advance
There's a couple of questions scattered in there, so I'll try to answer them separately.
About the type inference, yes, separating the parameter lists helps the compile in inferring the type of f.
This is because Scala has linear local type inference (from left to right) and it uses the first parameter list to infer A (from the type of l). Then it can use this information to infer the type of f.
Given for example
dropWhile(List(1, 2, 3))(x => x < 3)
the compiler will perform this steps:
first parameter list
A is unknown.
a List[A] is expected
a List[Int] is provided (this is inferred by the type of the elements in the List)
=> A is an Int
second parameter list
we know A = Int
so we're expecting a function Int => Boolean as f
If you don't separate the two parameter lists, the compiler can't "stop" and decide the type of A before type-checking f. f would be part of the "conversation" while deciding the type of A so you would need to annotate it.
This is something Haskell can do better, since it uses a different type system (Hindley-Milner) that can also use information deriving from the context in which the function is applied. This is why it's also called "complete" or "universal".
Why does Scala doesn't feature an Hindley-Milner type system? Long story short, because Scala also supports sub-typing, which hardly co-exists with such a powerful type system. More on the subject:
Why is Scala's type inference not as powerful as Haskell's?
http://www.codecommit.com/blog/scala/what-is-hindley-milner-and-why-is-it-cool
http://www.scala-lang.org/old/node/4654
About partial application, the question "why is it useful" is definitely too broad to be answered here. However, in the specific dropWhile case, suppose you have a list of functions representing different "drop" conditions. Using a partially applied function you could do:
val list = List(1, 2, 3)
val conditions: List[Int => Boolean] = List(_ < 1, _ < 2, _ < 3)
conditions.map(dropWhile(list)) // List(List(1, 2, 3), List(2, 3), List(3))
Obviously, with a non-curried function (i.e. a single parameter list) you could have achieved the same with
val list = List(1, 2, 3)
val conditions: List[Int => Boolean] = List(_ < 1, _ < 2, _ < 3)
conditions.map(cond => dropWhile(list, cond))
but currying allows for more flexibility while composing functions.
More on the subject:
https://softwareengineering.stackexchange.com/questions/185585/what-is-the-advantage-of-currying
What's the difference between multiple parameters lists and multiple parameters per list in Scala?
Let's take a look at this code:
scala> val a = List(Some(4), None)
a: List[Option[Int]] = List(Some(4), None)
scala> a.flatMap( e=> e)
List[Int] = List(4)
Why would applying flatMap with the function { e => e } on a List[Option[T]] returns a List[T] with the None elements removed?
Specifically, what is the conceptual reasoning behind it -- is it based on some existing theory in functional programming? Is this behavior common in other functional languages?
This, while indeed useful, does feel a bit magical and arbitrary at the same time.
EDIT:
Thank you for your feedbacks and answer.
I have rewritten my question to put more emphasis on the conceptual nature of the question. Rather than the Scala specific implementation details, I'm more interested in knowing the formal concepts behind it.
Let's first look at the Scaladoc for Option's companion object. There we see an implicit conversion:
implicit def option2Iterable[A](xo: Option[A]): Iterable[A]
This means that any option can be implicitly converted to an Iterable, resulting in a collection with zero or one elements. If you have an Option[A] where you need an Iterable[A], the compiler will add the conversion for you.
In your example:
val a = List(Some(4), None)
a.flatMap(e => e)
We are calling List.flatMap, which takes a function A => GenTraversableOnce[B]. In this case, A is Option[Int] and B will be inferred as Int, because through the magic of implicit conversion, e when returned in that function will be converted from an Option[Int] to an Iterable[Int] (which is a subtype of GenTraversableOnce).
At this point, we've essentially done the following:
List(List(1), Nil).flatMap(e => e)
Or, to make our implicit explicit:
List(Option(1), None).flatMap(e => e.toList)
flatMap then works on Option as it does for any linear collection in Scala: take a function of A => List[B] (again, simplifying) and produce a flattened collection of List[B], un-nesting the nested collections in the process.
I assume you mean the support for mapping and filtering at the same time with flatMap:
scala> List(1, 2).flatMap {
| case i if i % 2 == 0 => Some(i)
| case i => None
| }
res0: List[Int] = List(2)
This works because Option's companion object includes an implicit conversion from Option[A] to Iterable[A], which is a GenTraversableOnce[A], which is what flatMap expects as the return type for its argument function.
It's a convenient idiom, but it doesn't really exist in other functional languages (at least the ones I'm familiar with), since it relies on Scala's weird mix of subtyping, implicit conversions, etc. Haskell for example provides similar functionality through mapMaybe for lists, though.
A short answer to your question is: the flatMap method of the List type is defined to work with a more general function type, not just a function that only produces a List[B] result type.
The general result type is IterableOnce[B], as shown in the faltMap method signature: final def flatMap[B](f: (A) => IterableOnce[B]): List[B]. The flatMap implementation is rather simple in that it applies the f function to each element and iterates over the result in a nested while loop. All results from the nested loop are added to a result of type List[B].
Therefore the flatMap works with any function that produces an IterableOnce[B] result from each list element. The IterableOnce is a trait that defines a minimal interface that is inherited by all iterable classes, including all collection types (Set, Map and etc.) and the Option class.
The Option class implementation returns collection.Iterator.empty for None and collection.Iterator.single(x) for Some(x). Therefore the flatMap method skips None element.
The question uses the identity function. It is better to use flatten method when the purpose is to flat iterable elements.
scala> val a = List(Some(4), None)
a: List[Option[Int]] = List(Some(4), None)
scala> a.flatten
res0: List[Int] = List(4)
How can I define a function that is accepting all the tuples(1 to 22) as argument, I have something as follows in mind:
def foo (v=Tuple) =...
foo((1,2))
foo((1,2,3))
EDIT:
To answer the comment: I am actually trying to create a Tensor class which is a set of values and a set of indices. The indices can be covariant and/or contravariant (cf Wikipedia1 and Wikipedia2). I wanted to have a special syntax like Tensor((1,2),(3,4),values) which would create a tensor with values, two covariant indices having length (2,3) and two contravariant indices with length (3,4). So using this syntax I could also write Tensor((1,2,3),3,values) (with an implicit Int=>Tuple1).
I agree that Tuples are not suitable for this, better to use Lists. However the syntax is not so nice then...
This really isn't what tuples are for (cf. the comments and answers here). Tuples are for doing things like returning multiple values from a method, where in Java you would have to create a lightweight class. If you have an arbitrary number of elements, you should use a collection.
Another way to provide a convenient API to your users (aside from implicit conversion) is to use multiple parameter lists with varargs:
def tensor(cov: Int*)(contrav: Int*)(values: Int*) = // ...
Your examples would be written
tensor(1,2)(3,4)(values)
tensor(1,2,3)(3)(values)
There is no trait specifically for tuples, but you could use a typeclass approach, as demonstrated in this answer.
If your goal is really to have a List but allow callers to pass in tuples (for convenience), you can modify that solution so that the type class produces a List rather than a Product.
In brief, the idea is that you provide implicit conversions from the types that callers can pass to the type you're actually going to use:
def foo(x: IndexList) = x.indices
sealed case class IndexList(indices: List[Int])
object IndexList {
implicit def val2indices(i: Int) = IndexList(List(i))
implicit def tuple2toIndices(t: (Int, Int)): IndexList =
product2indices(t)
// etc
implicit def list2indices(l: List[Int]) = IndexList(l)
private def product2indices(p: Product) =
IndexList(p.productIterator.toList.asInstanceOf[List[Int]])
}
You can then call your method with any type for which you've provided a conversion:
foo(1)
foo((2,3))
foo(List(1,2,3))
All case classes, including Tuples, extend scala.Product but unfortunately there's no marker trait specifically for tuples, so someone could sneak ordinary case classes into your method. Of course, there's no way to treat all arities in a uniform way and still be typesafe, but you can use productElement(n: Int) to extract the nth value, or productIterator to iterate over all the values.
But... This is heresy around here, but have you considered overloading? :)
What you probably want to use is an HList, not a tuple. An HList (heterogenous list) is basically an arbitrary-length, typed tuple.
There are a few examples of HLists in scala (they are not part of the standard library)
http://jnordenberg.blogspot.com/2008/08/hlist-in-scala.html
a great and comprehensive series by Mark Harrah (of SBT fame)
Miles Sabin's github examples, taken from his recent talk at Scala eXchange
Check this out. It actually works better than I expected ;)
scala> def f[T <: Product](x: T) = x
f: [T <: Product](x: T)T
scala> f(1)
<console>:9: error: inferred type arguments [Int] do not conform to method f's type parameter bounds [T <: Product]
scala> f(1, "2") // you don't even need the extra parenthesis
res0: (Int, java.lang.String) = (2,3)
scala> f(1, "2", BigInt("3"))
res1: (Int, java.lang.String, scala.math.BigInt) = (1,2,3)
This question already has answers here:
What's the difference between multiple parameters lists and multiple parameters per list in Scala?
(4 answers)
Closed 9 years ago.
Multiple parameters lists, e.g. def foo(a:Int)(b:Int) = {} and multiple parameters per list, e.g. def foo(a:Int, b:Int) = {} are semantically equivalent so far as I can tell, and most functional languages have only one way of declaring multiple parameters, e.g. F#.
The only reason I can figure out for supporting both these styles of function definitions is to allow syntax-like language extensions using a parameter list that has only one parameter in it.
def withBufferedWriter(file: File)(block: BufferedWriter => Unit)
can now be called with the syntax-looking
withBufferedWriter(new File("myfile.txt")) { out =>
out write "whatever"
...
}
However, there could be other ways of supporting the use of curly braces without having multiple parameter lists.
A related question: why is the use of multiple parameter lists in Scala called "currying"? Currying is usually defined as a technique for making an n-ary function unary for the sake of supporting partial application. However, in Scala one can partially apply a function without making a "curried" (multiple parameter lists with one param each) version of the function.
It makes you able to do e.g.:
scala> def foo(as: Int*)(bs: Int*)(cs: Int*) = as.sum * bs.sum * cs.sum
foo: (as: Int*)(bs: Int*)(cs: Int*)Int
scala> foo(1, 2, 3)(4, 5, 6, 7, 9)(10, 11)
res7: Int = 3906
As well as allowing you to write methods that look like part of the language (which you already spotted), it's worth noting that the type inferencer will work with one block at a time.
So in this:
def foo[T](a: T, b: T)(op: (T,T)=>T) = op(a,b)
foo(1,2){_+_}
T will first be inferred as Int, which will then be used as the type of the two underscores in the closure.
This is how the compiler then knows, with complete type safety, that the + operation is valid.
To answer your "related question," currying is simply a way of turning a function of multiple arguments, for example (A, B, C) => D, into a function which takes one argument and returns a function, e.g. A => (B => (C => D)) (parentheses shown but not necessary).
The tuple-ized form and the curried form are isomorphic, and we may translate freely between them. All of these are equivalent, but have different syntactic implications:
(A, B, C, D, E) => F
((A, B), (C, D, E)) => F
(A, B) => (C, D, E) => F
When you declare separate parameter groups, this is the kind of currying you're doing. The multi-parameter-group method is a method which returns a function... you can see this in the REPL:
scala> def foo(a:Int, b:Int)(c:Int, d:Int, e:Int):Int = 9
foo: (a: Int,b: Int)(c: Int,d: Int,e: Int)Int
scala> foo _
res4: (Int, Int) => (Int, Int, Int) => Int = <function2>
Back references in default arguments:
case class Foo(bar: Int)
def test(f: Foo)(i: Int = f.bar) = i*i
test(Foo(3))()
I know one of the motivations was implicit parameter lists. "implicit" is a property of the list, not the parameter. Another was probably case classes: only the first parameter list become case fields.