I'm following the tutorial Pattern matching & functional composition on Scala compose and andThen methods. There's such an example:
scala> def addUmm(x: String) = x + " umm"
scala> def addAhem(x: String) = x + " ahem"
val ummThenAhem = addAhem(_).compose(addUmm(_))
When I try to use it I get an error:
<console>:7: error: missing parameter type for expanded function ((x$1) => addAhem(x$1).compose(((x$2) => addUmm(x$2))))
val ummThenAhem = addAhem(_).compose(addUmm(_))
^
<console>:7: error: missing parameter type for expanded function ((x$2) => addUmm(x$2))
val ummThenAhem = addAhem(_).compose(addUmm(_))
^
<console>:7: error: type mismatch;
found : java.lang.String
required: Int
val ummThenAhem = addAhem(_).compose(addUmm(_))
However, this works:
val ummThenAhem = addAhem _ compose addUmm _
or even
val ummThenAhem = addAhem _ compose addUmm
What's wrong with the code in the tutorial? Isn't the latter expression the same as the first one without parenthesis?
Well, this:
addUhum _
is an eta expansion. It converts methods into functions. On the other hand, this:
addUhum(_)
is an anonymous function. In fact, it is a partial function application, in that this parameter is not applied, and the whole thing converted into a function. It expands to:
x => addUhum(x)
The exact rules for expansion are a bit difficult to explain, but, basically, the function will "start" at the innermost expression delimiter. The exception is partial function applications, where the "x" is moved outside the function -- if _ is used in place of a parameter.
Anyway, this is how it expands:
val ummThenAhem = x => addAhem(x).compose(y => addUmm(y))
Alas, the type inferencer doesn't know the type of x or y. If you wish, you can see exactly what it tried using the parameter -Ytyper-debug.
addAhem is a method. compose method is defined on functions. addAhem _ converts addAhem from method to function, so compose can be called on it. compose expects a function as it's argument. You are giving it a method addUmm by converting addUmm into a function with addUmm _ (The underscore can be left out because the compiler can automatically convert a method into a function when it knows that a function is expected anyway). So your code:
addAhem _ compose addUmm
is the same as
(addAhem _).compose(addUmm)
but not
addAhem(_).compose(addUmm(_))
PS
I didn't look at the link you provided.
From compose documentation:
Composes two instances of Function1 in a new Function1, with this
function applied last.
so you should write
scala> val ummThenAhem = (addAhem _).compose(addUmm _)
ummThenAhem: String => java.lang.String = <function1>
to treat addAhem and addUmm as partially applied functions (i.e function1)
scala> addAhem _
res0: String => java.lang.String = <function1>
I believe the tutorial was written for an earlier version of Scala (probably 2.7.7 or earlier). There have been some changes in the compiler since then, namely, extensions to the type system, which now cause the type inferencing to fail on the:
addUhum(_).compose(addAhem(_))
The lifting to a function still works with that syntax if you just write:
addUhum(_)
Related
2 different examples, the first one works:
import cats.syntax.either._
val e = 10.asRight[String]
def i2s(i:Int):String = i.toString
e.map(i => List(i2s(i))) //using explicit parameter
e.map(List(i2s(_))) //using no-name _ parameter
Now the same example with Option is not compiled:
e.map(Option(i2s(_)))
The error:
Error:(27, 15) type mismatch;
found : Option[Int => String]
required: Int => ?
e.map(Option(i2s(_)))
With explicit parameter it works fine:
e.map(i => Option(i2s(i)))
In both cases apply method is invoked with List and Option. List.apply signature:
def apply[A](xs: A*): List[A] = ???
Option.apply signature:
def apply[A](x: A): Option[A]
Please explain the difference.
Both of your List examples compile but they don't mean the same thing and don't produce the same results.
e.map(i => List(i2s(i))) //res0: scala.util.Either[String,List[String]] = Right(List(10))
e.map(List(i2s(_))) //java.lang.IndexOutOfBoundsException: 10
The 1st is easy to understand, so what's going on with the 2nd?
What's happening is that you're using eta expansion to create an Int => String function from the i2s() method. You then populate a List with that single function as the only element in the list, and then try to retrieve the value at index 10, which doesn't exist, thus the exception.
If you change the 1st line to val e = 0.asRight[String] then the exception goes away because something does exist at index 0, the function that was just put in there.
This compiles because a List instance will accept an Int as a parameter (via the hidden apply() method), but an Option instance does not have an apply() method that takes an Int (*) so that can't be compiled.
(*) The Option object does have an apply() method, but that's a different animal.
There are multiple things at play here as to why your first example with List[A] works. First, let's look at the expansion that happens on the expression:
val res: Either[String, Int => String] =
e.map[Int => String](List.apply[Int => String](((x$1: Int) => FlinkTest.this.i2s(x$1))));
Notice two things:
The expansion of the lambda expression happens inside List.apply, and perhaps not as you expected, for it to be outside of List.apply, like this:
e.map(i => List(i2s(i))
The return type from .map is somehow not Either[String, List[Int => String]], but Either[String, Int => String]. This is due to the fact that in it's hierarchy chain, List[A] extends PartialFunction[Int, A], thus allowing it to transform the result into a function type.
This doesn't work for Option[A], as it doesn't extend PartialFunction anywhere in it's type hierarchy.
The key takeaway here is that the expansion of the lambda expression doesn't work as you expect, as List(i2s(_)) expands to List(i2s(x => i2s(x)) and not List(i => i2s(i)). For more on underscore expansion, see What are all the uses of an underscore in Scala?
I`m experimenting with Scala and slightly confused by type inference problem.
Given the definition below, which compiles successfully:
case class SequentialHistory[+E](events: Seq[E])
trait Event
case class SampleEvent(value: String) extends Event
def addEvent[E, E2 >: E](event: E)(history:SequentialHistory[E2]):
SequentialHistory[E2] = history.copy(events = event +: history.events)
I would like to understand why type inference not working properly and compilation of:
val hist = SequentialHistory(Seq.empty[Event])
//hist: SequentialHistory[Event] = SequentialHistory(List())
val histWithEvent = addEvent(SampleEvent("Does not compile"))(hist)
//error
result in compiler error:
Error:(21, 62) type mismatch;
found : SequentialHistory[Event]
required: SequentialHistory[SampleEvent]
addEvent(SampleEvent("Does not compile"))(hist)
^
but, if I swap parameter lists in addEvent method definition:
def addEvent2[E, E2 >: E](history: SequentialHistory[E2])(event: E):
SequentialHistory[E2] = history.copy(events = event +: history.events)
that would solve the problem, types inferred correctly and snippet below compiles:
val hist2 = SequentialHistory(Seq.empty[Event])
//hist2: SequentialHistory[Event] = SequentialHistory(List())
val histWithEvent2 = addEvent2(hist)(SampleEvent("Compiles"))
//histWithEvent1: SequentialHistory[Event] =
// SequentialHistory(List(SampleEvent1(Compiles)))
Why Scala compiler can`t infer types correctly in first version of addEvent?
I can't explain it with references to the compiler code or quotes from the Scala specification, but here's my reasoning.
When your function has multiple parameter lists, it's "curried", so it's a function from the first argument that returns another function, etc. So if you try to apply it just to one argument, all types will be inferred:
scala> val partial = addEvent(SampleEvent(""))(_)
partial: SequentialHistory[SampleEvent] => SequentialHistory[SampleEvent] = <function1>
Now you can't pass the second argument with a wider type. What happens here is eta expansion. But another way round it won't work:
scala> val partial = addEvent2(hist)(_)
<console>:14: error: missing parameter type for expanded function ((x$1) => addEvent2(hist)(x$1))
val partial = addEvent2(hist)(_)
Here E2 is fixed by the hist argument, but compiler can't infer E. But if you provide it both arguments, it sees that their types conform to the restrictions. I think in this sense the second definition is the same as defining it like def addEvent2[E, E2 >: E](event: E, history: SequentialHistory[E2]).
In the spec of scala 6.26.2, there are 4 kind of conversions for methods:
MethodConversions
The following four implicit conversions can be applied to methods which are not applied to some argument list.
Evaluation. A parameterless method m of type => T is always converted to type T by evaluating the expression to which m is bound.
Implicit Application. If the method takes only implicit parameters, implicit argu- ments are passed following the rules of §7.2.
Eta Expansion. Otherwise, if the method is not a constructor, and the expected type pt is a function type (Ts′) ⇒ T′, eta-expansion (§6.26.5) is performed on the expression e.
Empty Application. Otherwise, if e has method type ()T , it is implicitly applied to the empty argument list, yielding e().
Some scala code:
def hello(): String = "abc"
def world1(s: String) = s
def world2(s: => String) = s
def world3(s: () => String) = s
world1(hello)
world2(hello)
world3(hello)
My question is, for world1, world2 and world3, which conversion rule is applied to each of them?
def world1(s: String) = s
world1(hello)
Empty application: hello is evaluated as hello() and then passed as the argument of world1
def world2(s: => String) = s
world2(hello)
Eta-expansion + Evaluation: hello is expanded into a function and evaluated lazily.
def world3(s: () => String) = s
world3(hello)
Eta-expansion: hello is expanded to a Function0[String]
Eta-expansion is necessary because methods and functions are two different things, even though the scala compiler is very good at hiding this difference.
The difference comes from the fact that methods are a JVM concept, whereas first-class functions are a scala-specific concept.
You can try this out in a REPL. First, let's define a method hello
scala> def hello(s: String) = s"hello $s"
hello: (s: String)String
scala> hello.hashCode
<console>:9: error: missing arguments for method hello;
follow this method with `_' if you want to treat it as a partially applied function
hello.hashCode
^
Woops, hello is not a value (in the JVM sense)! Not having an hashcode is a proof of it.
Let's get a Function1 out of the hello method
scala> hello _
res10: String => String = <function1>
scala> (hello _).hashCode
res11: Int = 1710358635
A function is a value, so it has an hashcode.
What is the parameter "_" in below method call signify ?
Is this a wildcard that accepts a parameter of any type ?
val integerSorter = msort[Int]((a, b) => a < b) _
The method msort signature :
def msort[T](less: (T, T) => Boolean)(xs: List[T]): List[T] = {
The easiest way to explain this is probably to let the compiler do most of the explaining—just try the first line without the underscore:
scala> val integerSorter = msort[Int]((a, b) => a < b)
<console>:11: error: missing arguments for method msort;
follow this method with `_' if you want to treat it as a partially applied function
val integerSorter = msort[Int]((a, b) => a < b)
^
So there you have it—the msort method has two parameter lists, but you've only passed arguments for the first, and the trailing underscore is the syntax that Scala provides to tell the compiler that you want partial application in that situation.
(If you try that line in the REPL with the underscore, you'll see that the inferred type of integerSorter is List[Int] => List[Int], so to answer your second question, no, the underscore doesn't allow you to provide a parameter of any type.)
For more information, see section 6.7 of the language specification:
The expression e _ is well-formed if e is of method type or if e is a
call-by-name parameter. If e is a method with parameters, e _
represents e converted to a function type by eta expansion
(§6.26.5).
Reading the section on eta expansion may also be helpful.
msort takes two parameters, a function that returns a boolean, and a list of items to be sorted. the function integerSorter supplies the first parameter, and the underscore represents the list that still needs to be specified. look up currying (http://www.scala-lang.org/old/node/135.html) for a more detailed explanation.
Background
I recently attended a beginner Scala meetup and we were talking about the difference between methods and functions (also discussed in-depth here).
For example:
scala> val one = 1
one: Int = 1
scala> val addOne = (x: Int) => x + 1
addOne: Int => Int = <function1>
This demonstrates that vals can not only have an integer type, but can have a function type. We can see the type in the scala repl:
scala> :type addOne
Int => Int
We can also define methods in objects and classes:
scala> object Foo {
| def timesTwo(op: Int) = op * 2
| }
defined module Foo
And while a method doesn't have a type (but rather is has a type signature), we can lift it into a function to see what it is:
scala> :type Foo.timesTwo
<console>:9: error: missing arguments for method timesTwo in object Foo;
follow this method with `_' if you want to treat it as a partially applied function
Foo.timesTwo
^
scala> :type Foo.timesTwo _
Int => Int
So far, so good. We even talked about how functions are actually objects with an apply method and how we can de-syntactic sugarify expressions to show this:
scala> Foo.timesTwo _ apply(4)
res0: Int = 8
scala> addOne.apply(3)
res1: Int = 4
To me, this is quite helpful in learning the language because I can internalize what the syntax is actually implying.
Problematic Example
We did, however, run into a situation that we could not identify. Take, for example, a list of strings. We can map functions over the values demonstrating basic Scala collections and functional programming stuff:
scala> List(1,2,3).map(_*4)
res2: List[Int] = List(4, 8, 12)
Ok, so what is the type of List(1,2,3).map()? I would expect we would do the same :type trick in the repl:
scala> :type List(1,2,3).map _
<console>:8: error: Cannot construct a collection of type Nothing with elements of type Nothing based on a collection of type List[Int].
List(1,2,3).map _
^
From the API definition, I know the signature is:
def map[B](f: (A) ⇒ B): List[B]
But there is also a full signature:
def map[B, That](f: (A) ⇒ B)(implicit bf: CanBuildFrom[List[A], B, That]): That
Question
So there are two things I don't quite understand:
Why doesn't the normal function lift trick work with List.map? Is there a way to de-syntactic sugar the erroneous statement to demonstrate what is going on?
If the reason that the method can't be lifted is due to the full signature "implicit", what exactly is going on there?
Finally, is there a robust way to inspect both types and signatures from the REPL?
The problem you've encountered has to do with the fact that, in Scala, functions are monomorphic, while methods can be polymorphic. As a result, the type parameters B and That must be known in order to create a function value for List.map.
The compiler attempts to infer the parameters but can't come up with anything sensible. If you supply parameters, you'll get a valid function type:
scala> List(1,2,3).map[Char, List[Char]] _
res0: (Int => Char) => List[Char] = <function1>
scala> :type res0
(Int => Char) => List[Char]
Without an actual function argument, the inferred type of the function is Int => Nothing, but the target collection is also Nothing. There is no suitable CanBuildFrom[List[Int], Nothing, Nothing] in scope, which we can see by entering implicitly[CanBuildFrom[List[Int], Nothing, Nothing]] in the REPL (comes up with same error). If you supply the type parameters, then you can get a function:
scala> :type List(1,2,3).map[Int, List[Int]] _
(Int => Int) => List[Int]
I don't think you can inspect method signatures in the REPL. That's what Scaladoc is for.