I can't understand why this code is breaking.
The both methods differ in input parameters.
object foo {
def x: String => String = x => x
def x: Option[String] => String = x => x.getOrElse("None")
}
Compiler says method x is defined twice; ...
How can I work around this?
I would like to keep the lambda notation, so pipelining is easier when using the function.
The answer to this specific question becomes clearer if you add the brackets to the methods:
object foo {
def x(): String => String = x => x
def x(): Option[String] => String = x => x.getOrElse("None")
}
This makes it clear that the methods only differ by return type, and polymorphism (in Scala) relies on the argument types being different.
The broader issue here is that polymorphism only applies to method invocation. It is resolved at compile time, not run time, so you can't have a value that is a polymorphic function.
This is OK:
object foo {
def poly(i: Int): Int = i
def poly(s: String): String = s
}
val i = foo.poly(1)
val s = foo.poly("a")
val y: Int => Int = foo.poly
This is NOT OK, because it would require x to hold a polymorphic function type:
val x = foo.poly
(Also note that the last two lines involve the compiler performing eta expansion from method to function)
Related
I was reading about scala anonymous functions here and saw that they can take the format:
{ case p1 => b1 … case pn => bn }
However, I thought that this was how partial functions were written. In fact, in this blog post, the author calls a partial function an anonymous function. At first he says that collect takes a partial function but then appears to call it an anonymous function ("collect can handle the fact that your anonymous function...").
Is it just that some anonymous functions are partial functions? If so, are all partial functions anonymous? Or are they only anonymous if in format like Alvin Alexander's example here:
val divide2: PartialFunction[Int, Int] = {
case d: Int if d != 0 => 42 / d
}
Anonymous and partial are different concepts. We would not say the following function is anonymous
val divide2: PartialFunction[Int, Int] = {
case d: Int if d != 0 => 42 / d
}
because it is bound to the name divide2, however we could say divide2 is defined in terms of the anonymous (function) value
{ case d: Int if d != 0 => 42 / d }
in the same sense x is defined in terms of anonymous value 42 in the following definition
val x: Int = 42
Orthogonal concept of partial refers to special subtype of function, as opposed to whether values of particular type are bound to a name or not.
From the documentation on pattern matching anonymous functions that you linked:
If the expected type is SAM-convertible to scala.Functionk[S1,…,Sk, R], the expression is taken to be equivalent to the anonymous
function:
(x1:S1,…,xk:Sk) => (x1,…,xk) match {
case p1 => b1 … case pn => bn
}
Here, each xi is a fresh name. As was shown here, this anonymous
function is in turn equivalent to the following instance creation
expression, where T is the weak least upper bound of the types of all
bi.
new scala.Functionk[S1,…,Sk, T] {
def apply(x1:S1,…,xk:Sk): T = (x1,…,xk) match {
case p1 => b1 …
case pn => bn
}
}
If the expected type is scala.PartialFunction[S, R], the expression is taken to be equivalent
to the following instance creation expression:
new scala.PartialFunction[S, T] {
def apply(x: S): T = x match {
case p1 => b1 … case pn => bn
}
def isDefinedAt(x: S): Boolean = {
case p1 => true … case pn => true
case _ => false
}
}
Your first code snippet is a pattern matching anonymous function, but not necessarily a partial function. It would only be turned into a PartialFunction if it was given to a method with a PartialFunction parameter or assigned to a variable of type PartialFunction.
So you are right that just some (pattern matching) anonymous functions are partial functions (AFAIK, function literals defined with fat arrows, such as x => x, can only ever be used to create FunctionN instances and not PartialFunction instances).
However, not all partial functions are anonymous functions. A sugar-free way to define PartialFunctions is extending the PartialFunction trait (which extends Function1) and manually overriding the isDefinedAt and apply methods. For example, divide2 could also be defined like this, with an anonymous class:
val divide2 = new PartialFunction[Int, Int] {
override def isDefinedAt(x: Int) = x != 0
override def apply(x: Int) = 42 / x
}
You probably won't see this very often, though, as it's a lot easier to just use pattern matching to define a PartialFunction.
In the blog post by Alvin Alexander you linked, the author refers to the pattern matching anonymous partial function literal as an anonymous function only because it happens to be both a partial function and an anonymous function. You could also define the function like so:
List(42, "cat").collect(new PartialFunction[Any, Int] {
def isDefinedAt(x: Any) = x.isInstanceOf[Int]
def apply(x: Any) = x match {
case i: Int => i + 1
}
})
It's no longer an anonymous function, although it's still an anonymous object that's the instance of an anonymous class. Or you could define a singleton object beforehand and then use that.
object Foo extends PartialFunction[Any, Int] {
def isDefinedAt(x: Any) = x.isInstanceOf[Int]
def apply(x: Any) = x match {
case i: Int => i + 1
}
}
List(42, "cat").collect(Foo)
No matter how you define it, though, it's a partial function.
Here's another way of writing your pattern matching partial function.
val divide = new PartialFunction[Int, Int] {
def apply(x: Int) = 42 / x
def isDefinedAt(x: Int) = x != 0
}
In essence, a partial function is a function that isn't defined for a set of inputs. It could be like in the example that it doesn't make sense to divide by 0, or you could want to restrict some specific values.
The nifty thing with partial functions is that it has synergies with orElse, andThen and collect. Depending on whether or not you're inputing a 0 in the divide function, your variable can be passed along to andThen if it wasn't a 0, can go through orElse if it was a 0. Finally, collect will only apply your partial function if it is defined on that input.
The way you create a partial function is usually through pattern matching with case as shown in your example.
Last thing, an anonymous function in Scala is like a lambda in Python. It's just a way of creating a function without "naming" it.
Eg
val f: Int => Int = (x: Int) => x * x
collect {
case a: Int => 1-a
}
I was wondering how it works if I want to defining a function that takes one or more parameters and a callable (a function), and why the annotation is like this.
I will take the code from this answer as example:
// Returning T, throwing the exception on failure
#annotation.tailrec
final def retry[T](n: Int)(fn: => T): T = {
util.Try { fn } match {
case util.Success(x) => x
case _ if n > 1 => retry(n - 1)(fn)
case util.Failure(e) => throw e
}
}
In this function there are a few interesting things:
The annotation tailrec.
Generic type function retry[T]
Parameter int
callable fn
My question is on point 4. Why and how the definition of this function takes two round brackets. If you want to pass a callable function to any function should you always use a round brackets next to the "list" of optional parameter? Why not put together with the parameters?
Thank you in advance
You can have multiple parameter lists in function declaration. It is mostly the same as merging all the parameters into one list (def foo(a: Int)(b: Int) is more or less equivalent to def foo(a: Int, b: Int)) with a few differences:
You can reference parameters from previous list(s) in declaration: def foo(a : Int, b: Int = a + 1) does not work, but def foo(a: Int)(b: Int = a +1) does.
Type parameters can be inferred between parameter lists: def foo[T](x: T, f: T => String): String ; foo(1, _.toString) doesn't work (you'd have to write foo[Int](1, _.toString), but def foo[T](x: T)(f: T => String); foo(1)(_.toString) does.
You can only declare the entire list as implicit, so, multiple lists are helpful when you need some parameters to be implicit, and not the others: def foo(a: Int)(implicit b: Configuration)
Then, there are some syntactical advantages - things you could do with the single list, but they'd just look uglier:
Currying:
def foo(a: Int)(b: Int): String
val bar: Int => String = foo(1)
You could write this with the single list too, but it wouldn't look as nice:
def foo(a: Int, b: Int): String
val bar: Int => String = foo(1, _)
Finally, to your question:
def retry[T](n: Int)(f: => T)
is nice, because parenthesis are optional around lists with just a single argument. So, this lets you write things like
retry(3) {
val c = createConnection
doStuff(c)
closeConnection(c)
}
which would look a lot uglier if f was merged into the same list.
Also, currying is useful:
val transformer = retry[Int](3)
Seq(1,2,3).map { n => transformer(n + 1) }
Seq(4,5,6).map { n => transformer(n * 2) }
To be honest you don't have to use multiple parameter lists in order to pass function as an argument. E.g.
def pass(string: String, fn: String => Unit): Unit = fn(string)
would totally work. However, how would you call it?
pass("test", s => println(s))
Some would find it clumsy. You would rather want to pass it like:
pass("test")(s => println(s))
or even
pass("test") { s =>
println(s)
}
to make it look as if function is a block appended to the pass called with one parameter.
With single parameter list you will be forced to write it like:
pass("test", println)
pass("test", s => println(s))
pass("test", { s => println(s) })
With last parameter curried you just get more comfortable syntax. In languages like Haskell, where currying happens automatically, you don't have to bother with syntactic design decisions like this one. Unfortunately Scala requires that you made these decisions explicitly.
If you want to pass a callable function to any function should you
always use a round brackets next to the "list" of optional parameter?
Why not put together with the parameters?
There is no such obligation, it's (mostly) a matter of style. IMO, it also leads to cleaner syntax. With two parameter lists, where the second one is the function yielding the result, we can call the retry method:
val res: Try[Int] retry(3) {
42
}
Instead, of we used a single parameter list, our method call would look like this:
val res: Try[Int] = retry(3, () => {
42
})
I find the first syntax cleaner, which also allows you to use retry as a curried method when only supplying the first parameter list
However, if we think of a more advanced use case, Scala type inference works between parameter lists, not inside them. That means that if we have a method:
def mapFun[T, U](xs: List[T])(f: T => U): List[U] = ???
We would be able to call our method without specifying the type of T or U at the call site:
val res: List[Int] = mapFun(List.empty[Int])(i => i + 1)
But if we used a single parameter list,
def mapFun2[T, U](xs: List[T], f: T => U): List[U] = ???
This won't compile:
val res2 = mapFun2(List.empty[Int], i => i + 1)
Instead, we'd need to write:
val res2 = mapFun2[Int, Int](List.empty[Int], i => i + 1)
To aid the compiler at choosing the right types.
There is a significant difference in how Scala resolves implicit conversions from "Magnet Pattern" for non-overloaded and overloaded methods.
Suppose there is a trait Apply (a variation of a "Magnet Pattern") implemented as follows.
trait Apply[A] {
def apply(): A
}
object Apply {
implicit def fromLazyVal[A](v: => A): Apply[A] = new Apply[A] {
def apply(): A = v
}
}
Now we create a trait Foo that has a single apply taking an instance of Apply so we can pass it any value of arbitrary type A since there an implicit conversion from A => Apply[A].
trait Foo[A] {
def apply(a: Apply[A]): A = a()
}
We can make sure it works as expected using REPL and this workaround to de-sugar Scala code.
scala> val foo = new Foo[String]{}
foo: Foo[String] = $anon$1#3a248e6a
scala> showCode(reify { foo { "foo" } }.tree)
res9: String =
$line21$read.foo.apply(
$read.INSTANCE.Apply.fromLazyVal("foo")
)
This works great, but suppose we pass a complex expression (with ;) to the apply method.
scala> val foo = new Foo[Int]{}
foo: Foo[Int] = $anon$1#5645b124
scala> var i = 0
i: Int = 0
scala> showCode(reify { foo { i = i + 1; i } }.tree)
res10: String =
$line23$read.foo.apply({
$line24$read.`i_=`($line24$read.i.+(1));
$read.INSTANCE.Apply.fromLazyVal($line24$read.i)
})
As we can see, an implicit conversion has been applied only on the last part of the complex expression (i.e., i), not to the whole expression. So, i = i + 1 was strictly evaluated at the moment we pass it to an apply method, which is not what we've been expecting.
Good (or bad) news. We can make scalac to use the whole expression in the implicit conversion. So i = i + 1 will be evaluated lazily as expected. To do so we (surprize, surprize!) we add an overload method Foo.apply that takes any type, but not Apply.
trait Foo[A] {
def apply(a: Apply[A]): A = a()
def apply(s: Symbol): Foo[A] = this
}
And then.
scala> var i = 0
i: Int = 0
scala> val foo = new Foo[Int]{}
foo: Foo[Int] = $anon$1#3ff00018
scala> showCode(reify { foo { i = i + 1; i } }.tree)
res11: String =
$line28$read.foo.apply($read.INSTANCE.Apply.fromLazyVal({
$line27$read.`i_=`($line27$read.i.+(1));
$line27$read.i
}))
As we can see, the entire expression i = i + 1; i made it under the implicit conversion as expected.
So my question is why is that? Why the scope of which an implicit conversion is applied depends on the fact whether or not there is an overloaded method in the class.
Now, that is a tricky one. And it's actually pretty awesome, I didn't know that "workaround" to the "lazy implicit does not cover full block" problem. Thanks for that!
What happens is related to expected types, and how they affect type inference works, implicit conversions, and overloads.
Type inference and expected types
First, we have to know that type inference in Scala is bi-directional. Most of the inference works bottom-up (given a: Int and b: Int, infer a + b: Int), but some things are top-down. For example, inferring the parameter types of a lambda is top-down:
def foo(f: Int => Int): Int = f(42)
foo(x => x + 1)
In the second line, after resolving foo to be def foo(f: Int => Int): Int, the type inferencer can tell that x must be of type Int. It does so before typechecking the lambda itself. It propagates type information from the function application down to the lambda, which is a parameter.
Top-down inference basically relies on the notion of expected type. When typechecking a node of the AST of the program, the typechecker does not start empty-handed. It receives an expected type from "above" (in this case, the function application node). When typechecking the lambda x => x + 1 in the above example, the expected type is Int => Int, because we know what parameter type is expected by foo. This drives the type inference into inferring Int for the parameter x, which in turn allows to typecheck x + 1.
Expected types are propagated down certain constructs, e.g., blocks ({}) and the branches of ifs and matches. Hence, you could also call foo with
foo({
val y = 1
x => x + y
})
and the typechecker is still able to infer x: Int. That is because, when typechecking the block { ... }, the expected type Int => Int is passed down to the typechecking of the last expression, i.e., x => x + y.
Implicit conversions and expected types
Now, we have to introduce implicit conversions into the mix. When typechecking a node produces a value of type T, but the expected type for that node is U where T <: U is false, the typechecker looks for an implicit T => U (I'm probably simplifying things a bit here, but the gist is still true). This is why your first example does not work. Let us look at it closely:
trait Foo[A] {
def apply(a: Apply[A]): A = a()
}
val foo = new Foo[Int] {}
foo({
i = i + 1
i
})
When calling foo.apply, the expected type for the parameter (i.e., the block) is Apply[Int] (A has already been instantiated to Int). We can "write" this typechecker "state" like this:
{
i = i + 1
i
}: Apply[Int]
This expected type is passed down to the last expression of the block, which gives:
{
i = i + 1
(i: Apply[Int])
}
at this point, since i: Int and the expected type is Apply[Int], the typechecker finds the implicit conversion:
{
i = i + 1
fromLazyVal[Int](i)
}
which causes only i to be lazified.
Overloads and expected types
OK, time to throw overloads in there! When the typechecker sees an application of an overload method, it has much more trouble deciding on an expected type. We can see that with the following example:
object Foo {
def apply(f: Int => Int): Int = f(42)
def apply(f: String => String): String = f("hello")
}
Foo(x => x + 1)
gives:
error: missing parameter type
Foo(x => x + 1)
^
In this case, the failure of the typechecker to figure out an expected type causes the parameter type not to be inferred.
If we take your "solution" to your issue, we have a different consequence:
trait Foo[A] {
def apply(a: Apply[A]): A = a()
def apply(s: Symbol): Foo[A] = this
}
val foo = new Foo[Int] {}
foo({
i = i + 1
i
})
Now when typechecking the block, the typechecker has no expected type to work with. It will therefore typecheck the last expression without expression, and eventually typecheck the whole block as an Int:
{
i = i + 1
i
}: Int
Only now, with an already typechecked argument, does it try to resolve the overloads. Since none of the overloads conforms directly, it tries to apply an implicit conversion from Int to either Apply[Int] or Symbol. It finds fromLazyVal[Int], which it applies to the entire argument. It does not push it inside the block anymore, giving:
fromLazyVal({
i = i + 1
i
}): Apply[Int]
In this case, the whole block is lazified.
This concludes the explanation. To summarize, the major difference is the presence vs absence of an expected type when typechecking the block. With an expected type, the implicit conversion is pushed down as much as possible, down to just i. Without the expected type, the implicit conversion is applied a posteriori on the entire argument, i.e., the whole block.
By assigning a variable (or value?) a method name with a space and an underscore, you tell scala to treat the method as a function, which apparently means doing more than simply taking the value generated by a call to the method and assigning to the variable. What else is/can go on through such an assignment?
Since Scala runs on the JVM, it's easier to understand in terms of simple Java-like classes without Scala's syntactic sugar.
Remember that Scala functions are essentially members of a class similar to the following (signature deliberately simplified):
class Function[X, Y] {
def apply(x: X): Y
}
Application of a function f to an argument x is desugared into a method application f.apply(x).
Now suppose that you have another class Foo with method bar:
class Foo {
def bar(x: Int): String
}
If you now have an instance foo of type Foo, then whenever its method bar is transformed into a function by writing:
val f = foo.bar(_)
a new instance of an anonymous subclass of Function is created:
val f = new Function[Int, String] {
def apply(x: Int) = foo.bar(x)
}
If you use this syntax inside a class, this is closed over instead of an instance foo.
This is what all those weirdly named classes Main$$anon$1$$anonfun$1 are: they are the anonymous classes that represent functions. The functions can appear quite implicitly (for example, as blocks passed to the for-loops).
That's all there is to it semantically. The rest is just syntactic sugar.
Here is a complete runnable example that demonstrates the conversion of an instance method into a function:
with sugar, from the outside (a)
with sugar, from the inside (b)
without sugar, from the outside (c)
without sugar, from the inside (d)
You can save it into a file and execute with scala <filename.scala>:
/** A simple greeter that prints 'hello name' multiple times */
case class Hey(name: String) { thisHeyInst =>
def hello(x: Int): String = ("hello " + name + " ") * x
def withSugarFromInside = hello(_)
def noSugarFromInside = new Function[Int, String] {
def apply(y: Int) = thisHeyInst.hello(y)
}
}
val heyAlice = Hey("Alice")
val heyBob = Hey("Bob")
val heyCharlie = Hey("Charlie")
val heyDonald = Hey("Donald")
val a = heyAlice.hello(_)
val b = heyBob.withSugarFromInside
val c = new Function[Int, String] { def apply(y: Int) = heyCharlie.hello(y) }
val d = heyDonald.noSugarFromInside
println(a(3))
println(b(3))
println(c(3))
println(d(3))
In all four cases, a greeting is printed three times.
What _ actually does is an eta-conversion. It takes compile-time construction called method and returns runtime construction called anonymous function, which is actually an instance of scala's Function. Exactly the class depends on arity, so it might be Function1, Function2, Function3 and so on. The point here is to make First-class citizen, which may act like a value.
OOP needs a little more than some new object. Before making the code that creates instance, compiler generates a new class (extending FunctionN) in compile-time, but theoretically it shouldn't be necessary a whole new class. For Java 8 it could be native Java-lambdas.
Btw, you may extend Function1 by yourself and even eta-abstract it again:
scala> object f extends (Int => Int) { def apply(a: Int) = a }
scala> f(1)
res0: Int = 1
scala> f.apply _
res1: Int => Int = <function1>
scala> res1(5)
res2: Int = 5
As a conclusion a little copy-paste from #Daniel C. Sobral's answer:
the former can be easily converted into the latter:
val f = m _
Scala will expand that, assuming m type is (List[Int])AnyRef
into (Scala 2.7):
val f = new AnyRef with Function1[List[Int], AnyRef] {
def apply(x$1: List[Int]) = this.m(x$1)
}
On Scala 2.8, it actually uses an AbstractFunction1 class to reduce
class sizes.
Or simply saying val f = m _ is same as val f = (x: List[Int]) => m(x)
To make this answer more modern and precise let's see what's happening using scalac 2.11.2 and javap:
$ echo "object Z{def f(a: Int) = a}" > Z.scala //no eta-abstraction here
$ scalac Z.scala
$ ls
Z$.class Z.class Z.scala
$ echo "object Z{def f(a: Int) = a; val k = f _}" > Z.scala
$ scalac Z.scala
$ ls
Z$$anonfun$1.class Z.class //new anonfun class added for lambda
Z$.class Z.scala
$ javap -c Z\$\$anonfun\$1.class
Compiled from "Z.scala" // I've simplified output a bit
public final class Z$$anonfun$1 extends scala.runtime.AbstractFunction1$mcII$sp implements scala.Serializable {
public final int apply(int);
Code:
calling apply$mcII$sp(int)
public int apply$mcII$sp(int); //that's it
Code:
0: getstatic #25 // reading Field Z$.MODULE$:LZ$;, which points to `object Z`
3: iload_1
4: invokevirtual #28 // calling Method Z$.f
7: ireturn
public final java.lang.Object apply(java.lang.Object); //just boxed version of `apply`
Code:
unboxToInt
calling apply(int) method
boxToInteger
public Z$$anonfun$1();
Code:
AbstractFunction1$mcII$sp."<init>":()V //initialize
}
So it still extends AbstractFunction1
I'll try to provide some examples how a function or method are assigned to values with underscore.
If it's need to reference a zero-argument function
scala> val uuid = java.util.UUID.randomUUID _
uuid: () => java.util.UUID = <function0>
scala> uuid()
res15: java.util.UUID = 3057ef51-8407-44c8-a09e-e2f4396f566e
scala> uuid()
uuid: java.util.UUID = c1e934e4-e722-4279-8a86-004fed8b9090
Check how it's different when one does
scala> val uuid = java.util.UUID.randomUUID
uuid: java.util.UUID = 292708cb-14dc-4ace-a56b-4ed80d7ccfc7
In first case one assigned a reference to a function. And then calling uuid() generates new UUID every time.
In second, function randomUUID has been called and value assigned to a val uuid.
There are some other cases why _ might be useful.
It's possible to use a function with two arguments and create a function with a single argument out of it.
scala> def multiply(n: Int)(m: Int) = n*m
multiply: (n: Int)(m: Int)Int
scala> val by2 = multiply(2) _
by2: Int => Int = <function1>
scala> by2(3)
res16: Int = 6
To be able to do, it's crucial to define function multiply as curried. It's called function currying.
Suppose I want to pass, as an argument to a higher-order function, a function object that takes two parameter lists. What is the type of such an object?
For example, this might be such an object:
object passMe extends <???> {
def apply(x: Double)(implicit ctx: Context): String = . . .
}
What should I put after extends? Or how would I declare a function that would accept passMe as a parameter?
def foundation(xs: Seq[Double], oneOfThoseObjects: <???>)(implicit ctx: Context): Unit = {
. . .
xs.map(oneOfThoseObjects).toSet. . . .
}
My example uses an implicit parameter list, since that's mainly what I'm interested in, but it would still help to know how to declare a function with two explicit parameter lists. I haven't found anything yet in the Function1, Function2, ... series that suggests what such a declaration would look like.
A function with two parameter lists can be modeled as a function that returns another function. This is a technique called currying. For example, the addition function can be written like this:
scala> val add:Function1[Int,Function1[Int, Int]] = x => (y => x + y)
add: Int => (Int => Int) = <function1>
And called like this:
scala> add(1)(2)
res1: Int = 3
As you can see, the same type can be written both with Function1 and with => syntax.
After some playing around in the REPL, I arrived at this:
(Double => (Context => String))
Which, in English, says, "Function taking a double and returning a function taking a Context and returning a String"
Thus, your passMe object would look something like this:
object passMe extends (Double => (Context => String)) {
def apply(d: Double) = {
def f(c: Context) = "...something..."
f _
}
}
However, because of the implicit parameter, this won't work, because with the implicit parameter, you end up with a type mismatch (you need a scala.collection.immutable.WrappedString). So I eventually ended up with this:
object passMe extends (Double => scala.collection.immutable.WrappedString) {
def apply(d: Double) = {
def f(implicit c: Context) = "...something..."
f _
}
}