2 different examples, the first one works:
import cats.syntax.either._
val e = 10.asRight[String]
def i2s(i:Int):String = i.toString
e.map(i => List(i2s(i))) //using explicit parameter
e.map(List(i2s(_))) //using no-name _ parameter
Now the same example with Option is not compiled:
e.map(Option(i2s(_)))
The error:
Error:(27, 15) type mismatch;
found : Option[Int => String]
required: Int => ?
e.map(Option(i2s(_)))
With explicit parameter it works fine:
e.map(i => Option(i2s(i)))
In both cases apply method is invoked with List and Option. List.apply signature:
def apply[A](xs: A*): List[A] = ???
Option.apply signature:
def apply[A](x: A): Option[A]
Please explain the difference.
Both of your List examples compile but they don't mean the same thing and don't produce the same results.
e.map(i => List(i2s(i))) //res0: scala.util.Either[String,List[String]] = Right(List(10))
e.map(List(i2s(_))) //java.lang.IndexOutOfBoundsException: 10
The 1st is easy to understand, so what's going on with the 2nd?
What's happening is that you're using eta expansion to create an Int => String function from the i2s() method. You then populate a List with that single function as the only element in the list, and then try to retrieve the value at index 10, which doesn't exist, thus the exception.
If you change the 1st line to val e = 0.asRight[String] then the exception goes away because something does exist at index 0, the function that was just put in there.
This compiles because a List instance will accept an Int as a parameter (via the hidden apply() method), but an Option instance does not have an apply() method that takes an Int (*) so that can't be compiled.
(*) The Option object does have an apply() method, but that's a different animal.
There are multiple things at play here as to why your first example with List[A] works. First, let's look at the expansion that happens on the expression:
val res: Either[String, Int => String] =
e.map[Int => String](List.apply[Int => String](((x$1: Int) => FlinkTest.this.i2s(x$1))));
Notice two things:
The expansion of the lambda expression happens inside List.apply, and perhaps not as you expected, for it to be outside of List.apply, like this:
e.map(i => List(i2s(i))
The return type from .map is somehow not Either[String, List[Int => String]], but Either[String, Int => String]. This is due to the fact that in it's hierarchy chain, List[A] extends PartialFunction[Int, A], thus allowing it to transform the result into a function type.
This doesn't work for Option[A], as it doesn't extend PartialFunction anywhere in it's type hierarchy.
The key takeaway here is that the expansion of the lambda expression doesn't work as you expect, as List(i2s(_)) expands to List(i2s(x => i2s(x)) and not List(i => i2s(i)). For more on underscore expansion, see What are all the uses of an underscore in Scala?
Related
I was wondering how it works if I want to defining a function that takes one or more parameters and a callable (a function), and why the annotation is like this.
I will take the code from this answer as example:
// Returning T, throwing the exception on failure
#annotation.tailrec
final def retry[T](n: Int)(fn: => T): T = {
util.Try { fn } match {
case util.Success(x) => x
case _ if n > 1 => retry(n - 1)(fn)
case util.Failure(e) => throw e
}
}
In this function there are a few interesting things:
The annotation tailrec.
Generic type function retry[T]
Parameter int
callable fn
My question is on point 4. Why and how the definition of this function takes two round brackets. If you want to pass a callable function to any function should you always use a round brackets next to the "list" of optional parameter? Why not put together with the parameters?
Thank you in advance
You can have multiple parameter lists in function declaration. It is mostly the same as merging all the parameters into one list (def foo(a: Int)(b: Int) is more or less equivalent to def foo(a: Int, b: Int)) with a few differences:
You can reference parameters from previous list(s) in declaration: def foo(a : Int, b: Int = a + 1) does not work, but def foo(a: Int)(b: Int = a +1) does.
Type parameters can be inferred between parameter lists: def foo[T](x: T, f: T => String): String ; foo(1, _.toString) doesn't work (you'd have to write foo[Int](1, _.toString), but def foo[T](x: T)(f: T => String); foo(1)(_.toString) does.
You can only declare the entire list as implicit, so, multiple lists are helpful when you need some parameters to be implicit, and not the others: def foo(a: Int)(implicit b: Configuration)
Then, there are some syntactical advantages - things you could do with the single list, but they'd just look uglier:
Currying:
def foo(a: Int)(b: Int): String
val bar: Int => String = foo(1)
You could write this with the single list too, but it wouldn't look as nice:
def foo(a: Int, b: Int): String
val bar: Int => String = foo(1, _)
Finally, to your question:
def retry[T](n: Int)(f: => T)
is nice, because parenthesis are optional around lists with just a single argument. So, this lets you write things like
retry(3) {
val c = createConnection
doStuff(c)
closeConnection(c)
}
which would look a lot uglier if f was merged into the same list.
Also, currying is useful:
val transformer = retry[Int](3)
Seq(1,2,3).map { n => transformer(n + 1) }
Seq(4,5,6).map { n => transformer(n * 2) }
To be honest you don't have to use multiple parameter lists in order to pass function as an argument. E.g.
def pass(string: String, fn: String => Unit): Unit = fn(string)
would totally work. However, how would you call it?
pass("test", s => println(s))
Some would find it clumsy. You would rather want to pass it like:
pass("test")(s => println(s))
or even
pass("test") { s =>
println(s)
}
to make it look as if function is a block appended to the pass called with one parameter.
With single parameter list you will be forced to write it like:
pass("test", println)
pass("test", s => println(s))
pass("test", { s => println(s) })
With last parameter curried you just get more comfortable syntax. In languages like Haskell, where currying happens automatically, you don't have to bother with syntactic design decisions like this one. Unfortunately Scala requires that you made these decisions explicitly.
If you want to pass a callable function to any function should you
always use a round brackets next to the "list" of optional parameter?
Why not put together with the parameters?
There is no such obligation, it's (mostly) a matter of style. IMO, it also leads to cleaner syntax. With two parameter lists, where the second one is the function yielding the result, we can call the retry method:
val res: Try[Int] retry(3) {
42
}
Instead, of we used a single parameter list, our method call would look like this:
val res: Try[Int] = retry(3, () => {
42
})
I find the first syntax cleaner, which also allows you to use retry as a curried method when only supplying the first parameter list
However, if we think of a more advanced use case, Scala type inference works between parameter lists, not inside them. That means that if we have a method:
def mapFun[T, U](xs: List[T])(f: T => U): List[U] = ???
We would be able to call our method without specifying the type of T or U at the call site:
val res: List[Int] = mapFun(List.empty[Int])(i => i + 1)
But if we used a single parameter list,
def mapFun2[T, U](xs: List[T], f: T => U): List[U] = ???
This won't compile:
val res2 = mapFun2(List.empty[Int], i => i + 1)
Instead, we'd need to write:
val res2 = mapFun2[Int, Int](List.empty[Int], i => i + 1)
To aid the compiler at choosing the right types.
Some places (e.g. Play Form.bindFromRequest) use an empty parameter list before the implicit parameter list. Why?
IMHO, it has the disadvantage of requiring extra parentheses when the parameter is explicitly passed as in form.bindFromRequest()(request).
Not sure what is the advantage then?
A def with a parameter list has a different type than one with no parameter list. This doesn't matter under direct invocation, but if you pass that method as an argument to another method, it does.
In example, if you define your method like so:
def func1 = { println("Hello"); 1 }
You can't pass it to this method:
def consume(f: () => Double) = // ...
as it's type is simply Double, albeit a very lazy one. On the other hand this function will work fine:
def func2() = { println("Hello"); 2 }
I'm not saying this is the explicit reason why they did it, but if they had an actual reason, it will almost certainly be tied to type.
EDIT:
The difference between them in practical terms it mostly comes down to where they can be used.
The call-by-value item can only be used in a parameter list to a function/method (afaik). Since it can only be passed in a parameter list, you can't store it in a variable and use it in multiple locations (without changing it to an explicit instance of T).
And as you can see here, they aren't interchangeable per se:
scala> def f(s: => String) { println(s) }
f: (s: => String)Unit
scala> def s1 = () => { "Hello" }
s1: () => String
scala> f(s1)
<console>:10: error: type mismatch;
found : () => String
required: String
f(s1)
^
So imagine I wanted to store an array of callbacks that users pass to me.. I can't use => T here:
scala> val a: Array[=> Int] = Array()
<console>:1: error: identifier expected but '=>' found.
val a: Array[=> Int] = Array()
^
scala> val a: Array[() => Int] = Array()
a: Array[() => Int] = Array()
So if I want to store such items, and pass them around internally, using => T (and keeping it lazily evaluated) is not an option.
I think it's best to not think of the => in => T and () => T as meaning the same thing. Hope this helps a little.
Given
val strings = Set("Hi", "there", "friend")
def numberOfCharsDiv2(s: String) = scala.util.Try {
if (s.length % 2 == 0) s.length / 2 else throw new RuntimeException("grr")
}
Why can't I flatMap away the Try resulting from the method call? i.e.
strings.flatMap(numberOfCharsDiv2)
<console>:10: error: type mismatch;
found : scala.util.Try[Int]
required: scala.collection.GenTraversableOnce[?]
strings.flatMap(numberOfCharsDiv2)
or
for {
s <- strings
n <- numberOfCharsDiv2(s)
} yield n
<console>:12: error: type mismatch;
found : scala.util.Try[Int]
required: scala.collection.GenTraversableOnce[?]
n <- numberOfCharsDiv2(s)
However if I use Option instead of Try there's no problem.
def numberOfCharsDiv2(s: String) = if (s.length % 2 == 0)
Some(s.length / 2) else None
strings.flatMap(numberOfCharsDiv2) # => Set(1, 3)
What's the rationale behind not allowing flatMap on Try?
Let's look at the signature of flatMap.
def flatMap[B](f: (A) => GenTraversableOnce[B]): Set[B]
Your numberOfCharsDiv2 is seen as String => Try[Int]. Try is not a subclass of GenTraversableOnce and that is why you get the error. You don't strictly need a function that gives a Set only because you use flatMap on a Set. The function basically has to return any kind of collection.
So why does it work with Option? Option is also not a subclass of GenTraversableOnce, but there exists an implicit conversion inside the Option companion object, that transforms it into a List.
implicit def option2Iterable[A](xo: Option[A]): Iterable[A] = xo.toList
Then one question remains. Why not have an implicit conversion for Try as well? Because you will probably not get what you want.
flatMap can be seen as a map followed by a flatten.
Imagine you have a List[Option[Int]] like List(Some(1), None, Some(2)). Then flatten will give you List(1,2) of type List[Int].
Now look at an example with Try. List(Success(1), Failure(exception), Success(2)) of type List[Try[Int]].
How will flatten work with the failure now?
Should it disappear like None? Then why not work directly with Option?
Should it be included in the result? Then it would be List(1, exception, 2). The problem here is that the type is List[Any], because you have to find a common super class for Int and Throwable. You lose the type.
These should be reasons why there isn't an implicit conversion. Of course you are free to define one yourself, if you accept the above consequences.
The problem is that in your example, you're not flatmapping over Try. The flatmap you are doing is over Set.
Flatmap over Set takes a Set[A], and a function from A to Set[B]. As Kigyo points out in his comment below this isn't the actual type signature of flatmap on Set in Scala, but the general form of flat map is:
M[A] => (A => M[B]) => M[B]
That is, it takes some higher-kinded type, along with a function that operates on elements of the type in that higher-kinded type, and it gives you back the same higher-kinded type with the mapped elements.
In your case, this means that for each element of your Set, flatmap expects a call to a function that takes a String, and returns a Set of some type B which could be String (or could be anything else).
Your function
numberOfCharsDiv2(s: String)
correctly takes a String, but incorrectly returns a Try, rather then another Set as flatmap requires.
Your code would work if you used 'map', as that allows you to take some structure - in this case Set and run a function over each element transforming it from an A to a B without the function's return type conforming to the enclosing structure i.e. returning a Set
strings.map(numberOfCharsDiv2)
res2: scala.collection.immutable.Set[scala.util.Try[Int]] = Set(Success(1), Failure(java.lang.RuntimeException: grr), Success(3))
It is a Monad in Scala 2.11:
scala> import scala.util._
import scala.util._
scala> val x: Try[String] = Success[String]("abc")
x: scala.util.Try[String] = Success(abc)
scala> val y: Try[String] = Failure[String](new Exception("oops"))
y: scala.util.Try[String] = Failure(java.lang.Exception: oops)
scala> val z = Try(x)
z: scala.util.Try[scala.util.Try[String]] = Success(Success(abc))
scala> val t = Try(y)
t: scala.util.Try[scala.util.Try[String]] = Success(Failure(java.lang.Exception: oops))
scala> z.flatten
res2: scala.util.Try[String] = Success(abc)
scala> t.flatten
res3: scala.util.Try[String] =
Failure(java.lang.UnsupportedOperationException: oops)
Kigyo explains well why Scala does not do this implicitly. To put it simply, Scala does not want to automatically throw away the exception that is preserved in a Try.
Scala does provide a simple way to explicitly translate a Try into an Option though. This is how you can use a Try in a flatmap:
strings.flatMap(numberOfCharsDiv2(_).toOption)
def t[A] = (l:List[A]) => l tail
def r[A] = (r:List[A]) => r reverse
def tr[A] :List[A] => List[A] = t compose r
tr(List(1,2,3,4))
List(3, 2, 1)
As expected.
But somehow, every variant (with several variations of type annotations) I have tried on
def tr = tail compose reverse
fails (not found - value tail). I am missing something obvious, but I am stuck.
Well let's start from the back, what compose function does.
It is defined in the Function1 as def compose[A](g: (A) => T1): (A) => R, and description
Composes two instances of Function1 in a new Function1, with this function applied last.
What it does it takes another function g, taking parameter of some type - A, and returning same type T1. It creates new function which , when called is equivalent to call f(g(x)).
The two functions you have defined:
def t[A] = (l:List[A]) => l.tail
def r[A] = (r:List[A]) => r.reverse
They take no parameters, but return functions which take a list and call tail and reverse respectively on the argument.
The same you could write like:
def t[A](l: List[A]): List[A] = l.tail
def r[A](l: List[A]): List[A] = l.reverse
so the composition of two functions is t(r(List(1,2,3,4))).
The tail or reverse however are not functions, they are method defined in a List class. They must be called on an object, and there is no way you could do tail(reverse(arg)).
compose is a function (i.e. Function1) composition.
List[T].tail and List[T].reverse are List[T] methods
in scala, A => B will be treated as Function1 which define the method compose, so if u want to use tail compose reverse, you need to give scala compiler hint that what "class" it is, and then change a "method" to a "function".
the example :
def tr[A] = ((x: List[A]) => x.tail) compose ((x: List[A]) => x.reverse)
Background
I recently attended a beginner Scala meetup and we were talking about the difference between methods and functions (also discussed in-depth here).
For example:
scala> val one = 1
one: Int = 1
scala> val addOne = (x: Int) => x + 1
addOne: Int => Int = <function1>
This demonstrates that vals can not only have an integer type, but can have a function type. We can see the type in the scala repl:
scala> :type addOne
Int => Int
We can also define methods in objects and classes:
scala> object Foo {
| def timesTwo(op: Int) = op * 2
| }
defined module Foo
And while a method doesn't have a type (but rather is has a type signature), we can lift it into a function to see what it is:
scala> :type Foo.timesTwo
<console>:9: error: missing arguments for method timesTwo in object Foo;
follow this method with `_' if you want to treat it as a partially applied function
Foo.timesTwo
^
scala> :type Foo.timesTwo _
Int => Int
So far, so good. We even talked about how functions are actually objects with an apply method and how we can de-syntactic sugarify expressions to show this:
scala> Foo.timesTwo _ apply(4)
res0: Int = 8
scala> addOne.apply(3)
res1: Int = 4
To me, this is quite helpful in learning the language because I can internalize what the syntax is actually implying.
Problematic Example
We did, however, run into a situation that we could not identify. Take, for example, a list of strings. We can map functions over the values demonstrating basic Scala collections and functional programming stuff:
scala> List(1,2,3).map(_*4)
res2: List[Int] = List(4, 8, 12)
Ok, so what is the type of List(1,2,3).map()? I would expect we would do the same :type trick in the repl:
scala> :type List(1,2,3).map _
<console>:8: error: Cannot construct a collection of type Nothing with elements of type Nothing based on a collection of type List[Int].
List(1,2,3).map _
^
From the API definition, I know the signature is:
def map[B](f: (A) ⇒ B): List[B]
But there is also a full signature:
def map[B, That](f: (A) ⇒ B)(implicit bf: CanBuildFrom[List[A], B, That]): That
Question
So there are two things I don't quite understand:
Why doesn't the normal function lift trick work with List.map? Is there a way to de-syntactic sugar the erroneous statement to demonstrate what is going on?
If the reason that the method can't be lifted is due to the full signature "implicit", what exactly is going on there?
Finally, is there a robust way to inspect both types and signatures from the REPL?
The problem you've encountered has to do with the fact that, in Scala, functions are monomorphic, while methods can be polymorphic. As a result, the type parameters B and That must be known in order to create a function value for List.map.
The compiler attempts to infer the parameters but can't come up with anything sensible. If you supply parameters, you'll get a valid function type:
scala> List(1,2,3).map[Char, List[Char]] _
res0: (Int => Char) => List[Char] = <function1>
scala> :type res0
(Int => Char) => List[Char]
Without an actual function argument, the inferred type of the function is Int => Nothing, but the target collection is also Nothing. There is no suitable CanBuildFrom[List[Int], Nothing, Nothing] in scope, which we can see by entering implicitly[CanBuildFrom[List[Int], Nothing, Nothing]] in the REPL (comes up with same error). If you supply the type parameters, then you can get a function:
scala> :type List(1,2,3).map[Int, List[Int]] _
(Int => Int) => List[Int]
I don't think you can inspect method signatures in the REPL. That's what Scaladoc is for.