While trying to play with Options in scala I have come across this peculiar problem.
I started off with creating a List[Option[Int]] as follows:
scala> List(Some(1),Some(2),None,Some(3))
res0: List[Option[Int]] = List(Some(1), Some(2), None, Some(3))
Then I tried to map an addition to 1 over the entries of the list in res0 as follows:
scala> res0 map (_ + 1)
This gave me the error:
<console>:9: error: type mismatch;
found : Int(1)
required: String
res0 map (_ + 1)
^
Then I tried flatMapping an addition over the entries as follows:
scala> res0 flatMap (_ + 1)
This gave me the same error:
<console>:9: error: type mismatch;
found : Int(1)
required: String
res0 flatMap (_ + 1)
^
But something like res0.flatMap(r => r) works just fine with a result of:
res9: List[Int] = List(1, 2, 3)
Can anybody tell me why adding the entry to 1 would fail for both map and flatMap?
The first two things you tried failed because you are trying to add an Option to an Int, and that's not possible.
The weird error message happens because Scala assumes, since Option doesn't have a + method, that you are trying String concatenation, but you'd have to either add an Option to a String, or a String to an Option, and you are doing neither, hence the error message.
In the last case, you are not trying to add anything, you are simply returning Option as is, hence no error message.
To increment all values that are not None, you need to map also each Option element of the list, like so:
scala> res0.map(_.map(_ + 1))
res1: List[Option[Int]] = List(Some(2), Some(3), None, Some(4))
If you want to filter out the Nones, you would indeed use a flatMap:
scala> res0.flatMap(_.map(_ + 1))
res2: List[Int] = List(2, 3, 4)
Both the function given to flatMap as well as the function given to map take a value of the list's element type - in this case Option[Int]. But your function _ + 1 expects an Int, not an Option[Int], so you can't use it as the argument to either map or flatMap in this case. Further the function given to flatMap should return an iterable¹, but your function would return a number.
This will do what you want: res0 flatMap (_ map (_ + 1)). Here the function given to flatMap takes an Option[Int] and returns an Option[Int] by calling map on the option. flatMap then takes the options returned by the function and concatenates them.
¹ Technically a GenTraversableOnce.
You try to invokee.+(1) on every element e in a List[Option[Int]], but + is not a function declared by Option[_]. String concatenation, however, would be possible (I assume there is an implicit from Any to String), but only if the second argument were a string as well (not sure why the implicit whose existence I assumed isn't considered here).
You can overcome this problem by doing working with a default value as suggested by #korefn, or "hide" the differentiation between Some(x) and None in another invocation of map, namely by
map(_.map(_ + 1))
They are failing because the types are wrong and the compiler correctly states it as such.
The map case fails because map expects a function A => B. In your code, A => B is really Int => Int which will not work because calling map on your list means that A is actually Option[Int].
Furthermore, flatMap expects a function of the form A => F[B]. So you will get your answer if you did res0 flatMap { o => o map { a => a + 1 } }. This basically is the expansion of:
for {
element <- res0 // o above
value <- element // a above
} yield value + 1
Try to use get to extract the value from Some[Int] to Int allowing for calculation value + 1 ie:
res0 map{_.getOrElse(0) + 1}
As pointed out by #Sepp2k you could alternatively use a collect to avoid having a default for None
res0 collect {case Some(x) => x + 1 }
Related
First, map treats x => x identical than identity
List(1,2,3).map(x => x) //res0: List[Int] = List(1, 2, 3)
List(1,2,3).map(identity) //res1: List[Int] = List(1, 2, 3)
Now let's transform a List[Option[Int]] into List[Int] discarding all None. We can do that by .flatten. But the point of this question is to understand how flatMap treats identity
val maybeNums = List(Some(1), None, Some(-2), None, None, Some(33))
// Works OK, result = List[Int] = List(1, -2, 33)
maybeNums.flatMap(x => x)
maybeNums.flatMap(x => x.map(identity))
// Not working:
maybeNums.flatMap(identity)
Error:(5, 20) type mismatch;
found : Option[Int] => Option[Int]
required: Option[Int] => scala.collection.GenTraversableOnce[?]
Question: why does maybeNums.flatMap(identity) give a compilation error while maybeNums.flatMap(x => x) works OK?
Funny thing i had a similar problem to yours. This behavior is caused by the fact that Option is not GenTraversableOnce but there exists an implicit conversion to one. Compiler informs you what is wrong, but unfortunately as it is the case frequently in Scala it does not point the real reason of a mistake. If your collection was containing elements of GenTraversableOnce type, flatMap method would work just fine.
At first i thought that implicit conversion would solve this problem, but it turns out that eta expansion needs types to match explicitly. What's more interesting is that the following code compiles:
ys.flatMap(identity(_))
I assume that in this case implicit conversion from Option[Int] => Option[Int] to Option[Int] => GenTraversableOnce[Int] happens.
In the case of x => x right x is converted with the previously mentioned implicit conversion so the code compiles
A bit more detailed explanation of Lampart's answer.
It comes down to how type inference works in Scala and use of expected types.
For maybeNums.flatMap(???) to work, ??? must have type Option[Int] => GenTraversableOnce[?A] (where ?A stands for some unknown type). That's the expected type.
When ??? is a lambda expression like x => x, the argument is typed as Option[Int] and the body is typed with the expected type GenTraversableOnce[?A]. The type of the body without expected type is Option[Int], so the implicit conversion from Option[Int] to GenTraversableOnce[Int] is found and inserted.
When ??? is identity, it's short for identity[?B] for some yet unknown type ?B (in the same sense as ?A above, but they don't have to be the same, of course), and so has type ?B => ?B. So the compiler needs to solve an equation with two unknown types: ?B => ?B == Option[Int] => GenTraversableOnce[?A]. It matches argument types to pick ?B = Option[Int] for ?B, but can't find a suitable ?A. When the error is printed, ?B is substituted and the type which I wrote as ?A above is printed as ? (because it's the only remaining unknown).
In case identity(_), it's expanded to x => identity(x). Again, the type of the argument is inferred to be Option[Int] and the body has calculated type Option[Int] and expected type GenTraversableOnce[?A].
I want to iterate over a list of values using a beautiful one-liner in Scala.
For example, this one works well:
scala> val x = List(1,2,3,4)
x: List[Int] = List(1, 2, 3, 4)
scala> x foreach println
1
2
3
4
But if I use the placeholder _, it gives me an error:
scala> x foreach println(_ + 1)
<console>:6: error: missing parameter type for expanded function ((x$1) =>x$1.$plus(1))
x foreach println(_ + 1)
^
Why is that? Can't compiler infer type here?
This:
x foreach println(_ + 1)
is equivalent to this:
x.foreach(println(x$1 => x$1 + 1))
There's no indication as to what might be the type of x$1, and, to be honest, it doesn't make any sense to print a function.
You obviously (to me) meant to print x$0 + 1, where x$0 would the the parameter passed by foreach, instead. But, let's consider this... foreach takes, as a parameter, a Function1[T, Unit], where T is the type parameter of the list. What you are passing to foreach instead is println(_ + 1), which is an expression that returns Unit.
If you wrote, instead x foreach println, you'd be passing a completely different thing. You'd be passing the function(*) println, which takes Any and returns Unit, fitting, therefore, the requirements of foreach.
This gets slightly confused because of the rules of expansion of _. It expands to the innermost expression delimiter (parenthesis or curly braces), except if they are in place of a parameter, in which case it means a different thing: partial function application.
To explain this better, look at these examples:
def f(a: Int, b: Int, c: Int) = a + b + c
val g: Int => Int = f(_, 2, 3) // Partial function application
g(1)
Here, we applies the second and third arguments to f, and returned a function requiring just the remaining argument. Note that it only worked as is because I indicated the type of g, otherwise I'd have to indicate the type of the argument I was not applying. Let's continue:
val h: Int => Int = _ + 1 // Anonymous function, expands to (x$1: Int => x$1 + 1)
val i: Int => Int = (_ + 1) // Same thing, because the parenthesis are dropped here
val j: Int => Int = 1 + (_ + 1) // doesn't work, because it expands to 1 + (x$1 => x$1 + 1), so it misses the type of `x$1`
val k: Int => Int = 1 + ((_: Int) + 1) // doesn't work, because it expands to 1 + (x$1: Int => x$1 + 1), so you are adding a function to an `Int`, but this operation doesn't exist
Let discuss k in more detail, because this is a very important point. Recall that g is a function Int => Int, right? So, if I were to type 1 + g, would that make any sense? That's what was done in k.
What confuses people is that what they really wanted was:
val j: Int => Int = x$1 => 1 + (x$1 + 1)
In other words, they want the x$1 replacing _ to jump to outside the parenthesis, and to the proper place. The problem here is that, while it may seem obvious to them what the proper place is, it is not obvious to the compiler. Consider this example, for instance:
def findKeywords(keywords: List[String], sentence: List[String]) = sentence.filter(keywords contains _.map(_.toLowerCase))
Now, if we were to expand this to outside the parenthesis, we would get this:
def findKeywords(keywords: List[String], sentence: List[String]) = (x$1, x$2) => sentence.filter(keywords contains x$1.map(x$2.toLowerCase))
Which is definitely not what we want. In fact, if the _ did not get bounded by the innermost expression delimiter, one could never use _ with nested map, flatMap, filter and foreach.
Now, back to the confusion between anonymous function and partial application, look here:
List(1,2,3,4) foreach println(_) // doesn't work
List(1,2,3,4) foreach (println(_)) // works
List(1,2,3,4) foreach (println(_ + 1)) // doesn't work
The first line doesn't work because of how operation notation works. Scala just sees that println returns Unit, which is not what foreachexpects.
The second line works because the parenthesis let Scala evaluate println(_) as a whole. It is a partial function application, so it returns Any => Unit, which is acceptable.
The third line doesn't work because _ + 1 is anonymous function, which you are passing as a parameter to println. You are not making println part of an anonymous function, which is what you wanted.
Finally, what few people expect:
List(1,2,3,4) foreach (Console println _ + 1)
This works. Why it does is left as an exercise to the reader. :-)
(*) Actually, println is a method. When you write x foreach println, you are not passing a method, because methods can't be passed. Instead, Scala creates a closure and passes it. It expands like this:
x.foreach(new Function1[Any,Unit] { def apply(x$1: Any): Unit = Console.println(x$1) })
The underscore is a bit tricky. According to the spec, the phrase:
_ + 1
is equivalent to
x => x + 1
Trying
x foreach println (y => y + 1)
yields:
<console>:6: error: missing parameter type
x foreach println (y => y + 1)
If you add some types in:
x foreach( println((y:Int) => y + 1))
<console>:6: error: type mismatch;
found : Unit
required: (Int) => Unit
x foreach( println((y:Int) => y + 1))
The problem is that you are passing an anonymous function to println and it's not able to deal with it. What you really want to do (if you are trying to print the successor to each item in the list) is:
x map (_+1) foreach println
scala> for(x <- List(1,2,3,4)) println(x + 1)
2
3
4
5
There is a strange limitation in Scala for the nesting depth of expressions with underscore. It's well seen on the following example:
scala> List(1) map(1+_)
res3: List[Int] = List(2)
scala> Some(1) map (1+(1+_))
<console>:5: error: missing parameter type for expanded function ((x$1) => 1.+(x$1))
Some(1) map (1+(1+_))
^
Looks like a bug for me.
Welcome to Scala version 2.8.0.Beta1-prerelease (Java HotSpot(TM) Client VM, Java 1.6.0_17).
Type in expressions to have them evaluated.
Type :help for more information.
scala> val l1 = List(1, 2, 3)
l1: List[Int] = List(1, 2, 3)
scala>
scala> l1.foreach(println(_))
1
2
3
Let's say we have this list of tuples:
val data = List(('a', List(1, 0)), ('b', List(1, 1)), ('c', List(0)))
The list has this signature:
List[(Char, List[Int])]
My task is to get the "List[Int]" element from a tuple inside "data" whose key is, for instance, letter "b". If I implement a method like "findIntList(data, 'b')", then I expect List(1, 1) as a result. I have tried the following approaches:
data.foreach { elem => if (elem._1 == char) return elem._2 }
data.find(x=> x._1 == ch)
for (elem <- data) yield elem match {case (x, y: List[Bit]) => if (x == char) y}
for (x <- data) yield if (x._1 == char) x._2
With all the approaches (except Approach 1, where I employ an explicit "return"), I get either a List[Option] or List[Any] and I don't know how to extract the "List[Int]" out of it.
One of many ways:
data.toMap.get('b').get
toMap converts a list of 2-tuples into a Map from the first element of the tuples to the second. get gives you the value for the given key and returns an Option, thus you need another get to actually get the list.
Or you can use:
data.find(_._1 == 'b').get._2
Note: Only use get on Option when you can guarantee that you'll have a Some and not a None. See http://www.scala-lang.org/api/current/index.html#scala.Option for how to use Option idiomatic.
Update: Explanation of the result types you see with your different approaches
Approach 2: find returns an Option[List[Int]] because it can not guarantee that a matching element gets found.
Approach 3: here you basically do a map, i.e. you apply a function to each element of your collection. For the element you are looking for the function returns your List[Int] for all other elements it contains the value () which is the Unit value, roughly equivalent to void in Java, but an actual type. Since the only common super type of ´List[Int]´ and ´Unit´ is ´Any´ you get a ´List[Any]´ as the result.
Approach 4 is basically the same as #3
Another way is
data.toMap.apply('b')
Or with one intermediate step this is even nicer:
val m = data.toMap
m('b')
where apply is used implicitly, i.e., the last line is equivalent to
m.apply('b')
There are multiple ways of doing it. One more way:
scala> def listInt(ls:List[(Char, List[Int])],ch:Char) = ls filter (a => a._1 == ch) match {
| case Nil => List[Int]()
| case x ::xs => x._2
| }
listInt: (ls: List[(Char, List[Int])], ch: Char)List[Int]
scala> listInt(data, 'b')
res66: List[Int] = List(1, 1)
You can try something like(when you are sure it exists) simply by adding type information.
val char = 'b'
data.collect{case (x,y:List[Int]) if x == char => y}.head
or use headOption if your not sure the character exists
data.collect{case (x,y:List[Int]) if x == char => y}.headOption
You can also solve this using pattern matching. Keep in mind you need to make it recursive though. The solution should look something like this;
def findTupleValue(tupleList: List[(Char, List[Int])], char: Char): List[Int] = tupleList match {
case (k, list) :: _ if char == k => list
case _ :: theRest => findTupleValue(theRest, char)
}
What this will do is walk your tuple list recursively. Check whether the head element matches your condition (the key you are looking for) and then returns it. Or continues with the remainder of the list.
I had a List of Scala tuples like the following:
val l = List((1,2),(2,3),(3,4))
and I wanted to map it in a list of Int where each item is the sum of the Ints in a the corresponding tuple. I also didn't want to use to use the x._1 notation so I solved the problem with a pattern matching like this
def addTuple(t: (Int, Int)) : Int = t match {
case (first, second) => first + second
}
var r = l map addTuple
Doing that I obtained the list r: List[Int] = List(3, 5, 7) as expected. At this point, almost by accident, I discovered that I can achieve the same result with an abbreviated form like the following:
val r = l map {case(first, second) => first + second}
I cannot find any reference to this syntax in the documentation I have. Is that normal? Am I missing something trivial?
See Section 8.5 of the language reference, "Pattern Matching Anonymous Functions".
An anonymous function can be defined by a sequence of cases
{case p1 =>b1 ... case pn => bn }
which appear as an expression without a prior match. The expected type of such an expression must in part be defined. It must be either scala.Functionk[S1, ..., Sk, R] for some k > 0, or scala.PartialFunction[S1, R], where the argument type(s) S1, ..., Sk must be fully determined, but the result type R may be undetermined.
The expected type deternines whether this is translated to a FunctionN or PartialFunction.
scala> {case x => x}
<console>:6: error: missing parameter type for expanded function ((x0$1) => x0$1 match {
case (x # _) => x
})
{case x => x}
^
scala> {case x => x}: (Int => Int)
res1: (Int) => Int = <function1>
scala> {case x => x}: PartialFunction[Int, Int]
res2: PartialFunction[Int,Int] = <function1>
{case(first, second) => first + second} is treated as a PartialFunction literal. See examples in "Partial Functions" section here: http://programming-scala.labs.oreilly.com/ch08.html or section 15.7 of Programming in Scala.
Method map accepts a function. In your first example you create a function, assign it to a variable, and pass it to the map method. In the second example you pass your created function directly, omitting assigning it to a variable. You are doing just the same thing.
I want to iterate over a list of values using a beautiful one-liner in Scala.
For example, this one works well:
scala> val x = List(1,2,3,4)
x: List[Int] = List(1, 2, 3, 4)
scala> x foreach println
1
2
3
4
But if I use the placeholder _, it gives me an error:
scala> x foreach println(_ + 1)
<console>:6: error: missing parameter type for expanded function ((x$1) =>x$1.$plus(1))
x foreach println(_ + 1)
^
Why is that? Can't compiler infer type here?
This:
x foreach println(_ + 1)
is equivalent to this:
x.foreach(println(x$1 => x$1 + 1))
There's no indication as to what might be the type of x$1, and, to be honest, it doesn't make any sense to print a function.
You obviously (to me) meant to print x$0 + 1, where x$0 would the the parameter passed by foreach, instead. But, let's consider this... foreach takes, as a parameter, a Function1[T, Unit], where T is the type parameter of the list. What you are passing to foreach instead is println(_ + 1), which is an expression that returns Unit.
If you wrote, instead x foreach println, you'd be passing a completely different thing. You'd be passing the function(*) println, which takes Any and returns Unit, fitting, therefore, the requirements of foreach.
This gets slightly confused because of the rules of expansion of _. It expands to the innermost expression delimiter (parenthesis or curly braces), except if they are in place of a parameter, in which case it means a different thing: partial function application.
To explain this better, look at these examples:
def f(a: Int, b: Int, c: Int) = a + b + c
val g: Int => Int = f(_, 2, 3) // Partial function application
g(1)
Here, we applies the second and third arguments to f, and returned a function requiring just the remaining argument. Note that it only worked as is because I indicated the type of g, otherwise I'd have to indicate the type of the argument I was not applying. Let's continue:
val h: Int => Int = _ + 1 // Anonymous function, expands to (x$1: Int => x$1 + 1)
val i: Int => Int = (_ + 1) // Same thing, because the parenthesis are dropped here
val j: Int => Int = 1 + (_ + 1) // doesn't work, because it expands to 1 + (x$1 => x$1 + 1), so it misses the type of `x$1`
val k: Int => Int = 1 + ((_: Int) + 1) // doesn't work, because it expands to 1 + (x$1: Int => x$1 + 1), so you are adding a function to an `Int`, but this operation doesn't exist
Let discuss k in more detail, because this is a very important point. Recall that g is a function Int => Int, right? So, if I were to type 1 + g, would that make any sense? That's what was done in k.
What confuses people is that what they really wanted was:
val j: Int => Int = x$1 => 1 + (x$1 + 1)
In other words, they want the x$1 replacing _ to jump to outside the parenthesis, and to the proper place. The problem here is that, while it may seem obvious to them what the proper place is, it is not obvious to the compiler. Consider this example, for instance:
def findKeywords(keywords: List[String], sentence: List[String]) = sentence.filter(keywords contains _.map(_.toLowerCase))
Now, if we were to expand this to outside the parenthesis, we would get this:
def findKeywords(keywords: List[String], sentence: List[String]) = (x$1, x$2) => sentence.filter(keywords contains x$1.map(x$2.toLowerCase))
Which is definitely not what we want. In fact, if the _ did not get bounded by the innermost expression delimiter, one could never use _ with nested map, flatMap, filter and foreach.
Now, back to the confusion between anonymous function and partial application, look here:
List(1,2,3,4) foreach println(_) // doesn't work
List(1,2,3,4) foreach (println(_)) // works
List(1,2,3,4) foreach (println(_ + 1)) // doesn't work
The first line doesn't work because of how operation notation works. Scala just sees that println returns Unit, which is not what foreachexpects.
The second line works because the parenthesis let Scala evaluate println(_) as a whole. It is a partial function application, so it returns Any => Unit, which is acceptable.
The third line doesn't work because _ + 1 is anonymous function, which you are passing as a parameter to println. You are not making println part of an anonymous function, which is what you wanted.
Finally, what few people expect:
List(1,2,3,4) foreach (Console println _ + 1)
This works. Why it does is left as an exercise to the reader. :-)
(*) Actually, println is a method. When you write x foreach println, you are not passing a method, because methods can't be passed. Instead, Scala creates a closure and passes it. It expands like this:
x.foreach(new Function1[Any,Unit] { def apply(x$1: Any): Unit = Console.println(x$1) })
The underscore is a bit tricky. According to the spec, the phrase:
_ + 1
is equivalent to
x => x + 1
Trying
x foreach println (y => y + 1)
yields:
<console>:6: error: missing parameter type
x foreach println (y => y + 1)
If you add some types in:
x foreach( println((y:Int) => y + 1))
<console>:6: error: type mismatch;
found : Unit
required: (Int) => Unit
x foreach( println((y:Int) => y + 1))
The problem is that you are passing an anonymous function to println and it's not able to deal with it. What you really want to do (if you are trying to print the successor to each item in the list) is:
x map (_+1) foreach println
scala> for(x <- List(1,2,3,4)) println(x + 1)
2
3
4
5
There is a strange limitation in Scala for the nesting depth of expressions with underscore. It's well seen on the following example:
scala> List(1) map(1+_)
res3: List[Int] = List(2)
scala> Some(1) map (1+(1+_))
<console>:5: error: missing parameter type for expanded function ((x$1) => 1.+(x$1))
Some(1) map (1+(1+_))
^
Looks like a bug for me.
Welcome to Scala version 2.8.0.Beta1-prerelease (Java HotSpot(TM) Client VM, Java 1.6.0_17).
Type in expressions to have them evaluated.
Type :help for more information.
scala> val l1 = List(1, 2, 3)
l1: List[Int] = List(1, 2, 3)
scala>
scala> l1.foreach(println(_))
1
2
3