Why can't I flatMap a Try? - scala

Given
val strings = Set("Hi", "there", "friend")
def numberOfCharsDiv2(s: String) = scala.util.Try {
if (s.length % 2 == 0) s.length / 2 else throw new RuntimeException("grr")
}
Why can't I flatMap away the Try resulting from the method call? i.e.
strings.flatMap(numberOfCharsDiv2)
<console>:10: error: type mismatch;
found : scala.util.Try[Int]
required: scala.collection.GenTraversableOnce[?]
strings.flatMap(numberOfCharsDiv2)
or
for {
s <- strings
n <- numberOfCharsDiv2(s)
} yield n
<console>:12: error: type mismatch;
found : scala.util.Try[Int]
required: scala.collection.GenTraversableOnce[?]
n <- numberOfCharsDiv2(s)
However if I use Option instead of Try there's no problem.
def numberOfCharsDiv2(s: String) = if (s.length % 2 == 0)
Some(s.length / 2) else None
strings.flatMap(numberOfCharsDiv2) # => Set(1, 3)
What's the rationale behind not allowing flatMap on Try?

Let's look at the signature of flatMap.
def flatMap[B](f: (A) => GenTraversableOnce[B]): Set[B]
Your numberOfCharsDiv2 is seen as String => Try[Int]. Try is not a subclass of GenTraversableOnce and that is why you get the error. You don't strictly need a function that gives a Set only because you use flatMap on a Set. The function basically has to return any kind of collection.
So why does it work with Option? Option is also not a subclass of GenTraversableOnce, but there exists an implicit conversion inside the Option companion object, that transforms it into a List.
implicit def option2Iterable[A](xo: Option[A]): Iterable[A] = xo.toList
Then one question remains. Why not have an implicit conversion for Try as well? Because you will probably not get what you want.
flatMap can be seen as a map followed by a flatten.
Imagine you have a List[Option[Int]] like List(Some(1), None, Some(2)). Then flatten will give you List(1,2) of type List[Int].
Now look at an example with Try. List(Success(1), Failure(exception), Success(2)) of type List[Try[Int]].
How will flatten work with the failure now?
Should it disappear like None? Then why not work directly with Option?
Should it be included in the result? Then it would be List(1, exception, 2). The problem here is that the type is List[Any], because you have to find a common super class for Int and Throwable. You lose the type.
These should be reasons why there isn't an implicit conversion. Of course you are free to define one yourself, if you accept the above consequences.

The problem is that in your example, you're not flatmapping over Try. The flatmap you are doing is over Set.
Flatmap over Set takes a Set[A], and a function from A to Set[B]. As Kigyo points out in his comment below this isn't the actual type signature of flatmap on Set in Scala, but the general form of flat map is:
M[A] => (A => M[B]) => M[B]
That is, it takes some higher-kinded type, along with a function that operates on elements of the type in that higher-kinded type, and it gives you back the same higher-kinded type with the mapped elements.
In your case, this means that for each element of your Set, flatmap expects a call to a function that takes a String, and returns a Set of some type B which could be String (or could be anything else).
Your function
numberOfCharsDiv2(s: String)
correctly takes a String, but incorrectly returns a Try, rather then another Set as flatmap requires.
Your code would work if you used 'map', as that allows you to take some structure - in this case Set and run a function over each element transforming it from an A to a B without the function's return type conforming to the enclosing structure i.e. returning a Set
strings.map(numberOfCharsDiv2)
res2: scala.collection.immutable.Set[scala.util.Try[Int]] = Set(Success(1), Failure(java.lang.RuntimeException: grr), Success(3))

It is a Monad in Scala 2.11:
scala> import scala.util._
import scala.util._
scala> val x: Try[String] = Success[String]("abc")
x: scala.util.Try[String] = Success(abc)
scala> val y: Try[String] = Failure[String](new Exception("oops"))
y: scala.util.Try[String] = Failure(java.lang.Exception: oops)
scala> val z = Try(x)
z: scala.util.Try[scala.util.Try[String]] = Success(Success(abc))
scala> val t = Try(y)
t: scala.util.Try[scala.util.Try[String]] = Success(Failure(java.lang.Exception: oops))
scala> z.flatten
res2: scala.util.Try[String] = Success(abc)
scala> t.flatten
res3: scala.util.Try[String] =
Failure(java.lang.UnsupportedOperationException: oops)

Kigyo explains well why Scala does not do this implicitly. To put it simply, Scala does not want to automatically throw away the exception that is preserved in a Try.
Scala does provide a simple way to explicitly translate a Try into an Option though. This is how you can use a Try in a flatmap:
strings.flatMap(numberOfCharsDiv2(_).toOption)

Related

Why does Scala Option.tapEach return Iterable, not Option?

The scaladoc of Option.tapEach states "returns: The same logical collection as this" just as expected for an operation named after tap & foreach. However, it does not return an Option but an Iterable backed by a List:
scala> import scala.util.chaining._
scala> Option(5).tap(_.foreach(_ => ()))
val res0: Option[Int] = Some(5)
scala> Option(5).tapEach(_ => ())
val res1: Iterable[Int] = List(5)
(Verified for Scala 2.13.5 and 3.0.0-RC1)
Is there a good reason to return Iterable instead of Option, or has this just been overlooked (and might be fixed eventually)?
It seems whether Option is considered a full collection is a bit of a can of worms as indicated by the discussion at Make Option extend IterableOnce #8038. I think the relevant comment is
So it can definitely be a IterableOnce because you can get an iterator
of zero to one elements. But it can't be a Iterable because you you
can't implement fromSpecific(c: IterableOnce[A]): K without throwing
away data.
however tapEach uses fromSpecific in its definition
override def tapEach[U](f: A => U): C = fromSpecific(new View.Map(this, { (a: A) => f(a); a })
So key to remember is Option since Scala 2.13 is an IterableOnce but not a full Iterable. IterableOnce is smaller compared to Iterable, so if a capability is needed from Iterable it is provided via implicit conversion as per docs
This member is added by an implicit conversion from Option[A]
to Iterable[A] performed by method option2Iterable in scala.Option.
that is
option2iterable(Option(5)).tapEach(_ => ())
hence the Iterable[Int] return type.
Also consider the following note
Many of the methods in here are duplicative with those in the
Traversable hierarchy, but they are duplicated for a reason: the
implicit conversion tends to leave one with an Iterable in situations
where one could have retained an Option.
so contributors would have to bake a specialised version in Option to preserve the type, or perhaps we could provide our own specialised extension implementation, something like
scala> implicit class OptionTapOps[A](v: Option[A]) {
| def tapEach[B](f: A => B): Option[A] = { v.foreach(f); v }
| }
class OptionTapOps
scala> Option(5).tapEach(_ => ())
val res11: Option[Int] = Some(5)

Why does not sequence work with List of Validations

I think I understand what sequence is. I am wondering why it does not work with List[ValidationNel]. For instance:
The sequence works fine with List[Option]]
scala> val os = List(1.some, 2.some)
os: List[Option[Int]] = List(Some(1), Some(2))
scala> os.sequence
res10: Option[List[Int]] = Some(List(1, 2))
... but does not work with List[ValidationNel]
scala> val vs: List[ValidationNel[String, Int]] = List(Success(1), Success(2))
vs: List[scalaz.ValidationNel[String,Int]] = List(Success(1), Success(2))
scala> vs.sequence
<console>:15: error: could not find implicit value for parameter ev:scalaz.Leibniz.===[scalaz.ValidationNel[String,Int],G[B]]
... however sequenceU does work with List[ValidationNel]
scala> vs.sequenceU
res14: scalaz.Validation[scalaz.NonEmptyList[String],List[Int]] = Success(List(1, 2))
My questions are: Why does not sequence work with List[ValidationNel] ? Why does sequenceU work with it ?
.sequenceU uses the Unapply technique to derive the correct types, where as for .sequence you need to manually provide the types for it.
To make things a bit more annoying, the first type argument of sequence needs a type parameter that takes one type parameter not two like ValidationNel. So you either type lambda it, or do a local type definition.
Try
type X = ValidationNel[String,X]
vs.sequence[X, Int]
or
vs.sequence[({type l[A]=ValidationNel[String,A]})#l,Int]
Not an expert on Scalaz.
To be short, because ValidationNel is not monad, so this conversion is not valid:
List[ValidationNel[SomeType]] => ValidationNel[List[SomeType]]
as the error message shows: implicit value not found, which indicates no such conversion.

Comparing reflected collection types in Scala

I have a reflected collection type I need to compare to see if it's a certain kind of collection. How can I do that?
val a = List(1,2,3)
val b = currentMirror.classSymbol(a.getClass).typeSignature
println("Q? "+ (b =:= typeOf[List[_]]))
This is always false, of course. In practice I have a list of collections, generalized with _ . I need to know if a given Type (typically specific in its parameters) is one of these collections. For example I need to know if List[Int] is a List[_].
The actual case is this: Map of generic type -> function for something I need:
val collectionHandlers = Map(
typeOf[scala.collection.immutable.List[_]] -> fnList,
typeOf[scala.collection.immutable.Map[_,_]] -> fnMap,
//...
)
val aListType = // something here that is a TypeSignature of List[Int] as above
collectionHandlers( _magicClean(aListType) )()
Where _magicClean is what I need help with. How to I "generalize" the specifically-typed collection so that map lookup will work?
The problem is that a's classSymbol is ::. It also doesn't contain the type information for the contained type Int (the toString only shows generics).
Say, if we have a method like this:
def typeTagOf[A](a: A)(implicit tt: TypeTag[A]) = tt
We can use it to get the exact Type of a, and compare it with typeOf[List[_]].
scala> typeTagOf(a).tpe <:< typeOf[List[_]]
res165: Boolean = true
scala> typeTagOf(1).tpe <:< typeOf[List[_]]
res166: Boolean = false
But it's not true that List[Int] is List[_], which is why I used <:< instead of =:=. That is, a List[Int] is sub-type of a List[_], but they are not exactly the same type. It's not really clear what you need this for. If you're only matching collection types (without the inner type), you can probably get away with using List[_], etc. But there are caveats. For example, if you have this:
val list: List[_] = List(1, 2, 3)
You cannot recover the type information and prove it is a list of only Ints, because _ is an existential type.
scala> typeTagOf(list)
res170: reflect.runtime.universe.TypeTag[List[_$1]] forSome { type _$1 } = TypeTag[scala.List[_$1]]

Scala parenthesis semantics

When appending values to a MAP, why does Scala require the additional parenthesis-block to make this statement work?
Does NOT compile:
vmap += (item.getName(), item.getString()) // compiler output: "found: String"
However, this Does compile:
vmap += ((item.getName(), item.getString())) // note the second set of enclosures
TIA
EDIT: vmap is defined as
val vmap = new mutable.HashMap[String, String]()
Epilogue:
At the time of this edit there are posts detailing two possible explanations, both of which appear to contain elements of truth to them. Which one is actually Correct? I couldn't say with any degree of certainty...I'm just a guy who's still learning the language. That being said, I have changed the answer selection based upon the feeling that the one answer is (at least to some extent) encompassed within the other - so I've opted for the larger picture, as I think it will provide a broader meaning for someone else in search of an answer. Ironically, I was trying to get a better understanding of how to flatten out some of the little nuances of the language, and what I've come to realize is there are more of those than I had suspected. I'm not saying that's a bad thing - in fact (IMO) it's to be expected from any language that is as flexible and complex - but it sure does make a guy miss the black/white world of Assembly from time-to-time...
To draw this to an end, a couple observations:
1) the selected answer contains a link to a website full of Scala brain-benders (Which I found extremely helpful in trying to understand some of the aforementioned quarks in the language.) Highly recommended.
2) I did come across another interesting twist - whereas the single-parenthesis (example above) does not work, change it the following and it works just fine...
vmap += ("foo" -> "bar")
Which probably has something to do with matching method/function signatures, but that is just a guess on my part.
The accepted answer is actually wrong.
The reason you don't get tupling for Map.+= is that the method is overloaded with a second method that takes two or more args.
The compiler will only try tupling if the number of args is wrong. But if you give it two args, and there's a method that takes two, that's what it chooses, even if it fails to type check.
It doesn't start trying all possible combinations until something works because that would be fraught with obscurity. (Cf implicits.)
scala> def f(p: Pair[String, Int]) = { val (s,i) = p; s.toInt + i }
f: (p: Pair[String,Int])Int
scala> f("2",3) // ok to tuple
res0: Int = 5
scala> trait F { def +=(p: Pair[String, Int]) = f(p) }
defined trait F
scala> val foo = new F {}
foo: F = $anon$1#6bc77f62
scala> foo += ("2",3) // ok to tuple
res1: Int = 5
scala> trait G { def +=(p: Pair[String, Int]) = f(p); def +=(p:(String,Int),q:(String,Int),r:(String,Int)*) = f(p)+f(q)+(r map f).sum }
defined trait G
scala> val goo = new G {}
goo: G = $anon$1#183aeac3
scala> goo += ("2",3) // sorry
<console>:12: error: type mismatch;
found : String("2")
required: (String, Int)
goo += ("2",3)
^
scala> goo += (("2",3),("4",5),("6",7))
res3: Int = 27
I'd be remiss not to mention your friend and mine, -Xlint, which will warn about untoward arg adaptations:
apm#mara:~/tmp$ skala -Xlint
Welcome to Scala version 2.11.0-20130811-132927-95a4d6e987 (OpenJDK 64-Bit Server VM, Java 1.7.0_25).
Type in expressions to have them evaluated.
Type :help for more information.
scala> def f(p: Pair[String, Int]) = { val (s,i) = p; s.toInt + i }
f: (p: Pair[String,Int])Int
scala> f("2",3)
<console>:9: warning: Adapting argument list by creating a 2-tuple: this may not be what you want.
signature: f(p: Pair[String,Int]): Int
given arguments: "2", 3
after adaptation: f(("2", 3): (String, Int))
f("2",3)
^
res0: Int = 5
scala> List(1).toSet()
<console>:8: warning: Adapting argument list by inserting (): this is unlikely to be what you want.
signature: GenSetLike.apply(elem: A): Boolean
given arguments: <none>
after adaptation: GenSetLike((): Unit)
List(1).toSet()
^
res3: Boolean = false
On the perils of adaptation, see the Adaptive Reasoning puzzler and this new one that is pretty common because we learn that the presence or absence of parens is largely a matter of style, and when parens really matter, using them wrong results in a type error.
Adapting a tuple in the presence of overloading:
scala> class Foo {
| def f[A](a: A) = 1 // A can be (Int,Int,Int)
| def f[A](a: A, a2: A) = 2
| }
defined class Foo
scala> val foo = new Foo
foo: Foo = Foo#2645d22d
scala> foo.f(0,0,0)
<console>:10: warning: Adapting argument list by creating a 3-tuple: this may not be what you want.
signature: Foo.f[A](a: A): Int
given arguments: 0, 0, 0
after adaptation: Foo.f((0, 0, 0): (Int, Int, Int))
foo.f(0,0,0)
^
res9: Int = 1
Because the Map += method takes a Tuple2[A, B] as parameter.
You have to mark the Tuple2[A, B] with surrounding parentheses, otherwise the compiler won't infer the type to Tuple2.
A Tuple2 is a simple pair A -> B.
val x = (5, 7); // the type is inferred to Tuple2[Int, Int];
A Map iteration makes this even more obvious:
map.foreach { case (key, value) => ...};
(key, value)// is a Tuple2
The first set of enclosing parentheses are treated as fruitless/skip-able by the compiler.
The second set of enclosing parentheses creates the needed Tuple2.
val x = (item.getName(), item.getString());//Tuple2[String, String]
vmap += x; // THIS COMPILES
vmap += (item.getName(), item.getString())// is identical to
vmap += item.getName(), item.getString() // now it thinks you are adding a String, the result of item.getName()
vmap += ( (item.getName(), item.getString()) )// skips the first set, sees the Tuple, compiles.
From the SLS:
Postfix operators have lower precedence than infix operators,
so foo bar baz = foo.bar(baz)
In this case: vmap += (item.getName(), item.getString()); = vmap.+=(item.getName());

How do I convert Array[Node] to NodeSeq?

I'm trying to integrate a Lift application into some existing Java code. In one of my snippets, I have an Array of Java objects that I need to map that into a NodeSeq. I can get an Array of Node's, but not a NodeSeq. (At least, not in very functional-looking way).
import scala.xml.NodeSeq
// pretend this is code I can't do anything about
val data = Array("one", "two", "three")
// this is the function I need to write
def foo: NodeSeq = data.map { s => <x>{s}</x> }
// ^
// error: type mismatch;
// found : Array[scala.xml.Elem]
// required: scala.xml.NodeSeq
What's the cleanest way to do this?
scala> import collection.breakOut
import collection.breakOut
scala> def foo: NodeSeq = data.map { s => <x>{s}</x> }(breakOut)
foo: scala.xml.NodeSeq
The method map actually has two argument lists. The first accepts a function, which you passed. The second accepts a CanBuildFrom object which is used to create a builder that then builds the returning sequence. This argument is implicit, so usually the compiler fills it for you. It accepts 3 type parameters: From, T, To. There are several predef implicits (including in object NodeSeq), but none of them matches From=Array, T=Node, To=NodeSeq.
breakOut solves this: it is a generic method that returns a CanBuildFrom instance by searching for an implicit CanBuildFrom[Nothing, T, To]. According to the implicit search rules, any CanBuildFrom that matches T, To and has From > Nothing is acceptable. In this case: canBuildFrom in object Array
I would simply convert map output to sequence (given that Seq[Node] is a super-class of NodeSeq)
scala> def foo: NodeSeq = data.map { s => <x>{s}</x> } toSeq
foo: scala.xml.NodeSeq
or use foldLeft instead of map
scala> def foo: NodeSeq = (Seq[Node]() /: data) {(seq, node)=> seq ++ <x>{node}</x>}
foo: scala.xml.NodeSeq
You are looking for this method on the NodeSeq companion object.
NodeSeq.fromSeq(s: Seq[Node])