Comparing reflected collection types in Scala - scala

I have a reflected collection type I need to compare to see if it's a certain kind of collection. How can I do that?
val a = List(1,2,3)
val b = currentMirror.classSymbol(a.getClass).typeSignature
println("Q? "+ (b =:= typeOf[List[_]]))
This is always false, of course. In practice I have a list of collections, generalized with _ . I need to know if a given Type (typically specific in its parameters) is one of these collections. For example I need to know if List[Int] is a List[_].
The actual case is this: Map of generic type -> function for something I need:
val collectionHandlers = Map(
typeOf[scala.collection.immutable.List[_]] -> fnList,
typeOf[scala.collection.immutable.Map[_,_]] -> fnMap,
//...
)
val aListType = // something here that is a TypeSignature of List[Int] as above
collectionHandlers( _magicClean(aListType) )()
Where _magicClean is what I need help with. How to I "generalize" the specifically-typed collection so that map lookup will work?

The problem is that a's classSymbol is ::. It also doesn't contain the type information for the contained type Int (the toString only shows generics).
Say, if we have a method like this:
def typeTagOf[A](a: A)(implicit tt: TypeTag[A]) = tt
We can use it to get the exact Type of a, and compare it with typeOf[List[_]].
scala> typeTagOf(a).tpe <:< typeOf[List[_]]
res165: Boolean = true
scala> typeTagOf(1).tpe <:< typeOf[List[_]]
res166: Boolean = false
But it's not true that List[Int] is List[_], which is why I used <:< instead of =:=. That is, a List[Int] is sub-type of a List[_], but they are not exactly the same type. It's not really clear what you need this for. If you're only matching collection types (without the inner type), you can probably get away with using List[_], etc. But there are caveats. For example, if you have this:
val list: List[_] = List(1, 2, 3)
You cannot recover the type information and prove it is a list of only Ints, because _ is an existential type.
scala> typeTagOf(list)
res170: reflect.runtime.universe.TypeTag[List[_$1]] forSome { type _$1 } = TypeTag[scala.List[_$1]]

Related

How to turn an HList of Const functors into a Seq?

I have a higher-kinded data type like this
case class Foo[F[_] : Functor](a: F[Int], b: F[Double])
and a value that labels every member with some kind of string (the name of a CSV column for example):
val colNames = new Foo[Const[String, *]](a = Const("AAA"), b = Const("BBB"))
Now I need to somehow convert colNames into a Seq[String]. There is the standard scala method productIterator of course, but it only returns Iterator[Any].
Using shapeless I can get an HList where every type in the list is Const[String, _]:
val colNamesHlist: Const[String, _] :: Const[String, _] :: HNil = Generic[Foo[Const[String, *]]].to(colNames)
But how can I turn this into a regular list?
I am also open to any ideas not involving shapeless, though I don't think it can be avoided.
You may do this:
val colNameList: List[String] =
Generic[Foo[Const[String, *]]]
.to(colNames)
.toList[Const[String, A] forSome { type A }]
.map(_.getConst)
You can see the code running here.

Restrict Generic function fun[T] with multiple parameters on higher types on T to be exactly T

I have the following function
def f[T](l:List[T], v:T)
this works without errors for
f(List(1,2,3), "this is a string")
even when T is not same in both parameters. Is it possible to rewrite the function so that the compiler gives me an error when T is not same in both parameters? (I cant rely on the user to explicitly specify the type in function f[Int])
[Apologies for the horrible title]
Type unification happens only within a parameter list. You can use separate parameter lists, although this only ensures that v.type <:< T, not that v.type =:= T.
def f[T](l:List[T])(v:T): List[T] = v :: l
f(List(1,2,3))("this is a string")
Error:(81, 20) type mismatch;
found : String("this is a string")
required: Int
f(List(1,2,3))("this is a string")
Another option is using an implicit, although it requires a type cast:
def f[T, U](l: List[T], v: U)(implicit ev: T =:= U): List[T] =
v.asInstanceOf[T] :: l
val l1 = List("a", "b", "c")
val l2: List[Any] = List("a", "b", "c")
val s1 = "d"
val s2: Any = "d"
f(l1, s1) // compiles
f(l2, s2) // compiles
f(l1, s2) // Cannot prove that String =:= Any.
f(l2, s1) // Cannot prove that Any =:= String.
First things first, Scala handles type classes in such situations by finding the common supertype that could be generalized as T. In your example, you have Int and String whom common super is Any. So def f[T](l:List[T], v:T): T = v invoked with f(List(1, 2, 3), "scala") yields Any = "scala".
Now, there are several ways you could handle that depending on the desired behaviour. This question contains neat answers for the opposite question (ensuring type difference) Enforce type difference . On the other hand, you could resolve to using ClassTag or introducing some implicits that would allow you to distinguish types. Nevertheless, regardless of the approach your method most likely will require following definition def f[U, T](l: List[U], v: T) and implement one of the abovementioned type guards.

Why can't I flatMap a Try?

Given
val strings = Set("Hi", "there", "friend")
def numberOfCharsDiv2(s: String) = scala.util.Try {
if (s.length % 2 == 0) s.length / 2 else throw new RuntimeException("grr")
}
Why can't I flatMap away the Try resulting from the method call? i.e.
strings.flatMap(numberOfCharsDiv2)
<console>:10: error: type mismatch;
found : scala.util.Try[Int]
required: scala.collection.GenTraversableOnce[?]
strings.flatMap(numberOfCharsDiv2)
or
for {
s <- strings
n <- numberOfCharsDiv2(s)
} yield n
<console>:12: error: type mismatch;
found : scala.util.Try[Int]
required: scala.collection.GenTraversableOnce[?]
n <- numberOfCharsDiv2(s)
However if I use Option instead of Try there's no problem.
def numberOfCharsDiv2(s: String) = if (s.length % 2 == 0)
Some(s.length / 2) else None
strings.flatMap(numberOfCharsDiv2) # => Set(1, 3)
What's the rationale behind not allowing flatMap on Try?
Let's look at the signature of flatMap.
def flatMap[B](f: (A) => GenTraversableOnce[B]): Set[B]
Your numberOfCharsDiv2 is seen as String => Try[Int]. Try is not a subclass of GenTraversableOnce and that is why you get the error. You don't strictly need a function that gives a Set only because you use flatMap on a Set. The function basically has to return any kind of collection.
So why does it work with Option? Option is also not a subclass of GenTraversableOnce, but there exists an implicit conversion inside the Option companion object, that transforms it into a List.
implicit def option2Iterable[A](xo: Option[A]): Iterable[A] = xo.toList
Then one question remains. Why not have an implicit conversion for Try as well? Because you will probably not get what you want.
flatMap can be seen as a map followed by a flatten.
Imagine you have a List[Option[Int]] like List(Some(1), None, Some(2)). Then flatten will give you List(1,2) of type List[Int].
Now look at an example with Try. List(Success(1), Failure(exception), Success(2)) of type List[Try[Int]].
How will flatten work with the failure now?
Should it disappear like None? Then why not work directly with Option?
Should it be included in the result? Then it would be List(1, exception, 2). The problem here is that the type is List[Any], because you have to find a common super class for Int and Throwable. You lose the type.
These should be reasons why there isn't an implicit conversion. Of course you are free to define one yourself, if you accept the above consequences.
The problem is that in your example, you're not flatmapping over Try. The flatmap you are doing is over Set.
Flatmap over Set takes a Set[A], and a function from A to Set[B]. As Kigyo points out in his comment below this isn't the actual type signature of flatmap on Set in Scala, but the general form of flat map is:
M[A] => (A => M[B]) => M[B]
That is, it takes some higher-kinded type, along with a function that operates on elements of the type in that higher-kinded type, and it gives you back the same higher-kinded type with the mapped elements.
In your case, this means that for each element of your Set, flatmap expects a call to a function that takes a String, and returns a Set of some type B which could be String (or could be anything else).
Your function
numberOfCharsDiv2(s: String)
correctly takes a String, but incorrectly returns a Try, rather then another Set as flatmap requires.
Your code would work if you used 'map', as that allows you to take some structure - in this case Set and run a function over each element transforming it from an A to a B without the function's return type conforming to the enclosing structure i.e. returning a Set
strings.map(numberOfCharsDiv2)
res2: scala.collection.immutable.Set[scala.util.Try[Int]] = Set(Success(1), Failure(java.lang.RuntimeException: grr), Success(3))
It is a Monad in Scala 2.11:
scala> import scala.util._
import scala.util._
scala> val x: Try[String] = Success[String]("abc")
x: scala.util.Try[String] = Success(abc)
scala> val y: Try[String] = Failure[String](new Exception("oops"))
y: scala.util.Try[String] = Failure(java.lang.Exception: oops)
scala> val z = Try(x)
z: scala.util.Try[scala.util.Try[String]] = Success(Success(abc))
scala> val t = Try(y)
t: scala.util.Try[scala.util.Try[String]] = Success(Failure(java.lang.Exception: oops))
scala> z.flatten
res2: scala.util.Try[String] = Success(abc)
scala> t.flatten
res3: scala.util.Try[String] =
Failure(java.lang.UnsupportedOperationException: oops)
Kigyo explains well why Scala does not do this implicitly. To put it simply, Scala does not want to automatically throw away the exception that is preserved in a Try.
Scala does provide a simple way to explicitly translate a Try into an Option though. This is how you can use a Try in a flatmap:
strings.flatMap(numberOfCharsDiv2(_).toOption)

Scala: existential types for a Map

I want to use a map of varying types on an unknown A:
val map: Map[Foo[A], Bar[A]] = ...
...
val foo = new Foo[Qux]
val bar: Bar[Qux] = map(foo)
This doesn't work, because A is an unknown. I have to define it instead as:
val map: Map[Foo[_], Bar[_]] = ...
...
val foo = new Foo[Qux]
val bar: Bar[Qux] = map(foo).asInstanceOf[Bar[Qux]]
This works, but the cast is ugly. I'd rather find a better way. I gather the answer is to use existential types with the forSome keyword, but I'm confused as to how that works. Should it be:
Map[Foo[A], Bar[A]] forSome { type A }
or:
Map[Foo[A] forSome { type A }, Bar[A]]
or:
Map[Foo[A forSome { type A }], Bar[A]]
Actually, none of these work.
Map[Foo[A], Bar[A]] forSome { type A }
is a Map where all keys are of the same type Foo[A] and values of type Bar[A] (but the type A may be different for different maps of this type); in second and third examples, A in Bar[A] is completely different from A under forSome.
This ugly workaround should work:
// need type members, so can't use tuples
case class Pair[A, B](a: A, b: B) {
type T1 = A
type T2 = B
}
type PairedMap[P <: Pair[_, _]] = Map[P#T1, P#T2]
type FooBarPair[A] = Pair[Foo[A], Bar[A]]
val map: PairedMap[FooBarPair[_]] = ...
I want to use a map of varying types on an unknown A
So, you want a variant of Map[K,V] with following interface, correct?
trait DependentMap[K[_],V[_]] {
def add[A](key: K[A], value: V[A]): DependentMap[K,V]
def get[A](key: K[A]): Option[V[A]]
}
Whether it is or not might be a bit hard to tell from the type signature, so let's create a few dummy values and see if the type checker accepts what we want it to accept and rejects what we want it to reject.
// dummy definitions just to check that the types are correct
case class Foo[+A](a: A)
case class Bar[+A](a: A)
val myMap: DependentMap[Foo,Bar] = null
myMap.add(Foo( 42), Bar( 43)) // typechecks
myMap.add(Foo("foo"), Bar("bar")) // typechecks
myMap.add(Foo( 42), Bar("bar")) // type mismatch
val r1: Option[Bar[ Int]] = myMap.get(Foo( 42)) // typechecks
val r2: Option[Bar[String]] = myMap.get(Foo("foo")) // typechecks
val r3: Option[Bar[String]] = myMap.get(Foo( 42)) // type mismatch
So far so good. But look what happens once we start to play with inheritance:
val fooInt: Foo[Int] = Foo(42)
val fooAny: Foo[Any] = fooInt
val barStr: Bar[String] = Bar("bar")
val barAny: Bar[Any] = barStr
println(fooInt == fooAny) // true
myMap.add(fooAny, barAny).get(fooInt) // Bar("bar")?
Since fooInt and fooAny are the same value, we'd naïvely expect this get to succeed. According to the type signature of get, it would have to succeed with a value of type Bar[Int], but where would this value come from? The value we put in has the type Bar[Any] and also the type Bar[String], but certainly not the type Bar[Int]!
Here's another surprising case.
val fooListInt: Foo[List[Int]] = Foo(List[Int]())
val fooListStr: Foo[List[String]] = Foo(List[String]())
println(fooListInt == fooListStr) // true!
myMap.add(fooListInt, Bar(List(42))).get(fooListStr) // Bar(List(42))?
This time fooListInt and fooListStr look like they should be distinct, but in fact they both have the value Nil, of type List[Nothing], which is a subtype of both List[Int] and List[String]. So if we want to mimic the behaviour of Map[K,V] on such a key, get would again have to succeed, but it can't since we gave it a Bar[List[Int]] and it needs to produce a Bar[List[String]].
All this to say, our DependentMap should not consider keys to be equal unless the type A given to add is also equal to the type A given to get. Here is an implementation which uses type tags to ensure that this is the case.
import scala.reflect.runtime.universe._
class DependentMap[K[_],V[_]](
inner: Map[
(TypeTag[_], Any),
Any
] = Map()
) {
def add[A](
key: K[A], value: V[A]
)(
implicit tt: TypeTag[A]
): DependentMap[K,V] = {
val realKey: (TypeTag[_], Any) = (tt, key)
new DependentMap(inner + ((realKey, value)))
}
def get[A](key: K[A])(implicit tt: TypeTag[A]): Option[V[A]] = {
val realKey: (TypeTag[_], Any) = (tt, key)
inner.get(realKey).map(_.asInstanceOf[V[A]])
}
}
And here are a few examples demonstrating that it works as expected.
scala> val myMap: DependentMap[Foo,Bar] = new DependentMap
scala> myMap.add(Foo(42), Bar(43)).get(Foo(42))
res0: Option[Bar[Int]] = Some(Bar(43))
scala> myMap.add(Foo("foo"), Bar("bar")).get(Foo("foo"))
res1: Option[Bar[String]] = Some(Bar(bar))
scala> myMap.add(Foo(42), Bar("bar")).get(Foo(42))
error: type mismatch;
scala> myMap.add[Any](Foo(42), Bar("bar")).get(Foo(42))
res2: Option[Bar[Int]] = None
scala> myMap.add[Any](Foo(42), Bar("bar")).get[Any](Foo(42))
res3: Option[Bar[Any]] = Some(Bar(bar))
scala> myMap.add(Foo(List[Int]()), Bar(List(43))).get(Foo(List[Int]()))
res4: Option[Bar[List[Int]]] = Some(Bar(List(43)))
scala> myMap.add(Foo(List[Int]()), Bar(List(43))).get(Foo(List[String]()))
res5: Option[Bar[List[String]]] = None
This works, but the cast is ugly. I'd rather find a better way.
Then you're probably disappointed that my get is implemented using a cast. Let me try to convince you that there is no other way.
Instead of a map which may or may not contain our key, let's consider a much simpler case.
def safeCast[A,B](
value: A
)(
implicit tt1: TypeTag[A], tt2: TypeTag[B]
): Option[B] = {
if (tt1 == tt2)
Some(value.asInstanceOf[B])
else
None
}
Given a type tag for A and a type tag for B, if the two type tags are equal, then we know that A and B are the same type, and therefore the typecast is safe. But how could we possibly implement this without a typecast? The equality check returns a mere boolean, not a witness of equality like in some other languages. Therefore there is nothing which statically distinguishes the two branches of the if and the compiler cannot possibly know that a cast-free conversion is legal in the "true" branch but illegal in the other.
In the more complicated case of our DependentMap, we also have to compare our type tag with the one we stored when we did an add, and so we also have to use a cast.
I gather the answer is to use existential types with the forSome keyword
I can see why you'd think so. You want a bunch of associations from key to value, where each (key, value) pair has the type (Foo[A], Bar[A]) forSome {type A}. And indeed, if you were not concerned with performance, you could store those associations in a list:
val myList: List[(Foo[A], Bar[A]) forSome {type A}]
Since the forSome is inside the brackets, this allows each entry in the list to use a different A. And since Foo[A] and Bar[A] are both to the left of the forSome, in each entry the As must match.
In the case of Map, however, there is no place to put the forSome to obtain the result you want. The problem with Map is that it has two type parameters, which prevents us from from putting them both to the left of the forSome without putting the forSome outside of the brackets. It wouldn't make sense to do so: since the two type parameters are independent, there is nothing which links one occurrence of the left type parameter to a corresponding occurrence of the right type parameter, and so there is no way to know which As should match. Consider the following counter-example:
case class NotMap[K,V](k1: K, k2: K, v1: V, v2: V, v3: V)
There isn't the same number of K values as there are V values, so in particular there is no correspondence between the K values and the V values. If there was some special syntax like Map[{Foo[A], Bar[A]} forSome {type A}] which would allow us to specify that for each entry in the Map, the As should match, then it would also be possible to use that syntax to specify the nonsense type NotMap[{Foo[A], Bar[A]} forSome {type A}]. Therefore there is no such syntax, and we need to use a type other than Map, such as DependentMap or HMap.
What about something like
def map[A]: Map[Foo[A], Bar[A]] = ...
val myMap = map[Qux]
...
val foo = new Foo[Qux]
val bar: Bar[Qux] = myMap(foo)
Or (inspired by Alexey Romanov's answer)
type MyMap[A] = Map[Foo[A],Bar[A]]
val map:MyMap[Qux] = ...
...
val foo = new Foo[Qux]
val bar: Bar[Qux] = map(foo)

Is it possible to have a manifest defined based on another manifest in Scala?

Is it possible to have a manifest defined based on another manifest in Scala?
I've pretty much resigned myself to the belief that this is not possible because the Scala Manifest information was not intended to be used dynamically.
Here's the problem. I have a function that can return more than one type of object (String, Int, List[Int], List[List[String]], etc.) To support these multiple types, the return type is set to Any, but due to type erasure the information about the types supported in Lists, Maps, etc is lost. In an attempt to recover some of the details, I return a Manifest along with the return type.
However, the returned information may then be placed in another list or map and that is then returned from another function. I want to update the manifest to include the fact that the type is now a List or Map of the previous type as defined by the manifest.
Here's some example code
def returnWithManifest[T: Manifest](x: T) = (x, manifest[T])
// May return String, Int, List[Int], List[List[String]], ...
def contrivedExample(t: String): (Any, Manifest[_]) = t match {
case "String" => returnWithManifest("test")
case "Int" => returnWithManifest(1)
case "Boolean" => returnWithManifest(true)
case "List[Int]" => returnWithManifest(List(1,2,3))
case "List[List[String]]" =>
returnWithManifest(List(List("a","b"),List("c","d")))
case _ => returnWithManifest(None)
}
scala> val v1 = contrivedExample("List[Int]")
v1: (Any, Manifest[_]) = (List(1, 2, 3),scala.collection.immutable.List[Int])
scala> val x = v1._1
x: Any = List(1, 2, 3)
scala> val m = v1._2
m: scala.reflect.Manifest[_] = scala.collection.immutable.List[Int]
scala> val v2 = List(x)
v2: List[Any] = List(List(1, 2, 3))
From the manifest of 'v1' I know that v1 is of type List[Int] so when I create 'v2' I should have all the information I need to create a manifest identifying that the type is List[List[Int]], but instead I only have List[Any] to work with. Perhaps syntax like the following:
val v2: m = List(x)
val v2 = List[m](x)
I realize it looks like I'm trying to define a type dynamically, but in reality the information is metadata related to type erasure of statically known types. I guess if this can be solved, then type erasure can be solved. However, at the very least I thought I should be able to do something like:
scala> val m2 = m.wrapInList()
m2: scala.reflect.Manifest[_] =
scala.collection.immutable.List[scala.collection.immutable.List[Int]]
Edit: Adriaan Moors is right pointing out this works:
def makeListManifest[T: Manifest] = manifest[List[T]]
You'll just have to call it explicitly, passing it the manifest you have obtained.
My old answer:
huynhjl is partially right: currently this won't work automatically. We'd need the compiler to be smart enough to compile this:
def makeListManifest[T](m: Manifest[T]) = manifest[List[T]]
without any extra implicit parameter. While it certainly looks feasible (all the needed info is here), it isn't implemented yet (2.9.0.1), as I believe manifests are now either inserted locally if the compiler has all the static type info it needs or looked up in the implicit scope, but not generated from other (possibly implicitly available) manifests.
What you can do, however, is construct that manifest yourself with the methods on the companion object:
scala> import reflect.Manifest
scala> Manifest.classType(classOf[List[_]], manifest[Int])
res0: scala.reflect.Manifest[List[_]] = scala.collection.immutable.List[Int]
So you can implement makeListManifest yourself:
scala> def makeListManifest[T](m: Manifest[T]) = Manifest.classType(classOf[List[_]], m)
makeListManifest: [T](m: scala.reflect.Manifest[T])scala.reflect.Manifest[List[_]]
Note that although the right manifest will be returned, the static return type of makeListManifest is only Manifest[List[_]]. But you could safely cast to Manifest[List[T]] here.
Note that Scala's type system allows you to do better than returning Any. There are several ways to define type unions (a.k.a. "disjunctive types"). See e.g.
How to define "type disjunction" (union types)?
http://cleverlytitled.blogspot.com/2009/03/so-i-read-both-jim-mcbeath-and-michids.html
http://www.chuusai.com/2011/06/09/scala-union-types-curry-howard/
Of course you can also have your own ADT for the return type, which is IMHO the cleanest solution:
trait ReturnValue
case class ReturnInt(value: Int) extends ReturnValue
case class ReturnString(value: String) extends ReturnValue
case class ReturnIntList(value: List[Int]) extends ReturnValue
...
[Edit]
Going with the definition from the second link, we could write:
def contrivedExample(t: String): String or List[Int] or List[List[String]] = t match {
case "String" => "test"
case "List[Int]" => List(1,2,3)
case "List[List[String]]" => List(List("a","b"),List("c","d"))
}
Now we can retrieve verbosely but safely the type:
def otherContrivedExample(t: String or List[Int] or List[List[String]]) = t match {
case DisjointType(Some(DisjointType(Some(s),_)), _) => println("processing String: " + s)
case DisjointType(Some(DisjointType(_,Some(s))), _) => println("processing List[String]: " + s)
case DisjointType(_,Some(s)) => println("processing List[List[Int]]: head=" + s.head)
}
val x = contrivedExample("List[List[String]]")
otherContrivedExample(x)
//--> processing List[List[Int]]: head=List(a, b)
As you can see that the matched variable s has the right type, despite we didn't mention it. I'm pretty sure the extraction process could be simplified by using implicit magic and / or special extractors.
I think because your function returns Manifest[_], the compiler has lost the necessary information to recover the type. If m was of type Manifest[List[Int]] that would be a different story.