Related
Scala suffers type erasure, but it also introduces TypeTag to overcome type erasure problem.
Sometimes we can use implicit parameters(or even value paramters are enough) to solve the same problem that we countered.
For exammple:
// implicit paramters
def getKindName[T](implicit x: T): String = {
x match {
case _: People => "Mammals"
case _: Sparrow => "Birds"
case _: Shark => "Fishes"
case _: Crocodile => "Reptiles"
case _ => ???
}
}
// TypeTag
def getKindName[T: TypeTag]: String = {
typeOf[T] match {
case t if t =:= typeOf[People] => "Mammals"
case t if t =:= typeOf[Sparrow] => "Birds"
case t if t =:= typeOf[Shark] => "Fishes"
case t if t =:= typeOf[Crocodile] => "Reptiles"
case _ => ???
}
}
So my question are:
why and when use Type Tag and implicit paramters ?
what are the differences of perfomance between Type Tag and implicit paramters?
If you could give me a link of doc in scala to prove your opinion, it is perfect. Thank you in advance.
TypeTags are nearly always used as implicit parameters. If you aren't aware, getKindName[T: TypeTag] means getKindName[T](implicit tag: TypeTag[T]).
The exception would be when you store a TypeTag which you got as an implicit parameter somewhere. So "implicit parameters vs TypeTag" doesn't make sense to begin with.
The two snippets do not behave at all similarly, so they can't be solving the same problem, whatever that problem is. E.g. you can define
implicit val x: Animal = new People
getKindName1[Sparrow] // returns "Mammals"
or without any extra implicits being defined, getKindName1[Sparrow] will not compile at all (and it isn't clear why you would have e.g. an implicit Sparrow in the first place).
It's a sad fact of life on Scala that if you instantiate a List[Int], you can verify that your instance is a List, and you can verify that any individual element of it is an Int, but not that it is a List[Int], as can be easily verified:
scala> List(1,2,3) match {
| case l : List[String] => println("A list of strings?!")
| case _ => println("Ok")
| }
warning: there were unchecked warnings; re-run with -unchecked for details
A list of strings?!
The -unchecked option puts the blame squarely on type erasure:
scala> List(1,2,3) match {
| case l : List[String] => println("A list of strings?!")
| case _ => println("Ok")
| }
<console>:6: warning: non variable type-argument String in type pattern is unchecked since it is eliminated by erasure
case l : List[String] => println("A list of strings?!")
^
A list of strings?!
Why is that, and how do I get around it?
This answer uses the Manifest-API, which is deprecated as of Scala 2.10. Please see answers below for more current solutions.
Scala was defined with Type Erasure because the Java Virtual Machine (JVM), unlike Java, did not get generics. This means that, at run time, only the class exists, not its type parameters. In the example, JVM knows it is handling a scala.collection.immutable.List, but not that this list is parameterized with Int.
Fortunately, there's a feature in Scala that lets you get around that. It’s the Manifest. A Manifest is class whose instances are objects representing types. Since these instances are objects, you can pass them around, store them, and generally call methods on them. With the support of implicit parameters, it becomes a very powerful tool. Take the following example, for instance:
object Registry {
import scala.reflect.Manifest
private var map= Map.empty[Any,(Manifest[_], Any)]
def register[T](name: Any, item: T)(implicit m: Manifest[T]) {
map = map.updated(name, m -> item)
}
def get[T](key:Any)(implicit m : Manifest[T]): Option[T] = {
map get key flatMap {
case (om, s) => if (om <:< m) Some(s.asInstanceOf[T]) else None
}
}
}
scala> Registry.register("a", List(1,2,3))
scala> Registry.get[List[Int]]("a")
res6: Option[List[Int]] = Some(List(1, 2, 3))
scala> Registry.get[List[String]]("a")
res7: Option[List[String]] = None
When storing an element, we store a "Manifest" of it too. A Manifest is a class whose instances represent Scala types. These objects have more information than JVM does, which enable us to test for the full, parameterized type.
Note, however, that a Manifest is still an evolving feature. As an example of its limitations, it presently doesn't know anything about variance, and assumes everything is co-variant. I expect it will get more stable and solid once the Scala reflection library, presently under development, gets finished.
You can do this using TypeTags (as Daniel already mentions, but I'll just spell it out explicitly):
import scala.reflect.runtime.universe._
def matchList[A: TypeTag](list: List[A]) = list match {
case strlist: List[String #unchecked] if typeOf[A] =:= typeOf[String] => println("A list of strings!")
case intlist: List[Int #unchecked] if typeOf[A] =:= typeOf[Int] => println("A list of ints!")
}
You can also do this using ClassTags (which saves you from having to depend on scala-reflect):
import scala.reflect.{ClassTag, classTag}
def matchList2[A : ClassTag](list: List[A]) = list match {
case strlist: List[String #unchecked] if classTag[A] == classTag[String] => println("A List of strings!")
case intlist: List[Int #unchecked] if classTag[A] == classTag[Int] => println("A list of ints!")
}
ClassTags can be used so long as you don't expect the type parameter A to itself be a generic type.
Unfortunately it's a little verbose and you need the #unchecked annotation to suppress a compiler warning. The TypeTag may be incorporated into the pattern match automatically by the compiler in the future: https://issues.scala-lang.org/browse/SI-6517
You can use the Typeable type class from shapeless to get the result you're after,
Sample REPL session,
scala> import shapeless.syntax.typeable._
import shapeless.syntax.typeable._
scala> val l1 : Any = List(1,2,3)
l1: Any = List(1, 2, 3)
scala> l1.cast[List[String]]
res0: Option[List[String]] = None
scala> l1.cast[List[Int]]
res1: Option[List[Int]] = Some(List(1, 2, 3))
The cast operation will be as precise wrt erasure as possible given the in-scope Typeable instances available.
I came up with a relatively simple solution that would suffice in limited-use situations, essentially wrapping parameterized types that would suffer from the type erasure problem in wrapper classes that can be used in a match statement.
case class StringListHolder(list:List[String])
StringListHolder(List("str1","str2")) match {
case holder: StringListHolder => holder.list foreach println
}
This has the expected output and limits the contents of our case class to the desired type, String Lists.
More details here: http://www.scalafied.com/?p=60
There is a way to overcome the type erasure issue in Scala. In Overcoming Type Erasure in matching 1 and Overcoming Type Erasure in Matching 2 (Variance) are some explanation of how to code some helpers to wrap the types, including Variance, for matching.
I found a slightly better workaround for this limitation of the otherwise awesome language.
In Scala, the issue of type erasure does not occur with arrays. I think it is easier to demonstrate this with an example.
Let us say we have a list of (Int, String), then the following gives a type erasure warning
x match {
case l:List[(Int, String)] =>
...
}
To work around this, first create a case class:
case class IntString(i:Int, s:String)
then in the pattern matching do something like:
x match {
case a:Array[IntString] =>
...
}
which seems to work perfectly.
This will require minor changes in your code to work with arrays instead of lists, but should not be a major problem.
Note that using case a:Array[(Int, String)] will still give a type erasure warning, so it is necessary to use a new container class (in this example, IntString).
Since Java does not know the actual element type, I found it most useful to just use List[_]. Then the warning goes away and the code describes reality - it is a list of something unknown.
I'm wondering if this is a suited workaround:
scala> List(1,2,3) match {
| case List(_: String, _*) => println("A list of strings?!")
| case _ => println("Ok")
| }
It does not match the "empty list" case, but it gives a compile error, not a warning!
error: type mismatch;
found: String
requirerd: Int
This on the other hand seems to work....
scala> List(1,2,3) match {
| case List(_: Int, _*) => println("A list of ints")
| case _ => println("Ok")
| }
Isn't it kinda even better or am I missing the point here?
Not a solution but a way to live with it without sweeping it under the rug altogether:
Adding the #unchecked annotation. See here - http://www.scala-lang.org/api/current/index.html#scala.unchecked
I wanted to add an answer which generalises the problem to: How do a get a String representation of the type of my list at runtime
import scala.reflect.runtime.universe._
def whatListAmI[A : TypeTag](list : List[A]) = {
if (typeTag[A] == typeTag[java.lang.String]) // note that typeTag[String] does not match due to type alias being a different type
println("its a String")
else if (typeTag[A] == typeTag[Int])
println("its a Int")
s"A List of ${typeTag[A].tpe.toString}"
}
val listInt = List(1,2,3)
val listString = List("a", "b", "c")
println(whatListAmI(listInt))
println(whatListAmI(listString))
Using pattern match guard
list match {
case x:List if x.isInstanceOf(List[String]) => do sth
case x:List if x.isInstanceOf(List[Int]) => do sth else
}
Let's suppose we have a generic class Container:
case class Container[+A](value: A)
We then want to pattern match a Container with a Double and a Container of Any:
val double = Container(3.3)
var container: Container[Any] = double
To do this, we would normally write:
container match {
case c: Container[String] => println(c.value.toUpperCase)
case c: Container[Double] => println(math.sqrt(c.value))
case _ => println("_")
}
However, the compiler gives two warnings, one for each of the first two cases. For example, the first warning says: "non-variable type argument String in type pattern Container[String] is unchecked since it is eliminated by erasure". Because of the erasure, it is impossible during runtime to distinguish between different kinds of containers and the first catch will be matched. As a consequence, container of type Container[Double] will be matched by the first case, which catches Container[String] objects, so toUpperCase method will be called on a Double and a java.lang.ClassCastException will be thrown.
How to match a Container parametrized by a particular type?
Maybe this will help
def matchContainer[A: Manifest](c: Container[A]) = c match {
case c: Container[String] if manifest <:< manifest[String] => println(c.value.toUpperCase)
case c: Container[Double] if manifest <:< manifest[Double] => println(math.sqrt(c.value))
case c: Container[_] => println("other")
}
Edit:
As Impredicative pointed out, Manifest is deprecated. Instead you could do the following:
import reflect.runtime.universe._
def matchContainer[A: TypeTag](c: Container[A]) = c match {
case c: Container[String] if typeOf[A] <:< typeOf[String] => println("string: " + c.value.toUpperCase)
case c: Container[Double] if typeOf[A] <:< typeOf[Double] => println("double" + math.sqrt(c.value))
case c: Container[_] => println("other")
}
In general rarry's answer is correct, for your case however it can be simplified, because your container only contains a single value of a generic type, so you can match on that value's type directly:
container match {
case Container(x: String) => println("string")
case Container(x: Double) => println("double")
case _ => println("w00t")
}
A possible workaround for this could be to use isInstanceOf and asInstanceOf.
container match {
case Container(x) if x.isInstanceOf[String] =>
println(x.asInstanceOf[String].toUpperCase)
case Container(x) if x.isInstanceOf[Double] =>
println(math.sqrt(x.asInstanceOf[Double]))
case _ => println("_")
}
This works, but it doesn't look elegant at all. Professor Martin Odersky, the creator of Scala, says that isInstanceOf and asInstanceOf should be avoided.
As Rob Norris pointed me out, on the forum of the course "Functional programming in Scala" from Coursera, matching by type is a bad practice: case foo: Bar => .... Scala encourages to take advantage of static typing and avoid checking type during runtime. This is consistent with the philosophy of Haskell/ML world. Instead of matching types, case clauses should match constructors.
To solve the Container matching problem, a special container for each type can be defined:
class Container[+A](val value: A)
case class StringContainer(override val value: String)
extends Container(value)
case class DoubleContainer(override val value: Double)
extends Container(value)
And now constructors will be matched, not types:
container match {
case StringContainer(x) => println(x.toUpperCase)
case DoubleContainer(x) => println(math.sqrt(x))
case _ => println("_")
}
Apparently, we could be defined unapply methods in two objects, StringContainer and DoubleContainer and use the same match as above, instead of extending the Container class:
case class Container[+A](val value: A)
object StringContainer {
def unapply(c: Container[String]): Option[String] = Some(c.value)
}
object DoubleContainer {
def unapply(c: Container[Double]): Option[Double] = Some(c.value)
}
But this does not work, again, because of JVM type erasure.
A reference to Rob Norris post, which lead me to this answer can be found here: https://class.coursera.org/progfun-002/forum/thread?thread_id=842#post-3567 . Unfortunately, you can't access it unless you are enrolled in the Coursera course.
Note: you also have an alternative with Miles Sabin's Shapeless library (already mentioned by Miles in 2012 here).
You can see an example in "Ways to pattern match generic types in Scala" from Jaakko Pallari
Typeable is a type class that provides the ability to cast values from Any type to a specific type.
The result of the casting operation is an Option where the Some value will contain the successfully casted value, and the None value represents a cast failure.
TypeCase bridges Typeable and pattern matching. It's essentially an extractor for Typeable instances
import shapeless._
def extractCollection[T: Typeable](a: Any): Option[Iterable[T]] = {
val list = TypeCase[List[T]]
val set = TypeCase[Set[T]]
a match {
case list(l) => Some(l)
case set(s) => Some(s)
case _ => None
}
}
val l1: Any = List(1, 2, 3)
val l2: Any = List[Int]()
val s: Any = Set(1, 2, 3)
extractCollection[Int](l1) // Some(List(1, 2, 3))
extractCollection[Int](s) // Some(Set(1, 2, 3))
extractCollection[String](l1) // None
extractCollection[String](s) // None
extractCollection[String](l2) // Some(List()) // Shouldn't this be None? We'll get back to this.
While Typeable may look like it has what it takes to solve type erasure, it's still subject to the same behaviour as any other runtime code.
This can be seen in the last lines of the previous code examples where empty lists were recognized as string lists even when they were specified to be integer lists. This is because Typeable casts are based on the values of the list. If the list is empty, then naturally that is a valid string list and a valid integer list (or any other list for that matter)
Could someone explain why the following code compiles?
Option("foo") match {
case x: List[String] => println("A")
case _ => println("B")
}
This gives me an (expected) warning about type erasure, but it still compiles. I expected this to throw a type error, like it does if I matched on "foo" instead of Option("foo").
Thanks!
The code is commented, so let's take a moment to savor that:
/** If we can absolutely rule out a match we can fail early.
* This is the case if the scrutinee has no unresolved type arguments
* and is a "final type", meaning final + invariant in all type parameters.
*/
Notice that None is not final, for instance. I know, right?
If you ever try scalac -Ypatmat-debug, the comment here might help:
https://github.com/scala/scala/pull/650
Reachability is almost within reach:
https://issues.scala-lang.org/browse/SI-6146
But I don't see any promises about what might someday be warnable. For performance reasons? One could also say, why should it warn about an instanceOf[Foo[_]]?
For now, the spec sections 8.2 - 8.4 motivate why matching against Foo[a] is interesting (because of the bounds a acquires). I think I'll go read that again. After some coffee.
trait Foo[+A]
final class Fuzz[+A] extends Foo[A]
final object Fooz extends Foo[Nothing]
object Futz extends Foo[Nothing]
//error
Fooz match {
case x: List[_] => println("A")
case _ => println("B")
}
//no error
Futz match { ... }
I would assume that the compiler is treating both Option and List as Product, which is why it compiles. As you say, the warning about type erasure is expected. Here's an example that uses another Product:
scala> Option("foo") match {
| case x: Tuple2[String,String] => println("TUPLE")
| case x: List[String] => println("LIST")
| case _ => println("OTHER")
| }
<console>:9: warning: non variable type-argument String in type pattern (String, String) is unchecked since it is eliminated by erasure
case x: Tuple2[String,String] => println("TUPLE")
^
<console>:10: warning: non variable type-argument String in type pattern List[String] is unchecked since it is eliminated by erasure
case x: List[String] => println("LIST")
^
UPDATE w/r/t case classes (because of the comment below):
scala> case class Foo(bar: Int)
defined class Foo
scala> val y: Product = Foo(123)
y: Product = Foo(123)
I noticed that an error is shown when the class of the value you match is declared as final (and we know that String is final). I still don't know why there's no error without it.
It's a sad fact of life on Scala that if you instantiate a List[Int], you can verify that your instance is a List, and you can verify that any individual element of it is an Int, but not that it is a List[Int], as can be easily verified:
scala> List(1,2,3) match {
| case l : List[String] => println("A list of strings?!")
| case _ => println("Ok")
| }
warning: there were unchecked warnings; re-run with -unchecked for details
A list of strings?!
The -unchecked option puts the blame squarely on type erasure:
scala> List(1,2,3) match {
| case l : List[String] => println("A list of strings?!")
| case _ => println("Ok")
| }
<console>:6: warning: non variable type-argument String in type pattern is unchecked since it is eliminated by erasure
case l : List[String] => println("A list of strings?!")
^
A list of strings?!
Why is that, and how do I get around it?
This answer uses the Manifest-API, which is deprecated as of Scala 2.10. Please see answers below for more current solutions.
Scala was defined with Type Erasure because the Java Virtual Machine (JVM), unlike Java, did not get generics. This means that, at run time, only the class exists, not its type parameters. In the example, JVM knows it is handling a scala.collection.immutable.List, but not that this list is parameterized with Int.
Fortunately, there's a feature in Scala that lets you get around that. It’s the Manifest. A Manifest is class whose instances are objects representing types. Since these instances are objects, you can pass them around, store them, and generally call methods on them. With the support of implicit parameters, it becomes a very powerful tool. Take the following example, for instance:
object Registry {
import scala.reflect.Manifest
private var map= Map.empty[Any,(Manifest[_], Any)]
def register[T](name: Any, item: T)(implicit m: Manifest[T]) {
map = map.updated(name, m -> item)
}
def get[T](key:Any)(implicit m : Manifest[T]): Option[T] = {
map get key flatMap {
case (om, s) => if (om <:< m) Some(s.asInstanceOf[T]) else None
}
}
}
scala> Registry.register("a", List(1,2,3))
scala> Registry.get[List[Int]]("a")
res6: Option[List[Int]] = Some(List(1, 2, 3))
scala> Registry.get[List[String]]("a")
res7: Option[List[String]] = None
When storing an element, we store a "Manifest" of it too. A Manifest is a class whose instances represent Scala types. These objects have more information than JVM does, which enable us to test for the full, parameterized type.
Note, however, that a Manifest is still an evolving feature. As an example of its limitations, it presently doesn't know anything about variance, and assumes everything is co-variant. I expect it will get more stable and solid once the Scala reflection library, presently under development, gets finished.
You can do this using TypeTags (as Daniel already mentions, but I'll just spell it out explicitly):
import scala.reflect.runtime.universe._
def matchList[A: TypeTag](list: List[A]) = list match {
case strlist: List[String #unchecked] if typeOf[A] =:= typeOf[String] => println("A list of strings!")
case intlist: List[Int #unchecked] if typeOf[A] =:= typeOf[Int] => println("A list of ints!")
}
You can also do this using ClassTags (which saves you from having to depend on scala-reflect):
import scala.reflect.{ClassTag, classTag}
def matchList2[A : ClassTag](list: List[A]) = list match {
case strlist: List[String #unchecked] if classTag[A] == classTag[String] => println("A List of strings!")
case intlist: List[Int #unchecked] if classTag[A] == classTag[Int] => println("A list of ints!")
}
ClassTags can be used so long as you don't expect the type parameter A to itself be a generic type.
Unfortunately it's a little verbose and you need the #unchecked annotation to suppress a compiler warning. The TypeTag may be incorporated into the pattern match automatically by the compiler in the future: https://issues.scala-lang.org/browse/SI-6517
You can use the Typeable type class from shapeless to get the result you're after,
Sample REPL session,
scala> import shapeless.syntax.typeable._
import shapeless.syntax.typeable._
scala> val l1 : Any = List(1,2,3)
l1: Any = List(1, 2, 3)
scala> l1.cast[List[String]]
res0: Option[List[String]] = None
scala> l1.cast[List[Int]]
res1: Option[List[Int]] = Some(List(1, 2, 3))
The cast operation will be as precise wrt erasure as possible given the in-scope Typeable instances available.
I came up with a relatively simple solution that would suffice in limited-use situations, essentially wrapping parameterized types that would suffer from the type erasure problem in wrapper classes that can be used in a match statement.
case class StringListHolder(list:List[String])
StringListHolder(List("str1","str2")) match {
case holder: StringListHolder => holder.list foreach println
}
This has the expected output and limits the contents of our case class to the desired type, String Lists.
More details here: http://www.scalafied.com/?p=60
There is a way to overcome the type erasure issue in Scala. In Overcoming Type Erasure in matching 1 and Overcoming Type Erasure in Matching 2 (Variance) are some explanation of how to code some helpers to wrap the types, including Variance, for matching.
I found a slightly better workaround for this limitation of the otherwise awesome language.
In Scala, the issue of type erasure does not occur with arrays. I think it is easier to demonstrate this with an example.
Let us say we have a list of (Int, String), then the following gives a type erasure warning
x match {
case l:List[(Int, String)] =>
...
}
To work around this, first create a case class:
case class IntString(i:Int, s:String)
then in the pattern matching do something like:
x match {
case a:Array[IntString] =>
...
}
which seems to work perfectly.
This will require minor changes in your code to work with arrays instead of lists, but should not be a major problem.
Note that using case a:Array[(Int, String)] will still give a type erasure warning, so it is necessary to use a new container class (in this example, IntString).
Since Java does not know the actual element type, I found it most useful to just use List[_]. Then the warning goes away and the code describes reality - it is a list of something unknown.
I'm wondering if this is a suited workaround:
scala> List(1,2,3) match {
| case List(_: String, _*) => println("A list of strings?!")
| case _ => println("Ok")
| }
It does not match the "empty list" case, but it gives a compile error, not a warning!
error: type mismatch;
found: String
requirerd: Int
This on the other hand seems to work....
scala> List(1,2,3) match {
| case List(_: Int, _*) => println("A list of ints")
| case _ => println("Ok")
| }
Isn't it kinda even better or am I missing the point here?
Not a solution but a way to live with it without sweeping it under the rug altogether:
Adding the #unchecked annotation. See here - http://www.scala-lang.org/api/current/index.html#scala.unchecked
I wanted to add an answer which generalises the problem to: How do a get a String representation of the type of my list at runtime
import scala.reflect.runtime.universe._
def whatListAmI[A : TypeTag](list : List[A]) = {
if (typeTag[A] == typeTag[java.lang.String]) // note that typeTag[String] does not match due to type alias being a different type
println("its a String")
else if (typeTag[A] == typeTag[Int])
println("its a Int")
s"A List of ${typeTag[A].tpe.toString}"
}
val listInt = List(1,2,3)
val listString = List("a", "b", "c")
println(whatListAmI(listInt))
println(whatListAmI(listString))
Using pattern match guard
list match {
case x:List if x.isInstanceOf(List[String]) => do sth
case x:List if x.isInstanceOf(List[Int]) => do sth else
}