I'm trying to dynamically filter (or collect) a list based on type:
If I do this specifying the type explicitly, it works fine
scala> var aList = List("one", 2, 3.3)
aList: List[Any] = List(one, 2, 3.3)
scala> aList.collect{case x:Int => x}
res10: List[Int] = List(2)
If I want to write a method to do this generically, then it doesn't:
scala> def collectType[T](l:List[Any]):List[T] = l.collect{case x:T => x}
warning: there were unchecked warnings; re-run with -unchecked for details
collectType: [T](l: List[Any])List[T]
scala> collectType[Int](aList)
res11: List[Int] = List(one, 2, 3.3)
scala> collectType[Double](aList)
res16: List[Double] = List(one, 2, 3.3)
scala> collectType[String](aList)
res14: List[String] = List(one, 2, 3.3)
I thought at first that it was naming the type 'Integer' rather than using Integer as the type, but that doesn't seem to be the case as:
collectType[Int](aList).foreach(x => println(x))
java.lang.ClassCastException: java.lang.String cannot be cast to java.lang.Integer
It's as though it's deferring checking the type until it's forced to
What am I missing about Types?
Is there a way to achieve what I want to achieve?
After reading through the linked questions, this is what I've come up with. Pretty simple now that it's been pointed out. Taggable is a trait which knows how to hold a map of tags for a class
def matches[F <: Taggable](thing:Taggable)(implicit m:Manifest[F]):Boolean = {
thing match {
case e if (m >:> singleType(e)) => true
case x => false
}
}
def findByType[G <: Taggable](list:List[Taggable])(implicit m:Manifest[G]) = {
list.collect{case x if (matches[G](x)) => x}
}
You are missing type erasure. At runtime your method is actually
def collectType(l:List):List = l.collect {case x:Object => x}
Related
Consider a flatMap written over some case matching. For example:
list.flatMap( v =>
v match {
case Cond1 => if(something) Some(Int) else None
//..Other conditions yielding Option[Int]
case CondN => if(somethingelse) Seq(Int) else Seq()
})
However this wont compile. If the seq is all of Option[Int] or all of Seq[Int] the flatMap works. But not if the Seq is a mix of Options and Seqs. Why is such a restriction in place? Does this solve a particular ambiguity that I cannot think of as of now.
EDIT1
Adding code snippet from REPL
scala> val a = Seq(Option(1), Seq(2,3))
a: Seq[Equals] = List(Some(1), List(2, 3))
scala> val b = Seq(Seq(1), Seq(2,3))
b: Seq[Seq[Int]] = List(List(1), List(2, 3))
scala> a.flatMap(x=>x)
<console>:9: error: type mismatch;
found : Equals
required: scala.collection.GenTraversableOnce[?]
a.flatMap(x=>x)
^
scala> b.flatMap(x=>x)
res24: Seq[Int] = List(1, 2, 3)
EDIT2
After Filippo's answer I tried the following piece of code in the REPL and it worked.
scala> val options = Seq("opt1", "opt2")
options: Seq[String] = List(opt1, opt2)
scala> options.flatMap( x =>
| x match {
| case "opt1" => Some(1)
| case "opt2" => Seq(2,3)
| case _ => None
| })
res27: Seq[Int] = List(1, 2, 3)
How is the resolution different in each of the scenarios.
More importantly when I map instead of flatMap the result is the same as the Seq a that I had created.
scala> options.map( x =>
| x match {
| case "opt1" => Some(1)
| case "opt2" => Seq(2,3)
| case _ => None
| })
res28: Seq[Equals] = List(Some(1), List(2, 3))
Option is a GenTraversableOnce but scala needs some help here:
val a: Seq[TraversableOnce[Int]] = Seq(Option(1), Seq(2,3))
a.flatMap(x=>x)
res0: Seq[Int] = List(1, 2, 3)
EDIT after additions to the question
I think that if the type of your sequence is the one you are expecting, everything boils down to the function passed to the flatMap. If scala can't figure out that the function is (A) => Traversable[A] when the starting sequence is Seq[A], I think we should make some types explicit.
Now, back to your first sample, I would refactor it as:
list.flatMap {
case Cond1 if something => Seq(Int)
case CondN if somethingelse => Seq(Int)
case _ => Seq()
}
No doubt scala is now able to infer the types correctly.
Imagine I have this class:
class Example {
val list = List(new Apple(), new Orange(), Banana());
def getIfPresent[T <: Fruit] : Option[T] = list.collectFirst { case x : T => x }
}
You use it like this:
val example = new Example();
match example.getIfPresent[Apple] {
case Some(apple) => apple.someAppleSpecificMethod();
case None => println("No apple");
}
Now, of course, this doesn't work in the JVM, because of type erasure. getIfPresent just matches on the type Fruit in the collectFirst partial function, instead of the actual type specified in the call.
I have tried to get my head around type tags and class tags, and really have no idea how I would implement the above method. The examples that I see are trying to do very different things. How could I achieve a method that does what I want, either with TypeTags or some other mechanism I'm unaware of?
Edit: m-z's answer below is the full solution, but here is how it looks with my example code:
class Example {
val list = List(new Apple(), new Orange(), Banana());
def getIfPresent[T <: Fruit : ClassTag] : Option[T] = list.collectFirst { case x : T => x }
}
Just needed to add : ClassTag!
You can do this using ClassTag, to an extent.
import scala.reflect.ClassTag
// Modify to apply whatever type bounds you find necessary
// Requires Scala ~2.11.5 or greater (not sure of the exact version, but 2.11.1 does not work, and 2.11.5 does)
def findFirst[A : ClassTag](list: List[Any]): Option[A] =
list collectFirst { case a: A => a }
val l = List(1, "a", false, List(1, 2, 3), List("a", "b"))
scala> findFirst[Boolean](l)
res22: Option[Boolean] = Some(false)
scala> findFirst[Long](l)
res23: Option[Long] = None
But there are some caveats with ClassTag, in that it will only match the class, and not the type:
scala> findFirst[List[String]](l)
res24: Option[List[String]] = Some(List(1, 2, 3)) // No!
You can use a TypeTag to get around this, but it won't work with a List[Any]. Here is one possible (sort of ugly) trick:
import scala.reflect.runtime.universe.{typeOf, TypeTag}
case class Tagged[A : TypeTag](a: A) {
def tpe = typeOf[A]
}
implicit class AnyTagged[A : TypeTag](a: A) {
def tag = Tagged(a)
}
def findFirst[A : TypeTag](list: List[Tagged[_]]): Option[A] =
list collectFirst { case tag # Tagged(a) if(tag.tpe =:= typeOf[A]) => a.asInstanceOf[A] }
The only way I can think of to hold onto the TypeTag of each element is to literally hold onto it with a wrapper class. So I have to construct the list like this:
val l = List(1.tag, "a".tag, false.tag, List(1, 2, 3).tag, List("a", "b").tag)
But it works:
scala> findFirst[List[String]](l)
res26: Option[List[String]] = Some(List(a, b))
There may be a more elegant way to construct such a list with TypeTags.
For fun, you can also try to do this with shapeless using an HList and select. The difference is that instead of returning Option[A], select will return A (the type you want), but if the HList contains no A, it won't compile.
import shapeless._
val l = 1 :: "a" :: false :: List(1, 2, 3) :: List("a", "b") :: HNil
scala> l.select[Boolean]
res0: Boolean = false
scala> l.select[Boolean]
res1: Boolean = false
scala> l.select[List[String]]
res2: List[String] = List(a, b)
scala> l.select[Long]
<console>:12: error: Implicit not found: shapeless.Ops.Selector[shapeless.::[Int,shapeless.::[String,shapeless.::[Boolean,shapeless.::[List[Int],shapeless.::[List[String],shapeless.HNil]]]]], Long]. You requested an element of type Long, but there is none in the HList shapeless.::[Int,shapeless.::[String,shapeless.::[Boolean,shapeless.::[List[Int],shapeless.::[List[String],shapeless.HNil]]]]].
l.select[Long]
^
I know that maps results are split into
None
Some
However I would like to only get back (when using a lookup) the actual object that is being pointed towards. Let me make myself a bit more clear:
I have the map abc
val abc = Map( 1 -> List("a","b"), 2 -> List("z","y") )
And I want to somehow get back
List("a","b")
I tried:
abc(1) // return a List
I tried to pattern match:
abc get 1 match {
case Some(x) => x
}
But the pattern match returns an Object
I tried to achieve this using a for :
(for((int, string) <- abc; if(int == 1)) yield string)
But it returns an Immutable iterator object.
I know this answer is very very basic but I can't get my head around this and I can't find anything on the internet that relates to this
REPLed it up:
$ scala
Welcome to Scala version 2.10.4 (Java HotSpot(TM) 64-Bit Server VM, Java 1.8.0_05).
Type in expressions to have them evaluated.
Type :help for more information.
scala> val abc = Map( 1 -> List("a","b"), 2 -> List("z","y") )
abc: scala.collection.immutable.Map[Int,List[String]] = Map(1 -> List(a, b), 2 -> List(z, y))
I see a difference here: You got an Option. But I got the List back. As you can see, I am on Scala 2.10.4
scala> abc(1)
res0: List[String] = List(a, b)
scala> abc get 1 match { case Some(x) => x }
<console>:9: warning: match may not be exhaustive.
It would fail on the following input: None
abc get 1 match { case Some(x) => x }
^
res1: List[String] = List(a, b)
Again, I see something different than what you see. I got back a List and not an Object.
scala> (for((int, string) <- abc; if(int == 1)) yield string)
res2: scala.collection.immutable.Iterable[List[String]] = List(List(a, b))
Yes, I got an immutable Iterator[List[String]] back. If you just need the List[String] type you can as well do this:
scala> res2.toList.flatMap(identity)
res3: List[String] = List(a, b)
Basically identity function boils down to this:
scala> res2.toList.flatMap(i => i)
res4: List[String] = List(a, b)
Or you can even do this:
scala> res2.toList.flatten
res5: List[String] = List(a, b)
Some points: You should avoid apply on a Map and do a get because you will see this:
scala> abc(3)
java.util.NoSuchElementException: key not found: 3
scala> abc.get(3)
res7: Option[List[String]] = None
I prefer something like:
scala> abc.get(1).getOrElse(List.empty[String])
res10: List[String] = List(a, b)
You probably added a default value to the map that only has Object as common supertype with List.
The withDefaultValue method uses a lower type bound on the type of the values in the Map. So when you do Map( 1 -> List("a","b"), 2 -> List("z","y") ) .withDefaultValue("x") the result will be a Map[Int, Object]
Let
val a = List ("a", 1, 2.34, "b", List(6,7))
a: List[Any] = List(a, 1, 2.34, b, List(6, 7))
and so
a.collect { case s: String => s }
res: List[String] = List(a, b)
However
a.collect { case s: List[Int] => s }
warns that
non-variable type argument Int in type pattern List[Int] is unchecked
since it is eliminated by erasure
a.collect { case s: List[Int] => s }
^
res: List[List[Int]] = List(List(6, 7))
Hence to ask whether there is a warning-free / correct approach to collect a list of integers.
Many Thanks.
On the JVM there isn't enough information at runtime to know whether a List is a List[Int]. Even a List[Any] might just happen to contain only Ints, and there is absolutely no way to tell which compile-time type it had. You could, however, code one of several things:
For each List in a, yield the subset which are Ints.
Yield the Lists in a which contain only Ints.
Same as #1, but eliminate the empty Lists.
For example, #1 could be coded as
a collect { case list:List[_] => list collect { case x:Int => x } }
As an complement to #AmigoNico's answer, there are utility functions out there that encapsulate all this behind a clean, type-safe function, for example Shapeless's Typeable type class.
Using shapeless 2.0.0-M1:
scala> val a = List ("a", 1, 2.34, "b", List(6,7))
a: List[Any] = List(a, 1, 2.34, b, List(6, 7))
scala> import syntax.typeable._
import syntax.typeable._
scala> a.flatMap(_.cast[List[Int]])
res0: List[List[Int]] = List(List(6, 7))
scala> a.flatMap(_.cast[List[String]])
res1: List[List[String]] = List()
See Type safe cast in the shapeless feature overview.
I have an heterogeneous List like the following one:
val l = List(1, "One", true)
and I need to filter its objects by extracting only the ones belonging to a given Class. For this purpose I wrote a very simple method like this:
def filterByClass[A](l: List[_], c: Class[A]) =
l filter (_.asInstanceOf[AnyRef].getClass() == c)
Note that I am obliged to add the explicit conversion to AnyRef in order to avoid this compilation problem:
error: type mismatch;
found : _$1 where type _$1
required: ?{val getClass(): ?}
Note that implicit conversions are not applicable because they are ambiguous:
both method any2stringadd in object Predef of type (x: Any)scala.runtime.StringAdd
and method any2ArrowAssoc in object Predef of type [A](x: A)ArrowAssoc[A]
are possible conversion functions from _$1 to ?{val getClass(): ?}
l filter (_.getClass() == c)
However in this way the invocation of:
filterByClass(l, classOf[String])
returns as expected:
List(One)
but of course the same doesn't work, for example, with Int since they extends Any but not AnyRef, so by invoking:
filterByClass(l, classOf[Int])
the result is just the empty List.
Is there a way to make my filterByClass method working even with Int, Boolean and all the other classes extending Any?
The collect method already does what you want. For example to collect all Ints in a collection you could write
xs collect { case x: Int => x }
This of course only works when you hardcode the type but as primitives are handled differently from reference types it is actually better to do so. You can make your life easier with some type classes:
case class Collect[A](collect: PartialFunction[Any,A])
object Collect {
implicit val collectInt: Collect[Int] = Collect[Int]({case x: Int => x})
// repeat for other primitives
// for types that extend AnyRef
implicit def collectAnyRef[A <: AnyRef](implicit mf: ClassManifest[A]) =
Collect[A]({ case x if mf.erasure.isInstance(x) => x.asInstanceOf[A] })
}
def collectInstance[A : Collect](xs: List[_ >: A]) =
xs.collect(implicitly[Collect[A]].collect)
Then you can use it without even passing a Class[A] instance:
scala> collectInstance[Int](l)
res5: List[Int] = List(1)
scala> collectInstance[String](l)
res6: List[String] = List(One)
Using isInstanceOf:
scala> val l = List(1, "One", 2)
l: List[Any] = List(1, One, 2)
scala> l . filter(_.isInstanceOf[String])
res1: List[Any] = List(One)
scala> l . filter(_.isInstanceOf[Int])
res2: List[Any] = List(1, 2)
edit:
As the OP requested, here's another version that moves the check in a method. I Couldn't find a way to use isInstanceOf and so I changed the implementation to use a ClassManifest:
def filterByClass[A](l: List[_])(implicit mf: ClassManifest[A]) =
l.filter(mf.erasure.isInstance(_))
Some usage scenarios:
scala> filterByClass[String](l)
res5: List[Any] = List(One)
scala> filterByClass[java.lang.Integer](l)
res6: List[Any] = List(1, 2)
scala> filterByClass[Int](l)
res7: List[Any] = List()
As can be seen above, this solution doesn't work with Scala's Int type.
The class of an element in a List[Any] is never classOf[Int], so this is behaving as expected. Your assumptions apparently leave this unexpected, but it's hard to give you a better way because the right way is "don't do that."
What do you think can be said about the classes of the members of a heterogenous list? Maybe this is illustrative. I'm curious how you think java does it better.
scala> def f[T: Manifest](xs: List[T]) = println(manifest[T] + ", " + manifest[T].erasure)
f: [T](xs: List[T])(implicit evidence$1: Manifest[T])Unit
scala> f(List(1))
Int, int
scala> f(List(1, true))
AnyVal, class java.lang.Object
scala> f(List(1, "One", true))
Any, class java.lang.Object
This worked for me. Is this what you want?
scala> val l = List(1, "One", true)
l: List[Any] = List(1, One, true)
scala> l filter { case x: String => true; case _ => false }
res0: List[Any] = List(One)
scala> l filter { case x: Int => true; case _ => false }
res1: List[Any] = List(1)
scala> l filter { case x: Boolean => true; case _ => false }
res2: List[Any] = List(true)
Despite my solution could be less elegant than this one I find mine quicker and easier. I just defined a method like this:
private def normalizeClass(c: Class[_]): Class[_] =
if (classOf[AnyRef].isAssignableFrom((c))) c
else if (c == classOf[Int]) classOf[java.lang.Integer]
// Add all other primitive types
else classOf[java.lang.Boolean]
So by using it in my former filterByClass method as it follows:
def filterByClass[A](l: List[_], c: Class[A]) =
l filter (normalizeClass(c).isInstance(_))
the invocation of:
filterByClass(List(1, "One", false), classOf[Int])
just returns
List(1)
as expected.
At the end, this problem reduces to find a map between a primitive and the corresponding boxed type.
Maybe a help can arrive from scala.reflect.Invocation (not included in the final version of 2.8.0), the getAnyValClass function in particular (here slightly edited)
def getAnyValClass(x: Any): java.lang.Class[_] = x match {
case _: Byte => classOf[Byte]
case _: Short => classOf[Short]
case _: Int => classOf[Int]
case _: Long => classOf[Long]
case _: Float => classOf[Float]
case _: Double => classOf[Double]
case _: Char => classOf[Char]
case _: Boolean => classOf[Boolean]
case _: Unit => classOf[Unit]
case x#_ => x.asInstanceOf[AnyRef].getClass
}
With this function the filter is as easy as
def filterByClass[T: Manifest](l:List[Any]) = {
l filter (getAnyValClass(_) == manifest[T].erasure)
}
and the invocation is:
filterByClass[Int](List(1,"one",true))