I broke Scala type safety somehow - scala

Can some explain to me how I was able to create a case class with the wrong type? Bellow the worksheet output:
case class OptObj(os: Option[String], s: String)
//defined class OptObj
val osCrazy: Option[Any] = Option(Option(Option(1)))
//osCrazy: Option[Any] = Some(Some(Some(1)))
def anyOpt: Option[Any] = osCrazy
//anyOpt: Option[Any]
def getOpt[T]: Option[T] = anyOpt.map(_.asInstanceOf[T])
//getOpt: [T]=> Option[T]
val o = OptObj(getOpt[String], "It has the wrong type inside, but was built!")
//o: OptObj = OptObj(Some(Some(Some(1))),It has the wrong type inside, but was built)
o == OptObj(None,"")
//res0: Boolean = false
o == o
//res1: Boolean = true
o.os
//res2: Option[String] = Some(Some(Some(1)))
o.os.get
//java.lang.ClassCastException: scala.Some cannot be cast to java.lang.String
The object is created with the wrong type inside, shouldn't object creation fail? It just fails when I try to make some operation on the os attribute.

The documentation for Any.asInstanceOf explains what you are seeing. The short of it is that the code compiles due to type erasure.
Note that the success of a cast at runtime is modulo Scala's erasure semantics. Therefore the expression 1.asInstanceOf[String] will throw a ClassCastException at runtime, while the expression List(1).asInstanceOf[List[String]] will not. In the latter example, because the type argument is erased as part of compilation it is not possible to check whether the contents of the list are of the requested type.

Related

Scala overload constructor with mutable List of different type

I am trying to overload a constructor with mutable list of Int and Long , it mentions that the method is already defined.
I need updateList to be either mutable.MutableList[Int] or mutable.MutableList[Long]
object PercentileDistribution {
def apply(updateList: mutable.MutableList[Int], percentileDistribution: PercentileDistribution): PercentileDistribution = {
updateList.foreach { x =>
percentileDistribution.update(x)
}
percentileDistribution
}
def apply(updateList: mutable.MutableList[Long], percentileDistribution: PercentileDistribution): PercentileDistribution = {
updateList.foreach { x =>
percentileDistribution.update(x)
}
percentileDistribution
}
}
Being new to scala I am facing some issues, any help is appreciated.
The error clearly refers to the upcasting that is happening in your code.
An Int can be represented as a Long hence you have essentially written the same method with one methods parameter being the upcasted version of the parameter of the other apply method.
You can simply use the apply method that has the MutableList[Long] type and remove the one with Int.
Follow this documentation from the official scala docs and you will get a good idea as to how types behave in Scala
The error refers to type erasure. The other answer incorrectly states that it is related to casting (although type erasure can lead to casting-related problems).
A quick example done in a Scala REPL session:
scala> class Foo {
| def bar(list: List[Int]) = "ints"
| def bar(list: List[String]) = "strings"
| }
<console>:12: error: double definition:
def bar(list: List[Int]): String at line 11 and
def bar(list: List[String]): String at line 12
have same type after erasure: (list: List)String
def bar(list: List[String]) = "strings"
^
The message here is saying that both bar methods will have a type signature like def bar(list: List): String in the compiled output; type erasure is taking away the [Int] and [String] parameters, making the two methods indistinguishable. It's an annoyance you just have to put up with if you're running code on the JVM.
My recommended workaround is to distinguish the methods by their names, e.g. instead of apply you might call it forInts and forLongs.
Also note that type erasure causes another problem:
scala> List(1,2,3).isInstanceOf[List[String]]
<console>:11: warning: fruitless type test: a value of type List[Int] cannot also be a List[String] (the underlying of List[String]) (but still might match its erasure)
List(1,2,3).isInstanceOf[List[String]]
^
res5: Boolean = true
and
scala> List(1, 2, 3) match {
| case l: List[String] => l // you'd think this shouldn't match, but it does
| }
<console>:12: warning: fruitless type test: a value of type List[Int] cannot also be a List[String] (the underlying of List[String]) (but still might match its erasure)
case l: List[String] => l // you'd think this shouldn't match, but it does
^
res2: List[Int] with List[String] = List(1, 2, 3)
scala> res2.head
java.lang.ClassCastException: java.lang.Integer cannot be cast to java.lang.String
... 33 elided
Since in the compiled bytecode, List[Int] and List[String] are both only represented as List, the runtime can't actually distinguish between the two, so an isInstanceOf check may provide the wrong answer, possibly leading to ClassCastExceptions as it tries to treat an Int as a String.
In your case you might get away with it, since it is safe to cast an Int to a Long, but it'd be best to avoid unchecked casting altogether.

Type erasure in a nested list with a given context bound

I am going through the book Scala with Cats. I am trying to understand the subtleties of the scala type system. I came up with the following example:
object Example extends App {
sealed trait Serializer[T] {
def serialize(seq: List[T]): String
}
implicit object StringSerializer extends Serializer[String] {
def serialize(seq: List[String]): String = seq.toString()
}
implicit object IntSerializer extends Serializer[Int] {
def serialize(seq: List[Int]): String = seq.toString()
}
def f1[T0 : Serializer](x: List[List[T0]])(implicit s0: Serializer[T0]): List[String] = {
x.map(lst => s0.serialize(lst))
}
// some dummy data
val col1 = List("a", "b", "c", "d", "e")
val col2 = List(12, 200, 80900, 201200, 124420000)
val col3 = List(121, 12121, 71240000, 44356, 845)
val data = List(col1, col2, col3)
f1(data)
}
Now this does not compile, the following error comes up:
could not find implicit value for evidence parameter of type
Example.Serializer[Any]
Now I understand why this happens; it is due to my function f1. Because I have a List[Int] and List[String] passed in to the function, the common parent type is Any. So the Type info gets erased, which passes into serializer.
However, given I have put a context bound, shouldn't the compiler first look for the implicit definitons before this takes place? Clearly it does not so my understanding is incorrect. What is the Scala way of getting round this problem.
Any explanations will be greatly appreciated!
The issue is that the inferred type of data is List[List[Any]], so that when you call f1, the type that's inferred for T0 is Any, which doesn't have a Serializer instance. Even if you don't define data as a val, and instead write something like f1(List(col1, col2, col3)), the inferred type of T0 will still be Any.
Scala just doesn't really provide any way that you can do the kind of thing you're aiming for. The closest solution is probably something like the magnet pattern—for example you could add something like this:
trait SerializableList {
type T
def values: List[T]
def instance: Serializer[T]
final def apply(): String = instance.serialize(values)
}
object SerializableList {
implicit def fromSerializer[T0](ts: List[T0])
(implicit T: Serializer[T0]): SerializableList =
new SerializableList {
type T = T0
val values = ts
val instance = T
}
}
And then define f1 like this:
def f1(x: List[SerializableList]): List[String] = {
x.map(_())
}
And this actually works for your case, provided that you pass an expression where the element types haven't been inferred yet:
scala> f1(List(col1, col2, col3))
res3: List[String] = List(List(a, b, c, d, e), List(12, 200, 80900, 201200, 124420000), List(121, 12121, 71240000, 44356, 845))
But if you try f1(data) it still won't work, since the static type of data is already List[List[Any]]:
scala> f1(data)
^
error: type mismatch;
found : List[List[Any]]
required: List[SerializableList]
In my view it isn't really a good idea to use implicit conversions like this, anyway, though, even if they're powered by a type class.
As a footnote, what you're seeing doesn't really have anything to do with type erasure, which in Scala and Java is about losing access to generic types in runtime reflection. For example, this is an example of how type erasure can enable unsafe programs in Scala:
def broken() = List(1, 2, 3) match { case xs: List[String] => xs.head }
This compiles (with a warning), but crashes with a ClassCastException at runtime.
It's at least arguable that type erasure is a good thing, since unerased types undermine parametricity, and that the only problem in Scala is that its type erasure isn't more complete. The only issue with broken, in this view, is the ability to match on runtime type at all—not the fact that it doesn't work for generic types.
In your case there's no runtime reflection, and the fact that you've lost specific type information when Any is inferred isn't erasure, at least in the sense that term is typically used in this context. Instead it's a matter of the least upper bound being taken.

Scala unexplainable program behavior

For the following code:
object Test {
class MapOps(map: Map[String, Any]) {
def getValue[T](name: String): Option[T] = {
map.get(name).map{_.asInstanceOf[T]}
}
}
implicit def toMapOps(map: Map[String, Any]): MapOps = new MapOps(map)
def main(args: Array[String]): Unit = {
val m: Map[String, Any] = Map("1" -> 1, "2" -> "two")
val a = m.getValue[Int]("2").get.toString
println(s"1: $a")
val b = m.getValue[Int]("2").get
println(s"2: $b")
}
}
val a is computed without exception and the console prints 1: two,
but when computing val b, the java.lang.ClassCastException: java.lang.String cannot be cast to java.lang.Integer is thrown.
Besides, if I execute
val c = m.getValue[Int]("2").get.getClass.toString
println(s"1: $c")
The console prints "int".
Can someone explain why this code behaves like this?
This is certainly odd.
If you look at the following statement in the Scala REPL:
scala> val x = m.getValue[Int]("2")
x: Option[Int] = Some(two)
What I think is happening is this: the asInstanceOf[T] statement is simply flagging to the compiler that the result should be an Int, but no cast is required, because the object is still just referenced via a pointer. (And Int values are boxed inside of an Option/Some) .toString works because every object has a .toString method, which just operates on the value "two" to yield "two". However, when you attempt to assign the result to an Int variable, the compiler attempts to unbox the stored integer, and the result is a cast exception, because the value is a String and not a boxed Int.
Let's verify this step-by-step in the REPL:
$ scala
Welcome to Scala 2.12.1 (Java HotSpot(TM) 64-Bit Server VM, Java 1.8.0_151).
Type in expressions for evaluation. Or try :help.
scala> class MapOps(map: Map[String, Any]) {
| def getValue[T](name: String): Option[T] = {
| map.get(name).map{_.asInstanceOf[T]}
| }
| }
defined class MapOps
scala> import scala.language.implicitConversions
import scala.language.implicitConversions
scala> implicit def toMapOps(map: Map[String, Any]): MapOps = new MapOps(map)
toMapOps: (map: Map[String,Any])MapOps
scala> val a = m.getValue[Int]("2").get.toString
a: String = two
scala> println(s"1: $a")
1: two
So far so good. Note that no exceptions have been thrown so far, even though we have already used .asInstanceOf[T] and used get on the resulting value. What's significant is that we haven't attempted to do anything with the result of the get call (nominally a boxed Int that is actually the String value "two") except to invoke it's toString method. That works, because String values have toString methods.
Now let's perform the assignment to an Int variable:
scala> val b = m.getValue[Int]("2").get
java.lang.ClassCastException: java.lang.String cannot be cast to java.lang.Integer
at scala.runtime.BoxesRunTime.unboxToInt(BoxesRunTime.java:101)
... 29 elided
Now we get the exception! Note also the function in the stack trace that caused it: unboxToInt - it's clearly trying to convert the value stored in the Some to an Int and it fails because it's not a boxed Int but a String.
A big part of the problem is type erasure. Don't forget that a Some(Banana) and a Some(Bicycle) are - at runtime - both just Some instances with a pointer to some object. .asInstanceOf[T] cannot verify the type, because that information has been erased. However, the compiler is able to track what the type should be based upon what you've told it, but it can only detect the error when its assumptions are proven wrong.
Finally, with regard to the getClass call on the result. This is a bit of compiler sleight-of-hand. It's not actually calling a getClass function on the object, but - because it thinks it's dealing with an Int, which is a primitive - it simply substitutes an int class instance.
scala> m.getValue[Int]("2").get.getClass
res0: Class[Int] = int
To verify that the object actually is a String, you can cast it to an Any as follows:
scala> m.getValue[Int]("2").get.asInstanceOf[Any].getClass
res1: Class[_] = class java.lang.String
Further verification about the return value of get follows; note the lack of an exception when we assign the result of this method to a variable of type Any (so no casting is necessary), the fact that the valid Int with key "1" is actually stored under Any as a boxed Int (java.lang.Integer), and that this latter value can be successfully unboxed to a regular Int primitive:
scala> val x: Any = m.getValue[Int]("2").get
x: Any = two
scala> x.getClass
res2: Class[_] = class java.lang.String
scala> val y: Any = m.getValue[Int]("1").get
y: Any = 1
scala> y.getClass
res3: Class[_] = class java.lang.Integer
scala> val z = m.getValue[Int]("1").get
z: Int = 1
scala> z.getClass
res4: Class[Int] = int
Expanding on the last part of eje211's answer.
You told the compiler that a String was an Int, and now you're looking for a sensible explanation of the resulting behavior. That's understandable, but it's not really useful. Once you tell the compiler a lie, all bets are off. You can spend time investigating exactly when and where the compiler inserts checks that happen to discover your deceit, but your time would probably be better spent writing code that doesn't cause you to lie.
As the earlier answer pointed out, you can do that (avoid accidental lying) by using pattern matching. You'll need a ClassTag to make pattern matching work in cases like the above, but the end result will be code that is type-safe and correct.
It's an Int because you request an Int on this line :
val b = m.getValue[Int]("2").get
This calls this method:
def getValue[T](name: String): Option[T] = {
map.get(name).map{_.asInstanceOf[T]}
}
and applies it this way:
def getValue[Int](name: String): Option[Int] = {
map.get(name).map{_.asInstanceOf[Int]}
}
So if you ask for an Int, you get an Int.
In the case of "two", this is what happens in that case:
"two".asInstanceOf[Int]
That's what throws your exception.
A String is not an Int. You can't cast it that way. You can cast it this way:
"2".toInt
But that's different.
In general, using asInstanceOf[] is dangerous. Try pattern matching instead. If you must use, it's up you to make sure that the casts you attempt are valid. You're basically telling the compiler to bypass its own type-checks, particularly when you cast from Any.
It works when you add .toString because then, you change the type back to String, which is what it really was in the first place. The lie of what type the data was is corrected.

Scala asInstanceOf generic type does not fail in Try

Trying to cast a String to a Double obviously should fail:
scala> Try("abc".asInstanceOf[Double])
res11: scala.util.Try[Double] = Failure(java.lang.ClassCastException: java.lang.String cannot be cast to java.lang.Double)
However, if I define the above as a function:
scala> def convertAsTypeOf[T](anyValue: Any): Try[T] = Try(anyValue.asInstanceOf[T])
convertAsTypeOf: [T](anyValue: Any)scala.util.Try[T]
Strangely it returns Success when I try to cast a String to a Double:
scala> convertAsTypeOf[Double]("abc")
res10: scala.util.Try[Double] = Success(abc)
If I try to get the value out of Success, I get the following exception:
scala> convertAsTypeOf[Double]("abc").get
java.lang.ClassCastException: java.lang.String cannot be cast to java.lang.Double
at scala.runtime.BoxesRunTime.unboxToDouble(BoxesRunTime.java:119)
Why does this happen? Is there any way to fix this and make asInstanceOf[T] when it's in a generic function?
Because of type erasure, the type T is unknown at runtime, so asInstanceOf can't actually check the cast that it does. In fact, it compiles as a no-op. However, when the .get is finally done a cast is made to Double, and because the type is known here we can get a ClassCastException.
If you change convertAsTypeOf to use ClassTag:
def convertAsTypeOf[T](anyValue: Any)(implicit tag: ClassTag[T]): Try[T] =
Try(tag.runtimeClass.cast(anyValue).asInstanceOf[T])
You will then get errors when you expect:
scala> convertAsTypeOf[Double]("abc")
res1: scala.util.Try[Double] = Failure(java.lang.ClassCastException: Cannot cast java.lang.String to double)
The ClassTag represents the type T at runtime, allowing the value to be checked against it.
The reason this happens is type erasure, which makes asInstanceOf[T] for generic T useless. I don't know why the compiler doesn't issue any warning.
You can force the check to happen by using a ClassTag. Something like this:
import reflect.ClassTag
def convertAsTypeOf[T : ClassTag](anyValue: Any): Try[T] = Try {
anyValue match { case t: T => t }
}
This works because the compiler will use the implicit ClassTag in the case match if it is available (and generate a warning if it is not).

Why won't Scala use implicit conversion here?

I'm trying to call this set method documented here, in the Java library jOOQ, with signature:
<T> ... set(Field<T> field, T value)
This Scala line is a problem:
.set(table.MODIFIED_BY, userId)
MODIFIED_BY is a Field<Integer> representing the table column. userId is Int. Predef has an implicit conversion from Int to Integer, so why doesn't it use it? I get this:
type mismatch; found: org.jooq.TableField[gen.tables.records.DocRecord,Integer]
required: org.jooq.Field[Any]
Note: Integer <: Any
(and org.jooq.TableField[gen.tables.records.DocRecord,Integer] <:
org.jooq.Field[Integer]), but Java-defined trait Field is invariant in type T.
You may wish to investigate a wildcard type such as `_ <: Any`. (SLS 3.2.10)
Update - About Vinicius's Example
Rather than try to explain this in comments, here is a demonstration that there is no implicit conversion being called when you use a type with covariant parameter, like List[+T]. Let's say I put this code in a file, compile, and run it...
case class Foo(str: String)
object StackOver1 extends App {
implicit def str2Foo(s: String): Foo = {
println("In str2Foo.")
new Foo(s)
}
def test[T](xs: List[T], x: T): List[T] = {
println("test " + x.getClass)
xs
}
val foo1 = new Foo("foo1")
test(List(foo1), "abc")
}
You'll see that it calls test, but never the implicit conversion from String "abc" to Foo. Instead it's picking a T for test[T] that is a common base class between String and Foo. When you use Int and Integer it picks Any, but it's confusing because the runtime representation of the Int in the list is Integer. So it looks like it used the implicit conversion, but it didn't. You can verify by opening a Scala prompt...
scala> :type StackOver1.test(List(new java.lang.Integer(1)), 2)
List[Any]
I don't know anything aboutjOOQ, but I think the issue is that Scala does not understand java generics very well. Try:
scala> def test[T](a : java.util.ArrayList[T], b: T) = { println(a,b) }
scala> val a = new java.util.ArrayList[Integer]()
scala> val b = 12
scala> test(a,b)
<console>:11: error: type mismatch;
found : java.util.ArrayList[Integer]
required: java.util.ArrayList[Any]
Note: Integer <: Any, but Java-defined class ArrayList is invariant in type E.
You may wish to investigate a wildcard type such as `_ <: Any`. (SLS 3.2.10)
test(a,b)
Sounds familiar??
And to fix, just inform the type T to call the method: test[Integer](a,b) works fine.
EDIT:
There a few things involved here:
Erasure -> When compiled the type of the generic will disappear by erasure. The compiler will use Object which Scala, will treat as Any. However a ArrayList[Integer] is not an ArrayList[Any], even though Integer is any. The same way that TableField[gen.tables.records.DocRecord,Integer] is not a Field[Any].
Type inference mechanism -> it will figure out what type T should be and to do that it will use the intersection dominator of the types passed (in our case the first common ancestor). Page 36 of Scala Language Spec, which in our examples above will lead use to Any.
Implicit conversion -> it is the last step and would be called if there was some type to be converted to another one, but since the type of the arguments were determined to be the first common ancestor, there is no need to convert and we will never have a implicit conversion if we don't force the type T.
A example to show how the common ancestor is used to determine T:
scala> def test[T](a: T, b: T): T = a
scala> class Foo
scala> class Boo extends Foo
scala> test(new Boo,new Foo)
res2: Foo = Boo#139c2a6
scala> test(new Boo,new Boo)
res3: Boo = Boo#141c803
scala> class Coo extends Foo
scala> test(new Boo,new Coo)
res4: Foo = Boo#aafc83
scala> test(new Boo,"qsasad")
res5: Object = Boo#16989d8
Summing up, the implicit method does not get called, because the type inference mechanism, determines the types before getting the argument and since it uses the common ancestor, there is no need for a implicit conversion.
Your code produces an error due to erasure mechanism which disappear with the type information that would be important to determine the correct type of the argument.
#RobN, thanks for questioning my answer, I learned a lot with the process.