The intended output of the following code is the digit, 0.
object Bad extends App {
implicit val massTable: Map[String, Int] =
Map("H" -> 1, "He" -> 4, "O" -> 16)
Implementation.doWork()
}
object Implementation {
def doWork()(implicit massTable: Map[String, Int]) = println("0".toInt)
}
The actual output is an exception:
java.util.NoSuchElementException: key not found: 0
at scala.collection.immutable.Map$Map3.apply(Map.scala:156)
at Implementation$.doWork(Main.scala:20)
at Bad$.delayedEndpoint$Bad$1(Main.scala:16)
...
Investigation shows that Map implements apply(string): Int causing the implicit parameter supplied to doWork, massTable, to also and unintentionally function as an implicit conversion from String => Int.
The result is that, the above toInt call is the synthetic method toInt for the type Int, rather than the toInt method provided by StringOps.
What is the best way to avoid this sort of issue and achieve the desired output?
Related
I am going through the book Scala with Cats. I am trying to understand the subtleties of the scala type system. I came up with the following example:
object Example extends App {
sealed trait Serializer[T] {
def serialize(seq: List[T]): String
}
implicit object StringSerializer extends Serializer[String] {
def serialize(seq: List[String]): String = seq.toString()
}
implicit object IntSerializer extends Serializer[Int] {
def serialize(seq: List[Int]): String = seq.toString()
}
def f1[T0 : Serializer](x: List[List[T0]])(implicit s0: Serializer[T0]): List[String] = {
x.map(lst => s0.serialize(lst))
}
// some dummy data
val col1 = List("a", "b", "c", "d", "e")
val col2 = List(12, 200, 80900, 201200, 124420000)
val col3 = List(121, 12121, 71240000, 44356, 845)
val data = List(col1, col2, col3)
f1(data)
}
Now this does not compile, the following error comes up:
could not find implicit value for evidence parameter of type
Example.Serializer[Any]
Now I understand why this happens; it is due to my function f1. Because I have a List[Int] and List[String] passed in to the function, the common parent type is Any. So the Type info gets erased, which passes into serializer.
However, given I have put a context bound, shouldn't the compiler first look for the implicit definitons before this takes place? Clearly it does not so my understanding is incorrect. What is the Scala way of getting round this problem.
Any explanations will be greatly appreciated!
The issue is that the inferred type of data is List[List[Any]], so that when you call f1, the type that's inferred for T0 is Any, which doesn't have a Serializer instance. Even if you don't define data as a val, and instead write something like f1(List(col1, col2, col3)), the inferred type of T0 will still be Any.
Scala just doesn't really provide any way that you can do the kind of thing you're aiming for. The closest solution is probably something like the magnet pattern—for example you could add something like this:
trait SerializableList {
type T
def values: List[T]
def instance: Serializer[T]
final def apply(): String = instance.serialize(values)
}
object SerializableList {
implicit def fromSerializer[T0](ts: List[T0])
(implicit T: Serializer[T0]): SerializableList =
new SerializableList {
type T = T0
val values = ts
val instance = T
}
}
And then define f1 like this:
def f1(x: List[SerializableList]): List[String] = {
x.map(_())
}
And this actually works for your case, provided that you pass an expression where the element types haven't been inferred yet:
scala> f1(List(col1, col2, col3))
res3: List[String] = List(List(a, b, c, d, e), List(12, 200, 80900, 201200, 124420000), List(121, 12121, 71240000, 44356, 845))
But if you try f1(data) it still won't work, since the static type of data is already List[List[Any]]:
scala> f1(data)
^
error: type mismatch;
found : List[List[Any]]
required: List[SerializableList]
In my view it isn't really a good idea to use implicit conversions like this, anyway, though, even if they're powered by a type class.
As a footnote, what you're seeing doesn't really have anything to do with type erasure, which in Scala and Java is about losing access to generic types in runtime reflection. For example, this is an example of how type erasure can enable unsafe programs in Scala:
def broken() = List(1, 2, 3) match { case xs: List[String] => xs.head }
This compiles (with a warning), but crashes with a ClassCastException at runtime.
It's at least arguable that type erasure is a good thing, since unerased types undermine parametricity, and that the only problem in Scala is that its type erasure isn't more complete. The only issue with broken, in this view, is the ability to match on runtime type at all—not the fact that it doesn't work for generic types.
In your case there's no runtime reflection, and the fact that you've lost specific type information when Any is inferred isn't erasure, at least in the sense that term is typically used in this context. Instead it's a matter of the least upper bound being taken.
For the following code:
object Test {
class MapOps(map: Map[String, Any]) {
def getValue[T](name: String): Option[T] = {
map.get(name).map{_.asInstanceOf[T]}
}
}
implicit def toMapOps(map: Map[String, Any]): MapOps = new MapOps(map)
def main(args: Array[String]): Unit = {
val m: Map[String, Any] = Map("1" -> 1, "2" -> "two")
val a = m.getValue[Int]("2").get.toString
println(s"1: $a")
val b = m.getValue[Int]("2").get
println(s"2: $b")
}
}
val a is computed without exception and the console prints 1: two,
but when computing val b, the java.lang.ClassCastException: java.lang.String cannot be cast to java.lang.Integer is thrown.
Besides, if I execute
val c = m.getValue[Int]("2").get.getClass.toString
println(s"1: $c")
The console prints "int".
Can someone explain why this code behaves like this?
This is certainly odd.
If you look at the following statement in the Scala REPL:
scala> val x = m.getValue[Int]("2")
x: Option[Int] = Some(two)
What I think is happening is this: the asInstanceOf[T] statement is simply flagging to the compiler that the result should be an Int, but no cast is required, because the object is still just referenced via a pointer. (And Int values are boxed inside of an Option/Some) .toString works because every object has a .toString method, which just operates on the value "two" to yield "two". However, when you attempt to assign the result to an Int variable, the compiler attempts to unbox the stored integer, and the result is a cast exception, because the value is a String and not a boxed Int.
Let's verify this step-by-step in the REPL:
$ scala
Welcome to Scala 2.12.1 (Java HotSpot(TM) 64-Bit Server VM, Java 1.8.0_151).
Type in expressions for evaluation. Or try :help.
scala> class MapOps(map: Map[String, Any]) {
| def getValue[T](name: String): Option[T] = {
| map.get(name).map{_.asInstanceOf[T]}
| }
| }
defined class MapOps
scala> import scala.language.implicitConversions
import scala.language.implicitConversions
scala> implicit def toMapOps(map: Map[String, Any]): MapOps = new MapOps(map)
toMapOps: (map: Map[String,Any])MapOps
scala> val a = m.getValue[Int]("2").get.toString
a: String = two
scala> println(s"1: $a")
1: two
So far so good. Note that no exceptions have been thrown so far, even though we have already used .asInstanceOf[T] and used get on the resulting value. What's significant is that we haven't attempted to do anything with the result of the get call (nominally a boxed Int that is actually the String value "two") except to invoke it's toString method. That works, because String values have toString methods.
Now let's perform the assignment to an Int variable:
scala> val b = m.getValue[Int]("2").get
java.lang.ClassCastException: java.lang.String cannot be cast to java.lang.Integer
at scala.runtime.BoxesRunTime.unboxToInt(BoxesRunTime.java:101)
... 29 elided
Now we get the exception! Note also the function in the stack trace that caused it: unboxToInt - it's clearly trying to convert the value stored in the Some to an Int and it fails because it's not a boxed Int but a String.
A big part of the problem is type erasure. Don't forget that a Some(Banana) and a Some(Bicycle) are - at runtime - both just Some instances with a pointer to some object. .asInstanceOf[T] cannot verify the type, because that information has been erased. However, the compiler is able to track what the type should be based upon what you've told it, but it can only detect the error when its assumptions are proven wrong.
Finally, with regard to the getClass call on the result. This is a bit of compiler sleight-of-hand. It's not actually calling a getClass function on the object, but - because it thinks it's dealing with an Int, which is a primitive - it simply substitutes an int class instance.
scala> m.getValue[Int]("2").get.getClass
res0: Class[Int] = int
To verify that the object actually is a String, you can cast it to an Any as follows:
scala> m.getValue[Int]("2").get.asInstanceOf[Any].getClass
res1: Class[_] = class java.lang.String
Further verification about the return value of get follows; note the lack of an exception when we assign the result of this method to a variable of type Any (so no casting is necessary), the fact that the valid Int with key "1" is actually stored under Any as a boxed Int (java.lang.Integer), and that this latter value can be successfully unboxed to a regular Int primitive:
scala> val x: Any = m.getValue[Int]("2").get
x: Any = two
scala> x.getClass
res2: Class[_] = class java.lang.String
scala> val y: Any = m.getValue[Int]("1").get
y: Any = 1
scala> y.getClass
res3: Class[_] = class java.lang.Integer
scala> val z = m.getValue[Int]("1").get
z: Int = 1
scala> z.getClass
res4: Class[Int] = int
Expanding on the last part of eje211's answer.
You told the compiler that a String was an Int, and now you're looking for a sensible explanation of the resulting behavior. That's understandable, but it's not really useful. Once you tell the compiler a lie, all bets are off. You can spend time investigating exactly when and where the compiler inserts checks that happen to discover your deceit, but your time would probably be better spent writing code that doesn't cause you to lie.
As the earlier answer pointed out, you can do that (avoid accidental lying) by using pattern matching. You'll need a ClassTag to make pattern matching work in cases like the above, but the end result will be code that is type-safe and correct.
It's an Int because you request an Int on this line :
val b = m.getValue[Int]("2").get
This calls this method:
def getValue[T](name: String): Option[T] = {
map.get(name).map{_.asInstanceOf[T]}
}
and applies it this way:
def getValue[Int](name: String): Option[Int] = {
map.get(name).map{_.asInstanceOf[Int]}
}
So if you ask for an Int, you get an Int.
In the case of "two", this is what happens in that case:
"two".asInstanceOf[Int]
That's what throws your exception.
A String is not an Int. You can't cast it that way. You can cast it this way:
"2".toInt
But that's different.
In general, using asInstanceOf[] is dangerous. Try pattern matching instead. If you must use, it's up you to make sure that the casts you attempt are valid. You're basically telling the compiler to bypass its own type-checks, particularly when you cast from Any.
It works when you add .toString because then, you change the type back to String, which is what it really was in the first place. The lie of what type the data was is corrected.
Here is a toy example:
object Example {
import collection.mutable
import collection.immutable.TreeSet
val x = TreeSet(1, 5, 8, 12)
val y = mutable.Set.empty ++= x
val z = TreeSet.empty ++ y
// This gives an error: unspecified parameter
// val z = TreeSet.empty() ++ y
}
Apparently TreeSet.empty and TreeSet.empty() are not the same thing. What's going on under the hood? When can I safely omit (or not omit in this case) the parentheses?
Update
I have sent some code to the console and then deleted it in intellij before eval the above code, here it is:
implicit object StringOrdering extends Ordering[String] {
def compare(o1: String, o2: String) = {
o1.length - o2.length
}
}
object StringOrdering1 extends Ordering[String] {
def compare(o1: String, o2: String) = {
o2.length - o1.length
}
}
This is a special case, and isn't quite relevant to when you can and cannot omit parentheses.
This is the signature for TreeSet.empty:
def empty[A](implicit ordering: Ordering[A]): TreeSet[A]
It has an implicit parameter list that requires an Ordering for the contained type A. When you call TreeSet.empty, the compiler will try to implicitly find the correct Ordering[A].
But when you call TreeSet.empty(), the compiler thinks you are trying to explicitly provide the implicit parameter. Except you leave out the parameter in the list, which is a compile error (wrong number of arguments). The only way this will work is if you explicitly pass some Ordering: TreeSet.empty(Ordering.Int).
Side note: Your above code does not actually compile with TreeSet.empty, because it succumbs to an ambiguous implicit error for Ordering. There is probably some implicit Ordering[Int] in your scope that you are not including in the question. It would be better to make the type explicit and use TreeSet.empty[Int].
I have a map of String to Functions which details all of the valid functions that are in a language. When I add a function to my map, I am required to specify the type (in this case Int).
var functionMap: Map[String, (Nothing) => Any] = Map[String, (Nothing) => Any]()
functionMap += ("Neg" -> expr_neg[Int])
def expr_neg[T: Numeric](value: T)(implicit n: Numeric[T]): T = {
n.negate(value)
}
Instead, how can I do something like:
functionMap += ("Neg" -> expr_neg)
without the [Int] and add it in later on when I call:
(unaryFunctionMap.get("abs").get)[Int](-45)
You're trying to build your function using type classes (in this case, Numeric). Type classes rely on implicit parameters. Implicits are resolved at compile time. Your function name string values are only known at runtime, therefore you shouldn't build your solution on top of type classes like this.
An alternative would be to store a separate function object in your map for each parameter type. You could store the parameter type with a TypeTag:
import scala.reflect.runtime.universe._
var functionMap: Map[(String, TypeTag[_]), (Nothing) => Any] = Map()
def addFn[T: TypeTag](name: String, f: T => Any) =
functionMap += ((name, typeTag[T]) -> f)
def callFn[T: TypeTag](name: String, value: T): Any =
functionMap((name, typeTag[T])).asInstanceOf[T => Any](value)
addFn[Int]("Neg", expr_neg)
addFn[Long]("Neg", expr_neg)
addFn[Double]("Neg", expr_neg)
val neg10 = callFn("Neg", 10)
No type class implicit needs to be resolved to call callFn(), because the implicit Numeric was already resolved on the call to addFn.
What happens if we try to resolve the type class when the function is called?
The first problem is that a Function1 (or Function2) can't have implicit parameters. Only a method can. (See this other question for more explanation.) So if you want something that acts like a Function1 but takes an implicit parameter, you'll need to create your own type that defines the apply() method. It has to be a different type from Function1, though.
Now we get to the main problem: all implicits must be able to be resolved at compile time. At the location in code where the method is run, all the type information needed to choose the implicit value needs to be available. In the following code example:
unaryFunctionMap("abs")(-45)
We don't really need to specify that our value type is Int, because it can be inferred from the value -45 itself. But the fact that our method uses a Numeric implicit value can't be inferred from anything in that line of code. We need to specify the use of Numeric somewhere at compile time.
If you can have a separate map for unary functions that take a numeric value, this is (relatively) easy:
trait UnaryNumericFn {
def apply[T](value: T)(implicit n: Numeric[T]): Any
}
var unaryNumericFnMap: Map[String, UnaryNumericFn] = Map()
object expr_neg extends UnaryNumericFn {
override def apply[T](value: T)(implicit n: Numeric[T]): T = n.negate(value)
}
unaryNumericFnMap += ("Neg" -> expr_neg)
val neg3 = unaryNumericFnMap("Neg")(3)
You can make the function trait generic on the type class it requires, letting your map hold unary functions that use different type classes. This requires a cast internally, and moves the specification of Numeric to where the function is finally called:
trait UnaryFn[-E[X]] {
def apply[T](value: T)(implicit ev: E[T]): Any
}
object expr_neg extends UnaryFn[Numeric] {
override def apply[T](value: T)(implicit n: Numeric[T]): T = n.negate(value)
}
var privateMap: Map[String, UnaryFn[Nothing]] = Map()
def putUnary[E[X]](key: String, value: UnaryFn[E]): Unit =
privateMap += (key -> value)
def getUnary[E[X]](key: String): UnaryFn[E] =
privateMap(key).asInstanceOf[UnaryFn[E]]
putUnary("Neg", expr_neg)
val pos5 = getUnary[Numeric]("Neg")(-5)
But you still have to specify Numeric somewhere.
Also, neither of these solutions, as written, support functions that don't need type classes. Being forced to be this explicit about which functions take implicit parameters, and what kinds of implicits they use, starts to defeat the purpose of using implicits in the first place.
You can't. Because expr_neg is a method with a type parameter T and an implicit argument n depending on that parameter. For Scala to lift that method to a function, it needs to capture the implicit, and therefore it must know what kind of type you want.
I am struggling with a Scala implicit conversion problem. The following code snippet illustrates my problem :
import org.junit.{ Test, Before, After };
class ImplicitsTest {
implicit def toStringWrapper(str: String) = new StringWrapper(str);
#Test
def test(){
val res1: Predicate = "str" startsWith "other";
}
}
class StringWrapper(str: String){
def startsWith(other: String): Predicate = null;
}
trait Predicate
How can I force the String literal "str" to be converted through the implicit conversion toStringWrapper to get startsWith return Predicate instead of Boolean?
The code example doesn't compile. I am aware that String has already a startsWith method, I just want to use a different one, and I thought that using implicit conversions might be a way to do it.
Scala thankfully doesn't let you sneak replacement methods in without you noticing--if you call a method on a class, and the class has that method, that's the method call you get. To do otherwise would likely cause all sorts of confusion.
That leaves you with two other options:
(1) Rename the method
(2) Add a specific method to do the conversion.
The second approach works like so: you define a class that has both the method that you want and a uniquely-named method that returns itself, and optionally an implicit conversion from that class back to string if you want to be able to use the custom item like the original string (as if it had extended String):
object ImplicitExample {
class CustomString(s: String) {
def original = s
def custom = this
def startsWith(other: String): Int = if (s.startsWith(other)) 1 else 0
}
implicit def string_to_custom(s: String) = new CustomString(s)
implicit def custom_to_string(c: CustomString) = c.original
def test = {
println("This".custom.startsWith("Thi"))
println("This".custom.length())
}
}
scala> ImplicitExample.test
1
4
scala>
An implicit conversion is triggered in Scala only if the receiver does not contain the method being invoked or if an expression has a type different than the expected type.
Since the String object above contains the method startsWith, no implicit conversion is triggered. The compiler does, however, check if the type of the right hand expression "str".startsWith("other"); (that is Boolean) can be converted to a Predicate. Since there is no such implicit conversion in scope, it reports an error.
Note, also, that implicit conversions must be unambiguous. For instance, you might try to override Scala string behaviour for some other methods using implicit conversions. Java String objects are not Scala sequences (Seq), meaning that they do not have methods such as sorted, which returns the sorted version of the sequence. If you invoke it on a string:
scala> "string" sorted
res1: String = ginrst
This works because an implicit conversion defined in the Predef object gets triggered. Note that providing your own implicit conversion results in an error:
scala> implicit def wrap(s: String) = new { def sorted = "Hi!" }
wrap: (s: String)java.lang.Object{def sorted: java.lang.String}
scala> "string" sorted
<console>:7: error: type mismatch;
found : java.lang.String
required: ?{val sorted: ?}
Note that implicit conversions are not applicable because they are ambiguous:
...