Scala ClassCastException on Option.orNull - scala

When I try to run following code:
def config[T](key: String): Option[T] = {
//in reality this is a map of various instance types as values
Some("string".asInstanceOf[T])
}
config("path").orNull
I'm getting error:
java.lang.String cannot be cast to scala.runtime.Null$
java.lang.ClassCastException
Following attempts are working fine:
config[String]("path").orNull
config("path").getOrElse("")
Since getOrElse works its confusing why null is so special and throws an error. Is there a way for orNull to work without specifying generic type ?
scalaVersion := "2.12.8"

Just to show how you may avoid the use asInstanceOf to get the values from a typed config.
sealed trait Value extends Product with Serializable
final case class IntValue(value: Int) extends Value
final case class StringValue(value: String) extends Value
final case class BooleanValue(value: Boolean) extends Value
type Config = Map[String, Value]
sealed trait ValueExtractor[T] {
def extract(config: Config)(fieldName: String): Option[T]
}
object ValueExtractor {
implicit final val IntExtractor: ValueExtractor[Int] =
new ValueExtractor[Int] {
override def extract(config: Config)(fieldName: String): Option[Int] =
config.get(fieldName).collect {
case IntValue(value) => value
}
}
implicit final val StringExtractor: ValueExtractor[String] =
new ValueExtractor[String] {
override def extract(config: Config)(fieldName: String): Option[String] =
config.get(fieldName).collect {
case StringValue(value) => value
}
}
implicit final val BooleanExtractor: ValueExtractor[Boolean] =
new ValueExtractor[Boolean] {
override def extract(config: Config)(fieldName: String): Option[Boolean] =
config.get(fieldName).collect {
case BooleanValue(value) => value
}
}
}
implicit class ConfigOps(val config: Config) extends AnyVal {
def getAs[T](fieldName: String)(default: => T)
(implicit extractor: ValueExtractor[T]): T =
extractor.extract(config)(fieldName).getOrElse(default)
}
Then, you can use it like this.
val config = Map("a" -> IntValue(10), "b" -> StringValue("Hey"), "d" -> BooleanValue(true))
config.getAs[Int](fieldName = "a")(default = 0) // res: Int = 10
config.getAs[Int](fieldName = "b")(default = 0) // res: Int = 0
config.getAs[Boolean](fieldName = "c")(default = false) // res: Boolean = false
Now, the problem becomes how to create the typed config from a raw source.
And even better, how to directly map the config to a case class.
But, those are more complex, and probably is better to just use something already done, like pureconfig.
Just as an academic exercise, lets see if we can support Lists & Maps.
Lets start with lists, a naive approach would be to have another case class for values which are lists, and create a factory of extractors for every kind of list (this process is formally know as implicit derivation).
import scala.reflect.ClassTag
final case class ListValue[T](value: List[T]) extends Value
...
// Note that, it has to be a def, since it is not only one implicit.
// But, rather a factory of implicits.
// Also note that, it needs another implicit parameter to construct the specific implicit.
// In this case, it needs a ClasTag for the inner type of the list to extract.
implicit final def listExtractor[T: ClassTag]: ValueExtractor[List[T]] =
new ValueExtractor[List[T]] {
override def extract(config: Config)(fieldName: String): Option[List[T]] =
config.get(fieldName).collect {
case ListValue(value) => value.collect {
// This works as a safe caster, which will remove all value that couldn't been casted.
case t: T => t
}
}
}
Now, you can use it like this.
val config = Map("l" ->ListValue(List(1, 2, 3)))
config.getAs[List[Int]](fieldName = "l")(default = List.empty)
// res: List[Int] = List(1, 2, 3)
config.getAs[List[String]](fieldName = "l")(default = List("Hey"))
// res: String = List() - The default is not used, since the field is a List...
// whose no element could be casted to String.
However, this approach is limited to plain types, if you need a List of other generic type, like a List of Lists. Then, this won't work.
val config = Map("l" ->ListValue(List(List(1, 2), List(3))))
val l = config.getAs[List[List[String]]](fieldName = "l")(default = List.empty)
// l: List[List[String]] = List(List(1, 2), List(3)) ???!!!
l.head
// res: List[String] = List(1, 2)
l.head.head
// java.lang.ClassCastException: java.lang.Integer cannot be cast to java.lang.String
The problem here is type erasure, which ClassTags can not solve, you may try to use TypeTags which can preserve the complete type, but the solution becomes more cumbersome.
For Maps the solution is quite similar, especially if you fix the key type to String (assuming what you really want is a nested config). But, this post is too long now, so I would leave it as an exercise for the reader.
Nevertheless, as already said, this can be broken easily, and is not completely robust.
There are better approaches, but I myself am not very skilled on those (yet), and even if I would be, the answer would be more long and really not necessary at all.
Lucky for you, even if pureconfig does not support YAML directly, there is a module which does, pureconfig-yaml.
I would suggest you to take a look to the module, and if you have further problems ask a new question tagging pureconfig and yaml directly. Also, if it is just a small doubt, you may try asking in thegitter channel.

Related

Can I avoid using structural typing in this scenario?

I have some code that uses both a third-party library and my own library. In my own library, I don't want to have a dependency on the third-party one so I want one of my methods to accept a more generic type as a parameter. Unfortunately, I cannot extend or mixin a trait to the 3rd party classes since they are generated using factory methods & the classes are final.
I can get around this issue by using structural typing but I was wondering if there is an alternative? I don't want to have to iterate through each record returned by the factory method and "new up" instances of a separate type if possible.
I've boiled it down to a scenario like the following:
Third-party library code that cannot be changed
// Class inside library cannot be extended due to it being 'final'
final class SpecificRecord(val values: IndexedSeq[String]) {
def get(i: Int): String = {
values(i)
}
}
// A companion object simply to create some sample data in an iterator
object SpecificRecord{
def generateSpecificRecords(): Iterator[SpecificRecord] = new Iterator[SpecificRecord] {
var pointerLocation: Int = 0
private val state = IndexedSeq(
IndexedSeq("Row1 Col1", "Row1 Col2", "Row 1 Col3"),
IndexedSeq("Row2 Col1", "Row2 Col2", "Row 2 Col3")
)
override def hasNext: Boolean = {
if (pointerLocation < state.length) true else false
}
override def next(): SpecificRecord = {
val record = new SpecificRecord(state(pointerLocation))
pointerLocation += 1
record
}
}
}
As you can see above, the SpecificRecord class is final and the specificRecords val is an Iterator with a bunch of SpecificRecord in it. I don't want to have to iterate through each specificRecord and create a new, more generic, object if possible.
My code that can be changed
val specificRecords: Iterator[SpecificRecord] = SpecificRecord.generateSpecificRecords()
type gettable = {
def get(i: Int): String
}
def printRecord(records: Iterator[gettable]): Unit = {
for (record <- records) {
println(record.get(0), record.get(1), record.get(2))
}
}
printRecord(specificRecords)
This correctly prints:
(Row1 Col1,Row1 Col2,Row 1 Col3)
(Row2 Col1,Row2 Col2,Row 2 Col3)
I have a printRecord method that doesn't really care what type is passed in, as long as it has a method like get(Int): String. This a pretty decent solution but I was wondering if it would be possible to do this without structural typing?
This is a typical use case for type classes.
trait Gettable[T] {
def get(t: T, i: Int): String
}
object Gettable {
implicit object SpecificRecordGettable extends Gettable[SpecificRecord] {
def get(sr: SpecificRecord, i: Int) = sr.get(i)
}
}
def printRecord[T : Gettable](records: Iterator[T]) = {
val getter = implicitly[Gettable[T]]
records.foreach { record =>
println(getter.get(record, 0), getter.get(record, 1), getter.get(record, 2))
}
}
This is a bit more verbose than your approach with structured types: for each type you want to be gettable, you have to add an implicit object implementing the get, but it works without reflection, which is a good thing.
Another advantage of this approach is its flexibility: the underlying type does not have to have get specifically, you can implement anything with the implicit. E.g.:
implicit object ArrayGettable extends Gettable[Array[String]] {
def get(a: Array[String], i: Int) = a(i)
}
implicit object ProductGettable extends Gettable[Product] {
def get(p: Product, i: Int) = p.productIterator.drop(i).next.toString
}
Now, your printRecord works with string arrays too (as long as they have at least three elements), and even tuples and case classes.
Try this:
printRecord[Product](Iterator((1,2, "three"), ("foo", "bar", 5)))
Or this:
case class Foo(x: String, y: Int, z: Seq[Int])
printRecord[Product](Iterator(Foo("bar", 1, 1::2::Nil), ("foo", "bar", "baz")))
A similar but a little bit less verbose approach is to just define an implicit 'getter' without bothering with type classes:
def printRecord[T](records: Iterator[T])(implicit getter: (T,Int) => String) =
records.foreach { record =>
println(getter(record, 0), getter(record, 1), getter(record, 2))
}
object Getters {
implicit def getter(sr: SpecificRecord, i: Int) = sr.get(i)
implicit def getter(a: Array[String], i: Int) = a(i)
implicit def getter(p: Product, i: Int) = p.productIterator.drop(i).next.toString
}
This is fairly equivalent in usage, the difference being that type class lets you potentially define more than one method, but if you only ever need get, then this would save you a few keystrokes.

Scala: generic function to get a String converted

I have a map with a getter method. The key is always a String, the value is Any.
I want to allow the caller to use a method like the followings
get[Int](k: String)
get[Boolean](k:String)
and inside this method convert the string to the specific type specified by the user. The immediate solution came to my mind was
def get[T](k: String): T = k.asInstanceOf[T]
which does not work. Then I tried with
def cast[T](x: String, classTag: ClassTag[T]): T = classTag match {
case Int => x.toInt
case Boolean => x.toBoolean
...
}
which does not compile. I am not sure this is even possible. Any idea or I need to write all the methods I want? For example
def getInt(k: String): Int
def getBoolean(k: String): Boolean
...
This is a classical use case for typeclass pattern widely used in scala. I assume that you have a custom implementation of Map and get method.
trait Converter[T]{ // typeclass
def convert(t:String):T
}
implicit object ToIntConverter extends Converter[Int] {
def convert(t:String):Int = t.toInt
}
implicit object ToBooleanConverter extends Converter[Boolean] {
def convert(t:String):Boolean = t.toBoolean
}
// vvv approach bellow works starting from scala 2.12 vvv
//
// implicit val ToBooleanConverter: Converter[Boolean] = s => s.toBoolean
// implicit val ToIntConverter : Converter[Int] = s => s.toInt
def get[T](k:String)(implicit cv: Converter[T]):T= cv.convert(k)
println(get[Int]("1"))
println(get[Boolean]("true"))
I got the below to work.
val anyMap: Map[String, Any] = Map(
"a" -> 1,
"b" -> true
)
def getInst[T](amap: Map[String, Any])(k: String): T = amap.get(k) match {
case Some(thing) => thing.asInstanceOf[T]
case None => throw new IllegalArgumentException
}
getInst[Int](anyMap)("a")
getInst[Boolean](anyMap)("b")
It's not very safe to have something like Map[String, Any] as the cast might fail. Probably best to introduce some ad-hoc polymorphism in your map (not sure).

Class type required but E found (scala macro)

I'm trying to remove some of the boilerplate in an API I am writing.
Roughly speaking, my API currently looks like this:
def toEither[E <: WrapperBase](priority: Int)(implicit factory: (String, Int) => E): Either[E, T] = {
val either: Either[String, T] = generateEither()
either.left.map(s => factory(s, priority))
}
Which means that the user has to generate an implicit factory for every E used. I am looking to replace this with a macro that gives a nice compile error if the user provided type doesn't have the correct ctor parameters.
I have the following:
object GenericFactory {
def create[T](ctorParams: Any*): T = macro createMacro[T]
def createMacro[T](c: blackbox.Context)(ctorParams: c.Expr[Any]*)(implicit wtt: WeakTypeType[T]): c.Expr[T] = {
import c.universe._
c.Expr[T](q"new $wtt(..$ctorParams)")
}
}
If I provide a real type to this GenericFactory.create[String]("hey") I have no issues, but if I provide a generic type: GenericFactory.create[E]("hey") then I get the following compile error: class type required by E found.
Where have I gone wrong? Or if what I want is NOT possible, is there anything else I can do to reduce the effort for the user?
Sorry but I don't think you can make it work. The problem is that Scala (as Java) uses types erasure. It means that there is only one type for all generics kinds (possibly except for value-type specializations which is not important now). It means that the macro is expanded only once for all E rather then one time for each E specialization provided by the user. And there is no way to express a restriction that some generic type E must have a constructor with a given signature (and if there were - you wouldn't need you macro in the first place). So obviously it can not work because the compiler can't generate a constructor call for a generic type E. So what the compiler says is that for generating a constructor call it needs a real class rather than generic E.
To put it otherwise, macro is not a magic tool. Using macro is just a way to re-write a piece of code early in the compiler processing but then it will be processed by the compiler in a usual way. And what your macro does is rewrites
GenericFactory.create[E]("hey")
with something like
new E("hey")
If you just write that in your code, you'll get the same error (and probably will not be surprised).
I don't think you can avoid using your implicit factory. You probably could modify your macro to generate those implicit factories for valid types but I don't think you can improve the code further.
Update: implicit factory and macro
If you have just one place where you need one type of constructors I think the best you can do (or rather the best I can do ☺) is following:
Sidenote the whole idea comes from "Implicit macros" article
You define StringIntCtor[T] typeclass trait and a macro that would generate it:
import scala.language.experimental.macros
import scala.reflect.macros._
trait StringIntCtor[T] {
def create(s: String, i: Int): T
}
object StringIntCtor {
implicit def implicitCtor[T]: StringIntCtor[T] = macro createMacro[T]
def createMacro[T](c: blackbox.Context)(implicit wtt: c.WeakTypeTag[T]): c.Expr[StringIntCtor[T]] = {
import c.universe._
val targetTypes = List(typeOf[String], typeOf[Int])
def testCtor(ctor: MethodSymbol): Boolean = {
if (ctor.paramLists.size != 1)
false
else {
val types = ctor.paramLists(0).map(sym => sym.typeSignature)
(targetTypes.size == types.size) && targetTypes.zip(types).forall(tp => tp._1 =:= tp._2)
}
}
val ctors = wtt.tpe.decl(c.universe.TermName("<init>"))
if (!ctors.asTerm.alternatives.exists(sym => testCtor(sym.asMethod))) {
c.abort(c.enclosingPosition, s"Type ${wtt.tpe} has no constructor with signature <init>${targetTypes.mkString("(", ", ", ")")}")
}
// Note that using fully qualified names for all types except imported by default are important here
val res = c.Expr[StringIntCtor[T]](
q"""
(new so.macros.StringIntCtor[$wtt] {
override def create(s:String, i: Int): $wtt = new $wtt(s, i)
})
""")
//println(res) // log the macro
res
}
}
You use that trait as
class WrapperBase(val s: String, val i: Int)
case class WrapperChildGood(override val s: String, override val i: Int, val float: Float) extends WrapperBase(s, i) {
def this(s: String, i: Int) = this(s, i, 0f)
}
case class WrapperChildBad(override val s: String, override val i: Int, val float: Float) extends WrapperBase(s, i) {
}
object EitherHelper {
type T = String
import scala.util._
val rnd = new Random(1)
def generateEither(): Either[String, T] = {
if (rnd.nextBoolean()) {
Left("left")
}
else {
Right("right")
}
}
def toEither[E <: WrapperBase](priority: Int)(implicit factory: StringIntCtor[E]): Either[E, T] = {
val either: Either[String, T] = generateEither()
either.left.map(s => factory.create(s, priority))
}
}
So now you can do:
val x1 = EitherHelper.toEither[WrapperChildGood](1)
println(s"x1 = $x1")
val x2 = EitherHelper.toEither[WrapperChildGood](2)
println(s"x2 = $x2")
//val bad = EitherHelper.toEither[WrapperChildBad](3) // compilation error generated by c.abort
and it will print
x1 = Left(WrapperChildGood(left,1,0.0))
x2 = Right(right)
If you have many different places where you want to ensure different constructors exists, you'll need to make the macro much more complicated to generate constructor calls with arbitrary signatures passed from the outside.

Optional conversion of basic types in scala

I am trying to support an abstraction of an ID type for a framework. Example here:
object AmINecessary {
case class StringConverter[T](op: String => T)
implicit val toInt = new StringConverter[Int](_.toInt)
implicit val toLong = new StringConverter[Long](_.toLong)
}
class Foo[ID] {
// ID can be String, Long, or Int
import AmINecessary._
// If ID is string, return string, otherwise convert to ID
def getID(id: String)(implicit c: StringConverter[ID] = null): ID = if (c == null) id.asInstanceOf[ID] else c.op(id)
}
This is then used as:
val fooString = new Foo[String]
val fooLong = new Foo[Long]
val fooInt = new Foo[Int]
fooString.getID("asdf") // "asdf":String
fooLong.getID("1234") // 1234:Long
fooInt.getID("1234") // 1234:Int
fooInt.getID("asdf") // java.lang.NumberFormatException
This works as expected. My questions are:
using an optional implicit by defaulting it to null then branching on it feels bad. What is the scala way to accomplish that?
Is it really necessary to write implicit conversions for a string to long or int?
I think the best option would be to simply add an implicit StringConverter[String] and remove the default null value.
That way your fooString works without risking a ClassCastException for every other type.
object AmINecessary {
case class StringConverter[T](op: String => T)
implicit val toInt = new StringConverter[Int](_.toInt)
implicit val toLong = new StringConverter[Long](_.toLong)
implicit val idConverter = new StringConverter[String](identity)
}
class Foo[ID] {
import AmINecessary.StringConverter
def getID(id: String)(implicit c: StringConverter[ID]): ID = c.op(id)
}
Regarding your question 2, the type class approach is not really necessary (but note that there are no implicit conversions here). You can also do it like this:
abstract class Foo[ID] {
def getID(id: String): ID
}
class FooInt extends Foo[Int] {
def getID(id: String) = id.toInt
}
class FooLong extends Foo[Long] {
def getID(id: String) = id.toLong
}
class FooString extends Foo[String] {
def getID(id: String) = id
}
1) About implicit defaulting to null, you could just:
object Unsafe {
implicit val toT[T] = new StringConverter[T](_.asInstanceOf[T])
}
2) It doesn't seem as a good idea. First, because you're hiding asInstanceOf, which is unsafe operation (potential runtime exception). Second, the more explicit conversion is - the better.
If you expect some complex transformations - it's better to return option from your getID method:
def getId[T](id: String)(converter: Option[StringConverter] = None) = converter.map(_.op(id))
However, default parameters aren't the best approach either. I'd stick with compile-time error requiring user to either write own converter or explicitly do an asInstanceOf, in general.
In your concrete case asInstanceOf doesn't make much sense as the only type it would work for is String, like getId[String], so what's the point of calling getId then?

Same method, different argument types, implemented in a single go

I have a (Java) class with operations like this:
abstract class Holder {
def set(i: Int): Unit
def set(s: String): Unit
def set(b: Boolean): Unit
...
}
Essentially, the all perform the same task, but just take different argument types. I would love to create a generic Accessor[T] that performs something like this:
class Accessor[T](holder: Holder) {
def set(value: T) { holder.set(value) }
}
... but that gives:
<console>:16: error: overloaded method value set with alternatives:
(s: String)Unit <and>
(i: Int)Unit
(b: Boolean)Unit
cannot be applied to (T)
def set(value: T) { holder.set(value) }
Is there any way out?
Use reflection.
class Setter(obj: AnyRef) {
val clazz = obj.getClass
def set[T : Manifest](v: T): Boolean = try {
val paramType = manifest[T].erasure
val method = clazz.getMethod("set", paramType)
method.invoke(obj, v.asInstanceOf[AnyRef])
true
} catch {
case ex => false
}
}
val holder = ..
val setter = new Setter(holder)
setter.set(5) // returns true
setter.set(1.0) // double not accepted, returns false
There was an experimental shortcut for that in Scala, but it got removed before 2.8.0 was released.
I think matching should work nicely
def set(value: T) {
value match {
case s: String => holder.set(s)
case i: Int => holder.set(i)
case b: Boolean => holder.set(b)
}
}
I don't fully understand your use case, but one thing that you might try doing--if performance is not of utmost importance--is creating a wrapper class that converts to a universal form for you, and then have all your methods take that wrapper class (with appropriate implicit conversions in place). For example:
class Wrap(val data: String)
implicit def wrapString(s: String) = new Wrap(s)
implicit def wrapBoolean(b: Boolean) = if (b) new Wrap("T") else new Wrap("F")
implicit def wrapLong(l: Long) = new Wrap(l.toString+"L")
class User {
private[this] var myData = ""
def set(w: Wrap) { println("Setting to "+w.data); myData = w.data }
}
val u = new User
u.set(true)
u.set(50L)
u.set(50) // Int gets upconverted to Long for free, so this works
u.set("Fish")
// u.set(3.14159) // This is a type mismatch
This is a little bit like taking an Any except that you can restrict the types however you like and specify the conversion into whatever universal representation you have in mind. However, if there does not exist a universal form, then I'm not sure in what sense that you mean the code is doing the same thing each time. (Maybe you mean that you can conceive of a macro (or another program) that would generate the code automatically--Scala doesn't have that support built in, but you can of course write a Scala program that produces Scala code.)
Looking back at the results gathered so far, there are a couple of solutions suggested:
Use pattern matching (leads to fragmentation of the different strategies of dealing with the different parameter types)
Use reflection (to expensive for something that ideally should be super fast)
... and adding the one that I eventually ended up implementing: write an adapter per type of parameter.
To be a little more precise, the whole exercise was about writing a wrapper around Kyoto Cabinet. Kyoto Cabinet has methods for associating byte array keys with byte array values and String keys with String values. And then it basically replicates most of the operations for dealing with keys and values for both byte array as well as Strings.
In order to create a Map wrapper around Kyoto Cabinet's DB class, I defined a trait TypedDBOperations[T], with T being the type of parameter, and had it implemented twice. If I now construct a Map[Array[Byte], Array[Byte]], an implicit conversion will automatically assign it the proper instane of TypedDBOperations, calling the Array[Byte] based operations of the DB class.
This is the trait that I have been talking about:
trait TypedDBOperations[K,V] {
def get(db: DB, key: K): V
def set(db: DB, key: K, value: V): Boolean
def remove(db: DB, key: K): Boolean
def get(cursor: Cursor): (K, V)
}
And these are the implementations for both type of key value combinations:
implicit object StringDBOperations extends TypedDBOperations[String] {
def get(cursor: Cursor) = {
val Array(a, b) = cursor.get_str(false)
(a, b)
}
def remove(db: DB, key: String) = db.remove(key)
def set(db: DB, key: String, value: String) = db.set(key, value)
def get(db: DB, key: String) = db.get(key)
}
implicit object ByteArrayOperations extends TypedDBOperations[Array[Byte]] {
def get(cursor: Cursor) = {
val Array(a, b) = cursor.get(false)
(a, b)
}
def remove(db: DB, key: Array[Byte]) = db.remove(key)
def set(db: DB, key: Array[Byte], value: Array[Byte]) = db.set(key, value)
def get(db: DB, key: Array[Byte]) = db.get(key)
}
Not the most satisfying solution ever, but it gets the job done. Again, note there still is quite a bit of duplication, but it seems there's no way to get rid of it.