Scala trouble with casting to generics - scala

I tried to write some function which does different things according to the input type.
For example, I do this:
def Foo[T](inList: List[String]): ArrayBuffer[T] = {
val out: ArrayBuffer[T] = new ArrayBuffer[T]()
inList.map ( x => {
val str = x.substring(1)
out += str.asInstanceOf[T]
})
out
}
but if I call this function with Foo[Long](new List("123","2342")), I get an ArrayBuffer with Strings, not Longs. Sorry for my noob question, i want understand scala and generics.

Scala runs ontop of JVM, that does not know anything about generics, and concrete type of generic is not available at runtime.
So that, runtime equivalent of your code will look like
def Foo(inList: List[String]): (ArrayBuffer[Object]) = {
val out: ArrayBuffer[Object] = new ArrayBuffer[Object]()
inList.map ( x => {
val str=x.substring(1)
out += str.asInstanceOf[Object]
})
(out)
}
asInstanceOf will not convert strings to longs, instead it will throw exception about incompatible types. Instead, you should supply your function with a conversion from strings to another type.
Summing up, your code should look like this:
import scala.collection.mutable.ArrayBuffer
// here we define how convert string to longs
implicit def stringToLong(s: String) = s.toLong
// now this function requires to have converter from string to T in context
def Foo[T](inList: List[String])(implicit f: (String) => T): (ArrayBuffer[T]) = {
val out: ArrayBuffer[T] = new ArrayBuffer[T]()
inList.map { x =>
val str = x.substring(1)
out += f(str) // here we apply converter
}
out
}
// when function is called, appropriate implicit converter from context will be used
Foo[Long](List("51", "42"))

Casting a String to a Long does not make it a Long, it is an error, so you should get a runtime error saying that you cannot cast String to Long. Since T does not place any restrictions on what it can be, you cannot expect to call any method that isn't on all objects (toString is for example). You could provide that function yourself (which is almost the exact signature of map, so you could just use that):
def convert[T](in: List[String])(convert: String => T): List[T] =
in.map(convert)
Or make sure the objects are of a type that you know of the methods on (using a type bound):
trait CanConvertTo[T] {
def convert: T
}
case class Example(value: String) extends CanConvertTo[Long] {
def convert = value.toLong
}
def convert[T](in: List[CanConvertTo[T]]): List[T] = in.map(_.convert)

Related

Scala ClassCastException on Option.orNull

When I try to run following code:
def config[T](key: String): Option[T] = {
//in reality this is a map of various instance types as values
Some("string".asInstanceOf[T])
}
config("path").orNull
I'm getting error:
java.lang.String cannot be cast to scala.runtime.Null$
java.lang.ClassCastException
Following attempts are working fine:
config[String]("path").orNull
config("path").getOrElse("")
Since getOrElse works its confusing why null is so special and throws an error. Is there a way for orNull to work without specifying generic type ?
scalaVersion := "2.12.8"
Just to show how you may avoid the use asInstanceOf to get the values from a typed config.
sealed trait Value extends Product with Serializable
final case class IntValue(value: Int) extends Value
final case class StringValue(value: String) extends Value
final case class BooleanValue(value: Boolean) extends Value
type Config = Map[String, Value]
sealed trait ValueExtractor[T] {
def extract(config: Config)(fieldName: String): Option[T]
}
object ValueExtractor {
implicit final val IntExtractor: ValueExtractor[Int] =
new ValueExtractor[Int] {
override def extract(config: Config)(fieldName: String): Option[Int] =
config.get(fieldName).collect {
case IntValue(value) => value
}
}
implicit final val StringExtractor: ValueExtractor[String] =
new ValueExtractor[String] {
override def extract(config: Config)(fieldName: String): Option[String] =
config.get(fieldName).collect {
case StringValue(value) => value
}
}
implicit final val BooleanExtractor: ValueExtractor[Boolean] =
new ValueExtractor[Boolean] {
override def extract(config: Config)(fieldName: String): Option[Boolean] =
config.get(fieldName).collect {
case BooleanValue(value) => value
}
}
}
implicit class ConfigOps(val config: Config) extends AnyVal {
def getAs[T](fieldName: String)(default: => T)
(implicit extractor: ValueExtractor[T]): T =
extractor.extract(config)(fieldName).getOrElse(default)
}
Then, you can use it like this.
val config = Map("a" -> IntValue(10), "b" -> StringValue("Hey"), "d" -> BooleanValue(true))
config.getAs[Int](fieldName = "a")(default = 0) // res: Int = 10
config.getAs[Int](fieldName = "b")(default = 0) // res: Int = 0
config.getAs[Boolean](fieldName = "c")(default = false) // res: Boolean = false
Now, the problem becomes how to create the typed config from a raw source.
And even better, how to directly map the config to a case class.
But, those are more complex, and probably is better to just use something already done, like pureconfig.
Just as an academic exercise, lets see if we can support Lists & Maps.
Lets start with lists, a naive approach would be to have another case class for values which are lists, and create a factory of extractors for every kind of list (this process is formally know as implicit derivation).
import scala.reflect.ClassTag
final case class ListValue[T](value: List[T]) extends Value
...
// Note that, it has to be a def, since it is not only one implicit.
// But, rather a factory of implicits.
// Also note that, it needs another implicit parameter to construct the specific implicit.
// In this case, it needs a ClasTag for the inner type of the list to extract.
implicit final def listExtractor[T: ClassTag]: ValueExtractor[List[T]] =
new ValueExtractor[List[T]] {
override def extract(config: Config)(fieldName: String): Option[List[T]] =
config.get(fieldName).collect {
case ListValue(value) => value.collect {
// This works as a safe caster, which will remove all value that couldn't been casted.
case t: T => t
}
}
}
Now, you can use it like this.
val config = Map("l" ->ListValue(List(1, 2, 3)))
config.getAs[List[Int]](fieldName = "l")(default = List.empty)
// res: List[Int] = List(1, 2, 3)
config.getAs[List[String]](fieldName = "l")(default = List("Hey"))
// res: String = List() - The default is not used, since the field is a List...
// whose no element could be casted to String.
However, this approach is limited to plain types, if you need a List of other generic type, like a List of Lists. Then, this won't work.
val config = Map("l" ->ListValue(List(List(1, 2), List(3))))
val l = config.getAs[List[List[String]]](fieldName = "l")(default = List.empty)
// l: List[List[String]] = List(List(1, 2), List(3)) ???!!!
l.head
// res: List[String] = List(1, 2)
l.head.head
// java.lang.ClassCastException: java.lang.Integer cannot be cast to java.lang.String
The problem here is type erasure, which ClassTags can not solve, you may try to use TypeTags which can preserve the complete type, but the solution becomes more cumbersome.
For Maps the solution is quite similar, especially if you fix the key type to String (assuming what you really want is a nested config). But, this post is too long now, so I would leave it as an exercise for the reader.
Nevertheless, as already said, this can be broken easily, and is not completely robust.
There are better approaches, but I myself am not very skilled on those (yet), and even if I would be, the answer would be more long and really not necessary at all.
Lucky for you, even if pureconfig does not support YAML directly, there is a module which does, pureconfig-yaml.
I would suggest you to take a look to the module, and if you have further problems ask a new question tagging pureconfig and yaml directly. Also, if it is just a small doubt, you may try asking in thegitter channel.

Class type required but E found (scala macro)

I'm trying to remove some of the boilerplate in an API I am writing.
Roughly speaking, my API currently looks like this:
def toEither[E <: WrapperBase](priority: Int)(implicit factory: (String, Int) => E): Either[E, T] = {
val either: Either[String, T] = generateEither()
either.left.map(s => factory(s, priority))
}
Which means that the user has to generate an implicit factory for every E used. I am looking to replace this with a macro that gives a nice compile error if the user provided type doesn't have the correct ctor parameters.
I have the following:
object GenericFactory {
def create[T](ctorParams: Any*): T = macro createMacro[T]
def createMacro[T](c: blackbox.Context)(ctorParams: c.Expr[Any]*)(implicit wtt: WeakTypeType[T]): c.Expr[T] = {
import c.universe._
c.Expr[T](q"new $wtt(..$ctorParams)")
}
}
If I provide a real type to this GenericFactory.create[String]("hey") I have no issues, but if I provide a generic type: GenericFactory.create[E]("hey") then I get the following compile error: class type required by E found.
Where have I gone wrong? Or if what I want is NOT possible, is there anything else I can do to reduce the effort for the user?
Sorry but I don't think you can make it work. The problem is that Scala (as Java) uses types erasure. It means that there is only one type for all generics kinds (possibly except for value-type specializations which is not important now). It means that the macro is expanded only once for all E rather then one time for each E specialization provided by the user. And there is no way to express a restriction that some generic type E must have a constructor with a given signature (and if there were - you wouldn't need you macro in the first place). So obviously it can not work because the compiler can't generate a constructor call for a generic type E. So what the compiler says is that for generating a constructor call it needs a real class rather than generic E.
To put it otherwise, macro is not a magic tool. Using macro is just a way to re-write a piece of code early in the compiler processing but then it will be processed by the compiler in a usual way. And what your macro does is rewrites
GenericFactory.create[E]("hey")
with something like
new E("hey")
If you just write that in your code, you'll get the same error (and probably will not be surprised).
I don't think you can avoid using your implicit factory. You probably could modify your macro to generate those implicit factories for valid types but I don't think you can improve the code further.
Update: implicit factory and macro
If you have just one place where you need one type of constructors I think the best you can do (or rather the best I can do ☺) is following:
Sidenote the whole idea comes from "Implicit macros" article
You define StringIntCtor[T] typeclass trait and a macro that would generate it:
import scala.language.experimental.macros
import scala.reflect.macros._
trait StringIntCtor[T] {
def create(s: String, i: Int): T
}
object StringIntCtor {
implicit def implicitCtor[T]: StringIntCtor[T] = macro createMacro[T]
def createMacro[T](c: blackbox.Context)(implicit wtt: c.WeakTypeTag[T]): c.Expr[StringIntCtor[T]] = {
import c.universe._
val targetTypes = List(typeOf[String], typeOf[Int])
def testCtor(ctor: MethodSymbol): Boolean = {
if (ctor.paramLists.size != 1)
false
else {
val types = ctor.paramLists(0).map(sym => sym.typeSignature)
(targetTypes.size == types.size) && targetTypes.zip(types).forall(tp => tp._1 =:= tp._2)
}
}
val ctors = wtt.tpe.decl(c.universe.TermName("<init>"))
if (!ctors.asTerm.alternatives.exists(sym => testCtor(sym.asMethod))) {
c.abort(c.enclosingPosition, s"Type ${wtt.tpe} has no constructor with signature <init>${targetTypes.mkString("(", ", ", ")")}")
}
// Note that using fully qualified names for all types except imported by default are important here
val res = c.Expr[StringIntCtor[T]](
q"""
(new so.macros.StringIntCtor[$wtt] {
override def create(s:String, i: Int): $wtt = new $wtt(s, i)
})
""")
//println(res) // log the macro
res
}
}
You use that trait as
class WrapperBase(val s: String, val i: Int)
case class WrapperChildGood(override val s: String, override val i: Int, val float: Float) extends WrapperBase(s, i) {
def this(s: String, i: Int) = this(s, i, 0f)
}
case class WrapperChildBad(override val s: String, override val i: Int, val float: Float) extends WrapperBase(s, i) {
}
object EitherHelper {
type T = String
import scala.util._
val rnd = new Random(1)
def generateEither(): Either[String, T] = {
if (rnd.nextBoolean()) {
Left("left")
}
else {
Right("right")
}
}
def toEither[E <: WrapperBase](priority: Int)(implicit factory: StringIntCtor[E]): Either[E, T] = {
val either: Either[String, T] = generateEither()
either.left.map(s => factory.create(s, priority))
}
}
So now you can do:
val x1 = EitherHelper.toEither[WrapperChildGood](1)
println(s"x1 = $x1")
val x2 = EitherHelper.toEither[WrapperChildGood](2)
println(s"x2 = $x2")
//val bad = EitherHelper.toEither[WrapperChildBad](3) // compilation error generated by c.abort
and it will print
x1 = Left(WrapperChildGood(left,1,0.0))
x2 = Right(right)
If you have many different places where you want to ensure different constructors exists, you'll need to make the macro much more complicated to generate constructor calls with arbitrary signatures passed from the outside.

String companion object in scala

Given a type which has a "converter", I would like to have automatic conversion on method call using this type's companion object. That is, given the following definition,
case class Converted(name: String)
trait Converter[A] {
def perform: Converted
}
implicit val StringConverter = new Converter[String] {
def perform = Converted("String")
}
make the following code to work:
implicit def toConverter(a: String.type): Converted =
implicitly[Converter[String]].perform // Error: `Found String.type, required AnyRef`
def f(needsConverted: Converted) = ???
f(String) // <- That's what I would like to be able to write.
But this fails and both attempts for conversion fail. Note that I cannot change f because it is provided by a third-party library and there are many of them.
Can I make f(String) compile using implicits?
If not possible for Strings, what about classes which do have a companion object, can I do this generically like:
object TheClass
case class TheClass()
implicit val TheClassConverter = new Converter[TheClass] {
def perform = Converted("TheClass")
}
implicit def toConverter[A: Converter](a: A.type): Converted =
implicitly[Converter[A]].perform // Error: `Not found value A`
implicit def toConverter(a: TheClass.type): Converted =
implicitly[Converter[TheClass]].perform // This works but is not generic
f(TheClass) // This works.
Can I make the first toConverter to compile ?
Instead of defining an implicit instance for the type MyClass you can define an implicit instance for the companion type MyClass.type.
implicit val TheClassConverter: Converter[MyClass.type] = new Converted[MyClass.type] {
def perform = Converted("MyClass")
}
Can I make f(String) compile using implicits?
No. You can define a value called String, of course, but it won't be related to the type String.
implicit toConverter[A: Converter](a: A.type): Converted =
implicitly[Converter[A]].perform
A in A.type must be a value; it is not related to the type parameter A.
In fact, so far as Scala's type system is concerned, there is no relationship between a class/trait and its companion object. So you can't do what you want generically.
Of course, if you don't insist on using () instead of [], it becomes trivial:
def f1[A: Converter] = f(implicitly[Converter[A]].perform)
f1[String]
f1[TheClass]
Not sure, what are you trying to accomplish, but following works for me
case class Converted(name: String)
trait Converter[A] {
def perform: Converted
}
implicit def toConverted(name: String) = Converted("String")
implicit def toIntConverted(int: Int) = Converted("Int")
def f(needsConverted: Converted): String = needsConverted.name
f("some")
f(5)

How to overload the product method of a typeclass

I am trying to implement a ReadJsonCodec of sorts using the automatic type class derivation mechanism in Shapeless.
Here is my ReadCodecCompanionObject:
object ReadCodec extends LabelledProductTypeClassCompanion[ReadCodec] {
implicit object StringCodec extends SimpleCodec[String] {
def read(j: Json): String = j.stringOr(throw ...)
}
implicit object IntCodec ...
implicit object BooleanCodec ...
implicit object LongCodec ...
implicit object ShortCodec ...
implicit object DoubleCodec ...
implicit object BigDecimalCodec ...
implicit def readCodecInstance: LabelledProductTypeClass[ReadCodec] = new LabelledProductTypeClass[ReadCodec] {
def emptyProduct = new ReadCodec[HNil] {
// This will silently accept extra fields within a JsonObject
// To change this behavior make sure json is a JsonObject and that it is empty
def read(json: Json) = HNil
}
def product[F, T <: HList](name: String, FHead: ReadCodec[F], FTail: ReadCodec[T]) = new ReadCodec[F :: T] {
def read(json: Json): F :: T = {
val map = castOrThrow(json)
val fieldValue = map.getOrElse(name, throw new MappingException(s"Expected field $name on JsonObject $map"))
// Try reading the value of the field
// If we get a mapping exception, intercept it and add the name of this field to the path
// If we get another exception, don't touch!
// Pitfall: if handle did not accept a PartialFunction, we could transform an unknow exception into a match exception
val head: F = Try(FHead.read(fieldValue)).handle{ case MappingException(msg, path) => throw MappingException(msg, s"$name/$path")}.get
val tail = FTail.read(json)
head :: tail
}
}
def product[A, T <: HList](name: String, FHead: ReadCodec[Option[A]], FTail: ReadCodec[T]) = new ReadCodec[Option[A] :: T] {
def read(json: Json): Option[A] :: T = {
val map = castOrThrow(json)
val head: Option[A] = map.get(name).map { fieldValue =>
Try(FHead.read(fieldValue)).handle{ case MappingException(msg, path) => throw MappingException(msg, s"$name/$path")}.get.get
}
val tail = FTail.read(json)
head :: tail
}
}
def project[F, G](instance: => ReadCodec[G], to : F => G, from : G => F) = new ReadCodec[F] {
def read(json: Json): F = from(instance.read(json))
}
}
}
That's somewhat of a complicated piece of code to just grasp quickly but is really fairly simple once you understand it. The important part is the two def product methods. The problem I am having is that I want this codec to accept a Json AST that is missing a field if that field would be mapped to a value of type Option[A]. This means that I need the product function to know if the head of the HList is of type Option[A] or not.
Concretely:
case class Foo(a: String, b: Option[Boolean])
val test = read[Foo](json"""{"a" : "Beaver"}""") // should work
But currently this would fail because it makes no distinction for Option and expects a field b. I tried two ways of fixing this and neither of them worked.
The first one and the cleanest one is to overload the product method with a version where the F type parameter is replaced with Option[A]. This approach is fairly clean, although I am not sure whether or not it would play well with the shapeless derivation macro. However, it is not possible because Scala does not support the ability to overload functions when the type signature after erasure is the same. This is the version that is above. Unfortunately, this does not currently compile with the Scala compiler.
The second approach was to use a TypeTag to find out at runtime if F is of type Option[_] and behave appropriately. This version would almost certainly be less elegant and involve a cast but I could live with it. However, it seems impossible because the addition of a TypeTag changes the signature of the product method (adding an implicit parameter) and then the compiler complains that I am not defining the abstract method product.
Does anyone have any suggestions on the best way to proceed.
You need to get a typeclass with information about F passed through. But you're already passing a typeclass around, in the form of ReadCodec. So the solution is to replace that with one that contains all the information you need:
trait ReadCodecAndTypeTag[A] {
val rc: ReadCodec[A]
val tt: TypeTag[A]
}
But in that case you might as well delegate the decoding-from-optional-value-in-a-map to this typeclass as well:
trait OReadCodec[A] {
val rc: ReadCodec[A]
def missingField(name: String, map: Any): A =
throw new MappingException(s"Expected field $name on JsonObject $map")
}
implicit object StringCodec extends OReadCodec[String] {
val rc = new ReadCodec[String] {...}
}
implicit object IntCodec ...
...
implicit def OptionCodec[A](implicit orc: OReadCodec[A]) =
new OReadCodec[Option[A]] {
val rc = ...
override def missingField(name: String, map: Any) = None
}
...
def product[F, T <: HList](name: String, FHead: OReadCodec[F], FTail: OReadCodec[T]) =
new OReadCodec[F :: T] {
val rc = new ReadCodec[F :: T] {
def read(json: Json): F :: T = {
val map = castOrThrow(json)
val fieldValue = map.getOrElse(name, FHead.missingField(name, map))
val head: F = ...
...
}
}
}
implicit def ReadCodecFromOReadCodec[A](implicit orc: OReadCodec[A]) = orc.rc

Same method, different argument types, implemented in a single go

I have a (Java) class with operations like this:
abstract class Holder {
def set(i: Int): Unit
def set(s: String): Unit
def set(b: Boolean): Unit
...
}
Essentially, the all perform the same task, but just take different argument types. I would love to create a generic Accessor[T] that performs something like this:
class Accessor[T](holder: Holder) {
def set(value: T) { holder.set(value) }
}
... but that gives:
<console>:16: error: overloaded method value set with alternatives:
(s: String)Unit <and>
(i: Int)Unit
(b: Boolean)Unit
cannot be applied to (T)
def set(value: T) { holder.set(value) }
Is there any way out?
Use reflection.
class Setter(obj: AnyRef) {
val clazz = obj.getClass
def set[T : Manifest](v: T): Boolean = try {
val paramType = manifest[T].erasure
val method = clazz.getMethod("set", paramType)
method.invoke(obj, v.asInstanceOf[AnyRef])
true
} catch {
case ex => false
}
}
val holder = ..
val setter = new Setter(holder)
setter.set(5) // returns true
setter.set(1.0) // double not accepted, returns false
There was an experimental shortcut for that in Scala, but it got removed before 2.8.0 was released.
I think matching should work nicely
def set(value: T) {
value match {
case s: String => holder.set(s)
case i: Int => holder.set(i)
case b: Boolean => holder.set(b)
}
}
I don't fully understand your use case, but one thing that you might try doing--if performance is not of utmost importance--is creating a wrapper class that converts to a universal form for you, and then have all your methods take that wrapper class (with appropriate implicit conversions in place). For example:
class Wrap(val data: String)
implicit def wrapString(s: String) = new Wrap(s)
implicit def wrapBoolean(b: Boolean) = if (b) new Wrap("T") else new Wrap("F")
implicit def wrapLong(l: Long) = new Wrap(l.toString+"L")
class User {
private[this] var myData = ""
def set(w: Wrap) { println("Setting to "+w.data); myData = w.data }
}
val u = new User
u.set(true)
u.set(50L)
u.set(50) // Int gets upconverted to Long for free, so this works
u.set("Fish")
// u.set(3.14159) // This is a type mismatch
This is a little bit like taking an Any except that you can restrict the types however you like and specify the conversion into whatever universal representation you have in mind. However, if there does not exist a universal form, then I'm not sure in what sense that you mean the code is doing the same thing each time. (Maybe you mean that you can conceive of a macro (or another program) that would generate the code automatically--Scala doesn't have that support built in, but you can of course write a Scala program that produces Scala code.)
Looking back at the results gathered so far, there are a couple of solutions suggested:
Use pattern matching (leads to fragmentation of the different strategies of dealing with the different parameter types)
Use reflection (to expensive for something that ideally should be super fast)
... and adding the one that I eventually ended up implementing: write an adapter per type of parameter.
To be a little more precise, the whole exercise was about writing a wrapper around Kyoto Cabinet. Kyoto Cabinet has methods for associating byte array keys with byte array values and String keys with String values. And then it basically replicates most of the operations for dealing with keys and values for both byte array as well as Strings.
In order to create a Map wrapper around Kyoto Cabinet's DB class, I defined a trait TypedDBOperations[T], with T being the type of parameter, and had it implemented twice. If I now construct a Map[Array[Byte], Array[Byte]], an implicit conversion will automatically assign it the proper instane of TypedDBOperations, calling the Array[Byte] based operations of the DB class.
This is the trait that I have been talking about:
trait TypedDBOperations[K,V] {
def get(db: DB, key: K): V
def set(db: DB, key: K, value: V): Boolean
def remove(db: DB, key: K): Boolean
def get(cursor: Cursor): (K, V)
}
And these are the implementations for both type of key value combinations:
implicit object StringDBOperations extends TypedDBOperations[String] {
def get(cursor: Cursor) = {
val Array(a, b) = cursor.get_str(false)
(a, b)
}
def remove(db: DB, key: String) = db.remove(key)
def set(db: DB, key: String, value: String) = db.set(key, value)
def get(db: DB, key: String) = db.get(key)
}
implicit object ByteArrayOperations extends TypedDBOperations[Array[Byte]] {
def get(cursor: Cursor) = {
val Array(a, b) = cursor.get(false)
(a, b)
}
def remove(db: DB, key: Array[Byte]) = db.remove(key)
def set(db: DB, key: Array[Byte], value: Array[Byte]) = db.set(key, value)
def get(db: DB, key: Array[Byte]) = db.get(key)
}
Not the most satisfying solution ever, but it gets the job done. Again, note there still is quite a bit of duplication, but it seems there's no way to get rid of it.