I have a DSL and some runtime code. The problem is I got somewhere at runtime:
val clazz: Class[_ <: java.io.Serializable] = classOf[java.lang.String]
val value: java.io.Serializable = "abc"
and I have a class
class MyWrapper[T <: java.io.Serializable](val data: T)(implicit m: Manifest[T]) {
override def toString = m.runtimeClass
}
val wrapper = new MyWrapper(value)
The problem is I need to return java.lang.String from a call of toString. But I got java.io.Serializable. Unfortunately I am neither able to create fixed pattern matching for each java.io.Serializable subtype (this would be crazy) nor to create MyWrapper explicit with new MyWrapper[String](value). I don't know the type of value, maybe it is a subtype of Serializable.
Is there some way to pass type/manifest value at runtime if I know that value type is equal to variable clazz?
Update (solution from Régis Jean-Gilles don't work). REPL test:
val clazz: Class[_ <: java.io.Serializable] = classOf[java.lang.String]
val value: java.io.Serializable = "abc"
class MyWrapper[T <: java.io.Serializable](val data: T)(implicit m: Manifest[T]) {
override def toString = m.runtimeClass.getName
}
val wrapper = new MyWrapper(value)
//val wrapper = new MyWrapper(value)(ClassManifest.fromClass(clazz).asInstanceOf[ClassTypeManifest[java.io.Serializable]])
//val wrapper = new MyWrapper(value)(ClassManifest.fromClass(clazz))
System.err.println(wrapper.toString)
I got an error, if I tried to pass manifest explicit:
scala> :load test.scala
Loading test.scala...
clazz: Class[_ <: java.io.Serializable] = class java.lang.String
value: java.io.Serializable = abc
defined class MyWrapper
<console>:10: error: type mismatch;
found : scala.reflect.ClassManifest[_$1] where type _$1 <: java.io.Serializable
(which expands to) scala.reflect.ClassTag[_$1]
required: Manifest[java.io.Serializable]
val wrapper = new MyWrapper(value)(ClassManifest.fromClass(clazz))
^
<console>:8: error: not found: value wrapper
System.err.println(wrapper.toString)
Also I am unable to cast manifest explicit.
There is more strange error when I try to compile my application -
[error] found : scala.reflect.ClassManifest[(some other)_0(in method setProperty)]
[error] (which expands to) scala.reflect.ClassTag[(some other)_0(in method setProperty)]
[error] required: Manifest[_0(in method setProperty)]
[error] new Value.Static(default, Context.virtual(element))(ClassManifest.fromClass(elementProperty.typeClass)))
IMHO Régis Jean-Gilles very very close to solution. How to make it work with Scala 2.10?
If I understand correctly, you are using manifests to work around type erasure but at a specific point all you have is a Class[_] so you need to convert it back to a manifest. Correct?
If so, you can use ClassManifest.fromClass, and then explicitly pass it as the implicit value m.
val wrapper = new MyWrapper(value)(Manifest.classType(clazz))
You should probably abstract it away:
object MyWrapper {
def fromRuntimeType[T <: java.io.Serializable]( value: T ): MyWrapper[T] = {
new MyWrapper(value)(Manifest.classType(value.getClass))
}
}
Keep in mind though that because you only have a Class instance in the first place, you are at the mercy of type erasure again. This means that if the class (that you need to get a manifest for) is generic, you won't have any type parameter information in the manifest returned by ClassManifest.fromClass.
Wouldn't
override def toString = data.getClass.getName
already work for you? That yields java.lang.String.
Related
This is a follow up question on my previous question:
Scala - Passing Class of derived type in place of Class of supertype
Following is my use case:
I have a class called MultiplePredictionTester with the following implementation...
class MultiplePredictionTester[T <: SurvivalPredictor](predictorClasses: Seq[Class[T]],
restOfConstructorParams: Map[Class[T], Seq[Object]],
dataSplitter: DataSplitter,
titanic: DataFrame) extends PredictionTester {
import titanic.sqlContext.implicits._
override def test: Map[SurvivalPredictor, Double] = ???
}
The idea is to be able to take a sequence of classes derived from SurvivalPredictor and instantiate them inside, using a constructor where first argument is fetched from DataSplitter while the rest of the arguments come from the Map[Class[T], Seq[Object]]
I plan to invoke this class as:
object MultiQuickRunner extends App with ResourceGetter {
val conf: SparkConf = new SparkConf().setAppName("TitanicSurvivalPredictor").setMaster("local[4]")
val sc: SparkContext = new SparkContext(conf)
val sqlContext: SQLContext = new SQLContext(sc)
val test: Map[SurvivalPredictor, Double] =
new MultiplePredictionTester(Seq(classOf[SexBasedPredictor], classOf[TotallyRandomPredictor]),
Map[Class[SurvivalPredictor],Seq[Object]](),
new DataSplitter {},
new DataFrameLoader {}.load(getPathForResource("train.csv"),
sqlContext)).test
test.foreach { case (survivalPredictorInstance, accuracy) =>
println("Accuracy for " + survivalPredictorInstance.getClass.getName + " is " + accuracy)
}
}
However trying to compile something like that throws me the exception:
Error:(27, 5) no type parameters for constructor MultiplePredictionTester: (predictorClasses: Seq[Class[T]], restOfConstructorParams: Map[Class[T],Seq[Object]], dataSplitter: com.dhruvk.kaggle.DataSplitter, titanic: org.apache.spark.sql.DataFrame)com.dhruvk.kaggle.predictiontesters.implementations.MultiplePredictionTester[T] exist so that it can be applied to arguments (Seq[Class[_ >: com.dhruvk.kaggle.predictors.implementations.TotallyRandomPredictor with com.dhruvk.kaggle.predictors.implementations.SexBasedPredictor <: com.dhruvk.kaggle.predictors.SurvivalPredictor]], scala.collection.immutable.Map[Class[com.dhruvk.kaggle.predictors.SurvivalPredictor],Seq[Object]], com.dhruvk.kaggle.DataSplitter, org.apache.spark.sql.DataFrame)
--- because ---
argument expression's type is not compatible with formal parameter type;
found : Seq[Class[_ >: com.dhruvk.kaggle.predictors.implementations.TotallyRandomPredictor with com.dhruvk.kaggle.predictors.implementations.SexBasedPredictor <: com.dhruvk.kaggle.predictors.SurvivalPredictor]]
required: Seq[Class[?T]]
new MultiplePredictionTester(Seq(classOf[SexBasedPredictor], classOf[TotallyRandomPredictor]),
^
Any help is appreciated.
java.lang.Class is invariant. During class declaration you are telling compiler that it is some T which has a upper bound such that T <: SurvivalPredictor. But when initializing you are saying T could SexBasedPredictor, TotallyRandomPredictor or SurvivalPredictor. This is not possible.
Consider
class A
class B extends A
class Test[T <: A](x: Seq[Class[T]], m: Map[Class[T], Seq[T]])
Here I have a class Test with type parameter T with upper bound A. Now I can initialize Test with
new Test(Seq(classOf[A], classOf[A]), Map(classOf[A] -> Seq(new B)))
As java.lang.Class is invariant, once I state T as A. I cant change it. If I change to B, compiler will yell at me. On the other hand as Seq is covariant I can say new B there.
As my comment to Till Rohrman's answer to the linked question mentions, there is just one Class[T] for any specific class or trait T: classOf[T]. So your predictorClasses can have length at most 1, and restOfConstructorParams can have size of most one. This is not what you want. The correct signature would use existential types:
class MultiplePredictionTester(predictorClasses: Seq[Class[_ <: SurvivalPredictor]],
...
Class[_ <: SurvivalPredictor] is short for Class[T <: SurvivalPredictor] forSome { type T } and is the common type for all Class objects of subtypes of SurvivalPredictor.
It isn't clear what you want to use restOfConstructorParams for, but the type you gave is also wrong. It might be Seq[Any], for example.
I am trying to modify the Enterprise trait that extends the StarShip type.
(I apologize in advance for not knowing a proper title for this but I will update it after I understand the answer.)
For some reason I cannot get the parameters of the starShip that is passed.
I commented where I am getting the error.
Simple example to demonstrate this:
object Test {
trait Akira extends StarShip
trait Enterprise extends StarShip
sealed trait StarShip {
val captain: String
}
def doSomething[T: StarShip](starShip: T): T =
new T {
val captain = starShip.captain // Error: Cannot resolve symbol
}
doSomething(new Enterprise {
override val captain = "Picard"
})
}
Since I am passing an object of Enterprise I expect to get the same class out.
Edit:
Just realized that I want all variables in the starShip that is passed to be copied to the new class. With the exception that I will modify a few of them.
I believe the Monocle lib may solve my problem.
When I compile your code with the scala REPL, I get this:
<console>:17: error: Test.StarShip does not take type parameters
def doSomething[T: StarShip](starShip: T): T =
^
<console>:18: error: class type required but T found
new T {
^
<console>:19: error: value captain is not a member of type parameter T
val captain = starShip.captain // Error: Cannot resolve symbol
^
<console>:18: error: type mismatch;
found : T{}
required: T
new T {
^
As #dcastro mentioned, you probably wanted a type bound on T. However, even fixing that syntactical error isn't good enough because:
scala> def doSomething[T <: StarShip](starShip : T) : T = new T { val captain = starShip.captain }
<console>:8: error: class type required but T found
It is not possible to instantiate an object from a type parameter because the compiler does not know the actual type to instantiate until runtime. It could be anything, including a type written after the compilation of this function.
Based on your invocation of doSomething, I don't think you want the type parameter or the instantiation at all. That is, this works:
scala> def doSomething[T <: StarShip](starShip : T) : T = { val captain = starShip.captain; starShip }
doSomething: [T <: StarShip](starShip: T)T
scala> doSomething(new Enterprise { val captain = "Kirk" })
res0: Enterprise = $anon$1#26653222
So, I think with the above adjustments you have accomplished your goal. You've modified your trait (StarShip) by insantiating an Enterprise with an overriden value for the captain member.
I'm trying to call this set method documented here, in the Java library jOOQ, with signature:
<T> ... set(Field<T> field, T value)
This Scala line is a problem:
.set(table.MODIFIED_BY, userId)
MODIFIED_BY is a Field<Integer> representing the table column. userId is Int. Predef has an implicit conversion from Int to Integer, so why doesn't it use it? I get this:
type mismatch; found: org.jooq.TableField[gen.tables.records.DocRecord,Integer]
required: org.jooq.Field[Any]
Note: Integer <: Any
(and org.jooq.TableField[gen.tables.records.DocRecord,Integer] <:
org.jooq.Field[Integer]), but Java-defined trait Field is invariant in type T.
You may wish to investigate a wildcard type such as `_ <: Any`. (SLS 3.2.10)
Update - About Vinicius's Example
Rather than try to explain this in comments, here is a demonstration that there is no implicit conversion being called when you use a type with covariant parameter, like List[+T]. Let's say I put this code in a file, compile, and run it...
case class Foo(str: String)
object StackOver1 extends App {
implicit def str2Foo(s: String): Foo = {
println("In str2Foo.")
new Foo(s)
}
def test[T](xs: List[T], x: T): List[T] = {
println("test " + x.getClass)
xs
}
val foo1 = new Foo("foo1")
test(List(foo1), "abc")
}
You'll see that it calls test, but never the implicit conversion from String "abc" to Foo. Instead it's picking a T for test[T] that is a common base class between String and Foo. When you use Int and Integer it picks Any, but it's confusing because the runtime representation of the Int in the list is Integer. So it looks like it used the implicit conversion, but it didn't. You can verify by opening a Scala prompt...
scala> :type StackOver1.test(List(new java.lang.Integer(1)), 2)
List[Any]
I don't know anything aboutjOOQ, but I think the issue is that Scala does not understand java generics very well. Try:
scala> def test[T](a : java.util.ArrayList[T], b: T) = { println(a,b) }
scala> val a = new java.util.ArrayList[Integer]()
scala> val b = 12
scala> test(a,b)
<console>:11: error: type mismatch;
found : java.util.ArrayList[Integer]
required: java.util.ArrayList[Any]
Note: Integer <: Any, but Java-defined class ArrayList is invariant in type E.
You may wish to investigate a wildcard type such as `_ <: Any`. (SLS 3.2.10)
test(a,b)
Sounds familiar??
And to fix, just inform the type T to call the method: test[Integer](a,b) works fine.
EDIT:
There a few things involved here:
Erasure -> When compiled the type of the generic will disappear by erasure. The compiler will use Object which Scala, will treat as Any. However a ArrayList[Integer] is not an ArrayList[Any], even though Integer is any. The same way that TableField[gen.tables.records.DocRecord,Integer] is not a Field[Any].
Type inference mechanism -> it will figure out what type T should be and to do that it will use the intersection dominator of the types passed (in our case the first common ancestor). Page 36 of Scala Language Spec, which in our examples above will lead use to Any.
Implicit conversion -> it is the last step and would be called if there was some type to be converted to another one, but since the type of the arguments were determined to be the first common ancestor, there is no need to convert and we will never have a implicit conversion if we don't force the type T.
A example to show how the common ancestor is used to determine T:
scala> def test[T](a: T, b: T): T = a
scala> class Foo
scala> class Boo extends Foo
scala> test(new Boo,new Foo)
res2: Foo = Boo#139c2a6
scala> test(new Boo,new Boo)
res3: Boo = Boo#141c803
scala> class Coo extends Foo
scala> test(new Boo,new Coo)
res4: Foo = Boo#aafc83
scala> test(new Boo,"qsasad")
res5: Object = Boo#16989d8
Summing up, the implicit method does not get called, because the type inference mechanism, determines the types before getting the argument and since it uses the common ancestor, there is no need for a implicit conversion.
Your code produces an error due to erasure mechanism which disappear with the type information that would be important to determine the correct type of the argument.
#RobN, thanks for questioning my answer, I learned a lot with the process.
I use Jackson Scala Module.
I've created a little serialization tool which handle the json payloads received by Play2 framework.
def unserializePayloadAs[T](implicit requestContext: RequestContext[JsValue]): T = {
val json: String = Json.stringify(requestContext.request.body)
unserialize(json)
}
def unserialize[T](json: String): T = {
objectMapper.readValue(json)
}
The readValue of Jackson Scala Module has the signature:
def readValue[T: Manifest](content: String): T = {
readValue(content, constructType[T])
}
When trying to use my deserialization code, I have a stack.
Caused by: com.fasterxml.jackson.databind.JsonMappingException: Can not instantiate abstract type [simple type, class scala.runtime.Nothing$] (need to add/enable type information?)
at [Source: java.io.StringReader#7bb78579; line: 1, column: 2]
at com.fasterxml.jackson.databind.JsonMappingException.from(JsonMappingException.java:164) ~[jackson-databind-2.2.0.jar:2.2.0]
at com.fasterxml.jackson.databind.deser.std.ThrowableDeserializer.deserializeFromObject(ThrowableDeserializer.java:77) ~[jackson-databind-2.2.0.jar:2.2.0]
at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:121) ~[jackson-databind-2.2.0.jar:2.2.0]
at com.fasterxml.jackson.databind.ObjectMapper._readMapAndClose(ObjectMapper.java:2888) ~[jackson-databind-2.2.0.jar:2.2.0]
at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:2041) ~[jackson-databind-2.2.0.jar:2.2.0]
at com.fasterxml.jackson.module.scala.experimental.ScalaObjectMapper$class.readValue(ScalaObjectMapper.scala:157) ~[jackson-module-scala_2.10-2.2.0.jar:2.2.0]
at utils.CustomSerializer$$anon$1.readValue(CustomSerializer.scala:17) ~[na:na]
at utils.CustomSerializer$.unserialize(CustomSerializer.scala:39) ~[na:na]
at utils.CustomSerializer$.unserializePayloadAs(CustomSerializer.scala:35) ~[na:na]
As expected, it works fine when I add the manifest in my code:
def unserializePayloadAs[T: Manifest](implicit requestContext: RequestContext[JsValue]): T = {
val json: String = Json.stringify(requestContext.request.body)
unserialize(json)
}
def unserialize[T: Manifest](json: String): T = {
objectMapper.readValue(json)
}
Can someone explain what happens there?
When we call a method with a Manifest context bound, with a parametrized method with no Manifest, then the Manifest of Nothing is provided to the first method?
I would have expected some kind of compilation error, telling me I'm calling the readValue with a parametrized type that has no Manifest or something like that, which seems more fail-fast.
Take a slightly simpler example:
def getManifest[A: Manifest] = implicitly[Manifest[A]]
Or the equivalent desugared version:
def getManifest[A](implicit whatever: Manifest[A]) = whatever
And then:
scala> def getIt[T]: Manifest[T] = getManifest
<console>:8: error: type mismatch;
found : Manifest[Nothing]
required: Manifest[T]
Note: Nothing <: T, but trait Manifest is invariant in type T.
You may wish to investigate a wildcard type such as `_ <: T`. (SLS 3.2.10)
def getIt[T]: Manifest[T] = getManifest
^
What's happening is that when the compiler tries to infer the type parameter for getManifest, there are lots of types that have Manifest instances in scope, and it has no way to choose between them, so it goes with its default choice, Nothing.
(You're seeing this error at runtime instead of compile-time because, well, that's what happens when you use reflection-based approaches to serialization.)
We can add a context bound:
def getIt[T: Manifest]: Manifest[T] = getManifest
Or, equivalently:
def getIt[T](implicit whatever: Manifest[T]): Manifest[T] = getManifest
Now everything is fine—there's a single most precise Manifest instance in scope, so the compiler can (appropriately) resolve the type parameter for getManifest to T.
You can see this a little more dramatically in the following example:
scala> trait Foo[A]
defined trait Foo
scala> def getFoo[A: Foo] = implicitly[Foo[A]]
getFoo: [A](implicit evidence$1: Foo[A])Foo[A]
scala> implicit object stringFoo extends Foo[String]
defined module stringFoo
scala> def getIt[T]: Foo[T] = getFoo
<console>:10: error: type mismatch;
found : Foo[String]
required: Foo[T]
def getIt[T]: Foo[T] = getFoo
^
Since there's only one Foo instance in scope (for String), the compiler can pick it when it's resolving the type parameter for getFoo, and we end up with a different type mismatch.
Is it possible to get the type parameters of a manifest type included in the class object.
I might not be describing that correctly, so what I mean is, given this type.
Resultset[Band[Coldplay]]
I want a manfiest that represents the full type, so that it should be possible to get a class instance that has the type
Class[Resultset[Band[Coldplay]]]
All I can get is
Class[Resultset[_]]
You could build a manifest into a class:
case class Manifesting[A](value: A)(implicit val mf: Manifest[A]) { }
scala> Manifesting(5).mf.erasurescala> Manifesting(5).mf.erasure
res1: Class[_] = int
Or you can build all the manifests into a method:
def nested[A, B[A]](x: B[A])(implicit ma: Manifest[A], mb: Manifest[B[A]]) =
(ma.erasure, mb.erasure)
scala> nested(List("fish"))
res2: (Class[_$1], Class[_$1]) forSome { type _$1; type _$1 } =
(class java.lang.String,class scala.collection.immutable.List)
Or, in Scala 2.10 you can use TypeTag:
def nest2[A: scala.reflect.runtime.universe.TypeTag](x: A) =
implicitly[scala.reflect.runtime.universe.TypeTag[A]]
scala> nest2(Right(List("salmon","herring")))
res3: reflect.runtime.universe.TypeTag[scala.util.Right[Nothing,List[String]]] =
TypeTag[scala.util.Right[Nothing,List[String]]]