I am wondering if it is possible to instantiate an object (companion object) dynamically using Manifest. I want to parse json in MongoRecord but to do so, I have to understand which is the type that is passed.
def getCompanion[T](implicit mf : Manifest[T])={
if (mf <:< classOf[MongoRecord[C]]){
Class[C].asInstanceOf[MongoRecord].setFieldsFromJSON(request.body.toString)
}
}
but I am receiving an error during compilation:
error: object Class is not a value
Class[C].asInstanceOf[MongoRecord].setFieldsFromJSON(request.body.toString)
it is a difficult topic to me. Perhaps it is not feasible but I would like to know if it is possible?
Thanks
Related
Recently, I am reading the source code of Spray-json. I noted that the following hierarchy relation in JsonFormat.scala, please see below code snippet
/**
* A special JsonFormat signaling that the format produces a legal JSON root
* object, i.e. either a JSON array
* or a JSON object.
*/
trait RootJsonFormat[T] extends JsonFormat[T] with RootJsonReader[T] with RootJsonWriter[T]
To express the confusion more convenient, I draw the following diagram of hierarchy:
According to my limited knowledge of Scala, I think the JsonFormat[T] with should be removed from the above code. Then I cloned the repository of Spary-json, and comment the code JsonFormat[T] with
trait RootJsonFormat[T] extends RootJsonReader[T] with RootJsonWriter[T]
Then I compile it in SBT(use package/compile command) and it passed to the compiling process and generates a spray-json_2.11-1.3.4.jar successfully.
However, when I run the test cases via test command of SBT, it failed.
So I would like to know why. Thanks in advance.
I suggest you to not think of it in terms of OOP. Think of it in terms of type classes. In case when some entity must be serialized and deserialized at the same time, there is a type class JsonFormat that includes both JsonWriter and JsonReader. This is convenient since you don't need to search for 2 type class instances when you need both capabilities. But in order for this approach to work, there has to be an instance of JsonFormat type class. This is why you can't just throw it away from hierarchy. For instance:
def myMethod[T](t: T)(implicit format: JsonFormat[T]): Unit = {
format.read(format.write(t))
}
If you want this method to work properly there has to be a direct descendant of JsonFormat and a concrete implicit instance of it for a specific type T.
UPD: By creating an instance of the JsonFormat type class, you get instances for JsonWriter and JsonReader type classes automatically (in case when you need both). So this is also a way to reduce boilerplate.
I have an exception:
java.lang.ClassCastException: scala.collection.immutable.Map$ cannot
be cast to scala.collection.immutable.Map
which i'm getting in this part of code:
val iterator = new CsvMapper()
.disable(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES)
.readerFor(Map.getClass).`with`(CsvSchema.emptySchema().withHeader()).readValues(reader)
while (iterator.hasNext) {
println(iterator.next.asInstanceOf[Map[String, String]])
}
So, are there any options to avoid this issue, because this:
val iterator = new CsvMapper()
.disable(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES)
.readerFor(Map[String,String].getClass).`with`(CsvSchema.emptySchema().withHeader()).readValues(reader)
doesn't help, because I get
[error] Unapplied methods are only converted to functions when a function type is expected.
[error] You can make this conversion explicit by writing `apply _` or `apply(_)` instead of `apply`.
Thanks in advance
As has been pointed out in the earlier comments, in general you need classOf[X[_,_]] rather than X.getClass or X[A, B].getClass for a class that takes two generic types. (instance.getClass retrieves the class of the associated instance; classOf[X] does the same for some type X when an instance isn't available. Since Map is an object and objects are also instances, it retrieves the class type of the object Map - the Map trait's companion.)
However, a second problem here is that scala.collection.immutable.Map is abstract (it's actually a trait), and so it cannot be instantiated as-is. (If you look at the type of Scala Map instances created via the companion's apply method, you'll see that they're actually instances of classes such as Map.EmptyMap or Map.Map1, etc.) As a consequence, that's why your modified code still produced an error.
However, the ultimate problem here is that you required - as you mentioned - a Java java.util.Map and not a Scala scala.collections.immutable.Map (which is what you'll get by default it you just type Map in a Scala program). Just one more thing to watch out for when converting Java code examples to Scala. ;-)
I'm trying to read some data from hadoop into an RDD in Spark using the interactive Scala shell but I'm having trouble accessing some of the classes I need to deserialise the data.
I start by importing the necessary class
import com.example.ClassA
Which works fine. ClassA is located in a jar in the 'jars' path and has ClassB as a public static nested class
I'm then trying to use ClassB like so:
val rawData = sc.newAPIHadoopFile(dataPath, classOf[com.exmple.mapreduce.input.Format[com.example.ClassA$ClassB]], classOf[org.apache.hadoop.io.LongWritable], classOf[com.example.ClassA$ClassB])
This is slightly complicated by one of the other classes taking ClassB as a type, but I think that should be fine.
When I execute this line, I get the following error:
<console>:17: error: type ClassA$ClassB is not a member of package com.example
I have also tried using the import statement
import com.example.ClassA$ClassB
and it also seems fine with that.
Any advice as to how I could proceed to debug this would be appreciated
Thanks for reading.
update:
Changing the '$' to a '.' to reference the nested class seems to get past this problem, although I then got the following syntax error:
'<console>:17: error: inferred type arguments [org.apache.hadoop.io.LongWritable,com.example.ClassA.ClassB,com.example.mapreduce.input.Format[com.example.ClassA.ClassB]] do not conform to method newAPIHadoopFile's type parameter bounds [K,V,F <: org.apache.hadoop.mapreduce.InputFormat[K,V]]
Notice the types that the newAPIHadoopFile expects:
K,V,F <: org.apache.hadoop.mapreduce.InputFormat[K,V]
the important part here is that the generic type InputFormat expects the types K and V, i.e. the exact types of the first two parameters to the method.
In your case, the third parameter should be of type
F <: org.apache.hadoop.mapreduce.InputFormat[LongWritable, ClassA.ClassB]
does your class extend FileInputFormat<LongWritable, V>?
I have a function with the following signature:
myFunc[T <: AnyRef](arg: T)(implicit m: Manifest[T]) = ???
How can I invoke this function if I do not know the exact type of the argument at the compile time?
For example:
val obj: AnyRef = new Foo() // At compile time obj is defined as AnyRef,
val objClass = obj.getClass // At runtime I can figure out that it is actually Foo
// Now I would need to call `myFunc[Foo](obj.asInstanceOf[Foo])`,
// but how would I do it without putting [Foo] in the square braces?
I would want to write something logically similar to:
myFunc[objClass](obj.asInstanceOf[objClass])
Thank you!
UPDATE:
The question is invalid - As #DaoWen, #Jelmo and #itsbruce correctly pointed, the thing I was trying to do was a complete nonsense! I just overthought the problem severely.
THANK YOU guys! It's too bad I cannot accept all the answers as correct :)
So, the problem was caused by the following situation:
I am using Salat library to serialize the objects to/from BSON/JSON representation.
Salat has an Grater[T] class which is used for both serialization and deserialization.
The method call for deserialization from BSON looks this way:
val foo = grater[Foo].asObject(bson)
Here, the role of type parameter is clear. What I was trying to do then is to use the same Grater to serialize any entity from my domain model. So I wrote:
val json = grater[???].toCompactJSON(obj)
I immediately rushed for reflection and just didn't see an obvious solution lying on the surface. Which is:
grater[Entity].toCompactJSON(obj) // where Entity...
#Salat trait Entity // is a root of the domain model hierarchy
Sometimes things are much easier than we think they are! :)
It appears that while I was writing this answer the author of the question realized that he does not need to resolve Manifests at runtime. However, in my opinion it is perfectly legal problem which I resolved successfully when I was writing Yaml [de]serialization library, so I'm leaving the answer here.
It is possible to do what you want using ClassTags or even TypeTags. I don't know about Manifests because that API is deprecated and I haven't worked with it, but I believe that with manifests it will be easier since they weren't as sophisticated as new Scala reflection. FYI, Manifest's successor is TypeTag.
Suppose you have the following functions:
def useClasstag[T: ClassTag](obj: T) = ...
def useTypetag[T: TypeTag](obj: T) = ...
and you need to call then with obj: AnyRef as an argument while providing either ClassTag or TypeTag for obj.getClass class as the implicit parameter.
ClassTag is the easiest one. You can create ClassTag directly from Class[_] instance:
useClasstag(obj)(ClassTag(obj.getClass))
That's all.
TypeTags are harder. You need to use Scala reflection to obtain one from the object, and then you have to use some internals of Scala reflection.
import scala.reflect.runtime.universe._
import scala.reflect.api
import api.{Universe, TypeCreator}
// Obtain runtime mirror for the class' classloader
val rm = runtimeMirror(obj.getClass.getClassLoader)
// Obtain instance mirror for obj
val im = rm.reflect(obj)
// Get obj's symbol object
val sym = im.symbol
// Get symbol's type signature - that's what you really want!
val tpe = sym.typeSignature
// Now the black magic begins: we create TypeTag manually
// First, make so-called type creator for the type we have just obtained
val tc = new TypeCreator {
def apply[U <: Universe with Singleton](m: api.Mirror[U]) =
if (m eq rm) tpe.asInstanceOf[U # Type]
else throw new IllegalArgumentException(s"Type tag defined in $rm cannot be migrated to other mirrors.")
}
// Next, create a TypeTag using runtime mirror and type creator
val tt = TypeTag[AnyRef](rm, tc)
// Call our method
useTypetag(obj)(tt)
As you can see, this machinery is rather complex. It means that you should use it only if you really need it, and, as others have said, the cases when you really need it are very rare.
This isn't going to work. Think about it this way: You're asking the compiler to create a class Manifest (at compile time!) for a class that isn't known until run time.
However, I have the feeling you're approaching the problem the wrong way. Is AnyRef really the most you know about the type of Foo at compile time? If that's the case, how can you do anything useful with it? (You won't be able to call any methods on it except the few that are defined for AnyRef.)
It's not clear what you are trying to achieve and a little more context could be helpful. Anyway, here's my 2 cents.
Using Manifest will not help you here because the type parameter needs to be known at compile time. What I propose is something along these lines:
def myFunc[T](arg: AnyRef, klass: Class[T]) = {
val obj: T = klass.cast(arg)
//do something with obj... but what?
}
And you could call it like this:
myFunc(obj, Foo.class)
Note that I don't see how you can do something useful inside myFunc. At compile time, you cannot call any method on a object of type T beside the methods available for AnyRef. And if you want to use reflection to manipulate the argument of myFunc, then there is no need to cast it to a specific type.
This is the wrong way to work with a type-safe OO language. If you need to do this, your design is wrong.
myFunc[T <: AnyRef](arg: T)(implicit m: Manifest[T]) = ???
This is, of course, useless, as you have probably discovered. What kind of meaningful function can you call on an object which might be anything? You can't make any direct reference to its properties or methods.
I would want to write something logically similar to:
myFunc[objClass](obj.asInstanceOf[objClass])
Why? This kind of thing is generally only necessary for very specialised cases. Are you writing a framework that will use dependency injection, for example? If you're not doing some highly technical extension of Scala's capabilities, this should not be necessary.
I bet you know something more about the class, since you say you don't know the exact type. One big part of the way class-based OO works is that if you want to do something to a general type of objects (including all its subtypes), you put that behaviour into a method belonging to the class. Let subclasses override it if they need to.
Frankly, the way to do what you are attempting is to invoke the function in a context where you know enough about the type.
I am wondering what type do I put in place of XXX
def registerClass(cl:XXX) = kryo.register(classOf[cl])
EDIT: For why I want to do this.
I have to register many classes using the above code. I wanted to remove the duplication of calling kyro.register several times, hoping to write code like below:
Seq(com.mypackage.class1,com.mypackage.class2,com.mypackage.class3).foreach(registerClass)
Another question, can I pass String instead? and convert it somehow to a class in registerClass?
Seq("com.mypackage.class1","com.mypackage.class2").foreach(registerClass)
EDIT 2:
When I write com.mypackage.class1, it means any class defined in my source. So if I create a class
package com.mypackage.model
class Dummy(val ids:Seq[Int],val name:String)
I would provide com.mypackage.model.Dummy as input
So,
kryo.register(classOf[com.mypackage.model.Dummy])
Kryo is a Java Serialization library. The signature of the register class is
register(Class type)
You could do it like this:
def registerClass(cl:Class[_]) = kryo.register(cl)
And then call it like this:
registerClass(classOf[Int])
The type parameter to classOf needs to be known at compile time. Without knowing more about what you're trying to do, is there any reason you can't use:
def registerClass(cl:XXX) = kryo.register(cl.getClass)