For purposes of my app I need to be able to find out a list of fields of a type (not an instance) and types of those fields in runtime. So far I was only able to get a list of methods of a case class containing getters with classOf[MyCaseClass].getMethods and nothing useful from a simple class. Am I missing something? Are there any reflection libraries for that kinda purposes? How's that done correctly?
Using Scala 2.10 reflection:
scala> import reflect.runtime.{universe => ru}
import reflect.runtime.{universe=>ru}
scala> trait A { val field1: Int; val field2: Char; def meth1: Int }
defined trait A
scala> val fieldSymbols = ru.typeOf[A].members.collect{ case m: ru.MethodSymbol if m.isGetter => m }
fieldSymbols: Iterable[reflect.runtime.universe.MethodSymbol] = List(value field2, value field1)
The returned symbols contain all the type information, e.g.:
scala> fieldSymbols.map(_.typeSignature)
res16: Iterable[reflect.runtime.universe.Type] = List(=> scala.Char, => scala.Int)
You may want to take a look at this document on reflecting scala. getMethods is a method from Java reflection. What can't you find there? From the Javadoc:
String getName(): Returns the name of the method represented by this Method object, as a String.
Class[] getParameterTypes(): Returns an array of Class objects that represent the formal parameter types, in declaration order, of the method represented by this Method object.
Class getReturnType(): Returns a Class object that represents the formal return type of the method represented by this Method object.
You could read more about Java reflection.
Note that not all type information will be available at runtime because of erasure.
Related
I'm trying to compare the class type of a SparkSQL query.
scala> val plan = spark.sessionState.sqlParser.parsePlan("create table some_table as select 1")
scala> val childClass = plan.children.map(_.getClass).toList(0)
Class[_ <: org.apache.spark.sql.catalyst.plans.logical.LogicalPlan] = class org.apache.spark.sql.catalyst.plans.logical.Project
So looks like its of the type class org.apache.spark.sql.catalyst.plans.logical.Project, however, I get false when I compare this using isInstanceOf.
scala> childClass.isInstanceOf[org.apache.spark.sql.catalyst.plans.logical.Project]
^
res9: Boolean = false
I think the problem here is how the class org.apache.spark.sql.catalyst.plans.logical.Project is being compiled.
scala> org.apache.spark.sql.catalyst.plans.logical.Project.getClass
res21: Class[_ <: org.apache.spark.sql.catalyst.plans.logical.Project.type] = class org.apache.spark.sql.catalyst.plans.logical.Project$
The $ sign at the end is causing issues. However, if I do childClass.getName and do a string compare with the class, I get true but I don't think that's a good way of doing a class comparison here.
This is more of a scala question than a Spark question and the problem seems trivial but I can't seem to figure out how to compare the class type without comparing the raw string name itself.
org.apache.spark.sql.catalyst.plans.logical.Project.getClass calls the getClass method of the companion object for org.apache.spark.sql.catalyst.plans.logical.Project which is a singleton instance of the class org.apache.spark.sql.catalyst.plans.logical.Project$ (in this case, looking at Spark's code shows that to be a case class with a synthetic companion object).
You can get the class object for org.apache.spark.sql.catalyst.plans.logical.Project with:
classOf[org.apache.spark.sql.catalyst.plans.logical.Project]
i.e.
childClass.getClass == classOf[org.apache.spark.sql.catalyst.plans.logical.Project]
should be true.
I'm trying to use discriminators in existing project and something is wrong with my classes I guess.
Consider this scodec example. If I change TurnLeft and its codec to
sealed class TurnLeft(degrees: Int) extends Command {
def getDegrees: Int = degrees
}
implicit val leftCodec: Codec[TurnLeft] = uint8or16.xmap[TurnLeft](v => new TurnLeft(v), _.getDegrees)
I get
Error:(x, x) could not find Lazy implicit value of type scodec.Codec[Command]
val codec: Codec[Either[UnrecognizedCommand, Command]] = discriminatorFallback(unrecognizedCodec, Codec[Command])
It all works if I make degrees field value field. I suspect it's something tricky with shapeless. What should I do to make it work ?
Sample project that demonstrates the issue is here.
shapeless's Generic is defined for "case-class-like" types. To a first approximation, a case-class-like type is one whose values can be deconstructed to it's constructor parameters which can then be used to reconstruct an equal value, ie.
case class Foo ...
val foo = Foo(...)
val fooGen = Generic[Foo]
assert(fooGen.from(fooGen.to(foo)) == foo)
Case classes with a single constructor parameter list meet this criterion, whereas classes which don't have public (lazy) vals for their constructor parameters, or a companion with a matching apply/unapply, do not.
The implementation of Generic is fairly permissive, and will treat (lazy) val members which correspond to constructor parameters (by type and order) as being equivalent to accessible constructor arguments, so the closest to your example that we can get would be something like this,
sealed class TurnLeft(degrees: Int) extends Command {
val getDegrees: Int = degrees
}
scala> Generic[TurnLeft]
res0: shapeless.Generic[TurnLeft]{type Repr = Int :: HNil } = ...
In this case getDegrees is treated as the accessor for the single Int constructor parameter.
In the below code, I try to invoke an object's method that has an Int parameter (giving it a value of 3). This returns an error that Int and 3 are incompatible types.
//Using scala's Int does not work!
object MyObject{
def handleInt(id:Int) : Boolean = {
true
}
}
object testApp extends App {
val obj = MyObject.getClass
val method = obj.getDeclaredMethod("handleInt", classOf[Int]) //Int.getClass shows the same behavior
val rsp = method.invoke(obj, 3)
}
Error:(106, 41) the result type of an implicit conversion must be more
specific than AnyRef
val rsp = method.invoke(obj, 3)
Error:(106, 41) type mismatch; found : Int(3) required: Object
val rsp = method.invoke(obj, 3)
I tried modifying a lot of things here, the only way this could work is by changing all signatures to Java's Integer. The code will look like this:
//This works with Java's Integer
object MyObject{
def handleInt(id:Integer) : Boolean = {
true
}
}
object testApp extends App {
val obj = MyObject.getClass
val method = obj.getDeclaredMethod("handleInt", classOf[Integer])
val rsp = method.invoke(obj, 3)
}
My question(s) are:
Can someone explain why this happens? I think scala's Int wraps java's primitive int (which is why this is not considered an object), but I'm not sure.
Is there a way to achieve this using Scala's Int type?
Is it acceptable to mix scala and java types like this? Is it a good practice?
The first problem is that you invoke method on the wrong object: obj doesn't have handleInt method, MyObject does. The second is kind of an edge case: invoke takes Object... varargs and Scala doesn't automatically convert an Int directly to Object because that's not what you normally want to do. You can use a type ascription to say "box this Int into an Integer" and then Scala will upcast it to Object automatically.
So, combining these 2 fixes: you don't need to change handleInt or val method, just
val rsp = method.invoke(MyObject, 3: Integer)
To answer your final question: use of Integer, java.lang.Double, etc. shouldn't be common in Scala code, but it isn't particularly problematic. And reflection is one of the areas where you may have to use them.
No we cannot use Scala types.
Its Ok to mix Java and Scala types.
As reflection deals with reading class bytecode at Runtime. At bytecode level only types that are visible are the Java types because all scala types are boiled down to Java types.
I'm trying to convert a type tag into a java class that maintains/persists normally-erased type parameters. There are quite a few libraries that benefit from conversions like these (such as Jackson, and Guice). I'm currently trying to migrate Manifest based code to TypeTag since Manifests are insufficient for some corner cases.
The JVM treats Arrays special in comparison to other data types. The difference between a classOf[Int] and classOf[Array[Int]] is that the method Class.isArray() will return true for the latter.
The Manifest implementation was simple. Manifest.erasure was a Class instance where isArray() was already valid/true.
The TypeTag implementation is trickier. There is no quick and easy erasure method. In fact the 'similar' TypeTag variant, RuntimeMirror.runtimeClass, prefers not to handle creating any Array based classes on our behalf. Read the documentation:
Note: If the Scala symbol is ArrayClass, a ClassNotFound exception is
thrown because there is no unique Java class corresponding to a Scala
generic array
To work around this I try to detect if it is an Array. If it is an array, then I manually create the class object. However, I've come across an additional edge case when Array has an unknown type argument.
First let me show you an example that is not a HigherKinded type.
import scala.reflect.runtime.universe._
class A[T]
val innerType = typeOf[A[Array[_]]].asInstanceOf[TypeRefApi].args.head
innerType <:< typeOf[Array[_]] // Returns true.
So far so good.
class B[T[_]]
val innerType = typeOf[B[Array]].asInstanceOf[TypeRefApi].args.head
innerType <:< typeOf[Array[_]] // Returns false.
I can't create a typeOf[Array] since it complains about the missing parameter. How can I detect that B has an type parameter of Array?
Also, what would the class instance look like in this case? Is it an Array[Object]?
Decompose again:
scala> innerType match { case TypeRef(pre, sym, args) => sym == definitions.ArrayClass }
res13: Boolean = true
That might get you part-way.
Another way is to compare typeConstructors:
import scala.reflect.runtime.universe._
class B[T[_]]
val innerType = typeOf[B[Array]].asInstanceOf[TypeRefApi].args.head
innerType.typeConstructor =:= typeOf[Array[_]].typeConstructor
innerType: reflect.runtime.universe.Type = Array
res4: Boolean = true
This also works in general when we need to detect the Type is an Array (of any type).
The try to get erased type of such innerType (to compare) fails for Array (while works for others HigherKinded types):
class B[T[_]]
val innerType = typeOf[B[Array]].typeArgs.head
innerType.typeArgs
innerType.erasure // fails
innerType.erasure =:= typeOf[Array[_]].erasure
defined class B
innerType: reflect.runtime.universe.Type = Array
res4: List[reflect.runtime.universe.Type] = List()
java.util.NoSuchElementException: head of empty list
at scala.collection.immutable.Nil$.head(List.scala:431)
at scala.collection.immutable.Nil$.head(List.scala:428)
at scala.reflect.internal.transform.Erasure$ErasureMap.apply(Erasure.scala:126)
at scala.reflect.internal.transform.Transforms$class.transformedType(Transforms.scala:43)
at scala.reflect.internal.SymbolTable.transformedType(SymbolTable.scala:16)
at scala.reflect.internal.Types$TypeApiImpl.erasure(Types.scala:225)
at scala.reflect.internal.Types$TypeApiImpl.erasure(Types.scala:218)
... 36 elided
Suppose I have:
class X
{
val listPrimitive: List[Int] = null
val listX: List[X] = null
}
and I print out the return types of each method in Scala as follows:
classOf[ComplexType].getMethods().foreach { m => println(s"${m.getName}: ${m.getGenericReturnType()}") }
listPrimitive: scala.collection.immutable.List<Object>
listX: scala.collection.immutable.List<X>
So... I can determine that the listX's element type is X, but is there any way to determine via reflection that listPrimitive's element type is actually java.lang.Integer? ...
val list:List[Int] = List[Int](123);
val listErased:List[_] = list;
println(s"${listErased(0).getClass()}") // java.lang.Integer
NB. This seems not to be an issue due to JVM type erasure since I can find the types parameter of List. It looks like the scala compiler throws away this type information IFF the parameter type is java.lang.[numbers] .
UPDATE:
I suspect this type information is available, due to the following experiment. Suppose I define:
class TestX{
def f(x:X):Unit = {
val floats:List[Float] = x.listPrimitive() // type mismatch error
}
}
and X.class is imported via a jar. The full type information must be available in X.class in order that this case correctly fails to compile.
UPDATE2:
Imagine you're writing a scala extension to a Java serialization library. You need to implement a:
def getSerializer(clz:Class[_]):Serializer
function that needs to do different things depending on whether:
clz==List[Int] (or equivalently: List[java.lang.Integer])
clz==List[Float] (or equivalently: List[java.lang.Float])
clz==List[MyClass]
My problem is that I will only ever see:
clz==List[Object]
clz==List[Object]
clz==List[MyClass]
because clz is provided to this function as clz.getMethods()(i).getGenericReturnType().
Starting with clz:Class[_] how can I recover the element type information that was lost?
Its not clear to me that TypeToken will help me because its usages:
typeTag[T]
requires that I provide T (ie. at compile time).
So, one path to a solution... Given some clz:Class[_], can I determine the TypeTokens of its method's return types? Clearly this is possible as this information must be contained (somewhere) in a .class file for a scala compiler to correctly generate type mismatch errors (see above).
At the java bytecode level Ints have to be represented as something else (apparently Object) because a List can only contain objects, not primitives. So that's what java-level reflection can tell you. But the scala type information is, as you infer, present (at the bytecode level it's in an annotation, IIRC), so you should be able to inspect it with scala reflection:
import scala.reflect.runtime.universe._
val list:List[Int] = List[Int](123)
def printTypeOf[A: TypeTag](a: A) = println(typeOf[A])
printTypeOf(list)
Response to update2: you should use scala reflection to obtain a mirror, not the Class[_] object. You can go via the class name if need be:
import scala.reflect.runtime.universe._
val rm = runtimeMirror(getClass.getClassLoader)
val someClass: Class[_] = ...
val scalaMirrorOfClass = rm.staticClass(someClass.getName)
// or possibly rm.reflectClass(someClass) ?
val someObject: Any = ...
val scalaMirrorOfObject = rm.reflectClass(someObject)
I guess if you really only have the class, you could create a classloader that only loads that class? I can't imagine a use case where you wouldn't have the class, or even a value, though.