Calling method via reflection in Scala - scala

I want to call an arbitrary public method of an arbitrary stuff via reflection. I.e. let's say, I want to write method extractMethod to be used like:
class User { def setAvatar(avatar: Avatar): Unit = …; … }
val m = extractMethod(someUser, "setAvatar")
m(someAvatar)
From the Reflection. Overview document from Scala docs, I see the following direct way to do that:
import scala.reflect.ClassTag
import scala.reflect.runtime.universe._
def extractMethod[Stuff: ClassTag: TypeTag](
stuff: Stuff,
methodName: String): MethodMirror =
{
val stuffTypeTag = typeTag[Stuff]
val mirror = stuffTypeTag.mirror
val stuffType = stuffTypeTag.tpe
val methodSymbol = stuffType
.member(TermName(methodName)).asMethod
mirror.reflect(stuff)
.reflectMethod(methodSymbol)
}
However what I'm bothered with this solution is that I need to pass implicit ClassTag[Stuff] and TypeTag[Stuff] parameters (first one is needed for calling reflect, second one — for getting stuffType). Which may be quite cumbersome, especially if extractMethod is called from generics that are called from generics and so on. I'd accept this as necessity for some languages that strongly lack runtime type information, but Scala is based on JRE, which allows to do the following:
def extractMethod[Stuff](
stuff: Stuff,
methodName: String,
parameterTypes: Array[Class[_]]): (Object*) => Object =
{
val unboundMethod = stuff.getClass()
.getMethod(methodName, parameterTypes: _*)
arguments => unboundMethod(stuff, arguments: _*)
}
I understand that Scala reflection allows to get more information that basic Java reflection. Still, here I just need to call a method. Is there a way to somehow reduce requirements (e.g. these ClassTag, TypeTag) of the Scala-reflection-based extractMethod version (without falling back to pure-Java reflection), assuming that performance doesn't matter for me?

Yes, there is.
First, according to this answer, TypeTag[Stuff] is a strictly stronger requirement than ClassTag[Stuff]. Although we don't automatically get implicit ClassTag[Stuff] from implicit TypeTag[Stuff], we can evaluate it manually as ClassTag[Stuff](stuffTypeTag.mirror.runtimeClass(stuffTypeTag.tpe)) and then implicitly or explicitly pass it to reflect that needs it:
import scala.reflect.ClassTag
import scala.reflect.runtime.universe._
def extractMethod[Stuff: TypeTag](
stuff: Stuff,
methodName: String): MethodMirror =
{
val stuffTypeTag = typeTag[Stuff]
val mirror = stuffTypeTag.mirror
val stuffType = stuffTypeTag.tpe
val stuffClassTag = ClassTag[Stuff](mirror.runtimeClass(stuffType))
val methodSymbol = stuffType
.member(TermName(methodName)).asMethod
mirror.reflect(stuff)(stuffClassTag)
.reflectMethod(methodSymbol)
}
Second, mirror and stuffType can be obtained from stuff.getClass():
import scala.reflect.ClassTag
import scala.reflect.runtime.universe._
def extractMethod(stuff: Stuff, methodName: String): MethodMirror = {
val stuffClass = stuff.getClass()
val mirror = runtimeMirror(stuffClass.getClassLoader)
val stuffType = mirror.classSymbol(stuffClass).toType
val stuffClassTag = ClassTag[Stuff](mirror.runtimeClass(stuffType))
val methodSymbol = stuffType
.member(TermName(methodName)).asMethod
mirror.reflect(stuff)(stuffClassTag)
.reflectMethod(methodSymbol)
}
Therefore we obtained Scala-style reflection entities (i.e. finally MethodMirror) without requiring ClassTag and/or TypeTag to be passed explicitly or implicitly from the caller. Not sure, however, how it compares with the ways described in the question (i.e. passing tags from outside and pure Java) in the terms of performance.

Related

Is there anyway, in Scala, to get the Singleton type of something from the more general type?

I have a situation where I'm trying to use implicit resolution on a singleton type. This works perfectly fine if I know that singleton type at compile time:
object Main {
type SS = String with Singleton
trait Entry[S <: SS] {
type out
val value: out
}
implicit val e1 = new Entry["S"] {
type out = Int
val value = 3
}
implicit val e2 = new Entry["T"] {
type out = String
val value = "ABC"
}
def resolve[X <: SS](s: X)(implicit x: Entry[X]): x.value.type = {
x.value
}
def main(args: Array[String]): Unit = {
resolve("S") //implicit found! No problem
}
}
However, if I don't know this type at compile time, then I run into issues.
def main(args: Array[String]): Unit = {
val string = StdIn.readLine()
resolve(string) //Can't find implicit because it doesn't know the singleton type at runtime.
}
Is there anyway I can get around this? Maybe some method that takes a String and returns the singleton type of that string?
def getSingletonType[T <: SS](string: String): T = ???
Then maybe I could do
def main(args: Array[String]): Unit = {
val string = StdIn.readLine()
resolve(getSingletonType(string))
}
Or is this just not possible? Maybe you can only do this sort of thing if you know all of the information at compile-time?
If you knew about all possible implementations of Entry in compile time - which would be possible only if it was sealed - then you could use a macro to create a map/partial function String -> Entry[_].
Since this is open to extending, I'm afraid at best some runtime reflection would have to scan the whole classpath to find all possible implementations.
But even then you would have to embed this String literal somehow into each implementations because JVM bytecode knows nothing about mappings between singleton types and implementations - only Scala compiler does. And then use that to find if among all implementations there is one (and exactly one) that matches your value - in case of implicits if there are two of them at once in the same scope compilation would fail, but you can have more than one implementation as long as the don't appear together in the same scope. Runtime reflection would be global so it wouldn't be able to avoid conflicts.
So no, no good solution for making this compile-time dispatch dynamic. You could create such dispatch yourself by e.g. writing a Map[String, Entry[_]] yourself and using get function to handle missing pices.
Normally implicits are resolved at compile time. But val string = StdIn.readLine() becomes known at runtime only. Principally, you can postpone implicit resolution till runtime but you'll be able to apply the results of such resolution at runtime only, not at compile time (static types etc.)
object Entry {
implicit val e1 = ...
implicit val e2 = ...
}
import scala.reflect.runtime.universe._
import scala.reflect.runtime
import scala.tools.reflect.ToolBox
val toolbox = ToolBox(runtime.currentMirror).mkToolBox()
def resolve(s: String): Any = {
val typ = appliedType(
typeOf[Entry[_]].typeConstructor,
internal.constantType(Constant(s))
)
val instanceTree = toolbox.inferImplicitValue(typ, silent = false)
val instance = toolbox.eval(toolbox.untypecheck(instanceTree)).asInstanceOf[Entry[_]]
instance.value
}
resolve("S") // 3
val string = StdIn.readLine()
resolve(string)
// 3 if you enter S
// ABC if you enter T
// "scala.tools.reflect.ToolBoxError: implicit search has failed" otherwise
Please notice that I put implicits into the companion object of type class in order to make them available in the implicit scope and therefore in the toolbox scope. Otherwise the code should be modified slightly:
object EntryImplicits {
implicit val e1 = ...
implicit val e2 = ...
}
// val instanceTree = toolbox.inferImplicitValue(typ, silent = false)
// should be replaced with
val instanceTree =
q"""
import path.to.EntryImplicits._
implicitly[$typ]
"""
In your code import path.to.EntryImplicits._ is import Main._.
Load Dataset from Dynamically generated Case Class

How can I make a function generic on an MLReader

I am working in Spark 1.6.3. Here are two functions that do the same thing:
def modelFromBytesCV(modelArray: Array[Byte]): CountVectorizerModel = {
val tempPath: Path = KAZOO_TEMP_DIR.resolve(s"model_${System.currentTimeMillis()}")
Files.write(tempPath, modelArray)
CountVectorizerModel.read.load(tempPath.toString)
}
def modelFromBytesIDF(modelArray: Array[Byte]): IDFModel = {
val tempPath: Path = KAZOO_TEMP_DIR.resolve(s"model_${System.currentTimeMillis()}")
Files.write(tempPath, modelArray)
IDFModel.read.load(tempPath.toString)
}
I would like to make these functions generic. What I am hung up on is that the common trait between the CountVectorizerModel object and IDFModel is MLReadable[T] which itself must take as a type either CountVectorizerModel or IDFModel. This is sort of a recursive parent class loop that I can't figure out a solution to.
By comparison, the generic model writer is easy, because MLWritable is a common trait extended by all the models I am interested in:
def modelToBytes[M <: MLWritable](model: M): Array[Byte] = {
val tempPath: Path = KAZOO_TEMP_DIR.resolve(s"model_${System.currentTimeMillis()}")
model.write.overwrite().save(tempPath.toString)
Files.readAllBytes(tempPath)
}
How can I make a generic reader that will turn turn a spark-ml model into a byte array?
To make it work you'll need access to a specific MlReadable object.
import org.apache.spark.ml.util.MLReadable
def modelFromBytes[M](obj: MLReadable[M], modelArray: Array[Byte]): M = {
val tempPath: Path = ???
...
obj.read.load(tempPath.toString)
}
which could be later used as:
val bytes: Array[Byte] = ???
modelFromBytes(CountVectorizerModel, bytes)
Note that, despite the first appearance, there is nothing recursive here - MLReadable[M] refers to companion object, not class as such. So for example CountVectorizerModel object is MLReadable, while CountVectorizeModel class isn't.
Internally, Spark MLReader handles this in a different way - it creates an instance of the class using reflection, and then sets its Params. However this path won't be very useful for you here*.
If compatibility with the current API is required, you can try making readable object implicit:
def modelFromBytes[M](modelArray: Array[Byte])(implicit obj: MLReadable[M]): M = {
...
}
and then
implicit val readable: MLReadable[CountVectorizerModel] = CountVectorizerModel
modelFromBytes[CountVectorizerModel](bytes)
* Technically speaking it is possible to get companion object via reflection
def modelFromBytesCV[M <: MLWritable](
modelArray: Array[Byte])(implicit ct: ClassTag[M]): M = {
val tempPath: Path = ???
...
val cls = Class.forName(ct.runtimeClass.getName + "$");
cls.getField("MODULE$").get(cls).asInstanceOf[MLReadable[M]]
.read.load(tempPath.toString))
}
but I don't think that is a path worth exploring here. In particular we cannot really provide strict type bounds here - using MLWritable is a hack to limit human errors, but is rather useless for compiler.

Typeclass or add a method via implicit?

I'm designing sort of Services and faced with a design issue. Here is what I currently have:
trait Service {
def isFailed(): Boolean
def start(): Unit
def stop(): Unit
}
And in order to group Services related to each other in a group (in order to restart/recover the group, not other services) I created the following package object:
package object app {
type FaultTolerantServiceGroup = Seq[Service]
object FaultTolerantServiceGroup{
def apply(svcs: Service*): FaultTolerantServiceGroup = Seq(svcs: _*)
}
class FaultTolerantServiceGroupOps(val F: FaultTolerantServiceGroup){
def hasFailed: Boolean = F.forall(_.failed())
}
trait FaultTolerantServiceGroupSyntax{
implicit def serviceGroup2Ops(F: FaultTolerantServiceGroup) = new FaultTolerantServiceGroupOps(F)
}
}
So I added the method hasFailed to FaultTolerantServiceGroup. But I'm not sure about this decision.
Would it be better to define a typeclass, say
trait Watchable[T]{
def hasFailed(t: T): Boolean
}
And implicitly provide an instance of Watchable[FaultTolerantServiceGroup]?
In my humble opinion implicit functions become much harder to read afterwards. Even when reading my old code it can sometimes be confusing when objects have methods that appear out of the blue.
I have yet to see an instance where implicits are easier to reason about than declarative functions:
val failedGroup : FaultTolerantServiceGroup => Boolean = _.forall(_.failed())
The resulting code doesn't seem any better, or worse, than implicits but at least it's obvious where functionality is coming from:
val group : FaultTolerantServiceGroup = ???
//no implicit
val failed = failedGroup(group)
//with implicits : how does a Seq have a hasFailed method?
val failed = group.hasFailed
Explicit functions also make Iterable functions easier to read:
val groups : Iterable[FaultTolerantServiceGroup] = ???
val failedGroups = groups filter failedGroup

Proxy class support

Is there Scala way for runtime generation of proxy classes? not DynamicProxy but runtime types that extend provided class/interface and pass all calls through provided callback.
Java world uses cglib/javassist for that, but what is the best way to proxy in Scala?
def makeProxy[T](interceptor: Interceptor, implicit baseClass: Manifest[T]): T
Google says Scala macros can be used for this but I am unsure how.
Here is an example how (more-less) to do something like that with macro:
def testImpl[T : c.WeakTypeTag](c: Context): c.Expr[Any] = {
import c.universe._
val className = newTypeName(weakTypeTag[T].tpe.typeSymbol.name.toString) //not best way
val m = weakTypeOf[T].declarations.iterator.toList.map(_.asMethod) //`declaration` takes only current; `members` also takes inherited
.filter(m => !m.isConstructor && !m.isFinal).map { m => //all reflection info about method
q"""override def ${m.name} = 9""" //generating new method
}
c.Expr { q"""new $className { ..$m } """}
}
def t[T] = macro testImpl[T]
class Aaa{ def a = 7; def b = 8}
scala> t[Aaa].a
res39: Int = 9
scala> t[Aaa].b
res40: Int = 9
All such macro works only if overriden methods are not final as they can't change types (as it works in compile-time) - only create new and inherit. This example doesn't process classes with non-empty constructors and many other things. m here is instance of MethodSymbol and gives you full scala-style reflection about input class' method. You need only generate correct AST in response.
To read more about that:
macroses: http://docs.scala-lang.org/overviews/macros/overview.html
scala's runtime-reflection http://docs.scala-lang.org/overviews/reflection/environment-universes-mirrors.html
q"asiquotes" : http://docs.scala-lang.org/overviews/quasiquotes/expression-details.html
Another solution would be:
scala> def getClasss[T: ClassTag] = classTag[T].runtimeClass
getClasss: [T](implicit evidence$1: scala.reflect.ClassTag[T])Class[_]
Using this instance you can apply any asm/cglib/javassist or even DynamicProxy to it.

Get a class from a type scala

In scala, I want to be able to say
val user = Node.create[User](...) // return User object
So here's what I have so far:
def create[T : TypeTag](map: Map[String, Any]) {
val type = typeOf[T]
// create class from type here???
}
I've been digging around how to create classes from generic types and found out that using ClassManifest seems to be deprecated. Instead, type tags are here, so I'm able to do something like this typeOf[T] and actually get the type.. but then I'm lost. If I could get the class, then I could use something like class.newInstance and manually set the fields from there.
Question is: given a type, can I get a class instance of the given type?
The easiest way in fact is to use ClassTag:
def create[T : ClassTag](map: Map[String, Any]): T = {
val clazz: Class[_] = classTag[T].runtimeClass
clazz.newInstance(<constructor arguments here>).asInstanceOf[T]
}
ClassTag is a thin wrapper around Java Class, primarily used for arrays instantiation.
TypeTag facility is more powerful. First, you can use it to invoke Java reflection:
import scala.reflect.runtime.universe._
def create[T: TypeTag](map: Map[String, Any]): T = {
val mirror = runtimeMirror(getClass.getClassLoader) // current class classloader
val clazz: Class[_] = mirror.runtimeClass(typeOf[T].typeSymbol.asClass)
clazz.newInstance(<constructor arguments here>).asInstanceOf[T]
}
However, Scala reflection allows to instantiate classes without dropping back to Java reflection:
def create[T: TypeTag](map: Map[String, Any]): T = {
// obtain type symbol for the class, it is like Class but for Scala types
val typeSym = typeOf[T].typeSymbol.asClass
// obtain class mirror using runtime mirror for the given classloader
val mirror = runtimeMirror(getClass.getClassLoader) // current class classloader
val cm = mirror.reflectClass(typeSym)
// resolve class constructor using class mirror and
// a constructor declaration on the type symbol
val ctor = typeSym.decl(termNames.CONSTRUCTOR).asMethod
val ctorm = cm.reflectConstructor(cm)
// invoke the constructor
ctorm(<constructor arguments here>).asInstanceOf[T]
}
If you want to create a class with overloaded constructors, it may require more work though - you'll have to select correct constructor from declarations list, but the basic idea is the same. You can read more on Scala reflection here
There is a way to do it with reflection: either runtime reflection, or in a macro. Regarding runtime reflection way, you can have a look at my blog post where I tried to do something like what you are trying to do now. Using compile-time reflection with macros might be a better option, depending on your need.