I have a class TestClass with a companion object. How can I access a private field say xyz in the companion object using runtime reflection in scala when that private field is set from within the class as shown below.
class TestClass { TestClass.xyz = 100 }
object TestClass { private var xyz: Int = _ }
I tried the following
import scala.reflect.runtime.{currentMirror, universe => ru}
val testModuleSymbol = ru.typeOf[TestClass.type].termSymbol.asModule
val moduleMirror = currentMirror.reflectModule(testModuleSymbol)
val instanceMirror = currentMirror.reflect(moduleMirror.instance)
val xyzTerm = ru.typeOf[TestClass.type].decl(ru.TermName("xyz")).asTerm.accessed.asTerm
val fieldMirror = instanceMirror.reflectField(xyzTerm)
val context = fieldMirror.get.asInstanceOf[Int]
But I was getting the below error.
scala> val fieldMirror = instanceMirror.reflectField(xyzTerm)
scala.ScalaReflectionException: Scala field xyz of object TestClass isn't represented as a Java field, nor does it have a
Java accessor method. One common reason for this is that it may be a private class parameter
not used outside the primary constructor.
at scala.reflect.runtime.JavaMirrors$JavaMirror.scala$reflect$runtime$JavaMirrors$JavaMirror$$abort(JavaMirrors.scala:115)
at scala.reflect.runtime.JavaMirrors$JavaMirror.scala$reflect$runtime$JavaMirrors$JavaMirror$$ErrorNonExistentField(JavaMirrors.scala:127)
at scala.reflect.runtime.JavaMirrors$JavaMirror$JavaInstanceMirror.reflectField(JavaMirrors.scala:242)
at scala.reflect.runtime.JavaMirrors$JavaMirror$JavaInstanceMirror.reflectField(JavaMirrors.scala:233)
... 29 elided
This exception is thrown only when I refer the variable xyz in the TestClass (ie TestClass.xyz = 100). If this reference is removed from the class than my sample code works just fine.
Got this to work:
import scala.reflect.runtime.universe._
import scala.reflect.runtime.{universe => ru}
val runMirror = ru.runtimeMirror(getClass.getClassLoader)
val objectDef = Class.forName("org.myorg.TestClass")
val objectTypeModule = runMirror.moduleSymbol(objectDef).asModule
val objectType = objectTypeModule.typeSignature
val methodMap = objectType.members
.filter(_.isMethod)
.map(d => {
d.name.toString -> d.asMethod
})
.toMap
// get the scala Object
val instance = runMirror.reflectModule(objectTypeModule).instance
val instanceMirror = runMirror.reflect(instance)
// get the private value
val result = instanceMirror.reflectMethod(methodMap("xyz")).apply()
assert(result == 100)
Related
I want to register a scala Object Method into spark udf. Now i get the MethodMirror by scala reflect and Parameter by java reflect. But i cannot generate a Function object used to register into spark.udf. Such as:
object ArrayUdfs {
def array2String(arr: Seq[Long])= {
arr.mkString(",")
}
}
I want to register method 'array2String' into spark udf.
First i get the MethodMirror:
def getObjectMethod(clazzPath:String, methodName:String) = {
import scala.reflect.runtime.universe
lazy val runtimeMirror = universe.runtimeMirror(getClass.getClassLoader)
lazy val module = runtimeMirror.staticModule(clazzPath)
lazy val obj = runtimeMirror.reflectModule(module)
lazy val objMirror = runtimeMirror.reflect(obj.instance)
lazy val method = obj.symbol.typeSignature.member(universe.TermName(methodName)).asMethod
lazy val methodObject = objMirror.reflectMethod(method)
methodObject
}
lazy val method = getObjectMethod(clazzPath, funName);
Second. I get Parameter.
def getObjectMethodParams(clazzPath:String, methodName:String): Array[Parameter] = {
val methods = Class.forName(clazzPath).getDeclaredMethods()
var params: Array[Parameter] = null;
methods.foreach(method => {
if (method.getName == methodName) params = method.getParameters
})
params
}
val params = getObjectMethodParams(clazzPath, funName)
Third, register it into spark.udf
val function1: Function1[Seq[String], String] = (arr) => {
method.apply(arr).asInstanceOf[String]
}
spark.udf.register("array2String", function1)
So i want to replace 'Seq[String]' in 'Function1[Seq[String], String]' by 'params' object. The 'String' in 'Function1[Seq[String], String]' is same.
I need some education for class, object with Serialization
Say, I have a spark main job, which maps a dataframe to another dataframe:
def main(args: Array[String]){
val ss = SparkSession.builder
.appName("test")
.getOrCreate()
val mydf = ss.read("myfile")
// if call from object
val newdf = mydf.map(x=>Myobj.myfunc(x))
//if call from class
val myclass = new Myclass()
val newdf = mydf.map(x=>myclass.myfunc(x))
}
object Myobj {
def myfunc(x:Int):Int = {
x + 1
}
}
class Myclass{
def myfunc(x:Int):Int = {
x + 1
}
}
My questions are:
Which closure should I use to define myfunc within? an object or a class? What is the difference in terms of performance.
Should I extends Serializable for the object or class. Why?
I want to print/log some message from the object/class, what should I do?
Thanks
Given an anonymous object:
val anon = new {
val a = BigDecimal(1)
}
How can I use scala reflection to get value of a ?
I have tried using java reflection, it is trivial. But with scala reflection, it is not obvious.
Here is what I have tried:
package test
object ReflectTest extends App {
val anon = new {
val a = BigDecimal(1)
}
val instanceMirror = currentMirror.reflect(anon)
val anonType = typeOf[anon.type]
val anonTermSymbol = anonType.member(newTermName("a")).asTerm
val anonFieldMirror = instanceMirror.reflectField(anonTermSymbol)
val result = anonFieldMirror.get
println(result)
}
But encountered an exception:
Exception in thread "main" scala.ScalaReflectionException: expected a member of anonymous class $anon$1, you provided value test.ReflectTest.<refinement>.a
at scala.reflect.runtime.JavaMirrors$JavaMirror.scala$reflect$runtime$JavaMirrors$JavaMirror$$ErrorNotMember(JavaMirrors.scala:130)
at scala.reflect.runtime.JavaMirrors$JavaMirror$$anonfun$scala$reflect$runtime$JavaMirrors$JavaMirror$$checkMemberOf$1.apply(JavaMirrors.scala:225)
at scala.reflect.runtime.JavaMirrors$JavaMirror.ensuringNotFree(JavaMirrors.scala:214)
at scala.reflect.runtime.JavaMirrors$JavaMirror.scala$reflect$runtime$JavaMirrors$JavaMirror$$checkMemberOf(JavaMirrors.scala:224)
at scala.reflect.runtime.JavaMirrors$JavaMirror$JavaInstanceMirror.reflectField(JavaMirrors.scala:247)
at scala.reflect.runtime.JavaMirrors$JavaMirror$JavaInstanceMirror.reflectField(JavaMirrors.scala:243)
It seems that the runtime type is not the one recognized by scala reflection.
I have an array of quasi quotes called definitions which I want to inject to the quasi quote tree. How do I go about doing this?
private def generateDaoComponent(files: Array[File]) = {
val file = createNewFile(compDirectory)
val definitions = files.map(f => {
val daoName = f.getName.replace(".java", "")
val daoType = TypeName(daoName)
val daoTerm = TermName(daoName)
q"""def $daoTerm = getValueOrInstantiate($daoName, () => new $daoType(configuration))
"""
})
val tree = q"""
package database.dao {
import org.jooq.SQLDialect
import org.jooq.impl.DefaultConfiguration
import utility.StrongHashMap
trait $componentType extends StrongHashMap[String, Dao] {
this: DaoManager =>
private lazy val configuration = new DefaultConfiguration().set(connection).set(SQLDialect.POSTGRES_9_4)
${definitions.foreach(f => q"${f}")}
}
}"""
writeToFile(file, tree)
}
This is some crazy late night coding, but I found it thanks to this website
3 approaches to Scala code generation
I noticed when it passed the $params array into the quasi quote it used two .. in front of the class constructor like this:
val params = schema.fields.map { field =>
val fieldName = newTermName(field.name)
val fieldType = newTypeName(field.valueType.fullName)
q"val $fieldName: $fieldType"
}
val json = TypeSchema.toJson(schema)
// rewrite the class definition
c.Expr(
q"""
case class $className(..$params) {
def schema = ${json}
}
"""
)
There are two steps to the code I posted in the question to make this working.
1) Convert $definitions.toList to List
2) Add the two .. in front
So the final code looks like this:
val definitions = files.map(f => {
val daoName = f.getName.replace(".java", "")
val daoType = TypeName(daoName)
val daoTerm = TermName(new StringBuilder("get").append(daoName).toString())
q"""def $daoTerm = getValueOrInstantiate($daoName, () => new $daoType(configuration))"""
}).toList <--- HERE!
val tree = q"""
package database.dao {
import org.jooq.SQLDialect
import org.jooq.impl.DefaultConfiguration
import utility.StrongHashMap
trait $componentType extends StrongHashMap[String, Dao] {
this: DaoManager =>
private lazy val configuration = new DefaultConfiguration().set(connection).set(SQLDialect.POSTGRES_9_4)
..$definitions <-- AND HERE!
}
}"""
How does one access the parameters used to construct a Module from inside the Tester that is testing it?
In the test below I am passing the parameters explicitly both to the Module and to the Tester. I would prefer not to have to pass them to the Tester but instead extract them from the module that was also passed in.
Also I am new to scala/chisel so any tips on bad techniques I'm using would be appreciated :).
import Chisel._
import math.pow
class TestA(dataWidth: Int, arrayLength: Int) extends Module {
val dataType = Bits(INPUT, width = dataWidth)
val arrayType = Vec(gen = dataType, n = arrayLength)
val io = new Bundle {
val i_valid = Bool(INPUT)
val i_data = dataType
val i_array = arrayType
val o_valid = Bool(OUTPUT)
val o_data = dataType.flip
val o_array = arrayType.flip
}
io.o_valid := io.i_valid
io.o_data := io.i_data
io.o_array := io.i_array
}
class TestATests(c: TestA, dataWidth: Int, arrayLength: Int) extends Tester(c) {
val maxData = pow(2, dataWidth).toInt
for (t <- 0 until 16) {
val i_valid = rnd.nextInt(2)
val i_data = rnd.nextInt(maxData)
val i_array = List.fill(arrayLength)(rnd.nextInt(maxData))
poke(c.io.i_valid, i_valid)
poke(c.io.i_data, i_data)
(c.io.i_array, i_array).zipped foreach {
(element,value) => poke(element, value)
}
expect(c.io.o_valid, i_valid)
expect(c.io.o_data, i_data)
(c.io.o_array, i_array).zipped foreach {
(element,value) => poke(element, value)
}
step(1)
}
}
object TestAObject {
def main(args: Array[String]): Unit = {
val tutArgs = args.slice(0, args.length)
val dataWidth = 5
val arrayLength = 6
chiselMainTest(tutArgs, () => Module(
new TestA(dataWidth=dataWidth, arrayLength=arrayLength))){
c => new TestATests(c, dataWidth=dataWidth, arrayLength=arrayLength)
}
}
}
If you make the arguments dataWidth and arrayLength members of TestA you can just reference them. In Scala this can be accomplished by inserting val into the argument list:
class TestA(val dataWidth: Int, val arrayLength: Int) extends Module ...
Then you can reference them from the test as members with c.dataWidth or c.arrayLength