Given an anonymous object:
val anon = new {
val a = BigDecimal(1)
}
How can I use scala reflection to get value of a ?
I have tried using java reflection, it is trivial. But with scala reflection, it is not obvious.
Here is what I have tried:
package test
object ReflectTest extends App {
val anon = new {
val a = BigDecimal(1)
}
val instanceMirror = currentMirror.reflect(anon)
val anonType = typeOf[anon.type]
val anonTermSymbol = anonType.member(newTermName("a")).asTerm
val anonFieldMirror = instanceMirror.reflectField(anonTermSymbol)
val result = anonFieldMirror.get
println(result)
}
But encountered an exception:
Exception in thread "main" scala.ScalaReflectionException: expected a member of anonymous class $anon$1, you provided value test.ReflectTest.<refinement>.a
at scala.reflect.runtime.JavaMirrors$JavaMirror.scala$reflect$runtime$JavaMirrors$JavaMirror$$ErrorNotMember(JavaMirrors.scala:130)
at scala.reflect.runtime.JavaMirrors$JavaMirror$$anonfun$scala$reflect$runtime$JavaMirrors$JavaMirror$$checkMemberOf$1.apply(JavaMirrors.scala:225)
at scala.reflect.runtime.JavaMirrors$JavaMirror.ensuringNotFree(JavaMirrors.scala:214)
at scala.reflect.runtime.JavaMirrors$JavaMirror.scala$reflect$runtime$JavaMirrors$JavaMirror$$checkMemberOf(JavaMirrors.scala:224)
at scala.reflect.runtime.JavaMirrors$JavaMirror$JavaInstanceMirror.reflectField(JavaMirrors.scala:247)
at scala.reflect.runtime.JavaMirrors$JavaMirror$JavaInstanceMirror.reflectField(JavaMirrors.scala:243)
It seems that the runtime type is not the one recognized by scala reflection.
Related
I have a Scala test which uses Mockito to verify that certain DataFrame transformations are invoked. I broke it down to this simple problematic example
import org.apache.spark.sql.DataFrame
import org.scalatest.funsuite.AnyFunSuite
import org.apache.spark.sql.functions._
import org.mockito.{Mockito, MockitoSugar}
class SimpleTest extends AnyFunSuite{
def withGreeting(df: DataFrame):DataFrame = {
df.withColumn("greeting", lit("hello"))
}
test("sample test") {
val mockDF = MockitoSugar.mock[DataFrame]
val mockDF2 = MockitoSugar.mock[DataFrame]
MockitoSugar.doReturn(mockDF2).when(mockDF).transform(withGreeting)
mockDF.transform(withGreeting)
val orderVerifier = Mockito.inOrder(mockDF)
orderVerifier.verify(mockDF).transform(withGreeting)
}
}
I'm trying to assert that the transform was called on my mockDF, but it fails with
Argument(s) are different! Wanted:
dataset.transform(<function1>);
-> at org.apache.spark.sql.Dataset.transform(Dataset.scala:2182)
Actual invocations have different arguments:
dataset.transform(<function1>);
Why would the verify fail in this case?
You need to save lambda expression argument for transform as val for correct testing and pass it to all transform calls:
def withGreeting(df: DataFrame):DataFrame = {
df.withColumn("greeting", lit("hello"))
}
test("sample test") {
val mockDF = MockitoSugar.mock[DataFrame]
val mockDF2 = MockitoSugar.mock[DataFrame]
val withGreetingExpression = df => withGreeting(df)
MockitoSugar.doReturn(mockDF2).when(mockDF).transform(withGreetingExpression)
mockDF.transform(withGreetingExpression)
val orderVerifier = Mockito.inOrder(mockDF)
orderVerifier.verify(mockDF).transform(withGreetingExpression)
}
Mockito requires to provide same (or equal) arguments to the mocked functions calls. When you are passing lambda expression without saving each call transform(withGreeting) creates new object Function[DataFrame, DataFrame]
transform(withGreeting)
is the same as:
transform(new Function[DataFrame, DataFrame] {
override def apply(df: DataFrame): DataFrame = withGreeting(df)
})
And they aren't equal to each other - this is the cause of error message:
Argument(s) are different!
For example, try to execute:
println(((df: DataFrame) => withGreeting(df)) == ((df: DataFrame) => withGreeting(df))) //false
You can read more about objects equality in java (in the scala it's same):
wikibooks
javaworld.com
I have a class TestClass with a companion object. How can I access a private field say xyz in the companion object using runtime reflection in scala when that private field is set from within the class as shown below.
class TestClass { TestClass.xyz = 100 }
object TestClass { private var xyz: Int = _ }
I tried the following
import scala.reflect.runtime.{currentMirror, universe => ru}
val testModuleSymbol = ru.typeOf[TestClass.type].termSymbol.asModule
val moduleMirror = currentMirror.reflectModule(testModuleSymbol)
val instanceMirror = currentMirror.reflect(moduleMirror.instance)
val xyzTerm = ru.typeOf[TestClass.type].decl(ru.TermName("xyz")).asTerm.accessed.asTerm
val fieldMirror = instanceMirror.reflectField(xyzTerm)
val context = fieldMirror.get.asInstanceOf[Int]
But I was getting the below error.
scala> val fieldMirror = instanceMirror.reflectField(xyzTerm)
scala.ScalaReflectionException: Scala field xyz of object TestClass isn't represented as a Java field, nor does it have a
Java accessor method. One common reason for this is that it may be a private class parameter
not used outside the primary constructor.
at scala.reflect.runtime.JavaMirrors$JavaMirror.scala$reflect$runtime$JavaMirrors$JavaMirror$$abort(JavaMirrors.scala:115)
at scala.reflect.runtime.JavaMirrors$JavaMirror.scala$reflect$runtime$JavaMirrors$JavaMirror$$ErrorNonExistentField(JavaMirrors.scala:127)
at scala.reflect.runtime.JavaMirrors$JavaMirror$JavaInstanceMirror.reflectField(JavaMirrors.scala:242)
at scala.reflect.runtime.JavaMirrors$JavaMirror$JavaInstanceMirror.reflectField(JavaMirrors.scala:233)
... 29 elided
This exception is thrown only when I refer the variable xyz in the TestClass (ie TestClass.xyz = 100). If this reference is removed from the class than my sample code works just fine.
Got this to work:
import scala.reflect.runtime.universe._
import scala.reflect.runtime.{universe => ru}
val runMirror = ru.runtimeMirror(getClass.getClassLoader)
val objectDef = Class.forName("org.myorg.TestClass")
val objectTypeModule = runMirror.moduleSymbol(objectDef).asModule
val objectType = objectTypeModule.typeSignature
val methodMap = objectType.members
.filter(_.isMethod)
.map(d => {
d.name.toString -> d.asMethod
})
.toMap
// get the scala Object
val instance = runMirror.reflectModule(objectTypeModule).instance
val instanceMirror = runMirror.reflect(instance)
// get the private value
val result = instanceMirror.reflectMethod(methodMap("xyz")).apply()
assert(result == 100)
Using org.json4s, I'm trying to compile this code:
implicit val formats = org.json4s.DefaultJsonFormats
for {
parsed <- Try(parse(message)).toOption
purchase <- parsed.extractOpt[Item]
} yield {
val datetime = new DateTime(purchase.time)
val roundedTime = datetime.withMinuteOfHour(0).withSecondOfMinute(0).withMillisOfSecond(0)
Key(purchase.item_id, roundedTime) -> purchase.amount
}
I'm getting the following error:
"No org.json4s.Formats found. Try to bring an instance of
org.json4s.Formats in scope or use the org.json4s.DefaultFormats."
You're using the wrong implicit. You need to use DefaultFormats instead of DefaultJsonFormats:
implicit val formats = org.json4s.DefaultFormats
object Rec extends App {
val outStream = new java.io.ByteArrayOutputStream
{
val out = new java.io.PrintStream(new java.io.BufferedOutputStream(outStream))
}
}
This seemingly simple code causes a compile error:
$ scalac rec.scala
rec.scala:2: error: recursive value out needs type
val outStream = new java.io.ByteArrayOutputStream
^
one error found
But I don't see what is "recursive."
Scala compiler version 2.11.7 -- Copyright 2002-2013, LAMP/EPFL
Background: I was trying to write a unit test on println with Console.withOut
After putting braces where they belong code looks like this:
object Rec extends App {
val outStream = new java.io.ByteArrayOutputStream {
val out = new java.io.PrintStream(new java.io.BufferedOutputStream(outStream))
}
}
and this is how you create object of an anonymous class with a member out that uses the defined object recursively (outStream uses outStream in its definition).
I believe this is what you wanted to do
object Rec extends App {
val outStream = new java.io.ByteArrayOutputStream
val out = new java.io.PrintStream(new java.io.BufferedOutputStream(outStream))
}
If you for some reason need to create another scope, you can use locally
What does Predef.locally do, and how is it different from Predef.identity
You are assigning a value to outStream by invoking stuff to which you pass on the outStream (I marked it in CAPS). Hence the recursion.
object Rec extends App {
val OUTSTREAM = new java.io.ByteArrayOutputStream
{
val out = new java.io.PrintStream(new java.io.BufferedOutputStream(OUTSTREAM))
}
}
testOnly play.api.weibo.StatusesShowBatchSpec
[error] Could not create an instance of play.api.weibo.StatusesShowBatchSpec
[error] caused by java.lang.Exception: Could not instantiate class play.api.weibo.StatusesShowBatchSpec: null
[error] org.specs2.reflect.Classes$class.tryToCreateObjectEither(Classes.scala:93)
[error] org.specs2.reflect.Classes$.tryToCreateObjectEither(Classes.scala:211)
[error] org.specs2.specification.SpecificationStructure$$anonfun$createSpecificationEither$2.apply(BaseSpecification.scala:119)
[error] org.specs2.specification.SpecificationStructure$$anonfun$createSpecificationEither$2.apply(BaseSpecification.scala:119)
...
The spec
package play.api.weibo
import org.junit.runner.RunWith
import org.specs2.runner.JUnitRunner
class StatusesShowBatchSpec extends ApiSpec {
"'statuses show batch' api" should {
"read statuses" in {
val api = StatusesShowBatch(
accessToken = testAdvancedToken,
ids = "3677163356078857")
val res = awaitApi(api)
res.statuses must have size (1)
}
}
}
See full code here https://github.com/jilen/play-weibo/tree/spec2_error
Full stacktrace
https://gist.github.com/jilen/9050548
In the ApiSpec class you have a few variables which might be null at instantiation time:
val cfg = ConfigFactory.load("http.conf")
val testToken = cfg.getString("token.normal")
val testAdvancedToken = cfg.getString("token.advanced")
implicit val http = new SprayHttp {
val config = new SprayHttpConfig {
val system = ActorSystem("test")
val gzipEnable = true
}
val context = config.system.dispatcher
}
You can turn those vals into lazy vals to avoid this situation:
lazy val cfg = ConfigFactory.load("http.conf")
lazy val testToken = cfg.getString("token.normal")
lazy val testAdvancedToken = cfg.getString("token.advanced")
implicit lazy val http = new SprayHttp {
lazy val config = new SprayHttpConfig {
val system = ActorSystem("test")
val gzipEnable = true
}
val context = config.system.dispatcher
}
I was getting a very similar error using specs2 version 2.3.10 on Scala 2.10. Upgrading to 2.3.13 makes the error messages much more informative and provides an extra stacktrace to the root cause. This newer version was released very recently (8 days before this post!), so hopefully you're able to accommodate an update...
Some of my issues ended up being related the val vs. lazy val problem like in the accepted answer; however, I'm now able to pinpoint the exact line that these errors are occurring on in addition to debugging other initialization problems as well.