Deep access of fields in Scala using runtime reflection - scala

I have code that deeply walks a case class' constructor fields, which of course may themselves be complex (list of things, maps, options, and other case classes). The code I found to extract field values at runtime works great on the highest-level fields but explodes when I try to access deeper fields. Example below.
I real life my application introspects the fields at each level, so I know that 'stuff' is another case class (I have the Symbol/Type), and I know Dos' field Symbols/Types. But this is obtained at runtime so I think it's blowing up because it doesn't know [T]/Manifest[T]. Is there a way to get this at runtime via reflection? How might my code change? The examples I found seemed to all require various things[T], which I wouldn't have for 'dos', right?
case class Uno( name:String, age:Int, pets:List[String], stuff:Dos )
case class Dos( foo:String )
object Boom extends App {
val ru = scala.reflect.runtime.universe
val m = ru.runtimeMirror(getClass.getClassLoader)
val u = Uno("Marcus",19,List("fish","bird"),Dos("wow"))
println("NAME: "+unpack(u,"name")) // Works
println("PETS: "+unpack(u,"pets")) // Works
// ----- Goes Boom -------
val dos = unpack(u,"stuff")
println("Other: "+unpack(dos,"foo")) // Boom!
// -----------------------
// Get object value for named parameter of target
def unpack[T]( target:T, name:String )(implicit man:Manifest[T]) : Any = {
val im = m.reflect(target)
val fieldX = ru.typeOf[T].declaration(ru.newTermName(name)).asTerm.accessed.asTerm
im.reflectField(fieldX).get
}
}

You're exactly right, the type of your dos is Any.
FieldMirror.symbol.typeSignature is what you'd get from typeOf[Dos].
So consider returning a pair (Any, Type) from unpack to have something to pass to unpack(target, type, name). Somewhat like:
case class Uno(name: String, age: Int, pets: List[String], stuff: Dos)
case class Dos(foo: String)
object Boom extends App {
import scala.reflect.runtime.universe._
import scala.reflect.runtime.{ currentMirror => cm }
import scala.reflect.ClassTag
val u = Uno("Marcus", 19, List("fish", "bird"), Dos("wow"))
println("NAME: " + unpack(u, "name")) // Works
println("PETS: " + unpack(u, "pets")) // Works
// ----- Goes Boom -------
val (dos, dosT) = unpack(u, "stuff")
println("Other: " + unpack(dos, dosT, "foo")) // Boom! ...or fizzle
// -----------------------
def unpack[T: TypeTag](target: T, name: String): (Any, Type) = unpack(target, typeOf[T], name)
// Get object value for named parameter of target
def unpack[T](target: T, t: Type, name: String): (Any, Type) = {
val im = cm.reflect(target)(ClassTag(target.getClass))
val fieldX = t.declaration(newTermName(name)).asTerm.accessed.asTerm
val fm = im.reflectField(fieldX)
(fm.get, fm.symbol.typeSignature)
}
}

Related

Why Scala case class copy method parameterised only with the variables defined in the case class?

Why Scala case class copy method parameterised only with the variables defined in the case class?
The question based on on the Q&A:
Case class copy does not maintain state of an inherited trait
Short summary - when trait is defined with a field and extended by a case class, copy on the case class will create the new class instance only with the variables defined in case class, without the extended trait.
trait A {
var list: List[Int] = List()
def add(element: Int) = {
list = element :: list
}
}
case class B(str: String) extends A {
val b = B("foo")
println("B1: " + b.list)
b.add(1)
b.add(2)
b.add(3)
println("B2: " + b.list)
val b2 = b.copy(str = "bar")
println("B3: " + b.list)
println("B4: " + b2.list)
}
Here, B4: () will be empty, while B3:(3,2,1)
Because there is no reasonable way to do what you want consistently.
For your example, generated code for copy could be
def copy(str: String = this.str, list: List[Int] = this.list): B = {
val newB = B(str)
newB.list = list
newB
}
Good enough. Now what happens if you change list to be private, or a val instead of a var? In both cases newB.list = ... won't compile, so what code should the compiler generate?
If you really want to keep the value of list after copying (considering the mutability issue I mentioned in the comments of OP), you can write your own copy method.
case class B(str: String) extends A {
def copy(str: String = this.str, list: List[Int] = this.list): B = {
val newB = B(str)
list.reverse.foreach(newB.add)
newB
}
}

Filtering Lists in Scala's Monocle

Given the following code:
case class Person(name :String)
case class Group(group :List[Person])
val personLens = GenLens[Person]
val groupLens = GenLens[Group]
how can i "filter" out certain Persons from the selection, NOT by index but by a specific property of Person, like:
val trav :Traversal[Group, Person] = (groupLens(_.group) composeTraversal filterWith((x :Person) => /*expression of type Boolean here */))
I only found the filterIndex function, which does only include elements from the list based on the index, but this is not what I want.
filterIndex takes a function of type: (Int => Boolean)
and I want:
filterWith (made up name), that takes a (x => Boolean), where x has the type of the element of the list, namely Person in this short example.
This seems so practical and common that I assume somebody has thought about that and i (with my, i must admit limited understanding of the matter) don't see why it can't be done.
Am I missing this functionality, is it not implemented yet or just plainly impossible for whatever reason (please do explain if you have the time)?
Thank you.
A bad version
I'll start with a naive attempt to write something like this. I'm using a simple list version here, but you could get fancier (with Traverse or whatever) if you wanted.
import monocle.Traversal
import scalaz.Applicative, scalaz.std.list._, scalaz.syntax.traverse._
def filterWith[A](p: A => Boolean): Traversal[List[A], A] =
new Traversal[List[A], A] {
def modifyF[F[_]: Applicative](f: A => F[A])(s: List[A]): F[List[A]] =
s.filter(p).traverse(f)
}
And then:
import monocle.macros.GenLens
case class Person(name: String)
case class Group(group: List[Person])
val personLens = GenLens[Person]
val groupLens = GenLens[Group]
val aNames = groupLens(_.group).composeTraversal(filterWith(_.name.startsWith("A")))
val group = Group(List(Person("Al"), Person("Alice"), Person("Bob")))
And finally:
scala> aNames.getAll(group)
res0: List[Person] = List(Person(Al), Person(Alice))
It works!
Why it's bad
It works, except…
scala> import monocle.law.discipline.TraversalTests
import monocle.law.discipline.TraversalTests
scala> TraversalTests(filterWith[String](_.startsWith("A"))).all.check
+ Traversal.get what you set: OK, passed 100 tests.
+ Traversal.headOption: OK, passed 100 tests.
! Traversal.modify id = id: Falsified after 2 passed tests.
> Labels of failing property:
Expected List(崡) but got List()
> ARG_0: List(崡)
! Traversal.modifyF Id = Id: Falsified after 2 passed tests.
> Labels of failing property:
Expected List(ᜱ) but got List()
> ARG_0: List(ᜱ)
+ Traversal.set idempotent: OK, passed 100 tests.
Three out of five isn't very good.
A slightly better version
Let's start over:
def filterWith2[A](p: A => Boolean): Traversal[List[A], A] =
new Traversal[List[A], A] {
def modifyF[F[_]: Applicative](f: A => F[A])(s: List[A]): F[List[A]] =
s.traverse {
case a if p(a) => f(a)
case a => Applicative[F].point(a)
}
}
val aNames2 = groupLens(_.group).composeTraversal(filterWith2(_.name.startsWith("A")))
And then:
scala> aNames2.getAll(group)
res1: List[Person] = List(Person(Al), Person(Alice))
scala> TraversalTests(filterWith2[String](_.startsWith("A"))).all.check
+ Traversal.get what you set: OK, passed 100 tests.
+ Traversal.headOption: OK, passed 100 tests.
+ Traversal.modify id = id: OK, passed 100 tests.
+ Traversal.modifyF Id = Id: OK, passed 100 tests.
+ Traversal.set idempotent: OK, passed 100 tests.
Okay, better!
Why it's still bad
The "real" laws for Traversal aren't encoded in Monocle's TraversalLaws (at least not at the moment), and we additionally want something like this to hold:
For any f: A => A and g: A => A, t.modify(f.compose(g)) should equal t.modify(f).compose(t.modify(g)).
Let's try it:
scala> val graduate: Person => Person = p => Person("Dr. " + p.name)
graduate: Person => Person = <function1>
scala> val kill: Person => Person = p => Person(p.name + ", deceased")
kill: Person => Person = <function1>
scala> aNames2.modify(kill.compose(graduate))(group)
res2: Group = Group(List(Person(Dr. Al, deceased), Person(Dr. Alice, deceased), Person(Bob)))
scala> aNames2.modify(kill).compose(aNames2.modify(graduate))(group)
res3: Group = Group(List(Person(Dr. Al), Person(Dr. Alice), Person(Bob)))
So we're out of luck again. The only way our filterWith could actually be lawful is if we promise never to use it with an argument to modify that might change the result of the predicate.
This is why filterIndex is legit—its predicate takes as an argument something that modify can't touch, so you can't break the t.modify(f.compose(g)) === t.modify(f).compose(t.modify(g)) law.
Moral of the story
You could write an unlawful Traversal that does unlawful filtering stuff and use it all the time and it's pretty likely that it will never hurt you and that nobody will ever think you are a horrible person. So go for it, if you want. You'll probably never see a filterWith in a decent lens library, though.
You can use UnsafeSelect, https://www.optics.dev/Monocle/docs/unsafe_module.html#unsafeselect
import monocle.macros.GenLens
import org.scalatest.FunSuite
import monocle.function.all._
import monocle.unsafe.UnsafeSelect
case class Person(name :String, age: Int)
case class Group(group :List[Person])
class Example extends FunSuite{
test("filter elements of list") {
val group = Group(List(Person("adult1", 2), Person("adult2", 3), Person("child", 4)))
val filteredGroup = (GenLens[Group](_.group) composeTraversal each composePrism UnsafeSelect.unsafeSelect(_.name.startsWith("adult")) composeLens GenLens[Person](_.age) set 18) (group)
assert(filteredGroup.group.filter(_.name.startsWith("adult")).map(_.age) == List(18, 18))
}
}

Mapping a class as a column type within a class

I understand how this is done when using types such as Long, Int, String etc.. But say I have a class that has fields within another class like so:
case class Foo(a:String, b:String)
case class Bar(foo:Option[Foo], c:String)
How would I set up a mapper for my custom type (the Foo in my Bar class)?
class Bars(tag:Tag) extends Table[Bar](tag, "BARS") {
def foo = column[Foo]("FOO") // <- won't work
def c = column[String]("C")
def * = (foo, c) <> (Bar.tupled, Bar.unapply)
}
(documentation link)
Update:
DB Driver: slick.driver.PostgresDriver
Slick 2
I'm guessing the raw SQL would look like this:
"BARS" (
"A" VARCHAR(254) NOT NULL,
"B" VARCHAR(254) NOT NULL,
"C" VARCHAR(254) NOT NULL
);
Should be able to call Bar like so:
val bar = Bar(Foo("1", "2"), "3")
barTable.insert(bar)
bar.foo.a // 1
bar.foo.b // 2
bar.c // 3
You can write a mapper between the case class and some type that can be stored in the database.
See an example from Slick here:http://slick.typesafe.com/doc/1.0.0/lifted-embedding.html, at the end of the page.
One easy way in your case might be to transform your case class into json and store as a string. (And if your DB supports json type directly, like PostgreSQL, you can specify JSON type in column mapper, that would give you an advantage when making queries related to the content of your case classes.)
import org.json4s._
import org.json4s.native.Serialization
import org.json4s.native.Serialization.{read, write}
//place this in scope of your table definition
implicit val FooTypeMapper = MappedTypeMapper.base[Foo, String](
{ f => write(f) }, // map Foo to String
{ s => read[Too](s) } // map String to Foo
)
class Bars(tag:Tag) extends Table[Bar](tag, "BARS") {
def foo = column[Foo]("FOO") // <- should work now
def c = column[String]("C")
def * = (foo, c) <> (Bar.tupled, Bar.unapply)
}
With PostgreSQL >=9.3, you can also write:
def foo = column[Foo]("FOO", O.DBType("json"))
So that DB treats your json properly.
UPDATE: there is a connection property that should be set if send a String for a JSON field. Something like this:
val prop = new java.util.Properties
prop.setProperty("stringtype", "unspecified")
val db = Database.forURL(<db-uri>,
driver="org.postgresql.Driver",
user=<username>,
password=<password>,
prop=prop)
You need to provide columns for A and B in the Bars table:
class Bars(tag:Tag) extends Table[Bar](tag, "BARS") {
def a = column[String]("A")
def b = column[String]("B")
def foo = (a, b) <> (Foo.tupled, Foo.unapply)
def c = column[String]("C")
def * = (foo, c) <> (Bar.tupled, Bar.unapply)
}

Allocation of Function Literals in Scala

I have a class that represents sales orders:
class SalesOrder(val f01:String, val f02:Int, ..., f50:Date)
The fXX fields are of various types. I am faced with the problem of creating an audit trail of my orders. Given two instances of the class, I have to determine which fields have changed. I have come up with the following:
class SalesOrder(val f01:String, val f02:Int, ..., val f50:Date){
def auditDifferences(that:SalesOrder): List[String] = {
def diff[A](fieldName:String, getField: SalesOrder => A) =
if(getField(this) != getField(that)) Some(fieldName) else None
val diffList = diff("f01", _.f01) :: diff("f02", _.f02) :: ...
:: diff("f50", _.f50) :: Nil
diffList.flatten
}
}
I was wondering what the compiler does with all the _.fXX functions: are they instanced just once (statically), and can be shared by all instances of my class, or will they be instanced every time I create an instance of my class?
My worry is that, since I will use a lot of SalesOrder instances, it may create a lot of garbage. Should I use a different approach?
One clean way of solving this problem would be to use the standard library's Ordering type class. For example:
class SalesOrder(val f01: String, val f02: Int, val f03: Char) {
def diff(that: SalesOrder) = SalesOrder.fieldOrderings.collect {
case (name, ord) if !ord.equiv(this, that) => name
}
}
object SalesOrder {
val fieldOrderings: List[(String, Ordering[SalesOrder])] = List(
"f01" -> Ordering.by(_.f01),
"f02" -> Ordering.by(_.f02),
"f03" -> Ordering.by(_.f03)
)
}
And then:
scala> val orderA = new SalesOrder("a", 1, 'a')
orderA: SalesOrder = SalesOrder#5827384f
scala> val orderB = new SalesOrder("b", 1, 'b')
orderB: SalesOrder = SalesOrder#3bf2e1c7
scala> orderA diff orderB
res0: List[String] = List(f01, f03)
You almost certainly don't need to worry about the perfomance of your original formulation, but this version is (arguably) nicer for unrelated reasons.
Yes, that creates 50 short lived functions. I don't think you should be worried unless you have manifest evidence that that causes a performance problem in your case.
But I would define a method that transforms SalesOrder into a Map[String, Any], then you would just have
trait SalesOrder {
def fields: Map[String, Any]
}
def diff(a: SalesOrder, b: SalesOrder): Iterable[String] = {
val af = a.fields
val bf = b.fields
af.collect { case (key, value) if bf(key) != value => key }
}
If the field names are indeed just incremental numbers, you could simplify
trait SalesOrder {
def fields: Iterable[Any]
}
def diff(a: SalesOrder, b: SalesOrder): Iterable[String] =
(a.fields zip b.fields).zipWithIndex.collect {
case ((av, bv), idx) if av != bv => f"f${idx + 1}%02d"
}

Scala Macros: Checking for a certain annotation

Thanks to the answers to my previous question, I was able to create a function macro such that it returns a Map that maps each field name to its value of a class, e.g.
...
trait Model
case class User (name: String, age: Int, posts: List[String]) extends Model {
val numPosts: Int = posts.length
...
def foo = "bar"
...
}
So this command
val myUser = User("Foo", 25, List("Lorem", "Ipsum"))
myUser.asMap
returns
Map("name" -> "Foo", "age" -> 25, "posts" -> List("Lorem", "Ipsum"), "numPosts" -> 2)
This is where Tuples for the Map are generated (see Travis Brown's answer):
...
val pairs = weakTypeOf[T].declarations.collect {
case m: MethodSymbol if m.isAccessor =>
val name = c.literal(m.name.decoded)
val value = c.Expr(Select(model, m.name))
reify(name.splice -> value.splice).tree
}
...
Now I want to ignore fields that have #transient annotation. How would I check if a method has a #transient annotation?
I'm thinking of modifying the snippet above as
val pairs = weakTypeOf[T].declarations.collect {
case m: MethodSymbol if m.isAccessor && !m.annotations.exists(???) =>
val name = c.literal(m.name.decoded)
val value = c.Expr(Select(model, m.name))
reify(name.splice -> value.splice).tree
}
but I can't find what I need to write in exists part. How would I get #transient as an Annotation so I could pass it there?
Thanks in advance!
The annotation will be on the val itself, not on the accessor. The easiest way to access the val is through the accessed method on MethodSymbol:
def isTransient(m: MethodSymbol) = m.accessed.annotations.exists(
_.tpe =:= typeOf[scala.transient]
)
Now you can just write the following in your collect:
case m: MethodSymbol if m.isAccessor && !isTransient(m) =>
Note that the version of isTransient I've given here has to be defined in your macro, since it needs the imports from c.universe, but you could factor it out by adding a Universe argument if you're doing this kind of thing in several macros.