How can I look up the value of an object's property dynamically by name in Scala 2.10.x?
E.g. Given the class (it can't be a case class):
class Row(val click: Boolean,
val date: String,
val time: String)
I want to do something like:
val fields = List("click", "date", "time")
val row = new Row(click=true, date="2015-01-01", time="12:00:00")
fields.foreach(f => println(row.getProperty(f))) // how to do this?
class Row(val click: Boolean,
val date: String,
val time: String)
val row = new Row(click=true, date="2015-01-01", time="12:00:00")
row.getClass.getDeclaredFields foreach { f =>
f.setAccessible(true)
println(f.getName)
println(f.get(row))
}
You could also use the bean functionality from java/scala:
import scala.beans.BeanProperty
import java.beans.Introspector
object BeanEx extends App {
case class Stuff(#BeanProperty val i: Int, #BeanProperty val j: String)
val info = Introspector.getBeanInfo(classOf[Stuff])
val instance = Stuff(10, "Hello")
info.getPropertyDescriptors.map { p =>
println(p.getReadMethod.invoke(instance))
}
}
Related
Let's say we have two following case classes:
case class Person(name:String, age:Int, createdAt:LocalDate, lastModified:LocalDate)
case class Address(street:String, zip:Int, createdAt:LocalDate, lastModified:LocalDate)
I want to be able to do as follows:
for {
p <- query[Person]
.changedAfter(lift(LocalDate.of(2022,1,1)))
a <- query[Address].join(a => a.ownerId == p.id)
.changedAfter(lift(LocalDate.of(2022,1,1)))
} yield (p, a)
Where .changedAfter() will work for any entity containing createdAt and lastModified fields.
How would I go on to create such a modification?
Your can create a extension method like this.
import io.getquill._
import java.time._
val ctx = new SqlMirrorContext(PostgresDialect, SnakeCase)
import ctx._
trait EntityLike {
val createAt: LocalDate
val lastModified: LocalDate
}
case class Person(name: String, age: Int, createAt: LocalDate, lastModified: LocalDate) extends EntityLike
implicit class EntityOps[A <: EntityLike](q: Query[A]) {
val changeAfter = quote { (d: LocalDate) =>
q.filter(e => infix"${e.createAt} < ${d}".as[Boolean])
}
}
val d = LocalDate.of(2000, 1, 1)
val m = ctx.run(query[Person].changeAfter(lift(d)))
println(m)
I'm trying to use Spray JSON to marshall the 'Seq' collection below into a 'BidRequest' entity with the parameters as defined.
The Seq collection is mostly nested, therefore some 'Seq' parameter fields also have variable collection types that need to be marshalled.
Then after a computation, the aim is to unmarshall the results as an entity of 'BidResponse'.
What's the best approach to do this?
I'm using Akka-HTTP, Akka-Streams, Akka-Actor.
Seq collection:
val activeUsers = Seq(
Campaign(
id = 1,
country = "UK",
targeting = Targeting(
targetedSiteIds = Seq("0006a522ce0f4bbbbaa6b3c38cafaa0f")
),
banners = List(
Banner(
id = 1,
src ="https://business.URLTV.com/wp-content/uploads/2020/06/openGraph.jpeg",
width = 300,
height = 250
)
),
bid = 5d
)
)
BidRequest case class:
case class BidRequest(id: String, imp: Option[List[Impression]], site:Site, user: Option[User], device: Option[Device])
BidResponse case class:
case class BidResponse(id: String, bidRequestId: String, price: Double, adid:Option[String], banner: Option[Banner])
The other case classes:
case class Campaign(id: Int, country: String, targeting: Targeting, banners: List[Banner], bid: Double)
case class Targeting(targetedSiteIds: Seq[String])
case class Banner(id: Int, src: String, width: Int, height: Int)
case class Impression(id: String, wmin: Option[Int], wmax: Option[Int], w: Option[Int], hmin: Option[Int], hmax: Option[Int], h: Option[Int], bidFloor: Option[Double])
case class Site(id: Int, domain: String)
case class User(id: String, geo: Option[Geo])
case class Device(id: String, geo: Option[Geo])
case class Geo(country: Option[String])
I've so far tried using the code below but keep getting type mismatch errors:
import akka.http.scaladsl.marshallers.sprayjson.SprayJsonSupport._
import spray.json.DefaultJsonProtocol._
implicit val resFormat = jsonFormat2(BidResponse)
implicit val bidFormat = jsonFormat1(BidRequest)
implicit val cFormat = jsonFormat1(Campaign)
implicit val tFormat = jsonFormat1(Targeting)
implicit val bFormat = jsonFormat1(Banner)
implicit val iFormat = jsonFormat1(Impression)
implicit val sFormat = jsonFormat1(Site)
implicit val uFormat = jsonFormat1(User)
implicit val dFormat = jsonFormat1(Device)
implicit val gFormat = jsonFormat1(Geo)
The reason why you are getting Type errors with Spray JSON is because you need to use the corresponding jsonFormatN method, depending on the number of parameters in the case class.
In your case:
implicit val resFormat = jsonFormat5(BidResponse)
implicit val bidFormat = jsonFormat5(BidRequest)
implicit val cFormat = jsonFormat1(Campaign)
implicit val tFormat = jsonFormat1(Targeting)
implicit val bFormat = jsonFormat4(Banner)
...
I have the following 3 case classes:
case class Profile(name: String,
age: Int,
bankInfoData: BankInfoData,
userUpdatedFields: Option[UserUpdatedFields])
case class BankInfoData(accountNumber: Int,
bankAddress: String,
bankNumber: Int,
contactPerson: String,
phoneNumber: Int,
accountType: AccountType)
case class UserUpdatedFields(contactPerson: String,
phoneNumber: Int,
accountType: AccountType)
this is just enums, but i added anyway:
sealed trait AccountType extends EnumEntry
object AccountType extends Enum[AccountType] {
val values: IndexedSeq[AccountType] = findValues
case object Personal extends AccountType
case object Business extends AccountType
}
my task is - i need to write a funcc Profile and compare UserUpdatedFields(all of the fields) with SOME of the fields in BankInfoData...this func is to find which fields where updated.
so I wrote this func:
def findDiff(profile: Profile): Seq[String] = {
var listOfFieldsThatChanged: List[String] = List.empty
if (profile.bankInfoData.contactPerson != profile.userUpdatedFields.get.contactPerson){
listOfFieldsThatChanged = listOfFieldsThatChanged :+ "contactPerson"
}
if (profile.bankInfoData.phoneNumber != profile.userUpdatedFields.get.phoneNumber) {
listOfFieldsThatChanged = listOfFieldsThatChanged :+ "phoneNumber"
}
if (profile.bankInfoData.accountType != profile.userUpdatedFields.get.accountType) {
listOfFieldsThatChanged = listOfFieldsThatChanged :+ "accountType"
}
listOfFieldsThatChanged
}
val profile =
Profile(
"nir",
34,
BankInfoData(1, "somewhere", 2, "john", 123, AccountType.Personal),
Some(UserUpdatedFields("lee", 321, AccountType.Personal))
)
findDiff(profile)
it works, but wanted something cleaner..any suggestions?
Each case class extends Product interface so we could use it to convert case classes into sets of (field, value) elements. Then we can use set operations to find the difference. For example,
def findDiff(profile: Profile): Seq[String] = {
val userUpdatedFields = profile.userUpdatedFields.get
val bankInfoData = profile.bankInfoData
val updatedFieldsMap = userUpdatedFields.productElementNames.zip(userUpdatedFields.productIterator).toMap
val bankInfoDataMap = bankInfoData.productElementNames.zip(bankInfoData.productIterator).toMap
val bankInfoDataSubsetMap = bankInfoDataMap.view.filterKeys(userUpdatedFieldsMap.keys.toList.contains)
(bankInfoDataSubsetMap.toSet diff updatedFieldsMap.toSet).toList.map { case (field, value) => field }
}
Now findDiff(profile) should output List(phoneNumber, contactPerson). Note we are using productElementNames from Scala 2.13 to get the filed names which we then zip with corresponding values
userUpdatedFields.productElementNames.zip(userUpdatedFields.productIterator)
Also we rely on filterKeys and diff.
A simple improvement would be to introduce a trait
trait Fields {
val contactPerson: String
val phoneNumber: Int
val accountType: AccountType
def findDiff(that: Fields): Seq[String] = Seq(
Some(contactPerson).filter(_ != that.contactPerson).map(_ => "contactPerson"),
Some(phoneNumber).filter(_ != that.phoneNumber).map(_ => "phoneNumber"),
Some(accountType).filter(_ != that.accountType).map(_ => "accountType")
).flatten
}
case class BankInfoData(accountNumber: Int,
bankAddress: String,
bankNumber: Int,
contactPerson: String,
phoneNumber: Int,
accountType: String) extends Fields
case class UserUpdatedFields(contactPerson: String,
phoneNumber: Int,
accountType: AccountType) extends Fields
so it was possible to call
BankInfoData(...). findDiff(UserUpdatedFields(...))
If you want to further-improve and avoid naming all the fields multiple times, for example shapeless could be used to do it compile time. Not exactly the same but something like this to get started. Or use reflection to do it runtime like this answer.
That would be a very easy task to achieve if it would be an easy way to convert case class to map. Unfortunately, case classes don't offer that functionality out-of-box yet in Scala 2.12 (as Mario have mentioned it will be easy to achieve in Scala 2.13).
There's a library called shapeless, that offers some generic programming utilities. For example, we could write an extension function toMap using Record and ToMap from shapeless:
object Mappable {
implicit class RichCaseClass[X](val x: X) extends AnyVal {
import shapeless._
import ops.record._
def toMap[L <: HList](
implicit gen: LabelledGeneric.Aux[X, L],
toMap: ToMap[L]
): Map[String, Any] =
toMap(gen.to(x)).map{
case (k: Symbol, v) => k.name -> v
}
}
}
Then we could use it for findDiff:
def findDiff(profile: Profile): Seq[String] = {
import Mappable._
profile match {
case Profile(_, _, bankInfo, Some(userUpdatedFields)) =>
val bankInfoMap = bankInfo.toMap
userUpdatedFields.toMap.toList.flatMap{
case (k, v) if bankInfoMap.get(k).exists(_ != v) => Some(k)
case _ => None
}
case _ => Seq()
}
}
Spark throws Task not serializable when I use case class or class/object that extends Serializable inside a closure.
object WriteToHbase extends Serializable {
def main(args: Array[String]) {
val csvRows: RDD[Array[String] = ...
val dateFormatter = DateTimeFormat.forPattern("yyyy-MM-dd HH:mm:ss")
val usersRDD = csvRows.map(row => {
new UserTable(row(0), row(1), row(2), row(9), row(10), row(11))
})
processUsers(sc: SparkContext, usersRDD, dateFormatter)
})
}
def processUsers(sc: SparkContext, usersRDD: RDD[UserTable], dateFormatter: DateTimeFormatter): Unit = {
usersRDD.foreachPartition(part => {
val conf = HBaseConfiguration.create()
val table = new HTable(conf, tablename)
part.foreach(userRow => {
val id = userRow.id
val date1 = dateFormatter.parseDateTime(userRow.date1)
})
table.flushCommits()
table.close()
})
}
My first attempt was to use a case class:
case class UserTable(id: String, name: String, address: String, ...) extends Serializable
My second attempt was to use a class instead of a case class:
class UserTable (val id: String, val name: String, val addtess: String, ...) extends Serializable {
}
My third attempt was to use a companion object in the class:
object UserTable extends Serializable {
def apply(id: String, name: String, address: String, ...) = new UserTable(id, name, address, ...)
}
Most likely the function "doSomething" is defined on your class which isn't serilizable. Instead move the "doSomething" function to a companion object (e.g. make it static).
It was the dateFormatter, I placed it inside the partition loop and it works now.
usersRDD.foreachPartition(part => {
val id = userRow.id
val dateFormatter = DateTimeFormat.forPattern("yyyy-MM-dd HH:mm:ss")
val date1 = dateFormatter.parseDateTime(userRow.date1)
})
If I have a class like this:
case class Key(
id: Long,
var text: String,
var `type`: String
)
Is there any way to get from that class the list of arguments like this:
['id', 'text', 'type']
If so, how would I do that? Thanks!
I don't know the solution for myself, but I found this on github:
https://gist.github.com/heathermiller/354befb88b097330d42d
import scala.reflect.runtime.universe._
class AbstractParams[T: TypeTag] {
def tag: TypeTag[T] = typeTag[T]
override def toString: String = {
// Find all case class fields in concrete class instance and print them as "[field name] [field value]"
val tag = this.tag
val tpe = tag.tpe
val allAccessors = tpe.declarations.collect { case meth: MethodSymbol if meth.isCaseAccessor => meth }
val m = runtimeMirror(getClass.getClassLoader)
val im = m.reflect(this)
val ctorArg2Strings = allAccessors map { sym =>
val fldMirror = im.reflectField(sym)
val value = fldMirror.get
"[" + sym.name + ": " + value + "]"
}
ctorArg2Strings.mkString(" ")
}
}
case class MyParams(id: String, stuff: Int) extends AbstractParams[MyParams]
object Ex extends App {
val p = new MyParams("joe", 2)
println(s"$p")
// prints:
// [id: joe] [stuff: 2]
}
I hope this might help.
I know that it's not exactly what you was looking for, but you can adjust this solution for yourself.