I have a scala case class and object like below,
case class User(userId: Long, UserName: String, ts: Timestamp)
object User {
def getRdd(rdd: RDD[JsValue], type : String): RDD[User] = {
val rdd1: RDD[User] = rdd.map(doc => processEvent(doc))
.filter(event => event._1.equals(rddType)).map(event => {
User.get_class_obj(event)
})
rdd1
}
}
I want to call "getRdd" method of User Object from another object without creating instance of object/class. like below,
val object_name = "com.User"
val method_name = "getRdd"
I tried,
val no = Array(1, 2, 3, 4, 5,6,7,8,9,10)
val rdd = sc.parallelize(no)
Class.forName(object_name).getDeclaredMethod(method_name).invoke(rdd)
but fails with nosuchmethod error. I have gone through answers here but I don't want to create multiple instances every time. Is it possible to do it in one-liner.
Try with dollar sign
val object_name = "com.User$"
if object User is in package com.
From Java reflection point of view User is a class and User$ is its companion object.
You should also specify classes of arguments
Class.forName("com.User$").getDeclaredMethod("getRdd", classOf[RDD[_]], classOf[String])
Related
I'm trying to receive some filter list as query param of a get request, which is also an enum on my controller. To do so, I'm using play framework with scala. The problem is: I can't putting the enum type as query param on it, because IDE doesn't recognize as a valid type.
So, I have something like this on routes file
GET /service-orders/ controllers.ServiceOrdersController.listServiceOrders(status: ServiceStatus)
My enum file:
object ServiceStatus extends Enumeration {
type ServiceStatus = Value
val Pending = Value("pending")
val Started = Value("started")
val Completed = Value("completed")
val Error = Value("error")
}
On build.sbt, I use this trying to inject package on routes file
routesImport ++= Seq(
"serviceOrders.models.ServiceStatus"
),
I tried a lot of things, but with no success. I read in some place that I could use QueryStringBindable function, but I can't did this works well... Can you guys please help me to solve this?
Edit: Btw, there's a way to check if status is contained on a list of filters without making this?
.filter {
serviceOrder =>
status.map(serviceOrder.serviceStatus === _)
.reduceOption(_ || _)
.getOrElse(true: Rep[Boolean])
}
This was the only way I could thought to filter status by a list of filters received as query param from API.
You can implement the QueryStringBindable instance like so:
package serviceOrders.models
object ServiceStatus extends Enumeration {
type ServiceStatus = Value
val Pending = Value("pending")
val Started = Value("started")
val Completed = Value("completed")
val Error = Value("error")
implicit val queryStringBindable: QueryStringBindable[ServiceStatus] =
new QueryStringBindable[ServiceStatus] {
override def bind(
key: String,
params: Map[String, Seq[String]]
): Option[Either[String, ServiceStatus]] =
params.get(key).collect {
case Seq(s) =>
ServiceStatus.values.find(_.toString == s).toRight("invalid value")
}
override def unbind(key: String, value: ServiceStatus): String =
implicitly[QueryStringBindable[String]].unbind(key, value.toString)
}
}
In build.sbt you need this:
routesImport ++= Seq("serviceOrders.models.ServiceStatus._")
And this in your routes file:
GET /some/route controllers.SomeController.index(status: ServiceStatus)
Then you can create an index method that takes a ServiceStatus parameter in SomeController and Play will take care of the query parameters.
// edit:
You could actually use the QueryStringBindable.Parsing class to simplify the implementation further.
I'm writing a Scala library to operate upon Spark DataFrames. I have a bunch of classes, each of which contain a function that operates upon the supplied DataFrame:
class Foo(){val func = SomeFunction(,,,)}
class Bar(){val func = SomeFunction(,,,)}
class Baz(){val func = SomeFunction(,,,)}
The user of my library passes a parameter operation: String to indicate class to instantiate, the value passed has to be the name of one of those classes hence I have code that looks something like this:
operation match {
case Foo => new Foo().SomeFunction
case Bar => new Bar().SomeFunction
case Baz => new Baz().SomeFunction
}
I'm a novice Scala developer but this seems rather like a clunky way of achieving this. I'm hoping there is a simpler way to instantiate the desired class based on the value of operation given that it will be the same as the name of the desired class.
The reason I want to do this is that I want external contributors to contribute their own classes and I want to make it at easy as possible for them to do that, I don't want them to have to know they also need to go and change a pattern match.
For
case class SomeFunction(s: String)
class Foo(){val func = SomeFunction("Foo#func")}
class Bar(){val func = SomeFunction("Bar#func")}
class Baz(){val func = SomeFunction("Baz#func")}
//...
reflection-based version of
def foo(operation: String) = operation match {
case "Foo" => new Foo().func
case "Bar" => new Bar().func
case "Baz" => new Baz().func
// ...
}
is
import scala.reflect.runtime.universe
import scala.reflect.runtime.universe._
def foo(operation: String): SomeFunction = {
val runtimeMirror = universe.runtimeMirror(getClass.getClassLoader)
val classSymbol = runtimeMirror.staticClass(operation)
val constructorSymbol = classSymbol.primaryConstructor.asMethod
val classMirror = runtimeMirror.reflectClass(classSymbol)
val classType = classSymbol.toType
val constructorMirror = classMirror.reflectConstructor(constructorSymbol)
val instance = constructorMirror()
val fieldSymbol = classType.decl(TermName("func")).asTerm
val instanceMirror = runtimeMirror.reflect(instance)
val fieldMirror = instanceMirror.reflectField(fieldSymbol)
fieldMirror.get.asInstanceOf[SomeFunction]
}
Testing:
foo("Foo") //SomeFunction(Foo#func)
foo("Bar") //SomeFunction(Bar#func)
foo("Baz") //SomeFunction(Baz#func)
I'm trying (and failing) to get my head around how spray-json converts json feeds into objects. If I have a simple key -> value json feed then it seems to work ok but the data I want to read comes in a list like this:
[{
"name": "John",
"age": "30"
},
{
"name": "Tom",
"age": "25"
}]
And my code looks like this:
package jsontest
import spray.json._
import DefaultJsonProtocol._
object JsonFun {
case class Person(name: String, age: String)
case class FriendList(items: List[Person])
object FriendsProtocol extends DefaultJsonProtocol {
implicit val personFormat = jsonFormat2(Person)
implicit val friendListFormat = jsonFormat1(FriendList)
}
def main(args: Array[String]): Unit = {
import FriendsProtocol._
val input = scala.io.Source.fromFile("test.json")("UTF-8").mkString.parseJson
val friendList = input.convertTo[FriendList]
println(friendList)
}
}
If I change my test file so it just has a single person not in an array and run val friendList = input.convertTo[Person] then it works and everything parses but as soon as I try and parse an array it fails with the error Object expected in field 'items'
Can anyone point me the direction of what I'm doing wrong?
Well, as is often the way immediately after posting something to StackOverflow after spending hours trying to get something working, I've managed to get this to work.
The correct implementation of FriendsProtocol was:
object FriendsProtocol extends DefaultJsonProtocol {
implicit val personFormat = jsonFormat2(Person)
implicit object friendListJsonFormat extends RootJsonFormat[FriendList] {
def read(value: JsValue) = FriendList(value.convertTo[List[Person]])
def write(f: FriendList) = ???
}
}
Telling Spray how to read / write (just read in my case) the list object is enough to get it working.
Hope that helps someone else!
To make the Friend array easier to use extend the IndexedSeq[Person]trait by implementing the appropriate apply and length methods. This will allow the Standard Scala Collections API methods like map, filter and sortBy directly on the FriendsArray instance itself without having to access the underlying Array[Person] value that it wraps.
case class Person(name: String, age: String)
// this case class allows special sequence trait in FriendArray class
// this will allow you to use .map .filter etc on FriendArray
case class FriendArray(items: Array[Person]) extends IndexedSeq[Person] {
def apply(index: Int) = items(index)
def length = items.length
}
object FriendsProtocol extends DefaultJsonProtocol {
implicit val personFormat = jsonFormat2(Person)
implicit object friendListJsonFormat extends RootJsonFormat[FriendArray] {
def read(value: JsValue) = FriendArray(value.convertTo[Array[Person]])
def write(f: FriendArray) = ???
}
}
import FriendsProtocol._
val input = jsonString.parseJson
val friends = input.convertTo[FriendArray]
friends.map(x => println(x.name))
println(friends.length)
This will then print:
John
Tom
2
I'm using scala and slick here, and I have a baserepository which is responsible for doing the basic crud of my classes.
For a design decision, we do have updatedTime and createdTime columns all handled by the application, and not by triggers in database. Both of this fields are joda DataTime instances.
Those fields are defined in two traits called HasUpdatedAt, and HasCreatedAt, for the tables
trait HasCreatedAt {
val createdAt: Option[DateTime]
}
case class User(name:String,createdAt:Option[DateTime] = None) extends HasCreatedAt
I would like to know how can I use reflection to call the user copy method, to update the createdAt value during the database insertion method.
Edit after #vptron and #kevin-wright comments
I have a repo like this
trait BaseRepo[ID, R] {
def insert(r: R)(implicit session: Session): ID
}
I want to implement the insert just once, and there I want to createdAt to be updated, that's why I'm not using the copy method, otherwise I need to implement it everywhere I use the createdAt column.
This question was answered here to help other with this kind of problem.
I end up using this code to execute the copy method of my case classes using scala reflection.
import reflect._
import scala.reflect.runtime.universe._
import scala.reflect.runtime._
class Empty
val mirror = universe.runtimeMirror(getClass.getClassLoader)
// paramName is the parameter that I want to replacte the value
// paramValue is the new parameter value
def updateParam[R : ClassTag](r: R, paramName: String, paramValue: Any): R = {
val instanceMirror = mirror.reflect(r)
val decl = instanceMirror.symbol.asType.toType
val members = decl.members.map(method => transformMethod(method, paramName, paramValue, instanceMirror)).filter {
case _: Empty => false
case _ => true
}.toArray.reverse
val copyMethod = decl.declaration(newTermName("copy")).asMethod
val copyMethodInstance = instanceMirror.reflectMethod(copyMethod)
copyMethodInstance(members: _*).asInstanceOf[R]
}
def transformMethod(method: Symbol, paramName: String, paramValue: Any, instanceMirror: InstanceMirror) = {
val term = method.asTerm
if (term.isAccessor) {
if (term.name.toString == paramName) {
paramValue
} else instanceMirror.reflectField(term).get
} else new Empty
}
With this I can execute the copy method of my case classes, replacing a determined field value.
As comments have said, don't change a val using reflection. Would you that with a java final variable? It makes your code do really unexpected things. If you need to change the value of a val, don't use a val, use a var.
trait HasCreatedAt {
var createdAt: Option[DateTime] = None
}
case class User(name:String) extends HasCreatedAt
Although having a var in a case class may bring some unexpected behavior e.g. copy would not work as expected. This may lead to preferring not using a case class for this.
Another approach would be to make the insert method return an updated copy of the case class, e.g.:
trait HasCreatedAt {
val createdAt: Option[DateTime]
def withCreatedAt(dt:DateTime):this.type
}
case class User(name:String,createdAt:Option[DateTime] = None) extends HasCreatedAt {
def withCreatedAt(dt:DateTime) = this.copy(createdAt = Some(dt))
}
trait BaseRepo[ID, R <: HasCreatedAt] {
def insert(r: R)(implicit session: Session): (ID, R) = {
val id = ???//insert into db
(id, r.withCreatedAt(??? /*now*/))
}
}
EDIT:
Since I didn't answer your original question and you may know what you are doing I am adding a way to do this.
import scala.reflect.runtime.universe._
val user = User("aaa", None)
val m = runtimeMirror(getClass.getClassLoader)
val im = m.reflect(user)
val decl = im.symbol.asType.toType.declaration("createdAt":TermName).asTerm
val fm = im.reflectField(decl)
fm.set(??? /*now*/)
But again, please don't do this. Read this stackoveflow answer to get some insight into what it can cause (vals map to final fields).
Say you have a class Person, and create a collection class for it by extending e.g. ArrayBuffer:
class Persons extends ArrayBuffer[Person] {
// methods operation on the collection
}
Now, with ArrayBuffer, can create a collection with the apply() method on the companion object, e.g.:
ArrayBuffer(1, 2, 3)
You want to be able to do the same with Persons, e.g.:
Persons(new Person("John", 32), new Person("Bob", 43))
My first intuition here was to extend the ArrayBuffer companion object and getting the apply() method for free. But it seems that you can't extend objects. (I'm not quite sure why.)
The next idea was to create a Persons object with an apply() method that calls the apply
method of ArrayBuffer:
object Persons {
def apply(ps: Person*) = ArrayBuffer(ps: _*)
}
However, this returns an ArrayBuffer[Person] and not a Persons.
After some digging in the scaladoc and source for ArrayBuffer, I came up with the following, which I thought would make the Persons object inherit apply() from GenericCompanion:
EDIT:
object Persons extends SeqFactory[ArrayBuffer] {
def fromArrayBuffer(ps: ArrayBuffer[Person]) = {
val persons = new Persons
persons appendAll ps
persons
}
def newBuilder[Person]: Builder[Person, Persons] = new ArrayBuffer[Person] mapResult fromArrayBuffer
}
However, it gives the following error message:
<console>:24: error: type mismatch;
found : (scala.collection.mutable.ArrayBuffer[Person]) => Persons
required: (scala.collection.mutable.ArrayBuffer[Person(in method newBuilder)])
=> Persons
def newBuilder[Person]: Builder[Person, Persons] = new ArrayBuffer[Perso
n] mapResult fromArrayBuffer
^
Perhaps this should disencourage me from going further, but I'm having a great time learning Scala and I'd really like to get this working. Please tell me if I'm on the wrong track. :)
Rather than extending ArrayBuffer[Person] directly, you can use the pimp my library pattern. The idea is to make Persons and ArrayBuffer[Person] completely interchangeable.
class Persons(val self: ArrayBuffer[Person]) extends Proxy {
def names = self map { _.name }
// ... other methods ...
}
object Persons {
def apply(ps: Person*): Persons = ArrayBuffer(ps: _*)
implicit def toPersons(b: ArrayBuffer[Person]): Persons = new Persons(b)
implicit def toBuffer(ps: Persons): ArrayBuffer[Person] = ps.self
}
The implicit conversion in the Persons companion object allows you to use any ArrayBuffer method whenever you have a Persons reference and vice-versa.
For example, you can do
val l = Persons(new Person("Joe"))
(l += new Person("Bob")).names
Note that l is a Persons, but you can call the ArrayBuffer.+= method on it because the compiler will automatically add in a call to Persons.toBuffer(l). The result of the += method is an ArrayBuffer, but you can call Person.names on it because the compiler inserts a call to Persons.toPersons.
Edit:
You can generalize this solution with higher-kinded types:
class Persons[CC[X] <: Seq[X]](self: CC[Person]) extends Proxy {
def names = self map (_.name)
def averageAge = {
self map (_.age) reduceLeft { _ + _ } /
(self.length toDouble)
}
// other methods
}
object Persons {
def apply(ps: Person*): Persons[ArrayBuffer] = ArrayBuffer(ps: _*)
implicit def toPersons[CC[X] <: Seq[X]](c: CC[Person]): Persons[CC] =
new Persons[CC](c)
implicit def toColl[CC[X] <: Seq[X]](ps: Persons[CC]): CC[Person] =
ps.self
}
This allows you to do things like
List(new Person("Joe", 38), new Person("Bob", 52)).names
or
val p = Persons(new Person("Jeff", 23))
p += new Person("Sam", 20)
Note that in the latter example, we're calling += on a Persons. This is possible because Persons "remembers" the underlying collection type and allows you to call any method defined in that type (ArrayBuffer in this case, due to the definition of Persons.apply).
Apart from anovstrup's solution, won't the example below do what you want?
case class Person(name: String, age: Int)
class Persons extends ArrayBuffer[Person]
object Persons {
def apply(ps: Person*) = {
val persons = new Persons
persons appendAll(ps)
persons
}
}
scala> val ps = Persons(new Person("John", 32), new Person("Bob", 43))
ps: Persons = ArrayBuffer(Person(John,32), Person(Bob,43))
scala> ps.append(new Person("Bill", 50))
scala> ps
res0: Persons = ArrayBuffer(Person(John,32), Person(Bob,43), Person(Bill,50))