I have a case class like this
case class MyModel(category:Option[String],deploymentId:Option[String],key:Option[String],metaInfo:Option[String],name:Option[String],version:Option[Int],tenantId:Option[String])
There is a Java library, our app is interacting with which returns Java Bean. Now on many occasions have to populate Java Bean from MyModel case class e.g.
val javaModel = repositoryService.newModel()
myModel.category.map(javaModel.setCategory(_))
myModel.deploymentId.map(javaModel.setDeploymentId(_))
myModel.key.map(javaModel.setKey(_))
myModel.metaInfo.map(javaModel.setMetaInfo(_))
myModel.name.map(javaModel.setName(_))
myModel.tenantId.map(javaModel.setTenantId(_))
myModel.version.map(javaModel.setVersion(_))
repositoryService.saveModel(javaModel)
Above is shorten snippet but it gets quite versbose with a ton of other properties in MyModel which corrosponds to javaModel. I am trying to come up with a generic function which iterates through case class (MyModel) and finds property names from case class members and calls appropriate setter on javaModel Java bean.
I know I can use Shapeless for getting all keys (property names) from case class as follow
val modelLabel = LabelledGeneric[MyModel]
val modelKeys = Keys[modelLabel.Repr].apply
Now I am wondering if there is a way to use modelKey e.g. say when it finds category, when mapping over modelKeys, call appropriate set method on javaModel. e.g. for category it'll be calling setCategory and likewise for other properties (this may sound odd but preferably without using reflection? )
Related
I have always seen that, when we are using a map function, we can create a dataframe from rdd using case class like below:-
case class filematches(
row_num:Long,
matches:Long,
non_matches:Long,
non_match_column_desc:Array[String]
)
newrdd1.map(x=> filematches(x._1,x._2,x._3,x._4)).toDF()
This works great as we all know!!
I was wondering , why we specifically need case classes here?
We should be able to achieve same effect using normal classes with parameterized constructors (as they will be vals and not private):-
class filematches1(
val row_num:Long,
val matches:Long,
val non_matches:Long,
val non_match_column_desc:Array[String]
)
newrdd1.map(x=> new filematches1(x._1,x._2,x._3,x._4)).toDF
Here , I am using new keyword to instantiate the class.
Running above has given me the error:-
error: value toDF is not a member of org.apache.spark.rdd.RDD[filematches1]
I am sure I am missing some key concept on case classes vs regular classes here but not able to find it yet.
To resolve error of
value toDF is not a member of org.apache.spark.rdd.RDD[...]
You should move your case class definition out of function where you are using it. You can refer http://community.cloudera.com/t5/Advanced-Analytics-Apache-Spark/Spark-Scala-Error-value-toDF-is-not-a-member-of-org-apache/td-p/29878 for mode detail.
On your Other query - case classes are syntactic sugar and they provide following additional things
Case classes are different from general classes. They are specially used when creating immutable objects.
They have default apply function which is used as constructor to create object. (so Lesser code)
All the variables in case class are by default val type. Hence immutable. which is a good thing in spark world as all red are immutable
example for case class is
case class Book( name : string)
val book1 = Book("test")
you cannot change value of book1.name as it is immutable. and you do not need to say new Book() to create object here.
The class variables are public by default. so you don't need setter and getters.
Moreover while comparing two objects of case classes, their structure is compared instead of references.
Edit : Spark Uses Following class to Infer Schema
Code Link :
https://github.com/apache/spark/blob/branch-2.4/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/ScalaReflection.scala
If you check. in schemaFor function (Line 719 to 791). It converts Scala types to catalyst types. I this the case to handle non case classes for schema inference is not added yet. so the every time you try to use non case class with infer schema. It goes to other option and hence gives error of Schema for type $other is not supported.
Hope this helps
I am wondering if there is a way to get the quick documentation in IntelliJ to work for the class construction pattern many scala developers use below.
SomeClass(Param1,Parma2)
instead of
new SomeClass(param1,Param2)
The direct constructor call made with new obviously works but many scala devs use apply to construct objects. When that pattern is used the Intelij documentation look up fails to find any information on the class.
I don't know if there are documents in IntelliJ per se. However, the pattern is fairly easy to explain.
There's a pattern in Java code for having static factory methods (this is a specialization of the Gang of Four Factory Method Pattern), often along the lines of (translated to Scala-ish):
object Foo {
def barInstance(args...): Bar = ???
}
The main benefit of doing this is that the factory controls object instantiation, in particular:
the particular runtime class to instantiate, possibly based on the arguments to the factory. For example, the generic immutable collections in Scala have factory methods which may create optimized small collections if they're created with a sufficiently small amount of contents. An example of this is a sequence of length 1 can be implemented with basically no overhead with a single field referring to the object and a lookup that checks if the offset is 0 and either throws or returns its sole field.
whether an instance is created. One can cache arguments to the factory and memoize or "hashcons" the created objects, or precreate the most common instances and hand them out repeatedly.
A further benefit is that the factory is a function, while new is an operator, which allows the factory to be passed around:
class Foo(x: Int)
object Foo {
def instance(x: Int) = new Foo(x)
}
Seq(1, 2, 3).map(x => Foo(x)) // results in Seq(Foo(1), Foo(2), Foo(3))
In Scala, this is combined with the fact that the language allows any object which defines an apply method to be used syntactically as a function (even if it doesn't extend Function, which would allow the object to be passed around as if it's a function) and with the "companion object" to a class (which incorporates the things that in Java would be static in the class) to get something like:
class Foo(constructor_args...)
object Foo {
def apply(args...): Foo = ???
}
Which can be used like:
Foo(...)
For a case class, the Scala compiler automatically generates a companion object with certain behaviors, one of which is an apply with the same arguments as the constructor (other behaviors include contract-obeying hashCode and equals as well as an unapply method to allow for pattern matching).
I'm writing a script to automatically configure sharding for some specific MongoDB collections when the app is being deployed on a fresh cluster. The application is using the Lift framework and basically every sharded collection is mapped to a MongoRecord class extending a particular "ShardedCollection" trait. I need to call a particular method on those classes in order to get their collection name.
So the first step is to find in the code those specifics classes and for that I use ClassUtil . Then I need a way to instantiate them and for that I thought that java reflection should be able to do it. It's working but only if those classes do not belong to an outer class.
The configuration in this specific edge case is like:
class X {
class Y extends ShardedCollection {
}
}
So after reading some documentation I found that I had to call YConstructor.newInstance(XObject), newInstance taking as a first argument an XObject (as an instance of X) When Y is an inner class of X. My strategy is to recursively instantiate the enclosing classes until I'm getting the one that has the ShardedCollection trait.
The problem arise when X is no more a class but a trait, and then there is no constructor that I can use for it, but I still need to feed an XObject to newInstance .. Tricky :(
To be very concise from the java doc
If the constructor's declaring class is an inner class in a non-static context, the first argument to the constructor needs to be the enclosing instance
What do I do when the enclosing "thing" is a trait ? (assuming that I can't modify anything in the code base)
I understand that using something like
case class private A()
new A()#This will be a invalid call as A is private
But what I do not understand that as from an implementation perspective, what advantage does this provide while coding? Because calling A() twice will give 2 instances of the class anyways. If this syntax is not used to prevent instantiation like Java, then why would I want to not let someone instantiate my class using new?
Marking a case class constructor private is useless. As you've notices, case classes get a synthetic companion object with an apply method whose implementation is simply a call to the actual constructor.
Scala case classes have been designed to just "classes + the case modifier", meaning that everything that works on classes also works on case classes, which also include the (pointless) ability to specify access modifiers on the constructor.
In the activator template for simple rest API project in Scala the Book.scala file looks like the following.
package models
import play.api.libs.json.Json
object Book {
case class Book(name: String, author: String)
implicit val bookWrites = Json.writes[Book]
implicit val bookReads = Json.reads[Book]
var books = List(Book("TAOCP", "Knuth"), Book("SICP", "Sussman, Abelson"))
def addBook(b: Book) = books = books ::: List(b)
}
Why is there a Book object and a Book case class inside it? Why not just a Book case class (or just a Book class and not a case class)? What advantages/disadvantages are there in the above structure?
I'm sure this is just a small example that somebody put together, and so you shouldn't read too much into it. But it exhibits what some consider an anti-pattern: nesting case classes in other classes. Some best-practices guides, such as this one, suggest avoiding nesting case classes in other classes, and for good reason:
It is tempting, but you should almost never define nested case classes
inside another object/class because it messes with Java's
serialization. The reason is that when you serialize a case class it
closes over the "this" pointer and serializes the whole object, which
if you are putting in your App object means for every instance of a
case class you serialize the whole world.
And the thing with case classes specifically is that:
one expects a case class to be immutable (a value, a fact) and hence
one expects a case class to be easily serializable
Prefer flat hierarchies.
For example, this small program throws an exception, somewhat unexpectedly:
import java.io._
class Outer {
case class Inner(a: Int)
}
object Test extends App {
val inner = (new Outer).Inner(1)
val oos = new ObjectOutputStream(new FileOutputStream("/tmp/test"))
oos.writeObject(inner)
oos.close
}
If the only purpose of this outer Book object is to group together common functionality, a package would be the preferred structure.
Furthermore, even if an object were desired for some other reason, naming that object the same as the inner case class is confusing, especially since case classes automatically generate companion objects. So in this example there is a Book object, a Book.Book case class, and therefore also a Book.Book companion object.
The role of Book object in this code is more like a static book utils/manager class which hold a list of books. You can imagine that this is a Library class, which allow to add books.
The Book case class is just an anonymous class for Book instances. As m-z said, it is just an example, for more complicated class, you could move it to a standalone Book class.