Scala case classes can have 22+ properties these days, but AFAIA compiler does not compile apply/unapply methods then.
Is there a way to generate apply/unapply by means of a plugin at compile time or at least generate methods using IDE etc?
Note
please don't start asking - why do you need this? It is for mapping existing JSON schema from a mongoDB using Reactive Mongo
please don't advise to group properties into smaller case classes and etc. Schema was created by someone else & already exists on production.
Thank you for your answers in advance.
Yes, Scala supports >22 fields from version 2.11. However, there are certain limitations - the case class will no more have unapply or unapplyseq and tupled(you'll no longer convert case class to tuple) functions because scala still don't support tuple with more that 22 values.
val tup = (1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22) //will compile
val tup = (1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23) //will fail
Because of this, case class is much more like regular class class and many other libraries will be unable to fully utilize this case class. Such as, json serializer libraries.
I have faced with this issue when I tried to use macros read/write to serialize case class to json and viceversa in a playframework project it won't compile because case class no longer contain unapply() method. The one work around for this is to provide custom implicit read/writes for the case class instead of using macros.
case class Person(name: String, age: Int, lovesChocolate: Boolean)
implicit val personReads = Json.reads[Person] //this wond work, need to write custom read as below.
implicit val personReads = (
(__ \ 'name).read[String] and
(__ \ 'age).read[Int] and
(__ \ 'lovesChocolate).read[Boolean]
)(Person)
please don't start asking - why do you need this? It is for mapping
existing JSON schema from a mongoDB using Reactive Mongo
I'm assuming your is the same situation, you are using reactivemongo macros for json to/from case class serialization.
implicit val personReader: BSONDocumentReader[Person] = Macros.reader[Person]
implicit val personWriter: BSONDocumentWriter[Person] = Macros.writer[Person]
//or Handler
Macros.handler[Person]
Therefore, I would suggest you to use custom BSON reader and writer for the case class as documented here.
Related
I have always seen that, when we are using a map function, we can create a dataframe from rdd using case class like below:-
case class filematches(
row_num:Long,
matches:Long,
non_matches:Long,
non_match_column_desc:Array[String]
)
newrdd1.map(x=> filematches(x._1,x._2,x._3,x._4)).toDF()
This works great as we all know!!
I was wondering , why we specifically need case classes here?
We should be able to achieve same effect using normal classes with parameterized constructors (as they will be vals and not private):-
class filematches1(
val row_num:Long,
val matches:Long,
val non_matches:Long,
val non_match_column_desc:Array[String]
)
newrdd1.map(x=> new filematches1(x._1,x._2,x._3,x._4)).toDF
Here , I am using new keyword to instantiate the class.
Running above has given me the error:-
error: value toDF is not a member of org.apache.spark.rdd.RDD[filematches1]
I am sure I am missing some key concept on case classes vs regular classes here but not able to find it yet.
To resolve error of
value toDF is not a member of org.apache.spark.rdd.RDD[...]
You should move your case class definition out of function where you are using it. You can refer http://community.cloudera.com/t5/Advanced-Analytics-Apache-Spark/Spark-Scala-Error-value-toDF-is-not-a-member-of-org-apache/td-p/29878 for mode detail.
On your Other query - case classes are syntactic sugar and they provide following additional things
Case classes are different from general classes. They are specially used when creating immutable objects.
They have default apply function which is used as constructor to create object. (so Lesser code)
All the variables in case class are by default val type. Hence immutable. which is a good thing in spark world as all red are immutable
example for case class is
case class Book( name : string)
val book1 = Book("test")
you cannot change value of book1.name as it is immutable. and you do not need to say new Book() to create object here.
The class variables are public by default. so you don't need setter and getters.
Moreover while comparing two objects of case classes, their structure is compared instead of references.
Edit : Spark Uses Following class to Infer Schema
Code Link :
https://github.com/apache/spark/blob/branch-2.4/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/ScalaReflection.scala
If you check. in schemaFor function (Line 719 to 791). It converts Scala types to catalyst types. I this the case to handle non case classes for schema inference is not added yet. so the every time you try to use non case class with infer schema. It goes to other option and hence gives error of Schema for type $other is not supported.
Hope this helps
I am using uPickle/ScalaJS to deserialize a js.Dynamic object into a case class using this code fragment:
read[myClass](JSON.stringify(dynObj))
where myClass is the case class and dynObj is the js.Dynamic object.
Is there a boilerplate-free and simpler way to do this?
In order to serialize a case class, I have been able to serialize to js.Dynamic using Shapeless using this example as a starting point:
Converting nested case classes to nested Maps using Shapeless
I would like to be able to use uPickle to do this instead. How can I accomplish the round-trip with uPickle?
upickle.default.readJs[myClass](upickle.json.readJs(dynObj))
Should do it. You can wrap it in a nice helper if you find yourself doing it a lot.
Similar calls exist to write things to js.Dynamic, just the other way round
upickle.json.writeJs(upickle.default.writeJs[myClass](myClassInstance))
Though you can probably leave out the type parameter here since it'll be inferred
The answer above no longer applies for newer versions of upickle. In version 0.6.5 I had to use the following to deserialize a dynamic object:
val someJsObject: js.Dynamic = ...
upickle.WebJson.transform(someJsObject, implicitly[upickle.default.Reader[TargetType]])
To serialize, you will probably want something like:
val sourceObject: SourceType = ...
implicitly[upickle.default.Writer[SourceType]].write(upickle.WebJson.Builder, sourceObject)
I would like to define my Primary Keys as specific types - not just Long or String
For example
case class Project(
var id: ProjectId = 0,
One advantage of this is if I accidently compare different keys - then the compiler will pick it up.
Obviously this gives the compile error
overriding method id in trait KeyedEntity of type => Long;
variable id has incompatible type
Are there any example's where this type of approach is successfully implemented?
Appendix - a draft of what ProjectId could be
trait SelfType[T] {
val self : T
}
class Content_typeId( val self: Int) extends SelfType[Int]
class ProjectId( val self: Long) extends SelfType[Long]
object ProjectId {
implicit def baseToType(self: Long) = new ProjectId(self)
implicit def typeToBase(higherSelf: ProjectId) : Long = higherSelf.self
}
Thanks
Brent
Yup, it can be done, but you are going to want to upgrade to Squeryl 0.9.6. The latest available is RC3 at the moment. There are 2 changes that you'll want to take advantage of:
You no longer need to extend KeyedEntity. Instead, you can define an implicit KeyedEntityDef that Squeryl will use to determine which field(s) of your object constitute the primary key.
Squeryl 0.9.6 allows you to extend what types are supported using type classes.
RC3 is very stable, I'm using it myself in several production projects, but there is no official documentation for these features yet. You can find examples on the list, where I see you've also posted this question.
I'd also suggest looking at how both PrimitiveTypeMode (the process of exposing a TEF for a type) and PrimitiveTypeSupport (which is where the TypedExpressionFactory instances are defined). KeyedEntity itself is supported with a KeyedEntityDef By Squeryl and looking at that code may be helpful as well.
I am writing a play2 app that gets data via rest/json and stores it in mongodb using reactivemongo.
I am using a model built from case classes and implicit val myFormat = Json.format[myCaseClass]
Currently I have a case class for objects coming from mongodb. They contain the _id field and everything works. New objects coming in do naturally don't have this id field and so the Json.fromJson[myCaseClass](req.body) validator fails.
Do I really have to create another case class for new objects or is there a more DRY and elegant solution without duplicating the class and removing the _id?
I would use the parser combinator API and create a json format, or maybe even just a Reads[T], that handles incoming possibly id-less fields. Something like:
implicit val readsMyClass: Reads[MyClass] = (
(__ \ "id").readNullable[Id] and
(__ \ "someProperty").read[String]
)(create _)
def create(maybeId: Option[Id], someProperty: String) =
MyClass(maybeId.getOrElse(...generate id...), someProperty)
See the docs for more info: http://www.playframework.com/documentation/2.2.x/ScalaJsonCombinators
I followed the suggestions and _id: Option[BSONObjectID] does the trick.
It was not necessary to implement a reader because implicit val userFormat = Json.format[User] is able to create a macro containing the options.
Getting started with Scala and hunting around for Orika modules but for Scala. It is possible to create custom property builders in Orika but for stuff like case cases with value fields I would need to (with Scala 2.10+) reflex fields (mirroring) for setting immutables. Assuming there is a native approach with a Scala module?
I know Orika has lot more features but, if you just want to create a copy of an instance to a new instance and change some attributes of the new instance, scala has inbuilt feature for it. Use case classes and you can use copy method on it to create new instances.
case class Bird(name: String, color: String)
scala> val chicken = Bird("twitty", "yellow")
chicken: Bird = Bird(twitty,yellow)
scala> val coq = chicken.copy(color = "red")
coq: Bird = Bird(twitty,red)