I'm learning MongoDB and Casbah by writing a simple app. Got stuck when I try to convert an object with a list member into a MongoDB Object. Here is my class
case class BorrowerRecord( name: String, checkedOut: List[BookTag]) {
require(!name.isEmpty)
require(!checkedOut.isEmpty)
}
case class BookTag (subject: Subject, bookName: String) {
require(!bookName.isEmpty)
}
case class Subject (name: String, category: Category) {
require(!name.isEmpty)
}
Category is a sealed trait with 2 case class implementation, I intended to use this like "Enum"
sealed trait Category {
def name: String
}
object Category {
case object Computing extends Category { val name = "Computing"}
case object Math extends Category { val name = "Math"}
}
So, a instance of BorrowerRecord will keep what books a person checked out from the library, each book is identified by a BookTag object. A BookTag keeps some information about a book like bookname, subject name, Category, etc.
Lets say I've a BorrowerRecord and want to save it to MongoDB
val borrowOnToday = BorrowerRecord( "My Name", List( BookTag(Subject("J2EE", Category.Computing), "Head First Java"),
BookTag(Subject("Linear Algebra", Category.Math), "Algebra for Dummies")))
How should I convert this to MongoDBObject using Casbah ?
Or Casbah is not the way to go and there're other libraries that can help me persist this into MongoDB more easily?
To work with case classes use the salat (press <- and -> to move through the presentation).
It is pretty simple:
case class Alpha(x: String)
scala> val a = Alpha(x = "Hello world")
a: com.novus.salat.test.model.Alpha = Alpha(Hello world)
scala> val dbo = grater[Alpha].asDBObject(a)
dbo: com.mongodb.casbah.Imports.DBObject = { "_typeHint" :
"com.novus.salat.test.model.Alpha" , "x" : "Hello world"}
scala> val a_* = grater[Alpha].asObject(dbo)
a_*: com.novus.salat.test.model.Alpha = Alpha(Hello world)
Usually, I'm using them both: casbah to querying to/from Mongo, and salat to make a conversions to case classes and vice versa.
And yes, salat supports case classes with Lists (here is the list of supported collections).
I use my own library Subset (I've open-sourced it recently) along with MongoDB Java driver. Unlike Salat it's explicit, you have to declare all the serialization code, though Subset helps keep it quite simple. You'll get ability to create queries as a bonus.
For your data model, the code may look like
object BorrowerRecord {
val name = "name".fieldOf[String]
val checkedOut = "cout".fieldOf[List[BookTag]]
def toDBO(rec: BorrowerRecord): DBObject =
name(rec.name) ~ checkedOut(rec.checkedOut)
}
Subset knows how to serialize List[T], but it needs an implicit ValueWriter[BookTag] for that:
object BookTag {
val subject = "subj".fieldOf[Subject]
val name = "name".fieldOf[String]
implicit def writer = ValueWriter[BookTag](bt =>
(subject(bt.subject) ~ name(bt.name)).get
)
}
I hope you got the idea to continue with Subject and Category
Related
I'm using elastic4s library to query elasticsearch (ES). Version of elastic4s and ES itself 2.4.0.
Suppose I have a compound object that I put to ES like
case class MyObject(id: Long, vall: KeyVal, vals: Seq[KeyVal])
where KeyVal is
case class KeyVal(id: Long, name: String)
Now I queried ES and got the response which I want to deserialiize back to MyObject:
implicit object MyObjectHitAs extends HitAs[MyObject] {
override def as(hit: RichSearchHit): MyObject = {
MyObject(
hit.field("id").getValue[String]
KeyVal(hit.field("vall.id").getValue[Long], field("vall.name").getValue[String]),
//what should I code here to get the Seq[KeyVal] ???
)
}
}
Please explain how can I deserialize the Array of KeyVal. Thank you.
In the more recent versions of elastic4s, ie 5.0 onwards, you would use the HitReader typeclass. Your example would then look like this.
implicit object MyObjectHitAs extends HitReader[MyObject] {
override def read(hit: Hit): Either[Throwable, MyObject] = {
val obj = MyObject(
hit.sourceField("id").toString.toLong,
KeyVal(hit.sourceField("vall.id").toString.toLong, hit.sourceField("vall.name").toString),
hit.sourceField("vals").asInstanceOf[Seq[AnyRef]].map { entry =>
KeyVal(hit.sourceField("vall.id").toString.toLong, hit.sourceField("vall.name").toString)
}
)
Right(obj)
}
}
Although it is a lot easier to use the built in json mappers than hand craft it.
I'm attempting to write some code that tracks changes to a record and applies them at a later date. In a dynamic language I'd do this by simply keeping a log of List[(String, Any)] pairs, and then simply applying these as an update to the original record when I finally decide to commit the changes.
I need to be able to introspect over the updates, so a list of update functions isn't appropriate.
In Scala this is fairly trivial using reflection, however I'd like to implement a type-safe version.
My first attempt was to try with shapeless. This works well if we know specific types.
import shapeless._
import record._
import syntax.singleton._
case class Person(name:String, age:Int)
val bob = Person("Bob", 31)
val gen = LabelledGeneric[Person]
val updated = gen.from( gen.to(bob) + ('age ->> 32) )
// Result: Person("Bob", 32)
However I can't figure out how to make this work generically.
trait Record[T]
def update( ??? ):T
}
Given the way shapeless handles this, I'm not sure if this would even be possible?
If I accept a lot of boilerplate, as a poor mans version I could do something along the lines of the following.
object Contact {
sealed trait Field[T]
case object Name extends Field[String]
case object Age extends Field[Int]
}
// A typeclass would be cleaner, but too verbose for this simple example.
case class Contact(...) extends Record[Contact, Contact.Field] {
def update[T]( field:Contact.Field[T], value:T ) = field match {
case Contact.Name => contact.copy( name = value )
case Contact.Age => contact.copy( age = value )
}
}
However this isn't particularly elegant and requires a lot of boilerplate. I could probably write my own macro to handle this, however it seems like a fairly common thing - is there a way to handle this with Shapeless or a similar macro library already?
How about using the whole instance of the class as an update?
case class Contact(name: String, age: Int)
case class ContactUpdate(name: Option[String] = None, age: Option[Int] = None)
object Contact {
update(target: Contact, delta: ContactUpdate) = Contact(
delta.name.getOrElse(target.name)
target.age.getOrElse(delta.age)
)
}
// also, optionally this:
object ContactUpdate {
apply(name: String) = ContactUpdate(name = Option(name))
apply(age: Int) = ContactUpdate(age = Option(age))
}
I think, if you want the really type-safe solution, this is the cleanest and most readable, and also, possibly the least pain to implement, as you don't need to deal with Records, lenses and individual field descriptors, just ContactUpdate(name="foo") creates an update, and updates.map(Contact.update(target, _)) applies them all in sequence.
I described model as case class WorkAreaType(name: String) and findAll method
object WorkAreaType {
import system.db.Mongo.JsonFormats._
def findAll = Await.result(Mongo.workAreaTypes.find(Json.obj()).cursor[WorkAreaType].collect[List](), 3 seconds)
}
JsonFormats:
object JsonFormats {
implicit val workAreaTypeFormat = Json.format[WorkAreaType]
}
Mongo.workAreaTypes is a JSONCollection val workAreaTypes: JSONCollection = db.collection[JSONCollection]("WorkAreaTypes")
How can I read mongo _id object after call WorkAreaType.findAll in list?
The quick but not so great way is to define your case class as
case class WorkAreaType(_id:Option[BSONObjectID] , name: String)
The reason this is not so great is that this will make reactive mongo "leak" through your repository/dao/storage layer whatever you name it.
A "better" option would be to manipulate the json to "flatten"the object id before you read it to a scala object and inflate it after you write it back to json. This can be accomplished using Json transformers or JsZippers but takes a bit of research.
Schema.org is markup vocabulary (for the web) and defines a number of types in terms of properties (no methods). I am currently trying to model parts of that schema in Scala as internal model classes to be used in conjunction with a document-oriented database (MongoDB) and a web framework.
As can be seen in the definition of LocalBusiness, schema.org uses multiple inheritance to also include properties from the "Place" type. So my question is: How would you model such a schema in Scala?
I have come up with two solutions so far. The first one use regular classes to model a single inheritance tree and uses traits to mixin those additional properties.
trait ThingA {
var name: String = ""
var url: String = ""
}
trait OrganizationA {
var email: String = ""
}
trait PlaceA {
var x: String = ""
var y: String = ""
}
trait LocalBusinessA {
var priceRange: String = ""
}
class OrganizationClassA extends ThingA with OrganizationA {}
class LocalBusinessClassA extends OrganizationClassA with PlaceA with LocalBusinessA {}
The second version tries to use case classes. However, since case class inheritance is deprecated, I cannot model the main hierarchy so easily.
trait ThingB {
val name: String
}
trait OrganizationB {
val email: String
}
trait PlaceB {
val x: String
val y: String
}
trait LocalBusinessB {
val priceRange: String
}
case class OrganizationClassB(val name: String, val email: String) extends ThingB with OrganizationB
case class LocalBusinessClassB(val name: String, val email: String, val x: String, val y: String, val priceRange: String) extends ThingB with OrganizationB with PlaceB with LocalBusinessB
Is there a better way to model this? I could use composition similar to
case class LocalBusinessClassC(val thing:ThingClass, val place: PlaceClass, ...)
but then of course, LocalBusiness cannot be used when a "Place" is expected, for example when I try to render something on Google Maps.
What works best for you depends greatly on how you want to map your objects to the underlying datastore.
Given the need for multiple inheritance, and approach that might be worth considering would be to just use traits. This gives you multiple inheritance with the least amount of code duplication or boilerplating.
trait Thing {
val name: String // required
val url: Option[String] = None // reasonable default
}
trait Organization extends Thing {
val email: Option[String] = None
}
trait Place extends Thing {
val x: String
val y: String
}
trait LocalBusiness extends Organization with Place {
val priceRange: String
}
Note that Organization extends Thing, as does Place, just as in schema.org.
To instantiate them, you create anonymous inner classes that specify the values of all attributes.
object UseIt extends App {
val home = new Place {
val name = "Home"
val x = "-86.586104"
val y = "34.730369"
}
val oz = new Place {
val name = "Oz"
val x = "151.206890"
val y = "-33.873651"
}
val paulis = new LocalBusiness {
val name = "Pauli's"
override val url = "http://www.paulisbarandgrill.com/"
val x = "-86.713660"
val y = "34.755092"
val priceRange = "$$$"
}
}
If any fields have a reasonable default value, you can specify the default value in the trait.
I left fields without value as empty strings, but it probably makes more sense to make optional fields of type Option[String], to better indicate that their value is not set. You liked using Option, so I'm using Option.
The downside of this approach is that the compiler generates an anonymous inner class every place you instantiate one of the traits. This could give you an explosion of .class files. More importantly, though, it means that different instances of the same trait will have different types.
Edit:
In regards to how you would use this to load objects from the database, that depends greatly on how you access your database. If you use an object mapper, you'll want to structure your model objects in the way that the mapper expects them to be structured. If this sort of trick works with your object mapper, I'll be surprised.
If you're writing your own data access layer, then you can simply use a DAO or repository pattern for data access, putting the logic to build the anonymous inner classes in there.
This is just one way to structure these objects. It's not even the best way, but it demonstrates the point.
trait Database {
// treats objects as simple key/value pairs
def findObject(id: String): Option[Map[String, String]]
}
class ThingRepo(db: Database) {
def findThing(id: String): Option[Thing] = {
// Note that in this way, malformed objects (i.e. missing name) simply
// return None. Logging or other responses for malformed objects is left
// as an exercise :-)
for {
fields <- db.findObject(id) // load object from database
name <- field.get("name") // extract required field
} yield {
new Thing {
val name = name
val url = field.get("url")
}
}
}
}
There's a bit more to it than that (how you identify objects, how you store them in the database, how you wire up repository, how you'll handle polymorphic queries, etc.). But this should be a good start.
I'm in the process of learning Scala for a new project having come from Rails. I've defined a type that is going to be used in a number of my models which can basically be thought of as collection of 'attributes'. It's basically just a wrapper for a hashmap that delegates most of its responsibilities to it:
case class Description(attributes: Map[String, String]) {
override def hashCode: Int = attributes.hashCode
override def equals(other: Any) = other match {
case that: Description => this.attributes == that.attributes
case _ => false
}
}
So I would then define a model class with a Description, something like:
case class Person(val name: String, val description: Description)
However, when I persist a Person with a SalatDAO I end up with a document that looks like this:
{
name : "Russell",
description:
{
attributes:
{
hair: "brown",
favourite_color: "blue"
}
}
}
When in actual fact I don't need the nesting of the attributes tag in the description tag - what I actually want is this:
{
name : "Russell",
description:
{
hair: "brown",
favourite_color: "blue"
}
}
I haven't tried, but I reckon I could get that to work if I made Description extend a Map rather than contain one, but I'd rather not, because a Description isn't a type of Map, it's something which has some of the behaviour of a Map as well as other behaviour of its own I'm going to add later. Composition over inheritance and so on.
So my question is, how can I tell Salat (or Casbah, I'm actually a bit unclear as to which is doing the conversion as I've only just started using them) how to serialize and deserialize the Description class? In the casbah tutorial here it says:
It is also possible to create your own custom type serializers and
deserializers. See Custom Serializers and Deserializers.
But this page doesn't seem to exist. Or am I going about it the wrong way? Is there actually a really simple way to indicate this is what I want to happen, an annotation or something? Or can I simply delegate the serialization to the attributes map in some way?
EDIT: After having a look at the source for the JodaTime conversion helper I've tried the following but have had no luck getting it to work yet:
import org.bson.{ BSON, Transformer }
import com.mongodb.casbah.commons.conversions.MongoConversionHelper
object RegisterCustomConversionHelpers extends Serializers
with Deserializers {
def apply() = {
super.register()
}
}
trait Serializers extends MongoConversionHelper
with DescriptionSerializer {
override def register() = {
super.register()
}
override def unregister() = {
super.unregister()
}
}
trait Deserializers extends MongoConversionHelper {
override def register() = {
super.register()
}
override def unregister() = {
super.unregister()
}
}
trait DescriptionSerializer extends MongoConversionHelper {
private val transformer = new Transformer {
def transform(o: AnyRef): AnyRef = o match {
case d: Description => d.attributes.asInstanceOf[AnyRef]
case _ => o
}
}
override def register() = {
BSON.addEncodingHook(classOf[Description], transformer)
super.register()
}
}
When I call RegisterCustomConversionHelpers() then save a Person I don't get any errors, it just has no effect, saving the document the same way as ever. This also seems like quite a lot to have to do for what I want.
Salat maintainer here.
I don't understand the value of Description as a wrapper here. It wraps a map of attributes, overrides the default equals and hashcode impl of a case class - which seems unnecessary since the impl is delegated to the map anyhow and that is exactly what the case class does anyway - and introduces an additional layer of indirection to the serialized object.
Have you considered just:
case class Person(val name: String, val description: Map[String, String])
This will do exactly what you want out of box.
In another situation I would recommend a simple type alias but unfortunately Salat can't support type aliases right now due to some issues with how they are depicted in pickled Scala signatures.
(You probably omitted this from your example from brevity, but it is best practice for your Mongo model to have an _id field - if you don't, the Mongo Java driver will supply one for you)
There is a working example of a custom BSON hook in the salat-core test package (it handles java.net.URL). It could be that your hook is not working simply because you are not registering it in the right place? But still, I would recommend getting rid of Description unless it is adding some value that is not evident from your example above.
Based on #prasinous' answer I decided this wasn't going to be that easy so I've changed my design a bit to the following, which pretty much gets me what I want. Rather than persisting the Description as a field I persist a vanilla map then mix in a Described trait to the model classes I want to have a description, which automatically converts the map to Description when the object is created. Would appreciate it if anyone can point out any obvious problems to this approach or any suggestions for improvement.
class Description(val attributes: Map[String, String]){
//rest of class omitted
}
trait Described {
val attributes: Map[String, String]
val description = new Description(attributes)
}
case class Person(name: String, attributes: Map[String, String]) extends Described