How to parse this "pretty" BSON in Scala? - scala

In our project we are using Scala and Reactivemongo. (I'm very new to both)
When you print to the console a "pretty" Bson, it looks like this:
{ _id: BSONObjectID("52b006fe0100000100d47242"),
desc: BSONString({"_id:"BSONObjectID(52af03a5010000010036194d),"desc:"BSONString(www.foo.com"hits),"domains:"["0:"BSONString(www.foo.com)"],"func:"BSONString(Count),"maxr:"BSONInteger(5),"props:"["]"} hits),
domains: [
0: BSONString(jsonString)
],
func: BSONString(Count),
maxr: BSONInteger(5),
props: [
]
}
I need to be able to parse it back from the console into a corosponding Case Class.
Any help please?

taken from typesafe's activator template, you can simply use Json.format as an implicit val in the case class's companion object (ReactiveMongo 0.9, scala 2.10.2). example:
package models
import play.api.libs.json.Json
import reactivemongo.bson.BSONObjectID
import play.modules.reactivemongo.json.BSONFormats._
/**
* A message class
*
* #param _id The BSON object id of the message
* #param message The message
*/
case class Message(_id: BSONObjectID, message: String)
object Message {
/**
* Format for the message.
*
* Used both by JSON library and reactive mongo to serialise/deserialise a message.
*/
implicit val messageFormat = Json.format[Message]
}
i am using it, and you can use more parameters as long as JSON knows how to format them, or, if you have members that are case classes you created, if the have the same:
package models
import play.api.libs.json.Json
import reactivemongo.bson.BSONObjectID
import play.modules.reactivemongo.json.BSONFormats._
/**
* A message class
*
* #param _id The BSON object id of the message
* #param message The message
*/
case class Name(fName: String, lName: String, mInitial: String)
object Name {
implicit val nameFormat = Json.format[Name]
}
case class Message(_id: BSONObjectID, message: String, name: Name)
object Message {
/**
* Format for the message.
*
* Used both by JSON library and reactive mongo to serialise/deserialise a message.
*/
implicit val messageFormat = Json.format[Message]
}
i still have not figured a way to do it if you have auxiliary constructors, or if you implement apply(...) in the companion. however, the compiler will alert you to that.

You can check how they did it at this link in Reactivemongo documentation: http://stephane.godbillon.com/2012/10/18/writing-a-simple-app-with-reactivemongo-and-play-framework-pt-1.html
implicit object ArticleBSONReader extends BSONReader[Article] {
def fromBSON(document: BSONDocument) :Article = {
val doc = document.toTraversable
Article(
doc.getAs[BSONObjectID]("_id"),
doc.getAs[BSONString]("title").get.value,
doc.getAs[BSONString]("content").get.value,
doc.getAs[BSONString]("publisher").get.value,
doc.getAs[BSONDateTime]("creationDate").map(dt => new DateTime(dt.value)),
doc.getAs[BSONDateTime]("updateDate").map(dt => new DateTime(dt.value)))
}
}

Here a solution I'm using. I hope it helps ->
def bsonToString(bson: BSONDocument): String = {
val bsonAsKeyValueList = bson.toMap.toList.map {
case (key, document: BSONDocument) =>
(key, bsonToString(document))
case (key, value: BSONValue) =>
(key, value.toString)
}
val bsonAsKeyValueStrings = bsonAsKeyValueList.map {
case (key, value) =>
s"""("${key}" -> ${value})"""
}
bsonAsKeyValueStrings.mkString("BSONDocument", ", ", "")
}

Related

Scala implicit class

I am seeing some unexpected behavior with an implicit class. I have a case class that is retrieved from Mongo, and in certain cases wrapped into an implicit class with convenience lazy vals to retrieve data from other relevant collections. Code looks something like
case class MyClass(
_id: UUID, ...,
//some fields
)
and then
object MyImplicits {
implicit class MyRichClass(c: MyClass){
lazy val field1: Future[Iterable[T1]] = db.findAll[T1](bson)
lazy val field2: Future[Iterable[T2]] = db.findAll[T2](bson)
}
}
and the conversion is used like
import ...MyImplicits.MyRichClass
val c: MyClass...
c.field1.map(...)
My problem is Futures in field1 and field2 always contain an empty Iterable. I have spent best part of today debugging my queries, and they work correctly both in test code and Mongo cli.
Any tips would be greatly appreciated.
I have a wrapper around org.mongodb.scala.MongoClient, that has:
/**
* Finds all instances of T using raw mongo query
*
* #param bson search query
* #tparam T
* #return
*/
private[service] def findAll[T: ClassTag](bson: Bson): Future[Iterable[T]] = {
val colName = resolveCollection[T]
logger.info("colName = " + colName)
db.getCollection[T](colName).find(equal("ownerId", "user1")).toFuture().map(t => {
logger.info("t = " + t)
t
})
}
/**
* Returns collection name for given type
* #tparam T
* #return
*/
private def resolveCollection[T: ClassTag]: String = {
scala.reflect.classTag[T] match {
case `...tag` => "collectionName"
....
}
}
The
equal("ownerId", "user1")
is hardcoded for debugging purposes, as well as extra loggers.
There is actually no issues with the code. I was literally querying a wrong database, so, yes - all collections were in fact empty.

JSON4S does not serialize internal case class members

I have a case class inheriting from a trait:
trait Thing {
val name: String
val created: DateTime = DateTime.now
}
case class Door(override val name: String) extends Thing
This is akka-http, and I'm trying to return JSON to a get request:
...
~
path ("get" / Segment) { id =>
get {
onComplete(doorsManager ? ThingsManager.Get(id)) {
case Success(d: Door) => {
complete(200, d)
}
case Success(_) => {
complete(404, s"door $id not found")
}
case Failure(reason) => complete(500, reason)
}
}
} ~
...
but I only get the JSON of name. I do have the implicit Joda serializers in scope.
if i override the 'created' timestamp in the constructor of the case class, it does get serialized, but it defines the purpose, as I don't need (or want) the user to provide the timestamp. I've tried moving the timestamp into Door (either as override or just by skipping the trait) and the result is the same (that is, no 'created').
how do I tell JSON4S to serialize internal members (and inherited ones) too?
You have to define a custom format.
import org.json4s.{FieldSerializer, DefaultFormats}
import org.json4s.native.Serialization.write
case class Door(override val name: String) extends Thing
trait Thing {
val name: String
val created: DateTime = DateTime.now
}
implicit val formats = DefaultFormats + FieldSerializer[Door with Thing()]
val obj = new Door("dooor")
write(obj)

Return JSON errors in akka-http API

In an API I'm writing, I want to take and return JSON, even in cases of errors. I'm trying to figure out how to preserve all of the default RejectionHandler behavior, but convert the status codes and text into a JSON object. As the default behavior is specified within function calls, rather than as a data structure, it seems that the only way to do this is to convert the HttpEntity of the result it produces. Is there a simple way to do this?
You can write something like this in your HttpService
private val defaultRejectionHandler = RejectionHandler.default
implicit def myRejectionHandler =
RejectionHandler.newBuilder()
.handleAll[Rejection] { rejections ⇒
def prefixEntity(entity: ResponseEntity): ResponseEntity = entity match {
case HttpEntity.Strict(contentType, data) => {
import spray.json._
val text = ErrorResponse(0, "Rejection", data.utf8String).toJson.prettyPrint
HttpEntity(ContentTypes.`application/json`, text)
}
case _ =>
throw new IllegalStateException("Unexpected entity type")
}
mapResponseEntity(prefixEntity) {
defaultRejectionHandler(rejections).getOrElse {
complete(StatusCodes.InternalServerError)
}
}
}.handleNotFound {
complete(StatusCodes.Forbidden -> ErrorResponse(StatusCodes.NotFound.intValue, "NotFound", "Requested resource is not found"))
}.result()
where ErrorResponse is possibly
case class ErrorResponse(error: ErrorInfo)
case class ErrorInfo(code: Int, `type`: String, message: String)
for which you can defined json marshallers.
I had my pieced together my own version, but it had some rough edges I didn't like. ruslan's answer gave me some ideas for improvement. Here's what I came up with, synthesizing the best of both approaches:
/**
* Modifies the Akka-Http default rejection handler to wrap the default
* message in JSON wrapper, preserving the original status code.
*
* #param rejectionWrapper wraps the message in a structure to format the
* resulting JSON object
* #param writer writer for the wrapper type
* #tparam WrapperType type of the wrapper
* #return the modified rejection handler
*/
def defaultRejectionHandlerAsJson[WrapperType](rejectionWrapper: String => WrapperType)(implicit writer: JsonWriter[WrapperType]) = {
def rejectionModifier(originalMessage: String): String = {
writer.write(rejectionWrapper(originalMessage)).prettyPrint
}
modifiedDefaultRejectionHandler(rejectionModifier, ContentTypes.`application/json`)
}
/**
* Modifies the Akka-Http default rejection handler, converting the default
* message to some other textual representation.
*
* #param rejectionModifier the modifier function
* #param newContentType the new Content Type, defaulting to text/plain
* UTF-8
* #return the modified rejection handler
*/
def modifiedDefaultRejectionHandler(rejectionModifier: String => String, newContentType: ContentType.NonBinary = ContentTypes.`text/plain(UTF-8)`) = new RejectionHandler {
def repackageRouteResult(entity: ResponseEntity): ResponseEntity = entity match {
// If the entity isn't Strict (and it definitely will be), don't bother
// converting, just throw an error, because something's weird.
case strictEntity: HttpEntity.Strict =>
val modifiedMessage = rejectionModifier(strictEntity.data.utf8String)
HttpEntity(newContentType, modifiedMessage)
case other =>
throw new Exception("Unexpected entity type")
}
def apply(v1: Seq[Rejection]): Option[Route] = {
// The default rejection handler should handle all possible rejections,
// so if this isn't the case, return a 503.
val originalResult = RejectionHandler.default(v1).getOrElse(complete(StatusCodes.InternalServerError))
Some(mapResponseEntity(repackageRouteResult) {
originalResult
})
}
}

too many arguments for method apply: (car: play.api.data.Form[models.CarroFormData])

I have this error bellow and can't find the solution or how to debug what's being passed to apply.
Can anyone help?
too many arguments for method apply: (car:
play.api.data.Form[models.CarroFormData])(implicit messages:
play.api.i18n.Messages)play.twirl.api.HtmlFormat.Appendable in class
index
Controller Form
def add = Action { implicit request =>
CarroForm.form.bindFromRequest.fold(
// if any error in submitted data
errorForm => Ok(views.html.admin.index(errorForm, Seq.empty[Carro])),
data => repo.create(carro.name, carro.description, carro.img, carro.keywords).map { _ =>
// If successful, we simply redirect to the index page.
Redirect(routes.application.index)
})
}
This is the Model
package dal
import javax.inject.{ Inject, Singleton }
import play.api.db.slick.DatabaseConfigProvider
import slick.driver.JdbcProfile
import models._
import scala.concurrent.{ Future, ExecutionContext }
/**
* A repository for people.
*
* #param dbConfigProvider The Play db config provider. Play will inject this for you.
*/
#Singleton
class CarroRepository #Inject() (dbConfigProvider: DatabaseConfigProvider)(implicit ec: ExecutionContext) {
// We want the JdbcProfile for this provider
private val dbConfig = dbConfigProvider.get[JdbcProfile]
// These imports are important, the first one brings db into scope, which will let you do the actual db operations.
// The second one brings the Slick DSL into scope, which lets you define the table and other queries.
import dbConfig._
import driver.api._
/**
* Here we define the table. It will have a name of people
*/
private class CarroTable(tag: Tag) extends Table[Carro](tag, "carro") {
/** The ID column, which is the primary key, and auto incremented */
def id = column[Long]("id", O.PrimaryKey, O.AutoInc)
/** The name column */
def name = column[String]("name")
/** The description column */
def description = column[String]("description")
/** The img column */
def img = column[String]("img")
/** The keywords column */
def keywords = column[String]("keywords")
/**
* This is the tables default "projection".
*
* It defines how the columns are converted to and from the Person object.
*
* In this case, we are simply passing the id, name and page parameters to the Person case classes
* apply and unapply methods.
*/
def * = (id, name, description, img, keywords) <> ((Carro.apply _).tupled, Carro.unapply)
}
/**
* The starting point for all queries on the people table.
*/
private val carro = TableQuery[CarroTable]
/**
* Create a person with the given name and age.
*
* This is an asynchronous operation, it will return a future of the created person, which can be used to obtain the
* id for that person.
*/
def create(name: String, description: String, img:String, keywords: String): Future[Carro] = db.run {
// We create a projection of just the name and age columns, since we're not inserting a value for the id column
(carro.map(p => (p.name, p.description, p.img, p.keywords))
// Now define it to return the id, because we want to know what id was generated for the person
returning carro.map(_.id)
// And we define a transformation for the returned value, which combines our original parameters with the
// returned id
into ((nameAge, id) => Carro(id, nameAge._1, nameAge._2, nameAge._3, nameAge._4))
// And finally, insert the person into the database
) += (name, description, img, keywords)
}
/**
* List all the people in the database.
*/
def list(): Future[Seq[Carro]] = db.run {
carro.result
}
}
package models
import play.api.data.Form
import play.api.data.Forms._
import play.api.libs.json._
case class Carro(id: Long, name:String, description:String, img:String, keywords:String)
case class CarroFormData(name: String, description: String, img: String, keywords: String)
object CarroForm {
val form = Form(
mapping(
"name" -> nonEmptyText,
"description" -> nonEmptyText,
"img" -> nonEmptyText,
"keywords" -> nonEmptyText
)(CarroFormData.apply)(CarroFormData.unapply)
)
}
object Carros {
var carros: Seq[Carro] = Seq()
def add(carros: Carro): String = {
carros = carros :+ carro.copy(id = carro.length) // manual id increment
"User successfully added"
}
def delete(id: Long): Option[Int] = {
val originalSize = carro.length
carro = carro.filterNot(_.id == id)
Some(originalSize - carro.length) // returning the number of deleted users
}
def get(id: Long): Option[Carro] = carro.find(_.id == id)
def listAll: Seq[Carro] = carro
implicit val carroFormat = Json.format[Carro]
}
View Code
#(car: Form[CarroFormData])(implicit messages: Messages)
#import helper._
#main(new Main("Car Dealers", "Compra e venda de carros", "logo.png", "carro, compra, venda")) {
<div class="container">
<h1>Hello</h1>
#form(routes.AdminCarro.add()) {
#inputText(person("name"))
#inputText(person("description"))
#inputText(person("img"))
#inputText(person("keywords"))
)
<div class="buttons">
<input type="submit" value="Add Car"/>
</div>
}
</div>
}
At your controller code:
def add = Action { implicit request =>
CarroForm.form.bindFromRequest.fold(
// if any error in submitted data
errorForm => Ok(views.html.admin.index(errorForm, Seq.empty[Carro])),
data => repo.create(carro.name, carro.description, carro.img, carro.keywords).map { _ =>
// If successful, we simply redirect to the index page.
Redirect(routes.application.index)
}
)
}
At the errorForm, you are calling the index view with two arguments:
Ok(views.html.admin.index(errorForm, Seq.empty[Carro]))
But your view declares just one argument:
#(car: Form[CarroFormData])(implicit messages: Messages)
Just remove the Seq.empty[Carro] from the call in your controller and everything should work as expected. If you still getting the same error, see if there are other places where this view is called the same wrong way (with two arguments) or try to sbt clean your project before sbt run.

Scala: class type required but T found

I've found similar issues of this particular problem, however the problem was due to someone trying to instantiate T directly. Here I'm trying to create a trait that is a general interface to extend classes and store them automatically in a database such as Riak using classOf[T]. Using Scala 2.10.
Here's my code:
trait RiakWriteable[T] {
/**
* bucket name of data in Riak holding class data
*/
def bucketName: String
/**
* determine whether secondary indices will be added
*/
def enable2i: Boolean
/**
* the actual bucket
*/
val bucket: Bucket = enable2i match {
case true => DB.client.createBucket(bucketName).enableForSearch().execute()
case false => DB.client.createBucket(bucketName).disableSearch().execute()
}
/**
* register the scala module for Jackson
*/
val converter = {
val c = new JSONConverter[T](classOf[T], bucketName)
JSONConverter.registerJacksonModule(DefaultScalaModule)
c
}
/**
* store operation
*/
def store = bucket.store(this).withConverter(converter).withRetrier(DB.retrier).execute()
/**
* fetch operation
*/
def fetch(id: String): Option[T] = {
val u = bucket.fetch(id, classOf[T]).withConverter(converter).withRetrier(DB.retrier).r(DB.N_READ).execute()
u match {
case null => None
case _ => Some(u)
}
}
}
Compiler error is class type required but T found.
Example usage (pseudo-code):
class Foo
object Foo extends RiakWriteable[Foo]
Foo.store(object)
So I'm guessing that a manifest of T is not being properly defined. Do I need to implicitly define this somewhere?
Thanks!
Here's an intermediary solution, though it leaves out the converter registration (which I may leave permanently for this use case, not sure yet).
/**
* trait for adding write methods to classes
*/
trait RiakWriteable[T] {
/**
* bucket name of data in Riak holding class data
*/
def bucketName: String
/**
* determine whether secondary indices will be added
*/
def enable2i: Boolean
/**
* the actual bucket
*/
val bucket: Bucket = enable2i match {
case true => DB.client.createBucket(bucketName).enableForSearch().execute()
case false => DB.client.createBucket(bucketName).disableSearch().execute()
}
/**
* store operation
*/
def store(o: T) = bucket.store(o).withRetrier(DB.retrier).execute()
/**
* fetch operation
*/
def fetch(id: String)(implicit m: ClassTag[T]) = {
val u = bucket.fetch(id, classTag[T].runtimeClass).withRetrier(DB.retrier).r(DB.N_READ).execute()
u match {
case null => None
case _ => Some(u)
}
}
}