How to transform submitted json in Play 2.0? - scala

I'm trying to build a REST API using play 2.0. I have a User case class that contains some fields (like username & password) that shouldn't be updatable by the updateMember method.
Is there a good, functional way, of dealing with multiple Options somehow, because request.body.asJson returns an Option[JsValue], and my user lookup also returns an Option:
package controllers.api
import org.joda.time.LocalDate
import play.api.Play.current
import play.api.db.slick.{DB, Session}
import play.api.mvc._
import play.api.libs.json._
import play.api.libs.functional.syntax._
import models.{Gender, User, UserId}
import repositories.UserRepository
object Member extends Controller {
def updateMember(id: Long) = Action {
DB.withSession {
implicit session: Session =>
val json: JsValue = request.body.asJson // how to deal with this?
val repository = new UserRepository
repository.findById(new UserId(id)).map {
user =>
def usernameAppender = __.json.update(
__.read[JsObject].map { o => o ++ Json.obj("username" -> user.username) }
)
json.transform(usernameAppender) // transform not found
Ok("updated")
}.getOrElse(NotFound)
}
}
}
I could move the map call to where I try to parse the request, but then inside there I guess I'd need another map over the user Option like I already have. So in that style, I'd need a map per Option.
Is there a better way of dealing with multiple Options like this in FP?

You're basically dealing with a nested monad, and the main tool for working with such is flatMap, particularly if both options being None has the same semantic meaning to your program:
request.body.asJson.flatMap { requestJson =>
val repository = new UserRepository
repository.findById(new UserId(id)).map { user =>
def usernameAppender = __.json.update(
__.read[JsObject].map { o => o ++ Json.obj("username" -> user.username) }
)
requestJson.transform(usernameAppender)
Ok("updated") // EDIT: Do you not want to return the JSON?
}
}.getOrElse(NotFound)
But it might also be the case that your Nones have different meanings, in which case, you probably just want to pattern match, and handle the error cases separately:
request.body.asJson match {
case Some(requestJson) =>
val repository = new UserRepository
repository.findById(new UserId(id)).map { user =>
def usernameAppender = __.json.update(
__.read[JsObject].map { o => o ++ Json.obj("username" -> user.username) }
)
requestJson.transform(usernameAppender)
Ok("updated")
}.getOrElse(NotFound)
case None => BadRequest // Or whatever you response makes sense for this case
}

Related

Recursive HTTP-requests in Scala

I need to do recursive requests and then collect all models into one List, but not understand how to do it. Please tell me am I thinking right way?
package kindSir.main
import dispatch.Defaults._
import dispatch._
import kindSir.models._
import org.json4s._
import org.json4s.jackson.JsonMethods._
object ApplicationMain extends App {
def fetchMergeRequests(startPage: Int = 1): Future[List[MergeRequest]] = {
val requestsUrl = url(s"https://gitlab.com/api/v4/projects/gitlab-org%2Fgitlab-ce/merge_requests?state=opened&per_page=3&page=${startPage}")
Http(requestsUrl).map { res =>
(parse(res.getResponseBody), res.getHeader("X-Next-Page").toInt) match {
case (list#JArray(_), nextPage: Int) =>
val currentList: List[MergeRequest] = MergeRequest.parseList(list).get
val nextPageListFuture: Future[List[MergeRequest]] = fetchMergeRequests(nextPage)
// And how to merge these two lists?
case (list#JArray(_), _) => MergeRequest.parseList(list).get
case _ => throw new RuntimeException(s"No merge requests for project found")
}
}
}
}
The main problem you're dealing with here is that you're trying to combine data you already have (List[MergeRequest]) with the data you'll retrieve in future (Future[List[MergeRequest]]). There are a few things you need to do to handle this scenario:
Use flatMap instead of map on result of the HTTP request. This allows you to make further HTTP requests inside the recursion but map them back to a single Future.
Call map on the result of the recursion fetchMergeRequests(nextPage) to combine the data you already have with the future data from the recursion.
Wrap the other list in Future.successful() because flatMap requires all the pattern matches to return a Future — except for the exception.
I'm not familiar with the libraries you are using so I haven't tested it, but I think your code should work like this:
def fetchMergeRequests(startPage: Int = 1): Future[List[MergeRequest]] = {
val requestsUrl = url(s"https://gitlab.com/api/v4/projects/gitlab-org%2Fgitlab-ce/merge_requests?state=opened&per_page=3&page=${startPage}")
Http(requestsUrl).flatMap { res =>
(parse(res.getResponseBody), res.getHeader("X-Next-Page").toInt) match {
case (list#JArray(_), nextPage: Int) =>
val currentList: List[MergeRequest] = MergeRequest.parseList(list).get
val nextPageListFuture: Future[List[MergeRequest]] = fetchMergeRequests(nextPage)
nextPageListFuture.map(nextPageList => currentList ++ nextPageList)
case (list#JArray(_), _) =>
Future.successful(MergeRequest.parseList(list).get)
case _ => throw new RuntimeException(s"No merge requests for project found")
}
}
}

Could someone explain how is the this code executing?

How is the createUser or updateUser using the unmarshalValueJs function and passing arguments.
package controllers.user
import play.api._
import play.api.mvc._
import play.api.libs.json._
import play.api.libs.json.Reads._
import play.api.libs.functional.syntax._
import services.user.UserServiceComponent
import domain.user.User
trait UserController extends Controller {
self: UserServiceComponent =>0
def emailAlreadyExists(implicit reads: Reads[String]) =
Reads[String](js => reads.reads(js).flatMap { e =>
userService.tryFindByEmail(e).map(_ => JsError("error.custom.emailAlreadyExists")).getOrElse(JsSuccess(e))
})
implicit val userReads = (__ \ "email").read[String](email andKeep emailAlreadyExists)
.map(resource => UserResource(resource))
implicit val userWrites = new Writes[User] {
override def writes(user: User): JsValue = {
Json.obj(
"id" -> user.id,
"email" -> user.email
)
}
}
What create User passes to the unmarshalJsValue?
Where does resource come from?
def createUser = Action(parse.json) {request =>
unmarshalJsValue(request) { resource: UserResource =>
val user = User(Option.empty, resource.email)
userService.createUser(user)
Created
}
}
def updateUser(id: Long) = Action(parse.json) {request =>
unmarshalJsValue(request) { resource: UserResource =>
val user = User(Option(id), resource.email)
userService.updateUser(user)
NoContent
}
}
def findUserById(id: Long) = Action {
val user = userService.tryFindById(id)
if (user.isDefined) {
Ok(Json.toJson(user))
} else {
NotFound
}
}
def deleteUser(id: Long) = Action {
userService.delete(id)
NoContent
}
What is R over here?
What get passed back to the createUser?
def unmarshalJsValue[R](request: Request[JsValue])(block: R => Result)(implicit rds : Reads[R]): Result =
request.body.validate[R](rds).fold(
valid = block,
invalid = e => {
val error = e.mkString
Logger.error(error)
BadRequest(error)
}
)
}
case class UserResource(email: String)
R is a type variable. The definition def unmarshalJsValue[R](request: Request[JsValue])(block: R => Result)(implicit rds : Reads[R]) reads:
I'm a method called unmarshallJsValue.
I require a type parameter that could potentially be anything.
I require a value of Request[JsValue] (presumably a request contianin a JSON entity).
I require a function that, given some value of type R, produces a Result.
I require a value to be in implicit scope that is a Reads for type R. In other words, I need a way to convert a JsValue to an R, whatever R might happen to be. (If you look at the docs for Reads, you'll notice that its only method, reads, has this effect.)
To put it all together, entire function just says, give me some block of code that produces a Result from some piece of data I know how to convert to from JSON. The body of the function simply attempts to convert the JSON to the desired type (known as unmarshalling). If this succeeds, it runs the block. Otherwise, you get a BadRequest client error.
The compiler can't know whether the JSON that gets provided in real life by the client will be convertible to some type R, but it can require a JSON -> R converter, and some error handling code if it fails on real data.

How do you write a json4s CustomSerializer that handles collections

I have a class that I am trying to deserialize using the json4s CustomSerializer functionality. I need to do this due to the inability of json4s to deserialize mutable collections.
This is the basic structure of the class I want to deserialize (don't worry about why the class is structured like this):
case class FeatureValue(timestamp:Double)
object FeatureValue{
implicit def ordering[F <: FeatureValue] = new Ordering[F] {
override def compare(a: F, b: F): Int = {
a.timestamp.compareTo(b.timestamp)
}
}
}
class Point {
val features = new HashMap[String, SortedSet[FeatureValue]]
def add(name:String, value:FeatureValue):Unit = {
val oldValue:SortedSet[FeatureValue] = features.getOrElseUpdate(name, SortedSet[FeatureValue]())
oldValue += value
}
}
Json4s serializes this just fine. A serialized instance might look like the following:
{"features":
{
"CODE0":[{"timestamp":4.8828914447482E8}],
"CODE1":[{"timestamp":4.8828914541333E8}],
"CODE2":[{"timestamp":4.8828915127325E8},{"timestamp":4.8828910097466E8}]
}
}
I've tried writing a custom deserializer, but I don't know how to deal with the list tails. In a normal matcher you can just call your own function recursively, but in this case the function is anonymous and being called through the json4s API. I cannot find any examples that deal with this and I can't figure it out.
Currently I can match only a single hash key, and a single FeatureValue instance in its value. Here is the CustomSerializer as it stands:
import org.json4s.{FieldSerializer, DefaultFormats, Extraction, CustomSerializer}
import org.json4s.JsonAST._
class PointSerializer extends CustomSerializer[Point](format => (
{
case JObject(JField("features", JObject(Nil)) :: Nil) => new Point
case JObject(List(("features", JObject(List(
(feature:String, JArray(List(JObject(List(("timestamp",JDouble(ts)))))))))
))) => {
val point = new Point
point.add(feature, FeatureValue(ts))
point
}
},
{
// don't need to customize this, it works fine
case x: Point => Extraction.decompose(x)(DefaultFormats + FieldSerializer[Point]())
}
))
If I try to change to using the :: separated list format, so far I have gotten compiler errors. Even if I didn't get compiler errors, I am not sure what I would do with them.
You can get the list of json features in your pattern match and then map over this list to get the Features and their codes.
class PointSerializer extends CustomSerializer[Point](format => (
{
case JObject(List(("features", JObject(featuresJson)))) =>
val features = featuresJson.flatMap {
case (code:String, JArray(timestamps)) =>
timestamps.map { case JObject(List(("timestamp",JDouble(ts)))) =>
code -> FeatureValue(ts)
}
}
val point = new Point
features.foreach((point.add _).tupled)
point
}, {
case x: Point => Extraction.decompose(x)(DefaultFormats + FieldSerializer[Point]())
}
))
Which deserializes your json as follows :
import org.json4s.native.Serialization.{read, write}
implicit val formats = Serialization.formats(NoTypeHints) + new PointSerializer
val json = """
{"features":
{
"CODE0":[{"timestamp":4.8828914447482E8}],
"CODE1":[{"timestamp":4.8828914541333E8}],
"CODE2":[{"timestamp":4.8828915127325E8},{"timestamp":4.8828910097466E8}]
}
}
"""
val point0 = read[Point]("""{"features": {}}""")
val point1 = read[Point](json)
point0.features // Map()
point1.features
// Map(
// CODE0 -> TreeSet(FeatureValue(4.8828914447482E8)),
// CODE2 -> TreeSet(FeatureValue(4.8828910097466E8), FeatureValue(4.8828915127325E8)),
// CODE1 -> TreeSet(FeatureValue(4.8828914541333E8))
// )

How to query OrientDB asynchronously from Play controller?

I am writing a Play (2.2) controller in Scala, which should return the result of a query against OrientDB. Now, I have succeeded in writing a synchronous version of said controller, but I'd like to re-write it to work asynchronously.
My question is; given the below code (just put together for demonstration purposes), how do I re-write my controller to interact asynchronously with OrientDB (connecting and querying)?
import play.api.mvc.{Action, Controller}
import play.api.libs.json._
import com.orientechnologies.orient.`object`.db.OObjectDatabasePool
import java.util
import com.orientechnologies.orient.core.sql.query.OSQLSynchQuery
import scala.collection.JavaConverters._
object Packages extends Controller {
def packages() = Action { implicit request =>
val db = OObjectDatabasePool.global().acquire("http://localhost:2480", "reader", "reader")
try {
db.getEntityManager().registerEntityClass(classOf[models.Package])
val packages = db.query[util.List[models.Package]](new OSQLSynchQuery[models.Package]("select from Package")).asScala.toSeq
Ok(Json.obj(
"packages" -> Json.toJson(packages)
))
}
finally {
db.close()
}
}
}
EDIT:
Specifically, I wish to use OrientDB's asynchronous API. I know that asynchronous queries are supported by the API, though I'm not sure if you can connect asynchronously as well.
Attempted Solution
Based on Jean's answer, I've tried the following asynchronous implementation, but it fails due to a compilation error value execute is not a member of Nothing possible cause: maybe a semicolon is missing before 'value execute'?:
def getPackages(): Future[Seq[models.Package]] = {
val db = openDb
try {
val p = promise[Seq[models.Package]]
val f = p.future
db.command(
new OSQLAsynchQuery[ODocument]("select from Package",
new OCommandResultListener() {
var acc = List[ODocument]()
#Override
def result(iRecord: Any): Boolean = {
val doc = iRecord.asInstanceOf[ODocument]
acc = doc :: acc
true
}
#Override
def end() {
// This is just a dummy
p.success(Seq[models.Package]())
}
// Fails
})).execute()
f
}
finally {
db.close()
}
}
One way could be to start a promise, return the future representing the result of that promise, locally accumulate the results as they come and complete de promise ( thus resolving the future ) when orient db notifies you that the command has completed.
def executeAsync(osql: String, params: Map[String, String] = Map()): Future[List[ODocument]] = {
import scala.concurrent._
val p = promise[List[ODocument]]
val f =p.future
val req: OCommandRequest = database.command(
new OSQLAsynchQuery[ODocument]("select * from animal where name = 'Gipsy'",
new OCommandResultListener() {
var acc = List[ODocument]()
#Override
def result(iRecord:Any):Boolean= {
val doc = iRecord.asInstanceOf[ODocument]
acc=doc::acc
true
}
#Override
def end() {
p.success(acc)
}
}))
req.execute()
f
}
Be careful though, to enable graph navigation and field lazy loading, orientdb objects used to keep an internal reference to the database instance they were loaded from ( or to depend on a threadlocal database connected instance ) for lazily loading elements from the database. Manipulating these objects asynchronously may result in loading errors. I haven't checked changes from 1.6 but that seemed to be deeply embedded in the design.
It's as simple as wrapping the blocking call in a Future.
import play.api.libs.concurrent.Execution.Implicits.defaultContext
import scala.concurrent.Future
object Packages extends Controller {
def packages = Action.async { implicit request =>
val db = OObjectDatabasePool.global().acquire("http://localhost:2480", "reader", "reader")
db.getEntityManager().registerEntityClass(classOf[models.Package])
val futureResult: Future[Result] = Future(
db.query[util.List[models.Package]](new OSQLSynchQuery[models.Package]("select from Package")).asScala.toSeq
).map(
queryResult => Ok(Json.obj("packages" -> Json.toJson(packages)))
).recover {
// Handle each of the exception cases legitimately
case e: UnsupportedOperationException => UnsupportedMediaType(e.getMessage)
case e: MappingException => BadRequest(e.getMessage)
case e: MyServiceException => ServiceUnavailable(e.toString)
case e: Throwable => InternalServerError(e.toString + "\n" + e.getStackTraceString)
}
futureResult.onComplete { case _ =>
db.close()
}
futureResult
}
}
Note that I did not compile the code. There is a lot of room to improve the code.

Form binding with custom mapping to object - how?

I have these case class
case class Blog(id:Long, author:User, other stuff...)
case class Comment(id:Long, blog:Blog, comment:String)
and a form on the client side that submits the data
blog_id:"5"
comment:"wasssup"
I'm writing some simple code to let a user add a comment to a blog.
The user is logged in so the his user_id is not needed from the client side, we know who he is...
I would like to bind the blog_id to a Blog object loaded from db, and if it doesn't exist show an error.
The examples on play framework docs are not helpful.
They only show mappings for forms that represent a single Object and all of its fields.
Here I'm representing a tuple of a (b:Blog, comment:String) and for the Blog I'm only supplying it's id.
I'd like to have a mapping that would provide me with the conversion + validation + error messages, so i can write something like:
val form = Form(
tuple(
"blog_id" -> blogMapping,
"comment" -> nonEmptyText
)
)
form.bindFromRequest().fold(...
formWithErrors => {...
}, {
case (blog, comment) => {do some db stuff to create the comment}
...
The "blogMapping" wlil work like other mappings, it will bind the posted data to an object, in our case a blog loaded from db, and in case it's not successful it will provide an error that we can use on the formWithErrors => clause.
I'm not sure how to accomplish this, the docs are kinda lacking here...
any help is appreciated!
I ended up looking at how playframwork's current bindings look like and implementing something similar, but for Blog:
implicit def blogFromLongFormat: Formatter[Blog] = new Formatter[Blog] {
override val format = Some(("Blog does not exist", Nil))
def bind(key: String, data: Map[String, String]) = {
scala.util.control.Exception.allCatch[Long] either {
data.get(key).map(s => {
val blog_id = s.toLong
val blog = Daos.blogDao.retrieve(blog_id)
blog.map(Right(_)).getOrElse(Left(Seq(FormError(key, "Blog not found", Nil))))
}).get
} match {
case Right(e:Either[Seq[FormError],Blog]) => e
case Left(exception) => Left(Seq(FormError(key, "Invalid Blog Id", Nil)))
case _ => Left(Seq(FormError(key, "Error in form submission", Nil)))
}
}
def unbind(key: String, value: Blog) = Map(key -> value.id.toString)
}
val blogFromLongMapping: Mapping[Blog] = Forms.of[Blog]
To me, this doesn't really look like a binding problem.
The issue is around the Model-View-Controller split. Binding is a Controller activity, and it's about binding web data (from your View) to your data model (for use by the Model). Querying the data, on the other hand, would very much be handled by the Model.
So, the standard way to do this would be something like the following:
// Defined in the model somewhere
def lookupBlog(id: Long): Option[Blog] = ???
// Defined in your controllers
val boundForm = form.bindFromRequest()
val blogOption = boundForm.value.flatMap {
case (id, comment) => lookupBlog(id)
}
blogOption match {
case Some(blog) => ??? // If the blog is found
case None => ??? // If the blog is not found
}
However, if you are determined to handle database lookup in your binding (I'd strongly advise against this, as it will lead to spaghetti code in the long run), try something like the following:
class BlogMapping(val key: String = "") extends Mapping[Blog] {
val constraints = Nil
val mappings = Seq(this)
def bind(data: Map[String, String]) = {
val blogOpt = for {blog <- data.get(key)
blog_id = blog.toLong
blog <- lookupBlog(blog_id)} yield blog
blogOpt match {
case Some(blog) => Right(blog)
case None => Left(Seq(FormError(key, "Blog not found")))
}
}
def unbind(blog: Blog) = (Map(key -> blog.id.toString), Nil)
def withPrefix(prefix: String) = {
new BlogMapping(prefix + key)
}
def verifying(constraints: Constraint[Blog]*) = {
WrappedMapping[Blog, Blog](this, x => x, x => x, constraints)
}
}
val blogMapping = new BlogMapping()
val newform = Form(
tuple(
"blog_id" -> blogMapping,
"comment" -> nonEmptyText
)
)
// Example usage
val newBoundForm = newform.bindFromRequest()
val newBoundBlog = newBoundForm.get
The main thing we've done is to create a custom Mapping subclass. This can be a good idea under some circumstances, but I'd still recommend the first approach.
You can do it all in the form definition.
I have made some simple scala classes and objects from your example.
models/Blog.scala
package models
/**
* #author maba, 2013-04-10
*/
case class User(id:Long)
case class Blog(id:Long, author:User)
case class Comment(id:Long, blog:Blog, comment:String)
object Blog {
def findById(id: Long): Option[Blog] = {
Some(Blog(id, User(1L)))
}
}
object Comment {
def create(comment: Comment) {
// Save to DB
}
}
controllers/Comments.scala
package controllers
import play.api.mvc.{Action, Controller}
import play.api.data.Form
import play.api.data.Forms._
import models.{Comment, Blog}
/**
* #author maba, 2013-04-10
*/
object Comments extends Controller {
val form = Form(
mapping(
"comment" -> nonEmptyText,
"blog" -> mapping(
"id" -> longNumber
)(
(blogId) => {
Blog.findById(blogId)
}
)(
(blog: Option[Blog]) => Option(blog.get.id)
).verifying("The blog does not exist.", blog => blog.isDefined)
)(
(comment, blog) => {
// blog.get is always possible since it has already been validated
Comment(1L, blog.get, comment)
}
)(
(comment: Comment) => Option(comment.comment, Some(comment.blog))
)
)
def index = Action { implicit request =>
form.bindFromRequest.fold(
formWithErrors => BadRequest,
comment => {
Comment.create(comment)
Ok
}
)
}
}