my json looks like this.
[
{"name": "AABC Register Ltd (Architects accredited in building conservation), with effect from the 2016 to 2017 tax year"},
{"name": "Academic and Research Surgery Society of"},
{"name": "Academic Gaming and Simulation in Education and Training Society for"},
{"name": "Academic Primary Care Society for"}
]
and the code I use to parse this and add it to mongoDB is this
val sourceOrganisations: JsValue = Json.parse(getClass.getResourceAsStream("/json/ApprovedOrganisations.json"))
val organisations: Seq[String] = (sourceOrganisations.as[JsArray] \\ "name").map(jsval => jsval.toString())
println(organisations)
organisations.foreach(organisation => this.insert(Organisation(organisation)))
when querying mongo i get
{ "_id" : ObjectId("5c3dcf309770222486f50b4a"), "name" : "\"Accounting Association American\"" }
but i want this
{ "_id" : ObjectId("5c3dcf309770222486f50b4a"), "name" : "Accounting Association American" }
I have tried looking at why there might be extra quotes but can't find the reason
The problem is because you are doing a toString on JsValue.
When you do a toString on a JsValue which is a JsString you get a String which has the value "value_inside_JsString"
val jsValue: JsValue = JsString("a")
val extraQuotedString = "[[" + jsValue.toString() + "]]"
// extraQuotedString: String = [["a"]]
The correct way is to first cast it to JsString and the getting its value by .value.
val correctString = "[[" + jsValue.as[JsString].value + "]]"
// correctString: String = [[a]]
So,
val json = Json.parse(jsonString)
val organizationsAsJsValues = json match {
case jsArray: JsArray => jsArray \\ "name"
case _ => Seq.empty[JsValue]
}
val organizationAsNames = organizationsAsJsValues.flatMap({
case jsString: JsString => Some(jsString.value)
case _ => None
})
val organizations = organizationAsNames.map(name => Organization(name))
val organisations: Seq[String] = (sourceOrganisations.as[JsArray] \\ "name").map(jsval => jsval.toString().replaceAll("\"",""))
added
replaceAll("\"","") to line to fix it. have no clue why it added the qoutes though
jsval => jsval.toString()
This will give you the JSON representation of jsval. If that was a JsString, it will be the double quoted (and potentially escaped) text content.
You probably want to parse the JSON into a case class (sourceOrganisation = Json.parse(something).as[TheCaseClass]). That way, you can also avoid doing the manual parsing with jsArray \\ "name"
Or at least the final extract the String from the final jsval (if you know that it's a JsString, you could do jsval.as[String]).
Related
I have this following json input where I am trying to parse the name field in-order
scala> result
res6: play.api.libs.json.JsValue = {"L0":
{"name":"FASHION","id":"50000"},"L1":{"name":"ACCESSORIES AND TRAVEL","id":"51000"},"L2":{"name":"FASHION ACCESSORIES","id":"51001"},"L3":{"name":"MENS FASHION ACCESSORIES","id":"51100"},"L4":{"name":"MENS HATS","id":"51204"}}
scala> result \\ "name"
res5: Seq[play.api.libs.json.JsValue] = List("ACCESSORIES AND TRAVEL", "MENS HATS", "MENS FASHION ACCESSORIES", "FASHION ACCESSORIES", "FASHION")
What I am trying is to get those names in-order like
List("FASHION", "ACCESSORIES AND TRAVEL", "FASHION ACCESSORIES", "MENS FASHION ACCESSORIES", "MENS HATS")
Is there a way to achieve that with play Json library?
With Play JSON I always use case classes. So your example would look like:
import play.api.libs.json._
val json = """{"L0":
{"name":"FASHION","id":"50000"},"L1":{"name":"ACCESSORIES AND TRAVEL","id":"51000"},"L2":{"name":"FASHION ACCESSORIES","id":"51001"},"L3":{"name":"MENS FASHION ACCESSORIES","id":"51100"},"L4":{"name":"MENS HATS","id":"51204"}}
"""
case class Element(id: String, name: String)
object Element {
implicit val jsonFormat: Format[Element] = Json.format[Element]
}
Json.parse(json).validate[Map[String, Element]] match {
case JsSuccess(elems, _) => println(elems.toList.sortBy(_._1).map(e => e._2.name))
case other => println(s"Handle exception $other")
}
What this gives you, is that you can sort the result by the key - the info that is lost in your solution.
Compilation error:
No Json serializer found for type Seq[(models.Account, models.Company)]. Try to implement an implicit Writes or Format for this type.
How can I define an implicit writes for the result of a join query?
Controller:
def someEndpoint = Action.async { implicit request =>
val query = for {
a <- accounts if a.id === 10
c <- companies if a.companyID === c.id
} yield (a, c)
db.run(query.result).map(rows => Ok(Json.toJson(rows))) // Causes compilation error
}
Each of my models (account and company) have their own implicit writes (here's the company one):
case class Company(id: Int, name: String)
object Company {
implicit val writes = new Writes[Company] {
def writes(company: Company): JsValue = {
Json.obj(
"id" -> company.id,
"name" -> company.name
)
}
}
}
Is it possible to dynamically handle serializations for joins? I have a lot of things I will be joining together... Do I need to explicitly define a writes for each combination?
Writes.seq will help you
small writer
val w = (
(__ \ "account").write[Account] and
(__ \ "company").write[Company]
).tupled
helps you can transform Seq[(models.Account, models.Company)] to JsValue with
Writes.seq(w).writes(rows)
and last command will be
db.run(query.result).map(rows => Ok(Writes.seq(w).writes(rows))
or a more clear variant
db.run(query.result)
.map(
_.map{
case (a,c) => Json.obj("account" -> a, "company" -> c)
}
)
.map(rows =>
Ok(JsArray(rows))
)
it's the same thing, but you create object for every row yourself.
I think you expect the response of your query in JSON to be something like
[
{
"account" : { "number": "123", "companyID" : 1 },
"company" : { "id" : 1, "name" : "My company"}
} , ...
]
The problem is the response of the query is just a tuple, so "account" and "company" are not easy to calculate.
Instead of a tuple you could create a new case class with the joined data, but I understand you want to avoid that. In that case you can instead of a tuple use a Map that is something that will convert automatically to JSON.
Extra: Creating writers for case classes is very simple
import play.api.libs.json._
implicit val personWrites = Json.writes[Person]
Reference: https://www.playframework.com/documentation/2.4.x/ScalaJsonInception
Using Casbah, I query Mongo.
val mongoClient = MongoClient("localhost", 27017)
val db = mongoClient("test")
val coll = db("test")
val results: MongoCursor = coll.find(builder)
var matchedDocuments = List[DBObject]()
for(result <- results) {
matchedDocuments = matchedDocuments :+ result
}
Then, I convert the List[DBObject] into JSON via:
val jsonString: String = buildJsonString(matchedDocuments)
Is there a better way to convert from "results" (MongoCursor) to JSON (JsValue)?
private def buildJsonString(list: List[DBObject]): Option[String] = {
def go(list: List[DBObject], json: String): Option[String] = list match {
case Nil => Some(json)
case x :: xs if(json == "") => go(xs, x.toString)
case x :: xs => go(xs, json + "," + x.toString)
case _ => None
}
go(list, "")
}
Assuming you want implicit conversion (like in flavian's answer), the easiest way to join the elements of your list with commas is:
private implicit def buildJsonString(list: List[DBObject]): String =
list.mkString(",")
Which is basically the answer given in Scala: join an iterable of strings
If you want to include the square brackets to properly construct a JSON array you'd just change it to:
list.mkString("[", ",", "]") // punctuation madness
However if you'd actually like to get to Play JsValue elements as you seem to indicate in the original question, then you could do:
list.map { x => Json.parse(x.toString) }
Which should produce a List[JsValue] instead of a String. However, if you're just going to convert it back to a string again when sending a response, then it's an unneeded step.
I'm trying to write a function that returns a list (for querying purposes) that has some wildcard elements:
def createPattern(query: List[(String,String)]) = {
val l = List[(_,_,_,_,_,_,_)]
var iter = query
while(iter != null) {
val x = iter.head._1 match {
case "userId" => 0
case "userName" => 1
case "email" => 2
case "userPassword" => 3
case "creationDate" => 4
case "lastLoginDate" => 5
case "removed" => 6
}
l(x) = iter.head._2
iter = iter.tail
}
l
}
So, the user enters some query terms as a list. The function parses through these terms and inserts them into val l. The fields that the user doesn't specify are entered as wildcards.
Val l is causing me troubles. Am I going the right route or are there better ways to do this?
Thanks!
Gosh, where to start. I'd begin by getting an IDE (IntelliJ / Eclipse) which will tell you when you're writing nonsense and why.
Read up on how List works. It's an immutable linked list so your attempts to update by index are very misguided.
Don't use tuples - use case classes.
You shouldn't ever need to use null and I guess here you mean Nil.
Don't use var and while - use for-expression, or the relevant higher-order functions foreach, map etc.
Your code doesn't make much sense as it is, but it seems you're trying to return a 7-element list with the second element of each tuple in the input list mapped via a lookup to position in the output list.
To improve it... don't do that. What you're doing (as programmers have done since arrays were invented) is to use the index as a crude proxy for a Map from Int to whatever. What you want is an actual Map. I don't know what you want to do with it, but wouldn't it be nicer if it were from these key strings themselves, rather than by a number? If so, you can simplify your whole method to
def createPattern(query: List[(String,String)]) = query.toMap
at which point you should realise you probably don't need the method at all, since you can just use toMap at the call site.
If you insist on using an Int index, you could write
def createPattern(query: List[(String,String)]) = {
def intVal(x: String) = x match {
case "userId" => 0
case "userName" => 1
case "email" => 2
case "userPassword" => 3
case "creationDate" => 4
case "lastLoginDate" => 5
case "removed" => 6
}
val tuples = for ((key, value) <- query) yield (intVal(key), value)
tuples.toMap
}
Not sure what you want to do with the resulting list, but you can't create a List of wildcards like that.
What do you want to do with the resulting list, and what type should it be?
Here's how you might build something if you wanted the result to be a List[String], and if you wanted wildcards to be "*":
def createPattern(query:List[(String,String)]) = {
val wildcard = "*"
def orElseWildcard(key:String) = query.find(_._1 == key).getOrElse("",wildcard)._2
orElseWildcard("userID") ::
orElseWildcard("userName") ::
orElseWildcard("email") ::
orElseWildcard("userPassword") ::
orElseWildcard("creationDate") ::
orElseWildcard("lastLoginDate") ::
orElseWildcard("removed") ::
Nil
}
You're not using List, Tuple, iterator, or wild-cards correctly.
I'd take a different approach - maybe something like this:
case class Pattern ( valueMap:Map[String,String] ) {
def this( valueList:List[(String,String)] ) = this( valueList.toMap )
val Seq(
userId,userName,email,userPassword,creationDate,
lastLoginDate,removed
):Seq[Option[String]] = Seq( "userId", "userName",
"email", "userPassword", "creationDate", "lastLoginDate",
"removed" ).map( valueMap.get(_) )
}
Then you can do something like this:
scala> val pattern = new Pattern( List( "userId" -> "Fred" ) )
pattern: Pattern = Pattern(Map(userId -> Fred))
scala> pattern.email
res2: Option[String] = None
scala> pattern.userId
res3: Option[String] = Some(Fred)
, or just use the map directly.
It seems no easy way to use "in" clause in anorm:
val ids = List("111", "222", "333")
val users = SQL("select * from users where id in ({ids})").on('ids-> ???).as(parser *)
How to replace the ??? part?
I tried:
on('ids -> ids)
on('ids -> ids.mkString("'","','","'"))
on('ids -> ids.mkString("','")
But none works.
I see in the discussion the exactly same problem: https://groups.google.com/d/topic/play-framework/qls6dhhdayc/discussion, the author has a complex solution:
val params = List(1, 2, 3)
val paramsList = for ( i <- 0 until params.size ) yield ("userId" + i)
// ---> results in List("userId0", "userId1", "userId2")
User.find("id in ({%s})"
// produces "id in ({userId0},{userId1},{userId2})"
.format(paramsList.mkString("},{"))
// produces Map("userId0" -> 1, "userId1" -> 2, ...)
.on(paramsList.zip(params))
.list()
This is too much complicated.
Is there any easier way? Or should play provide something to make it easier?
Anorm now supports such a case (and more) since 2.3: "Using multi-value parameter"
Back to initial example it gives:
val ids = Seq("111", "222", "333")
val users = SQL("select * from users where id in ({ids})").on('ids-> ids).as(parser *)
Nailed it! There haven't really been any more updates on this thread, but it seems to still be relevant. Because of that, and because there isn't an answer, I thought I'd throw mine in for consideration.
Anorm doesn't support 'IN' clauses. I doubt they ever will. There's nothing you can do to make them work, I even read a post where anorm specifically took out those clauses because they made Anorm feel 'like an ORM'.
It's fairly easy, however, to wrap the SqlQuery in a short class that supports the IN clause, and then convert that class into a SqlQuery when needed.
Instead of pasting the code in here, because it gets a little long, here is the link to my blog, where I've posted the code and how to use it.
In clause with Anorm
Basically, when you have the code from my blog, your statements look like this:
RichSQL(""" SELECT * FROM users WHERE id IN ({userIds}) """).onList("userIds" -> userIds).toSQL.as(userParser *)(connection)
Maybe it's too late but here is a tip for using custom string interpolation that also works for solve the problem of IN clause.
I have implemented a helper class to define a string interpolation. You can see it below, and you can simply copy and paste, but first let's see how you can use it.
Instead of write something
SQL("select * from car where brand = {brand} and color = {color} and year = {year} order by name").on("brand" -> brand, "color" -> color, "year" -> year).as(Car.simple *)
You can simply write:
SQL"select * from car where brand = $brand and color = $color and year = $year order by name".as(Car.simple *)
So using string interpolation it's more concise and easier to read.
And for the case of using the IN clause, you can write:
val carIds = List(1, 3, 5)
SQLin"select * from car where id in ($carIds)".as(Car.simple *)
Or for your example:
val ids = List("111", "222", "333")
val users = SQLin"select * from users where id in ($ids)".as(parser *)
For more information about string interpolation, check this link
The code for this implicit class is the following:
package utils
object AnormHelpers {
def wild (str: String) = "%" + str + "%"
implicit class AnormHelper (val sc: StringContext) extends AnyVal {
// SQL raw -> it simply create an anorm.Sql using string interpolation
def SQLr (args: Any*) = {
// Matches every argument to an arbitrary name -> ("p0", value0), ("p1", value1), ...
val params = args.zipWithIndex.map(p => ("p"+p._2, p._1))
// Regenerates the original query substituting each argument by its name with the brackets -> "select * from user where id = {p0}"
val query = (sc.parts zip params).map{ case (s, p) => s + "{"+p._1+"}" }.mkString("") + sc.parts.last
// Creates the anorm.Sql
anorm.SQL(query).on( params.map(p => (p._1, anorm.toParameterValue(p._2))) :_*)
}
// SQL -> similar to SQLr but trimming any string value
def SQL (args: Any*) = {
val params = args.zipWithIndex.map {
case (arg: String, index) => ("p"+index, arg.trim.replaceAll("\\s{2,}", " "))
case (arg, index) => ("p"+index, arg)
}
val query = (sc.parts zip params).map { case (s, p) => s + "{"+ p._1 + "}" }.mkString("") + sc.parts.last
anorm.SQL(query).on( params.map(p => (p._1, anorm.toParameterValue(p._2))) :_*)
}
// SQL in clause -> similar to SQL but expanding Seq[Any] values separated by commas
def SQLin (args: Any*) = {
// Matches every argument to an arbitrary name -> ("p0", value0), ("p1", value1), ...
val params = args.zipWithIndex.map {
case (arg: String, index) => ("p"+index, arg.trim.replaceAll("\\s{2,}", " "))
case (arg, index) => ("p"+index, arg)
}
// Expands the Seq[Any] values with their names -> ("p0", v0), ("p1_0", v1_item0), ("p1_1", v1_item1), ...
val onParams = params.flatMap {
case (name, values: Seq[Any]) => values.zipWithIndex.map(v => (name+"_"+v._2, anorm.toParameterValue(v._1)))
case (name, value) => List((name, anorm.toParameterValue(value)))
}
// Regenerates the original query substituting each argument by its name expanding Seq[Any] values separated by commas
val query = (sc.parts zip params).map {
case (s, (name, values: Seq[Any])) => s + values.indices.map(name+"_"+_).mkString("{", "},{", "}")
case (s, (name, value)) => s + "{"+name+"}"
}.mkString("") + sc.parts.last
// Creates the anorm.Sql
anorm.SQL(query).on(onParams:_*)
}
}
}
It's probably late, but I add this for others looking for the same.
You could use some built-in database features to overcome this. This is one of the advantages Anorm has over ORMs. For example, if you are using PostgreSQL you could pass your list as an array and unnest the array in your query:
I assume ids are integer.
val ids = List(1, 2, 3)
val idsPgArray = "{%s}".format(ids.mkString(",")) //Outputs {1, 2, 3}
val users = SQL(
"""select * from users where id in (select unnest({idsPgArray}::integer[]))"""
).on('ids-> ???).as(parser *)
Executed query will be
select * from users where id in (select unnest('{1, 2, 3}'::integer[]))
which is equal to
select * from users where id in (1, 2, 3)
I had the same problem recently. Unfortunately there doesn't seem to be a way without using string interpolation and thus vulnerable to SQL injection.
What I ended up doing was kinda sanitizing it by transforming it to a list of ints and back:
val input = "1,2,3,4,5"
// here there will be an exception if someone is trying to sql-inject you
val list = (_ids.split(",") map Integer.parseInt).toList
// re-create the "in" string
SQL("select * from foo where foo.id in (%s)" format list.mkString(","))
User.find("id in (%s)"
.format(params.map("'%s'".format(_)).mkString(",") )
.list()
val ids = List("111", "222", "333")
val users = SQL("select * from users
where id in
(" + ids.reduceLeft((acc, s) => acc + "," + s) + ")").as(parser *)