Related
I want to change the keys and values for the keys key1 and key2 only when their values are val1 and val2 (both these mappings should be present for the transformation to take place). I am able to do it using the following code, but I do not think this is very elegant or efficient.
Is there a better way to do the same thing, perhaps using just one .map function applied over map?
Code:
val map = Map(
"key1" -> "val1",
"key2" -> "val2",
"otherkey1" -> "otherval1"
)
val requiredKeys = List("key1", "key2")
val interestingMap = map.filterKeys(requiredKeys.contains) // will give ("key1" -> "val1", "key2" -> "val2").
val changedIfMatched =
if (interestingMap.get("key1").get.equalsIgnoreCase("val1") && interestingMap.get("key2").get.equalsIgnoreCase("val2"))
Map("key1" -> "newval1", "key2" -> "newval2")
else
interestingMap
print(map ++ changedIfMatched) // to replace the old key->values with the new ones, if any.
Also can ++ operation to update the old key->value mappings be made more efficient?
Just do the check ahead of time:
map
.get("k1").filter(_.equalsIgnoreCase("v1"))
.zip(map.get("k2").filter(_.equalsIgnoreCase("v2")))
.headOption
.fold(map) { _ =>
map ++ Map("key1" -> "newVal1", "key2" -> "newVal2")
}
Here's an approach that checks that both key value pairs match.
EDIT: Added a mapValues method to the Map class. This technique can be used to do further checks on the values of the map.
val m = Map("key1" -> "val1", "key2" -> "VAL2", "otherkey1" -> "otherval1")
val oldKVs = Map("key1" -> "val1", "key2" -> "val2")
val newKVs = Map("newkey1" -> "newval1", "newkey2" -> "newval2")
implicit class MapImp[T,S](m: Map[T,S]) {
def mapValues[R](f: S => R) = m.map { case (k,v) => (k, f(v)) }
def subsetOf(m2: Map[T,S]) = m.toSet subsetOf m2.toSet
}
def containsKVs[T](m: Map[T,String], sub: Map[T,String]) =
sub.mapValues(_.toLowerCase) subsetOf m.mapValues(_.toLowerCase)
val m2 = if (containsKVs(m, oldKVs)) m -- oldKVs.keys ++ newKVs else m
println(m2)
// Map(otherkey1 -> otherval1, newkey1 -> newval1, newkey2 -> newval2)
It takes advantage of the fact that you can convert Maps into Sets of Tuple2.
I think this will be the most generic and resuable solution for the problem.
object Solution1 extends App {
val map = Map(
"key1" -> "val1",
"key2" -> "val2",
"otherkey1" -> "otherval1"
)
implicit class MapUpdate[T](map: Map[T, T]) {
def updateMapForGivenKeyValues: (Iterable[(T, T)], Iterable[(T, T)]) => Map[T, T] =
(fromKV: Iterable[(T, T)], toKV: Iterable[(T, T)]) => {
val isKeyValueExist: Boolean = fromKV.toIterator.forall {
(oldKV: (T, T)) =>
map.toIterator.contains(oldKV)
}
if (isKeyValueExist) map -- fromKV.map(_._1) ++ toKV else map
}
}
val updatedMap = map.updateMapForGivenKeyValues(List("key1" -> "val1", "key2" -> "val2"),
List("newKey1" -> "newVal1", "newVal2" -> "newKey2"))
println(updatedMap)
}
So the method updateMapForGivenKeyValues takes the List of old key value and new key value tuple. If all the key value pairs mentioned in the first parameter of the method exist in the map then only we will update the map with new key value pairs mentioned in the second parameter of the method. As the method is generic will can be used on any data type like String, Int, some case class etc.
we can easily re-use the method for different type of maps without even changing a single line of code.
Answer to modified question
val map = Map(
"key1" -> "val1",
"key2" -> "val2",
"otherkey1" -> "otherval1"
)
val requiredVals = List("key1"->"val1", "key2"->"val2")
val newVals = List("newval1", "newval2")
val result =
if (requiredVals.forall{ case (k, v) => map.get(k).exists(_.equalsIgnoreCase(v)) }) {
map ++ requiredVals.map(_._1).zip(newVals)
} else {
map
}
This solution use forall to check that all the key/value pairs in requiredKeys are found in the map by testing each pair in turn.
For each key/value pair (k, v) it does a get on the map using the key to retrieve the current value as Option[String]. This will be None if the key is not found or Some(s) if the key is found.
The code then calls exists on the Option[String]. This method will return false if value is None (the key is not found), otherwise it will return the result of the test that is passed to it. The test is _.equalsIgnoreCase(v) which does a case-insensitive comparison of the contents of the Option (_) and the value from the requireKeys list (v).
If this test fails then the original value of map is returned.
If this test succeeds then a modified version of the map is return. The expression requiredVals.map(_._1) returns the keys from the requireVals list, and the zip(newVals) associates the new values with the original keys. The resulting list of values is added to the map using ++ which will replace the existing values with the new ones.
Original answer
val map = Map(
"key1" -> "val1",
"key2" -> "val2",
"otherkey1" -> "otherval1"
)
val requiredVals = Map("key1"->"val1", "key2"->"val2")
val newVals = Map("newkey1" -> "newval1", "newkey2" -> "newval2")
val result =
if (requiredVals.forall{ case (k, v) => map.get(k).exists(_.equalsIgnoreCase(v)) }) {
map -- requiredVals.keys ++ newVals
} else {
map
}
Note that this replaces the old keys with the new keys, which appears to be what is described. If you want to keep the original keys and values, just delete "-- requiredVals.keys" and it will add the new keys without removing the old ones.
You can use the following code:
val interestingMap =
if(map.getOrElse("key1", "") == "val1" && map.getOrElse("key2", "") == "val2")
map - "key1" - "key2" + ("key1New" -> "val1New") + ("key2New" -> "val2New")
else map
The check part(if statement) can be tweaked to suit your specific need.
if any of these key-value pairs are not present in the map, the original map will be returned, otherwise, you will get a new map with two updates at the requested keys.
Regarding efficiency, as long as there are only two keys to be updated, I do not think there is a real performance difference between using + to add elements directly and using ++ operator to overwrite the keys wholesale. If your map is huge though, maybe using a mutable map proves to be a better option in the long run.
This is a scala question.
I currently have the following two collections objects:
val keywordLookup = Map("a" -> "1111",
"b" -> "2222",
"c" -> "3333",
"d" -> "4444",
"e" -> "5555")
val keywordList = Set("1111", "3333")
The keywordLookup is a lookup object. The keywordList contains a list of values that I need to find the Ids from the keywordLookup object.
I would like the get the following result:
Map("a" -> "1111", "c" -> "3333")
val filtered = keywordLookup.filter(kv => keywordList.contains(kv._2))
filtered is the Map you want as output
keywordLookup.filter(x => keywordList.contains(x._2))
Using flatMap on find,
keywordList.flatMap (k => keywordLookup.find( _._2 == k)).toMap
I'm learning Json4s library.
I have a json fragment like this:
{
"records":[
{
"name":"John Derp",
"address":"Jem Street 21"
},
{
"name":"Scala Jo",
"address":"in my sweet dream"
}
]
}
And, I have Scala code, which converts a json string into a List of Maps, like this:
import org.json4s._
import org.json4s.JsonAST._
import org.json4s.native.JsonParser
val json = JsonParser.parse( """{"records":[{"name":"John Derp","address":"Jem Street 21"},{"name":"Scala Jo","address":"in my sweet dream"}]}""")
val records: List[Map[String, Any]] = for {
JObject(rec) <- json \ "records"
JField("name", JString(name)) <- rec
JField("address", JString(address)) <- rec
} yield Map("name" -> name, "address" -> address)
println(records)
The output of records to screen gives this:
List(Map(name -> John Derp, address -> Jem Street 21), Map(name ->
Scala Jo, address -> in my sweet dream))
I want to understand what the lines inside the for loop mean. For example, what is the meaning of this line:
JObject(rec) <- json \ "records"
I understand that the json \ "records" produces a JArray object, but why is it fetched as JObject(rec) at left of <-? What is the meaning of the JObject(rec) syntax? Where does the rec variable come from? Does JObject(rec) mean instantiating a new JObject class from rec input?
BTW, I have a Java programming background, so it would also be helpful if you can show me the Java equivalent code for the loop above.
You have the following types hierarchy:
sealed abstract class JValue {
def \(nameToFind: String): JValue = ???
def filter(p: (JValue) => Boolean): List[JValue] = ???
}
case class JObject(val obj: List[JField]) extends JValue
case class JField(val name: String, val value: JValue) extends JValue
case class JString(val s: String) extends JValue
case class JArray(val arr: List[JValue]) extends JValue {
override def filter(p: (JValue) => Boolean): List[JValue] =
arr.filter(p)
}
Your JSON parser returns following object:
object JsonParser {
def parse(s: String): JValue = {
new JValue {
override def \(nameToFind: String): JValue =
JArray(List(
JObject(List(
JField("name", JString("John Derp")),
JField("address", JString("Jem Street 21")))),
JObject(List(
JField("name", JString("Scala Jo")),
JField("address", JString("in my sweet dream"))))))
}
}
}
val json = JsonParser.parse("Your JSON")
Under the hood Scala compiler generates the following:
val res = (json \ "records")
.filter(_.isInstanceOf[JObject])
.flatMap { x =>
x match {
case JObject(obj) => //
obj //
.withFilter(f => f match {
case JField("name", _) => true
case _ => false
}) //
.flatMap(n => obj.withFilter(f => f match {
case JField("address", _) => true
case _ => false
}).map(a => Map(
"name" -> (n.value match { case JString(name) => name }),
"address" -> (a.value match { case JString(address) => address }))))
}
}
First line JObject(rec) <- json \ "records" is possible because JArray.filter returns List[JValue] (i.e. List[JObject]). Here each value of List[JValue] maps to JObject(rec) with pattern matching.
Rest calls are series of flatMap and map (this is how Scala for comprehensions work) with pattern matching.
I used Scala 2.11.4.
Of course, match expressions above are implemented using series of type checks and casts.
UPDATE:
When you use Json4s library there is an implicit conversion from JValue to org.json4s.MonadicJValue. See package object json4s:
implicit def jvalue2monadic(jv: JValue) = new MonadicJValue(jv)
This conversion is used here: JObject(rec) <- json \ "records". First, json is converted to MonadicJValue, then def \("records") is applied, then def filter is used on the result of def \ which is JValue, then it is again implicitly converted to MonadicJValue, then def filter of MonadicJValue is used. The result of MonadicJValue.filter is List[JValue]. After that steps described above are performed.
You are using a Scala for comprehension and I believe much of the confusion is about how for comprehensions work. This is Scala syntax for accessing the map, flatMap and filter methods of a monad in a concise way for iterating over collections. You will need some understanding of monads and for comprehensions in order to fully comprehend this. The Scala documentation can help, and so will a search for "scala for comprehension". You will also need to understand about extractors in Scala.
You asked about the meaning of this line:
JObject(rec) <- json \ "records"
This is part of the for comprehension.
Your statement:
I understand that the json \ "records" produces a JArray object,
is slightly incorrect. The \ function extracts a List[JSObject] from the parser result, json
but why is it fetched as JObject(rec) at left of <-?
The json \ "records" uses the json4s extractor \ to select the "records" member of the Json data and yield a List[JObject]. The <- can be read as "is taken from" and implies that you are iterating over the list. The elements of the list have type JObject and the construct JObject(rec) applies an extractor to create a value, rec, that holds the content of the JObject (its fields).
how come it's fetched as JObject(rec) at left of <-?
That is the Scala syntax for iterating over a collection. For example, we could also write:
for (x <- 1 to 10)
which would simply give us the values of 1 through 10 in x. In your example, we're using a similar kind of iteration but over the content of a list of JObjects.
What is the meaning of the JObject(rec)?
This is a Scala extractor. If you look in the json4s code you will find that JObject is defined like this:
case class JObject(obj: List[JField]) extends JValue
When we have a case class in Scala there are two methods defined automatically: apply and unapply. The meaning of JObject(rec) then is to invoke the unapply method and produce a value, rec, that corresponds to the value obj in the JObject constructor (apply method). So, rec will have the type List[JField].
Where does the rec variable come from?
It comes from simply using it and is declared as a placeholder for the obj parameter to JObject's apply method.
Does JObject(rec) mean instantiating new JObject class from rec input?
No, it doesn't. It comes about because the JArray resulting from json \ "records" contains only JObject values.
So, to interpret this:
JObject(rec) <- json \ "records"
we could write the following pseudo-code in english:
Find the "records" in the parsed json as a JArray and iterate over them. The elements of the JArray should be of type JObject. Pull the "obj" field of each JObject as a list of JField and assign it to a value named "rec".
Hopefully that makes all this a bit clearer?
it's also helpful if you can show me the Java equivalent code for the loop above.
That could be done, of course, but it is far more work than I'm willing to contribute here. One thing you could do is compile the code with Scala, find the associated .class files, and decompile them as Java. That might be quite instructive for you to learn how much Scala simplifies programming over Java. :)
why I can't do this? for ( rec <- json \ "records", so rec become JObject. What is the reason of JObject(rec) at the left of <- ?
You could! However, you'd then need to get the contents of the JObject. You could write the for comprehension this way:
val records: List[Map[String, Any]] = for {
obj: JObject <- json \ "records"
rec = obj.obj
JField("name", JString(name)) <- rec
JField("address", JString(address)) <- rec
} yield Map("name" -> name, "address" -> address)
It would have the same meaning, but it is longer.
I just want to understand what does the N(x) pattern mean, because I only ever see for (x <- y pattern before.
As explained above, this is an extractor which is simply the use of the unapply method which is automatically created for case classes. A similar thing is done in a case statement in Scala.
UPDATE:
The code you provided does not compile for me against 3.2.11 version of json4s-native. This import:
import org.json4s.JsonAST._
is redundant with this import:
import org.json4s._
such that JObject is defined twice. If I remove the JsonAST import then it compiles just fine.
To test this out a little further, I put your code in a scala file like this:
package example
import org.json4s._
// import org.json4s.JsonAST._
import org.json4s.native.JsonParser
class ForComprehension {
val json = JsonParser.parse(
"""{
|"records":[
|{"name":"John Derp","address":"Jem Street 21"},
|{"name":"Scala Jo","address":"in my sweet dream"}
|]}""".stripMargin
)
val records: List[Map[String, Any]] = for {
JObject(rec) <- json \ "records"
JField("name", JString(name)) <- rec
JField("address", JString(address)) <- rec
} yield Map("name" -> name, "address" -> address)
println(records)
}
and then started a Scala REPL session to investigate:
scala> import example.ForComprehension
import example.ForComprehension
scala> val x = new ForComprehension
List(Map(name -> John Derp, address -> Jem Street 21), Map(name -> Scala Jo, address -> in my sweet dream))
x: example.ForComprehension = example.ForComprehension#5f9cbb71
scala> val obj = x.json \ "records"
obj: org.json4s.JValue = JArray(List(JObject(List((name,JString(John Derp)), (address,JString(Jem Street 21)))), JObject(List((name,JString(Scala Jo)), (address,JString(in my sweet dream))))))
scala> for (a <- obj) yield { a }
res1: org.json4s.JValue = JArray(List(JObject(List((name,JString(John Derp)), (address,JString(Jem Street 21)))), JObject(List((name,JString(Scala Jo)), (address,JString(in my sweet dream))))))
scala> import org.json4s.JsonAST.JObject
for ( JObject(rec) <- obj ) yield { rec }
import org.json4s.JsonAST.JObject
scala> res2: List[List[org.json4s.JsonAST.JField]] = List(List((name,JString(John Derp)), (address,JString(Jem Street 21))), List((name,JString(Scala Jo)), (address,JString(in my sweet dream))))
So:
You are correct, the result of the \ operator is a JArray
The "iteration" over the JArray just treats the entire array as the only value in the list
There must be an implicit conversion from JArray to JObject that permits the extractor to yield the contents of JArray as a List[JField].
Once everything is a List, the for comprehension proceeds as normal.
Hope that helps with your understanding of this.
For more on pattern matching within assignments, try this blog
UPDATE #2:
I dug around a little more to discover the implicit conversion at play here. The culprit is the \ operator. To understand how json \ "records" turns into a monadic iterable thing, you have to look at this code:
org.json4s package object: This line declares an implicit conversion from JValue to MonadicJValue. So what's a MonadicJValue?
org.json4s.MonadicJValue: This defines all the things that make JValues iterable in a for comprehension: filter, map, flatMap and also provides the \ and \\ XPath-like operators
So, essentially, the use of the \ operator results in the following sequence of actions:
- implicitly convert the json (JValue) into MonadicJValue
- Apply the \ operator in MonadicJValue to yield a JArray (the "records")
- implicitly convert the JArray into MonadicJValue
- Use the MonadicJValue.filter and MonadicJValue.map methods to implement the for comprehension
Just simplified example, how for-comprehesion works here:
scala> trait A
defined trait A
scala> case class A2(value: Int) extends A
defined class A2
scala> case class A3(value: Int) extends A
defined class A3
scala> val a = List(1,2,3)
a: List[Int] = List(1, 2, 3)
scala> val a: List[A] = List(A2(1),A3(2),A2(3))
a: List[A] = List(A2(1), A3(2), A2(3))
So here is just:
scala> for(A2(rec) <- a) yield rec //will return and unapply only A2 instances
res34: List[Int] = List(1, 3)
Which is equivalent to:
scala> a.collect{case A2(rec) => rec}
res35: List[Int] = List(1, 3)
Collect is based on filter - so it's enough to have filter method as JValue has.
P.S. There is no foreach in JValue - so this won't work for(rec <- json \ "records") rec. But there is map, so that will: for(rec <- json \ "records") yield rec
If you need your for without pattern matching:
for {
rec <- (json \ "records").filter(_.isInstanceOf[JObject]).map(_.asInstanceOf[JObject])
rcobj = rec.obj
name <- rcobj if name._1 == "name"
address <- rcobj if address._1 == "address"
nm = name._2.asInstanceOf[JString].s
vl = address._2.asInstanceOf[JString].s
} yield Map("name" -> nm, "address" -> vl)
res27: List[scala.collection.immutable.Map[String,String]] = List(Map(name -> John Derp, address -> Jem Street 21), Map(name -> Scala Jo, address -> in my sweet dream))
I have nested url parameters being passed to an endpoint, and I need these represented in as a JsValue. My initial assumption was that Play would parse them in a way similar to Rails, however parameters seem to only be split by & and =. Example:
Query params: ?test[testkey]=testvalue&test[newkey]=newvalue
Actual:
Map(
"test[testkey]" -> "testvalue" ,
"test[newkey]" -> "newvalue
)
Expected:
Map(
"test" -> Map(
"testkey" -> "testvalue" ,
"newkey" -> "newvalue"
)
)
Note that the end goal here is to be able to convert this into a JsObject.
I've started writing this myself, however simply porting the function from Rack is very un-scala-y and I feel like there has to be a quick way to get this that I am simply missing.
UPDATE
I am trying to find a generic solution which mimics the parsing that Rails uses (ie, with nested objects, lists, etc), and not just one level deep objects.
Just for fun, one option is to do something like:
import scala.util.matching.Regex
val pattern = new Regex("""(\w+)\[(\w+)\]""")
val qs : Map[String, Map[String, List[Seq[String]]]] = request.queryString.toList.map {
case (k, v) =>
pattern findFirstIn k match {
case Some(pattern(key, value)) => (key, value, v)
}
}.groupBy(_._1).mapValues(value => value.groupBy(_._2).mapValues {
value => value.map(x => x._3)
})
To convert this is to a JsValue, we can simply invoke:
import play.api.libs.json.Json
Json.toJson(qs)
This assumes that all your url params look like map[key]=value. You would have to modify the code a little to accommodate the standard key=value pattern.
I have a project set up with playframework 2.2.0 and play2-reactivemongo 0.10.0-SNAPSHOT. I'd like to query for few documents by their ids, in a fashion similar to this:
def usersCollection = db.collection[JSONCollection]("users")
val ids: List[String] = /* fetched from somewhere else */
val query = ??
val users = usersCollection.find(query).cursor[User].collect[List]()
As a query I tried:
Json.obj("_id" -> Json.obj("$in" -> ids)) // 1
Json.obj("_id.$oid" -> Json.obj("$in" -> ids)) // 2
Json.obj("_id" -> Json.obj("$oid" -> Json.obj("$in" -> ids))) // 3
for which first and second return empty lists and the third fails with error assertion 10068 invalid operator: $oid.
NOTE: copy of my response on the ReactiveMongo mailing list.
First, sorry for the delay of my answer, I may have missed your question.
Play-ReactiveMongo cannot guess on its own that the values of a Json array are ObjectIds. That's why you have to make a Json object for each id that looks like this: {"$oid": "526fda0f9205b10c00c82e34"}. When the ReactiveMongo Play plugin sees an object which first field is $oid, it treats it as an ObjectId so that the driver can send the right type for this value (BSONObjectID in this case.)
This is a more general problem actually: the JSON format does not match exactly the BSON one. That's the case for numeric types (BSONInteger, BSONLong, BSONDouble), BSONRegex, BSONDateTime, and BSONObjectID. You may find more detailed information in the MongoDB documentation: http://docs.mongodb.org/manual/reference/mongodb-extended-json/ .
I managed to solve it with:
val objectIds = ids.map(id => Json.obj("$oid" -> id))
val query = Json.obj("_id" -> Json.obj("$in" -> objectIds))
usersCollection.find(query).cursor[User].collect[List]()
since play-reactivemongo format considers BSONObjectID only when "$oid" is followed by string
implicit object BSONObjectIDFormat extends PartialFormat[BSONObjectID] {
def partialReads: PartialFunction[JsValue, JsResult[BSONObjectID]] = {
case JsObject(("$oid", JsString(v)) +: Nil) => JsSuccess(BSONObjectID(v))
}
val partialWrites: PartialFunction[BSONValue, JsValue] = {
case oid: BSONObjectID => Json.obj("$oid" -> oid.stringify)
}
}
Still, I hope there is a cleaner solution. If not, I guess it makes it a nice pull request.
I'm wondering if transforming id to BSONObjectID isn't more secure this way :
val ids: List[String] = ???
val bsonObjectIds = ids.map(BSONObjectID.parse(_)).collect{case Success(t) => t}
this will only generate valid BSONObjectIDs (and discard invalid ones)
If you do it this way :
val objectIds = ids.map(id => Json.obj("$oid" -> id))
your objectIds may not be valid ones depending on string id really being the stringify version of a BSONObjectID or not
If you import play.modules.reactivemongo.json._ it work without any $oid formatters.
import play.modules.reactivemongo.json._
...
val ids: Seq[BSONObjectID] = ???
val selector = Json.obj("_id" -> Json.obj("$in" -> ids))
usersCollection.find(selector).cursor[User].collect[Seq]()
I tried with the following and it worked for me:
val listOfItems = BSONArray(51, 61)
val query = BSONDocument("_id" -> BSONDocument("$in" -> listOfItems))
val ruleListFuture = bsonFutureColl.flatMap(_.find(query, Option.empty[BSONDocument]).cursor[ResponseAccDataBean]().
collect[List](-1, Cursor.FailOnError[List[ResponseAccDataBean]]()))