My program receives a scala map, the requirements is to validate this map (key-value pairs). Ex: validate a key value, change its value to an acceptable format etc. In a rare case, we update the key as well before passing the map to the down layer. Its not always required to update this map , but only when we detect that there are any unsupported keys or values. However, we have to check all key/value pairs. I'm doing some thing like this:
private def updateMap ( parameters: Map[String, String]): Map[String, String] = {
parameters.map{
case(k,v) => k match { case "checkPool" =>
(k, (if (k.contains("checkPool"))
v match {
case "1" => "true"
case _ => "false"
}
else v))
case "Newheader" => (k.replace("Newheader","header"),v)
case _ =>(k,v)
}
case _ => ("","")
}
}
Like this the code increases for doing the validation and converting the keys/values to supported ones. Is there a cleaner way of doing this validation in Scala for a map?
Thanks
It will be clearer if you put all your patterns above one another:
parameters.map{
case (k#"checkPool", "1") => k -> "true"
case (k#"checkPool", _") => k -> "false"
case ("Newheader", v) => "header" -> v
// put here all your other cases
case (k, v) => k -> v //last possible case, if nothing other matches
}
For clarity, you can also put different validators in partial functions:
type Validator = PartialFunction[(String, String), (String, String)
val checkPool: Validator = {
case (k#"checkPool", "1") => k -> "true"
case (k#"checkPool", _") => k -> "false"
}
val headers: Validator = {
case ("Newheader", v) => "header" -> v
}
And then put all your validators one after the other in your map:
parameters.map(
checkPool orElse
headers orElse
... orElse
PartialFunction(identity[(String, String)]) //this is the same as case (k, v) => k -> v
)
simple if else condition matching seems to be the best choice.
def updateMap(parameters: Map[String, String]): Map[String, String] = {
parameters.map(kv => {
var key = kv._1
var value = kv._2
if(key.contains("checkPool")){
value = if(value.equals("1")) "true" else "false"
}
else if(key.contains("Newheader")){
key = key.replace("Newheader", "header")
}
(key, value)
})
}
You can add more else if conditions
Related
I'm micro-optimising some code as a challenge.
I have a list of objects with a list of keys in each of them.
What's the most efficient way of grouping them by key, with each object being in every group of which it has a key.
This is what I have, but I have a feeling it can be improved.
I have many objects (100k+), each has ~2 keys, and there's less than 50 possible keys.
I've tried parallelising the list with listOfObjs.par, but there doesn't seem to be much of an improvement overall.
case class Obj(value: Option[Int], key: Option[List[String]])
listOfObjs
.filter(x => x.key.isDefined && x.value.isDefined)
.flatMap(x => x.key.get.map((_, x.value.get)))
.groupBy(_._1)
If you have that many object, the logical next step would be to distribute the work by using a MapReduce framework. At the end of the day you still need to go over every single object to determine the group it belongs in and your worst case will be bottlenecked by that.
The best you can do here is to replace these 3 operations by a fold so you only iterate through the collection once.
Edit: Updated the order based on Luis' recommendation in the comments
listOfObj.foldLeft(Map.empty[String, List[Int]]){ (acc, obj) =>
(obj.key, obj.value) match {
case (Some(k), Some(v)) =>
k.foldLeft(acc)((a, ky) => a + (ky -> {v +: a.getOrElse(ky, List.empty)}))))
case _ => acc
}
}
I got the impression you are looking for a fast alternative; thus a little bit of encapsulated mutability can help.
So, what about something like this:
def groupObjectsByKey(objects: List[Obj]): Map[String, List[Int]] = {
val iter =
objects.iterator.flatMap {
case Obj(Some(value), Some(keys)) =>
keys.iterator.map(key => key -> value)
case _ =>
Iterator.empty[(String, Int)]
}
val m =
mutable
.Map
.empty[String, mutable.Builder[Int, List[Int]]
iter.foreach {
case (k, v) =>
m.get(key = k) match {
case Some(builder) =>
builder.addOne(v)
case None =>
m.update(key = k, value = List.newBuilder[Int].addOne(v))
}
}
immutable
.Map
.mapFactory[String, List[Int]]
.fromSpecific(m.view.mapValues(_.result()))
}
Or if you don't care about the order of the elements of each group we can simplify and speed up the code a lot:
def groupObjectsByKey(objects: List[Obj]): Map[String, List[Int]] = {
val iter = objects.iterator.flatMap {
case Obj(Some(value), Some(keys)) =>
keys.iterator.map(key => key -> value)
case _ =>
Iterator.empty[(String, Int)]
}
val m = mutable.Map.empty[String, List[Int]]
iter.foreach {
case (k, v) =>
m.updateWith(key = k) match {
case Some(list) =>
Some(v :: list)
case None =>
Some(v :: Nil)
}
}
m.to(immutable.Map)
}
I have the following function definition:
private def extractUrl: String => (String, String)
= url =>
url
.split("/")
.toList
.filter(_.startsWith("localhost"))
.flatMap(e => e.split(":").toList)
.foldLeft[(String, String)](("", "")) { (acc, x) =>
acc match {
case ("", "") => (x, "")
case (a, "") => (a, x)
case z => z
}
}
the question is, is there another way to define an empty Tuple instead of ("", "")?
Empty tuple
("", "") is a tuple of empty strings with type (String, String).
Empty is unclear in that context, could be (None, None) or even (null, null) (bad)
You seem to use "" to represents a value that is not present. Try using None and Some[String], both sub types of Option[String], to indicate that a value is not present.
Analysis and comments
Potentially, your method seems not to do what is intended. (execute code below)
Think about using already present functions / methods / libraries for manipulating URLs (also see example below)
Think about using Option
object Fun {
import java.net.URL
def main(args: Array[String]): Unit = {
val url1 = "http://localhost:4000/a/b/c?x=1&y=2#asdf"
val url2 = "http://example.com:4000/a/localhostb/c?x=1&y=2#asdf"
val urls = List(url1, url2)
// your approach
println("Your approach")
urls.map( urlString => extractUrl(urlString ))
.foreach(println)
println("New approach")
urls.map(x => extractUrl2(x))
.filter( x => x.host.startsWith("localhost") )
.foreach(println)
}
case class HostPort(host: String, port: Option[String])
def extractUrl2: String => HostPort = urlString => {
val url = new URL(urlString)
HostPort(url.getHost,
url.getPort match {
case -1 => None
case i => Some(i.toString)
})
}
def extractUrl: String => (String, String) = url =>
url
.split("/")
.toList
.filter(_.startsWith("localhost"))
.flatMap(e => e.split(":").toList)
.foldLeft[(String, String)](("", "")) { (acc, x) =>
acc match {
case ("", "") => (x, "")
case (a, "") => (a, x)
case z => z
}
}
}
yields
Your approach
(localhost,4000)
(localhostb,)
New approach
HostPort(localhost,Some(4000))
I don't think it is possible to define an empty Tuple.
I tried to use (->) but that resolves to a Tuple2.type and not a Tuple2.
If the values of your Tuple are optional, use the type system to express that:
(Option[String], Option[String])
Better yet, you could define a case class for your data structure:
case class HostAndPort(host: Option[String], port: Option[String])
This would provide default values for each type within the tuple:
let my_tuple: (String, usize) = Default::default();
I am trying to learn some functional programming in Scala.
I have this Map:
val params: Map[String, QueryMap] = Map(
"a" -> SimpleQueryVal("1"),
"b" -> ComplexQueryVal("2", "3")
)
Where QueryMap is (might not be the best approach):
sealed trait QueryMap
case class SimpleQueryVal(value: String) extends QueryMap
case class ComplexQueryVal(values: String*) extends QueryMap
My result would be having a string like query parameters: ?a=1&b=2&b=3
I tried something, but my method return an Iterator[String] even I use mkString, looks ugly and I am sure that there's a very simple way of doing it.
def paramString(queryMap: Map[String, QueryMap]) = queryMap.keys.map { key =>
val params = queryMap(key) match {
case SimpleQueryVal(x) => "%s=%s".format(key, x)
case complexQuery: ComplexQueryVal => complexQuery.values.map { value =>
"%s=%s".format(key, value)
}
}
val result: String = params match {
case s: String => s + "&"
case s: ArrayBuffer[_] => s.mkString("&")
}
result.mkString
}
I would appreciate any idea that would make me learn something for today. :)
I think the result String can be built in a simpler, more straight forward, manner.
def paramString(queryMap: Map[String, QueryMap]): String = queryMap.map{
case (k, sq: SimpleQueryVal) => s"$k=${sq.value}"
case (k, cq: ComplexQueryVal)=> cq.values.map(k + "=" + _).mkString("&")
}.mkString("&")
A little cleaner:
def paramString(queryMap: Map[String, QueryMap]) = queryMap.flatMap {
case (key, SimpleQueryVal(x)) => Seq(s"$key=$x")
case (key, ComplexQueryVal(values # _*)) => values.map {v =>
s"$key=$v"
}
}.mkString("&")
No need for ArrayBuffer or to repeat the .mkString("&").
Keep in mind that this is good for just learning. If you're actually trying to handle HTTP query string parameters, you need to URLEncode the keys and the values and there's probably better libraries for that.
Try this:
def paramString(queryMap: Map[String, QueryMap]) = {
val qParams = queryMap.keys.map { key =>
queryMap(key) match {
case SimpleQueryVal(x) => "%s=%s".format(key, x)
case complexQuery: ComplexQueryVal => complexQuery.values.map { value =>
"%s=%s".format(key, value)
}.mkString("&")
}
}
qParams.mkString("&")
}
println(paramString(params))
Here, first you get a Set[String] like a=1 or b=2&b=3. Then you simply do another .mkString("&") to concatenate them all.
I am struggling with finding an elegant FP approach to solving the following problem in Scala:
Say I have a set of candidate keys
val validKeys = Set("key1", "key2", "key3")
And a list that
Starts with a key
has some number of non-keys (> 0) between each key
Does not end with a key
For example:
val myList = List("key3", "foo", "bar", "key1", "baz")
I'd like to transform this list into a map by choosing using valid keys as the key and aggregating non-keys as the value. So, in the example above:
("key3" -> "foo\nbar", "key1" -> "baz")
Thanks in advance.
Short and simple:
def create(a: List[String]): Map[String, String] = a match {
case Nil => Map()
case head :: tail =>
val (vals, rest) = tail.span(!validKeys(_))
create(rest) + (head -> vals.mkString("\n"))
}
Traversing a list from left to right, accumulating a result should suggest foldLeft
myList.foldLeft((Map[String, String](), "")) {
case ((m, lk), s) =>
if (validKeys contains s)
(m updated (s, ""), s)
else (m updated (lk, if (m(lk) == "") s else m(lk) + "\n" + s), lk)
}._1
// Map(key3 -> foo\nbar, key1 -> baz)
As a first approximation solution:
def group(list:List[String]):List[(String, List[String])] = {
#tailrec
def grp(list:List[String], key:String, acc:List[String]):List[(String, List[String])] =
list match {
case Nil => List((key, acc.reverse))
case x :: xs if validKeys(x) => (key, acc.reverse)::group(x::xs)
case x :: xs => grp(xs, key, x::acc)
}
list match {
case Nil => Nil
case x::xs => grp(xs, x, List())
}
}
val map = group(myList).toMap
Another option:
list.foldLeft((Map[String, String](), "")) {
case ((map, key), item) if validKeys(item) => (map, item)
case ((map, key), item) =>
(map.updated(key, map.get(key).map(v => v + "\n" + item).getOrElse(item)), key)
}._1
I am wondering if it is possible to do pattern matching on Jackson JSON objects in Scala. We are currently using jackson-module-scala in a Project heavily and would benefit from being able to do pattern matching of Json ObjectNode/JsonNode objects.
If this is not possible, how would I go about adding this functionality? I was thinking something in terms of implicit conversion from JsonNode/ObjectNode to MyClass, where MyClass would have unapply method, doing JsonNode.toString and regex matching. If my logic is correct, I could then do pattern matching on JsonNode objects. Of course, there could be better ways I am not aware of, or this one may not work for reasons I am not yet aware of. To illustrate my case, I would like to be able to perform something in terms of:
val mapper = new ObjectMapper()
mapper.registerModule(DefaultScalaModule)
val json = mapper.createObjectNode()
.put("key1","value1")
.put("key2","value2")
.put("key3","value3")
json match {
case MyClass("key1", "value1", "key2", y) => println("Found key1 with value1, where key2 is " + y)
case MyClass("key1", x) => println("Key1 value is " + x)
...
_ => println("No match found")
}
Have you tried to make use of the case class deserialization?
https://github.com/FasterXML/jackson-module-scala/blob/master/src/test/scala/com/fasterxml/jackson/module/scala/deser/CaseClassDeserializerTest.scala
If that doesn't work, I think you would be better off creating extractors to represent your domain objects. Code below assumes a scala Map, but it should give you an idea. No implicit conversion required.
case class DomainObjectA(v1: String, v2: String)
object DomainObjectAExtractor {
def unapply(m: Map[String, String]) = for {
v1 <- m.get("key1")
v2 <- m.get("key2")
} yield DomainObjectA(v1, v2)
}
case class DomainObjectB(v3, v4, v5)
object DomainObjectBExtractor {
def unapply(m: Map[String, String]) = for {
v3 <- m.get("key3")
v4 <- m.get("key4")
v5 <- m.get("key5")
} yield DomainObjectB(v3, v4, v5)
}
json match {
case DomainObjectAExtractor(a#DomainObjectA(_, _)) => a
case DomainObjectBExtractor(b#DomainObjectB(_, _, _)) => b
}
However, if you insist on trying to match against the key/value pairs, there may be ways to accomplish something which is acceptable for you. It is not possible to pass input into the unapply function from the case, which I think would be required if I understand what you want to do correctly. It may be possible to do this with macros which are experimental in the soon-to-be-officially-released scala 2.10. I haven't played with them enough to know if this is or is not possible though.
If ordering of keys was assumed, you could come up with a :: unapply operator similar to :: for list. This could extract the K, V pairs in this known order. Personally, this is too fragile for my tastes.
val json = Map(("key1" -> "one"), ("key2" -> "two"))
object -> {
def unapply[A, B](ab: (A, B)) = Some(ab)
}
object :: {
def unapply[K, V](m: Map[K, V]): Option[((K, V), Map[K, V])] =
m.headOption.map(_ -> m.tail)
}
scala> json match {
| case ("key1" -> "one") :: ("key2" -> value2) :: _ => value2
| }
res0: java.lang.String = two
You would not be able to extract keys in the wrong order though
scala> json match {
| case ("key2" -> value2) :: _ => value2
| case _ => "match fail"
| }
res2: java.lang.String = match fail
You could write Key1, Key2, Key3 as well. This may or may not scale well.
object && {
def unapply[A](a: A) = Some((a, a))
}
object Key2 {
def unapply[V](m: Map[String, V]) = m.get("key2")
}
object Key1 {
def unapply[V](m: Map[String, V]) = m.get("key1")
}
scala> json match {
| case Key2(value2) && Key1(value1) => (value2, value1)
| }
res5: (java.lang.String, java.lang.String) = (two,one)