scala: union of two maps whose key type is the same and whose value type is a collection of elements, but whose types are different - scala

I would like to create a union of two maps whose key type is the same and whose value type is a collection of elements, but whose types are different.
Consider the following contrived example:
case class Child(name: String)
val peopleToChildren: Map[String, Seq[Child]] =
Map("max" -> Seq(Child("a"), Child("b")),
"yaneeve" -> Seq(Child("y"), Child("d")))
case class Pet(name: String)
val peopleToPets: Map[String, Seq[Pet]] =
Map("max" -> Seq(Pet("fifi")),
"jill" -> Seq(Pet("bobo"), Pet("jack"), Pet("Roger rabbit")))
val peopleToChildrenAndDogs: Map[String, (Seq[Child], Seq[Pet])] = {
// people may have children
// people may have pets
// would like a map from people to a tuple with a potentially empty list of children and a
// potentially empty list of pets
// ???
}
What would be a way to do it which is concise, idiomatic, but still legible?
I found no single function that can do that in the standard scala collections library.
Proposed solutions can be based solely on the standard library, or propose an external solution.
I post it here since I could not easily find an online solution to a seemingly standard operation.

This appears to work.
val peopleToChildrenAndDogs: Map[String, (Seq[Child], Seq[Pet])] = {
(peopleToChildren.keySet ++ peopleToPets.keySet).map { k =>
k -> (peopleToChildren.getOrElse(k, Seq())
,peopleToPets.getOrElse(k, Seq()))
}.toMap
}
Get all the keys. For every key do a getOrElse() on each of the feeder Maps.

Just for the curious, here's how it could be done using Scalaz:
import scalaz._, Scalaz._
case class Child(name: String)
val peopleToChildren = Map(
"max" -> List(Child("a"), Child("b")),
"yaneeve" -> List(Child("y"), Child("d"))
)
case class Pet(name: String)
val peopleToPets = Map(
"max" -> List(Pet("fifi")),
"jill" -> List(Pet("bobo"), Pet("jack"), Pet("Roger rabbit"))
)
val peopleToChildrenAndPets: Map[String, (List[Child], List[Pet])] =
peopleToChildren.strengthR(nil[Pet]) |+| peopleToPets.strengthL(nil[Child])
Explanation:
nil[Pet] is just an alias for List.empty[Pet]
strengthR for a given Functor tuples contained values, so that its parameter is at the right. Here it is equivalent to peopleToChildren.mapValues(v => (v, nil[Pet]))
strengthL is the same, but element will be added to the left
|+| is an append operator for a given Semigroup. The one here is derived recursively:
for Map[K, V], it uses |+| to combine values of type V if a given key exists in both Maps. If the value is only present in one of them, it will be retained as is. Here, V = (List[Child], List[Pet])
for tuples (A, B), it again uses |+| to combine both As and Bs. Here, A = List[Child] and B = List[Pet]
for lists of any type (as well as strings, vectors or streams) it does concatenation. This is why I had to change type of Map values to be Lists - for generic Seqs this operation is not defined
Result:
peopleToChildrenAndPets: Map[String, (List[Child], List[Pet])] = Map(
"max" -> (List(Child("a"), Child("b")), List(Pet("fifi"))),
"jill" -> (
List(),
List(Pet("bobo"), Pet("jack"), Pet("Roger rabbit"))
),
"yaneeve" -> (List(Child("y"), Child("d")), List())
)

To answer my own question, the following is the way that I solved it, but it seems overly long and complex:
Welcome to the Ammonite Repl 1.0.2
(Scala 2.11.11 Java 1.8.0_91)
If you like Ammonite, please support our development at www.patreon.com/lihaoyi
# case class Child(name: String)
defined class Child
# val peopleToChildren: Map[String, Seq[Child]] =
Map("max" -> Seq(Child("a"), Child("b")),
"yaneeve" -> Seq(Child("y"), Child("d")))
peopleToChildren: Map[String, Seq[Child]] = Map("max" -> List(Child("a"), Child("b")), "yaneeve" -> List(Child("y"), Child("d")))
#
# case class Pet(name: String)
defined class Pet
# val peopleToPets: Map[String, Seq[Pet]] =
Map("max" -> Seq(Pet("fifi")),
"jill" -> Seq(Pet("bobo"), Pet("jack"), Pet("Roger rabbit")))
peopleToPets: Map[String, Seq[Pet]] = Map("max" -> List(Pet("fifi")), "jill" -> List(Pet("bobo"), Pet("jack"), Pet("Roger rabbit")))
#
# val peopleToChildrenAndDogs: Map[String, (Seq[Child], Seq[Pet])] = {
// people may have children
// people may have pets
// would like a map from people to a tuple with a potentially empty list of children and a
// potentially empty list of pets
val paddedPeopleToChildren = peopleToChildren.map{ case (person, children) => person -> (children, List.empty[Pet])}
val paddedPeopleToPets = peopleToPets.map{ case (person, pets) => person ->(List.empty[Child], pets)}
val notGoodEnough = paddedPeopleToPets ++ paddedPeopleToChildren // this is here to show that it does not work since it overwrites the value of a key - Map(max -> (List(Child(a), Child(b)),List()), jill -> (List(),List(Pet(bobo), Pet(jack), Pet(Roger rabbit))), yaneeve -> (List(Child(y), Child(d)),List()))
val allSeq = paddedPeopleToPets.toSeq ++ paddedPeopleToChildren.toSeq
val grouped = allSeq.groupBy(_._1).mapValues(_.map { case (_, tup) => tup })
val solution = grouped.mapValues(_.unzip).mapValues {case (wrappedChildren, wrappedPets) => (wrappedChildren.flatten, wrappedPets.flatten)}
solution
}
peopleToChildrenAndDogs: Map[String, (Seq[Child], Seq[Pet])] = Map(
"yaneeve" -> (ArrayBuffer(Child("y"), Child("d")), ArrayBuffer()),
"max" -> (ArrayBuffer(Child("a"), Child("b")), ArrayBuffer(Pet("fifi"))),
"jill" -> (ArrayBuffer(), ArrayBuffer(Pet("bobo"), Pet("jack"), Pet("Roger rabbit")))
)

Related

Read Hocon config as a Map[String, String] with key in dot notation and value

I have following HOCON config:
a {
b.c.d = "val1"
d.f.g = "val2"
}
HOCON represents paths "b.c.d" and "d.f.g" as objects. So, I would like to have a reader, which reads these configs as Map[String, String], ex:
Map("b.c.d" -> "val1", "d.f.g" -> "val2")
I've created a reader and trying to do it recursively:
import scala.collection.mutable.{Map => MutableMap}
private implicit val mapReader: ConfigReader[Map[String, String]] = ConfigReader.fromCursor(cur => {
def concat(prefix: String, key: String): String = if (prefix.nonEmpty) s"$prefix.$key" else key
def toMap(): Map[String, String] = {
val acc = MutableMap[String, String]()
def go(
cur: ConfigCursor,
prefix: String = EMPTY,
acc: MutableMap[String, String]
): Result[Map[String, Object]] = {
cur.fluent.mapObject { obj =>
obj.value.valueType() match {
case ConfigValueType.OBJECT => go(obj, concat(prefix, obj.pathElems.head), acc)
case ConfigValueType.STRING =>
acc += (concat(prefix, obj.pathElems.head) -> obj.asString.right.getOrElse(EMPTY))
}
obj.asRight
}
}
go(cur, acc = acc)
acc.toMap
}
toMap().asRight
})
It gives me the correct result but is there a way to avoid MutableMap here?
P.S. Also, I would like to keep implementation by "pureconfig" reader.
The solution given by Ivan Stanislavciuc isn't ideal. If the parsed config object contains values other than strings or objects, you don't get an error message (as you would expect) but instead some very strange output. For instance, if you parse a typesafe config document like this
"a":[1]
The resulting value will look like this:
Map(a -> [
# String: 1
1
])
And even if the input only contains objects and strings, it doesn't work correctly, because it erroneously adds double quotes around all the string values.
So I gave this a shot myself and came up with a recursive solution that reports an error for things like lists or null and doesn't add quotes that shouldn't be there.
implicit val reader: ConfigReader[Map[String, String]] = {
implicit val r: ConfigReader[String => Map[String, String]] =
ConfigReader[String]
.map(v => (prefix: String) => Map(prefix -> v))
.orElse { reader.map { v =>
(prefix: String) => v.map { case (k, v2) => s"$prefix.$k" -> v2 }
}}
ConfigReader[Map[String, String => Map[String, String]]].map {
_.flatMap { case (prefix, v) => v(prefix) }
}
}
Note that my solution doesn't mention ConfigValue or ConfigReader.Result at all. It only takes existing ConfigReader objects and combines them with combinators like map and orElse. This is, generally speaking, the best way to write ConfigReaders: don't start from scratch with methods like ConfigReader.fromFunction, use existing readers and combine them.
It seems a bit surprising at first that the above code works at all, because I'm using reader within its own definition. But it works because the orElse method takes its argument by name and not by value.
You can do the same without using recursion. Use method entrySet as following
import scala.jdk.CollectionConverters._
val hocon =
"""
|a {
| b.c.d = "val1"
| d.f.g = val2
|}""".stripMargin
val config = ConfigFactory.load(ConfigFactory.parseString(hocon))
val innerConfig = config.getConfig("a")
val map = innerConfig
.entrySet()
.asScala
.map { entry =>
entry.getKey -> entry.getValue.render()
}
.toMap
println(map)
Produces
Map(b.c.d -> "val1", d.f.g -> "val2")
With given knowledge, it's possible to define a pureconfig.ConfigReader that reads Map[String, String] as following
implicit val reader: ConfigReader[Map[String, String]] = ConfigReader.fromFunction {
case co: ConfigObject =>
Right(
co.toConfig
.entrySet()
.asScala
.map { entry =>
entry.getKey -> entry.getValue.render()
}
.toMap
)
case value =>
//Handle error case
Left(
ConfigReaderFailures(
ThrowableFailure(
new RuntimeException("cannot be mapped to map of string -> string"),
Option(value.origin())
)
)
)
}
I did not want to write custom readers to get a mapping of key value pairs. I instead changed my internal data type from a map to list of pairs (I am using kotlin), and then I can easily change that to a map at some later internal stage if I need to. My HOCON was then able to look like this.
additionalProperties = [
{first = "sasl.mechanism", second = "PLAIN"},
{first = "security.protocol", second = "SASL_SSL"},
]
additionalProducerProperties = [
{first = "acks", second = "all"},
]
Not the best for humans... but I prefer it to having to build custom parsing components.

how to access key values within nested map in Scala

I have a List of Maps. One of the maps has another map inside it (2 level deep). I need to access some of keys from the inner most map and finally change the values. The issue I'm facing is to retrieve the keys from the inner most map. I'm very new to Scala and tried different things without luck.
I have flatten the List to Map and then tried to retrieve the key, values. The thing is, I can print the entire inner map, but not sure how to iterate thru that.
Below is the code: at a very basic, I would like to retrieve the values corresponding to keys from innermost map; say for the keys "isStudentLoankRequested", "schoolStructure".
object ListToMaps {
def main(args: Array[String]) {
val dataInputKeysListvar = List(Map("identityKeyName" -> "MY_ID", "identityKeyValue" -> "abcd-1234-xyz"),
Map("identityKeyName" -> "OUR_ID", "identityKeyValue" -> "1234567890",
"identityInformation" -> Map("writeFrequency" -> "NEVER", "studentStatus" -> "SEP", "annualValue" -> 0,
"schoolStructure" -> "OTHER", "studentType" -> "FTS", "occupationDescription" -> "other",
"studentAccountBalanceRange" -> "string", "isStudentLoankRequested" -> "N", "schoolName" -> "myschool",
"parentsIncome" -> 100)),
Map("identityKeyName" -> "FINAL_DECISION_KEY", "identityKeyValue" -> "0000-ABCD-4567-IJKL"))
val x = dataInputKeysListvar.flatten.toMap
val y = x("identityInformation")
if (x.contains("identityInformation")){
println("value of y is" + y)
}
}
}
As you can see from the print stmt, I can print the entire map of the inner most map, but need help in terms of iterating thru that.
If you know at compile time which fields and values you need to change.
You can hard code the logic, like this:
def fixData(data: List[Map[String, Any]]): List[Map[String, Any]] =
data.map { outerMap =>
outerMap.get("identityInformation") match {
case Some(innerMap) =>
// Put as many key pairs you want to change.
// Note: if the key does not exists it will be added!
val updatedInnerMap = innerMap.asInstanceOf[Map[String, String]] ++ Map(
"isStudentLoankRequested" -> "No"
)
outerMap + ("identityInformation" -> updatedInnerMap)
case None =>
outerMap
}
}
If the key-values to change are dynamic, and / or some inner keys may not exists, or if the level of nesting can go on.
You can do something like this:
def updateMap(map: Map[String, Any], updates: Map[String, Any]): Map[String, Any] =
map.map {
case (key, value: Map[_, _]) =>
updates.get(key) match {
case Some(innerUpdates : Map[_, _]) =>
key -> updateMap(
map = value.asInstanceOf[Map[String, Any]],
updates = innerUpdates.asInstanceOf[Map[String, Any]]
)
case Some(newValue) =>
key -> newValue
case None =>
key -> value
}
case (key, value) =>
key -> updates.getOrElse(key, default = value)
}
def fixData(data: List[Map[String, Any]], updates: Map[String, Any]): List[Map[String, Any]] =
data.map(outerMap => updateMap(outerMap, updates))
Note: The above snippets use "unsafe" techniques like asInstanceOf because we lost type safety the moment you got a Map[String, Any]. Always that I see such structure, I think of JSON. I would suggest you to use an appropriate library for managing such kind of data, like circe, instead of writing code as the above.

How to sort a list of scala objects by sort order of other list?

I am having following 2 lists in scala.
case class Parents(name: String, savings: Double)
case class Children(parentName: String, debt: Double)
val parentList:List[Parents] = List(Parents("Halls",1007D), Parents("Atticus",8000D), Parents("Aurilius",900D))
val childrenList:List[Children] = List(Children("Halls",9379.40D), Children("Atticus",9.48D), Children("Aurilius",1100.75D))
val sortedParentList:List[Parents] = parentList.sortBy(_.savings).reverse
// sortedParentList = List(Parents(Atticus,8000.0), Parents(Halls,1007.0), Parents(Aurilius,900.0))
now my parenList is Sorted By savings in decreasing order, I want my childrenList to be sorted in the way that it follows parentList Order.
i.e. expected order will be following
// sortedParentList = List(Children(Atticus,9.48D), Children(Halls,9379.40D), Children(Aurilius,1100.75D))
Well, if you know both lists are initially in the same order (you can always ensure that by sorting both by name), you can just sort them both in one go:
val (sortedParentList, sortedChildrenList) = (parents zip children)
.sortBy(-_._1.savings)
.unzip
Or you can define the ordering ahead of time, and use it to sort both lists:
val order = parentList.map(p => p.name -> -p.savings).toMap
val sortedParentList = parentList.sortBy(order(_.name))
val sortedChildrenList = childrenList.sortBy(order(_.parentName))
Or you can sort parents first (maybe, they are already sorted), and then define the order:
val order = sortedParentList.zipWithIndex.map { case(p, idx) => p.name -> idx }.toMap
val sortedChildrenList = childrenList.sortBy(c => order(c.parentName))
case class Parents(name: String, savings: Double)
case class Children(parentName: String, debt: Double)
val familiesList: List[(Parents, Children)] = List(
Parents("Halls",1007D) -> Children("Halls",9379.40D),
Parents("Atticus",8000D) -> Children("Atticus",9.48D),
Parents("Aurilius",900D) -> Children("Aurilius",1100.75D))
val (sortedParents, sortedChildren) = familiesList.sortBy {
case (parents, _) => -parents.savings
}.unzip

Expanding a list of cars by a child collection as the key

I have a list of cars:
val cars = List(car1, car2, car3, car4, car5)
case class car(model: String, age: Int, partIds: Seq[String])
I now want to transform this list into a Map, where the key is the partId and the value is all the cars with that part.
val partMap: Map[String, List[Car]]
You will need to jump through some hoops by using intermediate types. The solution is to first get from your List[Car] into to List[PartId -> Car]. Dropping the Seq of parts makes your life easer. You can group your cars easily.
The mapValues is a function on Map. It will iterate over every tuple and will require some function that takes a type equal to the value of your Map...in my case before mapValues I had a Map[String, List[String -> Car]].
The mapValues wants a function with the signature (carMapping : List[(String, Car]) : A ... our desired type is of course List[Car]
here is a something on groupBy and a little about mapValues: http://markusjais.com/the-groupby-method-from-scalas-collection-library/
case class Car(model: String, age: Int, partIds: Seq[String])
object ListToMap extends App {
val car1 = Car("corolla", 1, Seq("b"))
val car2 = Car("camry", 5, Seq("b", "c"))
val car3 = Car("fit", 6, Seq("e"))
val car4 = Car("prelude", 2, Seq("e", "f"))
val car5 = Car("cobalt", 10, Seq("j"))
val cars = List(car1, car2, car3, car4, car5)
//For Every Car lets make the list of Tuples for PartId -> Car
def partMapping(car : Car) : Seq[(String, Car)] = car.partIds.map(part => part -> car)
def toPartMap(cars : List[Car]) : Map[String, List[Car]] =
cars
//Get the List of Tuples PartId -> Car and then flatten the internal list (same as map().flatten)
.flatMap(partMapping)
// group all the tuples by the partId
.groupBy(_._1)
// We currently have a Map[String, List[(partId -> Car)]] we need to clean that up a bit to remove the partId
.mapValues( carMapping => carMapping.map(_._2))
toPartMap(cars).foreach(println)
}
cars flatMap ( x => x.partIds map ((_, x))) groupBy (_._1) mapValues (_ map (_._2))

Best way to filter and sort a Map by set of keys

I have a Map instance (immutable):
val source = Map(
("foo", "spam"),
("bar", "hoge"),
("baz", "eggs"),
("qux", "corge"),
("quux", "grault")
)
and I have number of keys (Set or List) in some order which may or may not exist in source map:
baz
foo
quuuuux // does not exist in a source map
But what is the best and cleanest way to iterate over the source map with concise scala style, filter it by my keys and place filtered items into resulting map in the same order as keys are?
Map(baz -> eggs, foo -> spam)
P.S. To clarify - order of keys in resulting map must be the same as in filtration keys list
If you have:
val source = Map(
"foo" -> "spam",
"bar" -> "hoge",
"baz" -> "eggs",
"qux" -> "corge",
"quux" -> "grault"
)
and
val keys = List( "baz", "foo", "quuuux" )
Then, you can:
import scala.collection.immutable.SortedMap
SortedMap(source.toSeq:_*).filter{ case (k,v) => keys.contains(k) }
val keys = List("foo", "bar")
val map = Map("foo" -> "spam", "bar" -> "hoge", "baz" -> "eggs")
keys.foldLeft(ListMap.empty[String, String]){ (acc, k) =>
map.get(k) match {
case Some(v) => acc + (k -> v)
case None => acc
}
}
This will iterate over the keys, building a map containing only the matching keys.
Please note that you need a ListMap to preserve the ordering of keys, although the implementation of ListMap will return the elements in the opposite order they were inserted (since keys are prepended as head of the list)
LinkedHashMap would ensure exact insertion order, but it's a mutable data structure.
If you need an ordered Map, you could use something like a TreeMap with a custom key ordering. So given
import scala.collection.immutable.TreeMap
val source = Map(
("foo", "spam"),
("bar", "hoge"),
("baz", "eggs"),
("qux", "corge"),
("quux", "grault")
)
val order: IndexedSeq[String] = IndexedSeq("baz", "foo", "quuuuux")
implicit val keyOrdering: Ordering[String] = Ordering.by(order.indexOf)
You have choice, either iterate over the ordered keys:
val result1: TreeMap[String, String] = order.collect {
case key if source.contains(key) => key -> source(key)
}(collection.breakOut)
// or a bit shorter
val result2: TreeMap[String, String] = order.flatMap { key => source.get(key).map(key -> _) }(collection.breakOut)
or filter from the source map:
val result3: TreeMap[String, String] = TreeMap.empty ++ source.filterKeys(order.contains)
I am not sure which one would be the most efficient, but I suspect the flatMap one might be fastest, at least for your simple example. Though, imho, the last example is better readable than the others.