Related
I have a Map of type Map[A, Map[B, C]].
How can I inverse it to have a Map of type Map[B, Map[A, C]]?
There are lots of ways you could define this operation. I'll walk through a couple of the ones that I find the clearest. For the first implementation I'll start with a helper method:
def flattenNestedMap[A, B, C](nested: Map[A, Map[B, C]]): Map[(A, B), C] =
for {
(a, innerMap) <- nested
(b, c) <- innerMap
} yield (a, b) -> c
This flattens the nested map to a map from pairs to values. Next we can define another helper operation that gets us almost what we need.
def groupByBs[A, B, C](flattened: Map[(A, B), C]): Map[B, Map[(A, B), C]] =
flattened.groupBy(_._1._2)
Now we just need to remove the redundant B from the keys in the inner map:
def invert[A, B, C](nested: Map[A, Map[B, C]]): Map[B, Map[A, C]] =
groupByBs(flattenNestedMap(nested)).mapValues(
_.map {
case ((a, _), c) => a -> c
}
)
(Note that mapValues is lazy, which means that the result will be recomputed every time you use it. In general this isn't a problem, and there are easy workarounds, but they're not really relevant to the question.)
And we're done:
scala> invert(Map(1 -> Map(2 -> 3), 10 -> Map(2 -> 4)))
res0: Map[Int,Map[Int,Int]] = Map(2 -> Map(1 -> 3, 10 -> 4))
You could also skip the helper methods and just chain the operations in invert. I find breaking them up a little clearer, but that's a matter of style.
Alternatively you could use a couple of folds:
def invert[A, B, C](nested: Map[A, Map[B, C]]): Map[B, Map[A, C]] =
nested.foldLeft(Map.empty[B, Map[A, C]]) {
case (acc, (a, innerMap)) =>
innerMap.foldLeft(acc) {
case (innerAcc, (b, c)) =>
innerAcc.updated(b, innerAcc.getOrElse(b, Map.empty).updated(a, c))
}
}
Which does the same thing:
scala> invert(Map(1 -> Map(2 -> 3), 10 -> Map(2 -> 4)))
res1: Map[Int,Map[Int,Int]] = Map(2 -> Map(1 -> 3, 10 -> 4))
The foldLeft version has more of the shape of the straightforward imperative version—we're (functionally) iterating through the key-value pairs of the outer and inner maps and building up the result. Off the top of my head I'd guess it's also a little more efficient, but I'm not sure about that, and it's unlikely to matter much, so I'd suggest choosing the one you personally find clearer.
You can simply do it using map operation on given Map collection :
scala> Map("A" -> Map("B" -> "C"), "X" -> Map("Y" -> "Z"))
res1: scala.collection.immutable.Map[String,scala.collection.immutable.Map[String,String]] = Map(A -> Map(B -> C), X -> Map(Y -> Z))
scala> res1.map{ case (key, valueMap) => valueMap.map{ case (vmKey, vmValue) => (vmKey -> Map(key -> vmValue)) } }
res2: scala.collection.immutable.Iterable[scala.collection.immutable.Map[String,scala.collection.immutable.Map[String,String]]] = List(Map(B -> Map(A -> C)), Map(Y -> Map(X -> Z)))
I am trying to flatten a map where the keys are traversables, in the sense that:
Map( Set(1, 2, 3) -> 'A', Set(4, 5, 6) -> 'B')
should flatten to:
Map(5 -> B, 1 -> A, 6 -> B, 2 -> A, 3 -> A, 4 -> B)
Here is what I did:
def fuse[A, B, T <: Traversable[A]](mapOfTravs: Map[T, B]): Map[A, B] = {
val pairs = for {
trav <- mapOfTravs.keys
key <- trav
} yield (key, mapOfTravs(trav))
pairs.toMap
}
It works. But:
Is there a simpler way to do this?
I'm not very comfortable with the Scala type system and I'm sure this can be improved. I have to specify the types explicitly whenever I use my function:
val map2 = Map( Set(1, 2, 3) -> 'A', Set(4, 5, 6) -> 'B')
val fused2 = fuse[Int, Char, Set[Int]](map2)
val map1: Map[Traversable[Int], Char] = Map( Set(1, 2, 3) -> 'A', Set(4, 5, 6) -> 'B')
val fused1 = fuse[Int, Char, Traversable[Int]](map1)
P.S.: this fuse function does not make much sense when the key traversables have a non-null intersection.
This is basically what you're doing in the for comprehension, but simplified a little bit:
def fuse[A, B, T <: Traversable[A]](mapOfTravs: Map[T, B]): Map[A, B] = {
mapOfTravs.flatMap({ case (s, c) => s.map(i => i -> c) })
}
Not much you can do about the types, I'm sure there's some type lambda shenanigans that you can do, I'm just not sure how to do them...
UPDATE
Here's a slightly better for version, same as the above flatMap:
def fuse2[A, B, T <: Traversable[A]](mapOfTravs: Map[T, B]): Map[A, B] = {
for {
(keys, value) <- mapOfTravs
key <- keys
} yield key -> value
}
Like #Azzie, I was thinking zip, but maybe Azzie has the advantage with those zees.
scala> val m = Map( Set(1, 2, 3) -> 'A', Set(4, 5, 6) -> 'B')
m: scala.collection.immutable.Map[scala.collection.immutable.Set[Int],Char] = Map(Set(1, 2, 3) -> A, Set(4, 5, 6) -> B)
scala> (m map { case (k, v) => k zip (Stream continually v) }).flatten.toMap
res0: scala.collection.immutable.Map[Int,Char] = Map(5 -> B, 1 -> A, 6 -> B, 2 -> A, 3 -> A, 4 -> B)
scala> (m map { case (k, v) => k zipAll (Nil, null, v) }).flatten.toMap
res1: scala.collection.immutable.Map[Any,Char] = Map(5 -> B, 1 -> A, 6 -> B, 2 -> A, 3 -> A, 4 -> B)
scala> m flatMap { case (k, v) => k zip (Stream continually v) }
res2: scala.collection.immutable.Map[Int,Char] = Map(5 -> B, 1 -> A, 6 -> B, 2 -> A, 3 -> A, 4 -> B)
It's not obvious how to generalize it nicely.
This looks horrible and using 0 is kind of cheating but it does the job
m.map( {case (s,c) => s.zipAll(Set(c),0,c)} ).flatten.toMap
Since I'm apparently on a "hideously generic implicits" kick lately:
import scala.collection.MapLike
import scala.collection.TraversableLike
import scala.collection.generic.CanBuildFrom
implicit class Map_[
A,
B,
T1 : ({type L[X] = X => TraversableLike[A, T2]})#L,
T2,
M1 : ({type L[X] = X => MapLike[T1, B, M2]})#L,
M2 <: MapLike[T1, B, M2] with Map[T1, B]
](map: M1) {
def fuse[M3](implicit cbfM: CanBuildFrom[M2, (A, B), M3]) : M3 =
map.flatMap({ case (k, v) => k.toTraversable.map((_, v)) })
}
Examples:
scala> Map(Set(1, 2, 3) -> 'A', Set(4, 5, 6) -> 'B').fuse
res: scala.collection.immutable.Map[Int,Char] =
Map(5 -> B, 1 -> A, 6 -> B, 2 -> A, 3 -> A, 4 -> B)
scala> Map(Array(1, 2, 3) -> 'A', Array(4, 5, 6) -> 'B').fuse
res: scala.collection.immutable.Map[Int,Char] =
Map(5 -> B, 1 -> A, 6 -> B, 2 -> A, 3 -> A, 4 -> B)
I have to often transpose a "rectangular" collection-of-collections in Scala, e.g.: a list of maps, a map of lists, a map of maps, a set of lists, a map of sets etc. Since collections can be uniformly viewed as a mapping from a specific domain to a co-domain (e.g.: a List[A]/Array[A] is a mapping from the Int domain to the A co-domain, Set[A]is a mapping from the A domain to the Boolean co-domain etc.), I'd like to write a clean, generic function to do a transpose operation (e.g.: turn a map of lists to the transposed list of maps). However, I'm having trouble because other than the () operator, Scala doesn't seem to have a unified API to view collections abstractly as mappings ?
So I end up writing a separate transpose for each type of collection-of-collections as follows:
def transposeMapOfLists[A,B]( mapOfLists: Map[A,List[B]] ) : List[Map[A,B]] = {
val k = ( mapOfLists keys ) toList
val l = ( k map { mapOfLists(_) } ) transpose;
l map { v => ( k zip v ) toMap }
}
def transposeListOfMaps[A,B]( listOfMaps: List[Map[A,B]]) : Map[A,List[B]] = {
val k = ( listOfMaps(0) keys ) toList
val l = ( listOfMaps map { m => k map { m(_) } } ) transpose;
( k zip l ) toMap
}
def transposeMapOfMaps[A,B,C]( mapOfMaps: Map[A,Map[B,C]] ) : Map[B,Map[A,C]] = {
val k = ( mapOfMaps keys ) toList
val listOfMaps = k map { mapOfMaps(_) }
val mapOfLists = transposeListOfMaps( listOfMaps )
mapOfLists map { p => ( p._1, ( k zip p._2 ) toMap ) }
}
Can someone help me unify these methods into one generic collection-of-collections transpose ? It will also help me (and I am sure others) learn some useful Scala features in the process.
ps: I have ignored exception handling and have assumed the input collection-of-collections is rectangular, i.e., all of the inner collections' domain elements constitute the same set.
I'm sure the following messy version using type classes could be cleaned up a lot, but it works as a quick proof-of-concept. I don't see an easy way to get the return types right without dependent method types (I'm sure it's possible), so you'll have to use -Xexperimental:
trait Mapping[A, B, C] {
type M[D] <: PartialFunction[A, D]
def domain(c: C): Seq[A]
def fromPairs[D](ps: Seq[(A, D)]): M[D]
def codomain(c: C)(implicit ev: C <:< PartialFunction[A, B]) =
domain(c).map(c)
def toPairs(c: C)(implicit ev: C <:< PartialFunction[A, B]) =
domain(c).map(a => (a, c(a)))
}
implicit def seqMapping[A, B <: Seq[A]] = new Mapping[Int, A, B] {
type M[C] = Seq[C]
def domain(c: B) = 0 until c.size
def fromPairs[C](ps: Seq[(Int, C)]) = ps.sortBy(_._1).map(_._2)
}
implicit def mapMapping[A, B, C <: Map[A, B]] = new Mapping[A, B, C] {
type M[D] = Map[A, D]
def domain(c: C) = c.keys.toSeq
def fromPairs[D](ps: Seq[(A, D)]) = ps.toMap
}
def transpose[A, B, C, M, N](m: M)(implicit
pev: M <:< PartialFunction[A, N],
qev: N <:< PartialFunction[B, C],
mev: Mapping[A, N, M],
nev: Mapping[B, C, N]
) = nev.fromPairs(nev.domain(mev.codomain(m).head).map(b =>
b -> mev.fromPairs(mev.toPairs(m).map { case (a, c) => a -> c(b) })
))
And now for some tests:
scala> println(transpose(List(Map("a" -> 1, "b" -> 13), Map("b" -> 99, "a" -> 14))))
Map(a -> Vector(1, 14), b -> Vector(13, 99))
scala> println(transpose(Map('a' -> List(1, 2, 3), 'z' -> List(4, 5, 6))))
Vector(Map(a -> 1, z -> 4), Map(a -> 2, z -> 5), Map(a -> 3, z -> 6))
scala> println(transpose(Map("x" -> Map(4 -> 'a, 99 -> 'z), "y" -> Map(4 -> 'b, 99 -> 's))))
Map(4 -> Map(x -> 'a, y -> 'b), 99 -> Map(x -> 'z, y -> 's))
So it's working as desired.
I have a List
val l : List[Map[String,Any]] = List(Map("a" -> 1, "b" -> 2.8), Map("a" -> 3, "c" -> 4), Map("c" -> 5, "d" -> "abc"))
and I used the following code to find the sum for the keys "a" (Int), "b" (Double) and "c" (Int). "d" is included as noise.
l.map(n => n.mapValues( v => if (v.isInstanceOf[Number]) {v match {
case x:Int => x.asInstanceOf[Int]
case x:Double => x.asInstanceOf[Double]
}} else 0)).foldLeft((0,0.0,0))((t, m) => (
t._1 + m.get("a").getOrElse(0),
t._2 + m.get("b").getOrElse(0.0),
t._3 + m.get("c").getOrElse(0)))
I expect the output would be (4, 2.8, 9) but instead I was trashed with
<console>:10: error: overloaded method value + with alternatives:
(x: Int)Int <and>
(x: Char)Int <and>
(x: Short)Int <and>
(x: Byte)Int
cannot be applied to (AnyVal)
I think the exception was trying to tell me that '+' doesn't work with AnyVal. How do I get this to work to get my the result that I want? Thanks
m.foldLeft(0)(_+_._2)
it's a very clear and simple solution.
reference: http://ktuman.blogspot.com/2009/10/how-to-simply-sum-values-in-map-in.html
You can use foldLeft function:
scala> val l : List[Map[String,Any]] = List(Map("a" -> 1, "b" -> 2.8), Map("a" -> 3, "c" -> 4), Map("c" -> 5, "d" -> "abc"))
l: List[Map[String,Any]] = List(Map(a -> 1, b -> 2.8), Map(a -> 3, c -> 4), Map(c -> 5, d -> abc))
scala> val (sa, sb, sc) = l.foldLeft((0: Int, 0: Double, 0: Int)){
| case ((a, b, c), m) => (
| a + m.get("a").collect{case i: Int => i}.getOrElse(0),
| b + m.get("b").collect{case i: Double => i}.getOrElse(0.),
| c + m.get("c").collect{case i: Int => i}.getOrElse(0)
| )
| }
sa: Int = 4
sb: Double = 2.8
sc: Int = 9
Updated using incrop's idea of collect instead of match.
First, you totally miss the point of pattern matching
{case i: Int => i
case d: Double => d
case _ => 0}
is the proper replacement of all your function inside mapValues. Yet this is not the problem, your writing, while complex, does the same thing.
Your function in mapValues returns Double (because some branches return Int and others return Double, and in this case, Int is promoted to Double. If it were not, it would return AnyVal).
So you get a List[Map[String, Double]]. At this point, you have lost the Ints.
When you do m.get("a"), this returns Option[Double]. Option[A] has method getOrElse(default: A) : A (actually, default: => X) but it makes no difference here).
If you call getOrElse(0.0) instead of getOrElse(0), you get a Double. Your code still fails, because your fold start with (Int, Double, Double), and you would return (Double, Double, Double). If you start your fold with (0.0, 0.0, 0.0), it works, but you have lost your Ints, you get (4.0, 2.8, 9.0)
Now, about the error message. You pass an Int to a method expecting a Double (getOrElse), the Int should normally be converted to Double, and it would be as if you called with getOrElse(0.0). Except that Option is covariant (declared trait Option[+A]). if X is an ancestor of A, an Option[A] is also an Option[X]. So an Option[Double] is also Option[AnyVal] and Option[Any]. The call getOrElse(0) works if the option is considered an Option[AnyVal], and the result is AnyVal (would work with Any too, but AnyVal is more precise and this is the one the compiler chooses). Because the expression compiles as is, there is no need to promote the 0 to 0.0. Thus m.get("a").getOrElse(0) is of type AnyVal, which cannot be added to t._1. This is what your error message says.
You have knowledge that "a" is associated with Int, "b" with double, but you don't pass this knowledge to the compiler.
A nifty one-liner:
l.map(_.filterKeys(_ != "d")).flatten groupBy(_._1) map { case (k,v) => v map { case (k2,v2: Number) => v2.doubleValue} sum }
res0: scala.collection.immutable.Iterable[Double] = List(9.0, 4.0, 2.8)
In general, if you don't know the keys, but just want to sum values you can do
val filtered = for {
map <- l
(k, v) <- map
if v.isInstanceOf[Number]
} yield k -> v.asInstanceOf[Number].doubleValue
val summed = filtered.groupBy(_._1) map { case (k, v) => k -> v.map(_._2).sum }
scala> l
res1: List[Map[String,Any]] = List(Map(a -> 1, b -> 2.8), Map(a -> 3, c -> 4), Map(c -> 5, d -> abc))
scala> filtered
res2: List[(String, Double)] = List((a,1.0), (b,2.8), (a,3.0), (c,4.0), (c,5.0))
scala> summed
res3: Map[String,Double] = Map(c -> 9.0, a -> 4.0, b -> 2.8)
Update
You can filter map by type you want, for example
scala> val intMap = for (x <- l) yield x collect { case (k, v: Int) => k -> v }
intMap: List[scala.collection.immutable.Map[String,Int]] = List(Map(a -> 1), Map(a -> 3, c -> 4), Map(c -> 5))
and then sum values (see linked question)
scala> intMap reduce { _ |+| _ }
res4: scala.collection.immutable.Map[String,Int] = Map(a -> 4, c -> 9)
Am I missing something or can you not just do:
map.values.sum
?
What is the best way to turn a Map[A, Set[B]] into a Map[B, Set[A]]?
For example, how do I turn a
Map(1 -> Set("a", "b"),
2 -> Set("b", "c"),
3 -> Set("c", "d"))
into a
Map("a" -> Set(1),
"b" -> Set(1, 2),
"c" -> Set(2, 3),
"d" -> Set(3))
(I'm using immutable collections only here. And my real problem has nothing to do with strings or integers. :)
with help from aioobe and Moritz:
def reverse[A, B](m: Map[A, Set[B]]) =
m.values.toSet.flatten.map(v => (v, m.keys.filter(m(_)(v)))).toMap
It's a bit more readable if you explicitly call contains:
def reverse[A, B](m: Map[A, Set[B]]) =
m.values.toSet.flatten.map(v => (v, m.keys.filter(m(_).contains(v)))).toMap
Best I've come up with so far is
val intToStrs = Map(1 -> Set("a", "b"),
2 -> Set("b", "c"),
3 -> Set("c", "d"))
def mappingFor(key: String) =
intToStrs.keys.filter(intToStrs(_) contains key).toSet
val newKeys = intToStrs.values.flatten
val inverseMap = newKeys.map(newKey => (newKey -> mappingFor(newKey))).toMap
Or another one using folds:
def reverse2[A,B](m:Map[A,Set[B]])=
m.foldLeft(Map[B,Set[A]]()){case (r,(k,s)) =>
s.foldLeft(r){case (r,e)=>
r + (e -> (r.getOrElse(e, Set()) + k))
}
}
Here's a one statement solution
orginalMap
.map{case (k, v)=>value.map{v2=>(v2,k)}}
.flatten
.groupBy{_._1}
.transform {(k, v)=>v.unzip._2.toSet}
This bit rather neatly (*) produces the tuples needed to construct the reverse map
Map(1 -> Set("a", "b"),
2 -> Set("b", "c"),
3 -> Set("c", "d"))
.map{case (k, v)=>v.map{v2=>(v2,k)}}.flatten
produces
List((a,1), (b,1), (b,2), (c,2), (c,3), (d,3))
Converting it directly to a map overwrites the values corresponding to duplicate keys though
Adding .groupBy{_._1} gets this
Map(c -> List((c,2), (c,3)),
a -> List((a,1)),
d -> List((d,3)),
b -> List((b,1), (b,2)))
which is closer. To turn those lists into Sets of the second half of the pairs.
.transform {(k, v)=>v.unzip._2.toSet}
gives
Map(c -> Set(2, 3), a -> Set(1), d -> Set(3), b -> Set(1, 2))
QED :)
(*) YMMV
A simple, but maybe not super-elegant solution:
def reverse[A,B](m:Map[A,Set[B]])={
var r = Map[B,Set[A]]()
m.keySet foreach { k=>
m(k) foreach { e =>
r = r + (e -> (r.getOrElse(e, Set()) + k))
}
}
r
}
The easiest way I can think of is:
// unfold values to tuples (v,k)
// for all values v in the Set referenced by key k
def vk = for {
(k,vs) <- m.iterator
v <- vs.iterator
} yield (v -> k)
// fold iterator back into a map
(Map[String,Set[Int]]() /: vk) {
// alternative syntax: vk.foldLeft(Map[String,Set[Int]]()) {
case (m,(k,v)) if m contains k =>
// Map already contains a Set, so just add the value
m updated (k, m(k) + v)
case (m,(k,v)) =>
// key not in the map - wrap value in a Set and return updated map
m updated (k, Set(v))
}