Related
I am trying to flatten a map where the keys are traversables, in the sense that:
Map( Set(1, 2, 3) -> 'A', Set(4, 5, 6) -> 'B')
should flatten to:
Map(5 -> B, 1 -> A, 6 -> B, 2 -> A, 3 -> A, 4 -> B)
Here is what I did:
def fuse[A, B, T <: Traversable[A]](mapOfTravs: Map[T, B]): Map[A, B] = {
val pairs = for {
trav <- mapOfTravs.keys
key <- trav
} yield (key, mapOfTravs(trav))
pairs.toMap
}
It works. But:
Is there a simpler way to do this?
I'm not very comfortable with the Scala type system and I'm sure this can be improved. I have to specify the types explicitly whenever I use my function:
val map2 = Map( Set(1, 2, 3) -> 'A', Set(4, 5, 6) -> 'B')
val fused2 = fuse[Int, Char, Set[Int]](map2)
val map1: Map[Traversable[Int], Char] = Map( Set(1, 2, 3) -> 'A', Set(4, 5, 6) -> 'B')
val fused1 = fuse[Int, Char, Traversable[Int]](map1)
P.S.: this fuse function does not make much sense when the key traversables have a non-null intersection.
This is basically what you're doing in the for comprehension, but simplified a little bit:
def fuse[A, B, T <: Traversable[A]](mapOfTravs: Map[T, B]): Map[A, B] = {
mapOfTravs.flatMap({ case (s, c) => s.map(i => i -> c) })
}
Not much you can do about the types, I'm sure there's some type lambda shenanigans that you can do, I'm just not sure how to do them...
UPDATE
Here's a slightly better for version, same as the above flatMap:
def fuse2[A, B, T <: Traversable[A]](mapOfTravs: Map[T, B]): Map[A, B] = {
for {
(keys, value) <- mapOfTravs
key <- keys
} yield key -> value
}
Like #Azzie, I was thinking zip, but maybe Azzie has the advantage with those zees.
scala> val m = Map( Set(1, 2, 3) -> 'A', Set(4, 5, 6) -> 'B')
m: scala.collection.immutable.Map[scala.collection.immutable.Set[Int],Char] = Map(Set(1, 2, 3) -> A, Set(4, 5, 6) -> B)
scala> (m map { case (k, v) => k zip (Stream continually v) }).flatten.toMap
res0: scala.collection.immutable.Map[Int,Char] = Map(5 -> B, 1 -> A, 6 -> B, 2 -> A, 3 -> A, 4 -> B)
scala> (m map { case (k, v) => k zipAll (Nil, null, v) }).flatten.toMap
res1: scala.collection.immutable.Map[Any,Char] = Map(5 -> B, 1 -> A, 6 -> B, 2 -> A, 3 -> A, 4 -> B)
scala> m flatMap { case (k, v) => k zip (Stream continually v) }
res2: scala.collection.immutable.Map[Int,Char] = Map(5 -> B, 1 -> A, 6 -> B, 2 -> A, 3 -> A, 4 -> B)
It's not obvious how to generalize it nicely.
This looks horrible and using 0 is kind of cheating but it does the job
m.map( {case (s,c) => s.zipAll(Set(c),0,c)} ).flatten.toMap
Since I'm apparently on a "hideously generic implicits" kick lately:
import scala.collection.MapLike
import scala.collection.TraversableLike
import scala.collection.generic.CanBuildFrom
implicit class Map_[
A,
B,
T1 : ({type L[X] = X => TraversableLike[A, T2]})#L,
T2,
M1 : ({type L[X] = X => MapLike[T1, B, M2]})#L,
M2 <: MapLike[T1, B, M2] with Map[T1, B]
](map: M1) {
def fuse[M3](implicit cbfM: CanBuildFrom[M2, (A, B), M3]) : M3 =
map.flatMap({ case (k, v) => k.toTraversable.map((_, v)) })
}
Examples:
scala> Map(Set(1, 2, 3) -> 'A', Set(4, 5, 6) -> 'B').fuse
res: scala.collection.immutable.Map[Int,Char] =
Map(5 -> B, 1 -> A, 6 -> B, 2 -> A, 3 -> A, 4 -> B)
scala> Map(Array(1, 2, 3) -> 'A', Array(4, 5, 6) -> 'B').fuse
res: scala.collection.immutable.Map[Int,Char] =
Map(5 -> B, 1 -> A, 6 -> B, 2 -> A, 3 -> A, 4 -> B)
I have to often transpose a "rectangular" collection-of-collections in Scala, e.g.: a list of maps, a map of lists, a map of maps, a set of lists, a map of sets etc. Since collections can be uniformly viewed as a mapping from a specific domain to a co-domain (e.g.: a List[A]/Array[A] is a mapping from the Int domain to the A co-domain, Set[A]is a mapping from the A domain to the Boolean co-domain etc.), I'd like to write a clean, generic function to do a transpose operation (e.g.: turn a map of lists to the transposed list of maps). However, I'm having trouble because other than the () operator, Scala doesn't seem to have a unified API to view collections abstractly as mappings ?
So I end up writing a separate transpose for each type of collection-of-collections as follows:
def transposeMapOfLists[A,B]( mapOfLists: Map[A,List[B]] ) : List[Map[A,B]] = {
val k = ( mapOfLists keys ) toList
val l = ( k map { mapOfLists(_) } ) transpose;
l map { v => ( k zip v ) toMap }
}
def transposeListOfMaps[A,B]( listOfMaps: List[Map[A,B]]) : Map[A,List[B]] = {
val k = ( listOfMaps(0) keys ) toList
val l = ( listOfMaps map { m => k map { m(_) } } ) transpose;
( k zip l ) toMap
}
def transposeMapOfMaps[A,B,C]( mapOfMaps: Map[A,Map[B,C]] ) : Map[B,Map[A,C]] = {
val k = ( mapOfMaps keys ) toList
val listOfMaps = k map { mapOfMaps(_) }
val mapOfLists = transposeListOfMaps( listOfMaps )
mapOfLists map { p => ( p._1, ( k zip p._2 ) toMap ) }
}
Can someone help me unify these methods into one generic collection-of-collections transpose ? It will also help me (and I am sure others) learn some useful Scala features in the process.
ps: I have ignored exception handling and have assumed the input collection-of-collections is rectangular, i.e., all of the inner collections' domain elements constitute the same set.
I'm sure the following messy version using type classes could be cleaned up a lot, but it works as a quick proof-of-concept. I don't see an easy way to get the return types right without dependent method types (I'm sure it's possible), so you'll have to use -Xexperimental:
trait Mapping[A, B, C] {
type M[D] <: PartialFunction[A, D]
def domain(c: C): Seq[A]
def fromPairs[D](ps: Seq[(A, D)]): M[D]
def codomain(c: C)(implicit ev: C <:< PartialFunction[A, B]) =
domain(c).map(c)
def toPairs(c: C)(implicit ev: C <:< PartialFunction[A, B]) =
domain(c).map(a => (a, c(a)))
}
implicit def seqMapping[A, B <: Seq[A]] = new Mapping[Int, A, B] {
type M[C] = Seq[C]
def domain(c: B) = 0 until c.size
def fromPairs[C](ps: Seq[(Int, C)]) = ps.sortBy(_._1).map(_._2)
}
implicit def mapMapping[A, B, C <: Map[A, B]] = new Mapping[A, B, C] {
type M[D] = Map[A, D]
def domain(c: C) = c.keys.toSeq
def fromPairs[D](ps: Seq[(A, D)]) = ps.toMap
}
def transpose[A, B, C, M, N](m: M)(implicit
pev: M <:< PartialFunction[A, N],
qev: N <:< PartialFunction[B, C],
mev: Mapping[A, N, M],
nev: Mapping[B, C, N]
) = nev.fromPairs(nev.domain(mev.codomain(m).head).map(b =>
b -> mev.fromPairs(mev.toPairs(m).map { case (a, c) => a -> c(b) })
))
And now for some tests:
scala> println(transpose(List(Map("a" -> 1, "b" -> 13), Map("b" -> 99, "a" -> 14))))
Map(a -> Vector(1, 14), b -> Vector(13, 99))
scala> println(transpose(Map('a' -> List(1, 2, 3), 'z' -> List(4, 5, 6))))
Vector(Map(a -> 1, z -> 4), Map(a -> 2, z -> 5), Map(a -> 3, z -> 6))
scala> println(transpose(Map("x" -> Map(4 -> 'a, 99 -> 'z), "y" -> Map(4 -> 'b, 99 -> 's))))
Map(4 -> Map(x -> 'a, y -> 'b), 99 -> Map(x -> 'z, y -> 's))
So it's working as desired.
I have a List
val l : List[Map[String,Any]] = List(Map("a" -> 1, "b" -> 2.8), Map("a" -> 3, "c" -> 4), Map("c" -> 5, "d" -> "abc"))
and I used the following code to find the sum for the keys "a" (Int), "b" (Double) and "c" (Int). "d" is included as noise.
l.map(n => n.mapValues( v => if (v.isInstanceOf[Number]) {v match {
case x:Int => x.asInstanceOf[Int]
case x:Double => x.asInstanceOf[Double]
}} else 0)).foldLeft((0,0.0,0))((t, m) => (
t._1 + m.get("a").getOrElse(0),
t._2 + m.get("b").getOrElse(0.0),
t._3 + m.get("c").getOrElse(0)))
I expect the output would be (4, 2.8, 9) but instead I was trashed with
<console>:10: error: overloaded method value + with alternatives:
(x: Int)Int <and>
(x: Char)Int <and>
(x: Short)Int <and>
(x: Byte)Int
cannot be applied to (AnyVal)
I think the exception was trying to tell me that '+' doesn't work with AnyVal. How do I get this to work to get my the result that I want? Thanks
m.foldLeft(0)(_+_._2)
it's a very clear and simple solution.
reference: http://ktuman.blogspot.com/2009/10/how-to-simply-sum-values-in-map-in.html
You can use foldLeft function:
scala> val l : List[Map[String,Any]] = List(Map("a" -> 1, "b" -> 2.8), Map("a" -> 3, "c" -> 4), Map("c" -> 5, "d" -> "abc"))
l: List[Map[String,Any]] = List(Map(a -> 1, b -> 2.8), Map(a -> 3, c -> 4), Map(c -> 5, d -> abc))
scala> val (sa, sb, sc) = l.foldLeft((0: Int, 0: Double, 0: Int)){
| case ((a, b, c), m) => (
| a + m.get("a").collect{case i: Int => i}.getOrElse(0),
| b + m.get("b").collect{case i: Double => i}.getOrElse(0.),
| c + m.get("c").collect{case i: Int => i}.getOrElse(0)
| )
| }
sa: Int = 4
sb: Double = 2.8
sc: Int = 9
Updated using incrop's idea of collect instead of match.
First, you totally miss the point of pattern matching
{case i: Int => i
case d: Double => d
case _ => 0}
is the proper replacement of all your function inside mapValues. Yet this is not the problem, your writing, while complex, does the same thing.
Your function in mapValues returns Double (because some branches return Int and others return Double, and in this case, Int is promoted to Double. If it were not, it would return AnyVal).
So you get a List[Map[String, Double]]. At this point, you have lost the Ints.
When you do m.get("a"), this returns Option[Double]. Option[A] has method getOrElse(default: A) : A (actually, default: => X) but it makes no difference here).
If you call getOrElse(0.0) instead of getOrElse(0), you get a Double. Your code still fails, because your fold start with (Int, Double, Double), and you would return (Double, Double, Double). If you start your fold with (0.0, 0.0, 0.0), it works, but you have lost your Ints, you get (4.0, 2.8, 9.0)
Now, about the error message. You pass an Int to a method expecting a Double (getOrElse), the Int should normally be converted to Double, and it would be as if you called with getOrElse(0.0). Except that Option is covariant (declared trait Option[+A]). if X is an ancestor of A, an Option[A] is also an Option[X]. So an Option[Double] is also Option[AnyVal] and Option[Any]. The call getOrElse(0) works if the option is considered an Option[AnyVal], and the result is AnyVal (would work with Any too, but AnyVal is more precise and this is the one the compiler chooses). Because the expression compiles as is, there is no need to promote the 0 to 0.0. Thus m.get("a").getOrElse(0) is of type AnyVal, which cannot be added to t._1. This is what your error message says.
You have knowledge that "a" is associated with Int, "b" with double, but you don't pass this knowledge to the compiler.
A nifty one-liner:
l.map(_.filterKeys(_ != "d")).flatten groupBy(_._1) map { case (k,v) => v map { case (k2,v2: Number) => v2.doubleValue} sum }
res0: scala.collection.immutable.Iterable[Double] = List(9.0, 4.0, 2.8)
In general, if you don't know the keys, but just want to sum values you can do
val filtered = for {
map <- l
(k, v) <- map
if v.isInstanceOf[Number]
} yield k -> v.asInstanceOf[Number].doubleValue
val summed = filtered.groupBy(_._1) map { case (k, v) => k -> v.map(_._2).sum }
scala> l
res1: List[Map[String,Any]] = List(Map(a -> 1, b -> 2.8), Map(a -> 3, c -> 4), Map(c -> 5, d -> abc))
scala> filtered
res2: List[(String, Double)] = List((a,1.0), (b,2.8), (a,3.0), (c,4.0), (c,5.0))
scala> summed
res3: Map[String,Double] = Map(c -> 9.0, a -> 4.0, b -> 2.8)
Update
You can filter map by type you want, for example
scala> val intMap = for (x <- l) yield x collect { case (k, v: Int) => k -> v }
intMap: List[scala.collection.immutable.Map[String,Int]] = List(Map(a -> 1), Map(a -> 3, c -> 4), Map(c -> 5))
and then sum values (see linked question)
scala> intMap reduce { _ |+| _ }
res4: scala.collection.immutable.Map[String,Int] = Map(a -> 4, c -> 9)
Am I missing something or can you not just do:
map.values.sum
?
Inside a function of mine I construct a result set by filling a new mutable HashMap with data (if there is a better way - I'd appreciate comments). Then I'd like to return the result set as an immutable HashMap. How to derive an immutable from a mutable?
Discussion about returning immutable.Map vs. immutable.HashMap notwithstanding, what about simply using the toMap method:
scala> val m = collection.mutable.HashMap(1 -> 2, 3 -> 4)
m: scala.collection.mutable.HashMap[Int,Int] = Map(3 -> 4, 1 -> 2)
scala> m.toMap
res22: scala.collection.immutable.Map[Int,Int] = Map(3 -> 4, 1 -> 2)
As of 2.9, this uses the method toMap in TraversableOnce, which is implemented as follows:
def toMap[T, U](implicit ev: A <:< (T, U)): immutable.Map[T, U] = {
val b = immutable.Map.newBuilder[T, U]
for (x <- self)
b += x
b.result
}
scala> val m = collection.mutable.HashMap(1->2,3->4)
m: scala.collection.mutable.HashMap[Int,Int] = Map(3 -> 4, 1 -> 2)
scala> collection.immutable.HashMap() ++ m
res1: scala.collection.immutable.Map[Int,Int] = Map(1 -> 2, 3 -> 4)
or
scala> collection.immutable.HashMap(m.toSeq:_*)
res2: scala.collection.immutable.HashMap[Int,Int] = Map(1 -> 2, 3 -> 4)
If you have a map : logMap: Map[String, String]
just need to do : logMap.toMap()
I want to achieve something like the following:
(_ : Map[K,Int]).mapKey(k, _ + 1)
And the mapKey function applies its second argument (Int => Int) only to the value stored under k. Is there something inside the standard lib? If not I bet there's something in Scalaz.
Of course I can write this function myself (m.updated(k,f(m(k))) and its simple to do so. But I've come over this problem several times, so maybe its already done?
For Scalaz I imagine something along the following code:
(m: Map[A,B]).project(k: A).map(f: B => B): Map[A,B]
You could of course add
def changeForKey[A,B](a: A, fun: B => B): Tuple2[A, B] => Tuple2[A, B] = { kv =>
kv match {
case (`a`, b) => (a, fun(b))
case x => x
}
}
val theMap = Map('a -> 1, 'b -> 2)
theMap map changeForKey('a, (_: Int) + 1)
res0: scala.collection.immutable.Map[Symbol,Int] = Map('a -> 2, 'b -> 2)
But this would circumvent any optimisation regarding memory re-use and access.
I came also up with a rather verbose and inefficient scalaz solution using a zipper for your proposed project method:
theMap.toStream.toZipper.flatMap(_.findZ(_._1 == 'a).flatMap(elem => elem.delete.map(_.insert((elem.focus._1, fun(elem.focus._2)))))).map(_.toStream.toMap)
or
(for {
z <- theMap.toStream.toZipper
elem <- z.findZ(_._1 == 'a)
z2 <- elem.delete
} yield z2.insert((elem.focus._1, fun(elem.focus._2)))).map(_.toStream.toMap)
Probably of little use. I’m just posting for reference.
Here is one way:
scala> val m = Map(2 -> 3, 5 -> 11)
m: scala.collection.immutable.Map[Int,Int] = Map(2 -> 3, 5 -> 11)
scala> m ++ (2, m.get(2).map(1 +)).sequence
res53: scala.collection.immutable.Map[Int,Int] = Map(2 -> 4, 5 -> 11)
scala> m ++ (9, m.get(9).map(1 +)).sequence
res54: scala.collection.immutable.Map[Int,Int] = Map(2 -> 3, 5 -> 11)
This works because (A, Option[B]).sequence gives Option[(A, B)]. (sequence in general turns types inside out. i.e. F[G[A]] => [G[F[A]], given F : Traverse and G : Applicative.)
You can pimp it with this so that it creates a new map based on the old one:
class MapUtils[A, B](map: Map[A, B]) {
def mapValueAt(a: A)(f: (B) => B) = map.get(a) match {
case Some(b) => map + (a -> f(b))
case None => map
}
}
implicit def toMapUtils[A, B](map: Map[A, B]) = new MapUtils(map)
val m = Map(1 -> 1)
m.mapValueAt(1)(_ + 1)
// Map(1 -> 2)
m.mapValueAt(2)(_ + 1)
// Map(1 -> 1)