Related
I am still on studying period when it comes to scala and faces some problems that I would like to solve.
What I have at the moment is a Seq of items type X. Now I want to make a function that returns me a map of numbers mapped with set of items that appear on that original seq certain amount of time.
Here is small example what I want to do:
val exampleSeq[X]: Seq = [a, b, d, d, c, b, d]
val exampleSeq2[x]: Seq = [a, a, a, c, c, b, b, c]
myMagicalFunction(exampleSeq) returns Map[1 -> Set[a, c], 2 -> Set[b], 3 -> Set[d]]
myMagicalFunction(exampleSeq2) returns Map[2 -> Set[b], 3 -> Set[a, c]]
So far I have been able to create a function that maps the item with the times it appears:
function[X](seq: Seq[X]) = seq.groupBy(item => item).mapValues(_.size)
Return for my exampleSeq from that one is
Map(a -> 1, b -> 2, c -> 1, d -> 3)
Thank you for answers :)
One approach, for
val a = Seq('a', 'b', 'd', 'd', 'c', 'b', 'd')
this
val b = for ( (k,v) <- a.groupBy(identity).mapValues(_.size).toArray )
yield (v,k)
delivers
Array((2,b), (3,d), (1,a), (1,c))
and so
b.groupBy(_._1).mapValues(_.map(_._2).toSet)
res: Map(2 -> Set(b), 1 -> Set(a, c), 3 -> Set(d))
Note seq.groupBy(item => item) is equivalent to seq.groupBy(identity).
You are almost there! Departing from the collection element -> count, you only need a transformation to get to count -> Col[elem].
Lets say that freqItem = Map(a -> 1, b -> 2, c -> 1, d -> 3) you would do something like:
val freqSet = freqItem.toSeq.map(_.swap).groupBy(_._1).mapValues(_.toSet)
Note that we transform the Map into a Seq before swapping the (k,v) into (v,k) because mapping over a Map preserves the semantics of key uniqueness and you'd lose one of (1 -> a), (1 -> b) otherwise.
You can write your function as :
def f[T](l: Seq[T]): Map[Int, Set[T]] = {
l.map {
x => (x, l.count(_ == x))
}.distinct.groupBy(_._2).mapValues(_.map(_._1).toSet)
}
val l = List("a","a","a","b","b","b","b","c","c","d","e")
f(l)
res0: Map[Int,Set[String]] = Map(2 -> Set(c), 4 -> Set(b), 1 -> Set(d, e), 3 -> Set(a))
scala> case class A(name:String,age:Int)
defined class A
scala> val l = List(new A("a",1),new A("b",2),new A("a",1),new A("c",1) )
l: List[A] = List(A(a,1), A(b,2), A(a,1), A(c,1))
scala> f[A](l)
res1: Map[Int,Set[A]] = Map(2 -> Set(A(a,1)), 1 -> Set(A(b,2), A(c,1)))
I am trying to flatten a map where the keys are traversables, in the sense that:
Map( Set(1, 2, 3) -> 'A', Set(4, 5, 6) -> 'B')
should flatten to:
Map(5 -> B, 1 -> A, 6 -> B, 2 -> A, 3 -> A, 4 -> B)
Here is what I did:
def fuse[A, B, T <: Traversable[A]](mapOfTravs: Map[T, B]): Map[A, B] = {
val pairs = for {
trav <- mapOfTravs.keys
key <- trav
} yield (key, mapOfTravs(trav))
pairs.toMap
}
It works. But:
Is there a simpler way to do this?
I'm not very comfortable with the Scala type system and I'm sure this can be improved. I have to specify the types explicitly whenever I use my function:
val map2 = Map( Set(1, 2, 3) -> 'A', Set(4, 5, 6) -> 'B')
val fused2 = fuse[Int, Char, Set[Int]](map2)
val map1: Map[Traversable[Int], Char] = Map( Set(1, 2, 3) -> 'A', Set(4, 5, 6) -> 'B')
val fused1 = fuse[Int, Char, Traversable[Int]](map1)
P.S.: this fuse function does not make much sense when the key traversables have a non-null intersection.
This is basically what you're doing in the for comprehension, but simplified a little bit:
def fuse[A, B, T <: Traversable[A]](mapOfTravs: Map[T, B]): Map[A, B] = {
mapOfTravs.flatMap({ case (s, c) => s.map(i => i -> c) })
}
Not much you can do about the types, I'm sure there's some type lambda shenanigans that you can do, I'm just not sure how to do them...
UPDATE
Here's a slightly better for version, same as the above flatMap:
def fuse2[A, B, T <: Traversable[A]](mapOfTravs: Map[T, B]): Map[A, B] = {
for {
(keys, value) <- mapOfTravs
key <- keys
} yield key -> value
}
Like #Azzie, I was thinking zip, but maybe Azzie has the advantage with those zees.
scala> val m = Map( Set(1, 2, 3) -> 'A', Set(4, 5, 6) -> 'B')
m: scala.collection.immutable.Map[scala.collection.immutable.Set[Int],Char] = Map(Set(1, 2, 3) -> A, Set(4, 5, 6) -> B)
scala> (m map { case (k, v) => k zip (Stream continually v) }).flatten.toMap
res0: scala.collection.immutable.Map[Int,Char] = Map(5 -> B, 1 -> A, 6 -> B, 2 -> A, 3 -> A, 4 -> B)
scala> (m map { case (k, v) => k zipAll (Nil, null, v) }).flatten.toMap
res1: scala.collection.immutable.Map[Any,Char] = Map(5 -> B, 1 -> A, 6 -> B, 2 -> A, 3 -> A, 4 -> B)
scala> m flatMap { case (k, v) => k zip (Stream continually v) }
res2: scala.collection.immutable.Map[Int,Char] = Map(5 -> B, 1 -> A, 6 -> B, 2 -> A, 3 -> A, 4 -> B)
It's not obvious how to generalize it nicely.
This looks horrible and using 0 is kind of cheating but it does the job
m.map( {case (s,c) => s.zipAll(Set(c),0,c)} ).flatten.toMap
Since I'm apparently on a "hideously generic implicits" kick lately:
import scala.collection.MapLike
import scala.collection.TraversableLike
import scala.collection.generic.CanBuildFrom
implicit class Map_[
A,
B,
T1 : ({type L[X] = X => TraversableLike[A, T2]})#L,
T2,
M1 : ({type L[X] = X => MapLike[T1, B, M2]})#L,
M2 <: MapLike[T1, B, M2] with Map[T1, B]
](map: M1) {
def fuse[M3](implicit cbfM: CanBuildFrom[M2, (A, B), M3]) : M3 =
map.flatMap({ case (k, v) => k.toTraversable.map((_, v)) })
}
Examples:
scala> Map(Set(1, 2, 3) -> 'A', Set(4, 5, 6) -> 'B').fuse
res: scala.collection.immutable.Map[Int,Char] =
Map(5 -> B, 1 -> A, 6 -> B, 2 -> A, 3 -> A, 4 -> B)
scala> Map(Array(1, 2, 3) -> 'A', Array(4, 5, 6) -> 'B').fuse
res: scala.collection.immutable.Map[Int,Char] =
Map(5 -> B, 1 -> A, 6 -> B, 2 -> A, 3 -> A, 4 -> B)
I have to often transpose a "rectangular" collection-of-collections in Scala, e.g.: a list of maps, a map of lists, a map of maps, a set of lists, a map of sets etc. Since collections can be uniformly viewed as a mapping from a specific domain to a co-domain (e.g.: a List[A]/Array[A] is a mapping from the Int domain to the A co-domain, Set[A]is a mapping from the A domain to the Boolean co-domain etc.), I'd like to write a clean, generic function to do a transpose operation (e.g.: turn a map of lists to the transposed list of maps). However, I'm having trouble because other than the () operator, Scala doesn't seem to have a unified API to view collections abstractly as mappings ?
So I end up writing a separate transpose for each type of collection-of-collections as follows:
def transposeMapOfLists[A,B]( mapOfLists: Map[A,List[B]] ) : List[Map[A,B]] = {
val k = ( mapOfLists keys ) toList
val l = ( k map { mapOfLists(_) } ) transpose;
l map { v => ( k zip v ) toMap }
}
def transposeListOfMaps[A,B]( listOfMaps: List[Map[A,B]]) : Map[A,List[B]] = {
val k = ( listOfMaps(0) keys ) toList
val l = ( listOfMaps map { m => k map { m(_) } } ) transpose;
( k zip l ) toMap
}
def transposeMapOfMaps[A,B,C]( mapOfMaps: Map[A,Map[B,C]] ) : Map[B,Map[A,C]] = {
val k = ( mapOfMaps keys ) toList
val listOfMaps = k map { mapOfMaps(_) }
val mapOfLists = transposeListOfMaps( listOfMaps )
mapOfLists map { p => ( p._1, ( k zip p._2 ) toMap ) }
}
Can someone help me unify these methods into one generic collection-of-collections transpose ? It will also help me (and I am sure others) learn some useful Scala features in the process.
ps: I have ignored exception handling and have assumed the input collection-of-collections is rectangular, i.e., all of the inner collections' domain elements constitute the same set.
I'm sure the following messy version using type classes could be cleaned up a lot, but it works as a quick proof-of-concept. I don't see an easy way to get the return types right without dependent method types (I'm sure it's possible), so you'll have to use -Xexperimental:
trait Mapping[A, B, C] {
type M[D] <: PartialFunction[A, D]
def domain(c: C): Seq[A]
def fromPairs[D](ps: Seq[(A, D)]): M[D]
def codomain(c: C)(implicit ev: C <:< PartialFunction[A, B]) =
domain(c).map(c)
def toPairs(c: C)(implicit ev: C <:< PartialFunction[A, B]) =
domain(c).map(a => (a, c(a)))
}
implicit def seqMapping[A, B <: Seq[A]] = new Mapping[Int, A, B] {
type M[C] = Seq[C]
def domain(c: B) = 0 until c.size
def fromPairs[C](ps: Seq[(Int, C)]) = ps.sortBy(_._1).map(_._2)
}
implicit def mapMapping[A, B, C <: Map[A, B]] = new Mapping[A, B, C] {
type M[D] = Map[A, D]
def domain(c: C) = c.keys.toSeq
def fromPairs[D](ps: Seq[(A, D)]) = ps.toMap
}
def transpose[A, B, C, M, N](m: M)(implicit
pev: M <:< PartialFunction[A, N],
qev: N <:< PartialFunction[B, C],
mev: Mapping[A, N, M],
nev: Mapping[B, C, N]
) = nev.fromPairs(nev.domain(mev.codomain(m).head).map(b =>
b -> mev.fromPairs(mev.toPairs(m).map { case (a, c) => a -> c(b) })
))
And now for some tests:
scala> println(transpose(List(Map("a" -> 1, "b" -> 13), Map("b" -> 99, "a" -> 14))))
Map(a -> Vector(1, 14), b -> Vector(13, 99))
scala> println(transpose(Map('a' -> List(1, 2, 3), 'z' -> List(4, 5, 6))))
Vector(Map(a -> 1, z -> 4), Map(a -> 2, z -> 5), Map(a -> 3, z -> 6))
scala> println(transpose(Map("x" -> Map(4 -> 'a, 99 -> 'z), "y" -> Map(4 -> 'b, 99 -> 's))))
Map(4 -> Map(x -> 'a, y -> 'b), 99 -> Map(x -> 'z, y -> 's))
So it's working as desired.
I have a List
val l : List[Map[String,Any]] = List(Map("a" -> 1, "b" -> 2.8), Map("a" -> 3, "c" -> 4), Map("c" -> 5, "d" -> "abc"))
and I used the following code to find the sum for the keys "a" (Int), "b" (Double) and "c" (Int). "d" is included as noise.
l.map(n => n.mapValues( v => if (v.isInstanceOf[Number]) {v match {
case x:Int => x.asInstanceOf[Int]
case x:Double => x.asInstanceOf[Double]
}} else 0)).foldLeft((0,0.0,0))((t, m) => (
t._1 + m.get("a").getOrElse(0),
t._2 + m.get("b").getOrElse(0.0),
t._3 + m.get("c").getOrElse(0)))
I expect the output would be (4, 2.8, 9) but instead I was trashed with
<console>:10: error: overloaded method value + with alternatives:
(x: Int)Int <and>
(x: Char)Int <and>
(x: Short)Int <and>
(x: Byte)Int
cannot be applied to (AnyVal)
I think the exception was trying to tell me that '+' doesn't work with AnyVal. How do I get this to work to get my the result that I want? Thanks
m.foldLeft(0)(_+_._2)
it's a very clear and simple solution.
reference: http://ktuman.blogspot.com/2009/10/how-to-simply-sum-values-in-map-in.html
You can use foldLeft function:
scala> val l : List[Map[String,Any]] = List(Map("a" -> 1, "b" -> 2.8), Map("a" -> 3, "c" -> 4), Map("c" -> 5, "d" -> "abc"))
l: List[Map[String,Any]] = List(Map(a -> 1, b -> 2.8), Map(a -> 3, c -> 4), Map(c -> 5, d -> abc))
scala> val (sa, sb, sc) = l.foldLeft((0: Int, 0: Double, 0: Int)){
| case ((a, b, c), m) => (
| a + m.get("a").collect{case i: Int => i}.getOrElse(0),
| b + m.get("b").collect{case i: Double => i}.getOrElse(0.),
| c + m.get("c").collect{case i: Int => i}.getOrElse(0)
| )
| }
sa: Int = 4
sb: Double = 2.8
sc: Int = 9
Updated using incrop's idea of collect instead of match.
First, you totally miss the point of pattern matching
{case i: Int => i
case d: Double => d
case _ => 0}
is the proper replacement of all your function inside mapValues. Yet this is not the problem, your writing, while complex, does the same thing.
Your function in mapValues returns Double (because some branches return Int and others return Double, and in this case, Int is promoted to Double. If it were not, it would return AnyVal).
So you get a List[Map[String, Double]]. At this point, you have lost the Ints.
When you do m.get("a"), this returns Option[Double]. Option[A] has method getOrElse(default: A) : A (actually, default: => X) but it makes no difference here).
If you call getOrElse(0.0) instead of getOrElse(0), you get a Double. Your code still fails, because your fold start with (Int, Double, Double), and you would return (Double, Double, Double). If you start your fold with (0.0, 0.0, 0.0), it works, but you have lost your Ints, you get (4.0, 2.8, 9.0)
Now, about the error message. You pass an Int to a method expecting a Double (getOrElse), the Int should normally be converted to Double, and it would be as if you called with getOrElse(0.0). Except that Option is covariant (declared trait Option[+A]). if X is an ancestor of A, an Option[A] is also an Option[X]. So an Option[Double] is also Option[AnyVal] and Option[Any]. The call getOrElse(0) works if the option is considered an Option[AnyVal], and the result is AnyVal (would work with Any too, but AnyVal is more precise and this is the one the compiler chooses). Because the expression compiles as is, there is no need to promote the 0 to 0.0. Thus m.get("a").getOrElse(0) is of type AnyVal, which cannot be added to t._1. This is what your error message says.
You have knowledge that "a" is associated with Int, "b" with double, but you don't pass this knowledge to the compiler.
A nifty one-liner:
l.map(_.filterKeys(_ != "d")).flatten groupBy(_._1) map { case (k,v) => v map { case (k2,v2: Number) => v2.doubleValue} sum }
res0: scala.collection.immutable.Iterable[Double] = List(9.0, 4.0, 2.8)
In general, if you don't know the keys, but just want to sum values you can do
val filtered = for {
map <- l
(k, v) <- map
if v.isInstanceOf[Number]
} yield k -> v.asInstanceOf[Number].doubleValue
val summed = filtered.groupBy(_._1) map { case (k, v) => k -> v.map(_._2).sum }
scala> l
res1: List[Map[String,Any]] = List(Map(a -> 1, b -> 2.8), Map(a -> 3, c -> 4), Map(c -> 5, d -> abc))
scala> filtered
res2: List[(String, Double)] = List((a,1.0), (b,2.8), (a,3.0), (c,4.0), (c,5.0))
scala> summed
res3: Map[String,Double] = Map(c -> 9.0, a -> 4.0, b -> 2.8)
Update
You can filter map by type you want, for example
scala> val intMap = for (x <- l) yield x collect { case (k, v: Int) => k -> v }
intMap: List[scala.collection.immutable.Map[String,Int]] = List(Map(a -> 1), Map(a -> 3, c -> 4), Map(c -> 5))
and then sum values (see linked question)
scala> intMap reduce { _ |+| _ }
res4: scala.collection.immutable.Map[String,Int] = Map(a -> 4, c -> 9)
Am I missing something or can you not just do:
map.values.sum
?
I want to achieve something like the following:
(_ : Map[K,Int]).mapKey(k, _ + 1)
And the mapKey function applies its second argument (Int => Int) only to the value stored under k. Is there something inside the standard lib? If not I bet there's something in Scalaz.
Of course I can write this function myself (m.updated(k,f(m(k))) and its simple to do so. But I've come over this problem several times, so maybe its already done?
For Scalaz I imagine something along the following code:
(m: Map[A,B]).project(k: A).map(f: B => B): Map[A,B]
You could of course add
def changeForKey[A,B](a: A, fun: B => B): Tuple2[A, B] => Tuple2[A, B] = { kv =>
kv match {
case (`a`, b) => (a, fun(b))
case x => x
}
}
val theMap = Map('a -> 1, 'b -> 2)
theMap map changeForKey('a, (_: Int) + 1)
res0: scala.collection.immutable.Map[Symbol,Int] = Map('a -> 2, 'b -> 2)
But this would circumvent any optimisation regarding memory re-use and access.
I came also up with a rather verbose and inefficient scalaz solution using a zipper for your proposed project method:
theMap.toStream.toZipper.flatMap(_.findZ(_._1 == 'a).flatMap(elem => elem.delete.map(_.insert((elem.focus._1, fun(elem.focus._2)))))).map(_.toStream.toMap)
or
(for {
z <- theMap.toStream.toZipper
elem <- z.findZ(_._1 == 'a)
z2 <- elem.delete
} yield z2.insert((elem.focus._1, fun(elem.focus._2)))).map(_.toStream.toMap)
Probably of little use. I’m just posting for reference.
Here is one way:
scala> val m = Map(2 -> 3, 5 -> 11)
m: scala.collection.immutable.Map[Int,Int] = Map(2 -> 3, 5 -> 11)
scala> m ++ (2, m.get(2).map(1 +)).sequence
res53: scala.collection.immutable.Map[Int,Int] = Map(2 -> 4, 5 -> 11)
scala> m ++ (9, m.get(9).map(1 +)).sequence
res54: scala.collection.immutable.Map[Int,Int] = Map(2 -> 3, 5 -> 11)
This works because (A, Option[B]).sequence gives Option[(A, B)]. (sequence in general turns types inside out. i.e. F[G[A]] => [G[F[A]], given F : Traverse and G : Applicative.)
You can pimp it with this so that it creates a new map based on the old one:
class MapUtils[A, B](map: Map[A, B]) {
def mapValueAt(a: A)(f: (B) => B) = map.get(a) match {
case Some(b) => map + (a -> f(b))
case None => map
}
}
implicit def toMapUtils[A, B](map: Map[A, B]) = new MapUtils(map)
val m = Map(1 -> 1)
m.mapValueAt(1)(_ + 1)
// Map(1 -> 2)
m.mapValueAt(2)(_ + 1)
// Map(1 -> 1)