I want to merge two maps in fastest possible way by grouping by key and returning sum of values
val m1: Map[String, Long]
val m2: Map[String, Long]
currently, I do:
(m1.toSeq ++ m2.toSeq).groupBy(_._1).mapValues(_.map(_._2).sum)
What's faster approach? Would HashMap be faster?
val m3 = m1.foldLeft(m2) { case (accMap, (key, value)) =>
val accValue = accMap.getOrElse(key, 0)
accMap + (key -> (value + accValue))
}
If you have cats in scope, all you need is this:
import cats.syntax.all._
val combined = m1 |+| m2
Related
I have a function which return a Future[Seq[A]]
def foo(x:String): Future[Seq[A]]
Then how can I convert it into a Map[String, Seq[A]] ?
This is What I have tried
foo(x).map { e =>
(ResponseHeader.Ok, e.groupBy(_.use).mapValues(???))
}
*EDIT :
What I want to achieve is to group by my Seq based on 1 of its column as the key, and convert it into a Map[key, Seq[A]].
I tried to group by it, but I dont know what to put inside the mapValues
The mapValues call is not required, the groupBy will give you what you want:
val e: Seq[A] = ???
val map: Map[String, Seq[A]] = e.groupBy(_.use)
val res: Future[(Int, Map[String, Seq[A]])] =
foo("").map { e =>
(ResponseHeader.Ok, e.groupBy(_.use))
}
I am using Gatling and want to use feeders. Apparently, feeders require a List[Map[String, String]]. I have a list of tuples like
{("key", "abcde"),("key", "bcdef")...}
and I want to convert it into
{ Map("key", "abcde"), Map("key", "bcdef")...}
How can I do that? I am new to Scala programming and would appreciate any help here.
Try this:
val tupleList = List(("key", "abcde"),("key", "bcdef"))
val mapList = tupleList map { case (k,v) => Map(k-> v)}
try this:
val x: List[(String, String)] = List(("key", "abcde"),("key", "bcdef"))
val y: List[Map[String, String]] = x.map(p => Map(p._1 -> p._2))
since you are creating new Map for every tuple so duplicate key should not be an issue as suggested in comments
I want to convert list items to single map as a stage in my Akka Streams workflow. As an example, say I had the following class.
case class MyClass(myString: String, myInt: Int)
I want to convert a List of MyClass instances to a Map that keys them by myString.
So if I had List(MyClass("hello", 1), MyClass("world", 2), MyClass("hello", 3)), I would want a map of hello mapping to List(1, 3) and world mapping to List(2).
The following is what I have so far.
val flowIWant = {
Flow[MyClass].map { entry =>
entry.myString -> entry.myInt
} ??? // How to combine tuples into a single map?
}
Also, it would be ideal for the flow to end up producing the individual map entities so I can work with each individually for the next stage (I want to do an operation on each map entity individual).
I am not sure if this a fold type operation or what. Thanks for any help.
It is not really clear what you actually want to get. From the way you stated your problem, I see at least the following transformations you could have meant:
Flow[List[MyClass], Map[String, Int], _]
Flow[List[MyClass], Map[String, List[Int]], _]
Flow[MyClass, (String, Int), _]
Flow[MyClass, (String, List[Int]), _]
From your wording I suspect that most likely you want something like the last one, but it doesn't really make sense to have such a transformation, because it won't be able to emit anything - in order to combine all values corresponding to a key you need to read the entire input.
If you have an incoming stream of MyClass and want to get a Map[String, List[Int]] from it, then there is no other choice than to attach it to a folding sink and execute the stream until completion. For example:
val source: Source[MyClass, _] = ??? // your source of MyClass instances
val collect: Sink[MyClass, Future[Map[String, List[Int]]] =
Sink.fold[Map[String, List[Int]], MyClass](Map.empty.withDefaultValue(List.empty)) {
(m, v) => m + (v.myString -> (v.myInt :: m(v.myString)))
}
val result: Future[Map[String, List[Int]]] = source.toMat(collect)(Keep.right).run()
I think you want to scan it:
source.scan((Map.empty[String, Int], None: Option((String, Int))))((acc, next) => { val (map, _)
val newMap = map.updated(next._1 -> map.getOrElse(next._1, List()))
(newMap, Some(newMap.get(next._1)))}).map(_._2.get)
This way you can check the contents of the Map till the memory is exhausted. (The content related to the last element is in the value part of the initial tuple wrapped in an Option.)
This may be what you are looking for :
import akka.actor.ActorSystem
import akka.stream.ActorMaterializer
import akka.stream.scaladsl.{Sink, Source}
import scala.util.{Failure, Success}
object Stack {
def main(args: Array[String]): Unit = {
case class MyClass(myString: String, myInt: Int)
implicit val actorSystem = ActorSystem("app")
implicit val actorMaterializer = ActorMaterializer()
import scala.concurrent.ExecutionContext.Implicits.global
val list = List(MyClass("hello", 1), MyClass("world", 2), MyClass("hello", 3))
val eventualMap = Source(list).fold(Map[String, List[Int]]())((m, e) => {
val newValue = e.myInt :: m.get(e.myString).getOrElse(Nil)
m + (e.myString -> newValue)
}).runWith(Sink.head)
eventualMap.onComplete{
case Success(m) => {
println(m)
actorSystem.terminate()
}
case Failure(e) => {
e.printStackTrace()
actorSystem.terminate()
}
}
}
}
With this code, you'll get the following output :
Map(hello -> List(3, 1), world -> List(2))
If you would like to have the following output :
Vector(Map(), Map(hello -> List(1)), Map(hello -> List(1), world -> List(2)), Map(hello -> List(3, 1), world -> List(2)))
Just use scan instead of fold and run with Sink.seq.
The difference between fold and scan is that fold wait for the upstream to complete before pushing down, whereas scan push every updates to downstream.
Imagine you have a Map[Option[Int], String] and you want to have a Map[Int, String] discarding the entry which contain None as the key.
Another example, that should be somehow similar is List[(Option[Int], String)] and transform it to List[(Int, String)], again discarding the tuple which contain None as the first element.
What's the best approach?
collect is your friend here:
example data definition
val data = Map(Some(1) -> "data", None -> "")
solution for Map
scala> data collect { case ( Some(i), s) => (i,s) }
res4: scala.collection.immutable.Map[Int,String] = Map(1 -> data)
the same approach works for a list of tuples
scala> data.toList collect { case ( Some(i), s) => (i,s) }
res5: List[(Int, String)] = List((1,data))
Say I have been given a list of futures with each one linked to an key such as:
val seq: Seq[(Key, Future[Value])]
And my goal is to produce a list of key value tuples once all futures have completed:
val complete: Seq[(Key, Value)]
I am wondering if this can be achieved using a sequence call. For example I know I can do the following:
val complete = Future.sequence(seq.map(_._2).onComplete {
case Success(s) => s
case Failure(NonFatal(e)) => Seq()
}
But this will only returns me a sequence of Value objects and I lose the pairing information between Key and Value. The problem being that Future.sequence expects a sequence of Futures.
How could I augment this to maintain the key/value pairing in my complete sequence?
Thanks
Des
How about transforming your Seq[(Key, Future[Value])] to Seq[Future[(Key, Value)]] first.
val seq: Seq[(Key, Future[Value])] = // however your implementation is
val futurePair: Seq[Future[(Key, Value)]] = for {
(key, value) <- seq
} yield value.map(v => (key, v))
Now you can use sequence to get Future[Seq[(Key, Value)]].
val complete: Future[Seq[(String, Int)]] = Future.sequence(futurePair)
Just a different expression of the other answer, using unzip and zip.
scala> val vs = Seq(("one",Future(1)),("two",Future(2)))
vs: Seq[(String, scala.concurrent.Future[Int])] = List((one,scala.concurrent.impl.Promise$DefaultPromise#4e38d975), (two,scala.concurrent.impl.Promise$DefaultPromise#35f8a9d3))
scala> val (ks, fs) = vs.unzip
ks: Seq[String] = List(one, two)
fs: Seq[scala.concurrent.Future[Int]] = List(scala.concurrent.impl.Promise$DefaultPromise#4e38d975, scala.concurrent.impl.Promise$DefaultPromise#35f8a9d3)
scala> val done = (Future sequence fs) map (ks zip _)
done: scala.concurrent.Future[Seq[(String, Int)]] = scala.concurrent.impl.Promise$DefaultPromise#56913163
scala> done.value
res0: Option[scala.util.Try[Seq[(String, Int)]]] = Some(Success(List((one,1), (two,2))))
or maybe save on zippage:
scala> val done = (Future sequence fs) map ((ks, _).zipped)
done: scala.concurrent.Future[scala.runtime.Tuple2Zipped[String,Seq[String],Int,Seq[Int]]] = scala.concurrent.impl.Promise$DefaultPromise#766a52f5
scala> done.value.get.get.toList
res1: List[(String, Int)] = List((one,1), (two,2))