Convert List to Map in Akka Stream - scala

Question

I want to convert list items to single map as a stage in my Akka Streams workflow. As an example, say I had the following class.
case class MyClass(myString: String, myInt: Int)
I want to convert a List of MyClass instances to a Map that keys them by myString.
So if I had List(MyClass("hello", 1), MyClass("world", 2), MyClass("hello", 3)), I would want a map of hello mapping to List(1, 3) and world mapping to List(2).
The following is what I have so far.
val flowIWant = {
Flow[MyClass].map { entry =>
entry.myString -> entry.myInt
} ??? // How to combine tuples into a single map?
}
Also, it would be ideal for the flow to end up producing the individual map entities so I can work with each individually for the next stage (I want to do an operation on each map entity individual).
I am not sure if this a fold type operation or what. Thanks for any help.

Answer 1

It is not really clear what you actually want to get. From the way you stated your problem, I see at least the following transformations you could have meant:
Flow[List[MyClass], Map[String, Int], _]
Flow[List[MyClass], Map[String, List[Int]], _]
Flow[MyClass, (String, Int), _]
Flow[MyClass, (String, List[Int]), _]
From your wording I suspect that most likely you want something like the last one, but it doesn't really make sense to have such a transformation, because it won't be able to emit anything - in order to combine all values corresponding to a key you need to read the entire input.
If you have an incoming stream of MyClass and want to get a Map[String, List[Int]] from it, then there is no other choice than to attach it to a folding sink and execute the stream until completion. For example:
val source: Source[MyClass, _] = ??? // your source of MyClass instances
val collect: Sink[MyClass, Future[Map[String, List[Int]]] =
Sink.fold[Map[String, List[Int]], MyClass](Map.empty.withDefaultValue(List.empty)) {
(m, v) => m + (v.myString -> (v.myInt :: m(v.myString)))
}
val result: Future[Map[String, List[Int]]] = source.toMat(collect)(Keep.right).run()

Answer 2

I think you want to scan it:
source.scan((Map.empty[String, Int], None: Option((String, Int))))((acc, next) => { val (map, _)
val newMap = map.updated(next._1 -> map.getOrElse(next._1, List()))
(newMap, Some(newMap.get(next._1)))}).map(_._2.get)
This way you can check the contents of the Map till the memory is exhausted. (The content related to the last element is in the value part of the initial tuple wrapped in an Option.)

Answer 3

This may be what you are looking for :
import akka.actor.ActorSystem
import akka.stream.ActorMaterializer
import akka.stream.scaladsl.{Sink, Source}
import scala.util.{Failure, Success}
object Stack {
def main(args: Array[String]): Unit = {
case class MyClass(myString: String, myInt: Int)
implicit val actorSystem = ActorSystem("app")
implicit val actorMaterializer = ActorMaterializer()
import scala.concurrent.ExecutionContext.Implicits.global
val list = List(MyClass("hello", 1), MyClass("world", 2), MyClass("hello", 3))
val eventualMap = Source(list).fold(Map[String, List[Int]]())((m, e) => {
val newValue = e.myInt :: m.get(e.myString).getOrElse(Nil)
m + (e.myString -> newValue)
}).runWith(Sink.head)
eventualMap.onComplete{
case Success(m) => {
println(m)
actorSystem.terminate()
}
case Failure(e) => {
e.printStackTrace()
actorSystem.terminate()
}
}
}
}
With this code, you'll get the following output :
Map(hello -> List(3, 1), world -> List(2))
If you would like to have the following output :
Vector(Map(), Map(hello -> List(1)), Map(hello -> List(1), world -> List(2)), Map(hello -> List(3, 1), world -> List(2)))
Just use scan instead of fold and run with Sink.seq.
The difference between fold and scan is that fold wait for the upstream to complete before pushing down, whereas scan push every updates to downstream.