akka streams stops after parallelism - scala

Question

I tried to build a small PDF parser with akka-streams (and limited understandings of it yet) and Apache's pdfbox.
One thing which I don't really get, the stream stops exactly after the given number of parallelism which is given into the mapAsync.
so if a PDF-doc has 20 pages and the parallelism is set to 5, the first 5 pages get processed and the rest is ignored, if set to 20, everything is done fine. Anybody an idea what I'm doing wrong?
class PdfParser(ws: WSClient, conf: Configuration, parallelism: Int) {
implicit val system = ActorSystem("image-parser")
implicit val materializer = ActorMaterializer()
def documentPages(doc: PDDocument, key: String): Iterator[Page] = {
val pages: util.List[_] = doc.getDocumentCatalog.getAllPages
val pageList = (for {
i ← 0 until pages.size()
page = pages.get(i)
} yield Page(page, s"$key-$i.jpg")).toIterator
pageList
}
val pageToImage: Flow[Page, Image, NotUsed] = Flow[Page].map { p ⇒
val img = p.content.asInstanceOf[PDPage].convertToImage()
Image(img, p.name)
}
val imageToS3: Flow[Image, String, NotUsed] = Flow[Image].mapAsync(parallelism) { i ⇒
val s3 = S3.fromConfiguration(ws, conf)
val bucket = s3.getBucket("elsa-essays")
val baos = new ByteArrayOutputStream()
ImageIO.write(i.content, "jpg", baos)
val res = bucket add BucketFile(i.name, "image/jpeg", baos.toByteArray)
res.map { _ ⇒
"uploaded"
}.recover {
case e: S3Exception ⇒ e.message
}
}
val sink: Sink[String, Future[String]] = Sink.head[String]
def parse(path: Path, key: String): Future[String] = {
val stream: InputStream = new FileInputStream(path.toString)
val doc = PDDocument.load(stream)
val source = Source.fromIterator(() ⇒ documentPages(doc, key))
val runnable: RunnableGraph[Future[String]] = source.via(pageToImage).via(imageToS3).toMat(sink)(Keep.right)
val res = runnable.run()
res.map { s ⇒
doc.close()
stream.close()
s
}
}
}

Answer 1

The problem is in your Sink. That Sink.head will return one element from your materialized Stream. So the question is, why it´s received more than one value when mapAsync(>1) is used in stream materialization?. Maybe it´s because it uses more than one actor pushing values downstream.
In any case, change your sink to something like:
val sink: Sink[String, Future[String]] = Sink.fold("")((a, b) => b ++ a)
and it will work.