Writing custom Enumeratee with Scala and Play2 - scala

Question

I having hard time understanding Iteratee/Enumeratee/Enumerator concept. Looks like I understood how to create custom Iteratee - there are some good examples like that.
Now I'm going to write my custom Enumeratee. I start digging code for that, there is not so much comments there but a lot of fold(), fold0(), foldM(), joinI(). I understood that Enumeratee is really something made of Iteratee with sauce, but I still can't catch conception of writing my own. So, if somebody will help me with that example task it will give right direction. Lets consider such example:
val stringEnumerator = Enumerator("abc", "def,ghi", "jkl,mnopqrstuvwxyz")
val myEnumeratee: Enumeratee[String, Int] = ... // ???
val lengthEnumerator: Enumerator[Int] = stringEnumerator through myEnumeratee // should be equal to Enumerator(6, 6, 14)
myEnumeratee should resample stream by splitting given character flow by comma and returning length of each chunk ("abc" + "def" length is 6, "ghi" + "jkl" length is 6 and so on). How to write it?
P.S.
There is an Iteratee I've wrote for counting length of each chunk and eventually return List[Int]. Maybe it will help.

Answer 1

The fancy thing you are trying to do here is repartition the characters not according to their preexisting iteratee Input boundaries but by the comma boundaries. After that it is as simple as composing Enumeratee.map{_.length}. Here is your example using the scala interpreter in paste mode. You can see that result1 at the bottom is the repartitioned strings and result2 is just the count of each.
scala> :paste
// Entering paste mode (ctrl-D to finish)
import play.api.libs.iteratee._
import scala.concurrent.ExecutionContext.Implicits.global
import scala.concurrent.Await
import scala.concurrent.duration._
def repartitionStrings: Enumeratee[String, String] = {
Enumeratee.grouped[String](Traversable.splitOnceAt[String, Char](c => c != ',') transform Iteratee.consume())
}
val stringEnumerator = Enumerator("abc", "def,ghi", "jkl,mnopqrstuvwxyz")
val repartitionedEnumerator: Enumerator[String] = stringEnumerator.through(repartitionStrings)
val lengthEnumerator: Enumerator[Int] = stringEnumerator.through(repartitionStrings).through(Enumeratee.map{_.length}) // should be equal to Enumerator(6, 6, 14)
val result1 = Await.result(repartitionedEnumerator.run(Iteratee.getChunks[String]), 200 milliseconds)
val result2 = Await.result(lengthEnumerator.run(Iteratee.getChunks[Int]), 200 milliseconds)
// Exiting paste mode, now interpreting.
import play.api.libs.iteratee._
import scala.concurrent.ExecutionContext.Implicits.global
import scala.concurrent.Await
import scala.concurrent.duration._
repartitionStrings: play.api.libs.iteratee.Enumeratee[String,String]
stringEnumerator: play.api.libs.iteratee.Enumerator[String] = play.api.libs.iteratee.Enumerator$$anon$19#77e8800b
repartitionedEnumerator: play.api.libs.iteratee.Enumerator[String] = play.api.libs.iteratee.Enumerator$$anon$3#73216e8d
lengthEnumerator: play.api.libs.iteratee.Enumerator[Int] = play.api.libs.iteratee.Enumerator$$anon$3#2046e423
result1: List[String] = List(abcdef, ghijkl, mnopqrstuvwxyz)
result2: List[Int] = List(6, 6, 14)
Enumeratee.grouped is a powerful method that will group together traversable things (Seq, String, ...) according to a small internal custom Iteratee that you define. This iteratee should consume all the elements from the stream and produce the element that will be in the first element in the outer enumeratee and then will be rerun on the remaining input when it is time for the second outer element, and so on. We achieve this by using the special helper method Enumeratee.splitOnceAt which does precisely what we are looking for, we just need to compose it with a simple iteratee to concatenate all of these chunks together into the string that will be returned at the end (Iteratee.consume).