Generic stream builder with shapeless HLists - scala

Question

I have a simple builder which accepts a Source and Sink's from Akka streams and on compile time verifies that a method executed on those has types matching source and sinks.
class EventProcessorTask(config: EventProcessorConfig =
EventProcessorConfig()) {
def source[In, MatIn](source: Source[In, MatIn]): SourcedTask[In, MatIn] = new SourcedTask[In, MatIn](source, config)
}
class SourcedTask[In, MatIn](source: Source[In, MatIn], config: EventProcessorConfig) {
def withPartitioning[Id](partitioningF: In => Id): SourcedTaskWithPartitioning[In, MatIn, Id] =
new SourcedTaskWithPartitioning[In, MatIn, Id](source, partitioningF, config)
}
class SourcedTaskWithPartitioning[In, MatIn, Id](source: Source[In, MatIn], partitioningF: In => Id, config: EventProcessorConfig) {
def withSink[Out, T](sink: Sink[Out, T]): WiredTask[In, MatIn, Out :: HNil, Id, Sink[Out, T] :: HNil] =
new WiredTask[In, MatIn, Out :: HNil, Id, Sink[Out, T] :: HNil](source, sink :: HNil, partitioningF, config)
}
class WiredTask[In, MatIn, L <: HList, Id, SinksTypes <: HList](
source: Source[In, MatIn],
sinks: SinksTypes,
partitioningF: In => Id,
config: EventProcessorConfig
) {
def withSink[Out, T](sink: Sink[Out, T]): WiredTask[In, MatIn, Out :: L, Id, Sink[Out, T] :: SinksTypes] =
new WiredTask[In, MatIn, Out :: L, Id, Sink[Out, T] :: SinksTypes](
source, sink :: sinks, partitioningF, config
)
def execute[N <: Nat, P <: Product, F, R <: HList, SinksRev <: HList]
( executionMethod: In => Future[P])(
implicit generic: Generic.Aux[P, R],
rev: Reverse.Aux[L, R],
sinksRev: Reverse.Aux[SinksTypes, SinksRev],
executionContext: ExecutionContext,
l: Length.Aux[SinksRev, N]
): Unit = {
val sinksReversed = sinksRev(sinks)
// val sinksLength= sinksReversed.length.toInt
}
}
The code above compiles but when I try to build a Broadcast for the Sinks I can't even get the size of the list (commented out code). The next step would be to match all Sinks in SinksRev to corresponding type from P which would allow me to send messages produced by executionMethod which returns a tuple to Sinks corresponding to P types.
I.e.
new EventProcessorTask()
.source(Source.single("str"))
.withPartitioning(r => 1)
.withSink(Sink.head[Long])
.withSink(Sink.foreach((s: String) => ()))
.execute(
in => Future.successful((null.asInstanceOf[Long], null.asInstanceOf[String]))
)
Long should go to first Sink and String to second one.
Any help would be much appreciated. I might be doing something very wrong in here but the concept seemed nice at the time I started working on this (now not so much). Either way I would like to understand what I am missing in here.
To sum up, the questions are:
1. Why can't I get Int representation of SinksRev size?
2. How to match Sinks from SinksRev to corresponding elements in P to build a Broadcast based GraphShape?

Answer 1

I'm into shapeless and akka streams recently myself, so I decided to give this question a try. Your case seems pretty complicated, so I took what I understood, simplified it a bit a came out with code that seems to do something similar to what you probably want. I still cannot calculate the length but since it's a builder then += 1 could suffice.
Here's the result with Options instead of sinks. It takes list of Options and applies function to the content of options. As I mentioned, I simplified the case.
import shapeless._
object Shapes extends App {
import ops.function._
import syntax.std.function._
case class Thing[Types <: HList, Out] private(sources: List[Option[_]]) {
def withOption[T](o: Option[T]) = Thing[T :: Types, Out](o :: sources)
def withOutput[T] = Thing[Types, T](sources)
def apply[F](f: F)(implicit fp: FnToProduct.Aux[F, Types => Out]) = {
val a: Types = sources.foldLeft[HList](HNil)((m, v) ⇒ v.get :: m ).asInstanceOf[Types]
f.toProduct(a)
}
}
object Thing {
def withOption[T](o: Option[T]) = Thing[T :: HNil, AnyVal](o :: Nil)
}
val r = Thing
.withOption(Some(1))
.withOption(Some(2))
.withOption(Some(3))
.withOutput[Unit]
.apply {
(x: Int, y: Int, z: Int) ⇒ println(x + y + z)
}
println(r)
}