writing generic function using method name - scala

I am trying to write the following method:
case class Config2(repoName: String)
def buildAction[A, M, R <: HList]()
(implicit
gen: Generic.Aux[Config2, R],
mod: Modifier.Aux[R, M, A, A, R])
: (A, Config2) => Config2 = {
(arg: A, c: Config2) => {
val rec = mod.apply(gen.to(c), _ => arg)
gen.from(rec)
}
}
When trying to use it with:
buildAction[String, Witness.`'repoName`.T, String :: HList]()
I get an error:
could not find implicit value for parameter gen: shapeless.Generic.Aux[com.advancedtelematic.tuf.cli.Cli.Config2,shapeless.::[String,shapeless.HList]]
[error] val _ = buildAction[String, Witness.`'repoName`.T, String :: HList]()
am I missing some import here?
Second question is, can I somehow rewrite this signature so I don't don't have to specify all the types? In practive the Config2 type takes a long list of fields so it's not pratical to write this all the time
Update:
I simplified this to the following:
val CGen = LabelledGeneric[Config]
def buildAction[A, M]()
(implicit mod: Modifier.Aux[CGen.Repr, M, A, A, CGen.Repr])
: (A, Config) => Config = {
(arg: A, c: Config) => {
val rec = mod.apply(CGen.to(c), _ => arg)
CGen.from(rec)
}
}
Which allows me to just write:
buildAction[String, Witness.`'repoName`.T]()
But I still have to specify that Witness. Is the a way I could write buildAction[String]("repoName") and have some method provide the Witness implicitly?
Update: the following works!
val CGen = LabelledGeneric[Config]
def buildAction[A](witness: Witness)
(implicit mod: Modifier.Aux[CGen.Repr, witness.T, A, A, CGen.Repr]):
(A, Config) => Config = {
(arg: A, c: Config) => {
val rec = mod.apply(CGen.to(c), _ => arg)
CGen.from(rec)
}
}
buildAction[RepoName]('repoName)

am I missing some import here?
No, it's probably just that String :: HList must be String :: HNil
Second question is, can I somehow rewrite this signature so I don't don't have to specify all the types?
You can use a trick known as kinda-curried type parameters:
object buildAction {
class PartiallyApplied[A, M] {
def apply[R <: HList]()(implicit ...)
}
def apply[A, M] = new PartiallyApplied[A, M]
}
Used as
buildAction[String, Witness.`'foo`.T]()
Also, since your code mentions field name, you probably want LabelledGeneric in conjunction with ops.record.Updater

Related

Dynamically creating Akka Stream Flows at Runtime

I'm currently trying to dynamically create Akka Stream graph definitions at runtime. The idea being that users will be able to define flows interactively and attach them to existing/running BroadcastHubs. This means I don't know which flows or even how many flows will be used at compile time.
Unfortunately, I'm struggling with generics/type erasure. Frankly, I'm not even sure what I'm attempting to do is possible on the JVM.
I have a function that will return an Akka Streams Flow representing two connected Flows. It uses Scala's TypeTags to get around type erasure. If the output type of the first flow is the same as the input type of the second flow, it can be successfully connected. This works just fine.
import akka.NotUsed
import akka.stream.FlowShape
import akka.stream.scaladsl.GraphDSL.Implicits._
import akka.stream.scaladsl.{Flow, GraphDSL}
import scala.reflect.runtime.universe._
import scala.util.{Failure, Success, Try}
def connect[A: TypeTag, B: TypeTag, C: TypeTag, D: TypeTag](a: Flow[A, B, NotUsed],
b: Flow[C, D, NotUsed]): Try[Flow[A, D, NotUsed]] = {
Try {
if (typeOf[B] =:= typeOf[C]) {
val c = b.asInstanceOf[Flow[B, D, NotUsed]]
Flow.fromGraph {
GraphDSL.create(a, c)((m1, m2) => NotUsed.getInstance()) { implicit b =>
(s1, s2) =>
s1 ~> s2
FlowShape(s1.in, s2.out)
}
}
}
else
throw new RuntimeException(s"Connection failed. Incompatible types: ${typeOf[B]} and ${typeOf[C]}")
}
}
So If I have Flow[A,B] and Flow[C,D], the result would be Flow[A,D] assuming that B and C are the same type.
I also have function that attempts to merge/reduce a List of Flows down to a single Flow. Lets assume that this list is derived from a list of flow definitions from a file or web request.
def merge(fcs: List[Flow[_, _, NotUsed]]): Try[Option[Flow[_, _, NotUsed]]] = {
fcs match {
case Nil => Success(None)
case h :: Nil => Success(Some(h))
case h :: t =>
val n = t.head
connect(h, n) match {
case Success(fc) => merge(fc :: t)
case Failure(e) => Failure(e)
}
}
}
Unfortunately, since the Flows are stored inside a List, due to type erasure on standard Lists, I lose all of the type information and therefore am unable to connect the Flows at runtime. Here's an example:
def flowIdentity[A]() = Flow.fromFunction[A, A](x => x)
def flowI2S() = Flow.fromFunction[Int, String](_.toString)
val a = flowIdentity[Int]()
val b = flowIdentity[Int]()
val c = flowI2S()
val d = flowIdentity[String]()
val fcs: List[Flow[_, _, NotUsed]] = List(a, b, c, d)
val y = merge(fcs)
This results in the exception:
Failure(java.lang.RuntimeException: Connection failed. Incompatible types _$4 and _$3)
I've been looking into Miles Sabin'sShapeless, and thought I might be able to use HLists to retain type information. Unfortunately, that seems to work only if I know the individual types and length of the list at compile time. If I upcast a specific HList to just HList, it looks like I lose the type information again.
val fcs: HList = a :: b :: c :: d :: HNil
So my question is... is this even possible? Is there a way to do this with Shapeless generics magic (preferably without the need to use specific non-existential type extractors)? I'd like to find as generic a solution as possible, and any help would be appreciated.
Thanks!
I know this is an old post. As I had some time I gave it a try. Not sure this is exactly the solution, but I thought would post and get suggestions.
type FlowN[A, B] = Flow[A, B, NotUsed]
trait FlowMerger[L <: HList] {
type A
type D
def merge(flow: L): Option[FlowN[A, D]]
}
object FlowMerger extends LowPriorityImplicits {
def apply[L <: HList](v: L)(implicit ev: FlowMerger[L]): Option[FlowN[ev.A, ev.D]] = ev.merge(v)
type Aux[L <: HList, A1, D1] = FlowMerger[L] {
type A = A1
type D = D1
}
implicit def h1Instance[A1, D1]: FlowMerger.Aux[FlowN[A1, D1] :: HNil, A1, D1] = new FlowMerger[FlowN[A1, D1] :: HNil] {
override type A = A1
override type D = D1
override def merge(flow: FlowN[A1, D1] :: HNil): Option[FlowN[A, D]] = Option(flow.head)
}
}
trait LowPriorityImplicits {
implicit def hMulInstance[A1, B1, D1, E1, F1, L <: HList, T <: HList, T1 <: HList]
(implicit
isHC1: IsHCons.Aux[L, FlowN[A1, B1], T],
isHC2: IsHCons.Aux[T, FlowN[E1, F1], T1],
lx: Lazy[FlowMerger[T]],
typeableB: Lazy[Typeable[B1]],
typeableE: Lazy[Typeable[E1]]
): FlowMerger.Aux[L, A1, D1] = {
new FlowMerger[L] {
override type A = A1
override type D = D1
override def merge(flow: L): Option[FlowN[A, D]] = {
if (typeableB.value == typeableE.value) {
lx.value.merge(isHC1.tail(flow)).map(t => isHC1.head(flow) via t.asInstanceOf[FlowN[B1, D]])
} else None
}
}
}
}
You can use it as:
FlowMerger(fcs).map(flow => Source(List(1, 2, 3)) via flow runForeach println)
As you already noticed, the reason it didn't work was that the list erases the types you had. Therefore it is impossible.
If you know all of the types that can be used as intermediate types, you can solve that by adding a resolving function. Adding such a function will also simplify your connect method. I'll add a code snippet. I hope it will be clear.
def flowIdentity[A]() = Flow.fromFunction[A, A](x => x)
def flowI2S() = Flow.fromFunction[Int, String](_.toString)
def main(args: Array[String]): Unit = {
val idInt1 = flowIdentity[Int]()
val idInt2 = flowIdentity[Int]()
val int2String = flowI2S()
val idString = flowIdentity[String]()
val fcs = List(idInt1, idInt2, int2String, idString)
val source = Source(1 to 10)
val mergedGraph = merge(fcs).get.asInstanceOf[Flow[Int, String, NotUsed]]
source.via(mergedGraph).to(Sink.foreach(println)).run()
}
def merge(fcs: List[Flow[_, _, NotUsed]]): Option[Flow[_, _, NotUsed]] = {
fcs match {
case Nil => None
case h :: Nil => Some(h)
case h :: t =>
val n = t.head
val fc = resolveConnect(h, n)
merge(fc :: t.tail)
}
}
def resolveConnect(a: Flow[_, _, NotUsed], b: Flow[_, _, NotUsed]): Flow[_, _, NotUsed] = {
if (a.isInstanceOf[Flow[_, Int, NotUsed]] && b.isInstanceOf[Flow[Int, _, NotUsed]]) {
connectInt(a.asInstanceOf[Flow[_, Int, NotUsed]], b.asInstanceOf[Flow[Int, _, NotUsed]])
} else if (a.isInstanceOf[Flow[_, String, NotUsed]] && b.isInstanceOf[Flow[String, _, NotUsed]]) {
connectString(a.asInstanceOf[Flow[_, String, NotUsed]], b.asInstanceOf[Flow[String, _, NotUsed]])
} else {
throw new UnsupportedOperationException
}
}
def connectInt(a: Flow[_, Int, NotUsed], b: Flow[Int, _, NotUsed]): Flow[_, _, NotUsed] = {
a.via(b)
}
def connectString(a: Flow[_, String, NotUsed], b: Flow[String, _, NotUsed]): Flow[_, _, NotUsed] = {
a.via(b)
}
p.s
There is another bug hiding there, of an endless loop. When calling the merge recursion, the first element should be dropped, as it was already merged into the main flow.

type does not take parameters Scala

I have case class State and want to extend Variable class from it; but in Variable class only need value in constructor. Where can I put run function?
case class State[S, +A](run: S => (A, S)) {
//.....has `map` function
def map[B, X >: State[S, B]](f: A => B): X =
State(state => {
val (a, s2) = run(state)
(f(a), s2)
})
}
class Variable[+A](value: A) extends State[A, A] { // ERROR
def get: Variable[A] =
map(x => x)
def set(newValue: A): Variable[A] =
map(_ => newValue)
}
UPDATE
I've changed to something like this:
class Variable[+A](value: A, run: A => (A, A)) extends State[A, A](run) {
def get: Variable[A] =
map(x => x) // ERROR HERE
def set(newValue: A): State[A, A] =
map(_ => newValue)
}
object Variable {
def create[A](value: A): Variable[A] = new Variable[A](value, x => (x, x))
}
But I've gotten error:
type mismatch; found : com.libs.State[A,A] required: com.libs.Variable[A] Variable.scala /scala/src/com/libs line 4 Scala Problem
The problem is that you cannot define a Variable using map, since map defines a State which is only a super-type of Variable. How can your program know how to set the additional information of your subclass, using only map?
However, if you define type Variable[+A] = State[A, A], so that it is not a subclass but an alias for the same class, you will have some variance errors, since State is invariant in its type parameter S, so Variable must be too.

Scala type mismatch using multiple generic

I'm trying to use Scala to prototype the functionality of lazy endless list for my lambda calculus classes. The public constructor takes two arguments and should create LazyList[A,A].
class LazyList[A,B] private (private val first: A, private val mapper: A => B, private val successor: A => A) {
def this(first: A, successor: A => A) = this(first, (e: A) => e, successor)
def head: B = mapper(first)
def tail(): LazyList[A,B] = new LazyList(successor(first), mapper, successor)
def map[R](func: B => R) = new LazyList[A, R](first, func.compose(mapper), successor)
def at(index: Long): B = if (index == 0L) head else tail().at(index - 1)
def sublist(end: Long): List[B] = if (end == 0L) List(head) else head :: tail().sublist(end - 1)
override def toString = s"LazyList($first, $mapper, $successor)"
}
But the code compilation fails with error.
Error:(20, 65) type mismatch;
found : e.type (with underlying type A)
required: B
def this(first: A, successor: A => A) = this(first, (e: A) => e, successor)
^
What actually am I doing wrong?
Parameterized signatures inside the class don't have any info about the relation of type B to type A, and thus the compiler generally thinks that everywhere inside the LazyList's body B is not A, that's why it compiler complains when you try to assign A => A to A => B.
You should create an alternative constructor not as this(), but as a factory method in companion object. Note the A of this usage is a parameter which is not in any way related to the A inside LazyList's body:
object LazyList {
def apply[A](first: A, successor: A => A): LazyList[A, A] = new LazyList(first, identity, successor)
}

Generic stream builder with shapeless HLists

I have a simple builder which accepts a Source and Sink's from Akka streams and on compile time verifies that a method executed on those has types matching source and sinks.
class EventProcessorTask(config: EventProcessorConfig =
EventProcessorConfig()) {
def source[In, MatIn](source: Source[In, MatIn]): SourcedTask[In, MatIn] = new SourcedTask[In, MatIn](source, config)
}
class SourcedTask[In, MatIn](source: Source[In, MatIn], config: EventProcessorConfig) {
def withPartitioning[Id](partitioningF: In => Id): SourcedTaskWithPartitioning[In, MatIn, Id] =
new SourcedTaskWithPartitioning[In, MatIn, Id](source, partitioningF, config)
}
class SourcedTaskWithPartitioning[In, MatIn, Id](source: Source[In, MatIn], partitioningF: In => Id, config: EventProcessorConfig) {
def withSink[Out, T](sink: Sink[Out, T]): WiredTask[In, MatIn, Out :: HNil, Id, Sink[Out, T] :: HNil] =
new WiredTask[In, MatIn, Out :: HNil, Id, Sink[Out, T] :: HNil](source, sink :: HNil, partitioningF, config)
}
class WiredTask[In, MatIn, L <: HList, Id, SinksTypes <: HList](
source: Source[In, MatIn],
sinks: SinksTypes,
partitioningF: In => Id,
config: EventProcessorConfig
) {
def withSink[Out, T](sink: Sink[Out, T]): WiredTask[In, MatIn, Out :: L, Id, Sink[Out, T] :: SinksTypes] =
new WiredTask[In, MatIn, Out :: L, Id, Sink[Out, T] :: SinksTypes](
source, sink :: sinks, partitioningF, config
)
def execute[N <: Nat, P <: Product, F, R <: HList, SinksRev <: HList]
( executionMethod: In => Future[P])(
implicit generic: Generic.Aux[P, R],
rev: Reverse.Aux[L, R],
sinksRev: Reverse.Aux[SinksTypes, SinksRev],
executionContext: ExecutionContext,
l: Length.Aux[SinksRev, N]
): Unit = {
val sinksReversed = sinksRev(sinks)
// val sinksLength= sinksReversed.length.toInt
}
}
The code above compiles but when I try to build a Broadcast for the Sinks I can't even get the size of the list (commented out code). The next step would be to match all Sinks in SinksRev to corresponding type from P which would allow me to send messages produced by executionMethod which returns a tuple to Sinks corresponding to P types.
I.e.
new EventProcessorTask()
.source(Source.single("str"))
.withPartitioning(r => 1)
.withSink(Sink.head[Long])
.withSink(Sink.foreach((s: String) => ()))
.execute(
in => Future.successful((null.asInstanceOf[Long], null.asInstanceOf[String]))
)
Long should go to first Sink and String to second one.
Any help would be much appreciated. I might be doing something very wrong in here but the concept seemed nice at the time I started working on this (now not so much). Either way I would like to understand what I am missing in here.
To sum up, the questions are:
1. Why can't I get Int representation of SinksRev size?
2. How to match Sinks from SinksRev to corresponding elements in P to build a Broadcast based GraphShape?
I'm into shapeless and akka streams recently myself, so I decided to give this question a try. Your case seems pretty complicated, so I took what I understood, simplified it a bit a came out with code that seems to do something similar to what you probably want. I still cannot calculate the length but since it's a builder then += 1 could suffice.
Here's the result with Options instead of sinks. It takes list of Options and applies function to the content of options. As I mentioned, I simplified the case.
import shapeless._
object Shapes extends App {
import ops.function._
import syntax.std.function._
case class Thing[Types <: HList, Out] private(sources: List[Option[_]]) {
def withOption[T](o: Option[T]) = Thing[T :: Types, Out](o :: sources)
def withOutput[T] = Thing[Types, T](sources)
def apply[F](f: F)(implicit fp: FnToProduct.Aux[F, Types => Out]) = {
val a: Types = sources.foldLeft[HList](HNil)((m, v) ⇒ v.get :: m ).asInstanceOf[Types]
f.toProduct(a)
}
}
object Thing {
def withOption[T](o: Option[T]) = Thing[T :: HNil, AnyVal](o :: Nil)
}
val r = Thing
.withOption(Some(1))
.withOption(Some(2))
.withOption(Some(3))
.withOutput[Unit]
.apply {
(x: Int, y: Int, z: Int) ⇒ println(x + y + z)
}
println(r)
}

Replace Asynchronous Anonymous Class with Function

I am thinking this may be possible with the shapeless library.
I'm using the shapeless to convert an anonymous class to a closure. This requires the use of hlisted from the FnHListerAux trait.
All I want to do is get rid of the dummy function passed in and return a closure around this function which has the same type signature as F. This would be easy if there wasn't an asynchronously executed anonymous class. Is there a way to get around this?
def async[F, A <: HList, R](
shell: Shell,
success: F,
failure: FunctionTypes.Failure,
dummy: F)(implicit h: FnHListerAux[F, A => R],
u: FnUnHListerAux[A => R, F]): F =
{ (args: A) =>
require(shell != null, "Shell cannot be null")
require(shell.getDisplay() != null, "The shell must have a display")
val display = shell.getDisplay()
display.asyncExec(new Runnable() {
def run(): Unit = {
try {
success.hlisted(args)
} catch {
case e: Throwable =>
failure(e)
}
}
})
dummy.hlisted(args)
}.unhlisted
I'll start by simplifying a bit. Suppose I have a function f. I don't know it's arity in advance, and I don't care about what it returns. I want to wrap it with some functionality and get a function with the same argument types. I also don't care what this resulting function returns, so I may as well have it return Unit.
You could write a bunch (well, 22) of functions like the following:
def wrap[A](f: A => Unit): A => Unit = ???
def wrap[A, B](f: (A, B) => Unit): (A, B) => Unit = ???
def wrap[A, B, C](f: (A, B, C) => Unit): (A, B, C) => Unit = ???
But you don't want to.
Shapeless can definitely help you solve this problem more generically:
def wrap[F, A <: HList](f: F)(
implicit h: FnHListerAux[F, A => Unit], u: FnUnHListerAux[A => Unit, F]
): F = { (args: A) =>
println("Before!"); f.hlisted(args); println("After!")
}.unhlisted
Which gives us:
scala> def f(i: Int, s: String) { println(s * i) }
f: (i: Int, s: String)Unit
scala> val wf = wrap(f _)
wf: (Int, String) => Unit = <function2>
scala> wf(3, "ab")
Before!
ababab
After!
Note that f could return something other than Unit and this would still work, since everything in Scala is a Unit and the FunctionN traits are covariant in their return type.
Applying this approach to your code, we get the following:
def async[F, A <: HList](
shell: Shell, success: F, failure: FunctionTypes.Failure
)(
implicit h: FnHListerAux[F, A => Unit], u: FnUnHListerAux[A => Unit, F]
): F = { (args: A) =>
require(shell != null, "Shell cannot be null")
require(shell.getDisplay() != null, "The shell must have a display")
val display = shell.getDisplay()
display.asyncExec(new Runnable() {
def run(): Unit = {
try {
success.hlisted(args)
} catch {
case e: Throwable =>
failure(e)
}
}
})
}.unhlisted
No dummy required.