Generating adaptors for higher-kinded interfaces - scala

I often find myself in a scenario where I have defined an interface like so:
trait FooInterface [T[_]] {
def barA (): T[Int]
def barB (): T[Int]
def barC (): T[Int]
}
I then write a few different implementations each typed on the Higher Kinded Type that makes the most sense for that particular implementation:
object FooImpl1 extends FooInterface[Option] { ... }
object FooImpl2 extends FooInterface[Future] { ... }
object FooImpl3 extends FooInterface[({type X[Y] = ReaderT[Future, Database, Y]})#X] { ... }
All implementations are perfectly valid, all return their results wrapped in a particular Higher Kinded Type.
I then often come to writing some business logic, let's say that in the block of logic I am working with is using Future as a context, I might write something like this:
val foo: FooInterface[Future] = ???
def fn (): Future[Int] = Future { 42 }
val result: Future[Int] = for {
x <- foo.barA ()
y <- foo.barB ()
z <- foo.barC ()
w <- fn ()
} yield x + y + z + w
The above code would work really well with FooImpl2 however the other implementations do not slot in directly. In this scenario I always wind up writing simple adapters:
object FooImpl1Adapter extends FooInterface[Future] {
val t = new Exception ("Foo impl 1 failed.")
def barA (): Future[Int] = FooImpl1.barA () match {
case Some (num) => Future.successful (num)
case None => Future.failed (t)
}
def barB (): Future[Int] = FooImpl1.barB () match {
case Some (num) => Future.successful (num)
case None => Future.failed (t)
}
def barC (): Future[Int] = FooImpl1.barC () match {
case Some (num) => Future.successful (num)
case None => Future.failed (t)
}
}
case class FooImpl3Adapter (db: Database) extends FooInterface[Future] {
def barA (): Future[Int] = FooImpl3.barA ().run (db)
def barB (): Future[Int] = FooImpl3.barB ().run (db)
def barC (): Future[Int] = FooImpl3.barC ().run (db)
}
Writing adapters is fine but it involves a lot of boilerplate, especially for interfaces with lots of functions; what's more is that each method gets the exactly same adaptation treatment for each method. What I really want to do is lift an adaptor implementation from an existing implementation, only specify in adaption mechanism once.
I guess I want to be able to write something like this:
def generateAdapterFn[X[_], Y[_]] (implx: FooInterface[X])(f: X[?] => Y[?]): FooInterface[Y] = ???
So I could use it like so:
val fooImpl1Adapter: FooInterface[Future] = generateAdapterFn [?, Future] () { z => z match {
case Some (obj) => Future.successful (obj)
case None => Future.failed (t)
}}
The question is: How could I write the generateAdapterFn function?
I'm not really sure how to approach solving this, or if there are other common patterns or solutions to my problem. I suspect that to write the generateAdapterFn function I desire I would need to write a macro? If so how might that be done?

What you are looking for is a natural transformation from X to Y (what you called X[?] => Y[?]). In Cats is called a FunctionK (with the popular type alias ~>).
You could define a natural transformation between Option and Future as :
import cats.arrow.FunctionK
import scala.concurrent.Future
val option2future = new FunctionK[Option, Future] {
def apply[A](opt: Option[A]): Future[A] = opt match {
case Some(obj) => Future.succesful(obj)
case None => Future.failed(new Exception("none")) // t ??
}
}
With the kind projector compiler plugin this could be written more concise as :
val opt2fut = λ[FunctionK[Option, Future]]{
case Some(obj) => Future.succesful(obj)
case None => Future.failed(new Exception("none")) // t ??
}
Your generateAdapter function could then look like :
import cats.~>
def generateAdapter[X[_], Y[_]](implx: FooInterface[X])(f: X ~> Y): FooInterface[Y] =
new FooInterface[Y] {
def barA: Y[Int] = f(implx.barA)
def barB: Y[Int] = f(implx.barB)
def barC: Y[Int] = f(implx.barC)
}
You should then be able to create a FooInterface[Future]] as :
val fooFuture = generateAdapter(FooImpl1)(opt2fut)
Unrelated, you might be interested to read something about the free monad, which is used to solve similar problems as the one you are facing now.

Keep the code polymorphic for as long as possible. Instead of
val result: Future[Int] = for {
x <- foo.barA ()
y <- foo.barB ()
z <- foo.barC ()
w <- fn ()
} yield x + y + z + w
write
import scalaz.Monad
import scalaz.syntax.monad._
// or
import cats.Monad
import cats.syntax.all._
def result[M[_]: Monad](foo: FooInterface[M], fn: () => M[Int]): M[Int] = for {
x <- foo.barA ()
y <- foo.barB ()
z <- foo.barC ()
w <- fn ()
} yield x + y + z + w
This way, you avoid writing adapters for FooInterface altogether and only transform the final value (via a natural transformation (see Peter Neyens' answer) or also quite easily directly).

Expanding on Peter Neyen's answer (which I've marked as correct as it answers the important part of my question), here's a proof of concept for how generate the adapter at runtime using reflection:
def generateAdapterR[X[_], Y[_]](implx: FooInterface[X])(implicit
f: X ~> Y): FooInterface[Y] = {
import java.lang.reflect.{InvocationHandler, Method, Proxy}
object ProxyInvocationHandler extends InvocationHandler {
def invoke (
proxy: scala.AnyRef,
method: Method,
args: Array[AnyRef]): AnyRef = {
val fn = implx.getClass.getMethod (
method.getName,
method.getParameterTypes: _*)
val x = fn.invoke (implx, args: _*)
val fx = f.getClass.getMethods ()(0)
fx.invoke (f, x)
}
}
Proxy.newProxyInstance(
classOf[FooInterface[Y]].getClassLoader,
Array(classOf[FooInterface[Y]]),
ProxyInvocationHandler
).asInstanceOf[FooInterface[Y]]
}
Ideally it would be possible to type this function on T[_] as well, T being the type of the interface, so the function could be use to generate adaptors for any higher kinded interfaces at runtime.
Something like:
def genericGenerateAdapterR[T[_], X[_], Y[_]](implx: T[X[_]])(implicit
f: X ~> Y): T[Y[_]] = ???
Not really sure if that would be how to write it though...
I think the ideal solution would be to have a compiler plugin that generates the code in Peter Neyen's solution, avoiding reflection and avoiding boilerplate.

Related

Avoiding boilerplate with OptionT (natural transform?)

I have the following methods:
trait Tr[F[_]]{
def getSet(): F[Set[String]]
def checksum(): F[Long]
def value(): F[String]
def doRun(v: String, c: Long, s: Set[String]): F[Unit]
}
now I want to write the following for comprehension:
import cats._
import cats.data.OptionT
import cats.implicits._
def fcmprhn[F[_]: Monad](F: Tr[F]): OptionT[F, Unit] =
for {
set <- OptionT {
F.getSet() map { s =>
if(s.nonEmpty) Some(s) else None
}
}
checksum <- OptionT.liftF(F.checksum())
v <- OptionT.liftF(F.value())
_ <- OptionT.liftF(F.doRun(v, checksum, set))
//can be lots of OptionT.liftF here
} yield ()
As you can see there is too much of OptionT boilerplate. Is there a way to avoid it?
I think I can make use of F ~> OptionT[F, ?]. Can you suggest something?
One approach could be to nest the "F only" portion of the for-comprehension within a single liftF:
def fcmprhn[F[_]: Monad](F: Tr[F]): OptionT[F, Unit] =
for {
set <- OptionT {
F.getSet() map { s =>
if(s.nonEmpty) Some(s) else None
}
}
_ <- OptionT.liftF {
for {
checksum <- F.checksum()
v <- F.value()
_ <- F.doRun(v, checksum, set)
// rest of F monad for-comprehension
} yield ()
}
} yield ()
You can write it in "mtl-style" instead. mtl-style refers to mtl library in haskell but really it just means that instead of encoding effects as values (i.e. OptionT[F, ?]), we encode them as functions that take an abstract effect F[_] and give that F capabilities using type classes. That means instead of using OptionT[F, Unit] as our return type we can just use F[Unit] as our return type because F has to be able to handle errors.
This makes writing code like yours a small bit easier, but it's effect is amplified as you add monad transformers to the stack. Right now you only have to lift once, but what if you wanted a StateT[OptionT[F, ?], S, Unit] in the future. With mtl-style, all you need to do is add another type class constraint.
Here's what your code'd look like written in mtl-style:
def fcmprhn[F[_]](F: Tr[F])(implicit E: MonadError[F, Unit]): F[Unit] =
for {
set <- OptionT {
F.getSet() flatMap { s =>
if(s.nonEmpty) E.pure(s) else E.raiseError(())
}
}
checksum <- F.checksum()
v <- F.value()
_ <- F.doRun(v, checksum, set)
} yield ()
And now when you run the program you can specify the F[_] to be something like what you had before OptionT[F, ?]:
fcmprhn[OptionT[F, ?]](OptionT.liftF(yourOriginalTr))

Functional listener with state

Let's say in my pure Scala program i have a dependency to a Java service.
This Java service accepts a listener to notify me when some data changes.
Let's say the data is a tuple(x, y) and the java service calls the listener whenever X or Y changes but i'm interested only when X.
For this my listener has to save the last value of X, and forward the update/call only when oldX != X, so in order to have this my impure scala listener implementation has to hold a var oldX
val listener = new JavaServiceListener() {
var oldX;
def updated(val x, val y): Unit = {
if (oldX != x) {
oldX = x
//do stuff
}
}
javaService.register(listener)
How would i go about to design a wrapper for this kind of thing in Scala without val or mutable collections ? I can't at the JavaServiceListener level since i'm bound by the method signature, so I need another layer above which the java listener forwards to somehow
My preference would be to wrap it in a Monix Observable, then you can use distinctUntilChanged to eliminate consecutive duplicates. Something like:
import monix.reactive._
val observable = Observable.create(OverflowStrategy.Fail(10)){(sync) =>
val listener = new JavaServiceListener() {
def updated(val x, val y): Unit = {
sync.onNext(x)
}
}
javaService.register(listener)
Cancelable{() => javaService.unregister(listener)}
}
val distinctObservable = observable.distinctUntilChanged
Reactive programming allows you to use a pure model while the library handles all the difficult stuff.
First of all, if you are designing a purely functional program you cannot return Unit (neither Future[Unit], because Future does not suppress side effects).
If performance is not an issue I would make use of Kleisli[Option, xType, IO[Unit]] where T = Option. So the first thing you have to do is define (add the appropriate types)
def updated(oldX, x): Kleisli[Option, xType, xType] = Kleisli liftF {
if(x != oldX) None
else Some(x)
}
def doStuff(x, y): Kleisli[Option, xType, IO[Unit]] = Kleisli pure {
IO{
//doStuff
}
}
and now you can compose them in a for-comprehension something like that:
val result: Kleisli[Option, xType, IO[Unit]] = for{
xx <- updated(oldX, x)
effect <- doStuff(xx, y)
} yield effect
You can perform stateful compuation with ReaderWriterStateT, so you keep oldX as a state.
I found the solution I like with Cats and Cats-Effect:
trait MyListener {
def onChange(n: Int): Unit
}
class MyDistinctFunctionalListener(private val last: Ref[IO, Int], consumer: Int => Unit) extends MyListener {
override def onChange(newValue: Int): Unit = {
val program =
last
.getAndSet(newValue)
.flatMap(oldValue => notify(newValue, oldValue))
program.unsafeRunSync()
}
private def notify(newValue: Int, oldValue: Int): IO[Unit] = {
if (oldValue != newValue) IO(consumer(newValue)) else IO.delay(println("found duplicate"))
}
}
object MyDistinctFunctionalListener {
def create(consumer: Int => Unit): IO[MyDistinctFunctionalListener] =
Ref[IO].of(0).map(v => new MyDistinctFunctionalListener(v, consumer))
}
val printer: Int => Unit = println(_)
val functionalDistinctPrinterIO = MyDistinctFunctionalListener.create(printer)
functionalDistinctPrinterIO.map(fl =>
List(1, 1, 2, 2, 3, 3, 3, 4, 5, 5).foreach(fl.onChange)
).unsafeRunSync()
More stuff about handling shared state here https://github.com/systemfw/scala-italy-2018
it is debatable if this is worth it over the private var solution

yld V.S yield on SCALA

I am working in a project that use the following code:
case class R(f: Vector[String], s: Vector[String]) {
def apply(name: String): String = f(schema indexOf name)
def apply(names: S): Vector[String] = names map (this apply _)
}
def processCSV(file: String)(yld: R => Unit): Unit = {
val in = new Scanner(file)
val s= in.next('\n').split(",").toVector
while (in.hasNext) {
val f = schema map (n => in.next(if (n == s.last) '\n' else ','))
yld(R(f, s))
}
}
def execOp(op: Operator)(yld: R => Unit): Unit = op match {
case Scan(file, _, _, _) => processCSV(file)(yld)}
Then My question is what is the meaning of yld? That is the same of yield?
Exactly how works, someone can help me to understand how works this yld?
yield is a scala keyword used with for-comprehensions.
yld in that code is VERY different: it is just the name that the author of the code gave one of the parameters to the functions processCSV and execOp. Any other name could have been given to those parameters: fn, callback, cb, etc. Nothing special there. Given the type R => Unit, it is just a function that takes R as input and return Unit (equivalent to void in java). Essentially, a callback where the work happens as side-effects.

Scala generic types

I there, I'm trying to combine the Command and Chain of responsibility patterns with Scala style.
Basically, I would like to have one Executor who pass the command trough the chain and return the result. A command is anything that return a T :
class class Invoker {
type Command[T] = () => T
val handlers = new mutable.ListBuffer[PartialFunction[() => T, T]]
def invoke[T](command: => T): Future[T] = Future {
handlers.find(_.isDefinedAt(command)).map(_.apply(command))
}
}
But, because I'm a scala beginner, I have problems with the generic types for the list of handlers. I cannot figure how to define T in the declaration of handlers so that the invoke command return the correct type (wich should be T)
Can someone help me to implement this pattern ?
There are several sketchy places, but I think the closest you'll get to what you want would be (Though, it will not work, see below):
import scala.collection.mutable
import scala.concurrent.Future
object Invoker {
val handlers = new mutable.ListBuffer[PartialFunction[Any, Any]]
def invoke[T](command: () => T): Future[T] = Future {
handlers.collectFirst {
case pf if pf.isDefinedAt(command) => pf(command)
}.fold(throw new Exception("failure")) {
_.asInstanceOf[T]
}
}
}
However,
It will not work because, the partial functions or more specifically pattern matching will most certainly not work as you expect for Function0
You loose most of your type information through erasure and have to rely upon what you know about the partial functions.
In scala, the need to call asInstanceOf is a good indicator that something can be done better.
#1
When you define a list of partial functions that match on Function0 like for example this:
val T1: () => Int = () => 1
val T2: () => Int = () => 2
val T3: () => Int = () => 3
val pfs: Seq[PartialFunction[Any, Any]] = Seq(
PartialFunction[Any, Any] {
case T1 => T1()
},
PartialFunction[Any, Any] {
case T2 => T2()
},
PartialFunction[Any, Any] {
case T3 => T3()
}
)
Trying to find a match for any function literal will lead to a MatchError:
def invoke[T](c: () => T): T = {
pfs.collectFirst {
case pf if pf.isDefinedAt(c) => pf(c)
}.fold(throw new Exception("failure")) {
_.asInstanceOf[T]
}
}
invoke(() => 1) |-> scala.MatchError: <function0> (of class abc.A$A290$A$A290$$anonfun$ti1$1)
This would only work if you define Constants for allowed functions and only use those constants, when you call invoke.
You could define them with the Invoker object:
object Invoker {
val SomeCommand = () => 5 + 5
val AnotherCommand = () => 5 * 5
}
But that would utterly eliminate any flexibility.
FINALLY:
If you want to be able to perform any Command then why maintain a list of handlers anyway?
object Invoker {
def invoke[T](command: () => T): Future[T] = Future {
command()
}
}
should be sufficient for that.

Relation of free monad and AST

I'm referring to the Ken Scambler's source code listed below, also see GitHub source .
package kenbot.free
import scalaz._
import Scalaz._
import Free._
import scala.collection.mutable
// This example is based off the one in Runar Bjarnason's "Dead Simple Dependency Injection" talk.
// http://www.youtube.com/watch?v=ZasXwtTRkio
// 0. Fantasy API
// def put(key: String, value: String): Unit
// def get(key: String): String
// def delete(key: String): Unit
// 1. ADT
sealed trait KVS[+Next]
case class Put[Next](key: String, value: String, next: Next) extends KVS[Next] // <---- def put(key: String, value: String): Unit
case class Get[Next](key: String, onResult: String => Next) extends KVS[Next] // <---- def get(key: String): String
case class Delete[Next](key: String, next: Next) extends KVS[Next] // <---- def delete(key: String): Unit
object KVS {
type Script[A] = Free[KVS, A]
// 2. Functor definition
implicit val functor: Functor[KVS] = new Functor[KVS] {
def map[A,B](kvs: KVS[A])(f: A => B): KVS[B] = kvs match {
case Put(key, value, next) => Put(key, value, f(next))
case Get(key, onResult) => Get(key, onResult andThen f)
case Delete(key, next) => Delete(key, f(next))
}
}
// 3. Lifting functions
def put(key: String, value: String): Script[Unit] = liftF( Put(key, value, ()) )
def get(key: String): Script[String] = liftF(Get(key, identity))
def delete(key: String): Script[Unit] = liftF(Delete(key, ()))
// 4. Composite functions
def modify(key: String, f: String => String): Free[KVS, Unit] = for {
v <- get(key)
_ <- put(key, f(v))
} yield ()
// 5. Write scripts
val script: Free[KVS, Unit] = for {
id <- get("swiss-bank-account-id")
_ <- modify(id, (_ + 1000000))
_ <- put("bermuda-airport", "getaway car")
_ <- delete("tax-records")
} yield ()
// 6. Interpreters
// Building an immutable structure
def interpretPure(kvs: Script[Unit], table: Map[String, String] = Map.empty): Map[String, String] = kvs.resume.fold({
case Get(key, onResult) => interpretPure(onResult(table(key)), table)
case Put(key, value, next) => interpretPure(next, table + (key -> value))
case Delete(key, next) => interpretPure(next, table - key)
}, _ => table)
// Directly running effects
def interpretImpure(kvs: Script[Unit], table: mutable.Map[String, String]): Unit = kvs.go {
case Get(key, onResult) => onResult(table(key))
case Put(key, value, next) => table += (key -> value); next
case Delete(key, next) => table -= key; next
}
}
If a follow these slides, any script (see 5. in source code) is "transformed" into something like Suspend(Op(Suspend(Op(......(Result(Op))..)) within the free monad.
Unfortunately, the script under 5 is just a linear sequence of commands.
Even after searching the web for several hours, I wasn't able to find any examples, that gave more insight on the following questions:
The sequence of linear operations (which is also a tree of course) is represented by Suspend(Op(Suspend(Op(......(Result(Op))..)) and is thus a representation of the AST? Is this assumption right?
How is a real AST represented within the free monad? I assume, this happens, when control structures are included? (e.g. left and right tree branch, depending on condition) . Could someone please illustrate an example where real ASTs come into play? Maybe, an illustration of how an "if" could be implemented in the given example.
What is the general approach to include control structures into scripts (as given under 5 in source code?)
P.S.: Please try to stick to Scala / ScalaZ, as Haskell is not (yet) my field of expertise.
In Scalaz, the Free monad as the two cases (simplified and ignoring the GoSub optimization):
sealed abstract class Free[S[_], A]
case class Return[S[_], A](a: A) extends Free[S, A]
case class Suspend[S[_], A](a: S[Free[S, A]]) extends Free[S, A]
Let's first see what Free.liftF does, e.g. for
def get(key: String): Script[String] = liftF(Get(key, identity))
when doing get("key") we will get
get("key")
// definition of get
liftF(Get("key",identity)
// definition of liftF
Suspend(Get("key",identity).map(Return)
// definition of map for Get above
Suspend(Get("key", identity andThen Return))
// identity andThen f == f
Suspend(Get("key", Return))
Having that, let's start with your questions:
The sequence of linear operations (which is also a tree of course) is represented by Suspend(Op(Suspend(Op(......(Result(Op))..)) and is thus a representation of the AST? Is this assumption right?
Essentially yes, a program written in the DSL using the free monad arising from a functor represents a chain of "steps" where each step is either a Suspend containing one of your functor cases or a Return representing the end of the chain.
As an concrete example, script looks about like this:
Suspend(Get("swiss-bank-account-id",
id => Suspend(Get(id,
v => Suspend(Put(id, v+1000000,
_ => Suspend(Put("bermuda-airport","getaway car",
_ => Suspend(Delete("tax-records",
_ => Return(())
))))))))))
As you can see, we essentially just "fill" the holes of our functor with the continuation of the computation, terminating with a Return. In the sample DSL we will always have a linear chain, due to the fact that every case of the KVS functor only has one "hole" to fill, so no branching.
How is a real AST represented within the free monad? I assume, this happens, when control structures are included? (e.g. left and right tree branch, depending on condition) . Could someone please illustrate an example where real ASTs come into play? Maybe, an illustration of how an "if" could be implemented in the given example.
Let's extend our DSL with a branching construct:
case class Cond[Next](cond: Boolean, ifTrue: Free[KVS,Next], ifFalse: Free[KVS,Next]) extends KVS[Next]
def cond[A](c: Boolean)(ifTrue: => Script[A])(ifFalse: => Script[A]): Script[A] =
liftF(Cond(c,ifTrue,ifFalse))
after changing the interpreter cases, it can be used like this:
val script2: Script[Unit] = for {
id <- get("swiss-bank-account-id")
_ <- cond(id == "123") {
Free.point[KVS,Unit](())
} {
for {
_ <- modify(id, ("LOTS OF " + _))
_ <- put("bermuda-airport", "getaway car")
_ <- delete("tax-records")
} yield ()
}
} yield ()
So now you have a "real" AST where I interpret your "real" as "has branching nodes" instead of the linear chain form as was the case up until now. Output is as expected:
println(interpretPure(
script2,
Map("swiss-bank-account-id" -> "42", "42" -> "money", "tax-records" -> "acme corp")))
// Map(swiss-bank-account-id -> 42, 42 -> LOTS OF money, bermuda-airport -> getaway car)
println(interpretPure(
script2,
Map("swiss-bank-account-id" -> "123", "tax-records" -> "acme corp")))
// Map(swiss-bank-account-id -> 123, tax-records -> acme corp)
What is the general approach to include control structures into scripts (as given under 5 in source code?)
First of all, remember that you can for example use the standard if inside for-comprehensions:
val script3: Script[Unit] = for {
id <- get("swiss-bank-account-id")
_ <- if (id == "123") {
Free.point[KVS,Unit](())
} else {
for {
_ <- modify(id, ("LOTS OF " + _))
_ <- put("bermuda-airport", "getaway car")
_ <- delete("tax-records")
} yield ()
}
} yield ()
Secondly, remember that due to the fact that Script[A] is just Free[KVS,A] you have a monad at disposal, so any "control structure" defined in e.g. Scalaz for monads will work for you too:
val script4: Script[Unit] = modify("key",_+"!").untilM_ { get("key").map(_.length > 42) }
println(interpretPure(script4, Map("key" -> "")))
// Map(key -> !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!)
Have a look at Monad.scala and MonadSyntax.scala, there's also whileM and iterateWhile.