I have the following types and declarations:
import scalaz._, Scalaz._
trait Container[T]
type FreeContainer[A] = Free[Container, A]
type FreeFreeContainer[A] = Free[FreeContainer, A]
val fc: FreeContainer[Int]
val ffc: FreeFreeContainer[Int]
val t: Container ~> Id
val tranformed: Int = fc.foldMap(t) //ok
val tranformed2: Int = ffc.foldMap(t) //error
Is it possible to lift Container ~> Id to FreeContainer ~> Id?
Yes, via foldMap:
val t: Container ~> Id
val tt: FreeContainer ~> Id = new (FreeContainer ~> Id) {
def apply[A](fc: FreeContainer[A]): A = fc.foldMap(t)
}
Using the polymorphic lambda syntax of kind-projector, this can be simplified to
val tt: FreeContainer ~> Id = λ[FreeContainer ~> Id](_.foldMap(t))
So you can do
val tranformed2: Int = ffc.foldMap(λ[FreeContainer ~> Id](_.foldMap(t)))
Alternatively, you can just do two consecutive foldMaps, the first one with the identity natural transformation:
val tranformed2: Int = ffc.foldMap(NaturalTransformation.refl[FreeContainer]).foldMap(t)
Related
Suppose I have two algebras:
sealed trait OneAlgebra[A]
case class create() extends OneAlgebra[Unit]
case class update() extends OneAlgebra[Unit]
sealed trait TwoAlgebra[A]
case class add() extends TwoAlgebra[Unit]
case class delete() extends TwoAlgebra[Unit]
// Interpreter
def commonInterpreter: CommonAlgebra ~> IO = new (CommonAlgebra ~> IO) {
override def apply[A](commonAlgebra: CommonAlgebra[A]): IO[A] = commonAlgebra match {
case create => do something
case delete => do something
}
}
How to add multiple algebras in override apply method in a single interpreter? Any inputs?
One option is to write one interpreter for each and then compose them.
scala> val intp1 = new (OneAlgebra ~> IO){
| def apply[A](fa: OneAlgebra[A]) = fa match {
| case create() => IO(println("create"))
| case update() => IO(println("update"))
| }
| }
val intp1: OneAlgebra ~> cats.effect.IO = $anon$1#35696757
scala> val intp2 = new (TwoAlgebra ~> IO){
| def apply[A](fa: TwoAlgebra[A]) = fa match {
| case add() => IO(println("add"))
| case delete() => IO(println("delete"))
| }
| }
val intp2: TwoAlgebra ~> cats.effect.IO = $anon$1#53be89fb
scala> type CommonAlgebra[A] = EitherK[OneAlgebra, TwoAlgebra, A]
type CommonAlgebra
scala> val intp: CommonAlgebra ~> IO = intp1 or intp2
val intp: CommonAlgebra ~> cats.effect.IO = cats.arrow.FunctionK$$anon$2#434c79e3
scala> intp(EitherK(Left(create()))).unsafeRunSync
create
I am trying to implement a Websocket Login flow with Akka Flow. I get a myriad of nasty runtime exceptions around Inlets, Outlets and Connection issues.
My latest is:
java.lang.IllegalStateException: Illegal GraphDSL usage. Inlets [Map.in] were not returned in the resulting shape and not connected.
Snippet:
object Login {
def graph(system: ActorSystem, future: Future[LoginCommand.UserData], socketUrl: String) =
Source.fromGraph(GraphDSL.create() { implicit builder: GraphDSL.Builder[NotUsed] =>
import GraphDSL.Implicits._
val in = Source.fromFuture(future)
in.named("LoginData")
val fanIn = Zip[LoginResponse, LoginCommand.UserData]
val exasolLogin = builder.add(Http(system).webSocketClientFlow(WebSocketRequest(socketUrl)))
val encryptLoginData = FlowShape(exasolLogin.in, fanIn.out)
val exasolAnnounce = Http(system).webSocketClientFlow(WebSocketRequest(socketUrl))
val announceLogin = Source.single(LoginCommand)
in -> fanIn
announceLogin -> exasolAnnounce -> fanIn
fanIn -> encryptLoginData -> exasolLogin
SourceShape(exasolLogin.out)
})
}
I might be using the DSL totally wrong as I have not yet found a single writeup which explains Graphs, Shapes, Flows, Materialized values in depth.
Could someone point out what I am doing wrong or perhaps how this is supposed to be written?
EDIT 1:
Have now replaced -> with ~> and get nasty compile errors:
object Login {
def graph(system: ActorSystem, future: Future[LoginCommand.UserData], socketUrl: String) =
Source.fromGraph(GraphDSL.create() { implicit builder: GraphDSL.Builder[NotUsed] =>
import GraphDSL.Implicits._
val in = Source.fromFuture(future)
in.named("LoginData")
val fanIn = builder.add(Zip[LoginResponse, LoginCommand.UserData])
val exasolLogin = Http(system).webSocketClientFlow(WebSocketRequest(socketUrl))
val encryptLoginData = Flow[(LoginResponse, LoginCommand.UserData)].map(data => data._1)
val loginDataMessage = Flow[LoginCommand.UserData].map(data => TextMessage("bar"))
val exasolAnnounce = Http(system).webSocketClientFlow(WebSocketRequest(socketUrl))
val announceResponse = Flow[Message].map(data => LoginResponse("key", "mod", "exp"))
val loginMessage = Flow[LoginCommand].map(data => TextMessage("foo"))
val session = builder.add(Flow[Message].map(data => LoginCommand.SessionData(0, 1, "2", "db", "w", 59, 546, 45, "q", "TZ", "TZB")))
in ~> fanIn.in1
Source.single(LoginCommand) ~> loginMessage ~> exasolAnnounce ~> announceResponse ~> fanIn.in0
fanIn.out ~> encryptLoginData ~> loginDataMessage ~> exasolLogin ~> session
SourceShape(session.out)
})
}
which leads to
exasol-client/LoginGraph.scala:42: error: overloaded method value ~> with alternatives:
(to: akka.stream.SinkShape[exasol.LoginCommand.type])(implicit b: akka.stream.scaladsl.GraphDSL.Builder[_])Unit <and>
(to: akka.stream.Graph[akka.stream.SinkShape[exasol.LoginCommand.type], _])(implicit b: akka.stream.scaladsl.GraphDSL.Builder[_])Unit <and>
[Out](flow: akka.stream.FlowShape[exasol.LoginCommand.type,Out])(implicit b: akka.stream.scaladsl.GraphDSL.Builder[_])akka.stream.scaladsl.GraphDSL.Implicits.PortOps[Out] <and>
[Out](junction: akka.stream.UniformFanOutShape[exasol.LoginCommand.type,Out])(implicit b: akka.stream.scaladsl.GraphDSL.Builder[_])akka.stream.scaladsl.GraphDSL.Implicits.PortOps[Out] <and>
[Out](junction: akka.stream.UniformFanInShape[exasol.LoginCommand.type,Out])(implicit b: akka.stream.scaladsl.GraphDSL.Builder[_])akka.stream.scaladsl.GraphDSL.Implicits.PortOps[Out] <and>
[Out](via: akka.stream.Graph[akka.stream.FlowShape[exasol.LoginCommand.type,Out],Any])(implicit b: akka.stream.scaladsl.GraphDSL.Builder[_])akka.stream.scaladsl.GraphDSL.Implicits.PortOps[Out] <and>
[U >: exasol.LoginCommand.type](to: akka.stream.Inlet[U])(implicit b: akka.stream.scaladsl.GraphDSL.Builder[_])Unit
cannot be applied to (akka.stream.FlowShape[exasol.LoginCommand,akka.http.scaladsl.model.ws.TextMessage.Strict])
Source.single(LoginCommand) ~> loginMessage ~> exasolAnnounce ~> announceResponse ~> fanIn.in0
You need something like this:
object Login {
def graph(system: ActorSystem, future: Future[LoginCommand.UserData], socketUrl: String): Source[Message, NotUsed] =
Source.fromGraph(GraphDSL.create() { implicit builder: GraphDSL.Builder[NotUsed] =>
import GraphDSL.Implicits._
val in = Source.fromFuture(future)
in.named("LoginData")
val fanIn = builder.add(Zip[LoginResponse, LoginCommand.UserData])
val exasolLogin = builder.add(Http(system).webSocketClientFlow(WebSocketRequest(socketUrl)))
val encryptLoginData = Flow[(LoginResponse, LoginCommand.UserData)].map(data => TextMessage(data.toString)) //stub
val encryptAnnounceData = Flow[LoginCommand].map(data => TextMessage(data.toString)) //stub
val decryptAnnounceData = Flow[Message].map(message => LoginResponse(message)) //stub
val exasolAnnounce = Http(system).webSocketClientFlow(WebSocketRequest(socketUrl))
val announceLogin = Source.single(LoginCommand)
in ~> fanIn.in1
announceLogin ~> encryptAnnounceData ~> exasolAnnounce ~> decryptAnnounceData ~> fanIn.in0
fanIn.out ~> encryptLoginData ~> exasolLogin
SourceShape(exasolLogin.out)
})
}
Keep in mind, that -> and ~> are different operators (you should use ~>). And you need to add a shape to the builder only if you are going to manually connect the shape inlets and outlets.
I have a set of stream stages (Sources, Flows and Sinks) which I would like to add some MetaData information to.
Therefore rather than Sources producing A -> (A, StreamMetaData). I've managed to do this using custom stream stages whereby on grab(in) the element, I push(out, (elem, StreamMetaData)). In reality it is not 'converting' the existing Source but passing it to a flow to recreate a new source.
Now I'm trying to implement the below MetaStream stage:
Therefore given that the source is producing tuples of (A, StreamMetaData), I want to pass the A to an existing Flow for some computation to take place and then merge the output produced 'B' with the StreamMetaData. These will then be passed to a Sink which accepts (B, StreamMetaData).
How would you suggest I go about it. I've been informed partial graphs are the best bet and would help in completing such a task. UniformFanOut and UniformFanIn using Unzip((A streamMetaData), A, StreamMetaData) and Zip(A,B)
val fanOut = GraphDSL.create() { implicit b =>
val unzip = b.add(Unzip[T, StreamMetaData])
UniformFanOutShape(unzip.in, unzip.out0, unzip.out1)
}
val fanIn = GraphDSL.create() { implicit b =>
val zip = b.add(Zip[T ,StreamMetaData]())
UniformFanInShape(zip)
}
How can I connect the fanIn and fanOut so as to achieve the same behavior as in the picture?
I had something like this in mind;
def metaFlow[T, B, Mat](flow: Flow[T, B, Mat]): Unit = {
val wrappedFlow =
Flow.fromGraph(GraphDSL.create(){ implicit b =>
import GraphDSL.Implicits._
val unzip: FanOutShape2[(T, StreamMetaData), T, StreamMetaData] = b.add(Unzip[T, StreamMetaData])
val existingFlow = b.add(flow)
val zip: FanInShape2[B,StreamMetaData,(B,StreamMetaData)] = b.add(Zip[B, StreamMetaData])
unzip.out0 ~> existingFlow ~> zip.in0
unzip.out1 ~> zip.in1
FlowShape(unzip.in, zip.out)
})
}
Thanks in advance.
This aprox creating a new SourceShape stacking flow graph can work, a little bit different of your flowShape implementation.
def sourceGraph[A, B](f: A => B, source: Source[(A, StreamMetaData), NotUsed]) = Source.fromGraph(GraphDSL.create() { implicit builder: GraphDSL.Builder[NotUsed] =>
import GraphDSL.Implicits._
val unzip = builder.add(Unzip[A, StreamMetaData]())
val zip = builder.add(Zip[B, StreamMetaData]())
val flow0 = builder.add(Flow[A].map { f(_) })
val flow1 = source ~> unzip.in
unzip.out0 ~> flow0 ~> zip.in0
unzip.out1 ~> zip.in1
SourceShape(zip.out)
})
def flowGraph[A, B](f: A => B) = Flow.fromGraph(GraphDSL.create() { implicit builder =>
import GraphDSL.Implicits._
val unzip = builder.add(Unzip[A, StreamMetaData]())
val zip = builder.add(Zip[B, StreamMetaData]())
val flow0 = builder.add(Flow[A].map { f(_) })
unzip.out0 ~> flow0 ~> zip.in0
unzip.out1 ~> zip.in1
FlowShape(unzip.in, zip.out)
})
I have an Akka Streams Source which I want to split into two sources according to a predicate.
E.g. having a source (types are simplified intentionally):
val source: Source[Either[Throwable, String], NotUsed] = ???
And two methods:
def handleSuccess(source: Source[String, NotUsed]): Future[Unit] = ???
def handleFailure(source: Source[Throwable, NotUsed]): Future[Unit] = ???
I would like to be able to split the source according to _.isRight predicate and pass the right part to handleSuccess method and left part to handleFailure method.
I tried using Broadcast splitter but it requires Sinks at the end.
Although you can choose which side of the Source you want to retrieve items from it's not possible to create a Source that that yields two outputs which is what it seems like you would ultimately want.
Given the GraphStage below which essentially splits the left and right values into two outputs...
/**
* Fans out left and right values of an either
* #tparam L left value type
* #tparam R right value type
*/
class EitherFanOut[L, R] extends GraphStage[FanOutShape2[Either[L, R], L, R]] {
import akka.stream.{Attributes, Outlet}
import akka.stream.stage.GraphStageLogic
override val shape: FanOutShape2[Either[L, R], L, R] = new FanOutShape2[Either[L, R], L, R]("EitherFanOut")
override def createLogic(inheritedAttributes: Attributes): GraphStageLogic = new GraphStageLogic(shape) {
var out0demand = false
var out1demand = false
setHandler(shape.in, new InHandler {
override def onPush(): Unit = {
if (out0demand && out1demand) {
grab(shape.in) match {
case Left(l) =>
out0demand = false
push(shape.out0, l)
case Right(r) =>
out1demand = false
push(shape.out1, r)
}
}
}
})
setHandler(shape.out0, new OutHandler {
#scala.throws[Exception](classOf[Exception])
override def onPull(): Unit = {
if (!out0demand) {
out0demand = true
}
if (out0demand && out1demand) {
pull(shape.in)
}
}
})
setHandler(shape.out1, new OutHandler {
#scala.throws[Exception](classOf[Exception])
override def onPull(): Unit = {
if (!out1demand) {
out1demand = true
}
if (out0demand && out1demand) {
pull(shape.in)
}
}
})
}
}
.. you can route them to only receive one side:
val sourceRight: Source[String, NotUsed] = Source.fromGraph(GraphDSL.create(source) { implicit b => s =>
import GraphDSL.Implicits._
val eitherFanOut = b.add(new EitherFanOut[Throwable, String])
s ~> eitherFanOut.in
eitherFanOut.out0 ~> Sink.ignore
SourceShape(eitherFanOut.out1)
})
Await.result(sourceRight.runWith(Sink.foreach(println)), Duration.Inf)
... or probably more desirable, route them to two seperate Sinks:
val leftSink = Sink.foreach[Throwable](s => println(s"FAILURE: $s"))
val rightSink = Sink.foreach[String](s => println(s"SUCCESS: $s"))
val flow = RunnableGraph.fromGraph(GraphDSL.create(source, leftSink, rightSink)((_, _, _)) { implicit b => (s, l, r) =>
import GraphDSL.Implicits._
val eitherFanOut = b.add(new EitherFanOut[Throwable, String])
s ~> eitherFanOut.in
eitherFanOut.out0 ~> l.in
eitherFanOut.out1 ~> r.in
ClosedShape
})
val r = flow.run()
Await.result(Future.sequence(List(r._2, r._3)), Duration.Inf)
(Imports and initial setup)
import akka.NotUsed
import akka.stream.scaladsl.{GraphDSL, RunnableGraph, Sink, Source}
import akka.stream.stage.{GraphStage, InHandler, OutHandler}
import akka.stream._
import akka.actor.ActorSystem
import com.typesafe.config.ConfigFactory
import scala.concurrent.Future
import scala.concurrent.ExecutionContext.Implicits.global
import scala.concurrent.Await
import scala.concurrent.duration.Duration
val classLoader = getClass.getClassLoader
implicit val system = ActorSystem("QuickStart", ConfigFactory.load(classLoader), classLoader)
implicit val materializer = ActorMaterializer()
val values: List[Either[Throwable, String]] = List(
Right("B"),
Left(new Throwable),
Left(new RuntimeException),
Right("B"),
Right("C"),
Right("G"),
Right("I"),
Right("F"),
Right("T"),
Right("A")
)
val source: Source[Either[Throwable, String], NotUsed] = Source.fromIterator(() => values.toIterator)
Edit: this other answer with divertTo is a better solution than mine, IMO. I'll leave my answer as-is for posterity.
original answer:
This is implemented in akka-stream-contrib as PartitionWith. Add this dependency to SBT to pull it in to your project:
libraryDependencies += "com.typesafe.akka" %% "akka-stream-contrib" % "0.9"```
`PartitionWith` is shaped like a `Broadcast(2)`, but with potentially different types for each of the two outlets. You provide it with a predicate to apply to each element, and depending on the outcome, they get routed to the applicable outlet. You can then attach a `Sink` or `Flow` to each of these outlets independently as appropriate. Building on [cessationoftime's example](https://stackoverflow.com/a/39744355/147806), with the `Broadcast` replaced with a `PartitionWith`:
val eitherSource: Source[Either[Throwable, String], NotUsed] = Source.empty
val leftSink = Sink.foreach[Throwable](s => println(s"FAILURE: $s"))
val rightSink = Sink.foreach[String](s => println(s"SUCCESS: $s"))
val flow = RunnableGraph.fromGraph(GraphDSL.create(eitherSource, leftSink, rightSink)
((_, _, _)) { implicit b => (s, l, r) =>
import GraphDSL.Implicits._
val pw = b.add(
PartitionWith.apply[Either[Throwable, String], Throwable, String](identity)
)
eitherSource ~> pw.in
pw.out0 ~> leftSink
pw.out1 ~> rightSink
ClosedShape
})
val r = flow.run()
Await.result(Future.sequence(List(r._2, r._3)), Duration.Inf)
For this you can use a broadcast, then filter and map the streams within the GraphDSL:
val leftSink = Sink.foreach[Throwable](s => println(s"FAILURE: $s"))
val rightSink = Sink.foreach[String](s => println(s"SUCCESS: $s"))
val flow = RunnableGraph.fromGraph(GraphDSL.create(eitherSource, leftSink, rightSink)((_, _, _)) { implicit b => (s, l, r) =>
import GraphDSL.Implicits._
val broadcast = b.add(Broadcast[Either[Throwable,String]](2))
s ~> broadcast.in
broadcast.out(0).filter(_.isLeft).map(_.left.get) ~> l.in
broadcast.out(1).filter(_.isRight).map(_.right.get) ~> r.in
ClosedShape
})
val r = flow.run()
Await.result(Future.sequence(List(r._2, r._3)), Duration.Inf)
I expect you will be able to run the functions you want from within the map.
In the meantime this has been introduced to standard Akka-Streams:
https://doc.akka.io/api/akka/current/akka/stream/scaladsl/Partition.html.
You can split the input stream with a predicate and then use collect on each outputs to get only the types you are interested in.
You can use divertTo to attach alternative Sink to the flow to handle Lefts: https://doc.akka.io/docs/akka/current/stream/operators/Source-or-Flow/divertTo.html
source
.divertTo(handleFailureSink, _.isLeft)
.map(rightEither => handleSuccess(rightEither.right.get()))
I am looking for a SinkSource that provides a Sink and a Source. If an element flows into that Sink it should be provided at the corresponding Source. The following code shows what I mean:
object SinkSource {
def apply[T] = new {
def sink: Sink[T] = ???
def source: Source[T] = ???
}
}
val flowgraph = FlowGraph { implicit fgb =>
import FlowGraphImplicits._
val sinksource = SinkSource[Int]
Source(1 to 5) ~> sinksource.sink
sinksource.source ~> Sink.foreach(print)
}
implicit val actorSystem = ActorSystem(name = "System")
implicit val flowMaterializer = FlowMaterializer()
val materializedMap = flowgraph.run()
If executed this should print: 12345
So, does a SinkSource exist (haven't seen it in the API) or does anyone know how to implement it?
I should mention that I need distinct access to Sink and Source so that Flow isn't a solution in this particular form:
Source(1 to 5) ~> Flow[Int] ~> Sink.foreach(println)
As so often, ideas come to mind if question was already asked: It turned out, I don't need a Sink and a Source, JunctionInPort and JunctionOutPort are sufficient.
So here it goes:
object SinkSource {
def apply[T](implicit fgb: FlowGraphBuilder) = new SinkSource[T]
}
class SinkSource[T](implicit fgb: FlowGraphBuilder) {
import FlowGraphImplicits._
private val merge = Merge[T]
private val bcast = Broadcast[T]
Source.empty ~> merge
merge ~> bcast
bcast ~> Sink.ignore
def in: JunctionInPort[T] = merge
def out: JunctionOutPort[T] = bcast
}
val flowgraph = FlowGraph { implicit fgb =>
import FlowGraphImplicits._
val source = Source(1 to 5)
val sink = Sink.foreach(println)
val sinkSource = SinkSource[Int]
source ~> sinkSource.in
sinkSource.out ~> sink
}
implicit val actorSystem = ActorSystem(name = "System")
implicit val flowMaterializer = FlowMaterializer()
val materializedMap = flowgraph.run()