Protocol switching successfully or not - scala

I have the following code snippet, that does not compile:
import akka.actor.ActorSystem
import akka.Done
import akka.http.scaladsl.Http
import akka.stream.ActorMaterializer
import akka.stream.scaladsl._
import akka.http.scaladsl.model._
import akka.http.scaladsl.model.ws._
import scala.concurrent._
import scala.concurrent.duration._
object Main extends App {
implicit val system = ActorSystem()
implicit val materializer = ActorMaterializer()
import system.dispatcher
// Future[Done] is the materialized value of Sink.foreach,
// emitted when the stream completes
val incoming: Sink[Message, Future[Done]] =
Sink.foreach[Message] {
case message: TextMessage.Strict =>
println(message.text)
}
// flow to use (note: not re-usable!)
val sourceSocketFlow = RestartSource.withBackoff(
minBackoff = 3.seconds,
maxBackoff = 30.seconds,
randomFactor = 0.2,
maxRestarts = 3
) { () =>
Source.tick(2.seconds, 2.seconds, TextMessage("Hello world!!!"))
.viaMat(Http().webSocketClientFlow(WebSocketRequest("ws://127.0.0.1:8080/")))(Keep.right)
}
// the materialized value is a tuple with
// upgradeResponse is a Future[WebSocketUpgradeResponse] that
// completes or fails when the connection succeeds or fails
// and closed is a Future[Done] with the stream completion from the incoming sink
val (upgradeResponse, closed) =
sourceSocketFlow
.toMat(incoming)(Keep.both) // also keep the Future[Done]
.run()
// just like a regular http request we can access response status which is available via upgrade.response.status
// status code 101 (Switching Protocols) indicates that server support WebSockets
val connected = upgradeResponse.flatMap { upgrade =>
if (upgrade.response.status == StatusCodes.SwitchingProtocols) {
Future.successful(Done)
} else {
throw new RuntimeException(s"Connection failed: ${upgrade.response.status}")
}
}
connected.onComplete(println)
closed.foreach(_ => println("closed"))
}
The problem here is, that:
// the materialized value is a tuple with
// upgradeResponse is a Future[WebSocketUpgradeResponse] that
// completes or fails when the connection succeeds or fails
// and closed is a Future[Done] with the stream completion from the incoming sink
val (upgradeResponse, closed) =
sourceSocketFlow
.toMat(incoming)(Keep.both) // also keep the Future[Done]
.run()
does not return Future[WebSocketUpgradeResponse] at first position in the tuple but instead it returns NotUsed.
The question is, how to get the return type Future[WebSocketUpgradeResponse] to identify, that the connection was successfull.

RestartSource#withBackoff accepts a sourceFactory of a type () => Source[T, _] and returns a new source of a type Source[T, NotUsed]. So it's not possible to extract the materialized value from the wrapped source. That's probably because the materialized value will be different on every RestartSource restart.
The question is, how to get the return type Future[WebSocketUpgradeResponse] to identify, that the connection was successful.
If you want to check if the connection was established, and if WebSockets handshake succeeded, you can prematerialize the source using Source#preMaterialize. A slightly modified version of your code would look like the following:
val sourceSocketFlow: Source[Message, NotUsed] = RestartSource.withBackoff(
minBackoff = 3.seconds,
maxBackoff = 30.seconds,
randomFactor = 0.2,
maxRestarts = 3
) { () =>
val (response, source) = Source
.tick(2.seconds, 2.seconds, TextMessage("Hello world!!!"))
.viaMat(Http().webSocketClientFlow(WebSocketRequest("ws://mockbin.org/bin/82b160d4-6c05-4943-908a-a15122603e20")))(Keep.right).preMaterialize()
response.onComplete {
case Failure(e) ⇒
println(s"Connection failed")
case Success(value) ⇒
if (value.response.status == StatusCodes.SwitchingProtocols) {
println("Server supports websockets")
} else {
println("Server does not support websockets")
}
}
source
}
In case the connection fails, or websocket handshake fails, you don't have to do anything. Both cases will be handled by RestartSource.

Related

Akka-Http WebSocket: handling incoming messages very slow

I have a slightly modified example from akka-http docs for WebSocket client-side:
package org.enso
import akka.actor.ActorSystem
import akka.{Done, NotUsed}
import akka.http.scaladsl.Http
import akka.stream.ActorMaterializer
import akka.stream.scaladsl._
import akka.http.scaladsl.model._
import akka.http.scaladsl.model.ws._
import akka.pattern.after
import scala.concurrent.duration._
import scala.concurrent.{Future, Promise}
object SingleWebSocketRequest {
def main(args: Array[String]): Unit = {
implicit val system: ActorSystem = ActorSystem()
implicit val materializer: ActorMaterializer = ActorMaterializer()
import system.dispatcher
// print each incoming text message
val printSink: Sink[Message, Future[Done]] =
Sink.foreach { _ =>
println("foo")
}
val s1: Source[Message, NotUsed] =
Source(Stream.fill(100){
TextMessage("""{"method":"ping","responseSize":4}""")
})
// the Future[Done] is the materialized value of Sink.foreach
// and it is completed when the stream completes
val flow: Flow[Message, Message, Future[Done]] =
Flow.fromSinkAndSourceMat(printSink, s1)(Keep.left)
// upgradeResponse is a Future[WebSocketUpgradeResponse] that
// completes or fails when the connection succeeds or fails
// and closed is a Future[Done] representing the stream completion from above
val (upgradeResponse, closed) =
Http().singleWebSocketRequest(WebSocketRequest("ws://localhost:8080"), flow)
val connected = upgradeResponse.map { upgrade =>
// just like a regular http request we can access response status which is available via upgrade.response.status
// status code 101 (Switching Protocols) indicates that server support WebSockets
if (upgrade.response.status == StatusCodes.SwitchingProtocols) {
Done
} else {
throw new RuntimeException(s"Connection failed: ${upgrade.response.status}")
}
}
// in a real application you would not side effect here
// and handle errors more carefully
connected.onComplete(println)
closed.foreach {_ =>
println("closed")
}
}
}
This code sends requests to a Websocket server-side, which works fine with another JS client. The server-side works fine, sending many responses at once, as evident from this Wireshark dump
E1##ÿ K^B!¹åª&
<·*<·#/{
"response": "aaaa",
"status": "success"
}/{
"response": "aaaa",
"status": "success"
}/{
"response": "aaaa",
"status": "success"
}/...
But printing "foo", which is the side effect of handling server messages in the Sink, only happens every ~5s, which is very slow compared with what is observed in Wireshark.
How would I explain / debug this? Could it be something to do with buffers / back pressure?
Ok, with this Sink everything works nice:
val printSink: Sink[Message, Future[Done]] =
Sink.foreachAsync(1) {
case x: TextMessage => x.toStrict(FiniteDuration(100, "ms")).map { _ =>
logger.debug("foo")
receive = System.currentTimeMillis() :: receive
}
}
As it turns out, whenever working with akka – always consume all the input. This WS client API actually seems strange to me, for the http clients you get a helper method like discardResponseBytes precisely for this case. Anyway – consume the input, always

How to broadcast the received messages to two different flows

How to broadcast the received messages to two different flows
I am using akka stream websocket client to request and receive the data websocket server.
With the received data from the websocket, I would like to broadcast into two different flows.
The image below, should clarify the scenario:
As you can see on the image, it should be broadcasted to two different flows subsequently to seperate sink.
The websocket client can be created as the following:
import akka.actor.ActorSystem
import akka.Done
import akka.http.scaladsl.Http
import akka.stream.ActorMaterializer
import akka.stream.scaladsl._
import akka.http.scaladsl.model._
import akka.http.scaladsl.model.ws._
import scala.concurrent.Future
object WebSocketClientFlow {
def main(args: Array[String]) = {
implicit val system = ActorSystem()
implicit val materializer = ActorMaterializer()
import system.dispatcher
// Future[Done] is the materialized value of Sink.foreach,
// emitted when the stream completes
val incoming: Sink[Message, Future[Done]] =
Sink.foreach[Message] {
case message: TextMessage.Strict =>
println(message.text)
}
// send this as a message over the WebSocket
val outgoing = Source.single(TextMessage("hello world!"))
// flow to use (note: not re-usable!)
val webSocketFlow = Http().webSocketClientFlow(WebSocketRequest("ws://echo.websocket.org"))
// the materialized value is a tuple with
// upgradeResponse is a Future[WebSocketUpgradeResponse] that
// completes or fails when the connection succeeds or fails
// and closed is a Future[Done] with the stream completion from the incoming sink
val (upgradeResponse, closed) =
outgoing
.viaMat(webSocketFlow)(Keep.right) // keep the materialized Future[WebSocketUpgradeResponse]
.toMat(incoming)(Keep.both) // also keep the Future[Done]
.run()
// just like a regular http request we can access response status which is available via upgrade.response.status
// status code 101 (Switching Protocols) indicates that server support WebSockets
val connected = upgradeResponse.flatMap { upgrade =>
if (upgrade.response.status == StatusCodes.SwitchingProtocols) {
Future.successful(Done)
} else {
throw new RuntimeException(s"Connection failed: ${upgrade.response.status}")
}
}
// in a real application you would not side effect here
connected.onComplete(println)
closed.foreach(_ => println("closed"))
}
}
You can use SinkShape to get the required flow
Sink.fromGraph(GraphDSL.create(){
implicit b =>
val bcast = b.add(Broadcast[Message](2))
val flow1 = b.add(Flow[Message].map(m => m))
val flow2 = b.add(Flow[Message].map(m => m ))
val sink1 = b.add(Sink.foreach(println))
val sink2 = b.add(Sink.foreach(println))
bcast ~> flow1 ~> sink1
bcast ~> flow2 ~> sink2
SinkShape(bcast.in)
})
The entire code is
implicit val system = ActorSystem()
implicit val materializer = ActorMaterializer()
import system.dispatcher
// Future[Done] is the materialized value of Sink.foreach,
// emitted when the stream completes
val incomingSink = Sink.fromGraph(GraphDSL.create() {
implicit b =>
import GraphDSL.Implicits._
val bcast = b.add(Broadcast[Message](2))
val flow1 = b.add(Flow[Message].map(m => m))
val flow2 = b.add(Flow[Message].map(m => m ))
val sink1 = b.add(Sink.head[Message])
val sink2 = b.add(Sink.head[Message])
bcast ~> flow1 ~> sink1
bcast ~> flow2 ~> sink2
SinkShape(bcast.in)
}).mapMaterializedValue(_ => Future(Done))
// send this as a message over the WebSocket
val outgoing = Source.single(TextMessage("hello world!"))
// flow to use (note: not re-usable!)
val webSocketFlow = Http().webSocketClientFlow(WebSocketRequest("ws://echo.websocket.org"))
// the materialized value is a tuple with
// upgradeResponse is a Future[WebSocketUpgradeResponse] that
// completes or fails when the connection succeeds or fails
// and closed is a Future[Done] with the stream completion from the incoming sink
val (upgradeResponse, closed) =
outgoing
.viaMat(webSocketFlow)(Keep.right) // keep the materialized Future[WebSocketUpgradeResponse]
.toMat(incomingSink)(Keep.both) // also keep the Future[Done]
.run()
// just like a regular http request we can access response status which is available via upgrade.response.status
// status code 101 (Switching Protocols) indicates that server support WebSockets
val connected = upgradeResponse.flatMap { upgrade =>
if (upgrade.response.status == StatusCodes.SwitchingProtocols) {
Future.successful(Done)
} else {
throw new RuntimeException(s"Connection failed: ${upgrade.response.status}")
}
}
// in a real application you would not side effect here
connected.onComplete(println)
closed.foreach(_ => println("closed"))

Does composite flow make loop?

I am trying to understand, how the following code snippet works:
val flow: Flow[Message, Message, Future[Done]] =
Flow.fromSinkAndSourceMat(printSink, helloSource)(Keep.left)
Two guys gave a very wonderful explanation on this thread. I understand the concept of the Composite flow, but how does it work on the websocket client.
Consider the following code:
import akka.actor.ActorSystem
import akka.{ Done, NotUsed }
import akka.http.scaladsl.Http
import akka.stream.ActorMaterializer
import akka.stream.scaladsl._
import akka.http.scaladsl.model._
import akka.http.scaladsl.model.ws._
import scala.concurrent.Future
object SingleWebSocketRequest {
def main(args: Array[String]) = {
implicit val system = ActorSystem()
implicit val materializer = ActorMaterializer()
import system.dispatcher
// print each incoming strict text message
val printSink: Sink[Message, Future[Done]] =
Sink.foreach {
case message: TextMessage.Strict =>
println(message.text)
}
val helloSource: Source[Message, NotUsed] =
Source.single(TextMessage("hello world!"))
// the Future[Done] is the materialized value of Sink.foreach
// and it is completed when the stream completes
val flow: Flow[Message, Message, Future[Done]] =
Flow.fromSinkAndSourceMat(printSink, helloSource)(Keep.left)
// upgradeResponse is a Future[WebSocketUpgradeResponse] that
// completes or fails when the connection succeeds or fails
// and closed is a Future[Done] representing the stream completion from above
val (upgradeResponse, closed) =
Http().singleWebSocketRequest(WebSocketRequest("ws://echo.websocket.org"), flow)
val connected = upgradeResponse.map { upgrade =>
// just like a regular http request we can access response status which is available via upgrade.response.status
// status code 101 (Switching Protocols) indicates that server support WebSockets
if (upgrade.response.status == StatusCodes.SwitchingProtocols) {
Done
} else {
throw new RuntimeException(s"Connection failed: ${upgrade.response.status}")
}
}
// in a real application you would not side effect here
// and handle errors more carefully
connected.onComplete(println)
closed.foreach(_ => println("closed"))
}
}
It is a websocket client, that send a message to the websocket server and the printSink receives it and print it out.
How can it be, that printSink receives messages, there is no a connection between the Sink and Source.
Is it like a loop?
Stream flow is from left to right, how it comes that the Sink can consume messages from websocket server?
Flow.fromSinkAndSourceMat puts an independent Sink and a Source to a shape of the Flow. Elements going into that Sink do not end up at the Source.
From the Websocket client API perspective, it needs a Source from which requests will be sent to the server and a Sink that it will send the responses to. The singleWebSocketRequest could take a Source and a Sink separately, but that would be a bit more verbose API.
Here is a shorter example that demonstrates the same as in your code snippet but is runnable, so you can play around with it:
import akka._
import akka.actor._
import akka.stream._
import akka.stream.scaladsl._
implicit val sys = ActorSystem()
implicit val mat = ActorMaterializer()
def openConnection(userFlow: Flow[String, String, NotUsed])(implicit mat: Materializer) = {
val processor = Flow[String].map(_.toUpperCase)
processor.join(userFlow).run()
}
val requests = Source(List("one", "two", "three"))
val responses = Sink.foreach(println)
val userFlow = Flow.fromSinkAndSource(responses, requests)
openConnection(userFlow)

akka stream consume web socket

Getting started with akka-streams I want to build a simple example.
In chrome using a web socket plugin I simply can connect to a stream like this one https://blockchain.info/api/api_websocket via wss://ws.blockchain.info/inv and sending 2 commands
{"op":"ping"}
{"op":"unconfirmed_sub"}
will stream the results in chromes web socket plugin window.
I tried to implement the same functionality in akka streams but am facing some problems:
2 commands are executed, but I actually do not get the streaming output
the same command is executed twice (the ping command)
When following the tutorial of http://doc.akka.io/docs/akka/2.4.7/scala/http/client-side/websocket-support.html or http://doc.akka.io/docs/akka-http/10.0.0/scala/http/client-side/websocket-support.html#half-closed-client-websockets
Here is my adaption below:
object SingleWebSocketRequest extends App {
implicit val system = ActorSystem()
implicit val materializer = ActorMaterializer()
import system.dispatcher
// print each incoming strict text message
val printSink: Sink[Message, Future[Done]] =
Sink.foreach {
case message: TextMessage.Strict =>
println(message.text)
}
val commandMessages = Seq(TextMessage("{\"op\":\"ping\"}"), TextMessage("{\"op\":\"unconfirmed_sub\"}"))
val helloSource: Source[Message, NotUsed] = Source(commandMessages.to[scala.collection.immutable.Seq])
// the Future[Done] is the materialized value of Sink.foreach
// and it is completed when the stream completes
val flow: Flow[Message, Message, Future[Done]] =
Flow.fromSinkAndSourceMat(printSink, helloSource)(Keep.left)
// upgradeResponse is a Future[WebSocketUpgradeResponse] that
// completes or fails when the connection succeeds or fails
// and closed is a Future[Done] representing the stream completion from above
val (upgradeResponse, closed) =
Http().singleWebSocketRequest(WebSocketRequest("wss://ws.blockchain.info/inv"), flow)
val connected = upgradeResponse.map { upgrade =>
// just like a regular http request we can access response status which is available via upgrade.response.status
// status code 101 (Switching Protocols) indicates that server support WebSockets
if (upgrade.response.status == StatusCodes.SwitchingProtocols) {
Done
} else {
throw new RuntimeException(s"Connection failed: ${upgrade.response.status}")
}
}
// in a real application you would not side effect here
// and handle errors more carefully
connected.onComplete(println) // TODO why do I not get the same output as in chrome?
closed.foreach(_ => println("closed"))
}
when using the flow version from http://doc.akka.io/docs/akka-http/10.0.0/scala/http/client-side/websocket-support.html#websocketclientflow modified as outlined below, again, the result is twice the same output:
{"op":"pong"}
{"op":"pong"}
See the code:
object WebSocketClientFlow extends App {
implicit val system = ActorSystem()
implicit val materializer = ActorMaterializer()
import system.dispatcher
// Future[Done] is the materialized value of Sink.foreach,
// emitted when the stream completes
val incoming: Sink[Message, Future[Done]] =
Sink.foreach[Message] {
case message: TextMessage.Strict =>
println(message.text)
}
// send this as a message over the WebSocket
val commandMessages = Seq(TextMessage("{\"op\":\"ping\"}"), TextMessage("{\"op\":\"unconfirmed_sub\"}"))
val outgoing: Source[Message, NotUsed] = Source(commandMessages.to[scala.collection.immutable.Seq])
// val outgoing = Source.single(TextMessage("hello world!"))
// flow to use (note: not re-usable!)
val webSocketFlow = Http().webSocketClientFlow(WebSocketRequest("wss://ws.blockchain.info/inv"))
// the materialized value is a tuple with
// upgradeResponse is a Future[WebSocketUpgradeResponse] that
// completes or fails when the connection succeeds or fails
// and closed is a Future[Done] with the stream completion from the incoming sink
val (upgradeResponse, closed) =
outgoing
.viaMat(webSocketFlow)(Keep.right) // keep the materialized Future[WebSocketUpgradeResponse]
.toMat(incoming)(Keep.both) // also keep the Future[Done]
.run()
// just like a regular http request we can access response status which is available via upgrade.response.status
// status code 101 (Switching Protocols) indicates that server support WebSockets
val connected = upgradeResponse.flatMap { upgrade =>
if (upgrade.response.status == StatusCodes.SwitchingProtocols) {
Future.successful(Done)
} else {
throw new RuntimeException(s"Connection failed: ${upgrade.response.status}")
}
}
// in a real application you would not side effect here
connected.onComplete(println)
closed.foreach(_ => {
println("closed")
system.terminate
})
}
How can I achieve the same result as in chrome
display print of subscribed stream
at best periodically send update (ping statements) as outlined in https://blockchain.info/api/api via {"op":"ping"}messages
Note, I am using akka in version 2.4.17 and akka-http in version 10.0.5
A couple of things I notice are:
1) you need to consume all types of incoming messages, not only the TextMessage.Strict kind. The blockchain stream is definitely a Streamed message, as it contains loads of text and it will be delivered in chunks over the network. A more complete incoming Sink could be:
val incoming: Sink[Message, Future[Done]] =
Flow[Message].mapAsync(4) {
case message: TextMessage.Strict =>
println(message.text)
Future.successful(Done)
case message: TextMessage.Streamed =>
message.textStream.runForeach(println)
case message: BinaryMessage =>
message.dataStream.runWith(Sink.ignore)
}.toMat(Sink.last)(Keep.right)
2) your source of 2 elements might complete too early, i.e. before the websocket responses come back. You can concatenate a Source.maybe by doing
val outgoing: Source[Strict, Promise[Option[Nothing]]] =
Source(commandMessages.to[scala.collection.immutable.Seq]).concatMat(Source.maybe)(Keep.right)
and then
val ((completionPromise, upgradeResponse), closed) =
outgoing
.viaMat(webSocketFlow)(Keep.both)
.toMat(incoming)(Keep.both)
.run()
by keeping the materialized promise non-complete, you keep the source open and avoid the flow shutdown.

How to use Akka-HTTP client websocket send message

I'm trying client-side websocket by following doc at webSocketClientFlow.
sample code is:
import akka.actor.ActorSystem
import akka.Done
import akka.http.scaladsl.Http
import akka.stream.ActorMaterializer
import akka.stream.scaladsl._
import akka.http.scaladsl.model._
import akka.http.scaladsl.model.ws._
import scala.concurrent.Future
object WebSocketClientFlow {
def main(args: Array[String]) = {
implicit val system = ActorSystem()
implicit val materializer = ActorMaterializer()
import system.dispatcher
// Future[Done] is the materialized value of Sink.foreach,
// emitted when the stream completes
val incoming: Sink[Message, Future[Done]] =
Sink.foreach[Message] {
case message: TextMessage.Strict =>
println(message.text)
}
// send this as a message over the WebSocket
val outgoing = Source.single(TextMessage("hello world!"))
// flow to use (note: not re-usable!)
val webSocketFlow = Http().webSocketClientFlow(WebSocketRequest("ws://echo.websocket.org"))
// the materialized value is a tuple with
// upgradeResponse is a Future[WebSocketUpgradeResponse] that
// completes or fails when the connection succeeds or fails
// and closed is a Future[Done] with the stream completion from the incoming sink
val (upgradeResponse, closed) =
outgoing
.viaMat(webSocketFlow)(Keep.right) // keep the materialized Future[WebSocketUpgradeResponse]
.toMat(incoming)(Keep.both) // also keep the Future[Done]
.run()
// just like a regular http request we can access response status which is available via upgrade.response.status
// status code 101 (Switching Protocols) indicates that server support WebSockets
val connected = upgradeResponse.flatMap { upgrade =>
if (upgrade.response.status == StatusCodes.SwitchingProtocols) {
Future.successful(Done)
} else {
throw new RuntimeException(s"Connection failed: ${upgrade.response.status}")
}
}
// in a real application you would not side effect here
connected.onComplete(println)
closed.foreach(_ => println("closed"))
}
}
after had connection upgraded, how to use the connection send message to websocket server side?
I noticed from the doc:
The Flow that is returned by this method can only be materialized once. For each request a new flow must be acquired by calling the method again.
still confused, why we need construct the flow many times since an upgraded connection alrady ready.
You can create an actor based source and send new messages over the established websocket connection.
val req = WebSocketRequest(uri = "ws://127.0.0.1/ws")
val webSocketFlow = Http().webSocketClientFlow(req)
val messageSource: Source[Message, ActorRef] =
Source.actorRef[TextMessage.Strict](bufferSize = 10, OverflowStrategy.fail)
val messageSink: Sink[Message, NotUsed] =
Flow[Message]
.map(message => println(s"Received text message: [$message]"))
.to(Sink.ignore)
val ((ws, upgradeResponse), closed) =
messageSource
.viaMat(webSocketFlow)(Keep.both)
.toMat(messageSink)(Keep.both)
.run()
val connected = upgradeResponse.flatMap { upgrade =>
if (upgrade.response.status == StatusCodes.SwitchingProtocols) {
Future.successful(Done)
} else {
throw new RuntimeException(s"Connection failed: ${upgrade.response.status}")
}
}
ws ! TextMessage.Strict("Hello World")
ws ! TextMessage.Strict("Hi")
ws ! TextMessage.Strict("Yay!")
`