How to listen websocket server close events using akka-http websocket client - scala

I have websocket client connected to akka-http websocket server , how can i listen to connection close events happened on server( i.e server shutdown/ server closed websocket connection) ?
object Client extends App {
implicit val actorSystem = ActorSystem("akka-system")
implicit val flowMaterializer = ActorMaterializer()
val config = actorSystem.settings.config
val interface = config.getString("app.interface")
val port = config.getInt("app.port")
// print each incoming strict text message
val printSink: Sink[Message, Future[Done]] =
Sink.foreach {
case message: TextMessage.Strict =>
println(message.text)
case _ => {
sourceQueue.map(q => {
println(s"offering message on client")
q.offer(TextMessage("received unknown"))
})
println(s"received unknown message format")
}
}
val (source, sourceQueue) = {
val p = Promise[SourceQueue[Message]]
val s = Source.queue[Message](Int.MaxValue, OverflowStrategy.backpressure).mapMaterializedValue(m => {
p.trySuccess(m)
m
})
.keepAlive(FiniteDuration(1, TimeUnit.SECONDS), () => TextMessage.Strict("Heart Beat"))
(s, p.future)
}
val flow =
Flow.fromSinkAndSourceMat(printSink, source)(Keep.right)
val (upgradeResponse, sourceClose) =
Http().singleWebSocketRequest(WebSocketRequest("ws://localhost:8080/ws-echo"), flow)
val connected = upgradeResponse.map { upgrade =>
// just like a regular http request we can get 404 NotFound,
// with a response body, that will be available from upgrade.response
if (upgrade.response.status == StatusCodes.SwitchingProtocols || upgrade.response.status == StatusCodes.SwitchingProtocols) {
Done
} else {
throw new RuntimeException(s"Connection failed: ${upgrade.response.status}")
}
}
connected.onComplete(println)
}

Websocket connection termination is modeled as a regular stream completion, therefore in your case you can use the materialized Future[Done] yielded by Sink.foreach:
val flow = Flow.fromSinkAndSourceMat(printSink, source)(Keep.both)
val (upgradeResponse, (sinkClose, sourceClose)) =
Http().singleWebSocketRequest(..., flow)
sinkClose.onComplete {
case Success(_) => println("Connection closed gracefully")
case Failure(e) => println("Connection closed with an error: $e")
}

Related

how to use actors in akka client side websockets

i am using akka client websockets https://doc.akka.io/docs/akka-http/current/client-side/websocket-support.html
i have a server which takes requests and respond in json
this is the pattern of my request and response
request #1
{
"janus" : "create",
"transaction" : "<random alphanumeric string>"
}
response #1
{
"janus": "success",
"session_id": 2630959283560140,
"transaction": "asqeasd4as3d4asdasddas",
"data": {
"id": 4574061985075210
}
}
then based on response #1 i need to initiate request #2 and upon receiving response #2 i need to initiate request #3 and so on
for example
then based on id 4574061985075210 i will send request #2 and receive it response
request # 2 {
}
response # 2 {
}
----
how can i use the actors with source and sink and re use the flow
here is my initial code
import akka.http.scaladsl.model.ws._
import scala.concurrent.Future
object WebSocketClientFlow {
def main(args: Array[String]) = {
implicit val system = ActorSystem()
implicit val materializer = ActorMaterializer()
import system.dispatcher
val incoming: Sink[Message, Future[Done]] =
Sink.foreach[Message] {
case message: TextMessage.Strict =>
println(message.text)
//suppose here based on the server response i need to send another message to the server and so on do i need to repeat this same code here again ?????
}
val outgoing = Source.single(TextMessage("hello world!"))
val webSocketFlow = Http().webSocketClientFlow(WebSocketRequest("ws://echo.websocket.org"))
val (upgradeResponse, closed) =
outgoing
.viaMat(webSocketFlow)(Keep.right) // keep the materialized Future[WebSocketUpgradeResponse]
.toMat(incoming)(Keep.both) // also keep the Future[Done]
.run()
val connected = upgradeResponse.flatMap { upgrade =>
if (upgrade.response.status == StatusCodes.SwitchingProtocols) {
Future.successful(Done)
} else {
throw new RuntimeException(s"Connection failed: ${upgrade.response.status}")
}
}
connected.onComplete(println)
closed.foreach(_ => println("closed"))
}
}
and here i used Source.ActorRef
val url = "ws://0.0.0.0:8188"
val req = WebSocketRequest(url, Nil, Option("janus-protocol"))
implicit val system = ActorSystem()
implicit val materializer = ActorMaterializer()
import system.dispatcher
val webSocketFlow = Http().webSocketClientFlow(req)
val messageSource: Source[Message, ActorRef] =
Source.actorRef[TextMessage.Strict](bufferSize = 10, OverflowStrategy.fail)
val messageSink: Sink[Message, NotUsed] =
Flow[Message]
.map(message => println(s"Received text message: [$message]"))
.to(Sink.ignore)
val ((ws, upgradeResponse), closed) =
messageSource
.viaMat(webSocketFlow)(Keep.both)
.toMat(messageSink)(Keep.both)
.run()
val connected = upgradeResponse.flatMap { upgrade =>
if (upgrade.response.status == StatusCodes.SwitchingProtocols) {
Future.successful(Done)
} else {
throw new RuntimeException(s"Connection failed: ${upgrade.response.status}")
}
}
val source =
"""{ "janus": "create", "transaction":"d1403sa54a5s3d4as3das"}"""
val jsonAst = source.parseJson
ws ! TextMessage.Strict(jsonAst.toString())
now i need help in
how can i initiate the second request here because i need the "id" returned from the server to initiate request #2

how to use Akka http web-sockets

i have started using akka client websockets
i have a question regrading multiple request and response as here the document says
Note
The Flow that is returned by this method can only be materialized once. For each request a new flow must be acquired by calling the method again
and here is the sample code this example is for sending a message and printing any incoming message:
so i have a question if i have following scenario
request 1
{
"janus" : "create",
"transaction" : "123"
}
response #1
{
"janus": "success",
"transaction": "123",
"data": {
"id": 4574061985075210
}
}
and
request # 02 {
"janus": "attach","session_id":${sessionId},"plugin":"janus.plugin.echotest","transaction":"asqeasd4as3d4asdasddas"
}
response # 02 {
}
----
i need to instantiate request #2 based on response #1 so will the following code would be assumed correct ? i have a doubt becuase i am again opening web-sockets connection and according to my knowledge web-sockets connection is opened once and we send and receive messages over that connection but akka web-sockets documents does not support that (at least that is what i understood so far i maybe wrong that's why i am here )
here is my code
val source =
"""{ "janus": "create", "transaction":"d1403sa54a5s3d4as3das"}"""
val jsonAst = source.parseJson
val responseSession: Sink[Message, Future[Done]] =
Sink.foreach[Message] {
case message: TextMessage.Strict =>
println("message response " + message.text)
val strJson = message.text
val jsonResponse = strJson.parseJson
val jsonObj = jsonResponse.asJsObject
val janus = jsonObj.fields("janus").convertTo[String]
val data = jsonObj.fields("data").asJsObject
val sessionIDInt = sessionID.convertTo[BigInt]
getPluginData(sessionIDInt)//here i am repeating the code of webSocketClientFlow https://doc.akka.io/docs/akka-http/current/client-side/websocket-support.html#websocketclientflow
//is this the above approach is correct?
}
val flow: Flow[Message, Message, Promise[Option[Message]]] =
Flow.fromSinkAndSourceMat(
responseSession,
Source.single(TextMessage(jsonAst.toString())).concatMat(Source.maybe[Message])(Keep.right))(Keep.right)
val (upgradeResponse, closed) =
Http().singleWebSocketRequest(WebSocketRequest(url, Nil, Option("janus-protocol")), flow)
def getPluginData(sessionId: BigInt): Unit = {
val responsePlugin: Sink[Message, Future[Done]] =
Sink.foreach[Message] {
case message: TextMessage.Strict =>
println("plugin response " + message.text)
val strJson = message.text
val jsonResponse = strJson.parseJson
val jsonObj = jsonResponse.asJsObject
val janus = jsonObj.fields("janus").convertTo[String]
val sessionID = jsonObj.fields("session_id") .convertTo[BigInt]
val data = jsonObj.fields("data").asJsObject
val handelID = data.fields("id")
val handleIdInt = handelID.convertTo[BigInt]
//here i will call another method which has its own sink and source and the same code again ->getHandleDetails(sessionID,handleIdInt)
}
val requestPluginJson = s"""{ "janus": "attach","session_id":${sessionId},"plugin":"janus.plugin.echotest","transaction":"asqeasd4as3d4asdasddas"}"""
val jsonAstPlugin = requestPluginJson.parseJson // or JsonParser(source)
val requestPlugin = Source.single(TextMessage(jsonAstPlugin.toString()))
val webSocketFlow = Http().webSocketClientFlow(WebSocketRequest(url, Nil, Option("janus-protocol")))
val (upgradeResponse, closed) =
requestPlugin
.viaMat(webSocketFlow)(Keep.right) // keep the materialized Future[WebSocketUpgradeResponse]
.toMat(responsePlugin)(Keep.both) // also keep the Future[Done]
.run()
val connected = upgradeResponse.flatMap { upgrade =>
if (upgrade.response.status == StatusCodes.SwitchingProtocols) {
Future.successful(Done)
} else {
throw new RuntimeException(s"Connection failed: ${upgrade.response.status}")
}
}
connected.onComplete(println)
closed.foreach(_ => println("closed plugin"))
}

Play framework / Akka Streams: Detecting when a WebSocket has closed

When handling WebSockets with Akka Streams directly, I didn't find a proper way to know when the client disconnects (either normally or due to a crash or timeout). I'm using a basic example like the one from the official documentation:
import play.api.mvc._
import akka.stream.scaladsl._
def socket = WebSocket.accept[String, String] { request =>
// Log events to the console
val in = Sink.foreach[String](println)
// Send a single 'Hello!' message and then leave the socket open
val out = Source.single("Hello!").concat(Source.maybe)
Flow.fromSinkAndSource(in, out)
}
I need to know when a client is no longer connected.
Use watchTermination:
def socket = WebSocket.accept[String, String] { request =>
val in = Sink.foreach[String](println)
val out = Source.single("Hello!").concat(Source.maybe)
Flow.fromSinkAndSource(in, out)
.watchTermination() { (_, fut) =>
fut onComplete {
case Success(_) =>
println("Client disconnected")
case Failure(t) =>
println(s"Disconnection failure: ${t.getMessage}")
}
}
}

Akka-http first websocket client only receives the data once from a kafka topic

I am using akka-http websocket to push messages from a kafka topic to websocket clients.
For this purpose, i created a plain kafka consumer (using akka-streams-kafka connector) with offset set to "earliest" so that every new websocket client connecting gets all the data from the beginning.
The problem is that the first connected websocket client gets all the data and other ws clients (connecting after the first client has got all the data) do not get any. The kafka topic has 1million records.
I am using the BroadcastHub from Akka-streams.
Appreciate any suggestions.
lazy private val kafkaPlainSource: Source[String, NotUsed] = {
val consumerSettings = ConsumerSettings(system, new StringDeserializer, new StringDeserializer)
.withBootstrapServers(KAFKA_BROKERS)
.withGroupId(UUID.randomUUID().toString)
.withProperty(ConsumerConfig.AUTO_OFFSET_RESET_CONFIG, "earliest")
val kafkaSource = Consumer.plainSource(consumerSettings, Subscriptions.topics(KAFKA_TOPIC))
.mapAsync(PARALLELISM) { cr =>
Future {
cr.value
}
}
kafkaSource.toMat(BroadcastHub.sink)(Keep.right).run
}
def logicFlow: Flow[String, String, NotUsed] =
Flow.fromSinkAndSourceCoupled(Sink.ignore, kafkaSource)
val websocketFlow: Flow[Message, Message, Any] = {
Flow[Message]
.map {
case TextMessage.Strict(msg) => msg
case _ => println("ignore streamed message")
}
.via(logicFlow)
.map { msg: String => TextMessage.Strict(msg) }
}
lazy private val streamRoute =
path("stream") {
handleWebSocketMessages {
websocketFlow
.watchTermination() { (_, done) =>
done.onComplete {
case Success(_) =>
log.info("Stream route completed successfully")
case Failure(ex) =>
log.error(s"Stream route completed with failure : $ex")
}
}
}
}
def startServer(): Unit = {
bindingFuture = Http().bindAndHandle(wsRoutes, HOST, PORT)
log.info(s"Server online at http://localhost:9000/")
}
def stopServer(): Unit = {
bindingFuture
.flatMap(_.unbind())
.onComplete{
_ => system.terminate()
log.info("terminated")
}
}

How does one measure throughput of Akka WebSocket stream?

I am new to Akka and developed a sample Akka WebSocket server that streams a file's contents to clients using BroadcastHub (based on a sample from the Akka docs).
How can I measure the throughput (messages/second), assuming the clients are consuming as fast as the server?
// file source
val fileSource = FileIO.fromPath(Paths.get(path)
// Akka file source
val theFileSource = fileSource
.toMat(BroadcastHub.sink)(Keep.right)
.run
//Akka kafka file source
lazy val kafkaSourceActorStream = {
val (kafkaSourceActorRef, kafkaSource) = Source.actorRef[String](Int.MaxValue, OverflowStrategy.fail)
.toMat(BroadcastHub.sink)(Keep.both).run()
Consumer.plainSource(consumerSettings, Subscriptions.topics("perf-test-topic"))
.runForeach(record => kafkaSourceActorRef ! record.value().toString)
}
def logicFlow: Flow[String, String, NotUsed] = Flow.fromSinkAndSource(Sink.ignore, theFileSource)
val websocketFlow: Flow[Message, Message, Any] = {
Flow[Message]
.collect {
case TextMessage.Strict(msg) => Future.successful(msg)
case _ => println("ignore streamed message")
}
.mapAsync(parallelism = 2)(identity)
.via(logicFlow)
.map { msg: String => TextMessage.Strict(msg) }
}
val fileRoute =
path("file") {
handleWebSocketMessages(websocketFlow)
}
}
def startServer(): Unit = {
bindingFuture = Http().bindAndHandle(wsRoutes, HOST, PORT)
log.info(s"Server online at http://localhost:9000/")
}
def stopServer(): Unit = {
bindingFuture
.flatMap(_.unbind())
.onComplete{
_ => system.terminate()
log.info("terminated")
}
}
//ws client
def connectToWebSocket(url: String) = {
println("Connecting to websocket: " + url)
val (upgradeResponse, closed) = Http().singleWebSocketRequest(WebSocketRequest(url), websocketFlow)
val connected = upgradeResponse.flatMap{ upgrade =>
if(upgrade.response.status == StatusCodes.SwitchingProtocols )
{
println("Web socket connection success")
Future.successful(Done)
}else {
println("Web socket connection failed with error: {}", upgrade.response.status)
throw new RuntimeException(s"Web socket connection failed: ${upgrade.response.status}")
}
}
connected.onComplete { msg =>
println(msg)
}
}
def websocketFlow: Flow[Message, Message, _] = {
Flow.fromSinkAndSource(printFlowRate, Source.maybe)
}
lazy val printFlowRate =
Flow[Message]
.alsoTo(fileSink("output.txt"))
.via(flowRate(1.seconds))
.to(Sink.foreach(rate => println(s"$rate")))
def flowRate(sampleTime: FiniteDuration) =
Flow[Message]
.conflateWithSeed(_ ⇒ 1){ case (acc, _) ⇒ acc + 1 }
.zip(Source.tick(sampleTime, sampleTime, NotUsed))
.map(_._1.toDouble / sampleTime.toUnit(SECONDS))
def fileSink(file: String): Sink[Message, Future[IOResult]] = {
Flow[Message]
.map{
case TextMessage.Strict(msg) => msg
case TextMessage.Streamed(stream) => stream.runFold("")(_ + _).flatMap(msg => Future.successful(msg))
}
.map(s => ByteString(s + "\n"))
.toMat(FileIO.toFile(new File(file)))(Keep.right)
}
You could attach a throughput-measuring stream to your existing stream. Here is an example, inspired by this answer, that prints the number of integers that are emitted from the upstream source every second:
val rateSink = Flow[Int]
.conflateWithSeed(_ => 0){ case (acc, _) => acc + 1 }
.zip(Source.tick(1.second, 1.second, NotUsed))
.map(_._1)
.toMat(Sink.foreach(i => println(s"$i elements/second")))(Keep.right)
In the following example, we attach the above sink to a source that emits the integers 1 to 10 million. To prevent the rate-measuring stream from interfering with the main stream (which, in this case, simply converts every integer to a string and returns the last string processed as part of the materialized value), we use wireTapMat:
val (rateFut, mainFut) = Source(1 to 10000000)
.wireTapMat(rateSink)(Keep.right)
.map(_.toString)
.toMat(Sink.last[String])(Keep.both)
.run() // (Future[Done], Future[String])
rateFut onComplete {
case Success(x) => println(s"rateFut completed: $x")
case Failure(_) =>
}
mainFut onComplete {
case Success(s) => println(s"mainFut completed: $s")
case Failure(_) =>
}
Running the above sample prints something like the following:
0 elements/second
2597548 elements/second
3279052 elements/second
mainFut completed: 10000000
3516141 elements/second
607254 elements/second
rateFut completed: Done
If you don't need a reference to the materialized value of rateSink, use wireTap instead of wireTapMat. For example, attaching rateSink to your WebSocket flow could look like the following:
val websocketFlow: Flow[Message, Message, Any] = {
Flow[Message]
.wireTap(rateSink) // <---
.collect {
case TextMessage.Strict(msg) => Future.successful(msg)
case _ => println("ignore streamed message")
}
.mapAsync(parallelism = 2)(identity)
.via(logicFlow)
.map { msg: String => TextMessage.Strict(msg) }
}
wireTap is defined on both Source and Flow.
Where I last worked I implemented a performance benchmark of this nature.
Basically, it meant creating a simple client app that consumes messages from the websocket and outputs some metrics. The natural choice was to implement the client using akka-http client-side support for websockets. See:
https://doc.akka.io/docs/akka-http/current/client-side/websocket-support.html#singlewebsocketrequest
Then we used the micrometer library to expose metrics to Prometheus, which was our tool of choice for reporting and charting.
https://github.com/micrometer-metrics
https://micrometer.io/docs/concepts#_meters