I'm quite new in Akka Stream and I'd like to learn how handle a TCP socket for a project of mine. I took this piece of code from the Akka Stream official documentation.
import akka.stream.scaladsl.Framing
val connections: Source[IncomingConnection, Future[ServerBinding]] =
Tcp().bind(host, port)
connections.runForeach { connection =>
println(s"New connection from: ${connection.remoteAddress}")
val echo = Flow[ByteString]
.via(Framing.delimiter(ByteString("\n"), maximumFrameLength = 256, allowTruncation = true))
.map(_.utf8String)
.map(_ + "!!!\n")
.map(ByteString(_))
connection.handleWith(echo)
}
If I connect from the terminal using netcat I can see that the Akka Stream TCP socket works as expected. I also found out that If I need to close the connection using an user message, I can use a takeWhile as follow
import akka.stream.scaladsl.Framing
val connections: Source[IncomingConnection, Future[ServerBinding]] =
Tcp().bind(host, port)
connections.runForeach { connection =>
println(s"New connection from: ${connection.remoteAddress}")
val echo = Flow[ByteString]
.via(Framing.delimiter(ByteString("\n"), maximumFrameLength = 256, allowTruncation = true))
.map(_.utf8String)
.takeWhile(_.toLowerCase.trim != "exit") // < - - - - - - HERE
.map(_ + "!!!\n")
.map(ByteString(_))
connection.handleWith(echo)
}
What I can't find is how to manage a socket closed by a CMD + C action. Akka Stream use Akka.io to manage the TCP connection internally, so it must send some of its PeerClose messages when the socket is closed. So, my understanding of Akka.io tells me that I should receive a feedback from the socket closing but I can't find how to do that with Akka Stream. Is there a way to manage that ?
connection.handleWith(echo) is syntactic sugar for connection.flow.joinMat(echo)(Keep.right).run() which will have the materialized value of echo, which is generally not useful. Flow.via.map.takeWhile has NotUsed as a materialized value, so that's also basically useless. However, you can attach stages to echo which will materialize differently.
One of these is .watchTermination:
connections.runForeach { connection =>
println(s"New connection from: ${connection.remoteAddress}")
val echo: Flow[ByteString, ByteString, Future[Done]] = Flow[ByteString]
.via(Framing.delimiter(ByteString("\n"), maximumFrameLength = 256, allowTruncation = true))
.map(_.utf8String)
.takeWhile(_.toLowerCase.trim != "exit") // < - - - - - - HERE
.map(_ + "!!!\n")
.map(ByteString(_))
// change the materialized value to a Future[Done]
.watchTermination()(Keep.right)
// you may need to have an implicit ExecutionContext in scope, e.g. system.dispatcher,
// if you don't already
connection.handleWith(echo).onComplete {
case Success(_) => println("stream completed successfully")
case Failure(e) => println(e.getMessage)
}
}
This will not distinguish between your side or the remote side closing the connection normally; it will distinguish the stream failing.
Related
I have a java jar that generates the tcp stream on particular port.
I cam run the using java command like java -jar runner.jar and this start producing the stream of message on port 8888.
When I do nc -l 8888 I can see the messages.
I want to read this stream using scala and another framework or tool like akka, akka-stream.
Can anyone help me understand the best tool, framework or any other technique to read this tcp stream.
I tried using akka stream with following code :-
implicit val system = ActorSystem()
implicit val mater = ActorMaterializer() val ss = Tcp().outgoingConnection("127.0.0.1", 8888)
.to(Sink.foreach(println(_)))
Source.empty.to(ss).run()
I also tried
Tcp().outgoingConnection(new InetSocketAddress("127.0.0.1", 8888))
.runWith(Source.maybe[ByteString], Sink.foreach(bs => println(bs.utf8String)))
This doesn't work.
I only need to read the messages and process on my own.
Thanks
As I understand you want to setup TCP server, here is example of TCP Echo using akka streams
def server(system: ActorSystem, address: String, port: Int): Unit = {
implicit val sys = system
import system.dispatcher
implicit val materializer = ActorMaterializer()
val handler = Sink.foreach[Tcp.IncomingConnection] { conn =>
println("Client connected from: " + conn.remoteAddress)
conn handleWith Flow[ByteString]
}
val connections = Tcp().bind(address, port)
val binding = connections.to(handler).run()
binding.onComplete {
case Success(b) =>
println("Server started, listening on: " + b.localAddress)
case Failure(e) =>
println(s"Server could not bind to $address:$port: ${e.getMessage}")
system.terminate()
}
}
I have a very simple Akka WebSocket server that pushes lines from a file to a connected client with an interval of 400ms per line. Everything works fine, except for the fact that the web server seems to buffer messages for about a minute before broadcasting them.
So when a client connects, I see at the server end that every 400ms a line is read and pushed to the Sink, but on the client side I get nothing for a minute and then a burst of about 150 messages (corresponding to a minute of messages).
Is there a setting that I'm overlooking?
object WebsocketServer extends App {
implicit val actorSystem = ActorSystem("WebsocketServer")
implicit val materializer = ActorMaterializer()
implicit val executionContext = actorSystem.dispatcher
val file = Paths.get("websocket-server/src/main/resources/EURUSD.txt")
val fileSource =
FileIO.fromPath(file)
.via(Framing.delimiter(ByteString("\n"), Int.MaxValue))
val delayedSource: Source[Strict, Future[IOResult]] =
fileSource
.map { line =>
Thread.sleep(400)
println(line.utf8String)
TextMessage(line.utf8String)
}
def route = path("") {
extractUpgradeToWebSocket { upgrade =>
complete(upgrade.handleMessagesWithSinkSource(
Sink.ignore,
delayedSource)
)
}
}
val bindingFuture = Http().bindAndHandle(route, "localhost", 8080)
bindingFuture.onComplete {
case Success(binding) ⇒
println(s"Server is listening on ws://localhost:8080")
case Failure(e) ⇒
println(s"Binding failed with ${e.getMessage}")
actorSystem.terminate()
}
}
So the approach with Thread.sleep(400) was wrong. I should've used the .throttle mechanic on sources:
val delayedSource: Source[Strict, Future[IOResult]] =
fileSource
.throttle(elements = 1, per = 400.millis)
.map { line =>
println(line.utf8String)
TextMessage(line.utf8String)
}
This fixed the issue.
I am trying to connect to some server through websocket on localhost. When I try to do it in JS by
ws = new WebSocket('ws://localhost:8137');
it succeeds. However, when I use akka-http and akka-streams I get "connection failed" error.
object Transmitter {
implicit val system: ActorSystem = ActorSystem()
implicit val materializer: ActorMaterializer = ActorMaterializer()
import system.dispatcher
object Rec extends Actor {
override def receive: Receive = {
case TextMessage.Strict(msg) =>
Log.info("Recevied signal " + msg)
}
}
// val host = "ws://echo.websocket.org"
val host = "ws://localhost:8137"
val sink: Sink[Message, NotUsed] = Sink.actorRef[Message](system.actorOf(Props(Rec)), PoisonPill)
val source: Source[Message, NotUsed] = Source(List("test1", "test2") map (TextMessage(_)))
val flow: Flow[Message, Message, Future[WebSocketUpgradeResponse]] =
Http().webSocketClientFlow(WebSocketRequest(host))
val (upgradeResponse, closed) =
source
.viaMat(flow)(Keep.right) // keep the materialized Future[WebSocketUpgradeResponse]
.toMat(sink)(Keep.both) // also keep the Future[Done]
.run()
val connected: Future[Done.type] = upgradeResponse.flatMap { upgrade =>
if (upgrade.response.status == StatusCodes.SwitchingProtocols) {
Future.successful(Done)
} else {
Future.failed(new Exception(s"Connection failed: ${upgrade.response.status}")
}
}
def test(): Unit = {
connected.onComplete(Log.info)
}
}
It works completely OK with ws://echo.websocket.org.
I think attaching code of my server is reasonless, because it works with JavaScript client and problem is only with connection, however if you would like to look at it I may show it.
What am I doing wrong?
I have tested your client implementation with a websocket server from akka documentation,
and I did not get any connection error. Your websocket client connects successfully. That is why I am guessing the problem is with your server implementation.
object WebSocketServer extends App {
implicit val system = ActorSystem()
implicit val materializer = ActorMaterializer()
import Directives._
val greeterWebSocketService = Flow[Message].collect {
case tm: TextMessage => TextMessage(Source.single("Hello ") ++ tm.textStream)
}
val route =
get {
handleWebSocketMessages(greeterWebSocketService)
}
val bindingFuture = Http().bindAndHandle(route, "localhost", 8137)
println(s"Server online at http://localhost:8137/\nPress RETURN to stop...")
StdIn.readLine()
import system.dispatcher // for the future transformations
bindingFuture
.flatMap(_.unbind()) // trigger unbinding from the port
.onComplete(_ => system.terminate()) // and shutdown when done
}
By the way, I noticed that your actor's receive method does not cover all possible messages. According to that akka issue,
every message, even very small, can end up as Streamed. If you want to print all text messages a better implementation of the actor would be:
object Rec extends Actor {
override def receive: Receive = {
case TextMessage.Strict(text) ⇒ println(s"Received signal $text")
case TextMessage.Streamed(textStream) ⇒ textStream.runFold("")(_ + _).foreach(msg => println(s"Received streamed signal: $msg"))
}
}
Please find a working project on my github.
I found the solution: the server I used was running on IPv6 (as ::1), but akka-http treats localhost as 127.0.0.1 and ignores ::1. I had to rewrite server to force it to use IPv4 and it worked.
Getting started with akka-streams I want to build a simple example.
In chrome using a web socket plugin I simply can connect to a stream like this one https://blockchain.info/api/api_websocket via wss://ws.blockchain.info/inv and sending 2 commands
{"op":"ping"}
{"op":"unconfirmed_sub"}
will stream the results in chromes web socket plugin window.
I tried to implement the same functionality in akka streams but am facing some problems:
2 commands are executed, but I actually do not get the streaming output
the same command is executed twice (the ping command)
When following the tutorial of http://doc.akka.io/docs/akka/2.4.7/scala/http/client-side/websocket-support.html or http://doc.akka.io/docs/akka-http/10.0.0/scala/http/client-side/websocket-support.html#half-closed-client-websockets
Here is my adaption below:
object SingleWebSocketRequest extends App {
implicit val system = ActorSystem()
implicit val materializer = ActorMaterializer()
import system.dispatcher
// print each incoming strict text message
val printSink: Sink[Message, Future[Done]] =
Sink.foreach {
case message: TextMessage.Strict =>
println(message.text)
}
val commandMessages = Seq(TextMessage("{\"op\":\"ping\"}"), TextMessage("{\"op\":\"unconfirmed_sub\"}"))
val helloSource: Source[Message, NotUsed] = Source(commandMessages.to[scala.collection.immutable.Seq])
// the Future[Done] is the materialized value of Sink.foreach
// and it is completed when the stream completes
val flow: Flow[Message, Message, Future[Done]] =
Flow.fromSinkAndSourceMat(printSink, helloSource)(Keep.left)
// upgradeResponse is a Future[WebSocketUpgradeResponse] that
// completes or fails when the connection succeeds or fails
// and closed is a Future[Done] representing the stream completion from above
val (upgradeResponse, closed) =
Http().singleWebSocketRequest(WebSocketRequest("wss://ws.blockchain.info/inv"), flow)
val connected = upgradeResponse.map { upgrade =>
// just like a regular http request we can access response status which is available via upgrade.response.status
// status code 101 (Switching Protocols) indicates that server support WebSockets
if (upgrade.response.status == StatusCodes.SwitchingProtocols) {
Done
} else {
throw new RuntimeException(s"Connection failed: ${upgrade.response.status}")
}
}
// in a real application you would not side effect here
// and handle errors more carefully
connected.onComplete(println) // TODO why do I not get the same output as in chrome?
closed.foreach(_ => println("closed"))
}
when using the flow version from http://doc.akka.io/docs/akka-http/10.0.0/scala/http/client-side/websocket-support.html#websocketclientflow modified as outlined below, again, the result is twice the same output:
{"op":"pong"}
{"op":"pong"}
See the code:
object WebSocketClientFlow extends App {
implicit val system = ActorSystem()
implicit val materializer = ActorMaterializer()
import system.dispatcher
// Future[Done] is the materialized value of Sink.foreach,
// emitted when the stream completes
val incoming: Sink[Message, Future[Done]] =
Sink.foreach[Message] {
case message: TextMessage.Strict =>
println(message.text)
}
// send this as a message over the WebSocket
val commandMessages = Seq(TextMessage("{\"op\":\"ping\"}"), TextMessage("{\"op\":\"unconfirmed_sub\"}"))
val outgoing: Source[Message, NotUsed] = Source(commandMessages.to[scala.collection.immutable.Seq])
// val outgoing = Source.single(TextMessage("hello world!"))
// flow to use (note: not re-usable!)
val webSocketFlow = Http().webSocketClientFlow(WebSocketRequest("wss://ws.blockchain.info/inv"))
// the materialized value is a tuple with
// upgradeResponse is a Future[WebSocketUpgradeResponse] that
// completes or fails when the connection succeeds or fails
// and closed is a Future[Done] with the stream completion from the incoming sink
val (upgradeResponse, closed) =
outgoing
.viaMat(webSocketFlow)(Keep.right) // keep the materialized Future[WebSocketUpgradeResponse]
.toMat(incoming)(Keep.both) // also keep the Future[Done]
.run()
// just like a regular http request we can access response status which is available via upgrade.response.status
// status code 101 (Switching Protocols) indicates that server support WebSockets
val connected = upgradeResponse.flatMap { upgrade =>
if (upgrade.response.status == StatusCodes.SwitchingProtocols) {
Future.successful(Done)
} else {
throw new RuntimeException(s"Connection failed: ${upgrade.response.status}")
}
}
// in a real application you would not side effect here
connected.onComplete(println)
closed.foreach(_ => {
println("closed")
system.terminate
})
}
How can I achieve the same result as in chrome
display print of subscribed stream
at best periodically send update (ping statements) as outlined in https://blockchain.info/api/api via {"op":"ping"}messages
Note, I am using akka in version 2.4.17 and akka-http in version 10.0.5
A couple of things I notice are:
1) you need to consume all types of incoming messages, not only the TextMessage.Strict kind. The blockchain stream is definitely a Streamed message, as it contains loads of text and it will be delivered in chunks over the network. A more complete incoming Sink could be:
val incoming: Sink[Message, Future[Done]] =
Flow[Message].mapAsync(4) {
case message: TextMessage.Strict =>
println(message.text)
Future.successful(Done)
case message: TextMessage.Streamed =>
message.textStream.runForeach(println)
case message: BinaryMessage =>
message.dataStream.runWith(Sink.ignore)
}.toMat(Sink.last)(Keep.right)
2) your source of 2 elements might complete too early, i.e. before the websocket responses come back. You can concatenate a Source.maybe by doing
val outgoing: Source[Strict, Promise[Option[Nothing]]] =
Source(commandMessages.to[scala.collection.immutable.Seq]).concatMat(Source.maybe)(Keep.right)
and then
val ((completionPromise, upgradeResponse), closed) =
outgoing
.viaMat(webSocketFlow)(Keep.both)
.toMat(incoming)(Keep.both)
.run()
by keeping the materialized promise non-complete, you keep the source open and avoid the flow shutdown.
The second example at http://doc.akka.io/docs/akka/2.4.2/scala/stream/stream-io.html is the following. I added definitions of host & port.
val host = "localhost"
val port = 4444
val connections: Source[IncomingConnection, Future[ServerBinding]] =
Tcp().bind(host, port)
connections runForeach { (connection: IncomingConnection) =>
println(s"New connection from: ${connection.remoteAddress}")
val echo = Flow[ByteString]
.via(Framing.delimiter(
ByteString("\n"),
maximumFrameLength = 256,
allowTruncation = true))
.map(_.utf8String)
.map(_ + "!!!\n")
.map(ByteString(_))
connection.handleWith(echo)
}
How would you change the example so that you could also get a hold of the Future[ServerBinding] so that you could log successful binding and handle binding errors if, for example, the port were already in use?
You should be able to get this by moving away from the convenience of runForeach which hides the details of the Sink involved and instead use an explicit Sink. Something like this:
val host = "localhost"
val port = 4444
val connections: Source[IncomingConnection, Future[ServerBinding]] =
Tcp().bind(host, port)
val sink = Sink.foreach[IncomingConnection]{ connection =>
println(s"New connection from: ${connection.remoteAddress}")
val echo = Flow[ByteString]
.via(Framing.delimiter(
ByteString("\n"),
maximumFrameLength = 256,
allowTruncation = true))
.map(_.utf8String)
.map(_ + "!!!\n")
.map(ByteString(_))
connection.handleWith(echo)
}
val serverBinding:Future[ServerBinding] = connections.to(sink).run