We are using Akka HTTP to handle our web socket connections using the akka streams API. We are using a Flow that pipes the incoming messages to a "connection actor". A snippet of the code is below:
val connection = system.actorOf(ConnectionActor.props())
val in = Flow[Message]
.to(Sink.actorRef[Message](connection, WebSocketClosed))
val out = Source
.actorRef[Message](500, OverflowStrategy.fail)
.mapMaterializedValue(ws => connection ! WebSocketOpened(ws))
Flow.fromSinkAndSource(in, out)
When the web socket is closed, the connection actor is sent the "WebSocketClose" message and we clean up internal resources. We now have the requirement to know what the reason for closing the connection was according to the standard WebSocket CloseEvent codes.
Is there a way to get the close code from Akka HTTP and send it on to the connection actor so it can take the appropriate action?
I was able to handle client (browser) error code in an akka-http 10.2.6 server.
My use case was to pipe incoming messages to a Sink created by ActorSink.actorRef[T](). When creating the sink, 2 callbacks onCompleteMessage onFailureMessage can be set to converts normal WebSocket close (code=1000) or error to our custom message types.
I suppose that client close/error maps to Flow complete/failure, that means other sinks should be able to handle close/error in a similar way.
my code
`
As it turns out, this is not presently possible in Akka HTTP. See the following GitHub issue:
https://github.com/akka/akka-http/issues/2458
It looks as though this will need to be addressed before this is possible.
Related
I'm working with a Camel flow that uses a Netty TCP socket consumer to receive messages from a client program (which is outside of my control). The client should be opening a socket, sending us one message, then closing the socket, but we've been seeing cases where instead of one message Camel is "splitting" the text stream into two parts and trying to process them separately.
So I'm trying to figure out, since you can re-use the same socket for multiple Camel messages, but TCP sockets don't have a built-in concept of "frames" or a standard for message delimiters, how does Camel decide that a complete message has been received and is ready to process? I haven't been able to find a documented answer to this in the Netty component docs (https://camel.apache.org/components/3.15.x/netty-component.html), although maybe I'm missing something.
From playing around with a test script, it seems like one answer is "Camel assumes a message is complete and should be processed if it goes more than 1ms without receiving any input on the socket". Is this a correct statement, and if so is this behavior documented anywhere? Is there any way to change or configure this behavior? Really what I would prefer is for Camel to wait for an ETX character (or a much longer timeout) before processing a message, is that possible to set up?
Here's my test setup:
Camel flow:
from("netty:tcp://localhost:3003")
.log("Received: ${body}");
Python snippet:
DELAY_MS = 3
def send_msg(sock, msg):
print("Sending message: <{}>".format(msg))
if not sock.sendall(msg.encode()) is None:
print("Message failed to send")
time.sleep(DELAY_MS / 1000.0)
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
try:
print("Using DELAY_MS: {}".format(str(DELAY_MS)))
s.connect((args.hostname, args.port))
cutoff = int(math.floor(len(args.msg) / 2))
msg1 = args.msg[:cutoff]
send_msg(s, msg1)
msg2 = args.msg[cutoff:]
send_msg(s, msg2)
response = s.recv(1024)
except Exception as e:
print(e)
finally:
s.close()
I can see that with DELAY_MS=1 Camel logs one single message:
2022-02-21 16:54:40.689 INFO 19429 --- [erExecutorGroup] route1 : Received: a long string sent over the socket
But with DELAY_MS=2 it logs two separate messages:
2022-02-21 16:56:12.899 INFO 19429 --- [erExecutorGroup] route1 : Received: a long string sen
2022-02-21 16:56:12.899 INFO 19429 --- [erExecutorGroup] route1 : Received: t over the socket
After doing some more research, it seems like what I need to do is add a delimiter-based FrameDecoder to the decoders list.
Setting it up like this:
from("netty:tcp://localhost:3003?sync=true"
+ "&decoders=#frameDecoder,#stringDecoder"
+ "&encoders=#stringEncoder")
where frameDecoder is provided by
#Bean
ChannelHandlerFactory frameDecoder() {
ByteBuf[] ETX_DELIM = new ByteBuf[] { Unpooled.wrappedBuffer(new byte[] { (char)3 }) };
return ChannelHandlerFactories.newDelimiterBasedFrameDecoder(1024, ETX_DELIM,
false, "tcp");
}
seems to do the trick.
On the flip side though, it seems like this will hang indefinitely (or until lower-level TCP timeouts kick in?) if an ETX frame is not received, and I can't figure out any way to set a timeout on the decoder, so would still be eager for input if anyone knows how to do that.
I think the default "timeout" behavior I was seeing might've just been an artifact of Netty's read loop speed -- How does netty determine when a read is complete?
Is it possible to write a gatling script which connects to WebSocket and then performs actions (e.g. new HTTP requests) when certain messages are received (preferably with out of the box support for STOMP messages, but I could probably workaround this).
In other words, "real clients" should be simulated as best as possible. The real clients (angular applications) would load data based on certain WebSocket messages.
I'm looking for something similar to (pseude code, does not work):
val scn = scenario("WebSocket")
.exec(http("Home").get("/"))
.pause(1)
.exec(session => session.set("id", "Steph" + session.userId))
.exec(http("Login").get("/room?username=${id}"))
.pause(1)
.exec(
ws("Connect WS")
.open("/room/chat?username=${id}")
// ---------------------------------------------------------------------
// Is it possible to trigger/exec something (http call, full scenario)
// as reaction to a STOMP/WebSocket message?
.onMessage(check(perform some check, maybe regex?).as("idFromPayload"))
.exec(http("STOMP reaction").get("/entity/${idFromPayload}"))
// ---------------------------------------------------------------------
)
.exec(ws("Close WS").close)
// ideally, closing the websocket should only be done once the full scenario is over
// (or never, until the script terminates in "forever" scenarios)
Is this currently possible? If not, is this planned for future versions of gatling?
To the best of my knowledge, this is not possible with Gatling.
I have since switched to k6 which supports writing and executing test scripts with this kind of logic/behavior.
I have a simple WebSocket application, which is based on Akka HTTP/Reactive streams, like this https://github.com/calvinlfer/akka-http-streaming-response-examples/blob/master/src/main/scala/com/experiments/calvin/ws/WebSocketRoutes.scala#L82.
In other words, I have Sink, Source (which is produced from Publisher), and the Flow:
Flow.fromSinkAndSource(incomingMessages, outgoingMessages)
When I produce more, than 30 messages per second to the client, Akka closes a connection.
I cannot understand, where is a setting, which configure this behaviour. I know about OverflowStrategy, but I don't explicitly configure it.
It seems, that I have OverflowStrategy.fail(), or my problem looks like it.
You can tune Internal buffers.
There are two ways, how to do it:
1) application.conf:
akka.stream.materializer.max-input-buffer-size = 1024
2) You can configure it explicitly for your Flow:
Flow.fromSinkAndSource(incomingMessages, outgoingMessages)
.addAttributes(Attributes.inputBuffer(initial = 1, max = 1024))
i'm new to Netty and intend to create a tcp socket server which reads the info of each client and replies back towards client before processing requests immediately ,i.e. sort of an acknowledgement towards client as and when the message enters overriden channelRead method of ChannelInboundHandlerAdapter class.
Please guide me in the above specified objective.
i'm currently trying the basic netty 4.1.4 echo server example however i wanted server to send back acknowledgement to the client so i updated channelread method as follows :
#Override
public void channelRead(ChannelHandlerContext ctx, Object msg) {
ctx.write(msg);
ChannelFuture cf = ctx.channel().write("FROM SERVER");
System.out.println("Channelfuture is "+cf);
}
and the output obtained was as follows:
Channelfuture is DefaultChannelPromise#3f4ee9dd(failure: java.lang.UnsupportedOperationException: unsupported message type: String (expected: ByteBuf, FileRegion))
I understand the error that it is expecting bytebuf but how do i achieve it? also, whether this method would be able to send out acknowledgement towards client
You can use String.getBytes(Charset) and Unpooled.wrappedBuffer(byte[]) to convert to ByteBuf.
ChannelFuture cf = ctx.channel()
.write(Unpooled.wrappedBuffer("FROM SERVER".getBytes(CharsetUtil.UTF_8)));
Also note that ctx.channel().write(...); may not be what you want. Consider ctx.write(...); instead. The difference is that if your handler is a ChannelDuplexHandler it would receive a write event when you do channel().write(). Using ctx instead of channel will send the write out from your handlers point in the pipeline instead of from the end of the pipeline, which is usually what you want.
I'm trying to re-write one of our old Servers using ZeroMQ, for now I have the following Server setup, (which works for Zmq requests):
using (var context = ZmqContext.Create())
using (var server = context.CreateSocket(SocketType.REP)) {
server.Bind("tcp://x.x.x.x:5705");
while (true) { ... }
This kind of setup works fine if I use the Zmq client library to connect context.CreateSocket(SocketType.REQ)
But unfortunately we've got a lot of legacy code that needs to connect to this server and the sockets are created using .net socket libs:
Socket = new Socket(ipAddress.AddressFamily, SocketType.Stream, ProtocolType.Tcp);
Socket.Connect(ipAddress, port);
Is there a way to write a ZeroMQ Server to accept these traditional .net socket connections?
You can achieve this using ZMQ_STREAM sockets.
Please note that since zeroMQ 4.x, the RAW router option has been deprecated for a new ZMQ_STREAM socket type, that works the same way as ROUTER + RAW.
It seems it is bound to evolve, though.
I recently tried ZMQ_STREAM sockets in version 4.0.1.
You can open one, use zmq_rcv until you receive the whole message (you have to check it is whole yourself), or zmq_msg_rcv to let ZeroMQ handle it. You will receive an identifier message part, just like the identifier you would find in ROUTER sockets, directly followed by one ONLY body part. There is no empty delimiter between them like there would be using a REQ Socket talking to a ROUTER Socket. So if you route them, be sure to add it yourself.
Beware though: if there is latency on the other end or if your message exceeds ZeroMQ ZMQ_STREAM buffers (mine are 8192 bytes long), your message can be interpreted by zeroMQ as a series of messages.
In that case, you will receive as many different ZeroMQ messages including both the identifier part and the body part, and it is your job to aggregate them, knowing that if several clients are talking to the STREAM socket, they might get mixed up. I personnally use a hash table using the binary identifier as a key, and delete the entry from the table when I know the message is complete and sent to the next node.
Sending through a ZMQ_STREAM with zmq_msg_send or zmq_send works fine as is.
You probably have to use zmq's RAW socket type (instead of REP) to connect with and read client data without zmq-specific framing.
HTTP Server in C (from Pieter's blog)
http://hintjens.com/blog:42
RAW Socket type info
https://github.com/hintjens/libzmq/commit/777c38ae32a5d1799b3275d38ff8d587c885dd55