I am looking into the Swift Vapor framework.
I am trying to create a controller class that maps data obtained on an SSL link to a third party system (an Asterisk PBX server..) into a response body that is sent over some time down to the client.
So I need to send received text lines (obtained separately on the SSL connection) as they get in, without waiting for a 'complete response' to be constructed.
Seeing this example:
return Response(status: .ok) { chunker in
for name in ["joe\n", "pam\n", "cheryl\n"] {
sleep(1)
try chunker.send(name)
}
try chunker.close()
}
I thought it might be the way to go.
But what I see connecting to the Vapor server is that the REST call waits for the loop to complete, before the three lines are received as result.
How can I obtain to have try chunker.send(name) send it's characters back the client without first waiting for the loop to complete?
In the real code the controller method can potentially keep an HTTP connection to the client open for a long time, sending Asterisk activity data to the client as soon as it is obtained. So each .send(name) should actually pass immediately data to the client, not waiting for the final .close() call.
Adding a try chunker.flush() did not produce any better result..
HTTP requests aren't really designed to work like that. Different browsers and clients will function differently depending on their implementations.
For instance, if you connect with telnet to the chunker example you pasted, you will see the data is sent every second. But Safari on the other hand will wait for the entire response before displaying.
If you want to send chunked data like this reliably, you should use a protocol like WebSockets that is designed for it.
Related
I'm trying to create a server that sets up a Unix socket and listens for clients which send/receive data. I've made a small repository to recreate the problem.
The server runs and it can receive data from the clients that connect, but I can't get the server response to be read from the client without an error on the server.
I have commented out the offending code on the client and server. Uncomment both to recreate the problem.
When the code to respond to the client is uncommented, I get this error on the server:
thread '' panicked at 'called Result::unwrap() on an Err value: Os { code: 11, kind: WouldBlock, message: "Resource temporarily unavailable" }', src/main.rs:77:42
MRE Link
Your code calls set_read_timeout to set the timeout on the socket. Its documentation states that on Unix it results in a WouldBlock error in case of timeout, which is precisely what happens to you.
As to why your client times out, the likely reason is that the server calls stream.read_to_string(&mut response), which reads the stream until end-of-file. On the other hand, your client calls write_all() followed by flush(), and (after uncommenting the offending code) attempts to read the response. But the attempt to read the response means that the stream is not closed, so the server will wait for EOF, and you have a deadlock on your hands. Note that none of this is specific to Rust; you would have the exact same issue in C++ or Python.
To fix the issue, you need to use a protocol in your communication. A very simple protocol could consist of first sending the message size (in a fixed format, perhaps 4 bytes in length) and only then the actual message. The code that reads from the stream would do the same: first read the message size and then the message itself. Even better than inventing your own protocol would be to use an existing one, e.g. to exchange messages using serde.
I am using a FIX protocol to communicate with one of our counterparties. I have used Camel with Spring to build my communication routes.
I have a requirement where in my counterparty is expecting an ACK for every request it sends to me.
For example:
TradeCaptureRequestAck in response to TradeCaptureRequest
AllocationReportAck in response to AllocationReport
Confirmation_Ack in response to Confirmation
They are expecting a response irrespective of what happens at our end (even if something fails or exception occurs).
One way I know we can intercept the incoming message via MessageFactory. We can create a custom messagefactory and inject it in while creating QuickFixJComponent bean.
Problem with this approach is at factory level I will just be able to get the message type like TradeCaptureReport, AllocationReport etc. but not the content because factory only creates (and returns) the appropriate Message object. Actual work of populating this message object with incoming message data happens in Session class I guess (not sure about this).
Can someone please tell me if there is a way I can get or intercept the request message as soon as it reaches the route so that I can send the appropriate ACK to counterparty?
I am interested in learning Vapor, so I decided to work on a website that displays government issued weather alerts. Alert distribution is done via a TCP/IP data stream (streaming1.naad-adna.pelmorex.com port 8080).
What I have in mind is to use IBM's BlueSocket (https://github.com/IBM-Swift/BlueSocket) to create a socket, though after this point, I gave it a bit of thought but was unable to come to a conclusion on what the next steps would be.
Alerts are streamed over the data stream, so I am aware the socket would need to be opened and listened on but wasn't able to get to much past that.
A few things with the data stream are that the start and end of an alert is detected using the start and end tags of the XML document (alert and /alert). There are no special or proprietary headers added to the data, it's only raw XML. I know some alerts also include an XML declaration so I assume the encoding should be taken into account if the declaration is available.
I was then thinking of using XMLParser to parse the XML and use the data I am interested in from the alert.
So really, the main thing I am struggling with is, when the socket is open, what would be the method to listen to it, determine the start and end of the alert and then pass that XML alert for processing.
I would appreciate any input, I am also not restricted to BlueSocket so if there is a better option for what I am trying to achieve, I would be more than open to it.
So really, the main thing I am struggling with is, when the socket is
open, what would be the method to listen to it, determine the start
and end of the alert and then pass that XML alert for processing.
The method that you should use is read(into data: inout Data). It stores any available data that the server has sent into data. There are a few reasons for this method to fail, such as the connection disconnecting.
Here's an example of how to use it:
import Foundation
import Socket
let s = try Socket.create()
try s.connect(to: "streaming1.naad-adna.pelmorex.com", port: 8080)
while true {
if try Socket.wait(for: [s], timeout: 0, waitForever: true) != nil {
var alert = Data()
try s.read(into: &alert)
if let message = String(data: alert, encoding: .ascii) {
print(message)
}
}
}
s.close()
First create the socket. The default is what we want, a IPv4 TCP Stream.
Second connect() to the server using the hostname and port. Without this step, the socket isn't connected and cannot receive or send any data.
wait() until hostname has sent us some data. It returns a list of sockets that have data available to read.
read() the data, decode it and print it. By default this call will block if there is no data available on the socket.
close() the socket. This is good practice.
You might also like to consider thinking about:
non blocking sockets
error handling
streaming (a single call to read() might not give a complete alert).
I hope this answers your question.
Let's say I have a bunch of clients who all have their own numeric IDs. Each of them connect to my server through SockJS, with something like:
var sock = new SockJS("localhost:8080/sock/100");
In this case, 100 is that client's numeric ID, but it could be any number with any number of digits. How can I set up a SockJS router in my server-side code that allows for the client to set up a SockJS connection through a URL that varies based on what the user's ID is? Here's a simplified version of what I have on the server-side right now:
public void start() {
HttpServer server = vertx.createHttpServer();
SockJSHandler sockHandler = SockJSHandler.create(vertx);
router.route("/sock/*").handler(sockHandler);
server.requestHandler(router::accept).listen(8080);
}
This works fine if the client connects through localhost:8080/sock, but it doesn't seem to work if I add "/100" to the end of the URL. Instead of getting the default "Welcome to SockJS!" message, I just get "Not Found." I tried setting a path regex and I got an error saying that sub-routers can't use pattern URLs. So is there some way to allow for the client to connect through a variable URL, whether it's /sock/100, /sock/15, or /sock/1123123?
Ideally, I'd be able to capture the numeric ID that the client uses (like with routing REST API calls, when you could add "/:ID" to the routing path and then capture the value that the client uses), but I can't find anything that works for SockJS connections.
Since it seems that SockJS connections are considered to be the same as sub-routers, and sub-routers can't have pattern URLs, is there some work-around for this? Or is it not possible?
Edit
Just to add to what I said above, I've tried a couple different things which haven't seemed to work yet.
I tried setting up an initial, generic main router, which then re-directs to the SockJS handler. Here's the idea I had:
router.routeWithRegex("/sock/\\d+").handler(context -> {
context.reroute("/final");
});
router.route("/final").handler(SockJSHandler.create(vertx));
With this, if I access localhost:8080/sock/100 directly through the browser, it takes me to the "Welcome to SockJS!" page, and the Chrome network tab shows that a websocket connection has been created when I test it through my client.
However, I still get an error because the websocket shows a 200 status code rather than 101, and I'm not 100% sure as to why that is happening, but I would guess that it has to do with the response that the initial handler produces. If I try to set the initial handler's status code to 101, I still get an error, because then the initial handler fails.
If there's some way to work around these status codes (it seems like the websocket is expecting 101 but the initial handler is expecting 200, and I think I can only pick one), then that could potentially solve this. Any ideas?
I've been having some problems with the below code that I've pieced together. All the events work as advertised however, when a client drops off-line without first disconnecting the close event doesn't get call right away. If you give it a minute or so it will eventually get called. Also, I find if I continue to send data to the client it picks up a close event faster but never right away. Lastly, if the client gracefully disconnects, the end event is called just fine.
I understand this is related to the other listen events like upgrade and ondata.
I should also state that the client is an embedded device.
client http request:
GET /demo HTTP/1.1\r\n
Host: example.com\r\n
Upgrade: Websocket\r\n
Connection: Upgrade\r\n\r\n
//nodejs server (I'm using version 6.6)
var http = require('http');
var net = require('net');
var sys = require("util");
var srv = http.createServer(function (req, res){
});
srv.on('upgrade', function(req, socket, upgradeHead) {
socket.write('HTTP/1.1 101 Web Socket Protocol Handshake\r\n' +
'Upgrade: WebSocket\r\n' +
'Connection: Upgrade\r\n' +
'\r\n\r\n');
sys.puts('upgraded');
socket.ondata = function(data, start, end) {
socket.write(data.toString('utf8', start, end), 'utf8'); // echo back
};
socket.addListener('end', function () {
sys.puts('end'); //works fine
});
socket.addListener('close', function () {
sys.puts('close'); //eventually gets here
});
});
srv.listen(3400);
Can anyone suggest a solution to pickup an immediate close event? I am trying to keep this simple without use of modules. Thanks in advance.
close event will be called once TCP socket connection is closed by one or another end with few complications of rare cases when system "not realising" that socket been already closed, but this are rare cases. As WebSockets start from HTTP request server might just keep-alive till it timeouts the socket. That involves the delay.
In your case you are trying to perform handshake and then send data back and forth, but WebSockets are a bit more complex process than that.
The handshake process requires some security procedure to validate both ends (server and client) and it is HTTP compatible headers. But different draft versions supported by different platforms and browsers do implement it in a different manner so your implementation should take this in account as well and follow official documentation on WebSockets specification based on versions you need to support.
Then sending and receiving data via WebSockets is not pure string. Actual data sent over WebSockets protocol has data-framing layer, which involves adding header to each message you send. This header has details over message you sending, masking (from client to server), length and many other things. data-framing depends on version of WebSockets again, so implementations will vary slightly.
I would encourage to use existing libraries as they already implement everything you need in nice and clean manner, and have been used extensively across commercial projects.
As your client is embedded platform, and server I assume is node.js as well, it is easy to use same library on both ends.
Best suit here would be ws - actual pure WebSockets.
Socket.IO is not good for your case, as it is much more complex and heavy library that has multiple list of protocols support with fallbacks and have some abstraction that might be not what you are looking for.