Vertx request does not end on sendFile throwing - vert.x

I'm new to vert.x and I'm trying to create a simple download service.
I used Request#sendFile(fileName) and it works well, but if I pass a directory path to Request#sendFile(fileName) it throws an exception, which is totally fine.
The problem is that, even if I catch that exception with an handler, I can't send any data nor end the request, an that leaves the http client (the browser) stuck on an endless spinning progress.
That is an example that reproduces the problem:
VertxOptions options = new VertxOptions();
options.setBlockedThreadCheckInterval(1000*60*60);
Vertx vertx = Vertx.vertx(options);
HttpServer server = vertx.createHttpServer();
Router router = Router.router(vertx);
router
.route(HttpMethod.GET,"/foo")
.handler(ctx->{
// this path exist but is not a file, is a directory.
ctx.response().sendFile("docs/pdf",asr->{
if(asr.failed()) {
ctx.response()
.setStatusCode(404)
// I can't end the connection the only thing I can do is close it
// I've commented out this lambda because is not what I want to happen.
// It's just an hack to end the request all the same.
.end("File not found: "+"docs/pdf" /*, (x)->{ctx.response().close();}*/ );
}
});
});
server
.requestHandler(router)
.listen(3000);
I can this problem by checking first if the path references to a file which both exsist and is not a directory (which in fact I did in the real code), but that leaves me with doubt about what would happen if the IOException was something different (like reading a broken file, or an unauthorized file ...).
When this error happens no data is sent through the wire, I've both checked form the browser and sniffing packets TCP packets (0 bytes send from the server to the browser).
The only things that works is closing the connection with Response#close(), which at least closes the keep-alive http connection, and ends the browser request.
What I want to achieve is to send some information back to the client to tell something went wrong, possibly setting the status code to an appropriate 4** error and possibly adding some details to it (either in status text or in the response body).

You should add failureHandler to your router:
route.failureHandler(frc-> {
frc.response().setStatusCode( 400 ).end("Sorry! Not today");
});
see https://vertx.io/docs/vertx-web/java/#_error_handling

Related

Bidirectional communication of Unix sockets

I'm trying to create a server that sets up a Unix socket and listens for clients which send/receive data. I've made a small repository to recreate the problem.
The server runs and it can receive data from the clients that connect, but I can't get the server response to be read from the client without an error on the server.
I have commented out the offending code on the client and server. Uncomment both to recreate the problem.
When the code to respond to the client is uncommented, I get this error on the server:
thread '' panicked at 'called Result::unwrap() on an Err value: Os { code: 11, kind: WouldBlock, message: "Resource temporarily unavailable" }', src/main.rs:77:42
MRE Link
Your code calls set_read_timeout to set the timeout on the socket. Its documentation states that on Unix it results in a WouldBlock error in case of timeout, which is precisely what happens to you.
As to why your client times out, the likely reason is that the server calls stream.read_to_string(&mut response), which reads the stream until end-of-file. On the other hand, your client calls write_all() followed by flush(), and (after uncommenting the offending code) attempts to read the response. But the attempt to read the response means that the stream is not closed, so the server will wait for EOF, and you have a deadlock on your hands. Note that none of this is specific to Rust; you would have the exact same issue in C++ or Python.
To fix the issue, you need to use a protocol in your communication. A very simple protocol could consist of first sending the message size (in a fixed format, perhaps 4 bytes in length) and only then the actual message. The code that reads from the stream would do the same: first read the message size and then the message itself. Even better than inventing your own protocol would be to use an existing one, e.g. to exchange messages using serde.

Flutter/Dart + gRPC Stream error: Stream was terminated by peer (errorCode: 2)

I'm having this intermittent error come up when making a call from a Dart gRPC client to a Go gRPC server. The exact error message is:
gRPC Error (code: 2, codeName: UNKNOWN, message: HTTP/2 error: Stream
error: Stream was terminated by peer (errorCode: 2)
This seems to occur frequently on creating the client stub with the channel and making a call to the server. Subsequent calls seems to work fine with a response returned correctly to the client. Then this error intermittently pops up again every n-th call (could be after 5 calls, 10 calls sometimes no errors until the 20th call).
When I use a different client (eg: BloomRPC or Java app I wrote), this does not happen no matter how hard or fast I spam the calls to the server. This kind of implies the server is probably not the issue.
Can anyone shine a light on what might be happening? Or provide some hints as to where to start looking/debugging? I've tried stepping through the code, but not really able to follow what is happening when the gRPC call is made/returned.
Steps in the client code:
create ClientChannel with ip, ports
final cc = ClientChannel(serverIP,
port: serverPort,
options:
const ChannelOptions(credentials: ChannelCredentials.insecure()
));
created a stub based on above channel
_userQueryStub = UserQueryGRPCClient(cc,
options: CallOptions(timeout: Duration(seconds: 30)));
call function from stub (repeatedly)
GetUserInformationRequest request = GetUserInformationRequest(userId: userId);
GetUserInformationResponse response = await _userQueryStub.getUserInformation(request);
Any help will be greatly appreciated!

HTTP Sender and REST conventions

I'm writing a C# Web API server application, and will send JSON to it via a Mirth HTTP Sender destination. This post is about how to handle error conditions. Specifically, there are three scenarios I want to handle:
Sometimes we take the C# application server offline for a short period for system upgrade or maintenance, and Mirth is unable to connect at all. I want Mirth to queue all messages in order, and when the server is available, process them in the order they were received.
The server receives the request, but rejects it due to a problem with the content of the request, e.g., missing a required field. In accordance with REST conventions, the server will return a 400-level HTTP response. This message would be rejected every time it's submitted, so it should not be re-sent; just log the failure and move on to the next message.
The server receives the request, but something goes wrong on the server, and the server returns an HTTP 500 Server Error response. This would be the appropriate response, for example, when something in the server environment has gone wrong. One real-world example was the time the Web API server was running, but somebody rebooted the database server. REST conventions would suggest we continue to resend the message until the transient problem has been resolved.
For #1, initially I had it queue on failure/always, but it appears the response transformer never runs for messages that were queued (at least, the debug statements never showed in the log). I have turned queueing off, and set it to retry every ten seconds for an hour, and that seems to give the desired behavior. Am I on the right track here, or missing something?
For #2 and #3, returning any HTTP 400 or 500 error invokes the 1-hour retries. What I want is to apply the 1-hour retries for the 500 errors, but not the 400 errors. I’ve tried responseStatus = SENT in the response transformer, but the response transformer only runs once, after the hour has expired, and not for each retry.
This seems like a common problem, yet I’m not finding a solution. How are the rest of you handling this?
You're close!
So by default, the response transformer will only run if there's a response payload to transform. For connection problems, or possibly for 4xx/5xx responses that contain no payload, the response transformer won't execute.
However, if you set your response data types (From the Summary -> Set Data Types dialog, or from the Destinations -> Edit Response, Message Templates tab) to Raw, then the response transformer will execute all the time. The reason being that the Raw data type considers even an empty payload to be "transformable".
So turn queuing back on, and set your response data types to Raw. Then in the response transformer, if you look at the Reference tab there's a category for HTTP Sender:
You'll want the "response status line", that's the "HTTP/1.1 200 OK" line of the response that contains the response code. Here's a response transformer script that forces 4xx responses to error:
if (responseStatus == QUEUED) {
var statusLine = $('responseStatusLine');
if (statusLine) {
var parts = statusLine.split(' ');
if (parts.length >= 2) {
var responseCode = parseInt(parts[1], 10);
// Force 4xx responses to error
if (responseCode >= 400 && responseCode < 500) {
responseStatus = ERROR;
responseStatusMessage = statusLine;
}
}
}
}

Route SockJS connection at variable URL?

Let's say I have a bunch of clients who all have their own numeric IDs. Each of them connect to my server through SockJS, with something like:
var sock = new SockJS("localhost:8080/sock/100");
In this case, 100 is that client's numeric ID, but it could be any number with any number of digits. How can I set up a SockJS router in my server-side code that allows for the client to set up a SockJS connection through a URL that varies based on what the user's ID is? Here's a simplified version of what I have on the server-side right now:
public void start() {
HttpServer server = vertx.createHttpServer();
SockJSHandler sockHandler = SockJSHandler.create(vertx);
router.route("/sock/*").handler(sockHandler);
server.requestHandler(router::accept).listen(8080);
}
This works fine if the client connects through localhost:8080/sock, but it doesn't seem to work if I add "/100" to the end of the URL. Instead of getting the default "Welcome to SockJS!" message, I just get "Not Found." I tried setting a path regex and I got an error saying that sub-routers can't use pattern URLs. So is there some way to allow for the client to connect through a variable URL, whether it's /sock/100, /sock/15, or /sock/1123123?
Ideally, I'd be able to capture the numeric ID that the client uses (like with routing REST API calls, when you could add "/:ID" to the routing path and then capture the value that the client uses), but I can't find anything that works for SockJS connections.
Since it seems that SockJS connections are considered to be the same as sub-routers, and sub-routers can't have pattern URLs, is there some work-around for this? Or is it not possible?
Edit
Just to add to what I said above, I've tried a couple different things which haven't seemed to work yet.
I tried setting up an initial, generic main router, which then re-directs to the SockJS handler. Here's the idea I had:
router.routeWithRegex("/sock/\\d+").handler(context -> {
context.reroute("/final");
});
router.route("/final").handler(SockJSHandler.create(vertx));
With this, if I access localhost:8080/sock/100 directly through the browser, it takes me to the "Welcome to SockJS!" page, and the Chrome network tab shows that a websocket connection has been created when I test it through my client.
However, I still get an error because the websocket shows a 200 status code rather than 101, and I'm not 100% sure as to why that is happening, but I would guess that it has to do with the response that the initial handler produces. If I try to set the initial handler's status code to 101, I still get an error, because then the initial handler fails.
If there's some way to work around these status codes (it seems like the websocket is expecting 101 but the initial handler is expecting 200, and I think I can only pick one), then that could potentially solve this. Any ideas?

How to produce a response body with asynchronously created body chunks in Swift Vapor

I am looking into the Swift Vapor framework.
I am trying to create a controller class that maps data obtained on an SSL link to a third party system (an Asterisk PBX server..) into a response body that is sent over some time down to the client.
So I need to send received text lines (obtained separately on the SSL connection) as they get in, without waiting for a 'complete response' to be constructed.
Seeing this example:
return Response(status: .ok) { chunker in
for name in ["joe\n", "pam\n", "cheryl\n"] {
sleep(1)
try chunker.send(name)
}
try chunker.close()
}
I thought it might be the way to go.
But what I see connecting to the Vapor server is that the REST call waits for the loop to complete, before the three lines are received as result.
How can I obtain to have try chunker.send(name) send it's characters back the client without first waiting for the loop to complete?
In the real code the controller method can potentially keep an HTTP connection to the client open for a long time, sending Asterisk activity data to the client as soon as it is obtained. So each .send(name) should actually pass immediately data to the client, not waiting for the final .close() call.
Adding a try chunker.flush() did not produce any better result..
HTTP requests aren't really designed to work like that. Different browsers and clients will function differently depending on their implementations.
For instance, if you connect with telnet to the chunker example you pasted, you will see the data is sent every second. But Safari on the other hand will wait for the entire response before displaying.
If you want to send chunked data like this reliably, you should use a protocol like WebSockets that is designed for it.