Usually when I send a response to the browser I have to enter content-length in the http headers, otherwise the browser never stops loading (wait for more data)
But recently, I tested rust code:
let response = format!("HTTP/1.1 200 OK\r\n\r\n{}", contents);
stream.write(response.as_bytes()).unwrap();
The browser receives this without any problems, stops loading after receiving the response.(even though content-length is not specified in the response)
Can someone pls explain this?... What makes the browser satisfied with the response in this scenario (even though it does not contain: Content-length)
Content-length is optional as long as the connection is closed after the response is done. From RFC 7230 section 3.3.3 Message Body Length:
Otherwise, this is a response message without a declared message
body length, so the message body length is determined by the
number of octets received prior to the server closing the
connection.
Related
I'm currently trying to build a MATLAB based system to interface with the API of my stock broker. I'm however running into quite some issues with sending the http post requests to the server.
I already have it working perfectly when testing with POSTMAN, but for some reason it keeps refusing my MATLAB send requests. I now testing the actual requests through PIPEDREAM which lets me view the http request.
Image of the good and bad requests:
The Left is an image of my postman requests which it perfectly processes as JSON strings. However my MATLAB requests are not processed properly and also are 10 characters longer than the actual string value.
The (trimmed) code to send the requests can be seen here.
% http request classes
import matlab.net.*
import matlab.net.http.*
% prepare payload
username = "usr";
password = "XXXXXXXXXXXXX";
login_payload = struct("username", username, "password", password);
request = RequestMessage('POST', [ bunchOfHeaders ], jsonencode(login_payload));
% Send request to login api
[login_resp, c, h] = request.send("https://trading.somebroker.com/login/secure/login");
Does anyone have any clue what could be happening here? If I set the content-lenght to the "correct" length (same as length(login_payload)) it says my length is wrong even though my postman requests seem to not struggle with this.
Found the answer... Matlabs http stuff is absolutely braindead.
I had a closer look into the raw intercepted messages (pipedream just sends your request back to you and you can view it with string(login_resp)).
For some god darn reason matlab encases the json string with "s which makes the receiver treat the whole body as a string. This is caused by setting the "content-type" to "application/json". changing the content-type to "text/plain" did not encase it in "s and completely solved my issue
I'm writing a C# Web API server application, and will send JSON to it via a Mirth HTTP Sender destination. This post is about how to handle error conditions. Specifically, there are three scenarios I want to handle:
Sometimes we take the C# application server offline for a short period for system upgrade or maintenance, and Mirth is unable to connect at all. I want Mirth to queue all messages in order, and when the server is available, process them in the order they were received.
The server receives the request, but rejects it due to a problem with the content of the request, e.g., missing a required field. In accordance with REST conventions, the server will return a 400-level HTTP response. This message would be rejected every time it's submitted, so it should not be re-sent; just log the failure and move on to the next message.
The server receives the request, but something goes wrong on the server, and the server returns an HTTP 500 Server Error response. This would be the appropriate response, for example, when something in the server environment has gone wrong. One real-world example was the time the Web API server was running, but somebody rebooted the database server. REST conventions would suggest we continue to resend the message until the transient problem has been resolved.
For #1, initially I had it queue on failure/always, but it appears the response transformer never runs for messages that were queued (at least, the debug statements never showed in the log). I have turned queueing off, and set it to retry every ten seconds for an hour, and that seems to give the desired behavior. Am I on the right track here, or missing something?
For #2 and #3, returning any HTTP 400 or 500 error invokes the 1-hour retries. What I want is to apply the 1-hour retries for the 500 errors, but not the 400 errors. I’ve tried responseStatus = SENT in the response transformer, but the response transformer only runs once, after the hour has expired, and not for each retry.
This seems like a common problem, yet I’m not finding a solution. How are the rest of you handling this?
You're close!
So by default, the response transformer will only run if there's a response payload to transform. For connection problems, or possibly for 4xx/5xx responses that contain no payload, the response transformer won't execute.
However, if you set your response data types (From the Summary -> Set Data Types dialog, or from the Destinations -> Edit Response, Message Templates tab) to Raw, then the response transformer will execute all the time. The reason being that the Raw data type considers even an empty payload to be "transformable".
So turn queuing back on, and set your response data types to Raw. Then in the response transformer, if you look at the Reference tab there's a category for HTTP Sender:
You'll want the "response status line", that's the "HTTP/1.1 200 OK" line of the response that contains the response code. Here's a response transformer script that forces 4xx responses to error:
if (responseStatus == QUEUED) {
var statusLine = $('responseStatusLine');
if (statusLine) {
var parts = statusLine.split(' ');
if (parts.length >= 2) {
var responseCode = parseInt(parts[1], 10);
// Force 4xx responses to error
if (responseCode >= 400 && responseCode < 500) {
responseStatus = ERROR;
responseStatusMessage = statusLine;
}
}
}
}
I am implementing REST services using Apache-CXF running on servicemix and for that I have a camel route that does some processing, sends the message over queue, process some more and send back the reply. Something like this:
from("direct:start")
.process(A)
.process("activemq:abc")
.process(B);
On this route I have applied some basic validation and exception handler and when I have to stop the route in both cases, I use something like this:
exchange.getOut().setBody(response);
exchange.setProperty(Exchange.ROUTE_STOP, Boolean.TRUE);
I use soap UI, restclient-UI and putty to make http requests and I get proper response body displayed in all of them. Now I wanted to preserve request headers so I made a little change everywhere in the code so that response bodies are set in exchange.getIn() only. For example: in case of validation failure I do:
exchange.getIn().setBody(response);
exchange.setProperty(Exchange.ROUTE_STOP, Boolean.TRUE);
Just with this little change, the rest clients I am using to make request stopped displaying the response body. As per the server logs, response is being generated and also as per the logs in rest client, I am getting the proper response but they are unable to display the response body only in case when I stop the route in between. Normal response is displaying just fine. Only the restclient-UI was considerate enough to show the error as to why they are not displaying body and the error is:
Byte array conversion from response body stream failed.
Diggin deeper, I found the only response header which was there in success response but missing in error response:
Transfer-Encoding chunked
Error response is around 1000 characters long and contains a header called content-length. I am not sure but I think the problem has something to do with this itself. I would really like to play with exchange.getIn but these different kind of responses prepared by camel are confusing me. How can I make sure my camel responses are always displayed properly?
The Content-Length header will be preserved from the original request so you need to remove it so that camel cxf can work out the new body length on the response and set Content-Length with that.
I can't get one thing straight. The RFC 2616 in 4.4.5 states that Message Length can be determined "By the server closing the connection.".
This implies, that it is valid for a server to respond (e.g. returning a large image) with a response, that has no Content-Length in the header, but the client is supposed to keep fetching till the connection is closed and then assume all data has been downloaded.
But how is a client to know for sure that the connection was closed intentionally by the server? A server app could have crashed in the middle of sending the data and the server's OS would most likely send FIN packet to gracefully close the TCP connection with the client.
You are absolutely right, that mechanism is totally unreliable. This is covered in RFC 7230:
Since there is no way to distinguish a successfully completed,
close-delimited message from a partially received message interrupted
by network failure, a server SHOULD generate encoding or
length-delimited messages whenever possible. The close-delimiting
feature exists primarily for backwards compatibility with HTTP/1.0.
Fortunately most of HTTP traffic today are HTTP/1.1, with Content-Length or "Transfer-Encoding" to explicitly define the end of message.
The lesson is that, a message must have it own way of termination; we cannot repurpose the underlying transport layer's EOF as the message's EOF.
On that note, a (well-formed) html document, or a .gif, .avi etc, does define its own termination; we will know if we received an incomplete document. Therefore it is not so much of a problem to transmit it over HTTP/1.0 without Content-Length.
However, for plain text document, javascript, css etc. EOF is used to marked the end of the document, therefore it's problematic over HTTP/1.0.
I am having problems reading a text file content via winsock on C , does anyone have any idea how it should work? actually when I try to GET HTTP header from google am able to, but when I try on my xampp machine,
it just gives me 400 bad request.
HTTP/1.1 400 Bad Request
char *message = "GET / HTTP/1.1\r\n\r\n";
Ok the problem that I was receiving 400 bad request on my localhost via winsock was the my HTTP request, i just changed the 1.1 to 1.0 .. and it worked!!! what I am wanting now is printing nothing the content of the text file and not the whole banner?! :)
Read RFC 2616, in particular sections 5.2 and 14.23. An HTTP 1.1 request is required to include a Host header, and an HTTP 1.1 server is required to send a 400 reply if the header is missing and no host is specified in the request line.
char *message = "GET / HTTP/1.1\r\nHost: hostnamehere\r\n\r\n";
As for the text content, you need to read from the socket until you encounter a \r\n\r\n sequence (which terminates the response headers), then process the headers, then read the text content accordingly. The response headers tell you how to read the raw bytes of the text content and when to stop reading (refer to RFC 2616 section 4.4 for details). Once you have the raw bytes, the Content-Type header tells you how to interpret the raw bytes (data type, charset, etc).