How do i send a POST request without Transfer Encoding:chunked from Jersey ReST Client 2.22.2 - rest

When i send a POST request through Jersey ReST client it's automatically using Header transfer-encoding: [chunked].
Is there any way to force use of content-length: instead of transfer-encoding.?
WebTarget webTarget = client.target(connection.getServerUrl());
Invocation.Builder builder = webTarget.request(MediaType.APPLICATION_XML);
Response response = builder.post(Entity.xml(requestBroker));
After adding Content-Length property too the behavior is same
WebTarget webTarget = client.target(connection.getServerUrl());
Invocation.Builder builder = webTarget.request(MediaType.APPLICATION_XML);
Entity entity = Entity.xml(requestBroker);
client.property("Content-Length", entity.toString().getBytes().length);
Response response = builder.post(Entity.xml(requestBroker));

HTTP 1.1 version onwards chunked transfer encoding is default for POST, in this data is sent as chunks and hence the senders can begin transmitting dynamically-generated content before knowing the total size of that content. The size of each chunk is sent right before the chunk itself so that the receiver can tell when it has finished receiving data for that chunk. The data transfer is terminated by a final chunk of length zero.
Is there any way to force use of content-length: instead of
transfer-encoding
Set the Content-Length header before sending your POST request. But this will work only in http 1.0, and when you set the content length, and if the post request data size is more than the content length then the data received will be truncated.
In the version 1.1 of the HTTP protocol, the chunked transfer mechanism is considered to be always and anyways acceptable, even if not listed in the TE (transfer encoding) request header field, and when used with other transfer mechanisms, should always be applied last to the transferred data and never more than one time. Source Wikipedia - Chunked Transfer Encoding
Whereas in the response, we can avoid Transfer-Encoding by setting the BufferSize on response using response.setBufferSize(). But if our response size goes beyond the bufferSize it would fallback to Transfer-Encoding: Chunked.
Different Transfer Mechanisms
More Info:
Content-Length header versus chunked encoding
Remove Transfer-Encoding:chunked in the POST request?
avoiding chunked encoding of HTTP/1.1 response
Hope it Helps!

Related

Akka-HTTP: how to know if Content-type header was explicitly set in received response

Is the a way in akka-http to know if 'Content-type' header was explicitly set in HttpResponse that we received?
From sniffed Http dump I see, that there was no 'Content-Type' header, but
httpResponse.header[`Content-Type`].get.contentType.mediaType.toString()
and
httpResponse.entity.getContentType().mediaType.toString
stil return application/octet-stream.
This is default Content type not only for Akka-HTTP, but perhaps for other frameworks like Play too. Akka-http and other HTTP based technologies need to know how to parse content internally, based on this header. application/octet-stream means that it considers request body as just byte-stream.
Rule of thumb: if it is possible - try to specify Content-type.

Why is the browser satisfied with a response without content-length

Usually when I send a response to the browser I have to enter content-length in the http headers, otherwise the browser never stops loading (wait for more data)
But recently, I tested rust code:
let response = format!("HTTP/1.1 200 OK\r\n\r\n{}", contents);
stream.write(response.as_bytes()).unwrap();
The browser receives this without any problems, stops loading after receiving the response.(even though content-length is not specified in the response)
Can someone pls explain this?... What makes the browser satisfied with the response in this scenario (even though it does not contain: Content-length)
Content-length is optional as long as the connection is closed after the response is done. From RFC 7230 section 3.3.3 Message Body Length:
Otherwise, this is a response message without a declared message
body length, so the message body length is determined by the
number of octets received prior to the server closing the
connection.

Is Accept header needed for a POST method which doesn't return any content to client?

I have an endpoint which supports POST method with content-type as json(only). But the POST request doesn't return any content in its response body other than status codes. In this scenario, what is the correct behavior?
Client sends POST with Accept header as application/json
Client sends POST with Accept header as application/xml
Should the server return error in case 2?
RFC 7231 describes the semantics of the Accept header
A request without any Accept header field implies that the user agent will accept any media type in response.
If the header field is present in a request and none of the available representations for the response have a media type that is listed as acceptable, the origin server can either honor the header field by sending a 406 (Not Acceptable) response or disregard the header field by treating the response as if it is not subject to content negotiation.
The Accept header provided by the client should probably reflect the context of the request as seen by the client; for instance, a web browser might reasonably use a different Accept header for <img> than for <script>, in each case encouraging the server to provide useful representations.
In the case of a POST, what you are trying to negotiate is the representation of "the status of, or results obtained from, the action", rather than a representation of resource itself.
If the representation of the response is zero bytes long when the media-type is application/json, then I would expect the response to also be zero bytes long when the media-type is application/xml. So it isn't obvious to me to accept one but not the other.
Servers may ignore the Accept header.
If you're not returning anything in your response, it's kind of meaningless. It's up to you to decide whether you want to reject requests with Accept headers or not.
But I think most systems will not reject these requests.
https://datatracker.ietf.org/doc/html/rfc7231#section-5.3.2
A request without any Accept header field implies that the user agent
will accept any media type in response. If the header field is
present in a request and none of the available representations for
the response have a media type that is listed as acceptable, the
origin server can either honor the header field by sending a 406 (Not
Acceptable) response or disregard the header field by treating the
response as if it is not subject to content negotiation.
so either off 2 we can do

Is S3 REST for PUT (direct upload) operation chunked?

In S3 REST API, how does the PUT operation i.e. a direct upload not the multipart upload exactly send requests for such large files i.e. Gigabytes through HTTP? Is the direct upload also chunked (like the multipart upload) and has a defined size internally?
When tried doing a PUT (direct upload) operation using S3 REST API, the maximum I could upload was around 5GB which is what even Amazon says their maximum limit for direct upload is. But when tried uploading a file which larger then the limit it throws an exception "Your proposed upload exceeds the maximum allowed size" and also has a HTTP response returned where the header tag 'transfer-encoding' is 'chunked'.
Here's a randomly-selected error response from S3.
< HTTP/1.1 412 Precondition Failed
< x-amz-request-id: 207CAFB3CEXAMPLE
< x-amz-id-2: EXAMPLE/DCHbRTTnpavsMQIg/KRRnoEXAMPLEBJQrqR1TuaRy0SHEXAMPLE5otPHRZw4EXAMPLE=
< Content-Type: application/xml
< Transfer-Encoding: chunked
< Date: Fri, 23 Jun 2017 19:51:52 GMT
< Server: AmazonS3
<
<?xml version="1.0" encoding="UTF-8"?>
<Error><Code>...
The Transfer-Encoding: chunked response header only indicates that the error response body S3 is sending back to you will use chunked transfer encoding.
This is unrelated to what is permitted for uploads, and the presence of Transfer-Encoding: chunked in either direction (request or response) of an HTTP transaction is independent of whether it is present or supported in the opposite direction.
The PUT object REST API call does not support Transfer-Encoding: chunked on the request. It requires Content-Length: in the request headers, which precludes using chunked transfer encoding.
There is no chunking, blocking, etc., mechanism involved at the HTTP layer in standard uploads -- there is no meaningful internal structure "part-size," because there are no parts: it's a continuous TCP stream of un-encoded octets of exactly Content-Length length (number of octets/bytes), with retries and network errors handled by TCP, and HTTP unaware of these mechanisms.
If the Content-Length header you send exceeds the maximum allowed upload, you get the error about your proposed upload exceeding the maximum allowed size. If the connection is accidentally or intentionally severed before Content-Length number of octets are received by S3, the uploaded data is discarded, because partial objects are never created.

How to C - windows socket reading textfile content

I am having problems reading a text file content via winsock on C , does anyone have any idea how it should work? actually when I try to GET HTTP header from google am able to, but when I try on my xampp machine,
it just gives me 400 bad request.
HTTP/1.1 400 Bad Request
char *message = "GET / HTTP/1.1\r\n\r\n";
Ok the problem that I was receiving 400 bad request on my localhost via winsock was the my HTTP request, i just changed the 1.1 to 1.0 .. and it worked!!! what I am wanting now is printing nothing the content of the text file and not the whole banner?! :)
Read RFC 2616, in particular sections 5.2 and 14.23. An HTTP 1.1 request is required to include a Host header, and an HTTP 1.1 server is required to send a 400 reply if the header is missing and no host is specified in the request line.
char *message = "GET / HTTP/1.1\r\nHost: hostnamehere\r\n\r\n";
As for the text content, you need to read from the socket until you encounter a \r\n\r\n sequence (which terminates the response headers), then process the headers, then read the text content accordingly. The response headers tell you how to read the raw bytes of the text content and when to stop reading (refer to RFC 2616 section 4.4 for details). Once you have the raw bytes, the Content-Type header tells you how to interpret the raw bytes (data type, charset, etc).