I'm trying to upload a file onto my personal server.
I've written a small php page that works flawlessy so far.
The little weird thing is the fact that I generate all the body of the HTTP message I'm going to send (let's say that amounts to ~4 mb) and then I send the request to my server.
The server, then, asks for an HTTP challenge and my delegate connection:didReceiveAuthenticationChallenge:challenge replies to the server with the proper credentials and the data.
But, what's happened? The data has been sent twice!
In fact I've noticed that when I added the progressbar.. the apps sends the data (4mb), the server asks for authentication, the apps re-sends the data with the authentication (another 4mb). So, at the end, I've sent 8mb. That's wrong.
I started googling and searching for a solution but I can't figure out how to fix this.
The case scenarios are two (my guess):
Share the realm for the whole session (a minimal HTTP request, then challenge, then data)
Use the synchronized way to perform an HTTP connection (things that I do not want to do since it seems an ugly way to handle this kind of stuff to me)
Thank you
You've run into a flaw into the http protocol: you have to send all the data before getting the response with the auth challenge (when you send a request with no credentials). You can try doing a small round trip as the first request in the same session (as you've mentioned), like a HEAD request, then future requests will share the same nonce.
Too late to answer the original requester, but in time if somebody else read this.
TL;DR: Section 8.2.3 of RFC 2616 describes the 100 Continue status which is all what you need (were needing) in such a situation.
Also have a look at sections 10.1.1 and 14.20.
The client sends a request with an "Expect: 100-continue" header, pausing the request before sending the body. The server uses the already received headers to make its decision whether this request may be accepted or not (if the entity –the body– to be received is not too large, if the user's credentials are correct...). If the request is acceptable for the server, it replies with a "100 Continue" status code, the client sends the body and the server replies with the final status code for that request. To the contrary, if the request is not acceptable, the server replies with a 4xx status code ("413 Request Entity Too Large" if the provided body size is... too large, or a "401 Unauthorized" + the WWW-Authenticate: header) and the client does not send the body. Being answered with a 401 status code and the corresponding WWW-Authenticate: information, the client can now perform the request again and provides its credentials.
Related
We have 2 Windows services (same machine) that communicate on top of HTTP Protocol.
On specific machine we see the HTTP POST being sent from the client (Windows service) and arrives to the server (Windows service listening to REST CALLs) - 2 times, meaning i get 2 exact HTTP Post request on the service, but we see on client it was executed only 1 time.
Before going to wireshark/analyze the HTTP protocol, I wish to understand what explain this behavior.
When going to https://www.rfc-editor.org/rfc/rfc7231#section-4.3.3
"the origin server SHOULD send a 201 (Created) response containing a Location header
field that provides an identifier for the primary resource created"
I guess we should look in wireshark for 201 response? And if no response? Does the HTTP or network framework for my C# application is retrying the POST on the server side? because we dont see 2 requests sent from client code.
POST reply behavior
While true, more often than not the server replies with a 200-ok status code and some extra information.
Whether this is by mistake or to avoid chatty apis or some other architecture/design consideration, only the developer can tell.
So in theory you get a 201 with an identifier and then make a GET request with said identifier to retrieve details.
In practice a lot of times this does not occur. So it is not safe to assume this behavior.
Your problem
I highly doubt that there is a built in mechanism that retries post. There are plenty of reasons for that:
Duplicating entries. Imagine creating a PayPal payment. If the network has an error and you just did not receive the answer, the built in mechanism will charge you twice.
There are libraries that do that only when you are sure that the request is idempotent, that is the post contained some sort of identifier and the second request will fail.
First, the calls are HTTP GET (not POST).
We define the URL with hostname/FQDN, the solution to avoid duplicated calls was to work with ip address instead of hostname when sending the Rest API.
This is the long explanation of the problem, no root cause yet.
Used both Wireshark/Process Monitor to diag, not sure for the root cause.
Process Monitor: Filtering to display network calls
Wireshark: Filter to show only HTTP
The Client send a single HTTP Get request to:
/DLEManagement/API/Engine/RunLearningPeriod
The call was executed at 11:08:16.931906
We can see 2nd call at 11:08:54.511909 - We did not trigger.
HTTP Get executed from *Server.exe (in red) and the Server is at *Management.Webservice.exe (in red).
We see that a *Client.exe (Antivirus process, in blue) is sending TCPCopy packets in the window between we sent and received.
Also, we can see that the first request was made with APIPA IPv6 and the 2nd call is IPv4, We checked the network interface and it was disabled.
Wireshark screenshot:
Process Monitor screenshot:
Network configuration:
Our image board allows users to upload images by copy-pasting URLs. A client app sends a POST request to our API with an image URL given in the request body. Our web service receives the POST request and handles it by downloading the image from the given URL by using a server-side HTTP client (request in our case).
In successful case, the service finds the image, downloads it, and stores it to the server. The service returns HTTP 200 to the client.
Now, what if the image cannot be found? What if the download attempt results in HTTP 404? What HTTP error code should we use to response to the client?
HTTP 400 Bad Request is not applicable because the request was well-formed and all parameters were valid.
HTTP 404 Not Found is not applicable because the request URL was found and served although the image URL was not.
HTTP 502 Bad Gateway does not feel right either because there is nothing wrong with our server or the upstream server (the server of the source image). The user just happened to type in an image URL that does not exist.
Any experience on the matter? Which error code is the most correct?
First of all you should decide if this is a client error (4xx) or server error (5xx). From what you describe, it feels more like a client error. The client has requested the creation of a resource from another resource (the image URL) which does not exist.
There is no perfect match for this scenario, although one could make a case for each of the 2 following response codes:
HTTP 409 Conflict: From the RFC:
The request could not be completed due to a conflict with the current
state of the target resource. This code is used in situations where
the user might be able to resolve the conflict and resubmit the
request...
This applies to your case if you consider the target resource to be in a bad state (image not found). If someone provides an image at the specified URL, that effectively transitions your resource to a valid state.
This is also a good match because, as the RFC states, this code implies the user might be able to resolve the conflict (in your case the user would correct this by posting the image to the specified URL).
HTTP 424 Failed Dependency: From the RFC:
The 424 (Failed Dependency) status code means that the method could
not be performed on the resource because the requested action depended
on another action and that action failed...
This applies to your case in that "the requested action depended on another action and that action failed". The dependent action is the posting of an image to the other URL. What you have described is a case where that dependent action either failed or did not happen (which could also be called a failure).
Since the API determines on something that is not available, its service is unavailable as well.
The status code 503: Service Unavailable is the best fit for your situation.
According to the RFC description:
The server is currently unable to handle the request due to a temporary overloading or maintenance of the server. The implication is that this is a temporary condition which will be alleviated after some delay. If known, the length of the delay MAY be indicated in a Retry-After header. If no Retry-After is given, the client SHOULD handle the response as it would for a 500 response.
Alternatively, if your API supports a way of communicating errors (e.g. to tell the user that the information he submitted is incorrect) you may be able to use this method to tell the user that the external resource is unavailable. This might be a little friendlier and might avoid some error raises on the user's side.
Since the client app sends POST requests to your API server the response codes should be generated according to the received server in your case this is your API server.
If the server has received correct information from the client app and server determines the request as valid, it should return apropriate code with proper JSON or header based error messages.
http error codes were conceived assuming that all pages possibly served were stored locally, one way or another.
Your scenario does not match that assumption and it should therefore not come as a surprise that you don't find codes that fit your bill properly.
Your "not found" scenario is in fact an application error and you should notify your user of the situation by providing an error message on the form where he entered the URL (or return a fully dedicated error page or some such). Or choose an http error nonetheless and accept the notion that it will be a poor fit no matter what.
Now, what if the image cannot be found? What if the download attempt results in HTTP 404? What HTTP error code should we use to response to the client?
The main thing to keep in mind: you are trying to fool the client into thinking that you are a web site - just a dumb document store which might respond to some content editing messages.
For the client, the primary means of communication is the body of the response. See RFC 7231
Except when responding to a HEAD request, the server SHOULD send a representation containing an explanation of the error situation, and whether it is a temporary or permanent condition.
The status code is meta-data: aimed at giving the generic components participating in the exchange a chance to know what is going on (examples: the web browser doesn't need to know what page you are asking for to recognize a redirection response returned by the server, the web browser asking for credentials when it receives a 401 unauthorized response, web caches invalidating entries, or not, depending on the status code returned by the response).
HTTP 400 Bad Request is not applicable because the request was well-formed and all parameters were valid.
Yes, that's exactly right.
I would probably use 500 Internal Server Error, on the grounds that there's nothing wrong with the _document that the server received, the problems are all involved in the side effects of the server's implementation.
A different approach you might consider: 202 Accepted. Roughly translated "I got your message, I understood your message, and I'll get around to it later." If you don't need the side effects to be synchronous, you can defer judgment. That allows you to do things like applying a retry strategy.
The representation sent with this response ought to describe the request's current status and point to (or embed) a status monitor that can provide the user with an estimate of when the request will be fulfilled.
"I'll get to it later; if you want to know how it is going, go ask him -->"
Because 202 is a non-error status code, its effect on caches is different from those of a 4xx or 5xx. If you are already thinking ahead about caching, you'll want to the implications of that in mind.
I am learning MS Flow with a basic setup: a Microsoft Form is submitted and its data is sent to another server with an HTTP POST request.
Weirdly the POST data seems to not being sent with the request.
From the other side, I get no data at all though when debugging in Flow it says that all the data has been sent.
This has been driving crazy for the last few days.
Please see the attached screens, if you can spot anything I m doing wrong.
Sorry for this post due to my bad understanding of REST and HTTP requests. The endpoints was expecting normal HTTP POST requests (Querystring format: var=value&var2=value...)
As you can see, Flow HTTP is indeed sending a POST request but with a JSON payload while the server was trying to decode the common querystring format.
Thanks (I am leaving this here, as it may save someone time in the future)
Suppose client call server using GET API, is it possible that server send a response but client misses that response??
If yes how to handle such situation, as I want to make sure that client receives the data. For now I am using second REST call by client as ack of first.
It is certainly possible. For example if you are using a site with a REST API and a request is just sent to the API and your internet connection dies when the answer is supposed to arrive, then it is quite possible that the server has received your request, successfully handled it, even sent the response, but your computer did not receive it. It could be an issue on a server responsible for transmitting the request as well. The solution to this kind of issue is to have a timeout and if a request timed out, then resend it until it is no longer timed out.
I'm trying to build a simple Email Verification API. Below you can find the expected client requests in order:
The client gets an email address as an input. (e.g. mail#example.com)
The client sends a request: GET /emails/?email=mail#example.com
If mail#example.com has not been created before, meaning the previous request returns an empty list as a response, it sends a request: POST /emails/ where email#example.com is in the request body parameters.
The client sends a request: POST /email-verifications/ with email_id in the request body and creates a new email verification object. Upon successful creation, the client receives a token in the response body and 6-digit verification code is sent to the corresponding email address.
Now the client gets verification code as an input from the user.
The client sends a request: PATCH /email-verifications/id/ with token and code in the request body.
I'm not exactly sure about the last step since the corresponding update operation receives two inputs as token and code that won't be updated in the instance. Rather, they will be compared with the existing instance and upon success another field is_verified will be updated.
Is this a right way to implement such operation? Or are there any better practices that I can follow?
PATCH is often not the perfect fit for things, and I think that you probably also shouldn't be using it here.
We've ran into a similar issue as you did and wondered how to design it. In our example it wasn't a token and code but it was an API for changing a users password.
Also in our case, a client would send a new password to a server but the server would never return the password.
The most appropriate solution for us ended up being a special password resource, with a url like:
/users/x/password
A GET request on this url would always yield a 403, and only a PUT request will be supported here. I kinda have the feeling that your design problem should be solved the same way.