I am writing a web scraper and I use WWW::Mechanize module. I am performing a post, and pass invalid values to the arguments of the post. What I extracting is all the links from that page and print them to a text file. I would say that it's ok because the text file is empty which means that the page was not found but my problem is that the success() method is ok, and the status() method is 200.
I know it sounds a little strange but I try to get a page not found status or something to know that the page is not valid.
Does anyone have any idea of what is happening?
Whether or not your code will work depends on how the target site responds to requests for missing pages. If the server handles it by serving up an error page, you will get a successful (200) response, even though the page you requested isn't there.
More information from Google on "soft 404s" -- where missing pages return a valid page.
Here is an example from SO of configuring Apache to return a 200 response instead of a 404:
How can I replace Apache HTTP code 404 to 200
Related
The request sent to api is :
url: /users/{userid}respource/{resourceid}
method: POST
body: {SubResourceId: 6, Message: ''}
So it is a post api. It creates some mapping of user and resource and subresource.
In this case if user or resource is not found , then api returns 404. But what should be the status code if subresource(which is passed in Body) is not found in DB?
I think you should use a 4** error, because it is a client error, they are referencing a non-existent subresource. If you look at the definitions of the HTTP status errors you see that for example 400 bad request means bad syntax, where the client should not try the same request again. However a 404 means not found, yet it might be available in the future so the request can be tried again at a later time.
In all cases there should be an informative message offering more detail if available. So in this case you should probably send a 404 and a message explaining that the subresource cannot be found. Basically you are passing on a 404 from a deeper level.
There is an arguement to be made that 'the resource' that you refer to in the URI was found, and that 404 is not perfect. However looking at the various status codes available in the w3c Definitions there does not seem to be a more appropriate one.
The other option I would consider is a 500, internal server error. You could argue that the server should expect a request with a valid subresourceid. Since the subresource does not exist, the server cannot further process the request. Again, as long as their is a clear error message this can work, but it feels like more of a stretch than the 404.
But most importably, make sure you are consistent when it comes to error handling in your API(s).
Ideally the response code should be 200 with message saying that no sub-resource was found, it can also be "204 No Content". you can read more about it at wiki link.
It needs to be consistent with Response codes you have been using throughout the application.
we are using:
TYPO3 8.7.27
RealUrl 2.5.0
following scenario - a user enters a link that does not exist within our site - so we expect the behaviour that a 404 page gets rendered - we managed to achive that but we do not have the correct status code because we used a simply redirect within the install tool:
[FE][pageNotFound_handling] = "REDIRECT:/404"
[FE][pageNotFound_handling_statheader] = "HTTP/1.0 404 Not Found"
we also use our 404 page for cHash comparison errors but thats just a sidenote.
so what happens is the user requests the data from the wrong url, we send the correct 404 followed by a redirect to a certain page.
how can we directly render a page in the first place, the url should stay the same and we just render our whole TYPO3 page (no static file) with the 404 text-information.
You should use this instead:
[FE][pageNotFound_handling] = "/404"
This will instruct TYPO3 to download the page at the given "URL" and output its content together with the expected 404 status code. Notice that it might be necessary to use an absolute URL here.
From the DefaultConfigurationDescription.php:
pageNotFound_handling
... String: Static HTML file to show (reads content and outputs with correct headers), e.g. "notfound.html" or "http://www.example.org/errors/notfound.html"
You can drop the pageNotFound_handling_statheader option since it defaults to 404.
I am trying to create a surveymonkey webhook to receive my survey response and i am passing my SugarCRM custom entry point URL as "Subscription Url". But i am getting error " 'mycustomEntryPointUrl' did not return a success status code. Status code is 301". My Entry point is working fine if i run it in browser using its URL also my Sugar is working smoothly.
So, i just want to know any other reason which can cause this error.
Yes so HTTP status code 301 means the page has moved permanently. If you visit it in your browser, for example, you would see a network request to the page specified with a status code of 301, then a second one to the new page. Our API request won't do any redirect, so if a 301 is returned it will raise an error.
This sometimes happens when you go to a page with http and then it redirects to https due to rules on your server.
You also want to make sure your subscription URL supports a HEAD request without any redirect.
This started as a question, but in the process of posting this question, I solved it. So now, this is an FYI post...
==========
At first, whenever I tried to post to this URL, I would get an error message:
https:// www.dwolla.com/oauth/rest/register --> Bad URL
There is no operation listening for
http:// phx-dwol-web1.cloudworks.com/oauth/rest/register, but there is
an operation listening for
http:// www.dwolla.com/oauth/rest/register/, so you are being
redirected there.
I fixed by appending the auth values to the post URL:
https://www.dwolla.com/oauth/rest/register/?client_id={id}&client_secret={secret}
--> Good URL
Then, I was having trouble with request formatting. To fix this, make sure that you post JSON in the BODY of the post. Do NOT append to the URL, or submit as form fields.
Lastly, when I finally got a proper response from the Dwolla API, I was getting this error:
"New user must accept terms of service."
This is because the Dwolla API is CASE SENSITIVE, so the "ACCEPTTERMS" parameter in the JSON body must be submitted as "acceptTerms" in order for it to work.
Here is the documentation I am referencing:
http://developers.dwolla.com/dev/docs/register
Happy coding!
Thanks for the answer...it ultimately led to me solving my problem as well.
For completeness, I encountered this error when sending a poorly formed request to the api, where I had left off the trailing / before the get parameter {oauth_token}.
INCORRECT:
http://www.dwolla.com/oauth/rest/users?oauth_token={oauth_token}
CORRECT:
http://www.dwolla.com/oauth/rest/users/?oauth_token={oauth_token}
I am posting (HTTP POST) various values to the posterous api. I am successfully able to upload the title, body, and ONE media file, but when I try to add in a second media file I get a server 500.
They do allow media and media[] as parameters.
How do I upload multiple files with the iPhone SDK?
The 500 your getting is probably based on one of two things:
An incorrect request
An error on the server
Now, if its an incorrect, the HTTP server would be more helpful responding back with like a 415 (unsupported media type) or something. A 500 insists that something went wrong on the server and that your request was valid.
You'll have to dig into the server API or code (if you wrote it), or read the docs and figure out what's wrong with your second request ... seems like maybe your not setting the appropriate media type?
EDIT: Ok, so I looked at the API. It appears your posting XML, so your request content-type should be
Content-Type: application/xml
The API doc didn't specifically say, but that would be the correct type.
EDIT: Actually on second glance, are you just POSTing w/URI params? Their API doc isn't clear (I'm also looking rather quickly)