Almost each major cloud storage service provider like Google, Box, OneDrive use request parameters to convey pagination info to web services and update response object to further communicate next page info to the client.
Despite of having standard HTTP Headers like Range and Link and custom headers people tend to use query parameters to convey such info.
Using such standard headers to convey pagination info between client and server keeps the standard response unique as we don't need to pass such info via response attributes.
So, what is the best practice to implement pagination in REST APIs? With HTTP Headers or with Query Parameters???
According to the http specs:
A proxy MAY discard a Range
header field that contains a range unit it does not understand. That means any intermediate proxy, including caching servers, may throw out your `Range header before it gets to your server.
Link headers have nothing to do with pagination.
A proxy MUST forward unrecognized header fields unless [..] the proxy is specifically configured to block, or otherwise transform, such fields. I read that as a proxy can be configured to block all unrecognized headers, which is the same problem as for Range headers.
Beyond that, using query parameters for pagination (preferably offset/limit) is a standard. If you have a good reason to buck the standard, that's fine. The downside is now everybody who is using your API has to learn your way of doing things in addition to the way everybody else does things. This lowers adoption and increases support calls.
Related
I have a REST API and I want to handle all HTTP requests via POST request.
Is there any performance or other kind of issue in using just POST to perform all CRUD operations requested by a user, which sends a JSON containing some data and the operation to be performed?
Technically, the HTML used in the Web only supports GET and POST and this is more or less the reference implementation of a REST architecture.
So, while this is possible I wouldn't advocate for something like that as the idempotency property of PUT and DELETE provide some other benefits in case of network issues where a client can automatically resend the request regardless whether the initial request, whose response might have just got lost mid-way, actually performed its task or not. The result should always be an updated/created resource or a removed URI mapping to the actual resource (or even a removal of the actual resource) as DELETE technically just removes the URI mapping.
In regards to put some operations in the payload, it depends. This actually sounds very RPCy to me, similar to SOAP i.e. If the operation however is defined by a well-defined media-type, like in the JSON Patch case, I guess this is not wrong. Similar to the Web, however, a server should use some resource that is able to teach a client on how to build up a request, like HTML does with forms. This will not only teach the client on what fields the server supports for the target resource but also where to send the request to as well as the media-type and HTTP operation to use, which might be fixed to POST as in the HTML case.
I'm creating a headless API that's going to drive an Angular front end. I'm having a bit of trouble figuring out how I should handle user authentication though.
Obviously the API should run over SSL, but the question that's coming up is how should I send the request that contains the user's password: over GET or POST. It's a RESTFUL API, so what I'm doing is retrieving information meaning it should get a GET request. But sending the password over get means it's part of the URI, right? I know even a GET request is encrypted over HTTPS, but is that still the correct way? Or is this a case to break from RESTFUL and have the data in the body or something (can a GET request have data in the body?).
If you pass the credentials in a request header, you will be fine with either a GET or POST request. You have the option of using the established Authorization header with your choice of authentication scheme, or you can create custom headers that are specific to your API.
When using header fields as a means of communicating credentials, you do not need to fear the credentials being written to the access log as headers are not included in that log. Using header fields also conforms to REST standards, and should actually be utilized to communicate any meta-data relevant to the resource request/response. Such meta-data can include, but is not limited to, information like: collection size, pagination details, or locations of related resources.
In summary, always use header fields as a means of authentication/authorization.
mostly GET request will bind data in URL itself... so it is more redable than POST..
so if it is GET, there is a possibility to alive HISTORY LOG
Using ?user=myUsername&pass=MyPasswort is exactly like using a GET based form and, while the Referer issue can be contained, the problems regarding logs and history remain.
Sending any kind of sensitive data over GET is dangerous, even if it is HTTPS. These data might end up in log files at the server and will be included in the Referer header in links to or includes from other sides. They will also be saved in the history of the browser so an attacker might try to guess and verify the original contents of the link with an attack against the history.
You could send a data body with a get request too but this isn't supported by all libraries I guess.
Better to use POST or request headers. Look at other APIs and how they are handling it.
But you could still use GET with basic authentication like here: http://restcookbook.com/Basics/loggingin/
I'm designing api with method that should be an idempotent, and should not modify any data on the server. It should be method that process request and return response for given parameters.
One of the parameters is sensitive data. It's not an option to use additional encryption. Data is already encrypted, but security requirements are very demanding and even encrypted data should be treated very carefully.
According to REST spec, idempotent query method should be implemented as a GET HTTP method. Problem in this case is sensitive data that shouldn't be pass as a GET parameter in URL. Only option in HTTP standard is to pass sensitive data in a body part of HTTP request.
My question is what is better? Broke rest api design, and send query request as a POST, or pass encrypted data in URL? Maybe is there better solution I don't see?
According to REST spec, idempotent query method should be implemented
as a GET HTTP method.
2016
As far as I can tell with my limited English SHOULD != MUST. You won't break REST API design by sending a POST in this case. You can send your sensitive data in a HTTP header if that is possible. And ofc. you should use HTTPS if you want to send sensitive data to anywhere.
2019
I checked the HTTP 1.1 standard meanwhile. They don't explicitly use the MUST or SHOULD words in the specs for idempotency, but I got the impression they mean SHOULD. Another HTTP related thing here, that we use GET mostly because we can cache response with it. You don't necessarily want to cache sensitive data, so it might not make sense to insist on using GET on retrieval when security is more important by the parameters. You can find some tips about how to set cache-control headers here, but you can read the HTTP standard for that too.
From security perspective my non-expert opinion is the following:
Normally query parameters are not that sensitive, usually they are just random ids or keywords. So maybe the problem is with your design and you should hide these sensitive parameters (e.g. social security number) behind random ids instead of querying them explicitly. Another thought here, that user credentials must be in the Authorization header for example, not in the query string, so if the sensitive data is that kind, then you are doing it wrong.
As far as I understand the issue about sending sensitive data in URLs is that it can show up in browser history, cache, address bar and in server logs unencrypted. Even though many people call REST webservices directly from browser via AJAX (or the fetch API), that is not the intended way they should be used. Webservices are mostly for server side usage to scale out your application to multiple threads, cores or servers. So if you use a server side HTTP client which does not have history or cache to call the REST webservice programmatically, then all you need to do is encrypting your logs. If the client has cache, then you can encrypt that too if you feel it necessary. I think it is possible to filter these params from logs and store the cached content based on the salted hash of the URL, but I don't have much experience with that.
If you have a 3rd party client or a browser where you don't have that kind of control, then you can still assume that it follows the HTTP standard. So you can use the cache-control headers to disable cache for sensitive content. The address bar and history is not a problem by single page applications unless they move the sensitive data to there with the history API, but that can happen no matter what you do. It is possible to disable the Referrer header too. Only if you serve HTML with your webservice will you have a problem with browsers, because that assumes that javascript is disabled (so you cannot use location.replace to override browser history along with the sensitive querystring) and that the browser is your REST client. I think that is a very unlikely scenario, though it is possible to do it relative well with XML+XSL reusing most of the code or nowadays maybe with nodejs or some sort of transpiler on different languages.
So I think this can be solved even without POST if you do everything right. But this is just an opinion, I wait for security expert to correct me...
REST and CORS.. how are they different? is it even correct to compare them? because I have seen a seemingly REST API use custom X- headers to make a pre-flighted request(Docebo LMS API). This means that maybe CORS and REST are used for different purposes.. But on the surface, it seems that both are designed to give access to resources stored on a different server. Also, Simple XMLHTTP requests seem to work like HTTP.(The headers sent and received by the browser are through HTTP).. So, are XMLHTTP objects translated into HTTP by the browser? I am really taking in a ton of information right now and I cant seem to make any real progress in understanding these things... Any help is appreciated.
CORS - Cross Origin Resource Sharing. A concept and set of techniques that enables sharing of resource/data across domains. Example, from your page /yourDomain.net you try to make an ajax call to myDomain.net to post some data. Read this Wikipedia and MDN articles.
REST - REpresentational State Transfer. A set of standards & guidelines that defines a specific way for systems to talk to each other. It follows state-less http like standards where URIs reprsent resource and client can work on them using http verbs. e.g. GET weatherApp.com/weather/rome. Refer this.
HTTP - Hyper Text Transfer Protocol. THE standard protocol to transfer data to/from web servers. Check this W3 specifications and Wikipedia page.
XMLHttp - A type of request generally used to make ajax calls from client (mainly html, javascript) applications to web servers. It works on http standards. Not bound to XML though. Read this and this.
Now, all of REST, XMLHttp, CORS work on HTTP is some way, meaning they all use the http infrastructure.
And any/all of them might be used to create a fully functional modern application. For example, a web application might use XMLHttp request to make REST service call to get some data. It can also utilize CORS to get/post data to another domain. Need not say, the whole system relies on http!
They are totally different things. Rest is a specifical approach to prrforming data calls. Basically is characterized by a systen where the state is not stored on the server but rather passed in calls. You can read more here
Cors is a technique for enabling javascript to perform data ervice calls to domains otheir than the server donain that they came from. Normally web browsers prevent javascript and other web technologies from doing cross origin or cross domain calls. These are calls where a js script came from google.com lets say, and now it wats to call microsoft.com. well the browser would stop that call because google.com and microsoft.com are different domains.
That example is obvious, so lets try a less obvious one. Your script on blogs.yoursite.com tries to call a service at shopping.yoursite.com. now these sires are both yoursite.com but they could still be considered cross domain and usually are. CORS allows you(on the html developer side) to say i trust these domains. And by trusting them, now you can call their webservices even if they would have been a cross domain call.
I'm in the process of developing a REST service that allows a user to claim their listing based on a couple of pieces of information that appear on their invoice (invoice number and billing zip).
I've read countless articles and Stack Overflow questions about when to use GET and when to use POST. Overall, the common consensus is that GET should be used for idempotent operations and POST should be used for operations that create something on the server side. However, this article:
http://blog.teamtreehouse.com/the-definitive-guide-to-get-vs-post
has caused me to question using GET for this particular scenario, simply because of the fact that I'm using these 2 pieces of information as a mechanism to validate the identity of the user. I'm not updating anything on the server using this particular method call, but I also don't necessarily want to expose the information in the URL.
This is an internal web service and only the front-end that calls the service is publicly exposed, so I don't have to worry about the URL showing up in a user's browser history. My only concern would be the unlikely event that someone gain server log access, in which case, I'd have bigger problems.
I'm leaning toward POST for security reasons; however, GET feels like the correct method due to the fact that the request is idempotent. What is the recommended method in this case?
Independently of POST vs GET, I would recommend NOT basing your security as something as simple as a zip code and an invoice number. I would bet on the fact that invoice numbers are sequential (or close), and there aren't that many zip codes around - voila, I got full access to your listings.
If you're using another authentication method (typically in HTTP header), then you're good - it doesn't matter if you have an invoice number if the URL, so might as well use GET.
If you're not, then I guess POST isn't as bad as GET in term of exposing confidential content.
There isn't really any added security in a POST vs a GET. Sure, the request isn't in the URL, but it's REST we are talking about here, and the URL wouldn't be seen by a human anyway.
You question starts with some bad presumptions. Firstly, GET is not just for any old idempotent operation, it is for GETting resources from the server; it just happens that doing so should be side effect free. Secondly, the URL is not the only way for a GET request to send data to the server, you can use a payload with a GET request (at least as far as HTTP is concerned, some implementations are bad and don't support this or make it hard). Third, as pointed out, you have chosen some terrible data fields to secure your access. Finally, you are using a plain text protocol any way, so what neither method really offers and better security.
You should use the the verb that best describes what you are doing, you are getting some information from the server, so use GET. Use some proper security, such as basic HTTPS encryption. If you want to avoid these fields 'clogging' up the URL, you can send data in the payload of the request, something like:
GET /listings HTTP/1.1
Content-Type = application/json
{ "zip" : "IN0N0USZ1PC0D35",
"invoice" : "54859081145" }