Background:
I'm designing the authentication scheme for a REST web service. This doesn't "really" need to be secure (it's more of a personal project) but I want to make it as secure as possible as an exercise/learning experience. I don't want to use SSL since I don't want the hassle and, mostly, the expense of setting it up.
These SO questions were especially useful to get me started:
RESTful Authentication
Best Practices for securing a REST API / web service
Examples of the best SOAP/REST/RPC web APIs? And why do you like them? And what’s wrong with them?
I'm thinking of using a simplified version of Amazon S3's authentication (I like OAuth but it seems too complicated for my needs). I'm adding a randomly generated nonce, supplied by the server, to the request, to prevent replay attacks.
To get to the question:
Both S3 and OAuth rely on signing the request URL along with a few selected headers. Neither of them sign the request body for POST or PUT requests. Isn't this vulnerable to a man-in-the-middle attack, which keeps the url and headers and replaces the request body with any data the attacker wants?
It seems like I can guard against this by including a hash of the request body in the string that gets signed. Is this secure?
A previous answer only mentioned SSL in the context of data transfer and didn't actually cover authentication.
You're really asking about securely authenticating REST API clients. Unless you're using TLS client authentication, SSL alone is NOT a viable authentication mechanism for a REST API. SSL without client authc only authenticates the server, which is irrelevant for most REST APIs because you really want to authenticate the client.
If you don't use TLS client authentication, you'll need to use something like a digest-based authentication scheme (like Amazon Web Service's custom scheme) or OAuth 1.0a or even HTTP Basic authentication (but over SSL only).
These schemes authenticate that the request was sent by someone expected. TLS (SSL) (without client authentication) ensures that the data sent over the wire remains untampered. They are separate - but complementary - concerns.
For those interested, I've expanded on an SO question about HTTP Authentication Schemes and how they work.
REST means working with the standards of the web, and the standard for "secure" transfer on the web is SSL. Anything else is going to be kind of funky and require extra deployment effort for clients, which will have to have encryption libraries available.
Once you commit to SSL, there's really nothing fancy required for authentication in principle. You can again go with web standards and use HTTP Basic auth (username and secret token sent along with each request) as it's much simpler than an elaborate signing protocol, and still effective in the context of a secure connection. You just need to be sure the password never goes over plain text; so if the password is ever received over a plain text connection, you might even disable the password and mail the developer. You should also ensure the credentials aren't logged anywhere upon receipt, just as you wouldn't log a regular password.
HTTP Digest is a safer approach as it prevents the secret token being passed along; instead, it's a hash the server can verify on the other end. Though it may be overkill for less sensitive applications if you've taken the precautions mentioned above. After all, the user's password is already transmitted in plain-text when they log in (unless you're doing some fancy JavaScript encryption in the browser), and likewise their cookies on each request.
Note that with APIs, it's better for the client to be passing tokens - randomly generated strings - instead of the password the developer logs into the website with. So the developer should be able to log into your site and generate new tokens that can be used for API verification.
The main reason to use a token is that it can be replaced if it's compromised, whereas if the password is compromised, the owner could log into the developer's account and do anything they want with it. A further advantage of tokens is you can issue multiple tokens to the same developers. Perhaps because they have multiple apps or because they want tokens with different access levels.
(Updated to cover implications of making the connection SSL-only.)
Or you could use the known solution to this problem and use SSL. Self-signed certs are free and its a personal project right?
If you require the hash of the body as one of the parameters in the URL and that URL is signed via a private key, then a man-in-the-middle attack would only be able to replace the body with content that would generate the same hash. Easy to do with MD5 hash values now at least and when SHA-1 is broken, well, you get the picture.
To secure the body from tampering, you would need to require a signature of the body, which a man-in-the-middle attack would be less likely to be able to break since they wouldn't know the private key that generates the signature.
In fact, the original S3 auth does allow for the content to be signed, albeit with a weak MD5 signature. You can simply enforce their optional practice of including a Content-MD5 header in the HMAC (string to be signed).
http://s3.amazonaws.com/doc/s3-developer-guide/RESTAuthentication.html
Their new v4 authentication scheme is more secure.
http://docs.aws.amazon.com/general/latest/gr/signature-version-4.html
Remember that your suggestions makes it difficult for clients to communicate with the server. They need to understand your innovative solution and encrypt the data accordingly, this model is not so good for public API (unless you are amazon\yahoo\google..).
Anyways, if you must encrypt the body content I would suggest you to check out existing standards and solutions like:
XML encryption (W3C standard)
XML Security
Related
I'm trying to secure the rest API endpoint with JWT. I looked at a few related questions here and here.
I have some sensitive information in the JWT but I'm using HTTPS, so the data is already encrypted so would that be enough, or do I have to encrypt JWT ?
HTTPS gives you encryption in transport. This means that if anyone intercepts your message they will not be able to read it. But the secure connection is terminated by the browser (at the user side) and very often by a load balancer or API gateway at the server side. This means that the JWT will be freely available to anyone with access to the browser. There is also a possibility that it will be read or stolen from inside your network (once it's past the Gateway/load balancer). This one is a much lower threat, but it nevertheless exists.
If you have sensitive information in the JWT, then there are two options:
You can encrypt JWTs (so use JWEs). This way the token's contents will be secure in the browser. Even if someone reads or intercepts the token there, they will not be able to read it. JWE is a bit tricky to set up, though.
Use the Phantom Token pattern. In this pattern, you issue opaque tokens to the client and have your API Gateway exchange the opaque token for a JWT. This way your sensitive information is kept away from the browser but your APIs can still benefit from the power of the JWT. It also doesn't require setting up encryption. It leaves you vulnerable to any malicious actors inside your network (either from your organization or someone who manages to breach your defenses), but this is a much lower risk.
I have been looking into REST authentication schemes (many discussed on SO), and many of them seem to be overly complex for my purposes. I have formulated a simpler scheme from elements of the more complex ones: but I would like to know if there are any security holes in my approach.
Influencing factors:
TLS everywhere is too slow and resource heavy
I do not require security against eavesdropping as all information is public.
Proposed authentication scheme:
"Sign up" and "Login" are achieved via a TLS connection. On Login, a username and password are supplied and a shared secret key is returned by the server (and then stored in local storage by the client e.g. HTML5 local storage, App storage, etc).
Every other request takes place over cleartext HTTP
Client side algorithm:
Before sending, every request is "salted" with the shared secret key and an SHA hash is taken of the salted request.
This hash is inserted into the request in a custom HTTP header.
The salt is removed from the request.
The request is sent with the custom header.
Server side algorithm:
Server isolates and removes the custom Hash header from the request.
Server salts the request string with the shared secret key.
Server takes the hash of the salted request and compares it to the value of the custom hash header.
If they are the same, we have identified which user sent the request and can proceed with authorisation etc based on this knowledge.
Are there any vulnerabilities in this scheme that I have overlooked?
I would question your assumptions here.
TLS everywhere is too slow and resource heavy
TLS is becoming almost ubiquitous for APIs and one reason is because it is now relatively cheap for both clients and servers to support it. How much overhead? As usual "it depends" but certainly negligible enough for most modern APIs, even mass-consumer APIs like Facebook and Twitter, to be moving towards using it exclusively.
I do not require security against eavesdropping as all information is public.
This is a common myth about TLS. Even for public data, consider the implications:
Any intermediary agent can inject their own requests and responses. It could be junk, malicious, subtly incorrect, whatever. Secure communication is not just to keep the content private, it's also to maintain its integrity. (A common example is telcos and hotels injecting ads into websites.)
The data may be public, but the traffic may still be sensitive. What if you could monitor Apple's Wikipedia requests? Would it be interesting if there was a spike in requests to car-related articles? Without TLS, intermediaries can monitor requests a user is making.
None of which critiques your algorithm. You could ask on Cryptography Stack, but it's considered fairly risky to roll your own authentication and rarely worth it nowadays.
What you are describing is an MAC based authentication scheme. Instead of rolling your own implementation, you should look at Hawk or AWS authentication schemes.
A downside of such an authentication scheme is that the server that needs to validate the request needs to talk to the authentication server to get the secret key. This impacts the scalability of the system in a negative way.
Token based authentication schemes can validate the request without going back to the token issuing authority due to digital signatures.
Finally, I agree with #mahemoff that TLS is becoming ubiquitous and very cheap. Actually, depending on the circumstances, HTTPS may outperform HTTP.
I've just read this very interesting article: Principles for Standardized REST Authentication and I'm wondering why one should sign REST queries even when using SSL. In my understanding, signing REST queries lets the server ensure requests come from trusted clients.
Having said that, is signing really necessary considering that SSL also protects against man-in-the-middle attacks?
As stated on the Wikipedia article for HTTPS:
[...] HTTPS provides authentication of the web site and associated web server that one is communicating with, which protects against man-in-the-middle attacks. Additionally, it provides bidirectional encryption of communications between a client and server, which protects against eavesdropping and tampering with and/or forging the contents of the communication. In practice, this provides a reasonable guarantee that one is communicating with precisely the web site that one intended to communicate with (as opposed to an imposter), as well as ensuring that the contents of communications between the user and site cannot be read or forged by any third party. [...]
This is why you need HTTPS, so that the client "is sure" that it's requests are sent to the proper destination. The article you linked also says this:
If you are not validating the SSL certificate of the server, you don't know who is receiving your REST queries.
But HTTPS normally does not authenticate the client unless you configure the server to request a certificate from the client in order to perform mutual authentication. If you read the comments in the post you linked you will see people mentioning this:
If you are going to use https, why not use it fully, and ask for client side certificates too? Then you get a fully RESTful authentication method, because the client and the server are authenticated at the connection layer, and there is no need to bring authentication into the URI level.
But HTTPS with client-side certificates is more expensive and complex so most API providers keep "the normal" HTTPS to identify the server and use a lighter mechanism to identify the clients: the API keys. The API keys basically consist of a name which is public - for example "Johnny" - and a secret key which is private - for example a long string of randomly generated characters.
When you make a request to the server you include the name "Johnny" in the URL so that the server knows who sent the request. But the server doesn't just blindly trust you that you are "Johnny", you have to prove it by signing the request with the secret key which, because it's private, only the real "Johnny" knows.
A digital signature has legal implications such as non-repudiation, which any value transaction should require. It's not just a matter of authentication. A digital signature on an actual transaction is a much stronger piece of evidence in court than 'this conversation was carried out over SSL with mutual authentication so it must have been the defendant Your Honour'.
I'm writing REST API in PHP and recently I faced with authorization problem. I read a lot about basic authorization, about using private and public keys to create request signature. It is said that using request signature is more secure. But then I have a question:
-How should user will pass public key and generated signature?
I'm thinking about several options:
1) Create custom http header like X-Key, X-Signature
2) Use authorization header with custom scheme, like
AUTHORIZATION: SIGNATURE key='123' signature='abc'
3) Send this values as parameters. But I don't know if it acceptable for methods DELETE and PUT
What would you advice?
p.s. I don't want to implement oAuth
What are the desired properties of authentication scheme? Is this a publicly accessible or an intranet service? Are user accounts linked to something outside of scope of your API (linked 3rd party accounts etc). How are you going to distribute user credentials?
I would probably stick with plain old basic authorization, but encrypt everything at the transport level, making use of HTTPS mandatory. Rolling out your own cryptographic scheme is generally not a good idea. It's easier to fall victim to timing or replay attack than it seems. If you insist on client using a key pair for authentication, you can use HTTPS client certificates (though this is not widely used and maybe somewhat cumbersome solution).
There are a few security concerns about plain-text authentication over TLS. First, if someone implements MITM with forged certificate using either well known CA (maybe a government agency) or CA the client is forced to trust (big evil corporate proxy), they will get credentials. But you can't protect the client from its own environment anyway. Second, basic authentication can be prone to CSRF because browser knows how to do it and can remember credentials if you presented challenge and user filled the form. That's not a big problem if you adhere to REST principles and never allow state-changing GET requests. Also, if you are using JSON, never return arrays.
I'm building a picture diary on web application google app engine using python. Users can sign up and post pictures to their diary.
Also, I'm trying to conform as much as I can to the REST architecture of doing things.
The authentication scheme is based like this for the web application:
1. Post username/password from the frontend
2. Backend sets up a cookie if authentication is successful
3. The rest of the AJAX calls made are authenticated using this cookie.
Is there any way to conform to REST without using cookies ?
Now, I'm also building an android application where users can sign in and post/view their picture diary. I need to expose the data from web application's datastore so I'll be building a webservice to fetch data from the datastore.
The authentication scheme for the android client:
OPTION a
1. Post username/password over https to the web service
2. Web service returns a unique authorization token (store the token in the username/pwd table on the datastore)
3. Request subsequent services by adding this token to the Request Header of the request
4. Server maps the token to the username/pwd table and returns data if token is found
5. Authorization token expires after a certain period of time
OPTION b
1. Set up a secret key on the client and server side
2. Use "username:hash of password and secret key" in the authorization header of every request
3. server generates the password by extracting the password from the hash value using the same hash algorithm ; if successful returns data
btw, I didn't wanna use basic authorization because of its security vulnerabilities.
Which is better ?
Are there other significantly better ways to accomplish what I'm trying to do ? Security is quite a concern for me btw.
I'd appreciate if anyone has any insight into this issue. thanks.
I've been doing some research myself as to what would be the best solution. I think the 2-legged oauth might work in my case as Leonm suggested.
In this case the server has to provide the client with a consumer key/secret which in my case is hardcoded in the app.
The steps now would be:
1. Generate a signature using the oauth_parameters(consumer_key, signature_method, timestamp), request url, request parameters, and the SECRET.
2. Include the signature, oauth parameters when making a request.
3. Server verifies the request by generating the signature again except in this case it uses the SECRET that corresponds to the key
I think this way I am pretty much confirming to the REST principles. The server is statless as I far I understand now.
What are the pros/cons on doing things this way?
If "security is a concern" then I would say that you'd be a lot better off using open standards and a library to achieve what you want. The main reason for this is that if you do it yourself, you're very likely to forget something; these standards have had a lot of eyes looking at them, looking for holes.
Your options include (in increasing level of complexity)
Basic authentication and HTTPS
Everything is encrypted, which makes it impossible to compress or look into, it increases the overhead somewhat, using more horsepower on the server, and more perhaps battery power on the client. Simple to implement, since it's well supported by libraries.
Digest authentication
Unencrypted messages pass the wire, but the authentication is securely managed in the Authorization headers. See the wikipedia entry for more information.
OAuth
See how Google is providing OAuth for installed applications. I believe it isn't what you're looking for, since you're not asking to share data between applications, just authenticating users.
Roll your own
If you want to roll your own, I suggest looking at e.g. how Google's (now deprecated ?) ClientLogin used to work.
Clients would GET a protected resource, and get a 401 with instructions to perform a GoogleLogin authentication, including a URI for where to perform the login itself
Clients (knowing how to do this) POST a request in a specific manner to that URI
The server responds with a specific response including a (long) token
The client can now perform GET requests to the protected resource with that token.
Statelessness
You cite REST, which dictates that requests should not specifically depend on prior interaction: "... each request from client to server must contain all of the information necessary to understand the request, and cannot take advantage of any stored context on the server." (fielding) This means that a server shouldn't store conversational context (like an authentication token) in a table.
One way of fixing this is by using any of the token based approaches (where the server tells the client about a token it should use for future requests) where the token is not a random number, but a message to the server itself. To protect yourself from client tampering, it can be signed, and if you're afraid of clients looking at it, you can encrypt it.
Edit: Although I'm not certain, it seems unlikely that Google has a table of all authentication tokens ever issued; The length of their tokens suggests that the token is some encrypted message proving that whoever holds this token actually provided real credentials in some realm at some time.
OAuth does exactly what you want to do in a standard way.
You could use a combination of HTTPS and HTTP Basic Auth. Both are existing standards and should be secure enough when used together.