I am working on a REST API for a web application that up until now we have developed internally for a couple of companion applications. Now that we are looking at opening up to outside developers we want to add tokens to the API in order to help identify who is making requests and in general to help manage it's use. At this point we are using https and basic authentication for user authentication on the API.
The token scheme we've been discussing would be very simple where each developer would be assigned 1 or more tokens and these tokens would be passed as a parameter with each request.
My question is if you've done something similar before how did you do it (did you do more or less, how did you handle security, etc) and do you have any recommendations?
First, you might want look at http://OAuth.net. Depending on your usecases, it might provide the security you need.
As to the token, it's a BLOB to most protocols, including OAuth. You can put any information you need in it in any format.
Here is what we do,
First we assign each developer a key with associated secret.
The token itself is an encrypted name-value pairs. We put things like username, expiry, session id, roles etc in there. It's encrypted with our own secret so no one else can make it.
For easy of use with web API, we use the URL-safe version of Base64 so the token is always URL-safe.
Hope that helps!
You might also want to think about maybe adding a time based token that would allow you to limit the amount of time a request is valid. this will help with someone trying to do a replay attack.
You would have a handshake call to get/assign a time valid token based off the above developerKey. This token would be stored locally and passed back to the caller.
The developer would then use this key in a request to validate the request and the developer.
For example that key can then be used for 5 mins or for 10 requests or whatever you define. after that point the generated time based token is removed from the valid list and can no longer be used. the developer will then have to ask for a new token.
UUID is very good for any temporary random key you fancy dishing out. Unpredictable and fast to generate, with collisions so unlikely they are effectively unique. Make nice session keys also.
Related
I have used JWT before, but they were API that didn't need logout feature.
I need to implement logout feature for a API of an android app and SPA. When I looked it up I found that there are two ways to do it.
the easiest way is to delete the JWT Token from client side and call it a day.
The logic behind this is that since no session of any kind is maintained in server deleting the token in client side should be enough.
But it still leaves the possibility that, if the token falls in wrong hands they can still use it even after the user is no longer using the token.
Given if the app is well designed and uses HTTPS then chances of this happening is very low and can be minimized by keeping the valid time for the token short. But in my case the tokens are valid for 30 days.
the second option is to maintain a blacklist of tokens in server side
This solves the problem of the token still being usable even after user has logged out and stopped using it.
But it adds a complication of needing to run a cronjob to remove expired token form the blacklist table. Otherwise the table will eventually become ridiculously large.
It also kinda defeats the point of using JWT. Maintaining blacklist is very similar to maintaining session. We have to run an additional db query for every request. And it scales badly, as the no. of users grows the no. of token that needs to be blacklisted will also grow (this will be a bigger problem for API like mine with multiple front end apps and long validity period for the tokens).
Then I got an idea for third way.
Add jwt_secret row in user table that stores randomly generated string. Use it to sign the JWT Token then on every request use user id in the jwt payload to get the user form db(which is not an extra query, we have to do this anyway) and validate the token signature using jwt_secret of the user. When the user logs out we change the jwt_secret which makes all token out there useless.
At first I thought this was a great solution only to realize that in this setup if user logs out of one device or browser he/she gets logged out of all devices.
So is there a another option? Or a way to modify any of above approach to solve the problem. Or am I over thinking this and one of the above option should be used?
For logging out, which as you pointed out is a user initiated action, I don't think you need to do anything extra. If the user somehow did not delete his JWT, then so be it. He wouldn't be getting any extra access over to what he is already entitled.
However, your question seems to hint on the problem of how to know that a JWT is valid. Again, as you pointed out, if a JWT somehow fell into the wrong hands, then there may be no avoiding this. But, with each request you would typically be doing several types of validation against that JWT, e.g.
checking the claims of the JWT, such as the token expiry date
assuming the claims pass, then checking that user's ID against your database table to make sure the account is active, has not been suspended, etc.
My point here is that if you need to keep track on the server side that a logout has happened, you might need to persist this to a database. But, I don't think you would need this.
Considering that, by definition, a REST API is stateless: is the "logout" operation useless?
I mean, I'm creating a REST API using encrypted JWT. Each token has an expiration time of, let's say, 60 minutes. If I save on a database table the last tokens generated by the API, the "logout" would be done deleting them from the table of valid tokens. But, if I do that, I understand that the API will cease to be stateless, right?
So, I understand that I shouldn't do that. The only solution that I'm thinking is make the JWT expiration time shorter, to 5 minutes, don't implement a "logout" operation and just let the tokens expire.
Is this the correct approach?
I mean, I'm creating a REST API using encrypted JWT
The JSON Web Token (JWT) tokens encodes all the data about the grant into the token itself. The most important advantage of this approach is that you do not need a backend store for token storage at all. One disadvantage is that you can't easily revoke an access token, so they normally are granted with short expiry and the revocation is handled at the refresh token. Another disadvantage is that the tokens can get quite large if you are storing a lot of user credential information in them. So if:
If I save on a database table the last tokens generated by the API,
the "logout" would be done deleting them from the table of valid
tokens
Then you would lose the most important advantage of using JWT and also, still have all those disadvantages, which seems unreasonable to me.
So, I understand that I shouldn't do that. The only solution that I'm
thinking is make the JWT expiration time shorter, to 5 minutes, don't
implement a "logout" operation and just let the tokens expire.
Is this the correct approach?
In my opinion, if you're planning to use JWT, YES! it's better to rely on the token expiration. For more details on this approach you can check this question out.
Is “logout” useless on a REST API?
Regardless of the fact that you're using JWT and similar to any other decent questions on computer science, the answer would be It Depends. The most important advantage of Statelessness is that your API would be more scalable. If you choose this path, probably, every request on your API should be authenticated, since you may need to search a backend store for the given token or decode a JWT token. So, in this case you may have some performance cost on a single node but in a big picture, you would still have the scalability. I guess what i'm trying to say is, if you do not need that scalability, you're better off to choose a Stateful approach. Otherwise, pure REST principles is the way to go.
Automatic token expiry is a separate concern from an explicit "log out" mechanism and, as such, they are both perfectly valid actions regardless of whether your API is ReSTful or not.
When a user logs out they are making a conscious decision to invalidate their access token - for example, if they're using a public computer or borrowing someone else's device temporarily.
Automated expiry is used to ensure that the user must revalidate, in some fashion, on a regular basis. This is good for server-side security.
Access tokens are not about sharing session state between client and server - it's entirely possible to implement an access token system without shared state and the token itself doesn't implement session state, it's only used to verify that the user is who they claim to be. As such, access tokens are not really anything to do with the statefulness of the API.
I think it depends on the behavior that you want for your application, and how secure you need it to be. Do you really need to invalidate the token?
For instance, you could just remove your token from your frontend (browser or app). In theory, it is the only place that stores that particular token. If the token is compromised, it will still be valid until it expires, though.
If you really need to invalidate it server side, a common approach would be to create a blacklist with the token, and clear the expired entries from time to time.
But what if you need your application to accept just one token for each user, like in a bank app that you can only be logged in one device at time? For that purpose the blacklist won't do the job, so you will need to store a single token for each user and check if the passed token is the same. At logout, you would just clear that unique entry. Or you may just use sessions.
So, it is not useless, It just depends on your application.
I would argue that your API is already stateful just by the sheer fact that you have a token around. I also wouldn't get too hung up on REST purity, meaning that everything has to be stateless come hell or high water.
Put simply, if your application requires login, then you need a way to logout. You can't implement a short expiry because that's just going to be a really annoying experience to consumers of the API. And you can't just have no logout at all, because thats a potential security flaw.
I have a similar REST API that I support and I implemented a logout endpoint that is a DELETE call. It simply deletes the token information on the server side and clears any type of authentication for the logged in user.
TL;DR
No, a logout is not useless in a REST API. In fact, for APIs that require authentication, it is more or less a necessity.
With a short expiration time on the token I would think for most applications deleting the token from the client on logout would be a good solution. Anything more would rely on the server and no longer be stateless.
The good solution here would be to delete the token from the user.
So typically when you log in, you will get back a token from the server and store it in localStorage or sessionStorage (depending on the user wanting to be logged in after closing the tab) in the browser, and then send the token from there in the headers with any request that you make to your api.
Then if the user logs out, you don't even contact the api (you don't make any requests to your server), you just clear the sessionStorage or localStorage, use the command localStorage.clear() or sessionStorage.clear() , and then if the user will want to send more requests, he'll have to login again in order to get another token.
One drawback to this approach is, that if a virus, for example gets the token from the local or session Storage before the user logs out then, it will still be able to send requests as you, as the token will still be valid.
One solution to that would be to create a token blacklist in the database, and store the token there if the user logs out, until the token expiration time. However, every time the user would request something, the database would have to be consulted to check if his token is blacklisted, lengthening the process, and making your API stateful.
You can generate a new token that it already expired i.e. expiration is 1sec. and pass it to the user. Any upcoming request will be invalid. This is not optimal solution though..
I'm working with some people who have had a product in development for a while, with apps on various platforms calling a web service which they're using as an API to store data.
The security model they have developed is not completely unlike others i've worked with, but it's very non-standard in a lot of ways. First, they require an API key for each client, which makes sense. Then there are some methods you can hit without authentication, great. when you need to do authenticated updates, however, this is how they do it:
First you call an endpoint with a username/password to get a "session" going on the server side. this session id is returned to the client after a successful login. Everything else you do that requires authentication takes a session id parameter which is required and the idea is that if you don't supply a valid session id then reject your requests.
I spent some time a while back doing authentication using OAuth and requiring tokens for authenticated requests on another project, so this feels weak to me. One obvious issue is the potential for someone to hijack someone's session by means of a brute-force attack, but I expect they'll dismiss that as just plain unlikely. Their session Id's are GUID's, which I suppose are huge and difficult to hack on some scale, plus you have to hit a valid session that's already established.
Am I missing something obvious, or is this perhaps good enough? I don't want to raise a stink about it and propose a platform-wide migration to OAuth if I can't make the case that it's really necessary.
Thanks for any help.
This sounds acceptable, just make sure that GUIDs are indeed generated with a strong random engine that can't be brute-forced, and that they expire after a while of no activity.
First of all, requiring an API key is a form of authentication. It authenticates the client instead of the user.
Requiring an API key on all clients increases the risk of losing the key (especially when stored on untrusted clients like mobile devices).
Using session identifiers is not insecure, but it does require talking to the issuer (and likely doing a database lookup) for every call.
Security tokens can carry semantic information (claims) and can be validated on the called server (verifying signature), which improves scalability.
I'm building a picture diary on web application google app engine using python. Users can sign up and post pictures to their diary.
Also, I'm trying to conform as much as I can to the REST architecture of doing things.
The authentication scheme is based like this for the web application:
1. Post username/password from the frontend
2. Backend sets up a cookie if authentication is successful
3. The rest of the AJAX calls made are authenticated using this cookie.
Is there any way to conform to REST without using cookies ?
Now, I'm also building an android application where users can sign in and post/view their picture diary. I need to expose the data from web application's datastore so I'll be building a webservice to fetch data from the datastore.
The authentication scheme for the android client:
OPTION a
1. Post username/password over https to the web service
2. Web service returns a unique authorization token (store the token in the username/pwd table on the datastore)
3. Request subsequent services by adding this token to the Request Header of the request
4. Server maps the token to the username/pwd table and returns data if token is found
5. Authorization token expires after a certain period of time
OPTION b
1. Set up a secret key on the client and server side
2. Use "username:hash of password and secret key" in the authorization header of every request
3. server generates the password by extracting the password from the hash value using the same hash algorithm ; if successful returns data
btw, I didn't wanna use basic authorization because of its security vulnerabilities.
Which is better ?
Are there other significantly better ways to accomplish what I'm trying to do ? Security is quite a concern for me btw.
I'd appreciate if anyone has any insight into this issue. thanks.
I've been doing some research myself as to what would be the best solution. I think the 2-legged oauth might work in my case as Leonm suggested.
In this case the server has to provide the client with a consumer key/secret which in my case is hardcoded in the app.
The steps now would be:
1. Generate a signature using the oauth_parameters(consumer_key, signature_method, timestamp), request url, request parameters, and the SECRET.
2. Include the signature, oauth parameters when making a request.
3. Server verifies the request by generating the signature again except in this case it uses the SECRET that corresponds to the key
I think this way I am pretty much confirming to the REST principles. The server is statless as I far I understand now.
What are the pros/cons on doing things this way?
If "security is a concern" then I would say that you'd be a lot better off using open standards and a library to achieve what you want. The main reason for this is that if you do it yourself, you're very likely to forget something; these standards have had a lot of eyes looking at them, looking for holes.
Your options include (in increasing level of complexity)
Basic authentication and HTTPS
Everything is encrypted, which makes it impossible to compress or look into, it increases the overhead somewhat, using more horsepower on the server, and more perhaps battery power on the client. Simple to implement, since it's well supported by libraries.
Digest authentication
Unencrypted messages pass the wire, but the authentication is securely managed in the Authorization headers. See the wikipedia entry for more information.
OAuth
See how Google is providing OAuth for installed applications. I believe it isn't what you're looking for, since you're not asking to share data between applications, just authenticating users.
Roll your own
If you want to roll your own, I suggest looking at e.g. how Google's (now deprecated ?) ClientLogin used to work.
Clients would GET a protected resource, and get a 401 with instructions to perform a GoogleLogin authentication, including a URI for where to perform the login itself
Clients (knowing how to do this) POST a request in a specific manner to that URI
The server responds with a specific response including a (long) token
The client can now perform GET requests to the protected resource with that token.
Statelessness
You cite REST, which dictates that requests should not specifically depend on prior interaction: "... each request from client to server must contain all of the information necessary to understand the request, and cannot take advantage of any stored context on the server." (fielding) This means that a server shouldn't store conversational context (like an authentication token) in a table.
One way of fixing this is by using any of the token based approaches (where the server tells the client about a token it should use for future requests) where the token is not a random number, but a message to the server itself. To protect yourself from client tampering, it can be signed, and if you're afraid of clients looking at it, you can encrypt it.
Edit: Although I'm not certain, it seems unlikely that Google has a table of all authentication tokens ever issued; The length of their tokens suggests that the token is some encrypted message proving that whoever holds this token actually provided real credentials in some realm at some time.
OAuth does exactly what you want to do in a standard way.
You could use a combination of HTTPS and HTTP Basic Auth. Both are existing standards and should be secure enough when used together.
When using the OAuth protocol, you need a secret string obtained from the service you want to delegate to. If you are doing this in a web app, you can simply store the secret in your data base or on the file system, but what is the best way to handle it in a mobile app (or a desktop app for that matter)?
Storing the string in the app is obviously not good, as someone could easily find it and abuse it.
Another approach would be to store it on your server, and have the app fetch it on every run, never storing it on the phone. This is almost as bad, because you have to include the URL in the app.
The only workable solution I can come up with is to first obtain the Access Token as normal (preferably using a web view inside the app), and then route all further communication through our server, which would append the secret to the request data and communicate with the provider. Then again, I'm a security noob, so I'd really like to hear some knowledgeable peoples' opinions on this. It doesn't seem to me that most apps are going to these lengths to guarantee security (for example, Facebook Connect seems to assume that you put the secret into a string right in your app).
Another thing: I don't believe the secret is involved in initially requesting the Access Token, so that could be done without involving our own server. Am I correct?
Yes, this is an issue with the OAuth design that we are facing ourselves. We opted to proxy all calls through our own server. OAuth wasn't entirely flushed out in respect of desktop apps. There is no prefect solution to the issue that I've found without changing OAuth.
If you think about it and ask the question why we have secrets, is mostly for provision and disabling apps. If our secret is compromised, then the provider can only really revoke the entire app. Since we have to embed our secret in the desktop app, we are sorta screwed.
The solution is to have a different secret for each desktop app. OAuth doesn't make this concept easy. One way is have the user go and create an secret on their own and enter the key on their own into your desktop app (some facebook apps did something similar for a long time, having the user go and create facebook to setup their custom quizes and crap). It's not a great experience for the user.
I'm working on proposal for a delegation system for OAuth. The concept is that using our own secret key we get from our provider, we could issue our own delegated secret to our own desktop clients (one for each desktop app basically) and then during the auth process we send that key over to the top level provider that calls back to us and re-validates with us. That way we can revoke on own secrets we issue to each desktop client. (Borrowing a lot of how this works from SSL). This entire system would be prefect for value-add webservices as well that pass on calls to a third party webservice.
The process could also be done without delegation verification callbacks if the top level provider provides an API to generate and revoke new delegated secrets. Facebook is doing something similar by allowing facebook apps to allow users to create sub-apps.
There are some talks about the issue online:
http://blog.atebits.com/2009/02/fixing-oauth/
http://groups.google.com/group/twitter-development-talk/browse_thread/thread/629b03475a3d78a1/de1071bf4b820c14#de1071bf4b820c14
Twitter and Yammer's solution is a authentication pin solution:
https://dev.twitter.com/oauth/pin-based
https://www.yammer.com/api_oauth_security_addendum.html
With OAUth 2.0, you can store the secret on the server. Use the server to acquire an access token that you then move to the app and you can make calls from the app to the resource directly.
With OAuth 1.0 (Twitter), the secret is required to make API calls. Proxying calls through the server is the only way to ensure the secret is not compromised.
Both require some mechanism that your server component knows it is your client calling it. This tends to be done on installation and using a platform specific mechanism to get an app id of some kind in the call to your server.
(I am the editor of the OAuth 2.0 spec)
One solution could be to hard code the OAuth secret into the code, but not as a plain string. Obfuscate it in some way - split it into segments, shift characters by an offset, rotate it - do any or all of these things. A cracker can analyse your byte code and find strings, but the obfuscation code might be hard to figure out.
It's not a foolproof solution, but a cheap one.
Depending on the value of the exploit, some genius crackers can go to greater lengths to find your secret code. You need to weigh the factors - cost of previously mentioned server side solution, incentive for crackers to spend more efforts on finding your secret code, and the complexity of the obfuscation you can implement.
Do not store the secret inside the application.
You need to have a server that can be accessed by the application over https (obviously) and you store the secret on it.
When someone want to login via your mobile/desktop application, your application will simply forward the request to the server that will then append the secret and send it to the service provider. Your server can then tell your application if it was successful or not.
Then if you need to get any sensitive information from the service (facebook, google, twitter, etc), the application ask your server and your server will give it to the application only if it is correctly connected.
There is not really any option except storing it on a server. Nothing on the client side is secure.
Note
That said, this will only protect you against malicious client but not client against malicious you and not client against other malicious clients (phising)...
OAuth is a much better protocol in browser than on desktop/mobile.
There is a new extension to the Authorization Code Grant Type called Proof Key for Code Exchange (PKCE). With it, you don't need a client secret.
PKCE (RFC 7636) is a technique to secure public clients that don't use
a client secret.
It is primarily used by native and mobile apps, but the technique can
be applied to any public client as well. It requires additional
support by the authorization server, so it is only supported on
certain providers.
from https://oauth.net/2/pkce/
For more information, you can read the full RFC 7636 or this short introduction.
Here's something to think about. Google offers two methods of OAuth... for web apps, where you register the domain and generate a unique key, and for installed apps where you use the key "anonymous".
Maybe I glossed over something in the reading, but it seems that sharing your webapp's unique key with an installed app is probably more secure than using "anonymous" in the official installed apps method.
With OAuth 2.0 you can simply use the client side flow to obtain an access token and use then this access token to authenticate all further requests. Then you don't need a secret at all.
A nice description of how to implement this can be found here: https://aaronparecki.com/articles/2012/07/29/1/oauth2-simplified#mobile-apps
I don't have a ton of experience with OAuth - but doesn't every request require not only the user's access token, but an application consumer key and secret as well? So, even if somebody steals a mobile device and tries to pull data off of it, they would need an application key and secret as well to be able to actually do anything.
I always thought the intention behind OAuth was so that every Tom, Dick, and Harry that had a mashup didn't have to store your Twitter credentials in the clear. I think it solves that problem pretty well despite it's limitations. Also, it wasn't really designed with the iPhone in mind.
I agree with Felixyz. OAuth whilst better than Basic Auth, still has a long way to go to be a good solution for mobile apps. I've been playing with using OAuth to authenticate a mobile phone app to a Google App Engine app. The fact that you can't reliably manage the consumer secret on the mobile device means that the default is to use the 'anonymous' access.
The Google App Engine OAuth implementation's browser authorization step takes you to a page where it contains text like:
"The site <some-site> is requesting access to your Google Account for the product(s) listed below"
YourApp(yourapp.appspot.com) - not affiliated with Google
etc
It takes <some-site> from the domain/host name used in the callback url that you supply which can be anything on the Android if you use a custom scheme to intercept the callback.
So if you use 'anonymous' access or your consumer secret is compromised, then anyone could write a consumer that fools the user into giving access to your gae app.
The Google OAuth authorization page also does contain lots of warnings which have 3 levels of severity depending on whether you're using 'anonymous', consumer secret, or public keys.
Pretty scary stuff for the average user who isn't technically savvy. I don't expect to have a high signup completion percentage with that kind of stuff in the way.
This blog post clarifies how consumer secret's don't really work with installed apps.
http://hueniverse.com/2009/02/should-twitter-discontinue-their-basic-auth-api/
Here I have answer the secure way to storing your oAuth information in mobile application
https://stackoverflow.com/a/17359809/998483
https://sites.google.com/site/greateindiaclub/mobil-apps/ios/securelystoringoauthkeysiniosapplication
Facebook doesn't implement OAuth strictly speaking (yet), but they have implemented a way for you not to embed your secret in your iPhone app: https://web.archive.org/web/20091223092924/http://wiki.developers.facebook.com/index.php/Session_Proxy
As for OAuth, yeah, the more I think about it, we are a bit stuffed. Maybe this will fix it.
None of these solutions prevent a determined hacker from sniffing packets sent from their mobile device (or emulator) to view the client secret in the http headers.
One solution could be to have a dynamic secret which is made up of a timestamp encrypted with a private 2-way encryption key & algorithm. The service then decrypts the secret and determines if the time stamp is +/- 5 minutes.
In this way, even if the secret is compromised, the hacker will only be able to use it for a maximum of 5 minutes.
I'm also trying to come up with a solution for mobile OAuth authentication, and storing secrets within the application bundle in general.
And a crazy idea just hit me: The simplest idea is to store the secret inside the binary, but obfuscated somehow, or, in other words, you store an encrypted secret. So, that means you've got to store a key to decrypt your secret, which seems to have taken us full circle. However, why not just use a key which is already in the OS, i.e. it's defined by the OS not by your application.
So, to clarify my idea is that you pick a string defined by the OS, it doesn't matter which one. Then encrypt your secret using this string as the key, and store that in your app. Then during runtime, decrypt the variable using the key, which is just an OS constant. Any hacker peeking into your binary will see an encrypted string, but no key.
Will that work?
As others have mentioned, there should be no real issue with storing the secret locally on the device.
On top of that, you can always rely on the UNIX-based security model of Android: only your application can access what you write to the file system. Just write the info to your app's default SharedPreferences object.
In order to obtain the secret, one would have to obtain root access to the Android phone.