So, I have a RESTful API (built with Hapi.js) that has endpoints consumed by users and my front-end app (built with Next.js). GET api/candies is one of them, I'll take it as an example.
The front-end asks the list of candies stored in my DB and displays them on a page anyone can access (it has to be this way). The front-end doesn't provide an API token since people could read/use it. But, users who want to get this list of candies (to build whatever they want with it) must provide a valid API token (which they get by creating an account on my front-end app).
How could my API tell if a request for api/candies is from a user or from my front-end app, so it can verify (or not) the validity of their token?
I'm wondering if my problem isn't also about web scraping.
Can anyone help me please? :D
I thought about the same problem a while ago. If your frontend has a client side REST client (JS+XHR/fetch), then I don't think it is possible to do this reliably, because no matter how you identify your frontend REST client, your users will be able to copy it just by checking the HTTP requests in browser via CTRL+SHIFT+I. There are even automation tools, which use the browser e.g. Selenium. If you have a server side REST client (e.g. PHP+CURL), then just create a consumer id for the frontend and use a token. Even in this case I can easily write a few lines of code that uses the frontend for the same request. So if you want to sell the same service for money that you provide for free on your frontend, then you are out of luck here. This does not mean that there won't be consumers who are willing to pay for it.
I think your problem is bad business model.
Your requirement can be addressed by inspecting different headers sent by different user agents. You can also add custom headers from your front-end and validate the same on the backend.
Related
I am a bit confused. The requirement is that we need to create a REST API in Salesforce(Apex class) that has one POST method. Right now, I have been testing it with POSTMAN tool in 2 steps:
Making a POST request first with username, password, client_id, client_secret(that are coming from connected app in Salesforce), grant_type to receive access token.
Then I make another POST request in POSTMAN to create a lead in Salesforce, using the access token I received before and the body.
However, the REST API that I have in Salesforce would be called from various different web forms. So once someone fills out the webform, on the backend it would call this REST API in Salesforce and submits lead request.
I am wondering how would that happen since we can't use POSTMAN for that.
Thanks
These "various different web forms" would have to send requests to Salesforce just like Postman does. You'd need two POST calls (one for login, one to call the service you've created). It'll be bit out of your control, you provided the SF code and proven it works, now it's for these website developers to pick it up.
What's exactly your question? There are tons of libraries to connect to SF from Java, Python, .NET, PHP... Or they could hand-craft these HTTP messages, just Google for "PHP HTTP POST" or something...
https://developer.salesforce.com/index.php?title=Getting_Started_with_the_Force.com_Toolkit_for_PHP&oldid=51397
https://github.com/developerforce/Force.com-Toolkit-for-NET
https://pypi.org/project/simple-salesforce/ / https://pypi.org/project/salesforce-python/
Depending how much time they'll have they can:
cache the session id (so they don't call login every time), try to reuse it, call login again only if session id is blank / got "session expired or invalid" error back
try to batch it somehow (do they need to save these Leads to SF asap or in say hourly intervals is OK? How did YOU write the service, accepts 1 lead or list of records?
be smart about storing the credentials to SF (some secure way, not hardcoded). Ideally in a way that it's easy to use the integration against sandbox or production changing just 1 config file or environment variables or something like that
I am developing a web application with Spring Boot and a React.js SPA, but my question is not specific to those libraries/frameworks, as i assume reporting client-side JS errors to the server (for logging and analyzing) must be a common operation for many modern web applications.
So, suppose we have a JS client application that catches an error and a REST endpoint /errors that takes a JSON object holding the relevant information about what happened. The client app sends the data to the server, it gets stored in a database (or whatever) and everyone's happy, right?
Now I am not, really. Because now I have an open (as in allowing unauthenticated create/write operations) API endpoint everyone with just a little knowledge could easily spam.
I might validate the structure of JSON data the endpoint accepts, but that doesn't really solve the problem.
In questions like "Open REST API attached to a database- what stops a bad actor spamming my db?" or "Secure Rest-Service before user authentification", there are suggestions such as:
access quotas (but I don't want to save IPs or anything to identify clients)
Captchas (useless for error reporting, obviously)
e-mail verification (same, just imagine that)
So my questions are:
Is there an elegant, commonly used strategy to secure such an endpoint?
Would a lightweight solution like validating the structure of the data be enough in practice?
Is all this even necessary? After all I won't advertise my error handling API endpoint with a banner in the app...
I’ve seen it done three different ways…
Assuming you are using OAuth 2 to secure your API. Stand up two
error endpoints.
For a logged in user, if an errors occurs you would
hit the /error endpoint, and would authenticate using the existing
user auth token.
For a visitor, you can expose a /clientError (or
named in a way that makes sense to you) endpoint that takes the
client_credentials token for the client app.
Secure the /error endpoint using an api key that would be scope for
access to the error endpoint only.
This key would be specific to the
client and would be pass in the header.
Use a 3rd party tool such as Raygun.io, or any APM tool, such as New Relic.
I am writing a REST Api gateway for an Angular SPA and I am confronted with the problem of securing the data exposed by the API for the SPA against "data thiefs". I am aware that I can't do much against HTML scraping, but at least I don't want to offer such data thiefs the user experience and full power of our JSON sent to the SPA.
The difference between most "tutorials" and threads about this topic is that I am exposing this data to a public website (which means no user authentication required) which offers valuable statistics about a video game.
My initial idea on how to protect the Rest API for SPA:
Using JWTs everywhere. When a visitor opens the website the very first time the SPA requests a JWT from my REST Api and saves it in the HTTPS cookies. For all requests the SPA has to use the JWT to get a response.
Problems with that approach
The data thief could simply request the oauth token from our endpoint as well. I have no chance to verify that the token has actually been requested from my SPA or from the data thief?
Even if I solved that the attacker could read the saved JWT from the HTTPS cookies and use it in his own application. Sure I could add time expiration for the JWT
My question:
I am under the impression that this is a common problem and therefore I am wondering if there are any good solutions to protect against others than the SPA having direct access to my REST Api responses?
From the API's point of view, your SPA is in no way different than any other client. You obviously can't include a secret in the SPA as it is sent to anybody and cannot be protected. Also the requests it makes to the API can be easily sniffed and copied by another client.
So in short, as diacussed many times here, you can't authenticate the client application. Anybody can create a different client if they want.
One thing you can actually do is checking the referer/origin of requests. If a client is running in a browser, thr requests it can make are somewhat limited, and one such limitation is the referer and origin headers, which are always controlled by the browser, and not javascript. So you can actually make sure that if (and only if!) the client is running in an unmodified browser, it is downloaded from your domain. This is the default in browsers btw, so if you are not sending CORS headers, you already did this (browsers do, actually). However, this does not keep an attacker from building and running a non-browser client and fake any referer or origin he likes, or just disregard the same origin policy.
Another thing you could do is changing the API regularly just enough to stop rogue clients from working (and changing your client at the same time ofc). Obviously this is not secure at all, but can be annoying enough for an attacker. If downloading all your data once is a concern, this again doesn't help at all.
Some real things you should consider though are:
Does anybody actually want to download your data? How much is it worth? Most of the times nobody wants to create a different client, and nobody is that much interested in the data.
If it is that interesting, you should implement user authentication at the very least, and cover the remaining risk either via points below and/or in your contracts legally.
You could implement throttling to not allow bulk downloading. For example if the typical user accesses 1 record every 5 seconds, and 10 altogether, you can build rules based on the client IP for example to reasonably limit user access. Note though that rate limiting must be based on a parameter the client can't modify arbitrarily, and without authentication, that's pretty much the client IP only, and you will face issues with users behind a NAT (ie. corporate networks for example).
Similarly, you can implement monitoring to discover if somebody is downloading more data than it would be normal or necessary. However, without user authentication, your only option will be to ban the client IP. So again it comes down to knowing who the user is, ie. authentication.
I have seen many different posts regarding different solutions for authenticating a RESTful API and I have some questions, given this current scenario.
I've built a REST API that will allow clients of my software service (we are a B2B company) to access resources programmatically. Now that I've got the API working properly, I'd like to secure it in the most standardized way possible. I need to allow access to certain resources based on the caller of the API. That is to say, not all users of the API can access all resources.
I have URLS available in the following formats:
https://mydomain/api/students
https://mydomain/api/students/s123
https://mydomain/api/students/s123/classes
https://mydomain/api/students/s123/classes/c456
So far I've come up with these possible solutions:
Provide a unique key to each client that they can use to ultimately generate an encrypted token that will be passed as a GET parameter at the end of each REST call to (re)-authenticate every single request. Is this approach too expensive
https://mydomain.com/api/students/s123?token=abc123
Provide a value in the HTTP Authorization Header as seen here. Is this almost the same as #1? (Except I can't paste a URL into the browser) Do people use these headers anymore?
Use OAuth 2 (which I'm still a bit unclear on). Does OAuth 2 actually authenticate the client as a logged in user? And doesn't that go against the spirit of a REST API being stateless? I was hoping OAuth was the proper solution for me (since it's a public standard), but after reading up on it a little bit, I'm not so sure. Is it overkill and/or improper for REST API calls?
My goal is to provide an API that will not have to be changed for each client that wants to consume the API, but rather that I can provide a standard documentation made available to all of our clients.
I'll be happy to post additional details if I've been unclear.
There are 2 type of clients you probably want to prepare your API:
trusted clients - Which are written by you. They can have the username and password of the actual user, and they can send that data to your server with every request, possibly in a HTTP auth header. All you need is an encrypted connection by them.
3rd party clients - Which are written by some random developer. You can register them in your service and add a unique API key to each of them. After that if an user wants to use their services, you have to show her a prompt in which she can allow access to the 3rd party client. After that the 3rd party client will be assigned to the user's account with the given permissions and it will get an user specific access token. So when the client sends its API key and the user specific token along with the request, then it sends the requests in the name of the user.
OAuth can help you to control the second situation.
Your URLs do not have a meaning to the clients. By REST you have to decouple the clients from the URL structure by sending links annotated with semantics (e.g. link relations). So your documentation does not have to contain anything about the URL structure (maybe it can be useful for server side debug, but nothing more). You have to talk about different types of links. By generating these links on server side, you can check the permissions of the actual user (or 3rd party client) and skip the links which she does not have permission to follow.
We intend to develop rest based api. I explored the topic but it seems, you can secure api when your client is an app (So there are many ways, public key - private key etc). What about websites / mobile website, if we are accessing rest based api in website which do not use any login for accessing contents ( login would be optional ) then how could we restrict other people from accessing rest based api ?
Does it make sense using Oauth2.0 ? I don't have clear idea of that.
More clear question could be ,How can we secure get or post request exposed over web for the website which doesn't use any login ?
If it's simple get request or post request , which will return you json data on specific input, now i have mobile website , who will access those data using get request or post request to fetch data. Well, some else can also access it , problem is i am not using Login, user can access data directly. But how can we restrict other people from accessing that data.
What do you think is the difference between securing a website that is not using REST vs one that is using REST API?
OAuth provides authorisation capabilities for your site, in a REST architecture this means a user of the mobile application will have to provide their credentials before being allowed to access the resource. The application can then decide on if that user has access to the requested resource. However you've said your website doesn't need use authorisation.
You can use certificates however good luck managing the certificate for each client. My take on it is for your explanation you don't need to secure your website because you will never be able to manage a trust relationship between the client and the server. There are some options though:
You build your own client application that you ship out to people which can verify itself with the server using a packaged certificate with the client. E.g. iOS has this kind of feature if you build for that device.
You provide a capability to download a certificate that is 'installed' in the browser and used when communicating to your REST API
Use something like a handshaking protocol so when a client wants to make the first request it says; 'hi I'm a client can we chat?' And the server responds with 'yes for the next X minutes we can however make sure you send me this key everytime you tell me something YYYYYY' (you can use something like SecureUDID or equivalent for other devices than iOS).
There are probably others but you get the basic idea. Again in my opinion if your resource doesn't need authorisation then you don't need to secure that REST API. Can I ask what kind of data are you exposing via this REST API or functionality your providing? That might help provide a better answer.
You want authorization: only some agents (mobile clients) and/or users should be allowed to access those APIs.
To solve that problem, you need identification: a way for the server to tell who is who (or what), so the right decision can be made.
There are many different way to provide some form of identification, depending how much you care about security.
The simplest is a user agent string, specific to your mobile clients. But it can be faked easily. Slightly harder to fake are client based 'secrets' - embed some kind of secret or key in your mobile client code. You can make it really complicated and secret, but as ramsinb pointed out, you can't get security this way as it would require you to be able to guarantee that the secret you're shipping with the client (wether it's code, algorithm or any other fancy construct) can't be compromised or reverse engineered. Not happening when you don't control the client.
From there, 3 choices:
Security isn't really required, don't bother
Security isn't really required, but you still want to limit access to your API to either legit users/agents or people ready to invest some time hacking your protection - go with a specific user agent or a client embedded secret - don't invest much into it as it won't block people who really want access to get it anyway
Security IS required - and then I don't think there is a way around authentication, wether it's login/password, user specific (device specific?) keys, OpenID, etc... No matter what, you'll have to add to the user burden to some extent, although you can limit that burden by allowing authentication to persist (cookies, storage....)