From the official tutorials on https://community.algolia.com/instantsearch.js/, Algolia requires you to code the key into your application and used with each api request. If someone were to dig this up, what's to prevent them from spamming search requests with your api key?
If someone were to dig this up, what's to prevent them from spamming search requests with your api key?
Do you want the cruel truth? Nothing...
Unfortunately in a web app does not exist anyway of securing secrets, be they api-keys, tokens or any other name convention that may be used.
In a web app all is needed is to use F12 or view the page source and search for them in the raw html and JavaScript.
You can try to use JavaScript obfuscation to make it hard to find but will still be easy to reverse engineer. Even in a mobile app that have their code obfuscated and released as a binary is easy to extract this secrets.
If you want to understand a little more about Mobile Api Security Techinques please read this series of articles to find how api-keys, access tokens, HMAC and other techniques can be used and bypassed. While the article was wrote in the context of a mobile API is still valid in a web app context for the security techniques used to protect the API.
Possible Solution
The best approach is to always delegate your web app access to third part API's to a backend you can control.
In this backend you can then use a User Behaviour Analytics(UBA) solution to monitor bad use of this third part access.
Once UBA can be complex and expensive to deploy you could start by using the new Google reCaptcha V3 across all pages of your web app. ReCaptcha V3 does not require direct user interaction once it works on the background to differentiate humans from bots.
So I would have the web app requesting the Agolia search to my backend that would use reCaptcha V3 protection to differentiate abuse in the search functionality by bots or attackers.
Remember that this approach has the huge benefit of never reveal your Agolia API Key, thus attackers can never directly use it.
Related
I am building a mobile app and I was wondering if my API key can be seen when I make a GET request to my database through the API?
What I am currently doing is making an HTTP GET request where one of the params is the API Key. So if any user were to see this URL, they also can fetch this data.
Can any user see this URL being sent? I am doing this through the Flutter HTTP package.
Sensitive Data in URLs
What I am currently doing is making an HTTP GET request where one of the params is the API Key. So if any user were to see this URL, they also can fetch this data.
Using sensitive data, like API Keys, as an URL query parameter was made popular by many popular internet services since earlier days, thus a lot of tutorials and docs use this approach and this is a huge disservice for the security of any application, but a huge gain in terms of developer convenience.
Keeping sensitive data in an URL query parameter is a security disaster waiting to happen at any moment, but it's the type of disaster that happens and you don't notice until is too late, like when you discover you have been data-breached because an attacker was able to get the API Key from the logs server that was left open to the public or because you use a CI/CD pipeline that logs them by design (yes a famous one does that).
The correct place for an API key is as an header in the request, not as a URL parameter or a post parameter, but can still end-up in your logs server, but is less likely.
Extracting an API Key with a MitM Attack
I am building a mobile app and I was wondering if my API key can be seen when I make a GET request to my database through the API?
Yes, it can be seen and extracted from your mobile app binary or by intercepting the API requests made through HTTPS, also known as a Man in the Middle (MitM) attack.
Can any user see this URL being sent? I am doing this through the Flutter HTTP package.
To see how this can be easily achieved with MitM attack I invite you to read the article Steal that Api Key with a Man in the Middle Attack:
In order to help to demonstrate how to steal an API key, I have built and released in Github the Currency Converter Demo app for Android, which uses the same JNI/NDK technique we used in the earlier Android Hide Secrets app to hide the API key.
So, in this article you will learn how to setup and run a MitM attack to intercept https traffic in a mobile device under your control, so that you can steal the API key. Finally, you will see at a high level how MitM attacks can be mitigated.
If you want to learn more about API and Mobile security then I recommend you to read this answer I gave to the question How to secure an API REST for mobile app?, especially the sections Hardening and Shielding the Mobile App, Securing the API Server and A Possible Better Solution.
Do You Want To Go The Extra Mile?
In any response to a security question I always like to reference the excellent work from the OWASP foundation.
For APIS
OWASP API Security Top 10
The OWASP API Security Project seeks to provide value to software developers and security assessors by underscoring the potential risks in insecure APIs, and illustrating how these risks may be mitigated. In order to facilitate this goal, the OWASP API Security Project will create and maintain a Top 10 API Security Risks document, as well as a documentation portal for best practices when creating or assessing APIs.
For Mobile Apps
OWASP Mobile Security Project - Top 10 risks
The OWASP Mobile Security Project is a centralized resource intended to give developers and security teams the resources they need to build and maintain secure mobile applications. Through the project, our goal is to classify mobile security risks and provide developmental controls to reduce their impact or likelihood of exploitation.
OWASP - Mobile Security Testing Guide:
The Mobile Security Testing Guide (MSTG) is a comprehensive manual for mobile app security development, testing and reverse engineering.
Yes if you are not using ssl you are sending everything in plain text which can be intercepted.
if you are using ssl still your URL can be stored in logs on api server so instead of sending key in query parameter use POST request and append the key in custom header or form parameters.
I'm building an application where I need to make a request in the client-side frontend app to an external API, and I'm at a bit of a loss for how to make this maximally secure so that only valid requests can be forwarded to this external API and not whatever anyone wants.
As a first step in security, I've made it so that the client-side app can't speak to the external API directly, but must instead hit our own server-side API, which then proxies the request to the external API, so that the credentials for hitting the external API are at least stored solely server side and not client side.
This, however, has led to the same fundamental issue - how do I secure whatever credential/authentication system I use to authenticate requests I make from the client-side app to our own server-side app?
The issue is this is an online restaurant ordering service, and so we don't expect users to authenticate themselves with say, usernames and passwords before being able to place orders necessarily, and so order placement, which triggers the external API call, isn't gated behind any username/password scheme, and must be available to all consumers of the frontend app.
What's the best practice for security here? I've enabled CORS whitelisting as a minimum practice, such that only requests from our own domain are theoretically allowed by our server side API endpoint, but CORS is trivially bypassed if someone chooses to just spoof the origin URL.
What other options are available? I'm sure I must just be missing something trivial, since this must be an extraordinarily common issue with an established best practice, but I'm just somehow failing to find it.
Thank you!
As a Developer Advocate for API and Mobile security, seeing a developer that really cares about their app security always makes me smile, especially when they already show some effort was made to secure it, therefore accept my congratulations for your efforts.
My Answer Context
I'm building an application where I need to make a request in the client-side frontend app to an external API, and I'm at a bit of a loss for how to make this maximally secure so that only valid requests can be forwarded to this external API and not whatever anyone wants.
So, you have not detailed if it's a web app or a mobile app, and once my expertise relies on mobile and API security I will be answering with the assumption that is a mobile app.
The Challenge
The issue is this is an online restaurant ordering service, and so we don't expect users to authenticate themselves with say, usernames and passwords before being able to place orders necessarily, and so order placement, which triggers the external API call, isn't gated behind any username/password scheme, and must be available to all consumers of the frontend app.
You have here a complicated challenge to solve, because you have an app that is open to the public, no user authentication/identification of any sort, but that requires rules of access to the underline resources as if it was behind user authentication and authorization, but even if it was, it would still be vulnerable to being abused.
To understand why I need to clear a misconception that usually I find among developers of any seniority, that is about the difference between who and what is accessing an API server.
The Difference Between WHO and WHAT is Accessing the API Server
I wrote a series of articles around API and Mobile security, and in the article Why Does Your Mobile App Need An Api Key? you can read in detail the difference between who and what is accessing your API server, but I will extract here the main takes from it:
The what is the thing making the request to the API server. Is it really a genuine instance of your mobile app, or is it a bot, an automated script or an attacker manually poking around your API server with a tool like Postman?
The who is the user of the mobile app that we can authenticate, authorize and identify in several ways, like using OpenID Connect or OAUTH2 flows.
Think about the who as the user your API server will be able to Authenticate and Authorize access to the data, and think about the what as the software making that request in behalf of the user.
So, in your case you cannot identify who is in the request, thus you need a solution that is able to give a very high degree of confidence to the API backend that the request is indeed from what it expects, a genuine and unmodified instance of your app.
Possible Solutions
I'm building an application where I need to make a request in the client-side frontend app to an external API, and I'm at a bit of a loss for how to make this maximally secure so that only valid requests can be forwarded to this external API and not whatever anyone wants.
This requires very advanced solutions to properly secure, thus isn't at all trivial to achieve as you may think:
I'm sure I must just be missing something trivial, since this must be an extraordinarily common issue with an established best practice, but I'm just somehow failing to find it.
And yes, it's a common issue that often is neglected or not addressed properly, and the first step to solve it is to have a clear picture about the difference between who vs what is in a request, otherwise the devised solutions will fail to address the issue properly.
For Mobile Apps
Here I recommend you to go through this answer I gave to the question How to secure an API REST for mobile app?, especially the sections Hardening and Shielding the Mobile App, Securing the API Server and A Possible Better Solution.
This answer will show you several solutions, like WAFs and UBAs, but ends with a recommendation to use a Mobile App Attestation concept.
In a nutshell the Mobile App Attestation will allow the API backend to have a very high degree of confidence that the request is indeed from what it expects, a genuine and modified instance of the mobile app.
For Web Apps
You can learn some useful techniques to help your API backend to try to respond only to requests coming from what you expect, your genuine web app, and to do so I invite you to read my answer to the question Secure api data from calls out of the app, especially the section dedicated to Defending the API Server.
Do You Want To Go The Extra Mile?
In any response to a security question I always like to reference the excellent work from the OWASP foundation.
For APIS
OWASP API Security Top 10
The OWASP API Security Project seeks to provide value to software developers and security assessors by underscoring the potential risks in insecure APIs, and illustrating how these risks may be mitigated. In order to facilitate this goal, the OWASP API Security Project will create and maintain a Top 10 API Security Risks document, as well as a documentation portal for best practices when creating or assessing APIs.
For Mobile Apps
OWASP Mobile Security Project - Top 10 risks
The OWASP Mobile Security Project is a centralized resource intended to give developers and security teams the resources they need to build and maintain secure mobile applications. Through the project, our goal is to classify mobile security risks and provide developmental controls to reduce their impact or likelihood of exploitation.
OWASP - Mobile Security Testing Guide:
The Mobile Security Testing Guide (MSTG) is a comprehensive manual for mobile app security development, testing and reverse engineering.
For Web Apps
The Web Security Testing Guide:
The OWASP Web Security Testing Guide includes a "best practice" penetration testing framework which users can implement in their own organizations and a "low level" penetration testing guide that describes techniques for testing most common web application and web service security issues.
Ultimately your client needs to perform some operation on 3rd party API.
So we know that some operations should be allowed, and based on your description we also know that not every operation should be allowed.
So your security should be based on this premise. Don't create a dumb proxy that forwards every single request, but your intermediate API should only specifically allow the operations that you want it to allow, based on the rules you set.
If you don't have a username & password, you probably still have some other kind of rule that identifies a person (email/phone number?), which means you can create an authentication system.
Or maybe your 3rd party service should only be called after a user completed an order with a credit card, that logic needs to exist on your API.
I have actually one SPA in ReactJs + one mobile application in Flutter + one REST API developed with SailsJs running on a separate server. I managed user authentication with a secured session cookie sent back by the API when we are login with valid information (id/password).
So all the endpoints that require users to be authenticated are protected (unless there are others security best practices that I'm not aware of?). The session cookie expiration and validity are checked with each call to one of the protected endpoints.
I really read a massive amount of topics and blog posts talking about securing REST API. And my problem is never or barely represented. So now my main problem is :
How can I restrict my public API endpoints (login & register currently) that does not require users to be authenticated (since there are the endpoints used to achieve this mission...) to be used only in my trusted client apps (web and mobile)?
How can I prevent another app developed by another person to use these endpoints?
I don't want anyone to login via my API unless it is done in the client apps I am developing... I don't want anyone to replicate my applications and successfully use my API that way with 0 protection, without knowing it...
I see a lot of popular services with login API routes (Heroku for example) that can't be accessed in Postman with the same parameters (403 error code). So it is possible. But how they do that? There is nothing in specialized forums that handle this or I missed something!
I tough of a secret token stored in the client to authenticate it but it is literally public with web developer tools for example.
Need some advice.
Thanks
USER AUTHENTICATION IS NOT APP AUTHENTICATION
So all the endpoints that require users to be authenticated are protected...
This endpoints are only protected regarding to identify, authenticate and authorize Who his in the request, but not for What is doing the request, and this is a topic not very well understood among developers, be them juniors or seniors.
The Difference Between WHO and WHAT is Accessing the API Server
In an article I wrote, entitled Why Does Your Mobile App Need An Api Key? you can read with more detail the difference between Who and What is accessing your API server, from where I quote the following:
The what is the thing making the request to the API server. Is it really a genuine instance of your mobile app, or is it a bot, an automated script or an attacker manually poking around your API server with a tool like Postman?
The who is the user of the mobile app that we can authenticate, authorize and identify in several ways, like using OpenID Connect or OAUTH2 flows.
So the Who is the user of your API server that you will be able to Authenticate and Authorize access to the data, and the What is the software making that request in behalf of the user, your genuine app, a tampered one, an automated script or someone manually poking around with your API via cURL, Postman or similar tools.
By now I hope that you have enough knowledge to understand why user(who) authentication is not the same as app(what) authentication.
LOCK THE API SERVER TO THE APPS
How can I restrict my public API endpoints (login & register currently) that does not require users to be authenticated (since there are the endpoints used to achieve this mission...) to be used only in my trusted client apps (web and mobile)?
I think that by now it may be clear to you that it's not only the login and registration endpoints that need to be protected from What is doing the request.
How can I prevent another app developed by another person to use these endpoints?
I don't want anyone to login via my API unless it is done in the client apps I am developing... I don't want anyone to replicate my applications and successfully use my API that way with 0 protection, without knowing it...
This is extremely hard to achieve for web apps, but possible with an high degree of confidence for mobile apps when the Mobile App Attestation concept is implemented.
For web apps
Due to the nature of how the web was built, all it's necessary to inspect a web app is to hit F12 or inspect the page source, and then search for whatever you need to access the API server from another tool.
You can learn some useful techniques to help your API server to try to respond only to requests coming from What you expect, your genuine web app, and to do so I invite you to read my answer to the question Secure api data from calls out of the app, specially the section dedicated to Defending the API Server.
For mobile apps
To learn how you can lock your API server to your mobile app I recommend you to read my answer to
the question How to secure an API REST for mobile app? for the sections on Securing the API Server and A Possible Better Solution.
Endpoints to Secure
So all the endpoints that require users to be authenticated are protected (unless there are others security best practices that I'm not aware of?).
It's up to you if you only want to enhance the security of your login and register endpoints, but my advice is that you enhance the security of all them regarding the detection for What is accessing them.
POSTMAN WITH HEROKU AND OTHERS
I see a lot of popular services with login API routes (Heroku for example) that can't be accessed in Postman with the same parameters (403 error code). So it is possible. But how they do that? There is nothing in specialized forums that handle this or I missed something!
I never used Heroku, but when I am using an API that doesn't work in Postman, but works in other clients, let's say from cURL, then I disable Postman from sending it's own user-agent and normally the API will start accepting the requests.
If doesn't then they may be doing device fingerprinting:
A device fingerprint or machine fingerprint is information collected about the software and hardware of a remote computing device for the purpose of identification. The information is usually assimilated into a brief identifier using a fingerprinting algorithm. A browser fingerprint is information collected specifically by interaction with the web browser of the device.
The fingerprinting can be done in active or passive mode. In active mode some Javascript runs on the client to collect some data to send back to the API server, while in passive mode it uses the information available from the request in the server, like the http headers and request parameters.
While this raises the bar to fake What is doing the request, it can be bypassed by observing how a trusted client sends the request and mimic it. For an attacker it's just a little more work to enumerate all variants and then automate them.
DO YOU WANT TO GO THE EXTRA MILE?
I really read a massive amount of topics and blog posts talking about securing REST API.
First and foremost my congratulations for putting such effort in educating yourself about securing your API.
I don't know if you already read some of the OWASP resources I am about to link, but in any response to a security question I always like to reference the excellent work from the OWASP foundation ;)
For Web Apps
OWASP Web Top 10 Risks
The OWASP Top 10 is a powerful awareness document for web application security. It represents a broad consensus about the most critical security risks to web applications. Project members include a variety of security experts from around the world who have shared their expertise to produce this list.
The Web Security Testing Guide:
The OWASP Web Security Testing Guide includes a "best practice" penetration testing framework which users can implement in their own organizations and a "low level" penetration testing guide that describes techniques for testing most common web application and web service security issues.
For Mobile Apps
OWASP Mobile Security Project - Top 10 risks
The OWASP Mobile Security Project is a centralized resource intended to give developers and security teams the resources they need to build and maintain secure mobile applications. Through the project, our goal is to classify mobile security risks and provide developmental controls to reduce their impact or likelihood of exploitation.
OWASP - Mobile Security Testing Guide:
The Mobile Security Testing Guide (MSTG) is a comprehensive manual for mobile app security development, testing and reverse engineering.
For APIS
OWASP API Security Top 10
The OWASP API Security Project seeks to provide value to software developers and security assessors by underscoring the potential risks in insecure APIs, and illustrating how these risks may be mitigated. In order to facilitate this goal, the OWASP API Security Project will create and maintain a Top 10 API Security Risks document, as well as a documentation portal for best practices when creating or assessing APIs.
I'm new to OAuth and I would really appreciate if someone could give me a hand with my problem. I need to create a simple web application for track expenses, with some basic actions (user must be able to create an account and log in, list expenses, edit them, etc) with a REST API for each one, and the trick is that I need to be able to pass credentials to both the webpage and the API. So, after some research I've found some examples using Digest Authentication and HMAC Authentication but lot of posts also mentioned OAuth as an alternative approach, so my question is, given this scenario, would be proper to use OAuth? I mean, as far as I understand OAuth is suitable when you want to share resources with other application, which I'm not doing for this project; besides that, when you try to access the shared resource it appears a page requesting permission for the foreign application, would that page appear at some point in my application? (maybe after the login?)
Thanks in advance guys
In your current scenario it does not make sense to use OAuth. It's not what OAuth is designed for.
If your application ecosystem is going to have multiple webapps running on a single SSO (like google) then it is very helpful to have OAuth.
Suggestion: Decide based on your business/operation plan and implement accordingly.
Note: If you plan to have 10 apps in the span of the next 5 years but only have one app now it does not make sense to spend time to implement complex protocols like OAuth right now. Scale as you grow.
We intend to develop rest based api. I explored the topic but it seems, you can secure api when your client is an app (So there are many ways, public key - private key etc). What about websites / mobile website, if we are accessing rest based api in website which do not use any login for accessing contents ( login would be optional ) then how could we restrict other people from accessing rest based api ?
Does it make sense using Oauth2.0 ? I don't have clear idea of that.
More clear question could be ,How can we secure get or post request exposed over web for the website which doesn't use any login ?
If it's simple get request or post request , which will return you json data on specific input, now i have mobile website , who will access those data using get request or post request to fetch data. Well, some else can also access it , problem is i am not using Login, user can access data directly. But how can we restrict other people from accessing that data.
What do you think is the difference between securing a website that is not using REST vs one that is using REST API?
OAuth provides authorisation capabilities for your site, in a REST architecture this means a user of the mobile application will have to provide their credentials before being allowed to access the resource. The application can then decide on if that user has access to the requested resource. However you've said your website doesn't need use authorisation.
You can use certificates however good luck managing the certificate for each client. My take on it is for your explanation you don't need to secure your website because you will never be able to manage a trust relationship between the client and the server. There are some options though:
You build your own client application that you ship out to people which can verify itself with the server using a packaged certificate with the client. E.g. iOS has this kind of feature if you build for that device.
You provide a capability to download a certificate that is 'installed' in the browser and used when communicating to your REST API
Use something like a handshaking protocol so when a client wants to make the first request it says; 'hi I'm a client can we chat?' And the server responds with 'yes for the next X minutes we can however make sure you send me this key everytime you tell me something YYYYYY' (you can use something like SecureUDID or equivalent for other devices than iOS).
There are probably others but you get the basic idea. Again in my opinion if your resource doesn't need authorisation then you don't need to secure that REST API. Can I ask what kind of data are you exposing via this REST API or functionality your providing? That might help provide a better answer.
You want authorization: only some agents (mobile clients) and/or users should be allowed to access those APIs.
To solve that problem, you need identification: a way for the server to tell who is who (or what), so the right decision can be made.
There are many different way to provide some form of identification, depending how much you care about security.
The simplest is a user agent string, specific to your mobile clients. But it can be faked easily. Slightly harder to fake are client based 'secrets' - embed some kind of secret or key in your mobile client code. You can make it really complicated and secret, but as ramsinb pointed out, you can't get security this way as it would require you to be able to guarantee that the secret you're shipping with the client (wether it's code, algorithm or any other fancy construct) can't be compromised or reverse engineered. Not happening when you don't control the client.
From there, 3 choices:
Security isn't really required, don't bother
Security isn't really required, but you still want to limit access to your API to either legit users/agents or people ready to invest some time hacking your protection - go with a specific user agent or a client embedded secret - don't invest much into it as it won't block people who really want access to get it anyway
Security IS required - and then I don't think there is a way around authentication, wether it's login/password, user specific (device specific?) keys, OpenID, etc... No matter what, you'll have to add to the user burden to some extent, although you can limit that burden by allowing authentication to persist (cookies, storage....)