How Email Service Providers companies [like Emarsys\MailChimp\cheetahmail etc] are able to overcome the 3rd party cookies limitation, mostly in Safar & Firefox, and report end-user behavior following an email sent?
From my investigation, most ESPs append UTM params to the link URL and the relevant info is then stored in a cookie. However, as its a 3rd party cookie, its deleted after 2 days, and all events happening after aren’t reported with the campaign attribution.
I know in parallel, the cookie should be changed to be served from server-side - but it’s not enough.
Related
I want to know any broad ideas that will point me to this:
I want to capture the data that my website's user enters after being redirected to another third party website. Want to let the user authorize this before we do it as well.
How can this be done, what tools and practices are available already? What topics should I explore to accomplish this?
I am wondering - if a cookie can capture user activity on a third party website? Can I use use selenium?
As all websites work - you are sending a request, and you get back a response, and optionally a session token, which could be stored in a cookie. You could use this token to communicate continuously with that third party site (such as using Selenium, or just a regular REST API, assuming that third party site allows one)
"Broad ideas" - Web scraping + APIs, or security attacks
The situation
There's a REST API which is currently consumed by other backend systems.
Now, that same REST API is going to be used by a single page application soon, and that SPA needs some additional security measures (CSRF token verification among others), but those additional security measures should be enforced only against end users running ordinary browsers, and not against other backends, so that those existing other backends keep working without any changes.
The Question
How do you distinguish between when a browser is consuming a REST API and when another backend is consuming it?
Is there a header that will be sent by any modern browsers and can't be turned off or can't be tampered with?
Maybe the User-Agent? Or do REST libs (in any language) send that too?
Or the Referer? Or Origin? Or some other headers?
Or something else other than a header?
Is there a header that will be sent by any modern browsers and can't be turned off or can't be tampered with?
As far as I know, you aren't going to find what you are looking for.
User-Agent is close
The "User-Agent" header field contains information about the user agent originating the request, which is often used by servers to help identify the scope of reported interoperability problems, to work around or tailor responses to avoid particular user agent limitations, and for analytics regarding browser or operating system use. A user agent SHOULD send a User-Agent field in each request unless specifically configured not to do so.
But it certainly isn't "tamper-proof"; it's just a text header, many user agents will allow you to customize it, etc.
Am I correct in thinking that a public REST api, such as a registration endpoint can't authenticate a user. For example our endpoints should only accept requests coming from our mobile applications and future web apps.
I am pretty sure it's not logically possible. I think apple and google offer a way (acting like a ca) for the server to identify a request is coming from a registered device however that changes the API to a private one. As authentication is essentially moved to the app store so only owners of app store accounts can use it.
So keeping it completely public the only way to prevent bots attacking it is to use email confirmation with auto deletion if not confirmed in x amount of time, and IP rate limiting.
A contractor suggested we use a unique secret key hard coded in to each app however, that to me sounds illogical and completely wrong. A secret between the app and server is compromised the second a user downloads the application on their device.
So am I correct about the secret key, are there any other ways to secure the endpoint from bots? The only other thing I can think of is using a captcha.
There are 2 separate things here.
Securing Public REST APIs
You are correct that a secret, even an asymmetric one, would be compromised if the attacker downloads and reverse-engineers the application. Attackers can use debuggers, Chrome developer tools, and tools like Wireshark to reverse engineer and find the secret.
In general, an attacker can mimic the behavior of the app or user, i.e. same the same requests in very similar patterns.
The approach to this in today's industry is to filter out as many of those attackers as possible using tools like Web Application Firewalls, and increasingly, Bot Management solutions, which work for web, mobile, and APIs.
The better the tool, the more work the attacker has to do, ergo fewer are attackers willing to do it, ergo smaller chance of being attacked.
Secure Authentication
Authentication is a specific case of the above, however this problem is currently (one of) the biggest problem faced by web applications.
The main solutions employed today is are bot filtering techniques e.g. CAPTCHA (which resourceful attackers can overcome), and Multi-Factor Authentication.
It's quite uncommon these days to be able to register without email, however services such as 10 Minute Mail can be used to overcome this. It's actually not very difficult to set up your own mail server as well.
Login is more difficult because you can't expect people to go through email, and even other MFA methods get quite annoying. You'd be OK with MFA for your bank account, but not with MFA to your Domino's Pizza account.
Because of that, the latest fad in web attacks is Account Takeover.
All those security vendors which offer WAF and bot management also try to protect APIs such as registration & login, with various degrees of success. There are even dedicated products are starting to emerge.
I am writing a REST Api gateway for an Angular SPA and I am confronted with the problem of securing the data exposed by the API for the SPA against "data thiefs". I am aware that I can't do much against HTML scraping, but at least I don't want to offer such data thiefs the user experience and full power of our JSON sent to the SPA.
The difference between most "tutorials" and threads about this topic is that I am exposing this data to a public website (which means no user authentication required) which offers valuable statistics about a video game.
My initial idea on how to protect the Rest API for SPA:
Using JWTs everywhere. When a visitor opens the website the very first time the SPA requests a JWT from my REST Api and saves it in the HTTPS cookies. For all requests the SPA has to use the JWT to get a response.
Problems with that approach
The data thief could simply request the oauth token from our endpoint as well. I have no chance to verify that the token has actually been requested from my SPA or from the data thief?
Even if I solved that the attacker could read the saved JWT from the HTTPS cookies and use it in his own application. Sure I could add time expiration for the JWT
My question:
I am under the impression that this is a common problem and therefore I am wondering if there are any good solutions to protect against others than the SPA having direct access to my REST Api responses?
From the API's point of view, your SPA is in no way different than any other client. You obviously can't include a secret in the SPA as it is sent to anybody and cannot be protected. Also the requests it makes to the API can be easily sniffed and copied by another client.
So in short, as diacussed many times here, you can't authenticate the client application. Anybody can create a different client if they want.
One thing you can actually do is checking the referer/origin of requests. If a client is running in a browser, thr requests it can make are somewhat limited, and one such limitation is the referer and origin headers, which are always controlled by the browser, and not javascript. So you can actually make sure that if (and only if!) the client is running in an unmodified browser, it is downloaded from your domain. This is the default in browsers btw, so if you are not sending CORS headers, you already did this (browsers do, actually). However, this does not keep an attacker from building and running a non-browser client and fake any referer or origin he likes, or just disregard the same origin policy.
Another thing you could do is changing the API regularly just enough to stop rogue clients from working (and changing your client at the same time ofc). Obviously this is not secure at all, but can be annoying enough for an attacker. If downloading all your data once is a concern, this again doesn't help at all.
Some real things you should consider though are:
Does anybody actually want to download your data? How much is it worth? Most of the times nobody wants to create a different client, and nobody is that much interested in the data.
If it is that interesting, you should implement user authentication at the very least, and cover the remaining risk either via points below and/or in your contracts legally.
You could implement throttling to not allow bulk downloading. For example if the typical user accesses 1 record every 5 seconds, and 10 altogether, you can build rules based on the client IP for example to reasonably limit user access. Note though that rate limiting must be based on a parameter the client can't modify arbitrarily, and without authentication, that's pretty much the client IP only, and you will face issues with users behind a NAT (ie. corporate networks for example).
Similarly, you can implement monitoring to discover if somebody is downloading more data than it would be normal or necessary. However, without user authentication, your only option will be to ban the client IP. So again it comes down to knowing who the user is, ie. authentication.
I have seen many different posts regarding different solutions for authenticating a RESTful API and I have some questions, given this current scenario.
I've built a REST API that will allow clients of my software service (we are a B2B company) to access resources programmatically. Now that I've got the API working properly, I'd like to secure it in the most standardized way possible. I need to allow access to certain resources based on the caller of the API. That is to say, not all users of the API can access all resources.
I have URLS available in the following formats:
https://mydomain/api/students
https://mydomain/api/students/s123
https://mydomain/api/students/s123/classes
https://mydomain/api/students/s123/classes/c456
So far I've come up with these possible solutions:
Provide a unique key to each client that they can use to ultimately generate an encrypted token that will be passed as a GET parameter at the end of each REST call to (re)-authenticate every single request. Is this approach too expensive
https://mydomain.com/api/students/s123?token=abc123
Provide a value in the HTTP Authorization Header as seen here. Is this almost the same as #1? (Except I can't paste a URL into the browser) Do people use these headers anymore?
Use OAuth 2 (which I'm still a bit unclear on). Does OAuth 2 actually authenticate the client as a logged in user? And doesn't that go against the spirit of a REST API being stateless? I was hoping OAuth was the proper solution for me (since it's a public standard), but after reading up on it a little bit, I'm not so sure. Is it overkill and/or improper for REST API calls?
My goal is to provide an API that will not have to be changed for each client that wants to consume the API, but rather that I can provide a standard documentation made available to all of our clients.
I'll be happy to post additional details if I've been unclear.
There are 2 type of clients you probably want to prepare your API:
trusted clients - Which are written by you. They can have the username and password of the actual user, and they can send that data to your server with every request, possibly in a HTTP auth header. All you need is an encrypted connection by them.
3rd party clients - Which are written by some random developer. You can register them in your service and add a unique API key to each of them. After that if an user wants to use their services, you have to show her a prompt in which she can allow access to the 3rd party client. After that the 3rd party client will be assigned to the user's account with the given permissions and it will get an user specific access token. So when the client sends its API key and the user specific token along with the request, then it sends the requests in the name of the user.
OAuth can help you to control the second situation.
Your URLs do not have a meaning to the clients. By REST you have to decouple the clients from the URL structure by sending links annotated with semantics (e.g. link relations). So your documentation does not have to contain anything about the URL structure (maybe it can be useful for server side debug, but nothing more). You have to talk about different types of links. By generating these links on server side, you can check the permissions of the actual user (or 3rd party client) and skip the links which she does not have permission to follow.