I've always assumed that it's risky to identify users in urls within emails. For example, let's say my app is something like eventBrite. I'm inviting a set of users to an upcoming event. I create unique urls for each user's email which allows them to simply click those url's in the email to accept or decline. Ie, they will not have to authenticate with the website.
If they view the email on a mobile device or a public computer through webmail, then clicking the links will fully accept/decline.
Is this approach too risky? I had assumed you should avoid this as something could see those urls and make requests on them which would trigger false accepts/declines.
It'a an opinion but I would assume the link itself can be more secure than the email actually. You can make the accept link valid only through certain period of time (it would not make much sense otherwise anyways).
Moreover, you can make it pretty much arbitrary long. So it's basically arbitrarily hard to guess.
That would leave two options to "see" the link, that I can think of. Physically seeing it by eaves dropping. But you could generate a mail in the html form, which would allow you to hide the full link behind a hyper-ref text. Like Accept / Decline.
There are several parts to this answer:
Is it secure? Absolutely not. It's security through obscurity. You're betting somebody can't guess the link which, as long as it's a finite string then they totally can and as soon as they do, they can RSVP to your event.
Follow up Does it matter? Probably not. I imagine the chances of somebody trying to spoof an RSVP to an event are pretty slim. I absolutely wouldn't protect anything critical this way but if you're just doing something like event RSVP etc (no money changing hands) I don't see anything wrong with this approach. As luk32 said, you can also make the links valid for limited amounts of time etc.
The real issue here, (unless there's something you're not telling us and this is somehow a high value target) is how likely is somebody to accidentally stumble on one of these links and RSVP to an event they aren't going to? You can make the chances of that exceedingly unlikely by generating the links in a sufficiently random manner so that no two links are a like. In this case, I don't think security is the big concern so much as data integrity. That is, is the data you're receiving valid.
Related
Over the past few months random email addresses, some of which are on known spam lists, have been added at the rate of 2 or 3 a day to my website.
I know they aren't real humans - for a start the website is in a very narrow geographical area, and many of these emails are clearly from a different country, others are info# addresses that appear to have been harvested from a website, rather than something a human would use to sign up to a site.
What I can't work out is, what are reasons for somebody doing this? I can't see any benefit to an external party beyond being vaguely destructive. (I don't want to link to the site here, it's just a textbox where you enter email and press join).
These emails are never verified - my question isn't about how to prevent this, but what are some valid reasons why somebody might do this. I think it's important to understand why malicious users do what they do.
This is probably a list bombing attack, which is definitely not valid. The only valid use I can think of is for security research, and that's a corner case.
List bomb
I suspect this is part of a list bombing attack, which is when somebody uses a tool or service to maliciously sign up a victim for as much junk email as possible. I work in anti-spam and have seen victims' perspectives on this: it's nearly all opt-in verifications, meaning the damage is only one per service. It sounds like you're in the Confirmed Opt-In (COI) camp, so congratulations, it could be worse.
We don't have good solutions for list bombing. There are too many problems to entertain a global database of hashed emails that have recently opted into lists (so list maintainers could look up an address, conclude it's being bombed, and refuse to invite). A global database of hashed emails opting out of bulk mail (like the US Do Not Call list or the now-defunct Blue Frog's Do Not Intrude registry but without the controversial DDoS-the-spammers portion) could theoretically work in this capacity, though there'd still be a lot of hurdles to clear.
At the moment, the best thing you can do is to rate-limit (which this attacker is savvy enough to avoid) and use captchas. You can measure your success based on the click rate of the links in your COI emails; if it's still low, you still have a problem.
In your particular case, asking the user to identify a region via drop-down, with no default, may give you an easy way to reject subscriptions or trigger more complex captchas.
If you're interested in a more research-driven approach, you could try to fingerprint the subscription requests and see if you can identify the tool (if it's client-run, and I believe most are) or the service (if it's cloud-run, in which case you can hopefully just blacklist a few CIDR ranges instead). Pay attention to requesters' HTTP headers, especially the referer. Browser fingerprinting it its own arms race; take a gander at the EFF's Panopticlick or Brian Kreb's piece on AntiDetect.
Security research
The only valid case I can consider, whose validity is debatable, is that of security research (which is my field). When I'm given a possible phishing link, I'm going to anonymize it. This means I'll enter fake data rather than reveal my source. I'd never intentionally go after a subscription mechanism (at least with an email I don't control), but I suppose automation could accidentally stumble into such a thing.
You can avoid that by requiring POST requests to subscribe. No (well-designed) subscription mechanism should accept GET requests or action links without parameters (though there are plenty that do). No (well-designed) web crawler, for search or archiving or security, should generate POST requests, at least without several controls to ensure it's acceptable (such as already concluding that it's a bad actor's site). I'm going to be generous and not call out any security vendors that I know do this.
Through our platform I'm sending automatic emails with an unsubscribe link.
The unsubscribe has the following format:
http://example.com/unsubscribe?id={id}&c={code}
The Id represents the UserId in the database, the Code is just some hash that is linked to the User (so people can't unsubscribe random people).
Problem is that I'm getting exceptions that the UserId does not match the database (about 5-10%) - the Id is just way larger then the largest Id in our database.
Another weird thing is that the query parameters seem to be reversed. It is suddenly ?c={code}&id={id} instead of ?id={id}&c={code}.
The code is really really simple, and I can't believe there is anything that can go wrong in constructing the original message. In a way there don't seem to be any other code paths.
I can come up with two possible causes:
Some email tools mangle the url as some kind of anti spam / phishing / tracking / footprinting?
This is somehow related to character encoding going bad or something similar?
But in a way I have no clue and I'm really at a loss here. I really would like to fix this, because the last thing I want to do is spam people by accident that can't unsubscribe.
Can anyone point me in the right direction?
I was at a developer conference where the speaker argued that the following set of URLs are not RESTful:
/users/username/changepassword
/users/username/resetpassword
The main reason given was that the same URLs might be used in different context and that this didn't facilitate HATEOAS in a meaningful way.
He then continued to argue that a more viable approach is to use the following URLs:
/account/changepassword
/administration/server/users/username/resetpassword
According to the speaker this latter approach allowed for each use-case to have a specifically tailored (html-)form for each URL, which could then be posted to the same URL. No more problems with the same URL used in different contexts.
I would spontaneously say that neither of these URL sets are RESTful, simply due to the fact that they are both centered around actions (verbs) which in my eyes do not really qualify as resources except for in exceptional cases (like search). I feel like this setup is very RPC-like.
I would have suggested something more noun-like and granular like
//Change password
PUT /users/username/account/password
//Register reset
POST /users/username/account/password/resets
//Verify reset
PUT /users/username/account/password/resets/0/verification_code
What is your opinion? Is the speakers approach RESTful or not, or is there simply not enough information here?
I agree, the whole idea of a RESTful interface (as I understand it) is to allow access to "resources". So neither of those URL schemes seem very nice to me.
Having said that REST isn't set in stone, it is more of a guide than a set of rules. Some things don't sit that well with it, so you have to get as close as you can just using the HTTP verbs.
A password reset isn't a resource, however a password is. So, I would say something along these lines for a password reset operation ...
GET /users/antonyscott/password
PUT /users/antonyscott/password
With the 2nd call requiring authentication of some sort derived from the first call and passing in the new password. Actually that's more of a straight password change than a reset. If you're after a reset (ie - following a link in an email to confirm the reset) then what you had seems okay.
Obviously designing an API is an iterative process, so I would say have a go and see how it works, then refine it.
I'm doing a small newsletter software for my business, and I'm wondering what metrics should I collect. Obviously, bounces and clicks should be tracked, but I'm wondering should I track email opens (via an image or a bgsound element)?
Do popular webmail services and ISPs check for tracking images and possibly increase my spam score? I guess if it increases the chances of ending up in a spam filter, I'd rather not collect that metric.
Thanks.
It's generally bad form to try to track your users in that way. Email opens are a private thing.
If you have an image with a unique URL per message, yes you can track it, but IMO, you really shouldn't. Including unnecessary images in an email is bad for a number of reasons:
Images can increase your spam score. There's a time and place for images. They can improve a message, but used inappropriately, they can look spammy.
It is obvious what you are doing. Sooner or later, one of your customers is likely to get wise to it. Some people won't care; others will feel violated.
It's REALLY unreliable. Most email clients and webmails feature an option to block images by default. You will get massively understated results.
Also remember, some people open an email immediately before they click the "delete" button. You are much better off tracking clicks.
There may be some merit in tracking the images you want to include anyway, but I'd not treat it as anything more than a very basic indicator.
As always it depends on the individual ISP and Webmail services. However, I can share some anecdotal evidence: I periodically use mailchimp to send out mass email notifications, and email opens are tracked in mailchimp using the same approach you mentioned (See following link for reference: http://kb.mailchimp.com/reports/about-open-tracking). I never experienced any issues with ending up in the spam folder, I have only had challenges with bounce backs and ending up in the Gmail promotions tab.
So based on the fact that some companies are already doing this, I have to say it probably won't increase your spam score.
Most services offered online today that claim to "track" e-mails, do so by embedding images in the emails. My questions are:
Is this the only way to do it and if not, what are the other methods?
Are any of the methods actually fool-proof?
Has anybody had any luck with specific software or even an online group?
Yes, this is pretty much the only way to do it. Consider that an email is something that is inherently static. The only way to know if someone has "opened" an email is for the email to send some information back to your server. Most email clients these days support HTML emails, which means that you can get the client to request an image (or anything else) from your server by embedding the proper HTML tags. Other than this, you cannot force an email client to do anything it doesn't want to do. It's a separate program on a remote computer, and you have no control over it.
No, there's no foolproof way. There will always be emails you can't track. If someone downloads their email and disconnects from the internet before reading it, you can't track that email. Most email clients allow you to disable image loading now as well if you want to, so that can block tracking too.
I've usually written my own, so I wouldn't know what to recommend. I imagine most services will be quite similar, so I'd base a product/purchase decision on how easy their front-end is to use.
In addition to pixel tracking, a second way to track open rates is by looking for clickthroughs. If someone clicked through, then they must have opened it. This is infrequent, but it's important not to throw this data away.
More details:
How MailChimp tracks open rates
How CampaignMonitor tracks open rates
Wikipedia on email open rates
Hubspot on open rate issues
Facebook uses a bgsound element in addition to an img element like this:
<img src="http://www.facebook.com/email_open_log_pic.php?mid=999999999999"
style="border:0;width:1px;height:1px;" />
<bgsound src="http://www.facebook.com/email_open_log_pic.php?mid=99999999999&s=a"
volume="-10000" />
This is the best way, and it's hardly ideal - many e-mail clients block images to start with.
No, no methods are foolproof. A foolproof method of detecting if someone had read an e-mail would be a significant privacy issue.
I've used ExactTarget and CampaignMonitor's tracking systems. Both worked pretty well for tracking trends - i.e. twice as many people opened e-mail #1 than #2 - but you never know how many missed opens there are due to images not being shown.
Pixel tracking is the only way to track open rates. Then the links in your emails are also tracked through a redirect service for click rates. Absolutely nothing is going to be foolproof. You will have to use some guess work to figure out your actual open rate since some email clients will only take the text version and not the html and also some clients do not load images by default.
SilverPop is a popular one. They actually use PowerMTA on the back-end. Our company just ended up licensing PowerMTA and writing our own front-end and tracking.
No it's not the only way. Your HTML e-mail can refer to a web server for 'some content' which is then tracked. That could be an image, a stylesheet, some Javascript, etc. Most mail clients hate it and nothing automated is guaranteed to work.
Gain the trust of your recipient and invite them to your website. Track clicks.