Most services offered online today that claim to "track" e-mails, do so by embedding images in the emails. My questions are:
Is this the only way to do it and if not, what are the other methods?
Are any of the methods actually fool-proof?
Has anybody had any luck with specific software or even an online group?
Yes, this is pretty much the only way to do it. Consider that an email is something that is inherently static. The only way to know if someone has "opened" an email is for the email to send some information back to your server. Most email clients these days support HTML emails, which means that you can get the client to request an image (or anything else) from your server by embedding the proper HTML tags. Other than this, you cannot force an email client to do anything it doesn't want to do. It's a separate program on a remote computer, and you have no control over it.
No, there's no foolproof way. There will always be emails you can't track. If someone downloads their email and disconnects from the internet before reading it, you can't track that email. Most email clients allow you to disable image loading now as well if you want to, so that can block tracking too.
I've usually written my own, so I wouldn't know what to recommend. I imagine most services will be quite similar, so I'd base a product/purchase decision on how easy their front-end is to use.
In addition to pixel tracking, a second way to track open rates is by looking for clickthroughs. If someone clicked through, then they must have opened it. This is infrequent, but it's important not to throw this data away.
More details:
How MailChimp tracks open rates
How CampaignMonitor tracks open rates
Wikipedia on email open rates
Hubspot on open rate issues
Facebook uses a bgsound element in addition to an img element like this:
<img src="http://www.facebook.com/email_open_log_pic.php?mid=999999999999"
style="border:0;width:1px;height:1px;" />
<bgsound src="http://www.facebook.com/email_open_log_pic.php?mid=99999999999&s=a"
volume="-10000" />
This is the best way, and it's hardly ideal - many e-mail clients block images to start with.
No, no methods are foolproof. A foolproof method of detecting if someone had read an e-mail would be a significant privacy issue.
I've used ExactTarget and CampaignMonitor's tracking systems. Both worked pretty well for tracking trends - i.e. twice as many people opened e-mail #1 than #2 - but you never know how many missed opens there are due to images not being shown.
Pixel tracking is the only way to track open rates. Then the links in your emails are also tracked through a redirect service for click rates. Absolutely nothing is going to be foolproof. You will have to use some guess work to figure out your actual open rate since some email clients will only take the text version and not the html and also some clients do not load images by default.
SilverPop is a popular one. They actually use PowerMTA on the back-end. Our company just ended up licensing PowerMTA and writing our own front-end and tracking.
No it's not the only way. Your HTML e-mail can refer to a web server for 'some content' which is then tracked. That could be an image, a stylesheet, some Javascript, etc. Most mail clients hate it and nothing automated is guaranteed to work.
Gain the trust of your recipient and invite them to your website. Track clicks.
Related
This is a recurring question on the website, but after spending 20 minutes browsing through old questions I was unable to find a modern day solution.
I've previously employed this JS-based method to protect addresses. Before the JS-method I was using image and flash-based solutions. Below is my old way.
Animated example codepen: http://codepen.io/anon/pen/kIjKe/
HTML:
<span class="reverse eml">moc.niamod#tset</span><br>
CSS:
.reverse {
unicode-bidi: bidi-override;
direction: rtl;
}
.eml {
display: inline;
}
JS:
function reverseEmails() {
if (jQuery(".eml.reverse").length > 0) {
jQuery(".eml.reverse").each(function() {
var that = jQuery(this);
var email = that.text().split("").reverse().join("");
that.removeClass("reverse");
that.html("<a href='mailto:" + email + "'>" + email + "</a>");
});
}
}
None of these methods seem to work nowadays, since Node.js based scrapers are able to generate an image of the page they are scraping, then reading any human-readable data from said image - you can guess the rest.
Is there any method that works nowadays, in which users are still able to easily read / click / copy paste e-mail adresses, but JS-enabled bots could not?
This is personally my favorite method, which I have found to work so far, it's not bullet proof, in theory a bot that can parse CSS3 and will preform a text search can still find it or a spambot that triggered events in order to harvest email addresses would have to feed the page into basically a headless browser, somehow determine what might be JS-obfuscated email content these scenarios are enormous amount of work for possibly no benefit and no spammer would ever consider doing it, the fact is I have had no spam to date and it works great for humans, both to read or click on:
<style>
.email:after{ content:'#mydomain.com'; }
</style>
Contact me at:<div class="email">myemail</div>
<script>
$('.email').click(function(){
window.location.href='mailto:'+$(this).html()+'#mydomain.com';
});
</script>
The thing is that the email is not a link so bots never trigger the click event as they don't even know it will do anything.
Put the email address on a separate page which is only reachable by solving a CAPTCHA.
Granted, then the security is only as good as the security of the CAPTCHA.
Using your own obfuscations may be a serious alternative if you only have a limited number of addresses you want to protect. Some ideas I have used in the past;
Crossword puzzle. Make it really easy, with cues like famous song titles with one word missing (easy to google and no debate about possible second interpretations). You can fill in many letters to make it even easier.
Audio recording with background noise. I didn't want to use my own voice so I used a speech synthesizer, with a German accent (-: AT&T web demo IIRC) and mixed in a couple of seconds of music in the background (Frank Zappa's Peaches en regalia worked very well for me, but tastes differ).
Hand drawn image. I like to draw letter outlines but I doubt they are regular enough to pass any OCR.
The real beef here is not the stellar brilliance of these solutions, but the different approaches which I hope can inspire you to think in new directions. In the end, you will always be safer if you come up with your own unique solution; anything resembling a "new de facto standard" will be the lowest-hanging fruit that the scrapers will spend time trying to pluck.
Incidentally, I tried to think about usability for people with disabilities, so I actually deployed the audio version as a fallback for people who have issues with interacting with the other two, which are based on visual layout.
By the by, very few people want to send me email these days anyway (or maybe they do, but end up being rejected as spam?) which is frankly a relief. Those who do typically use the whois registration info for my domain name (which uses an anonymized address provided by the whois registrar) or are good guessers.
I suspect your intuition is correct, if an email address is displayed on a page then a bot can scrape it.
Your best bet is to involve the server-side in some way.
For example, if you just want visitors to be able to reach you, then you can add a "contact us" form like this one: https://store.theonion.com/t-contact.aspx
If you want visitors to be able to reach eachother, then you might need to build an anonymization system like Craigslist uses.
I've always assumed that it's risky to identify users in urls within emails. For example, let's say my app is something like eventBrite. I'm inviting a set of users to an upcoming event. I create unique urls for each user's email which allows them to simply click those url's in the email to accept or decline. Ie, they will not have to authenticate with the website.
If they view the email on a mobile device or a public computer through webmail, then clicking the links will fully accept/decline.
Is this approach too risky? I had assumed you should avoid this as something could see those urls and make requests on them which would trigger false accepts/declines.
It'a an opinion but I would assume the link itself can be more secure than the email actually. You can make the accept link valid only through certain period of time (it would not make much sense otherwise anyways).
Moreover, you can make it pretty much arbitrary long. So it's basically arbitrarily hard to guess.
That would leave two options to "see" the link, that I can think of. Physically seeing it by eaves dropping. But you could generate a mail in the html form, which would allow you to hide the full link behind a hyper-ref text. Like Accept / Decline.
There are several parts to this answer:
Is it secure? Absolutely not. It's security through obscurity. You're betting somebody can't guess the link which, as long as it's a finite string then they totally can and as soon as they do, they can RSVP to your event.
Follow up Does it matter? Probably not. I imagine the chances of somebody trying to spoof an RSVP to an event are pretty slim. I absolutely wouldn't protect anything critical this way but if you're just doing something like event RSVP etc (no money changing hands) I don't see anything wrong with this approach. As luk32 said, you can also make the links valid for limited amounts of time etc.
The real issue here, (unless there's something you're not telling us and this is somehow a high value target) is how likely is somebody to accidentally stumble on one of these links and RSVP to an event they aren't going to? You can make the chances of that exceedingly unlikely by generating the links in a sufficiently random manner so that no two links are a like. In this case, I don't think security is the big concern so much as data integrity. That is, is the data you're receiving valid.
The title being self explanatory, i would like to add some points to it.
1.Firstly, is it possible to track all these (READ,DELETED,SOFT BOUNCE,HARD BOUNCE) without using third party API?
2. If no, which third party services provide you the same ?
I am aware about the <img src="send-identifiers-to-this-url-to-track"/>, but this can only get me "No.of Reads/Forwards", but not deleted.
Anybody in for help ?
There are a number of things you can detect with your own software and no third party, though I don't know how they map onto the categories you list:
Tracking images and links, like you said (READ).
No mail server to take the message, e.g There might not be any MX record for the domain or your mailserver IP might be blocked by DNSBL (maybe what you mean by HARD BOUNCE?)
Mail server response codes. This might tell you for example:
Your email was rejected because the mailbox was over capacity etc (probably SOFT BOUNCE)
Rejected because it's spam (probably HARD BOUNCE)
Greylisted or something (probably SOFT BOUNCE)
There is basically nothing you can to detect the difference between unread and deleted messages though. That is also true of third party services. You will not even detect read messages unless your recipient turns on tracking images or clicks on a tracking link.
It's pretty hard to get this information, for instance I had to implement a postfix log analyzer that is mining the bounce information (soft/hard) and the reason why they occured and make it available via API in a structured form.
is there are program or script or anything that will make my email client/webmail to open webpage links from newly received emails automatically?
If someone knows or can make a add-on for thunderbird or make this to work I don't mind paying. Just I need links that are in emails I receive to be automatically opened in default internet browser tabs.
All I can say is I very very much doubt it. If there ever was one created it would most probably be removed by most download source providers as there is a fairly huge security risk there.
Additionally, aside from the security factor you have a simple load factor to account for. I regularly make newsletters for clients, each with a link to a text version, a online version, a link for the logo and so forth... So if you did have such a plugin you would also open these links, which seams crazy. Additionally, remember the Unsubscribe link for emails, on some you are asked to confirm on others you click(/open), your taken off. Which would be very undesired.
I do understand you probably have a reason X for doing this, like because its a in house email system which creates reports and it does this and does that etc. I think most people here do understand there are sometimes very odd usage cases for things, but I don't think anyone will be able to assist you here, sorry!
I'm doing a small newsletter software for my business, and I'm wondering what metrics should I collect. Obviously, bounces and clicks should be tracked, but I'm wondering should I track email opens (via an image or a bgsound element)?
Do popular webmail services and ISPs check for tracking images and possibly increase my spam score? I guess if it increases the chances of ending up in a spam filter, I'd rather not collect that metric.
Thanks.
It's generally bad form to try to track your users in that way. Email opens are a private thing.
If you have an image with a unique URL per message, yes you can track it, but IMO, you really shouldn't. Including unnecessary images in an email is bad for a number of reasons:
Images can increase your spam score. There's a time and place for images. They can improve a message, but used inappropriately, they can look spammy.
It is obvious what you are doing. Sooner or later, one of your customers is likely to get wise to it. Some people won't care; others will feel violated.
It's REALLY unreliable. Most email clients and webmails feature an option to block images by default. You will get massively understated results.
Also remember, some people open an email immediately before they click the "delete" button. You are much better off tracking clicks.
There may be some merit in tracking the images you want to include anyway, but I'd not treat it as anything more than a very basic indicator.
As always it depends on the individual ISP and Webmail services. However, I can share some anecdotal evidence: I periodically use mailchimp to send out mass email notifications, and email opens are tracked in mailchimp using the same approach you mentioned (See following link for reference: http://kb.mailchimp.com/reports/about-open-tracking). I never experienced any issues with ending up in the spam folder, I have only had challenges with bounce backs and ending up in the Gmail promotions tab.
So based on the fact that some companies are already doing this, I have to say it probably won't increase your spam score.