I yet read some posts on the argument, but I'd like to know if there are some "new" best practice to follow to avoid email clients (thunderbird, Outlook, gmail, ect) block remote images in a html email.
Of corse images in the email have alt description; but there is a way to be considered a secure host to which download images?
Thanks
The biggest thing that affects whether your image will load or not is user interaction. If the user has added you to their address book, responded to your email, sent replies back to you or clicked on links, the email client will add you to the white list and ensure that your emails will be delivered, rendered and isn't spam.
The best thing you can do is send engaging content and give the users a reason to interact with your email.
There are also services out there, like Return Path's Email Certification that will cost you quite a bit of money but ensure much better deliverability to their partner email providers.
Related
My company is developing a cloud contact management service and on our iOS app we're having some problems launching a particular enterprise email client app when the user presses the "Email" button on one of their contacts.
One member of our team came up with an idea to get around the problems with this enterprise app:
We let the user specify their email address in the app's settings and create our own email composing screen. Tapping the email button on a contact would open the composing screen, they would write their message and then we would send it on their behalf from our servers (or via service like mailchimp).
Basically, this would mean we would have to create an endpoint on our api that would accept a POST request with 'from', 'to', 'subject', and 'body' fields which would send the appropriate email.
This seems like a very bad idea as it's essentially creating a free, anonymous email service that could easily be abused send spam.
A few extra notes about our setup:
We don't verify an accounts email when they sign up
Even if we did verify the account's email, the user would need to be able to specify any email, as they may have signed up with personal email, but want to email someone from their work email.
Our API doesn't currently have any kind of rate limiting
Instead of having a from field in the request, we could instead send the id of the contact they want to email. This doesn't really change anything because if someone wants to abuse the send email endpoint they can also abuse the create contact endpoint.
So exactly how much of a bad idea is this, and how can I convince my team not to do this?
A few thoughts against doing it:
This is the perfect spamming service, which could damage the reputation of your company (reputational risk).
Your email servers would very quickly make it into blacklists (RBLs), making your outgoing emails land in spam folders in very many recipients' mailboxes.
Even if your servers are not yet in RBLs, if you send a forged email like that and proper email security is set up at the recipient end, your emails will still have a good chance to get classified as spam. Have a look at things like SPF and DKIM.
This could even have legal implications. Imagine the scenario when one of your users uses this service for something like blackmail. Would you be able to prove it was not you? Probably yes with the right controls, but would you want the hassle?
Still on the legal side, many countries (the EU, mainly) have data protection regulations which strictly control how personal data like email addresses can be used, especially for commercial advertisement. You probably want to adhere to that, but that would be hard with such a service (note that I'm not a lawyer, in such a case it's probably the abuser of your service that would offend these regulations and not you, I don't know, but it's something to consider).
If anyone can just send emails, it will be fairly easy to perform a denial of service attack against your services.
A few controls you could implement to mitigate some risks:
When adding a sender (from) address, you should validate that by for example sending a (cryptographically random) token and checking if the user can send it back (eg. by clicking a link in the email). If he can, that proves to some extent that he controls the email address and is probably a valid sender.
Limit the possible recipient addresses if you can. The best would be if recipients had to opt in to receive emails. If this is not possible, at least let recipients opt out from further emails. For this, you would have to add something like a footer to emails with "never again" links, and implement a facility to maintain recipients to which you must not send anymore.
Implement rate limiting. Depending on your exact scenario and use case, only allow to send the least number of emails acceptable for your application.
Implement proper logging so that you have an audit log of who exactly sent what email to whom. For this, log metadata like IP addresses as well. For this, you will likely have to authenticate your users.
On an operational level, have monitoring in place, and be prepared to ban offending users, based on a clear ToS shared with your users.
I have a software that sends notifications, quotes and invoices to "clients of my clients" by email. Sometimes people don't answer it very fast, so someone needs to call by phone to confirm if they received and get the feedback. I would like to automate this, to know if them, at least, read the email. I know this is very difficult due to how email works, but some companies already try to do this in a satisfactory way, like:
mailgun.com
mailchimp.com
sendwithus.com (YCombinator funded).
In HTML mail messages we can create a resource that points to the server, like a image. But mail clients usually ask permission to the user to load the images. So, problem
here.
But for text mail messages? Is there any way to know the email was read? How companies these companies do?
PS: I don't know what tags is the best to classify my answer, I shall appreciate any edit.
There is no way to be 100% sure if a email was opened, because of its architecture. There are some techniques to do this, but it always depends of user actions and mail client configurations. But:
For HTML messages you can use images and/or the return receipts (RFC 3798).
For text based messages you can use only the return receipts (RFC 3798).
About opening tracking:
Opens are tracked by including a transparent .png file, which will
only work if there is an HTML component to the email (i.e., text only
emails will not track opens). You should note that many email service
providers disable images by default, so this data will only show up if
the recipient clicks on display images button in his/her email.
(Text extracted from mailgun.com user docs)
References:
MailGun.com documentation.
Previous discutions on this thread.
As arnt says, you're fighting the design and basic operation of e-mail. Whenever you send a mail, there is a boundary between a MTA you control (or at least have an account on) and a MTA that is responsible for your target user's mail. What you can know is whether the user's MTA accepted the mail for delivery. Whatever happens afterwards is outside of your control.
Consider an example of a snail mail. When the package enters the recipient's box, you won't know whether they put the whole unopened envelope to a trash bin, or whether they opened is and read the contents very carefully. You can approximate that goal by using crude measures (like embedding a webcam-and-a-computer which will activate upon envelope opening and send you the snapshot of the face of the opener via a cell phone), but doing so is unreliable, unethical, and probably illegal in plenty of countries.
The "return receipts" or embedded image links are similar -- because the whole e-mail is already in the hands of the user's SW, they can do anything with it. A good MUA will probably ask before sending out dumb return receipts, and it also won't load remote images in HTML mail (because it's easy to create an http://trackme.example.org/mail/for/user/12345/message/666/image.png and have a database which says "hey, this URL belongs to Mr. Pichler, and is used in the first message we sent him). The most you can do is to ask nicely, and return receipts (RFC 3798) are a machine-readable way of doing just that.
I'm making a quick Google App Engine program that presents a publicly available form that users can fill out with their name and email address, then can enter a friend's name and e-mail address. The application pulls in the data via POST, then sends a pre-formatted e-mail like 'Hi, , your friend wants to invite you...'
What should I be doing to prevent spammers from exploiting this publicly facing e-mail sending program? Is there a good resource for best-practices in this field? I've spent a few hours searching, but I haven't really found anything definitive...
Principally creating a publicly available form that anyone can use to send[s] a pre-formatted e-mail is another name for creating a spam machine.
You can mitigate by making it harder for non-humans to use it, recaptcha is the typical way to achieve this.
You could send a confirmation email to the sender and require a secondary action (like clicking a link) before sending the email. Or, if you expect your users to return, ask them to sign up (with a similar confirmation) before allowing them to send email.
I would first impose some limit to the # of email addresses a specific user/IP can send. This won't solve the problem but will limit the damage in case someone does try to send spam to 1000 emails.
Second, you could try sending the emails in small batches if an user puts in a lot of email addresses. Send 5 at a time, and monitor to see if there's any spam complaints (you can probably automate this somehow). If no complaints after 2 days, keep sending the rest.
Let say I have a website that allows users to send articles on that website to a friend.
The way it works is that when the "send to a friend" link is clicked a form appears and it allows users to fill in the details and an email is sent to their friend.
The user can put in a "from" email address and a "to" email address into this form and a small amount of content.
When the email is received the from email address appears in the FROM and REPLY TO.
This website also sends a great deal of its own email communications to its users.
My question is:
Is there risk to allowing users (bots, attacks etc) to use this application to send emails from my SMTP, and how great is the risk?
My assumption is yes, this is not ideal.
Is it possibly worse than "not ideal"?
I do not know about bots using your form. Should it be a problem? I don't know.. I do know they program bots to be pretty clever, using your custom forms and all.
I do know that some email servers check if the FROM email address has the same IP address as the IP the mail was sent from. So imagine I put in my hotmail email address, and the mail server sees your server, it might flag the email as spam.
In the past I've an e-card websystem. It was a small joint venture with a girl I knew. She created the (cute) cards and I build her an e-card system. The website was pretty simple. Select card, enter email address, placing senders email address in the FROM and sent the email that you would have received an e-card.
Life was good...
Until I found that my entire web server IP was blacklisted at three major spam filtering mechanisms. And that 15% of all email recipients who used to receive e-cards from my site, would not receive their e-cards, because all my emails were blacklisted as spam from the get go. We have receive many many emails from angry "customers" demanding that their e-cards did not arrive. (I still find it funny how some people demanded the service, especially since it was a free service, go figure). My automatic reminder function was telling them the e-card still were not viewed, and they perhaps mistyped the email address, so that might have ticked them off :P
It was pretty annoying for my other customers as well, since they relied on sending out played newsletters and such and calling me that over 20% of the customers did not receive the newsletters.
Sending e-mails is hard. You should also check out Jeff's blog about this. So, learn from my mistake, and please put an email address associated with your email server in the FROM. This will spare you a lot of headaches ;)
yes this is definitely not ideal if this is a public website that any bot can access. but there are easy ways for you to limit spam use.
have your code limit any email
address to send ~50 emails a day and
only ~10 an hour based on your
needs. a bot would probably try to
send a million at once so limit them
on an hourly and daily basis.
store every email communication in a
database and come up with a good
program to monitor the most active
email senders. if you can verify
that an email is trusted, then let
them send as many emails as they
want
think about this site itself, it has very defined actions and reputation guidelines that limit you until you have proved you are trusted.
It may depend on whether you do any authentication to determine who's allowed to send emails. If the user has to be logged in to send articles, then you're probably fine. Bots will fail because they'll never be logged in.
The risk will increase the greater traffic you get to your site, and yes it's probably less than ideal. Unprotected, a bot will inevitably find your unprotected form, and start sending emails from your server.
There are some pretty easy solutions though, the most common probably being to implement something like Captcha
Fairly safe. I assume you do check the "From" address, if only by sending it one (standard!) mail first and asking the owner of that email address to confirm they are really humans ? This prevents most bots from finding and abusing your form. Of course, a directed attack with a human responding to your verification email will still allow spamming. But you've got a much better trail if you have received at least one reply from the alleged "From" address.
However, I don't think this will work reliably. The introduction of techniques like SPF will mean that mails from "example.com" will only be accepted if they originate from an outgoing SMTP server in the *.example.com domain. If you're faking emails with From: addresses #example.com, the receiving SMTP server will see that you are in fact not part of *.example.com and reject the email - and probably blacklist your IP range for good measure.
I found How do you make sure email you send programmatically is not automatically marked as spam? to (hopefully) be a solid guide to avoiding being marked as spam. Are there any other important tips/suggestions?
How do I track bounces,opens,clicks?
These are features found in paid services like Mail Chimp and Campaign Monitor.
Do the same as Mail Chimp and Campaign Monitor then. LIE about your stats.
There is no accurate way to track emails. If there was it'd just get blocked again. Most people don't want you to know these things and most email software ensures you don't. The stats provided by email tracking services are bogus.
Consider:
Most spam services will detect image
'bugs' and flag you as spam.
Image bugs don't do anything until
the user clicks 'show images'. This
does not mean they didn't open or
read it without images. How can you tell if a mail service downloaded the image preemptively to cache it or check it for image spam?
It can be difficult to determine the difference between a bounce and a reply due to differences in mail servers.
Only clicks can be tracked by redirecting through your server. Even then who can say that mail services won't start processing links in emails to determine whether the email is spam?
Opens can be tracked using a 1x1 picture file in an email. However, this is the same tactic that spammers use to validate email address existence, so you'll be fighting on the same side in that regard, unfortunately.
Clicks can be tracked by assigning a unique identifier to each link, determined by two variables: the URL that was clicked and the email address that clicked it. You can, for example, determine these on-send and store them in a database with the same unique identifier.
Bounces should bounce back to you with the email address intact.
I was looking at the email facebook sends out. In addition to an image, they use a bgsound element as a tracking bug like this:
<bgsound src="http://www.facebook.com/email_open_log_pic.php?mid=99999999&s=a"
volume="-10000" />
I'm guessing the bgsound src is fetched by some readers when the images are off.
Check out Ask MailChimp: How do you track email opens?
if you really want to track bounces, use a service like Email Delivered (www.emaildelivered.com)
i also use Return Path (www.returnpath.com) for a really good reading on whats being delivered to the inbox vs spam box and what esp's are totally rejecting my mail.
Two ideas, clicking links, and statistical fudgery.
Clickthroughs
I would like to add that you can mark emails as read by a user clicking a "view this email online" or by tracking click-throughs. If a user clicks on any <a> tag in your email, send it to a script first that logs the email as read and marks which link they clicked on. This will give you can get a more accurate number.
Stats
I wonder if there is any research into how many users don't show images. That way you could 'statistically' correct for the lower open counts. Just did a bit of reading and found:
A 2009 report from Merkle states that only 48% of email recipients see
images automatically. This means that if an email campaign relies
heavily on images, it’s probably not being read by over half of its
intended recipients. Source
The same site says:
In the latest MarketingSherpa Email Marketing Benchmark Report (2010), a survey of email recipients found that only 33% have images turned on by default.
Somewhere in between there could be a useful figure (35-40%) of users not displaying images in emails. That doesn't necessarily say that those users are opening the emails. Just that auto-displaying images isn't enabled.
If anyone can come up with some more facts/stats, we could potentially get a correction factor. Just with this information I don't think you can do much other than marketing smoke-and-mirrors. For example, 30% opened the emails. Based on 35% of users not displaying images, that means ~9% of users didn't display images, but explicitly chose to turn them on for this email (not really, but just go with it). Let's say that leaves 26% to unaccounted for. You could "correct" your 30% to 56%! All with the magic of bogus stats and a touch of marketing.