There are many articles and threads about guidelines while sending bulk emails. Most of the times it was mentioned that emails should be sent to the subscribed users. So that we can avoid "users clicking on spam in their mail boxes".
There are some features in famous social networking sites where we can send invitations to Yahoo contact list. In such scenarios the persons in our contact list actually did not subscribe with Linkedin to get invitations or other mails. But the hard thing to understand is how do Linkedin and Facebook mails don't go to spam?
Developers often attack this problem from "how can I do everything (DomainKeys, et al) to ensure the message will not end up in someone's spam box?".
I used to work on a website that mailed 20,000 users weekly.
The key point in keeping out of the spam box - beyond all the technical components laid out in the link above posted by Martin - is making sure that your recipients do not spam-can you.
Bayesian detection algorithms on mail servers source data from users who click "this is spam" in their mail clients, so if your users think you are SPAM, you are - and you'll get lower delivery percentages (be prepared: 100% is impossible).
Use your brand or website name as the From: field. Your customer is most likely to remember that, if nothing else. Attempts to "be close" to the customer by using a name of someone in your organization is usually a bad idea, even if you're consistent. Identify your organization UP FRONT, and in the subject line every time.
Make the unsubscribe link incredibly easy to find (not at the bottom!). Users will say "this is spam" to make the message go away if they can't figure out how to get off your list. You don't want these people on your list anyhow.
Double opt-in. Don't add anyone to your list without their explicit opt-in. Not only do you have their blessing, but the delivery of the opt-in message indicates that they're getting your mail. The ever-famous "call to action" of "please add us to your address book" is viable, but I personally never do this when a service asks me to.
Have a schedule on which you send, to whatever the user has agreed (weekly, monthly, etc). If you are not consistent, users will forget registering with you and the mail will seem random. Also, allow them to change the frequency as the next link right next to "unsubscribe" - many people unsubscribe because they are annoyed by too-frequent messages.
To avoid being considered as a SPAM, the easiest and most powerful method, is to not send a SPAM.
Check this to understand what is a SPAM and avoid sending one: http://en.wikipedia.org/wiki/Spam
Jeff Atwood has written quite a good article on this.
From now on, we are using a commercial provider. There are several on the market (like critsend, which is among the more inexpensive ones) which also do bounce handling and make sure that "permanent failures" are not sent out again in the next bulk.
Related
I am trying to avoid going into the spam folder when I send an email to users on my website.
Mainly I need them to activate their newly registered account and if it's in the spam folder, they most likely will never activate it.
I noticed that for the most part, it's Hotmail that blocks my emails.
I read a lot that the more people mark it as not-spam, and if they add the email to their contacts, that why it increases the chances of not going to spam folders in the future.
Is there a way to offer a link for "add this email to your contacts" in the html body of the email?
Also, what can I do to not get to the spam in general? I tried stripping all the html and just send plain text but still went to spam...
To prevent your emails from going to spam can entail optimizing a number of things such as
Text of the email (even if it's plain text.. spammy/salesy wording will still trigger spam
The domain in which your sending the email from
Whether or not your sending domain is authenticated (e.g. SPF, DKIM)
Checking that your not on spam lists
What people usually do is create a link to a page which provides step by step instructions on how users can whitelist the sender in various email clients and providers.
This website will actually auto-generate the instructions page for you: http://www.emaildeliveryjedi.com/email-whitelist.php
Mailchimp offers a solution which allows you to add an 'Add-to-Address-Book' link to your campaigns but it's not very broadly compatible with all clients. What they're doing is embedded hcard microdata.
Further:
Mailchimp Add-to-Address-Book Links
hCard
I'd recommend sending a test email to http://isnotspam.com/
They run a SpamAssassin test (and a few others) on the email and give you an output, which is a good metric to judge most spam filters by.
Another thing to look out for is that GMail's doesn't like when you mention money at all, especially large amounts.
if I wanted to create an "email list" along the lines of google groups (and eventually integrate with a web site), what services should I look into? I haven't really dealt with email before so I'm not 100% sure how it all works. I do know I need to be careful about being blacklisted, etc. Do I use something like sendgrid/mailman? I imagine all emails go to my server when a user hits "reply all" and on my end I push that email as though it came from the user but really it came from my "mail" server? It seems a user doesn't need to "reply all" to send to the list and there is no quick way to reply to just one user (which is a good thing for this app).
So the way I imagine this working:
1) I use something like sendgrid to "listen" for emails coming to list#mywebsite.com
2) When emails come (i.e from userx) I do some filtering for spam, etc and then push them all out to "all users/subscribers"
3) All users get something that is "from user x" (but it's really from my server, on google groups it says "via googlegroups") and the "reply to" is list#mywebsite.com
I imagine this is how it all works but I'm not 100% sure this is correct or if sendgrid will allow me to do this (and if it will cost too much).
Thanks
It really depends on what your needs and skills are. For simple mailing lists there is some very established software such as ListServ or MajorDomo
If you're not running a Unix variant, or you're not confident setting up this kind of software they might not be suitable though.
You can use SendGrid to handle your incoming email, but you will have to re-assemble the message before sending it out since it will be posted to your incoming parse URL in a, well, parsed way. The good news is that SendGrid can handle the spam check for you.
SendGrid currently doesn't charge for inbound parsing, but of course that may change. So as far as your cost goes, you would only use credits for the mail you send. Keep in mind that SendGrid only provides the Parse API to silver ($80/mo) customers or customers on the free (200 emails/day) plan, but there are plans to add a lower tier package that also has access to all APIs.
You can also use SendGrid to handle unsubscribes, but if you are using another solution for list management that's probably superfluous.
Googling for "mailing list hosting" brings up a wealth of sites, some of them free at least under some conditions. Many of them will use something like Mailman / Majordomo / Listserv; certainly you can run Mailman or Majordomo on your own server if you like, as well, so your question doesn't really make sense from that point of view. Yahoo Groups / Google Groups can also happily host a list for you. Do you need the list to be visible to subscribers only? Google Groups and Yahoo Groups will host private lists, too.
A mailing list basically works like Bcc: mail. You send to the mailing list's submission address; it then replaces some of the headers (often the Sender: and perhaps Reply-To:) and then redistributes the message to all the subscribers. Behind the scenes, it might rewrite some headers to make sure bounce messages come back to the list software for further handling (for example, to automatically unsubscribe addresses which consistently return a bounce), and often, it distributes the message with some delay in order not to saturate the outgoing bandwidth and/or choke a receiving server; though some parts of this are also handled by the regular mail transport agent (MTA -- Postfix or Sendmail or what have you).
You might look at ezmlm
EZmlm-idx is a nice mailing list add-on to Qmail. When we install Qmailadmin later on, you'll see that EZmlm-idx integrates seamlessly into Qmailadmin to provide a very user friendly mailing list management interface. As an added bonus, Vpopmail (which we will install as well) will let you control what users can and cannot use mailing lists!
Quoted from qmailrocks
You do not need to worry about sendgrid. Just get a cheap vps,configure DNS and, install MTA(postfix,qmail,sendmail) imap(courier,dovecot), a modern listserve, and let it do it's magic! On the lists where I am subscribed , I have the option to reply to a post, or the whole group.
Can someone help me understand what SendGrid actually adds to an application architecture? Is it's role really as limited as being an alternative delivery engine (SMTP) and post-send analytics? I was hoping that it would do more for transactional email, but everything that I see/read indicates that this is the limit.
My primary use case is for transactional email (new registrations, contact requests, etc.). I'd really like to hear how others have deployed SendGrid within the context of their own web applications. Your experiences may help me better understand how I can best deploy it within my own.
Thanks.
I implemented SendGrid at my last job, and shortly after left that job to go work at SendGrid. At the time, I simply set it up as an SMTP relay, and it took less than 5 minutes.
Scott's answer nailed it, though. We do a lot more than transactional messaging though. We have a newsletter product that you can plug into. A common use case is when a user registers on your site, you send them a transactional Email and fire off an API call to us to add them to one or more mailing lists based on their opt-in preferences. Then, whenever you want to send an Email to your users, you send us a single message via API with the name of the list, and we take care of delivering it on your behalf. We also help with unsubscribes -- eg. if a user unsubscribes from any message, we can use that event to trigger a call back to your site so you can handle anything on your end as well like un-selecting an opt-in checkbox the next time they view their user profile. I know several small businesses run by friends who do this exactly, including my last job who set this up after I left that company.
Another common setup is having one IP address for your transactional Email, and another IP for newsletters. This way if users flag a newsletter as spam (because they're too lazy to click an unsubscribe link, for example), it won't hurt the reputation of the transactional Emails getting through.
Don't hesitate to contact us with questions. We love to help!
If your primary use case is for transactional email, then SendGrid is the right solution for you. It is essentially a drop in replacement for your SMTP, but will scale way better than you could building something yourself from the ground up. It takes time to setup your own SMTP and time to make sure it's not falling over and sending out emails that never get to their destination.
All SendGrid does is email, so all the heavy lifting and boilerplate tedious setup is done for you. You mention transactional email, that is SendGrid's sweet spot. They make sure email gets where it's supposed to go. The big bonus is they give you a lot of tracking goodies along with making sure your email gets where it's supposed to go. There are a number of large companies that said the 15 minutes they spent switching to SendGrid upped their delivery rates substantially.
Unless you love running an SMTP server, instead of building your own product, I'd highly recommend offloading delivery to someone else.
I'm working on an application that will allow management to send registered users (opt-in) broadcast emails at regular intervals, or based on various other criteria. In any case, I'm curious as to whether I should send a separate email to each recipient or bcc all of them on a single message. Currently the email list would be about 1500 recipients, but it should scale all the way up to at least 25k without problems.
Thoughts? Am I getting into a range that I need to worry about being put on spam lists?
Yes, I've had spam list problem with mailing lists of that size, managing email lists for non-profits.
One wants to take extra precautions: make sure your email has SPF records, write a script to send the emails in batches, paced out over time. Definitely send them one one at a time, not as bcc, as direct mail has a better chance of arriving. Make it very easy to unsubscribe. Include people's subscribed email in the message sent -- often people have email forwarded to another account and then try to unsubscribe that account and get frustrated.
Even so, don't be surprised if you have to change your IP at some point.
You are getting into that range. This is the point where I would look to get a third party to send the email on my behalf. Let them worry about being marked as spammers, supply the bandwidth, etc.
I recently built an application with those same criteria. We do the emailing in-house, and send one email to each recipient.
Do use domain keys signing or be sure to use SPF records for your domain. We didn't do that at first, and were blacklisted by a number of different ISPs. Fortunately, it is fairly easy to get them to unblock you. Most will include an online form you can fill out or an email address you can use in the server bounce message.
Don't try to implement the actual email sending yourself. That's a huge waste of time. Either outsource the entire process to one of the many reputable vendors out there (Many organizations I deal with use Constant Contact, and it works well), or run a garden-variety mailing list server (e.g. Mailman) in-house.
Either way, take efforts to make it very easy to unsubscribe (good vendors have that covered), to authenticate that messages are from your company, and to show that your company is not spamming. Real mailing list server software supports all of these goals, by adding proper headers that identify the source very clearly and making unsubscription easy. For instance, Gmail will now offer to send unsubscribe requests in response to mailing list messages marked as 'spam', as has AOL for a long time.
Definitely set up SPF and DKIM if you can manage it.
Finally, whatever you do, make sure you keep logs of your subscriptions, so that if someone does accuse you of spamming, you can defend yourself.
The task is mostly uninteresting on a strictly technical level. You should worry about what happens when a recipient thinks that your list's content is spam and starts (a) complaining or (b) flagging the message as spam with one or more anti-spam service providers. Something like this is bound to happen with a list of the size you describe.
If you are prepared and have the time handle such cases, go for it, at least for a start. (Changing your mail server's IP address as Devin Ceartas suggests won't be of much use by the way.)
If you want to build your own thing, I have two pieces of advice:
Unsubscribing has to be easy, no more than one or two clicks. Using Mailman or any other mailing list manager that was intended for discussion mailing lists is asking for trouble.
BCCing the same message to 1500 (or 25k) recipients may take some load off your mail server, but it has one serious disadvantage: You won't be able to use VERP in order to determine if all addresses that have once been subscribed to your list are still valid. (Large mail providers tend to classify messages as spam if there are delivery attempts to many invalid addresses.)
If I were to build a newsletter emailing system, I will need to be able to generate reports on how many emails bounced, flagged as spam, unsubscribed, read vs. unread, click through rates etc....
So how do you keep track of user activity after the email has been sent? Am I right in assuming that you CAN NOT embed javascript code into emails to monitor user activity? How else do I gather data for my reports?
Once you send the e-mail, it's free like a baby bird kicked out of the nest. The writers of e-mail clients go to great lengths to make sure that they block any feature that will give you that kind of feedback you're asking for. While there are legitimate uses for this sort of information, spammers use such information to verify and clean their e-mail lists.
Many ISPs also block bounces because they give spammers information.
The best you can do is try to give your readers an incentive to click through back to your site. Then, you can gather information not available to a sender of e-mail.
You can easily track click-through rates by including a tracking query string bit in the URLs and route them through your site.
So a link might be: http://mysite.com/?LinkID=foobar
As for read vs. unread you can get an idea for that by including a small transparent image from your site that includes a tracking URL http://mysite.com/track.gif?EmailID=email. However this is not foolproof since emails can be read offline and most modern email clients do not display images without a user action to display images in the email.
For bounced, you'll have to track those by reading from a mailbox for the From email.
Can't think of way to track emails flagged as spam except to send it to several mailboxes that use some of the common spam filtering products and check the results. However, this isn't likely to be accurate because most can and are customized/trained by individual users.
If you want to do click tracking you'll have to replace all links from your message with links
that point back to your tracking script.
To do efficient tracking that you can actually use later for segmenting your list and better targeting you would have to track the subscriber's id and message and/or campaign id.
Some email marketing systems even track the link position in the message so you know exactly if the recipient clicked on the same link that was at the top of the message or in the middle and in the system that I have built I even track if they clicked a link in the html part of the message or the text part.
The tracker script would record all this information then redirect to the actual link.
Bounce tracking is done by processing the bounce messages that your server will receive or generate when a message cannot be delivered. I recommend using VERP: http://en.wikipedia.org/wiki/Variable_envelope_return_path
Open tracking is done by including the image with tracking code in the url. This would normally point to a script on your site that would record the subscriber id and message and/or campaign id then output the binary date for a transparent 1x1 px wide gif.
You can also track messages that are flagged as spam by some users of some ISPs like hotmail, yahoo, aol, and a few others. they offer feedback loops so every time someone clicks that "Spam" button in their webmail application they will send you a message that you can parse and determine the subscriber that actually flagged the message as spam. VERP also helps with this because the feedback loop messages don't always include the actual email address of the subscriber so you need another way to identify them. This page on wikipedia has a list of feedback ISPS that offer feedback loops : http://en.wikipedia.org/wiki/Feedback_Loop_%28email%29