I've been reading up on sending mass email to a user-base, and I'm not feeling comfortable using the PHP mail function. It tends to be too simple, spammy and unreliable.
But that leads me to my question... for a custom application, what should I be using to send email to potentially hundreds of people? ...or is mail okay to use?
I appreciate the help.
I would use a third-party service. There are several of them. They ensure the emails are sent from white-listed IP's and have spent a LOT of money on legal prep for terms, privacy policy, etc to ensure the ISP's play nicely with the incoming mail.
If you're only sending mail to potentially hundreds of people, and not hundreds of thousands of people, PHP's sendmail will handle the load fine. You should be worrying more about the newsletter content and the opt-out easiness than the capability of PHP to send your email. For small campaigns to hundreds of people, check out MyEmma.com as an example of a small-list solution.
What you're probably looking for is an API to offload your email calls to and let a service handle the delivery for you. Sending a large number of email messages from PHP can be tricky as if it's not done quickly enough you run the risk of time-outs, and tracking which have been sent is always troublesome if you want to re-try a big batch.
Not surprisingly there are several companies which offer an email API service to make this sort of thing significantly easier than doing it yourself:
SendGrid (PHP example)
Mandrill (PHP package) from MailChimp
Postmark (PHP libraries)
PostageApp (PHP example)
MailGun (PHP sample)
While I'm a developer for PostageApp, but I encourage you to try out many of these to see what works best for you.
In most cases you need to re-write a small portion of your application to work with the particular API or library used to access the API, and once that's done you can send a very large number of messages with one quick call. The delivery of those messages becomes the responsibility of your provider.
The truth is that the less money you're willing to spend on email sending, the more things you have to do yourself, such as:
White listing your sender IP address (especially if you're on shared hosting this can be a PITA, because the other users can mess things up for you).
Setting up SPF and DKIM to add trust for mail hosts (Hotmail, GMail, etc.)
Checking bounce emails
Handling ISP complaints
This is also amongst the things that third party providers charge you for; if you don't wish to bother with any of the above, feel free to use providers such as Mailchimp, Bluehornet, etc. Make sure they provide what you need before you whip your wallet, some might have surprising hidden costs (such as charging you extra for API usage, use of transactional emails, life-cycle emails, etc.)
If you don't mind doing a few of the above (like checking bounce / complaint emails and making some simple DNS changes) you could sign up for Amazon SES; it has a proper API and their email charges are the lowest I've seen so far and recently they have introduced DKIM (signed emails) support. You can also configure your sendmail (assuming dedicated hosting) to talk directly to SES, so it's easy to hook up any mail() based solution and run it.
First, thanks for everyone that has helped me with this thus far.
The answer I was looking for is http://mandrillapp.com/
This is the service behind MailChimp and it rules in every way!
Related
I am working on a project where I want to allow my clients send a large amount of emails at any given time.
The issue I am having is that usually hosts have a very limited amount of emails allowed to send.
At the moment I am using third party service called Mailgun and it works fine but it is rather expensive since I have to send up to 300 000 emails per months and the number is growing.
Even if I rent a virtual server for example on scaleway where I have to set everything up myself, they have limitations.
Or are those limitations if I use their smtp?
Can anyone clear this one for me?
So my questions are:
Can I create my own smtp or any other way to send emails?
Is it possible to set up my own server and email server and send emails without any limitations?
If yes can you give me a tutorial or a course on that?
Or is it too much of a headache to do it myself and I should stick to third party services?
Thank you!
SMTP is not and headache but IP reputation management is. That's why most hosting providers limit SMTP usage and third party service florish.
As for the question "should I stick to third party ?", there is no right answer possible, only opinions:
If your emails are information requested by users (like forgot my password stuff, or notifications) you could use your own servers, because your IP reputation is going to be good.
If you send marketing stuff, all I can say is that my biggest customers all rely on third party services. Probably for good reasons
We are hosting customer data on behalf of companies/clients, and one of our tasks is to send out a very specific transactional email from us (with our email address as sender and reply-to) to clients customers.
We are trying to move away from storing the personal part of a customers data, including his email address. Of course, in order to be able to send out an email to a customer we need to at some point have access to the email address, but in our view it's a step in the right direction to retrieve the email address from the client during a session instead of retreiving it from our own database.
The problem now is that our unwillingness to have email addresses stored anywhere rules out using email service providers like Sendgrid. Instead we need to send out lots of emails through our own server, and this might hurt deliverability. I've been looking for a kind of "self hosted Sendgrid". One who will enable us to send bulks of emails, and one we can tweak to not store the sent emails.
One solution I've found is sendy.co who defines themselves as:
Sendy is a self hosted application that runs on your web server.
This sound promising, but then I read that emails are sent through Amazon's cloud:
Sendy uses multi-threading to send emails via Amazon SES.
I suppose this leads us back where we started, because then Amazon is storing the email addresses.
As I understand, the high deliverability that ESPs achieve is not only caused by state-of-the-art email headers, but also by their servers being recognized by Google/Gmail, Microsoft and other email hosts. So maybe a high deliverability just isn't possible without an ESP. But is there an alternative approach that lets us acheive relativly high deliverability without needing to involve a 3rd party server to do the sending?
The reason that people tend to pay for this service is because it is reasonably difficult/complicated/time consuming. If sending this email is a core part of your business, you'll want to hire a deliverability engineer to handle this. If it's not, I'd start by contacting the various transactional ESPs and see if you can find one that has an enterprise offering they're willing to tailor to this use case.
I think that I'm a little late to this. I hope that you figured out your question by now.
If you haven't then:
One alternative that you can try is to host your own server with an on-premise option. I would check out SocketLabs Hurricane MTA.
SocketLabs is a cloud ESP, like Sendgrid. But they also have a powerful on-premise option.
https://www.socketlabs.com/blog/introducing-hurricane-mta-3-0/
I'm using my Google Apps account (Standard version) to send single emails (automatically) to real interested customers.
The problem is that my sender email account has been suspended, because it's been considered as a "spammer" by Google.
What can I do in order to send that kind of emails from now on? I use MailChimp to send massive emails, and I've seen it has an API, called STS, for sending single emails, but it also required an Amazon SES account, so I don't know whether to invest on those services or doing something else.
What do you recommend for this case?
You should try a service that specializes in sending "transactional" emails, such as Sendgrid, Postmarkapp and Mailjet.
They have APIs and libraries for most programming languages, but in your case you just need to configure the SMTP server they rovide in your email client and you're good to go. They help you not get into the spam folder of your recipients, and help make sure you don't get flagged as a spammer
With most of them you should get some prettty powerfull analytics as well, which is a nice bonus.
Can someone help me understand what SendGrid actually adds to an application architecture? Is it's role really as limited as being an alternative delivery engine (SMTP) and post-send analytics? I was hoping that it would do more for transactional email, but everything that I see/read indicates that this is the limit.
My primary use case is for transactional email (new registrations, contact requests, etc.). I'd really like to hear how others have deployed SendGrid within the context of their own web applications. Your experiences may help me better understand how I can best deploy it within my own.
Thanks.
I implemented SendGrid at my last job, and shortly after left that job to go work at SendGrid. At the time, I simply set it up as an SMTP relay, and it took less than 5 minutes.
Scott's answer nailed it, though. We do a lot more than transactional messaging though. We have a newsletter product that you can plug into. A common use case is when a user registers on your site, you send them a transactional Email and fire off an API call to us to add them to one or more mailing lists based on their opt-in preferences. Then, whenever you want to send an Email to your users, you send us a single message via API with the name of the list, and we take care of delivering it on your behalf. We also help with unsubscribes -- eg. if a user unsubscribes from any message, we can use that event to trigger a call back to your site so you can handle anything on your end as well like un-selecting an opt-in checkbox the next time they view their user profile. I know several small businesses run by friends who do this exactly, including my last job who set this up after I left that company.
Another common setup is having one IP address for your transactional Email, and another IP for newsletters. This way if users flag a newsletter as spam (because they're too lazy to click an unsubscribe link, for example), it won't hurt the reputation of the transactional Emails getting through.
Don't hesitate to contact us with questions. We love to help!
If your primary use case is for transactional email, then SendGrid is the right solution for you. It is essentially a drop in replacement for your SMTP, but will scale way better than you could building something yourself from the ground up. It takes time to setup your own SMTP and time to make sure it's not falling over and sending out emails that never get to their destination.
All SendGrid does is email, so all the heavy lifting and boilerplate tedious setup is done for you. You mention transactional email, that is SendGrid's sweet spot. They make sure email gets where it's supposed to go. The big bonus is they give you a lot of tracking goodies along with making sure your email gets where it's supposed to go. There are a number of large companies that said the 15 minutes they spent switching to SendGrid upped their delivery rates substantially.
Unless you love running an SMTP server, instead of building your own product, I'd highly recommend offloading delivery to someone else.
I'm working on an application that will allow management to send registered users (opt-in) broadcast emails at regular intervals, or based on various other criteria. In any case, I'm curious as to whether I should send a separate email to each recipient or bcc all of them on a single message. Currently the email list would be about 1500 recipients, but it should scale all the way up to at least 25k without problems.
Thoughts? Am I getting into a range that I need to worry about being put on spam lists?
Yes, I've had spam list problem with mailing lists of that size, managing email lists for non-profits.
One wants to take extra precautions: make sure your email has SPF records, write a script to send the emails in batches, paced out over time. Definitely send them one one at a time, not as bcc, as direct mail has a better chance of arriving. Make it very easy to unsubscribe. Include people's subscribed email in the message sent -- often people have email forwarded to another account and then try to unsubscribe that account and get frustrated.
Even so, don't be surprised if you have to change your IP at some point.
You are getting into that range. This is the point where I would look to get a third party to send the email on my behalf. Let them worry about being marked as spammers, supply the bandwidth, etc.
I recently built an application with those same criteria. We do the emailing in-house, and send one email to each recipient.
Do use domain keys signing or be sure to use SPF records for your domain. We didn't do that at first, and were blacklisted by a number of different ISPs. Fortunately, it is fairly easy to get them to unblock you. Most will include an online form you can fill out or an email address you can use in the server bounce message.
Don't try to implement the actual email sending yourself. That's a huge waste of time. Either outsource the entire process to one of the many reputable vendors out there (Many organizations I deal with use Constant Contact, and it works well), or run a garden-variety mailing list server (e.g. Mailman) in-house.
Either way, take efforts to make it very easy to unsubscribe (good vendors have that covered), to authenticate that messages are from your company, and to show that your company is not spamming. Real mailing list server software supports all of these goals, by adding proper headers that identify the source very clearly and making unsubscription easy. For instance, Gmail will now offer to send unsubscribe requests in response to mailing list messages marked as 'spam', as has AOL for a long time.
Definitely set up SPF and DKIM if you can manage it.
Finally, whatever you do, make sure you keep logs of your subscriptions, so that if someone does accuse you of spamming, you can defend yourself.
The task is mostly uninteresting on a strictly technical level. You should worry about what happens when a recipient thinks that your list's content is spam and starts (a) complaining or (b) flagging the message as spam with one or more anti-spam service providers. Something like this is bound to happen with a list of the size you describe.
If you are prepared and have the time handle such cases, go for it, at least for a start. (Changing your mail server's IP address as Devin Ceartas suggests won't be of much use by the way.)
If you want to build your own thing, I have two pieces of advice:
Unsubscribing has to be easy, no more than one or two clicks. Using Mailman or any other mailing list manager that was intended for discussion mailing lists is asking for trouble.
BCCing the same message to 1500 (or 25k) recipients may take some load off your mail server, but it has one serious disadvantage: You won't be able to use VERP in order to determine if all addresses that have once been subscribed to your list are still valid. (Large mail providers tend to classify messages as spam if there are delivery attempts to many invalid addresses.)