Can someone help me understand what SendGrid actually adds to an application architecture? Is it's role really as limited as being an alternative delivery engine (SMTP) and post-send analytics? I was hoping that it would do more for transactional email, but everything that I see/read indicates that this is the limit.
My primary use case is for transactional email (new registrations, contact requests, etc.). I'd really like to hear how others have deployed SendGrid within the context of their own web applications. Your experiences may help me better understand how I can best deploy it within my own.
Thanks.
I implemented SendGrid at my last job, and shortly after left that job to go work at SendGrid. At the time, I simply set it up as an SMTP relay, and it took less than 5 minutes.
Scott's answer nailed it, though. We do a lot more than transactional messaging though. We have a newsletter product that you can plug into. A common use case is when a user registers on your site, you send them a transactional Email and fire off an API call to us to add them to one or more mailing lists based on their opt-in preferences. Then, whenever you want to send an Email to your users, you send us a single message via API with the name of the list, and we take care of delivering it on your behalf. We also help with unsubscribes -- eg. if a user unsubscribes from any message, we can use that event to trigger a call back to your site so you can handle anything on your end as well like un-selecting an opt-in checkbox the next time they view their user profile. I know several small businesses run by friends who do this exactly, including my last job who set this up after I left that company.
Another common setup is having one IP address for your transactional Email, and another IP for newsletters. This way if users flag a newsletter as spam (because they're too lazy to click an unsubscribe link, for example), it won't hurt the reputation of the transactional Emails getting through.
Don't hesitate to contact us with questions. We love to help!
If your primary use case is for transactional email, then SendGrid is the right solution for you. It is essentially a drop in replacement for your SMTP, but will scale way better than you could building something yourself from the ground up. It takes time to setup your own SMTP and time to make sure it's not falling over and sending out emails that never get to their destination.
All SendGrid does is email, so all the heavy lifting and boilerplate tedious setup is done for you. You mention transactional email, that is SendGrid's sweet spot. They make sure email gets where it's supposed to go. The big bonus is they give you a lot of tracking goodies along with making sure your email gets where it's supposed to go. There are a number of large companies that said the 15 minutes they spent switching to SendGrid upped their delivery rates substantially.
Unless you love running an SMTP server, instead of building your own product, I'd highly recommend offloading delivery to someone else.
Related
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 4 years ago.
Improve this question
In Cuba, web access is extremely censored, so I created a tool that allows more than 50,000 people to browse the Internet through email. Cubans send me an email with an URL in the subject line, and I email them back with the response. Read more at https://apretaste.com.
It was working like a charm, till the communist government of Cuba started blocking my emails. My solution was rotation.
I started with Amazon SES, and I was changing the domain each time it was blocked, but Amazon adds a header to all emails, and once they blocked the header no email from SES was able to reach Cuba any more. The same happened with Mailgun and others, they all add headers.
Currently I am creating Gmail accounts and sending via SMTP, but Google blocks me for no reason and only allows to send 100 emails a day per account. Also I can only create few emails using the same IP address/phone, so I was forced to use anonymous proxies and fake Chinese phones. Now I am fighting a war on two fronts.
An email can be blocked by three parameters: IP address, domain, and email address.
It will be terrific if I can set up my own Postfix server at a VPS that auto-rotates the IP address. Even better if I can simulate "gmail.com", to avoid purchasing a new domain every day.
All the intents to create what I call "the ultimate sender" just either reach the spam folder or add unwanted headers making it too easy to block. I feel exhausted. I hit a knowledge barrier here.
I know I am crossing to the dark side, but this is for a very good cause. Thousands count on this service as their only source of unbiased news, social network and to feel part of the 21st century.
Can you please help me implementing "the ultimate sender", or pointing to another solution that I may be missing?
I have a few suggestions for you.
The first one relies on The Onion Router also known as Tor.
Since you are crossing to the dark side, why not also take a look into the darknet?
Take a look at this list of Tor email providers. If you have your own email server that can be accessed through Tor, it becomes much harder for anybody to stop people from using this service. After all, Tor was developed to offer people uncensored access to the web.
You can read about Tor in detail here, it uses Onion Routing and this is how you would set up your server to use Tor.
Here is an example how you could use it:
The steps that involve the setup, receiving an URL request and sending back the reply are as follows:
Set up an email server.
Configure your email server to use Tor.
Publish the public service name. (e.g. "duskgytldkxiuqc6.onion")
Deploy a client that takes the service name and a URL, and let it send an email with a request to your server.
The client now waits for a reply.
You send a reply and the client receives it.
You can change your service name on a regular basis, but you need to make it accessible to those who will use this service.
Having an own email server means being able to control the email header.
Here is one example how you could make use of it:
Configure your email server so that it receives and recognizes
emails which contain the requested URLs.
Before you send a reply modify the email header so that it shows a random IP address and a random sender email address including a random domain name.
Send your reply.
Sending an email that way means that you cannot be replied back to. But since your reply already contains the requested information there is no need to.
I hope this helps.
Crowd source it.
Find a way that volunteers can send some emails for you. This is the only long term approach that I can think of. A simple web interface with mail to links would be be enough to get started although there are other potential problems with this approach too.
Because you are talking about low numbers of users, you could also use crowdsourcing to create the single email address per person approach. They can create an account on a specific set of email providers and give you the credentials. This would allow the single email per user approach or could be used to rotate through a large set of email accounts to send emails.
The simplest solution is perhaps to set up a local SMTP server on your own computer. You don't even need a server per se.
https://sourceforge.net/projects/winsmtpserver/
There are many other such applications. They are usually used to test SMTP functions during local development, but there is nothing against actually sending spam through them.
I know this would be quite a large task, but how about pairing the users with one or just a few emails so they always receive an email from that email.
I'd assume people wouldn't have more than 100 queries per day, if so they could start receiving them from a backup email
I'd imagine it would look less suspicious for them to appear to be in constant contact with one unique email rather than 50,000 being in contact with one
I know this would be a huge undertaking, but I feel like it solves your issue.
Since the users are willing to receive emails form you then your shouldn't be blocked.
When you mentioned you are getting block does it mean your mail is going in spam or is getting lost in between sending and receiving or it is getting bounced back??
My suggestion would be to setup your own mail server and follow as below:
-Get approx 25 or more ip to rotate. (IP is the most imp part which is tracked and is accountable for the reputation of your mail server)
Don't start sending emails in bulk from the word go it is better to gradullay increase the email volume so that mail server reputation nicely built
keep changing the format of the email often
encourage user to add yourself to there contact list
your best part is user are willing to receive emails from you and you would reply to revived email is the USP of yours but still i will recommend you to register for FBL so that you would know which user is reporting you as spam and you can remove him from your list and never send him email again.
using best practice to send emails like dkim, SPF, dmarc are also vital.
Hope my answer was of some help to you. If you need step by step guide to step up mail server let me know.
My friend, do you remember what made Hillary Clinton lose the last elections to Trump?
It was the "mail" affair. And what was it? People discovered she shared confidential information through a non-official, non-governmental email account (i.e., she used some Gmail, Yahoo or another of a kind). Until here, nothing new with direct relation to your matters. But there is an small particularity on this history, and this can put, maybe not a solution, but maybe a light on a new path you could follow: Clinton actually never sent those emails; the email account she used had the password shared and the communication between people (Clinton-someone) occurred only using the drafts of the account.
How? One side logs in and accesses the drafts folder. There he/she reads the last message and edits it, cutting and writing new data - then save the draft message. On the next turn, the other side of the communication line logs in and do the same. And so forth, so never really sending those messages, but instead just updating the drafts (this "Hillary" method does schooled people... Dilma Rousseff, impeached ex-president of Brazil, actually did this method down there in Brazil too).
So, maybe if you could establish a pact with your user that he/she doesn't delete the account's password, you could pass those information by this method - without "really" exchanging emails. Maybe a "parent" email account (some that could reset a lost password) could be useful too.
Alternative: aren't you able to contract a regular HTTP webserver? You could rely on FTP to publish data to your user, he/she asks for it and you publish a page with that content.
Salvi, have you tried something with Telnet? OK, we are talking here about a text-only environment, but if nothing more would rest in the future, this could be better than nothing. Maybe you could implement a podcast-like, or push-like service based on it. Look what people do with it with references to your walk on the dark side...
If in Windows, open your command prompt.
Type telnet and press Enter.
Type "o" without quotes and press Enter.
Type "towel.blinkenlights.nl" without the quotes and press Enter.
I've been reading up on sending mass email to a user-base, and I'm not feeling comfortable using the PHP mail function. It tends to be too simple, spammy and unreliable.
But that leads me to my question... for a custom application, what should I be using to send email to potentially hundreds of people? ...or is mail okay to use?
I appreciate the help.
I would use a third-party service. There are several of them. They ensure the emails are sent from white-listed IP's and have spent a LOT of money on legal prep for terms, privacy policy, etc to ensure the ISP's play nicely with the incoming mail.
If you're only sending mail to potentially hundreds of people, and not hundreds of thousands of people, PHP's sendmail will handle the load fine. You should be worrying more about the newsletter content and the opt-out easiness than the capability of PHP to send your email. For small campaigns to hundreds of people, check out MyEmma.com as an example of a small-list solution.
What you're probably looking for is an API to offload your email calls to and let a service handle the delivery for you. Sending a large number of email messages from PHP can be tricky as if it's not done quickly enough you run the risk of time-outs, and tracking which have been sent is always troublesome if you want to re-try a big batch.
Not surprisingly there are several companies which offer an email API service to make this sort of thing significantly easier than doing it yourself:
SendGrid (PHP example)
Mandrill (PHP package) from MailChimp
Postmark (PHP libraries)
PostageApp (PHP example)
MailGun (PHP sample)
While I'm a developer for PostageApp, but I encourage you to try out many of these to see what works best for you.
In most cases you need to re-write a small portion of your application to work with the particular API or library used to access the API, and once that's done you can send a very large number of messages with one quick call. The delivery of those messages becomes the responsibility of your provider.
The truth is that the less money you're willing to spend on email sending, the more things you have to do yourself, such as:
White listing your sender IP address (especially if you're on shared hosting this can be a PITA, because the other users can mess things up for you).
Setting up SPF and DKIM to add trust for mail hosts (Hotmail, GMail, etc.)
Checking bounce emails
Handling ISP complaints
This is also amongst the things that third party providers charge you for; if you don't wish to bother with any of the above, feel free to use providers such as Mailchimp, Bluehornet, etc. Make sure they provide what you need before you whip your wallet, some might have surprising hidden costs (such as charging you extra for API usage, use of transactional emails, life-cycle emails, etc.)
If you don't mind doing a few of the above (like checking bounce / complaint emails and making some simple DNS changes) you could sign up for Amazon SES; it has a proper API and their email charges are the lowest I've seen so far and recently they have introduced DKIM (signed emails) support. You can also configure your sendmail (assuming dedicated hosting) to talk directly to SES, so it's easy to hook up any mail() based solution and run it.
First, thanks for everyone that has helped me with this thus far.
The answer I was looking for is http://mandrillapp.com/
This is the service behind MailChimp and it rules in every way!
With regards an OMS, what is the best method to send a confirmation email? The 2 options I have so far are;
A script on the order page sends an email once the record is written to the database.
A scheduled task on the server, send the email, polling the database every-so-often to find new entries.
Which method do systems currently use?
For e-commerce websites, it might be better to think about the best user experience.
Given that, you would want to send the email as soon as the order is received so the user knows that they have purchased the item. The sooner it gets into their inbox, the sooner they will be happy that they have made their purchase.
I agree with Digbyswift that sending the confirmation email once the record is written to the database is the least scalable. But I would argue that if your system has gotten to the point that you are taking so many orders that your system cannot keep up, you have a wonderful problem on your hands that you now probably have resources to handle.
At PostageApp, we handle the emails of a few e-commerce websites, so perhaps you would benefit from an arrangement with an email service provider to off-load this task so that all of your resources can be spent on keeping your site up and your databases running.
Here are some great alternatives if PostageApp is not your style:
Sendgrid
Postmark
Mailjet
This is a question of scalability. Sending a confirmation email once the record is written to the database is the least scalable. The more orders that are taken , the more emails are sent potentially tying up resources.
A scheduled task is certainly better as emails can be queued up and can be sent in a separate process.
A further option which you could consider is using neither and delegating the responsibility of sending emails to a 3rd party dedicated emailing service, i.e. via an API. This is much better since your hosting does not have to consider the load and you can utilise any reporting offered by the 3rd party. Plus many services offer a free quota up to a certain threshold. This will allow you OMS and business to scale appropriately.
If you apply a message based architecture; you could just publish an order created message and have any number of subscribers respond to that event. You could create a listener that sends the email in house (bespoke option) or another listener that called the API of a 3rd party emailer to send the email on your behalf (as per #Digbyswift)
What I've always liked about this approach is
You can have any number of listeners live at any one time.
You can create a new listener and change how you send the email without needing to change/redeploy the OMS application itself.
You can take the listener(s) off line and stop / delay the sending of the email without losing any notifications or affecting the OMS itself.
Realize there a lot of bulk email services, but what I'm trying to do doesn't feel like bulk email; though maybe I'm just thinking about it the wrong way. Case in point, I attempted to create an account on MailChimp and it wants a subscriber list.
I'm send one, and only one email per address; for example, replying to job postings on Craigslist. I'm not sending follow-up emails, and basically the first email I send is the last to an address unless I get a reply.
Sometimes the email I send is based on a template, other times they've asked for something, and I customize the email. I've automated creating resumes and sending emails in the past, but currently the fastest way just seems to manually do it; meaning I can't imagine sending more than 500 emails in a single day.
As far as I'm able to tell if Gmail, Yahoo, Hotmail, Lycos, AOL, etc. - do not offer the service of paying per email over the send limit, since I'd be willing to pay for the service and be done with this, as it's taking up a huge amount of time and overhead to deal with.
Currently, although I've seen sites giving higher numbers, most of the providers appear to limit emails sent to 20-40 a day; possible I'm doing something wrong, but if I am, I don't understand why these limits are not easy to see within an account, so I'm either able to stay within the limits, or see the service is not a good fit for my needs; currently I just get random error-messages, locked-out, etc.
(If needed, and the setup doesn't take more an a few hours, I'm more than willing to deal with configuring scripts, CSV files, templates, etc. on my computer - as long as they're well documented and the end result addresses the issue; meaning that if I have to send a one-off email, I just copy the template, make the changes, and point to the email. The system must also support attachments, since every email has at least one attachment; best solution would be if I'm able to load an attachment once to the system, then alias it in the local script.)
Use the mail account your internet service provider provides. It is unlikely that you hit their limits, and even if that happens, you, as a paying customer, can easily negotiate a higher limit.
if I wanted to create an "email list" along the lines of google groups (and eventually integrate with a web site), what services should I look into? I haven't really dealt with email before so I'm not 100% sure how it all works. I do know I need to be careful about being blacklisted, etc. Do I use something like sendgrid/mailman? I imagine all emails go to my server when a user hits "reply all" and on my end I push that email as though it came from the user but really it came from my "mail" server? It seems a user doesn't need to "reply all" to send to the list and there is no quick way to reply to just one user (which is a good thing for this app).
So the way I imagine this working:
1) I use something like sendgrid to "listen" for emails coming to list#mywebsite.com
2) When emails come (i.e from userx) I do some filtering for spam, etc and then push them all out to "all users/subscribers"
3) All users get something that is "from user x" (but it's really from my server, on google groups it says "via googlegroups") and the "reply to" is list#mywebsite.com
I imagine this is how it all works but I'm not 100% sure this is correct or if sendgrid will allow me to do this (and if it will cost too much).
Thanks
It really depends on what your needs and skills are. For simple mailing lists there is some very established software such as ListServ or MajorDomo
If you're not running a Unix variant, or you're not confident setting up this kind of software they might not be suitable though.
You can use SendGrid to handle your incoming email, but you will have to re-assemble the message before sending it out since it will be posted to your incoming parse URL in a, well, parsed way. The good news is that SendGrid can handle the spam check for you.
SendGrid currently doesn't charge for inbound parsing, but of course that may change. So as far as your cost goes, you would only use credits for the mail you send. Keep in mind that SendGrid only provides the Parse API to silver ($80/mo) customers or customers on the free (200 emails/day) plan, but there are plans to add a lower tier package that also has access to all APIs.
You can also use SendGrid to handle unsubscribes, but if you are using another solution for list management that's probably superfluous.
Googling for "mailing list hosting" brings up a wealth of sites, some of them free at least under some conditions. Many of them will use something like Mailman / Majordomo / Listserv; certainly you can run Mailman or Majordomo on your own server if you like, as well, so your question doesn't really make sense from that point of view. Yahoo Groups / Google Groups can also happily host a list for you. Do you need the list to be visible to subscribers only? Google Groups and Yahoo Groups will host private lists, too.
A mailing list basically works like Bcc: mail. You send to the mailing list's submission address; it then replaces some of the headers (often the Sender: and perhaps Reply-To:) and then redistributes the message to all the subscribers. Behind the scenes, it might rewrite some headers to make sure bounce messages come back to the list software for further handling (for example, to automatically unsubscribe addresses which consistently return a bounce), and often, it distributes the message with some delay in order not to saturate the outgoing bandwidth and/or choke a receiving server; though some parts of this are also handled by the regular mail transport agent (MTA -- Postfix or Sendmail or what have you).
You might look at ezmlm
EZmlm-idx is a nice mailing list add-on to Qmail. When we install Qmailadmin later on, you'll see that EZmlm-idx integrates seamlessly into Qmailadmin to provide a very user friendly mailing list management interface. As an added bonus, Vpopmail (which we will install as well) will let you control what users can and cannot use mailing lists!
Quoted from qmailrocks
You do not need to worry about sendgrid. Just get a cheap vps,configure DNS and, install MTA(postfix,qmail,sendmail) imap(courier,dovecot), a modern listserve, and let it do it's magic! On the lists where I am subscribed , I have the option to reply to a post, or the whole group.