Mandrill does not offer any support. I've sent them many tickets, but still no answer. I hope someone here will help me out.
I'm sending emails through SMTP. I'm able to send few thousand emails in start, but after few thousands, no email reaching any mailbox. Mandrill activity showing that the email is delivered, but it's not and there's no email in backlog. I have limits around 50K/hour.
I tried making another account after a day, and did some deposit too, but same thing happen with other account too. No error in SMTP client, no error in logs, nothing.
Other thing to notice is, when I hover over 'Delivered' it says "No SMTP event", but emails which are actually got delivered, showing some stats on hover over.
The Mandrill Delivered-status in the UI doesn't actually mean that it is sent, only that Mandrill have received the message for processing. This is of course extremely confusing.
The only way of seeing if an email is actually sent (i.e. successfully delivered to the receiving mail server) is to see if the message has smtp-events. Do note that it can take some time before the SMTP-events are available in the GUI/API (I have experienced a delay of between 2 minutes and 24 hours).
To see all emails that not currently confirmed delivered you can search for "NOT smtp_events.diag:250" in the search field.
After some research, one of the reason, I came to know that if TEST API KEY gets used to send emails, no mail is actually sent to inbox, but webhooks trigger normally and it shows status as delivered on the Mandrill UI. In order to check actual delivery of email to your inbox, please try to use different API KEY other than test account
It turns out to have been a delay in the emails being received by the mailserver (gmail). It appears as if they were accepted and not processed for a few hours.
I had run in to the same issue and this explanation given by OakHosting_James helped me a lot in understanding what is going on:
It turns out the message was sent from IP that is on an RBL (it happens - I get that). So the receiving server rejected the message at SMTP time.
They replied to say two things about that:
(i) Anyone can set up a blacklist and put any IP on for no good reason.
(ii) Some messages bounce in such a way that Mandrill is not able to detect that it's bounced - which is why their website said "delivered".
Let's take those issues:
(i) It's true, but this was UCEPROTECT-Level 1. They're not a pleasant blacklist to work with, but they're not a backyard project for someone with an axe to grind. I still get the fact that it's impossible for any sending network to remain 100% clean. I'm not frustrated that one of their IPs was listed temporarily. But I did feel fobbed off by being told that there are some tiny blacklists out there that no-one in their right mind would use. UCEPROTECT is not one of those, and they should have come clean: "Even with the best spam protection, we get blacklisted occasionally; we detect this very quickly and switch to other IPs."
(ii) I'm sure there are some after-the-fact bounces that Mandrill's system can miss. But this was rejected at SMTP time. How can they mark a message that never left their sending server as "delivered"?
So the solution (to some degree) to the RBL IP problem (i) could be an "Dedicated IP $29.95 / month" for your account in Mandrill. But using a dedicated IP can be a problem on it's own and is for most cases not advised.
In your mail.rb file you need to do the following :
ActionMailer::Base.smtp_settings = {
address: "smtp.mandrillapp.com",
port: 587,
enable_starttls_auto: true,
user_name: "yourname#gmail.com",
password: "apipassword",
authentication: "login"
}
Related
This question is not 100% technical. I've looked online and I couldn't find this being discussed.
We have a forgot username feature in our website which mails the username to the user's email address, using our own mail server.
We also put a message in the web-page asking the user to wait for 20 minutes for the mail to arrive since there can be occasional delays.
Our business tester raised an interesting point saying that 20 minutes seems an unacceptable time to ask the user to wait for. He said our technology should send emails immediately and the maximum lag should be 1 minute at worst.
Firstly, emails are usually received immediately by the recipient so there isn't a problem there.
But in our experience in using other websites, sometimes emails do take a while to arrive. I also remember reading somewhere that emails (at a network level) use a lower quality of service QoS unlike voip services. I can't seem to find it now.
Users can also experience delays in receiving emails because of issues in their own mail server.
Now, all we can do is send the mail using our mail server and ensure that the load and resources on the server is well managed.
1) Is there anything else we can do to ensure that our mails are sent quickly at all times.
2) What is the acceptable time, we can ask the user to wait until he logs a call with help-desk? I believe that there can be lags at a network/protocol level and the user's mail server which we can't do anything about.
Thanks.
All you can do is to inform user that your server "handed over" responsibility for email delivery to named SMTP server "beyond your control/responsibility". You may expect it would take a few (<5s) in most cases (>50%).
Your smtp-client may try initial delivery attempt.
On delivery success => inform the user that email delivery/delay is now beyond your control
On (initial) delivery failure => pass message to your SMTP server (with initial delivery attempt skipped).
I'm making a user management system for my app, and I need to send users a "forgot my password" email with a token that lets them reset their account password. I signed up for SendGrid through Azure (to get the 25,000 emails per month free, which sounded like a great deal) and wrote some code to use it, but after testing my program a bit I was dismayed to find that only a couple of my emails actually went through.
After going onto the SG control panel, I found that 4 out of the 6 test emails I sent went through, and all of the others were rejected as being spam. I sent an email to mail-tester.com to see what it though my spam score was and it gave me a 4.3/10.
The email in question was a single sentence with a link to the password reset, without any images or other elements. I only sent those 6 emails out, so the volume of my emails definitely wasn't the issue. Still, I'm very puzzled as to why my messages are getting flagged as spam.
Without going to the trouble of making an elaborate authentication setup, are there any basic changes I can make to my system to make it get through to users?
In this case it's most likely because you are sending such a short message, with a link to 'reset your password' from a non-whitelabelled email address (the email address you're sending from cannot be verified against the actual domain), and the link may also be a different URL. It's probably getting pulled up as a potential phishing email.
You can rectify this by white labeling your domain and email links via the SendGrid dashboard, it's easy to do and should improve your deliverability.
Also check out this article from the SendGrid support team about White Labeling.
A question from 2015 which is sadly still relevant today as usage of SendGrid increases.
My organization has blocked all SendGrid mails except for those on the paid tier using fixed IP addresses with resolvable public DNS names (such as sendgrid1.sampledomain.tld) which we then whitelist.
There are now far too many domain impersonation, phishing and other spam mails coming in from SendGrid for us to allow everything from them - roughly 10 000 mails over a seven day period, which is far too many to manually report to SendGrids abuse department.
So my answer would be that switching to the paid tier of SendGrid is the better option if you like a better chance of your mails arriving intact at their destination.
I receive only Spam Mails from Sendgrid.
Goes direct to Spam folder and try to report Sendgrid everywhere I can. Maybe they get blocked by most mail servers and make them think about their policy in "hosting" all these Spammers.
In my case my emails are marked as spam because of the anchor label different to the href being actually called.
And that's because of the 'click tracking' setting of sendgrid.
So, if you have something like
yourdomain.com
sendgrid may replace the href and you end up with something like:
yourdomain.com
The sendgrid page being called tracks the click and then redirects the user to the url you originally set. But this sometimes results in your email being marked as spam.
Try to set 'click tracking' in sendgrid dashboard to off: settings | tracking | click tracking.
details here: https://sendgrid.com/docs/ui/account-and-settings/tracking/
Always start by setting up Domain Authentication, formerly known as domain whitelabel as #MartynDavies says. Found under Settings -> Sender Authentication in the UI. Should look like this:
https://sendgrid.com/docs/ui/account-and-settings/how-to-set-up-domain-authentication/
To identify problems have a look at Activity and choose to see deferred, drops, bounces, blocks and spam reports.
https://app.sendgrid.com/email_activity
Under Suppressions you can see details for Blocks and Bounces among others:
https://app.sendgrid.com/suppressions/blocks
https://app.sendgrid.com/suppressions/bounces
There you can see errors like:
550 5.7.1 SPF check failed. em1234.mydomain.com does not declare 11.222.33.44 as a valid sender
If it says Verified but you see errors like this then contact SendGrid support.
One thing that has worked is to upgrade from the Free plan to Essentials or Bronze via the Azure Portal. This made a lot of the emails marked as spam pass through.
I had a similar issue when trying to send a user verification email using SendGrid.
In my case, using a custom domain as the sender identity solved the issue.
Make sure to also verify the domain before using it.
For the past 4 months we have been seeing large delays when sending emails through mandrill to gmail addresses. Sometimes it takes 15 minutes but other times it can be up to an hour. When i check the mandrill outbound section shortly after the email is sent it shows the email was delivered, but it usually takes a while before it actually shows up in my inbox. We are using this service for welcome emails and password resets so waiting long periods of time isn't acceptable.
It has been very hard to find any information on this issue. Has anyone seen this issue? Any recommendations on what i could do to fix it?
I had similar issues with delays on emails sent via Mandrill to gmail.
To fix the issue I viewed the "Sending Domains" page under "Settings" in Mandrill. I discovered the DKIM and SPF DNS records were either missing or not valid. Mandrill will provide you with new values by clicking on the "View... settings" link. After updating these settings we no longer experience the delay.
I've run into this issue a number of times. Our DNS settings were all good (DKIM and SPF confirmed my Mandrill) and after some investigation (looking at the headers of the delayed emails) the delay appeared to be entirely on Mandrill's side (once it was handed off to Gmail or Yahoo the delivery occurred within a second). When I contact Mandrill support they explained why we were seeing these delays:
In looking over the logs for your account we are seeing intermittent
delays for some of your recipients. Generally, the speed of delivery
in most cases depends largely on the receiving domain, and how quickly
they will receive and process emails. Most of the major email
providers limit how much email they'll receive in a certain period of
time, and will restrict delivery—Mandrill's sending servers are
designed to queue and back off sending if this occurs. In these cases,
the receiving mail server or ISP will return a specific kind of SMTP
response telling Mandrill's servers to 'back off' and 'try again
later,' which ultimately results in the message lingering on our mail
servers longer than expected (and since the message isn't passed off
to the receiving server at that point, and we're only getting a 'try
again' response, you won't see that information in the message headers
of the final email you receive. You'll only see that the email stayed
on our servers for a longer time period which can be confusing).
Additionally, even though we may hand the messages off to ISPs for
delivery almost immediately, it's still up to that ISP, like Gmail or
Yahoo, to actually to process that email and place it in the inbox.
Each receiving server is different though, so it may take a different
amount of time for Yahoo to process the mail than Gmail, for example.
In many cases, things like the time of day and overall email traffic
to that recipient server can affect how quickly they're able to
receive and process email.
All that said, the delays you're seeing generally aren't expected, and
while we see that messages are ultimately delivering, we are detecting
factors on our end where we may need to make some changes to help
mitigate further delays. Our delivery team is continuing to monitor
traffic to major ISPs and will make necessary adjustments as needed.
We still periodically see these delays, though they've improved is so the delays are rarely longer than 10 minutes or so, but it still can cause issues with things like password resets or confirmations that are time-sensitive. Bottom line: Mandrill is awesome for bulk mailing, but if you need instantaneous delivery you may want to rely on a different or self-hosted service.
I also had gmail showing emails sent through mandrill around 10 minutes later. And that is unacceptable to register confirmations and password resets.
I had configured my DKIM and SPF dns records and mandrill reported all green in this records.
But mail delivery to gmail was always delayed with no aparent reason.
After a while I decided do test/use my own email server to do this, instead of mandrill. Now there are no delays in gmail. I'm happy :)
After this I think I will only use mandrill for massive email delivery / marketing, where delays are not important. Time will tell.
Would like to hear other people about this subject.
In mandrillapp.com > Settings > Domains > Sending domains, verify these 3 points:
DKIM is valid,
SPF is valid,
domain is verified.
My experience has been that the Google SMTP servers are causing the delay (not Mandrill). Verify this by looking at the original email headers (in gmail, with email opened, in the top right More > Show Original) and pasting the email header into the google Message header analyzer will show you the path your email took and how long it was delayed at each server. This report will also tell you if you DKIM / SPF is invalid.
Why the delay is occurring is still a mystery to me. I suspect however that because the domain I am using to send is new, perhaps the gmail spam filters are grey listing the emails until enough users have opened emails and not clicked the spam button? I don't know.
I've set up a Mandrill webhook which will update my app whenever an email hard-bounces or is rejected, so I don't keep that particular email address in my database. The way it works is this: a user gives me an address, I send him a confirmation, and if I don't hear from Mandrill's webhook in 30 minutes, I assume it's OK.
So I ran a some tests with non-existing addresses, and they didn't go too well. Most of them appeared as delivered for hours, long after I'd assumed they were fine.
Also, I didn't account for the delay in receiving webhook batches. One mail bounced at 2:01pm, according to the outbound activity logs, but the webhook history shows a batch being sent only at 2:52pm.
My questions is: how long should I delay my app's assumption of deliverability in order to give Mandrill enough time to detect hard-bounces / rejections and then send me webhook batches? I can live with letting some 5% bad emails going by because of delayed in processing on peak-time or other extraordinary events, but it seems like my 30 minutes isn't enough to catch anything at all...
Not the answer you're looking for, but Mandrill doesn't let you do this. The only way of checking if an email has been delivered is to poll Mandrill (with the message/info.json API). To see if the message is delivered you have to check the smtp_events and look for an event with diag starting with 250. As you've already experienced it may take a long time between delivery and when a message is accessible through the API. In my experience the normal case is around 10 minutes, but it might take many, many hours (this is the case for bounced emails as well as emails that got delivered immediately).
If it is important to you to know when an email is delivered, I would recommend you to switch to another email provider. There are plenty of different ones out there. I've personally used Amazon SES. They're cheaper than Mandrill, and you can expect a delivery notification after a second or so. Do note that Amazon SES is a bit more bare-bone than Mandrill (they don't have support for open/click-tracking, templating, dedicated IP, etc.), so it might not be the right provider for you.
Our clients sometimes don't get the emails that we send out. It's a BIG loss. How do I assure that they receive the emails so that if it's not received in the other end, the program can resend it or do something about it.
None of the suggestions above will work 100% of the time. Many email clients will (rightly so) refuse to load foreign images, negating the usefulness of "web bugs". They will also refuse (or be unable to) return Outlook-style "receipts". And many mail servers either deliberately (to curb spam) or mistakenly (due to misconfiguration) won't return bounce messages. Or possibly an over-aggressive spam filter ate your message, so it arrived but was never seen by the end user. Plus there is the little matter of mail taking hours or days to reach the end user or bounce, and how do you correlate these late notifications or bounces with the mail you sent 4 days ago?
So basically, you can catch some but not all, no matter what you do. I'd say that any design that relies on being able to know with certainty whether the end user got your mail is fatally flawed.
One thing that you can do is set up a bounceback address that receives any mail that is undeliverable. Use the bounceback address as the From address -- you may want a different one for Reply-To so that replies get directed properly.
Check the bounceback mailbox daily and contact customers to get updated email addresses for the ones that fail. You may be able to automate a couple of retries to failed addresses before resorting to the manual contact in case the failure is only intermittent.
This would take some code outside your application that scans the mailbox and keeps some state information about the number of contacts, etc. and attempts the resend.
Depending on how you generate the mails, you might be able to make this process easier: generate a unique bounce address for every single email you send out. You could use bounces+1234#example.com, for example.
Many SMTP servers will allow you to use the part after the + as a parameter to an external script, etc.
The problem is that many (broken) SMTP servers don't return enough info with a bounce to identify the original message -- sometimes, when there are forwardings involved, you don't even get back the original addressee...
With the above trick you can reliably correlate outgoing messages with incoming bounces.
There is no standard way to know whether the email reached the destination. Many email clients support different types of receipts though. You can use any of those if you want.
There are some ways to know when the user actually read the email.
There are many techniques like adding an image to your email that is to be fetched from your web server. When the user reads the email, the request for the image comes to your server and you can capture the event.
The problem is that there is no way to know that the mail did not reach the destination.
I worked on a bulk email system in a previous life. Deliverability was one of our major issues. The most common cause of undelivered emails is a spam filter.
Here are the steps we took to ensure the highest delivery rates:
We used Return Path to test emails for that spam-like smell.
If you send a lot of emails, you need to make sure your SMTP server is not blacklisted.
Remind your users to add your FROM address to their "safe senders" list.
Use a system that collects bouncebacks and use them to scrub your mailing list. This will also help keep you off the blacklists.
If the emails are critical, consider sending them return-receipt-requested. This will not really guarantee anything, but it might give you some metrics on actual deliverability.
There's not really a good way to determine if the email actually arrives in their inbox, you can only confirm that you sent it. Attach a receipt that lets you know when they open it perhaps?
Microsoft Outlook provides similar functionality, however it is based on the email client. I'm not sure if other clients, like Thunderbird, support this.
However, there is nothing in the protocols that specify receipts.
One option that may work: send a link to a generate web page and monitor that page for hits. This provides its own issues however: confidentiality, etc.