My application uses Simple E-mail Service from AWS + GoogleMail for sending mails to clients.
But I have a lot of complaints about mail never received, even after more than 24 hours.
In the logs, I see:
250 2.0.0 Ok: queued as XXXXXXX
reportingMTA: a9-82.smtp-out.amazonses.com
Can anyone tell me:
Is e-mail queued on Amazon OR on recipient server?
Where can I find rules to help me avoid that?
This means Amazon 'did their job' by delivering the message and having it accepted.
What implies that you shouldn't want to avoid this to happen.
The problem is, as Amazon says:
Amazon SES successfully delivers the email to the recipient's mail
server. This notification does not indicate that the actual recipient
received the email because Amazon SES cannot control what happens to
an email after the receiving mail server accepts it. Delivery
notifications are available only through Amazon SNS.
I was in the very situation you are, I had some complaints about never receiving the message. What I realized after a lot of analysis:
There was in fact emails that after being said as delivered, later on came back as 'marked as spam' messages -> I asked them to whitelist my domain
I myself had a flaw in my verification of whether to send email to a client or not ( I was blacklisting all recipients in a batch if any of them sent back a "Permanent Bounce" response)
There was even cases of people saying that didn't receive (because they heard that some other people weren't actually receiving) and investigating further I found out they had in fact received but were just too lazy to take a look to give me correct information (!)
I found many types of "Successfully delivered notifications" (such as "250 2.0.0 Ok: queued as XXXXXXX", "250 ok: Message XXXXXXX accepted", "250 2.6.0 Queued mail for delivery", etc) and I thought there could be a problem related to this "queued" responses, but in the end I found none.
My advice is that you investigate further other possible reasons just like I did.
Related
I tried to send an email through sendgrid. I have a custom domain myself#contoso.com. If i try to send an email to user1#gmail.com with the from address as myself#contoso.com,then,the email is delivered to gmail with "via". When i try sending email to myself#contoso.com with the from address being myself#contoso.com the activity says the email is delivered however, in outlook client i have not received the email. What is it that I can try or do?
I have tried whitelisting contoso.com but it did not work
This is not an answer, as it doesn't help work out why the emails that are "Delivered" have not been received, but this is SendGrid's note on why Email messages with the “Delivered” status are not received:
Twilio SendGrid posts the Delivered event after the destination server accepts the message with a 250 OK response. Once an email is accepted by the destination server, we are unable to see what happens to the message. The receiving server could send it to the inbox, queue it for later delivery, put it into the spam folder, etc.
Often times, a recipient domain will initially accept a message for delivery, and then apply additional filtering afterwards. In this situation, we would have posted the Delivered event, but not have any insight into the additional filtering. Any additional Twilio SendGrid events for your email message would be triggered by recipient engagement (i.e. open/click events, unsubscribes, etc.).
Also note that at the time of writing, that answer had 26 downvotes (and zero upvotes) on the SendGrid website, probably indicating that many other users have experienced this problem.
I had several issue solving this problem. The most important part is to set "Sender Authentication" from sendgrid to your domain dns. There is a instruction here. I'm using godaddy, so the link to set dns is https://dcc.godaddy.com/manage//dns .
We send a lot of email messages from our Siebel 7.8 application, and we'd like to determine whether they have been successfully delivered or not.
According to the Bookshelf, if the SMTP server is down, the Communications Outbound Manager retries to send the message later, so that's not a problem. However, there are still plenty of issues which could cause an email to not be delivered, such as a typo in the address, the receiver having reached its storage quota, etc.
We send our messages this way:
var ps = TheApplication().NewPropertySet();
ps.SetProperty("ActivityId", outboundEmailActivityId);
ps.SetProperty("CommProfile", commProfile);
ps.SetProperty("ProcessMode", "Local");
var bs = TheApplication().GetService("Outbound Communications Manager");
bs.InvokeMethod("SendMessage", ps, psOut);
Using ProcessMode = Local allows us to detect a few errors. For example, if we try to send a message to a non-existant account in the same domain of our SMTP server, it returns 550 Unknown user and then 503 Must have sender and recipient first. The Outbound Communications Manager raises an exception, and we capture and handle it.
However, if we send a message to a non-existant account in a different domain, our SMTP server can't know that it will fail, and therefore it returns 250 Queued, and our code completes successfully. Later (it can range from seconds to a few hours later), we will receive a "Message undeliverable" error message, but at this point we only know that an outbound message failed, we don't know which one.
Is there any way in which Siebel can handle these 'Message undeliverable' notifications automatically?
We are thinking of writing our own process for that, but it seems like a huge task: we'd have to parse the delivery failure notification, identify the failing recipient, search for all the recent messages sent to that address, and somehow, guess which one failed (based on the Message-Id if we are lucky and can read it within Siebel, or on the Subject otherwise).
The problem is that SMTP is by its nature neither a synchronous nor reliable protocol (i.e. in the sense of "engineered for guaranteed delivery"). Your Siebel app server will connect to its assigned SMTP server and ask it to accept a message for delivery and at that time there are a few high level validations that can be perform (some of which you've mentioned but which can also include policy enforcement such as checking whether your (possibly anonymous) identity is authorized for relaying messages to external domains). Once that conversation ends, there is not much else you can reliably do because again, everything from that point is asynchronous and not guaranteed for delivery (any number of intermediate relay agents can be involved, each with their own potential for outages with or without retry, each with the ability to honor or ignore requests for delivery or read receipts or to report invalid recipients, throwing your message in a junk folder or not, etc.). Certainly you can attempt to work with any bounce notifications you do happen to get to try to correlate them back to the sender but that would be outside the context of your sending code.
This question is not 100% technical. I've looked online and I couldn't find this being discussed.
We have a forgot username feature in our website which mails the username to the user's email address, using our own mail server.
We also put a message in the web-page asking the user to wait for 20 minutes for the mail to arrive since there can be occasional delays.
Our business tester raised an interesting point saying that 20 minutes seems an unacceptable time to ask the user to wait for. He said our technology should send emails immediately and the maximum lag should be 1 minute at worst.
Firstly, emails are usually received immediately by the recipient so there isn't a problem there.
But in our experience in using other websites, sometimes emails do take a while to arrive. I also remember reading somewhere that emails (at a network level) use a lower quality of service QoS unlike voip services. I can't seem to find it now.
Users can also experience delays in receiving emails because of issues in their own mail server.
Now, all we can do is send the mail using our mail server and ensure that the load and resources on the server is well managed.
1) Is there anything else we can do to ensure that our mails are sent quickly at all times.
2) What is the acceptable time, we can ask the user to wait until he logs a call with help-desk? I believe that there can be lags at a network/protocol level and the user's mail server which we can't do anything about.
Thanks.
All you can do is to inform user that your server "handed over" responsibility for email delivery to named SMTP server "beyond your control/responsibility". You may expect it would take a few (<5s) in most cases (>50%).
Your smtp-client may try initial delivery attempt.
On delivery success => inform the user that email delivery/delay is now beyond your control
On (initial) delivery failure => pass message to your SMTP server (with initial delivery attempt skipped).
For the past 4 months we have been seeing large delays when sending emails through mandrill to gmail addresses. Sometimes it takes 15 minutes but other times it can be up to an hour. When i check the mandrill outbound section shortly after the email is sent it shows the email was delivered, but it usually takes a while before it actually shows up in my inbox. We are using this service for welcome emails and password resets so waiting long periods of time isn't acceptable.
It has been very hard to find any information on this issue. Has anyone seen this issue? Any recommendations on what i could do to fix it?
I had similar issues with delays on emails sent via Mandrill to gmail.
To fix the issue I viewed the "Sending Domains" page under "Settings" in Mandrill. I discovered the DKIM and SPF DNS records were either missing or not valid. Mandrill will provide you with new values by clicking on the "View... settings" link. After updating these settings we no longer experience the delay.
I've run into this issue a number of times. Our DNS settings were all good (DKIM and SPF confirmed my Mandrill) and after some investigation (looking at the headers of the delayed emails) the delay appeared to be entirely on Mandrill's side (once it was handed off to Gmail or Yahoo the delivery occurred within a second). When I contact Mandrill support they explained why we were seeing these delays:
In looking over the logs for your account we are seeing intermittent
delays for some of your recipients. Generally, the speed of delivery
in most cases depends largely on the receiving domain, and how quickly
they will receive and process emails. Most of the major email
providers limit how much email they'll receive in a certain period of
time, and will restrict delivery—Mandrill's sending servers are
designed to queue and back off sending if this occurs. In these cases,
the receiving mail server or ISP will return a specific kind of SMTP
response telling Mandrill's servers to 'back off' and 'try again
later,' which ultimately results in the message lingering on our mail
servers longer than expected (and since the message isn't passed off
to the receiving server at that point, and we're only getting a 'try
again' response, you won't see that information in the message headers
of the final email you receive. You'll only see that the email stayed
on our servers for a longer time period which can be confusing).
Additionally, even though we may hand the messages off to ISPs for
delivery almost immediately, it's still up to that ISP, like Gmail or
Yahoo, to actually to process that email and place it in the inbox.
Each receiving server is different though, so it may take a different
amount of time for Yahoo to process the mail than Gmail, for example.
In many cases, things like the time of day and overall email traffic
to that recipient server can affect how quickly they're able to
receive and process email.
All that said, the delays you're seeing generally aren't expected, and
while we see that messages are ultimately delivering, we are detecting
factors on our end where we may need to make some changes to help
mitigate further delays. Our delivery team is continuing to monitor
traffic to major ISPs and will make necessary adjustments as needed.
We still periodically see these delays, though they've improved is so the delays are rarely longer than 10 minutes or so, but it still can cause issues with things like password resets or confirmations that are time-sensitive. Bottom line: Mandrill is awesome for bulk mailing, but if you need instantaneous delivery you may want to rely on a different or self-hosted service.
I also had gmail showing emails sent through mandrill around 10 minutes later. And that is unacceptable to register confirmations and password resets.
I had configured my DKIM and SPF dns records and mandrill reported all green in this records.
But mail delivery to gmail was always delayed with no aparent reason.
After a while I decided do test/use my own email server to do this, instead of mandrill. Now there are no delays in gmail. I'm happy :)
After this I think I will only use mandrill for massive email delivery / marketing, where delays are not important. Time will tell.
Would like to hear other people about this subject.
In mandrillapp.com > Settings > Domains > Sending domains, verify these 3 points:
DKIM is valid,
SPF is valid,
domain is verified.
My experience has been that the Google SMTP servers are causing the delay (not Mandrill). Verify this by looking at the original email headers (in gmail, with email opened, in the top right More > Show Original) and pasting the email header into the google Message header analyzer will show you the path your email took and how long it was delayed at each server. This report will also tell you if you DKIM / SPF is invalid.
Why the delay is occurring is still a mystery to me. I suspect however that because the domain I am using to send is new, perhaps the gmail spam filters are grey listing the emails until enough users have opened emails and not clicked the spam button? I don't know.
Mandrill does not offer any support. I've sent them many tickets, but still no answer. I hope someone here will help me out.
I'm sending emails through SMTP. I'm able to send few thousand emails in start, but after few thousands, no email reaching any mailbox. Mandrill activity showing that the email is delivered, but it's not and there's no email in backlog. I have limits around 50K/hour.
I tried making another account after a day, and did some deposit too, but same thing happen with other account too. No error in SMTP client, no error in logs, nothing.
Other thing to notice is, when I hover over 'Delivered' it says "No SMTP event", but emails which are actually got delivered, showing some stats on hover over.
The Mandrill Delivered-status in the UI doesn't actually mean that it is sent, only that Mandrill have received the message for processing. This is of course extremely confusing.
The only way of seeing if an email is actually sent (i.e. successfully delivered to the receiving mail server) is to see if the message has smtp-events. Do note that it can take some time before the SMTP-events are available in the GUI/API (I have experienced a delay of between 2 minutes and 24 hours).
To see all emails that not currently confirmed delivered you can search for "NOT smtp_events.diag:250" in the search field.
After some research, one of the reason, I came to know that if TEST API KEY gets used to send emails, no mail is actually sent to inbox, but webhooks trigger normally and it shows status as delivered on the Mandrill UI. In order to check actual delivery of email to your inbox, please try to use different API KEY other than test account
It turns out to have been a delay in the emails being received by the mailserver (gmail). It appears as if they were accepted and not processed for a few hours.
I had run in to the same issue and this explanation given by OakHosting_James helped me a lot in understanding what is going on:
It turns out the message was sent from IP that is on an RBL (it happens - I get that). So the receiving server rejected the message at SMTP time.
They replied to say two things about that:
(i) Anyone can set up a blacklist and put any IP on for no good reason.
(ii) Some messages bounce in such a way that Mandrill is not able to detect that it's bounced - which is why their website said "delivered".
Let's take those issues:
(i) It's true, but this was UCEPROTECT-Level 1. They're not a pleasant blacklist to work with, but they're not a backyard project for someone with an axe to grind. I still get the fact that it's impossible for any sending network to remain 100% clean. I'm not frustrated that one of their IPs was listed temporarily. But I did feel fobbed off by being told that there are some tiny blacklists out there that no-one in their right mind would use. UCEPROTECT is not one of those, and they should have come clean: "Even with the best spam protection, we get blacklisted occasionally; we detect this very quickly and switch to other IPs."
(ii) I'm sure there are some after-the-fact bounces that Mandrill's system can miss. But this was rejected at SMTP time. How can they mark a message that never left their sending server as "delivered"?
So the solution (to some degree) to the RBL IP problem (i) could be an "Dedicated IP $29.95 / month" for your account in Mandrill. But using a dedicated IP can be a problem on it's own and is for most cases not advised.
In your mail.rb file you need to do the following :
ActionMailer::Base.smtp_settings = {
address: "smtp.mandrillapp.com",
port: 587,
enable_starttls_auto: true,
user_name: "yourname#gmail.com",
password: "apipassword",
authentication: "login"
}