I have created an emailing system using Amazon's Simple Email Service (SES) that handles bounces to invalid messages with their Notification(SNS) and Queue(SQS) services. Sending emails to valid addresses work as expected, but I am running into a problem when trying to report bounces.
There are 2 bounce situations: the first one works and the second one does not.
1) Emailing a fake address at an existing ISP (for eg: foo#gmail.com or foo2#yahoo.com) - correctly bounces and sends a Notification to my Queue through SNS
2) After emailing a fake address at a fake ISP (for eg: me#fake-website.com), the Queue never receives a bounce from SNS.
However, the bounce is recognize on some level by AWS because it is added to the Bounce-Statistics Graph in the console.
I can't remove these addresses from my email list if I am never notified that email has bounced.
After doing a lot of research, I initially thought that it was a problem with the AWS Suppression List But I dont think that's possible since i have tried sending to email addresses that were very unlikely to have been used in the past 12 days.
My other thought, is this is a soft bounce, and the system will only be updated if it continues to bounce for the next 12 hours.
Any suggestions or advice would be appreciated.
I receive bounce notifications from SES for invalid domains.
The difference is that the bounce is not immediate since there is no responding mail server. SES will hold the mail and retry several times before declaring it a bounce. I receive the bounce notification 12-16 hours after the initial message was sent if the domain is invalid. Usually from a misspelling.
Real Bounce Results
On 4/26 3:53 pm I sent a mail to an invalid domain (user#BLAHindsutrial.com instead of user#BLAHindustrial.com)
On 4/27 6:17 am I received the bounce from SES.
Related
Just a few moments ago, I sended two mail right after each other to two different mail accounts on the same domain. However if the first one that is send takes about a hour before it arrives and the second one that has been send arrives immediately.
Does anyone have a explanation for what is happening?
With kind regart,
Stefan
There are many possible explanations. The most likely is greylisting (an anti-spam procedure) with and without hitting whitelist.
fast delivery hit greylisting whitelist (no delay)
slow delivery received normal treatment, first delivery attempt received "try later" reply
Greylisting temporary whitelist is based on the following triple:
sender email address
IP address of sending host
recipient address.
[Preconditions]
I have developed a software running at google-appengine (GAE) that receives mail and reacts to them. So far so good, it all works as expected.
To receive mails into GAE you must send mail to xxx#myapp.appspotmail.com but as I wanted to use nicer mailadresses I have set up a catch-all adress for one of my domains that forwards all mails to myapp.appspotmail.com. This also works well.
[Question]
If this project gets popular it could potentially have some 10 000 users which all will send around 100 mails/day. The mails are sent automaticly from a 3G enabled device and contains status of the device. This makes me think that a spamfilter would probably consider the mails as spam. And as all mails are routed through the catch-all adress I belive this makes it even more suspicious.
Are there any spamfilters active for mails that are received to appspotmail.com
Are there any risk that the mailserver that forwards all mails will be blacklisted somehow?
Anything else you think of that would potentially prevent me from receiving all mails.
[Clarification]
The devices sending the mails are survaillancecameras which only has support for sending mails. Sometimes with status messages, and sometimes with a photo attached. The cameras can not be configured in any other way than to send mails or (MMS :-D).
Kind Regards
Thomas
For the past 4 months we have been seeing large delays when sending emails through mandrill to gmail addresses. Sometimes it takes 15 minutes but other times it can be up to an hour. When i check the mandrill outbound section shortly after the email is sent it shows the email was delivered, but it usually takes a while before it actually shows up in my inbox. We are using this service for welcome emails and password resets so waiting long periods of time isn't acceptable.
It has been very hard to find any information on this issue. Has anyone seen this issue? Any recommendations on what i could do to fix it?
I had similar issues with delays on emails sent via Mandrill to gmail.
To fix the issue I viewed the "Sending Domains" page under "Settings" in Mandrill. I discovered the DKIM and SPF DNS records were either missing or not valid. Mandrill will provide you with new values by clicking on the "View... settings" link. After updating these settings we no longer experience the delay.
I've run into this issue a number of times. Our DNS settings were all good (DKIM and SPF confirmed my Mandrill) and after some investigation (looking at the headers of the delayed emails) the delay appeared to be entirely on Mandrill's side (once it was handed off to Gmail or Yahoo the delivery occurred within a second). When I contact Mandrill support they explained why we were seeing these delays:
In looking over the logs for your account we are seeing intermittent
delays for some of your recipients. Generally, the speed of delivery
in most cases depends largely on the receiving domain, and how quickly
they will receive and process emails. Most of the major email
providers limit how much email they'll receive in a certain period of
time, and will restrict delivery—Mandrill's sending servers are
designed to queue and back off sending if this occurs. In these cases,
the receiving mail server or ISP will return a specific kind of SMTP
response telling Mandrill's servers to 'back off' and 'try again
later,' which ultimately results in the message lingering on our mail
servers longer than expected (and since the message isn't passed off
to the receiving server at that point, and we're only getting a 'try
again' response, you won't see that information in the message headers
of the final email you receive. You'll only see that the email stayed
on our servers for a longer time period which can be confusing).
Additionally, even though we may hand the messages off to ISPs for
delivery almost immediately, it's still up to that ISP, like Gmail or
Yahoo, to actually to process that email and place it in the inbox.
Each receiving server is different though, so it may take a different
amount of time for Yahoo to process the mail than Gmail, for example.
In many cases, things like the time of day and overall email traffic
to that recipient server can affect how quickly they're able to
receive and process email.
All that said, the delays you're seeing generally aren't expected, and
while we see that messages are ultimately delivering, we are detecting
factors on our end where we may need to make some changes to help
mitigate further delays. Our delivery team is continuing to monitor
traffic to major ISPs and will make necessary adjustments as needed.
We still periodically see these delays, though they've improved is so the delays are rarely longer than 10 minutes or so, but it still can cause issues with things like password resets or confirmations that are time-sensitive. Bottom line: Mandrill is awesome for bulk mailing, but if you need instantaneous delivery you may want to rely on a different or self-hosted service.
I also had gmail showing emails sent through mandrill around 10 minutes later. And that is unacceptable to register confirmations and password resets.
I had configured my DKIM and SPF dns records and mandrill reported all green in this records.
But mail delivery to gmail was always delayed with no aparent reason.
After a while I decided do test/use my own email server to do this, instead of mandrill. Now there are no delays in gmail. I'm happy :)
After this I think I will only use mandrill for massive email delivery / marketing, where delays are not important. Time will tell.
Would like to hear other people about this subject.
In mandrillapp.com > Settings > Domains > Sending domains, verify these 3 points:
DKIM is valid,
SPF is valid,
domain is verified.
My experience has been that the Google SMTP servers are causing the delay (not Mandrill). Verify this by looking at the original email headers (in gmail, with email opened, in the top right More > Show Original) and pasting the email header into the google Message header analyzer will show you the path your email took and how long it was delayed at each server. This report will also tell you if you DKIM / SPF is invalid.
Why the delay is occurring is still a mystery to me. I suspect however that because the domain I am using to send is new, perhaps the gmail spam filters are grey listing the emails until enough users have opened emails and not clicked the spam button? I don't know.
Mandrill does not offer any support. I've sent them many tickets, but still no answer. I hope someone here will help me out.
I'm sending emails through SMTP. I'm able to send few thousand emails in start, but after few thousands, no email reaching any mailbox. Mandrill activity showing that the email is delivered, but it's not and there's no email in backlog. I have limits around 50K/hour.
I tried making another account after a day, and did some deposit too, but same thing happen with other account too. No error in SMTP client, no error in logs, nothing.
Other thing to notice is, when I hover over 'Delivered' it says "No SMTP event", but emails which are actually got delivered, showing some stats on hover over.
The Mandrill Delivered-status in the UI doesn't actually mean that it is sent, only that Mandrill have received the message for processing. This is of course extremely confusing.
The only way of seeing if an email is actually sent (i.e. successfully delivered to the receiving mail server) is to see if the message has smtp-events. Do note that it can take some time before the SMTP-events are available in the GUI/API (I have experienced a delay of between 2 minutes and 24 hours).
To see all emails that not currently confirmed delivered you can search for "NOT smtp_events.diag:250" in the search field.
After some research, one of the reason, I came to know that if TEST API KEY gets used to send emails, no mail is actually sent to inbox, but webhooks trigger normally and it shows status as delivered on the Mandrill UI. In order to check actual delivery of email to your inbox, please try to use different API KEY other than test account
It turns out to have been a delay in the emails being received by the mailserver (gmail). It appears as if they were accepted and not processed for a few hours.
I had run in to the same issue and this explanation given by OakHosting_James helped me a lot in understanding what is going on:
It turns out the message was sent from IP that is on an RBL (it happens - I get that). So the receiving server rejected the message at SMTP time.
They replied to say two things about that:
(i) Anyone can set up a blacklist and put any IP on for no good reason.
(ii) Some messages bounce in such a way that Mandrill is not able to detect that it's bounced - which is why their website said "delivered".
Let's take those issues:
(i) It's true, but this was UCEPROTECT-Level 1. They're not a pleasant blacklist to work with, but they're not a backyard project for someone with an axe to grind. I still get the fact that it's impossible for any sending network to remain 100% clean. I'm not frustrated that one of their IPs was listed temporarily. But I did feel fobbed off by being told that there are some tiny blacklists out there that no-one in their right mind would use. UCEPROTECT is not one of those, and they should have come clean: "Even with the best spam protection, we get blacklisted occasionally; we detect this very quickly and switch to other IPs."
(ii) I'm sure there are some after-the-fact bounces that Mandrill's system can miss. But this was rejected at SMTP time. How can they mark a message that never left their sending server as "delivered"?
So the solution (to some degree) to the RBL IP problem (i) could be an "Dedicated IP $29.95 / month" for your account in Mandrill. But using a dedicated IP can be a problem on it's own and is for most cases not advised.
In your mail.rb file you need to do the following :
ActionMailer::Base.smtp_settings = {
address: "smtp.mandrillapp.com",
port: 587,
enable_starttls_auto: true,
user_name: "yourname#gmail.com",
password: "apipassword",
authentication: "login"
}
I'm usign amazon cloud services to host my webpage. Our web site, actually sends a lot of emails per hour. In one instant our server could be asked to send 30 mails or more.
Sometimes our clients complaint about not getting emails from the web, which is connected to our mail server to send emails. This doesn't happen if we send the email directly from our addresses to theirs, so I'm pretty much know is the web page who's causing the problem.
The thing is I don't know what is happening and neither know what to look for. I've checked memory and cpu of that server and everything seems to work fine
make sure your website sends the messages with a correct bounce address (aka envelope sender address). this does not have to be the same thing as the address in the From: header. by default, this is often something like "apache#www.example.com" - I don't know about amazon). these types of bounce addresses are bad because usually you don't receive the error message if something goes wrong. use a real email account. To check what bounce address you currently use, look at the message source of a received mail and see the Return-Path header.
check the logs of your mailserver for those missing messages. either it reports an error (in which case you should get the error to your bounce address) or it reports the message as sent to the target server (in which case you tell your clients to check THEIR maillogs since you can prove you have sent the message)