Just a few moments ago, I sended two mail right after each other to two different mail accounts on the same domain. However if the first one that is send takes about a hour before it arrives and the second one that has been send arrives immediately.
Does anyone have a explanation for what is happening?
With kind regart,
Stefan
There are many possible explanations. The most likely is greylisting (an anti-spam procedure) with and without hitting whitelist.
fast delivery hit greylisting whitelist (no delay)
slow delivery received normal treatment, first delivery attempt received "try later" reply
Greylisting temporary whitelist is based on the following triple:
sender email address
IP address of sending host
recipient address.
Related
I have created an emailing system using Amazon's Simple Email Service (SES) that handles bounces to invalid messages with their Notification(SNS) and Queue(SQS) services. Sending emails to valid addresses work as expected, but I am running into a problem when trying to report bounces.
There are 2 bounce situations: the first one works and the second one does not.
1) Emailing a fake address at an existing ISP (for eg: foo#gmail.com or foo2#yahoo.com) - correctly bounces and sends a Notification to my Queue through SNS
2) After emailing a fake address at a fake ISP (for eg: me#fake-website.com), the Queue never receives a bounce from SNS.
However, the bounce is recognize on some level by AWS because it is added to the Bounce-Statistics Graph in the console.
I can't remove these addresses from my email list if I am never notified that email has bounced.
After doing a lot of research, I initially thought that it was a problem with the AWS Suppression List But I dont think that's possible since i have tried sending to email addresses that were very unlikely to have been used in the past 12 days.
My other thought, is this is a soft bounce, and the system will only be updated if it continues to bounce for the next 12 hours.
Any suggestions or advice would be appreciated.
I receive bounce notifications from SES for invalid domains.
The difference is that the bounce is not immediate since there is no responding mail server. SES will hold the mail and retry several times before declaring it a bounce. I receive the bounce notification 12-16 hours after the initial message was sent if the domain is invalid. Usually from a misspelling.
Real Bounce Results
On 4/26 3:53 pm I sent a mail to an invalid domain (user#BLAHindsutrial.com instead of user#BLAHindustrial.com)
On 4/27 6:17 am I received the bounce from SES.
For the past 4 months we have been seeing large delays when sending emails through mandrill to gmail addresses. Sometimes it takes 15 minutes but other times it can be up to an hour. When i check the mandrill outbound section shortly after the email is sent it shows the email was delivered, but it usually takes a while before it actually shows up in my inbox. We are using this service for welcome emails and password resets so waiting long periods of time isn't acceptable.
It has been very hard to find any information on this issue. Has anyone seen this issue? Any recommendations on what i could do to fix it?
I had similar issues with delays on emails sent via Mandrill to gmail.
To fix the issue I viewed the "Sending Domains" page under "Settings" in Mandrill. I discovered the DKIM and SPF DNS records were either missing or not valid. Mandrill will provide you with new values by clicking on the "View... settings" link. After updating these settings we no longer experience the delay.
I've run into this issue a number of times. Our DNS settings were all good (DKIM and SPF confirmed my Mandrill) and after some investigation (looking at the headers of the delayed emails) the delay appeared to be entirely on Mandrill's side (once it was handed off to Gmail or Yahoo the delivery occurred within a second). When I contact Mandrill support they explained why we were seeing these delays:
In looking over the logs for your account we are seeing intermittent
delays for some of your recipients. Generally, the speed of delivery
in most cases depends largely on the receiving domain, and how quickly
they will receive and process emails. Most of the major email
providers limit how much email they'll receive in a certain period of
time, and will restrict delivery—Mandrill's sending servers are
designed to queue and back off sending if this occurs. In these cases,
the receiving mail server or ISP will return a specific kind of SMTP
response telling Mandrill's servers to 'back off' and 'try again
later,' which ultimately results in the message lingering on our mail
servers longer than expected (and since the message isn't passed off
to the receiving server at that point, and we're only getting a 'try
again' response, you won't see that information in the message headers
of the final email you receive. You'll only see that the email stayed
on our servers for a longer time period which can be confusing).
Additionally, even though we may hand the messages off to ISPs for
delivery almost immediately, it's still up to that ISP, like Gmail or
Yahoo, to actually to process that email and place it in the inbox.
Each receiving server is different though, so it may take a different
amount of time for Yahoo to process the mail than Gmail, for example.
In many cases, things like the time of day and overall email traffic
to that recipient server can affect how quickly they're able to
receive and process email.
All that said, the delays you're seeing generally aren't expected, and
while we see that messages are ultimately delivering, we are detecting
factors on our end where we may need to make some changes to help
mitigate further delays. Our delivery team is continuing to monitor
traffic to major ISPs and will make necessary adjustments as needed.
We still periodically see these delays, though they've improved is so the delays are rarely longer than 10 minutes or so, but it still can cause issues with things like password resets or confirmations that are time-sensitive. Bottom line: Mandrill is awesome for bulk mailing, but if you need instantaneous delivery you may want to rely on a different or self-hosted service.
I also had gmail showing emails sent through mandrill around 10 minutes later. And that is unacceptable to register confirmations and password resets.
I had configured my DKIM and SPF dns records and mandrill reported all green in this records.
But mail delivery to gmail was always delayed with no aparent reason.
After a while I decided do test/use my own email server to do this, instead of mandrill. Now there are no delays in gmail. I'm happy :)
After this I think I will only use mandrill for massive email delivery / marketing, where delays are not important. Time will tell.
Would like to hear other people about this subject.
In mandrillapp.com > Settings > Domains > Sending domains, verify these 3 points:
DKIM is valid,
SPF is valid,
domain is verified.
My experience has been that the Google SMTP servers are causing the delay (not Mandrill). Verify this by looking at the original email headers (in gmail, with email opened, in the top right More > Show Original) and pasting the email header into the google Message header analyzer will show you the path your email took and how long it was delayed at each server. This report will also tell you if you DKIM / SPF is invalid.
Why the delay is occurring is still a mystery to me. I suspect however that because the domain I am using to send is new, perhaps the gmail spam filters are grey listing the emails until enough users have opened emails and not clicked the spam button? I don't know.
Mandrill does not offer any support. I've sent them many tickets, but still no answer. I hope someone here will help me out.
I'm sending emails through SMTP. I'm able to send few thousand emails in start, but after few thousands, no email reaching any mailbox. Mandrill activity showing that the email is delivered, but it's not and there's no email in backlog. I have limits around 50K/hour.
I tried making another account after a day, and did some deposit too, but same thing happen with other account too. No error in SMTP client, no error in logs, nothing.
Other thing to notice is, when I hover over 'Delivered' it says "No SMTP event", but emails which are actually got delivered, showing some stats on hover over.
The Mandrill Delivered-status in the UI doesn't actually mean that it is sent, only that Mandrill have received the message for processing. This is of course extremely confusing.
The only way of seeing if an email is actually sent (i.e. successfully delivered to the receiving mail server) is to see if the message has smtp-events. Do note that it can take some time before the SMTP-events are available in the GUI/API (I have experienced a delay of between 2 minutes and 24 hours).
To see all emails that not currently confirmed delivered you can search for "NOT smtp_events.diag:250" in the search field.
After some research, one of the reason, I came to know that if TEST API KEY gets used to send emails, no mail is actually sent to inbox, but webhooks trigger normally and it shows status as delivered on the Mandrill UI. In order to check actual delivery of email to your inbox, please try to use different API KEY other than test account
It turns out to have been a delay in the emails being received by the mailserver (gmail). It appears as if they were accepted and not processed for a few hours.
I had run in to the same issue and this explanation given by OakHosting_James helped me a lot in understanding what is going on:
It turns out the message was sent from IP that is on an RBL (it happens - I get that). So the receiving server rejected the message at SMTP time.
They replied to say two things about that:
(i) Anyone can set up a blacklist and put any IP on for no good reason.
(ii) Some messages bounce in such a way that Mandrill is not able to detect that it's bounced - which is why their website said "delivered".
Let's take those issues:
(i) It's true, but this was UCEPROTECT-Level 1. They're not a pleasant blacklist to work with, but they're not a backyard project for someone with an axe to grind. I still get the fact that it's impossible for any sending network to remain 100% clean. I'm not frustrated that one of their IPs was listed temporarily. But I did feel fobbed off by being told that there are some tiny blacklists out there that no-one in their right mind would use. UCEPROTECT is not one of those, and they should have come clean: "Even with the best spam protection, we get blacklisted occasionally; we detect this very quickly and switch to other IPs."
(ii) I'm sure there are some after-the-fact bounces that Mandrill's system can miss. But this was rejected at SMTP time. How can they mark a message that never left their sending server as "delivered"?
So the solution (to some degree) to the RBL IP problem (i) could be an "Dedicated IP $29.95 / month" for your account in Mandrill. But using a dedicated IP can be a problem on it's own and is for most cases not advised.
In your mail.rb file you need to do the following :
ActionMailer::Base.smtp_settings = {
address: "smtp.mandrillapp.com",
port: 587,
enable_starttls_auto: true,
user_name: "yourname#gmail.com",
password: "apipassword",
authentication: "login"
}
We have set up a system where notifications get sent to a user with the following From address format: user-{0}#aol.com (replace {0} with an ID)
This way we can track what user we sent the message to originally. This format is not likely to change for various reasons. The issue we are running into is this: every email we send out with a dynamic address is creating a new SMTP relay.
My knowledge of relays is very limited. Our host (GoDaddy) limits SMTP relays to 250 per day. This is an application that potentially will be sending out thousands of emails per day.
Is my assumption that every 'unregistered' email address we use opens a new relay, correct? If so, are there any services or hosts that provide unlimited relays for a reasonable price?
Thanks.
so my assumption that every 'unregistered' email address we use opens a new relay, correct?
Relay is basically a classification. every email sent via relay opens an SMTP connection on the mail server and not "a new relay"...that said, It might be cheaper, if you can, to set up your own sending SMPT server. It comes as a component of IIS. If you need any additional information on setting this up, let me know.
Godaddy, yes, due to the different plans. You can try any other email providers like google and yahoo as well. Just make sure to setup a "catch" all mailbox for undelivered mails and any responses.
What kind of practical issues are there concerning sending tons of e-mail from a server? Will the likelihood of that e-mail being received be just the same as if it had been sent from g-mail or a personal e-mail account if I for example just blindly call the mail() function in PHP tens of thousands of times a day?
(note: you are not helping a spammer here, this relates to a notify feature I'm thinking about for a future link sharing site)
While you may technically be able to send thousands of mails per minute, in reality you must be carefull.
Say you send out 500 emails to yahoo for example. if enough people mark your message as spam, soon, ANY email you send to yahoo will be marked as spam, or [BULK]. Many isp's routinely tar-pit or outright reject email from servers on lists such as RBL (the real-time black hole list). If your mail IP gets put on one of these lists, you can kiss sending email normally from that ip ever again goodbye. Users are very finicky and it doesn't take many complaints to get your mail ip blocked at many domains.
Also since you are sending automated messages, there are heuristics used to determine if the same message is being sent to many users on the same domain. This also increases the chance your mail will be marked as spam.
This is why clean emails from some addresses always go into the spam box. Their company may have not been careful enough when sending what could be perceived as spam. Proceed with caution.
http://wiki.apache.org/spamassassin/AvoidingFpsForSenders
http://support.microsoft.com/kb/842851
http://www.blacklistedip.com/rbl_list.php
It helps to set a 'x-mailer' and ('X-MimeOLE' if your pretending to be outlook) of a real mail client.
It also helps to send it from a server that is a mail server for the domain in the from address, with forward & reverse DNS records setup.
No issues. Once a server is correctly configured as a mail server (SMTP) for a particular domain, there is no difference if the mail it sends out came to it from Outlook, or from the mail() function in PHP - both are getting the SMTP server to do all the heavy lifting
I always make sure to set my X-Mailer headers correctly (identifying that the message was sent from within PHP) to ensure that any overzealous anti-spam services recognize it as an automatated notification as opposed to bulk/junk email. e.g.
$headers .= "X-Mailer: PHP/".phpversion();
All the configuration and limits you'll encounter are with the SMTP server, not from PHP. You can configure SMTP to rate-limit to 2 messages per second for example, this means if you queue up 1,200 messages they'll be drip fed out over the next hour rather than all at once (two is a really low number, 5-25 is more realistic).
SMTP is the backbone of email and some SMTP servers can happily handle tens of thousands of messages per minute (or more!) - the only limitation you'll likely face is bandwdith ;)
Check with your hosting provider, especially if you're on shared hosting. For example: GoDaddy limits shared hosting accounts to sending 1000 emails per day on their server (http://support.godaddy.com/groups/web-hosting/forum/topic/e-mail-sending-limit/). I'm sure other providers have their own limits (I believe the provider one of the companies I worked for used limited outgoing emails to 250 per minute or something along those lines).
Edit: In my case the solution was to contact our hosting provider. They provided info to route outgoing emails through a server they had dedicated to sending outgoing emails. Solved the problem right away.