Firstly, I AM NOT A SPAMMER :) I am a legitimate developer, working for a company who is currently developing an enewsletter sending system for our clients.
Now, we sent out a campaign for one of our clients to 80k solicited emails, and we got a huge amount blocked due to spam, even though our client has used ymlp.com to send similar campaigns to the same mailing list in the past with no problems.
I have stumbled across a few nuggets of information such as:
How to send 100,000 emails weekly?
http://www.codinghorror.com/blog/2010/04/so-youd-like-to-send-some-email-through-code.html
So I have got the SPF record setup, and I am in the process of sorting out DKIM records too.
But my question is this... How can ymlp.com and other newsletter sending systems such as campaign monitor and mail chimp get by all the spam blockage, without getting their users to setup these DNS records?
Our client who uses ymlp.com has no other SPF records setup other than the one I have created for our system.
I have noticed the word spammer being thrown around way too freely in topics such as this, so again I must reiterate, I am not looking to spam people, this is a genuine question for a genuine system in development.
Edit: - We seem to pass all SPF / DKIM / DomainKeys checks ( brandonchecketts.com/emailtest.php ), yet still get rejected for spam by a fairly hefty chunk. Our read rate on campaigns sent is about 2% at the mo, which is way below the 7-8% we expect
Have you checked out reverse DNS? It does help on the deliverability front. Also, be sure to monitor the reputation of your IPs.
Hope that helps!
Related
I registered G-Suite free long ago for my domain. We use Google Drive for file sharing and emails under that domain. Recently Google seems forcing me to upgrade to their pay plan. They list some of our key emails to spam list so that those email can't send mail to group. It also list some of our partners emails to spam list so that they can't send mail to email group under our domain.
Google suggests that in order to manage spam list sending to a group under domain, we have to upgrade to a pay plan.
As we have many users, the pay plan will be too expensive. So I'm thinking to run my own mail server, however still want to use google drive for file sharing within users in domains.
I would like to ask if there will be any issue if I change MX records to my own email server and keep using G-Suite free for file sharing with google drive ?!
Thanks,
Klab
The answer to your question is "it depends". Your split brain approach absolutely does work. We have exactly that configuration where we have some MX records going to on-prem, some going to gmail AND THEN to on-prem and some going only to gmail. The mails flow well and users get their email. The reason that I say "it depends" is that it depends on what you mean by issue. There's no issue with mail delivery, but there are issues with management. For example ideally you will have domainA.com for your email and domainB.com for your Gsuite and keep them separate: you don't have to do this obviously, but I wish we had. If you must have only domainA.com with domainA registered as your GoogleID but not with your MX record it will work, but it will probably end up with a headache when you get a problem in two years when userX's emails don't arrive and you have to track through where they go. That may not be an issue for you, but if you end up with 100 sub domains and 100K users then it's irritating to say the least.
You have other options with GSuite Enterprise and I assume Free, you can route all your inbound emails from a mail gateway see the docs so you can have both. Your inbound mails hit your Exchange server which then forwards to GSuite, or you can set up mail routes doc to forward all your inbound emails to your Exchange server, so you keep your MX record as Google and then your forward those mails to Exchange, then you reply from Exchange and the recipient replies back to Google. We do that too. It does work, insofar that the mail is delivered but it gets confusing to debug issues. But if you must have only one domain and you have to split up users then it's one approach.
You also configure a non-Gmail mailbox see doc which routes all your messages to, say, Exchange.
However, before you do, I'd look more into the Gsuite anti-spam features. You can customise some of the Google spam filtering. See doc . You can't customise all of it: we have had hangouts with the Google spam team who (eventually) explained some of their internal workings and there are some spam messages that you simply can't get delivered because the spam filter is applied before the GSuite level. Most business-type spam, rather than the nasty malware or "adult" spam, though is managed at the Gsuite level and you can disable it by domain if you wish. Differentiating between what Google thinks is spam and what the business thinks is spam still crops up for us from time-to-time.
To address your core issue of spam emails not being delivered to your group, I do not know about the free tier: we have the Enterprise tier, but on the assumption that the Groups configuration is the same (which it may not be but if it is) you can configure message moderation docs for a group. You can set "spam messages" to "skip the moderation queues". We have done that where, as with you, legitimate mails get classed as spam because they come from, say, automated services. We have also in cases removed the "archive" ability so the group is really only a mail distribution list and that bypassed the moderation for us.
I enclose a screenshot of the Enterprise Groups moderation options page from the control panel so you can see what we get in Enterprise and if it's different from what you get in Free Tier
For the past 4 months we have been seeing large delays when sending emails through mandrill to gmail addresses. Sometimes it takes 15 minutes but other times it can be up to an hour. When i check the mandrill outbound section shortly after the email is sent it shows the email was delivered, but it usually takes a while before it actually shows up in my inbox. We are using this service for welcome emails and password resets so waiting long periods of time isn't acceptable.
It has been very hard to find any information on this issue. Has anyone seen this issue? Any recommendations on what i could do to fix it?
I had similar issues with delays on emails sent via Mandrill to gmail.
To fix the issue I viewed the "Sending Domains" page under "Settings" in Mandrill. I discovered the DKIM and SPF DNS records were either missing or not valid. Mandrill will provide you with new values by clicking on the "View... settings" link. After updating these settings we no longer experience the delay.
I've run into this issue a number of times. Our DNS settings were all good (DKIM and SPF confirmed my Mandrill) and after some investigation (looking at the headers of the delayed emails) the delay appeared to be entirely on Mandrill's side (once it was handed off to Gmail or Yahoo the delivery occurred within a second). When I contact Mandrill support they explained why we were seeing these delays:
In looking over the logs for your account we are seeing intermittent
delays for some of your recipients. Generally, the speed of delivery
in most cases depends largely on the receiving domain, and how quickly
they will receive and process emails. Most of the major email
providers limit how much email they'll receive in a certain period of
time, and will restrict delivery—Mandrill's sending servers are
designed to queue and back off sending if this occurs. In these cases,
the receiving mail server or ISP will return a specific kind of SMTP
response telling Mandrill's servers to 'back off' and 'try again
later,' which ultimately results in the message lingering on our mail
servers longer than expected (and since the message isn't passed off
to the receiving server at that point, and we're only getting a 'try
again' response, you won't see that information in the message headers
of the final email you receive. You'll only see that the email stayed
on our servers for a longer time period which can be confusing).
Additionally, even though we may hand the messages off to ISPs for
delivery almost immediately, it's still up to that ISP, like Gmail or
Yahoo, to actually to process that email and place it in the inbox.
Each receiving server is different though, so it may take a different
amount of time for Yahoo to process the mail than Gmail, for example.
In many cases, things like the time of day and overall email traffic
to that recipient server can affect how quickly they're able to
receive and process email.
All that said, the delays you're seeing generally aren't expected, and
while we see that messages are ultimately delivering, we are detecting
factors on our end where we may need to make some changes to help
mitigate further delays. Our delivery team is continuing to monitor
traffic to major ISPs and will make necessary adjustments as needed.
We still periodically see these delays, though they've improved is so the delays are rarely longer than 10 minutes or so, but it still can cause issues with things like password resets or confirmations that are time-sensitive. Bottom line: Mandrill is awesome for bulk mailing, but if you need instantaneous delivery you may want to rely on a different or self-hosted service.
I also had gmail showing emails sent through mandrill around 10 minutes later. And that is unacceptable to register confirmations and password resets.
I had configured my DKIM and SPF dns records and mandrill reported all green in this records.
But mail delivery to gmail was always delayed with no aparent reason.
After a while I decided do test/use my own email server to do this, instead of mandrill. Now there are no delays in gmail. I'm happy :)
After this I think I will only use mandrill for massive email delivery / marketing, where delays are not important. Time will tell.
Would like to hear other people about this subject.
In mandrillapp.com > Settings > Domains > Sending domains, verify these 3 points:
DKIM is valid,
SPF is valid,
domain is verified.
My experience has been that the Google SMTP servers are causing the delay (not Mandrill). Verify this by looking at the original email headers (in gmail, with email opened, in the top right More > Show Original) and pasting the email header into the google Message header analyzer will show you the path your email took and how long it was delayed at each server. This report will also tell you if you DKIM / SPF is invalid.
Why the delay is occurring is still a mystery to me. I suspect however that because the domain I am using to send is new, perhaps the gmail spam filters are grey listing the emails until enough users have opened emails and not clicked the spam button? I don't know.
We are involved in the project which is designed to gather UK hotels details that our client needs to create a paper guide with most popular and top rated places in the country.
At the begining of each year we automatically send emails out to hotel owners in order to ask them to update their hotel details.
Unfortunately Client reported that some of hotels never received any of the emails nor that email ended up in spam, especially on hotmail mailbox.
Is there any known approach which could help us to overcome that situation?
One of the solutions we tried was to resign from local SMTP server and purchase external SMTP server on turboSMTP, but without effect.
How would you advise us to you deal with that problem or what have you advised to other companies in the past? Surely there must be a way to resolve that problem completely and we would appreciate your prompt help with that.
Sending an email to multiple recipients within the same company may sometimes have that effect. That company’s email firewall often assumes it’s a spam attack.
There's a lot of factors that come into this. Thankfully, by going for an external SMTP relay, you can offload most of the issues to them.
What you can do, is make sure your domain and emails are configured to increase their validity. Two really key things for this:
SPF records
DKIM signing
SPF
SPF is basically a whitelist of IPs that can send email for your domain. SPF records are added to your DNS server. There are plenty of SPF generators online that can help (like this one). Your SMTP provider will also need to be included in your SPF record.
DKIM
DKIM digitally signs your email to verify that it's been sent by an authorised sender. Your SMTP provider will have info on how to set that up (turboSMTP docs).
If you want to explore more, I recommend Jeff Atwood's (co-founder of SO) article on how horrible email is: http://blog.codinghorror.com/so-youd-like-to-send-some-email-through-code/
I am trying to send mails with mailgun. My DNS config (SPF,DKIM) seems to be ok and are being validated in mailgun service. I can send mail to several users with gmail, live and most others mail providers. However, I have a problem when I sent an email for email accounts of my university.
The message is rejected with the following alert:
"554 5.7.1 : Client host rejected: MX-CIDR"
My current DNS settings are:
TXT # "v=spf1 include:mailgun.org ~all"
MX 10 mxa.mailgun.org.
MX 10 mxb.mailgun.org.
DKIM was validated as well. I checked my domain at mxtoolbox and the dns config pass in all tests. I did not find errors related with that alert in others questions. May someone help me to fix it?
Update 1:
Just some more informations:
1) I dont send, and I have absolutely no intention to send spam. I created an educational website, used by students and instructors, and they send messages sometimes between each others. I also send mail to confirm registers, recovery password, as a lot of others websites do. I only send messages to people who was agreed with my terms of service, that includes the information about my mail policy. It is a small service, I never sent more than 2,000 messages in a month (I have 800 registered users so far)
2) I do not believe I was blacklisted, mxtools verify several blacklists databases and my IP have passed in all verifications. Also, the server is not rejecting all messages from my IP, I can send messages with my personal email with the same domain, but I use different services to handle my personal inbox with my domain and the emails send by my website. So, I guess it may be a DNS record mistake.
3) I only use mailgun (or others transactional email services like mandrill or sendgrid) because it is highly recommended (and easy). I use a small VPS and it is hard to configure my own email service (I am a programmer, I am not an expert in that kind of configuration). If exists negative factors about the use of these systems, I really like to know and learn more.
I see no evidence posted that the reason the receiving mail server is rejecting your mail is because of your SPF records.
There isn't even any evidence here that the receiving mail servers are even performing SPF checks on their incoming mail.
Can you explain why exactly you believe that this has anything to do with SPF?
Just because someone's rejecting your mail, and you happen to be messing around with your SPF records, doesn't mean that the reason for your mail being rejected is due to your SPF records.
The only ones who can tell you exactly why your email is being rejected, and what needs to be done to fix it, is the receiving mail servers' administrators, and that's who you should be asking. They are the only ones who know exactly how their mail servers are configured, and how they work. Unless it's evident from the text of the error message, and it's not, anyone else's answer will be nothing but guesswork.
And actually my guess would be that, if anything, the error message seems to suggest that they have simply blacklisted your IP address range, period, for whatever reason. I would interpret "MX-CIDR" as meaning "MX's IP address' (you can Google what "CIDR" means by yourself); i.e.: sending mail server's IP address is explicitly blacklisted from sending them mail.
Now, taking from the referenced domain's web site, I quote:
"Our software automatically manages the delivery process to give your emails the best chance of landing in the inbox."
I would think that the only type of folks who would be concerned about having "the best chance of landing in" someone inbox would be all the typical spamming parasites. I browsed through the referenced website, and I couldn't shake off a slimy feeling I get after typically wandering into a typical spam spewer.
Is this domain being used to send spam?
If so, then you probably know the answer to your question, already.
Certain SPF libraries might reject emails when trying to perform a reverse lookup on the domain that you're sending from.
They usually get this from the MX records attached to the domain and if there's a mismatch it'll fail out with a rejection (more detail here: http://www.zytrax.com/books/dns/ch9/spf.html).
It's usually only a problem if the receiving server is not necessarily configured correctly, or is being super harsh on incoming mail due to an overwhelming amount of spam.
What kind of practical issues are there concerning sending tons of e-mail from a server? Will the likelihood of that e-mail being received be just the same as if it had been sent from g-mail or a personal e-mail account if I for example just blindly call the mail() function in PHP tens of thousands of times a day?
(note: you are not helping a spammer here, this relates to a notify feature I'm thinking about for a future link sharing site)
While you may technically be able to send thousands of mails per minute, in reality you must be carefull.
Say you send out 500 emails to yahoo for example. if enough people mark your message as spam, soon, ANY email you send to yahoo will be marked as spam, or [BULK]. Many isp's routinely tar-pit or outright reject email from servers on lists such as RBL (the real-time black hole list). If your mail IP gets put on one of these lists, you can kiss sending email normally from that ip ever again goodbye. Users are very finicky and it doesn't take many complaints to get your mail ip blocked at many domains.
Also since you are sending automated messages, there are heuristics used to determine if the same message is being sent to many users on the same domain. This also increases the chance your mail will be marked as spam.
This is why clean emails from some addresses always go into the spam box. Their company may have not been careful enough when sending what could be perceived as spam. Proceed with caution.
http://wiki.apache.org/spamassassin/AvoidingFpsForSenders
http://support.microsoft.com/kb/842851
http://www.blacklistedip.com/rbl_list.php
It helps to set a 'x-mailer' and ('X-MimeOLE' if your pretending to be outlook) of a real mail client.
It also helps to send it from a server that is a mail server for the domain in the from address, with forward & reverse DNS records setup.
No issues. Once a server is correctly configured as a mail server (SMTP) for a particular domain, there is no difference if the mail it sends out came to it from Outlook, or from the mail() function in PHP - both are getting the SMTP server to do all the heavy lifting
I always make sure to set my X-Mailer headers correctly (identifying that the message was sent from within PHP) to ensure that any overzealous anti-spam services recognize it as an automatated notification as opposed to bulk/junk email. e.g.
$headers .= "X-Mailer: PHP/".phpversion();
All the configuration and limits you'll encounter are with the SMTP server, not from PHP. You can configure SMTP to rate-limit to 2 messages per second for example, this means if you queue up 1,200 messages they'll be drip fed out over the next hour rather than all at once (two is a really low number, 5-25 is more realistic).
SMTP is the backbone of email and some SMTP servers can happily handle tens of thousands of messages per minute (or more!) - the only limitation you'll likely face is bandwdith ;)
Check with your hosting provider, especially if you're on shared hosting. For example: GoDaddy limits shared hosting accounts to sending 1000 emails per day on their server (http://support.godaddy.com/groups/web-hosting/forum/topic/e-mail-sending-limit/). I'm sure other providers have their own limits (I believe the provider one of the companies I worked for used limited outgoing emails to 250 per minute or something along those lines).
Edit: In my case the solution was to contact our hosting provider. They provided info to route outgoing emails through a server they had dedicated to sending outgoing emails. Solved the problem right away.