I've created an automated script that generates a custom HTML email using PHPMailer within PHP and sends it, from my custom domain email account, to whatever recipient.
The emails are all litmus tested and render perfectly in all clients, hold no comments whatsoever, no strings that seem to be spammy, all good. My domain is also not on any blacklist, etc.
The e-mails first pretty much dropped into the spam for every recipient.
I went back to my hosting service then, and they recommended me to create an SPF - record in the DNS zone of the domain of my e-mail.
This drastically improved the non-spam delivery rate of my emails.
Still, some clients still receive the emails of our company in their junk folder. I ran a spam-test using email on ACID, and all tests were passed, including SPF - record verifications, etc.
The only warnings that showed up were, guess what, that for outlook.com and gmail.com, the emails eventually drop into the spam folder (without telling why) which is obviously a big problem.
And indeed, the only clients still complaining about the spam troubles are indeed gmail users, mainly.
I then came across this very interesting post, and learned that google recommends the setup of all, SPF, DKIM and DMARC.
I first of all included the include:_spf.google.com part in my SPF record, but when it comes to DKIM, I'm stuck.
I'm confused if, to have a working DKIM - register within the DNS of your domain, you actually need to have a Google Workspace account? Because apparently you can only create a DKIM Key within that Workspace account, and I'm worried that this will may expire after my 14-trial period of the Workspace account, and rather further damage my email domain reputation instead of actually improving it.
Or am I misunderstanding things, and there's a different solution to setup a DKIM (and then also DMARC) for a given mail client like gmail for a domain? I'm asking because I have no experience whatsoever in setting up DKIM.
In order to set up DKIM from the Admin console you definitely need to have a Google Workspace account. Using a Cloud Identity free subscription would not work as you are not going to have access to services like Gmail.
To simply generate the key from the Admin console you need the right subscription with access to Gmail, which is only available in Google Workspace subscriptions.
It will most likely stop working after your subscription expires since it is linked to the Admin console from the Google Workspace account where it was generated and once the subscription gets suspended everything stops working.
Related
I have a question about the email deliverability to primarily Gmail and to some extent, Yahoo and Outlook email users. It’s a bit long since I’d like to lay down what I’ve done and tried so far, so please bear with me.
The Issue: Our marketing emails (in newsletter format, in hotel industry) using several email IaaS and PaaS are ending up in spam for Gmail users. About 10% will receive in Promotions tab.
The Providers: Here are the email IaaS and PaaS services I have been trying so far.
SendGrid (we are using their email API)
SparkPost
Mailgun
ElasticEmail
With each platform:
I’ve tried both our own dedicated IP addresses AND the platforms’ shared IPs.
New domains that haven’t been used to send emails
Changed contents of the email completely, including the HTML code (by designing new ones, just in case it’s an issue)
DKIM, SPF and all are setup correctly for each domains we used
I’ve checked all the domains and IPs (both dedicated and shared) against known blacklists and spams using MXTools and such. Everything is okay.
We also checked the emails we’re sending using:
www.isnotspam.com
www.mail-tester.com
Both are recommended by SendGrid. We score highly on both, and according to the analysis results from both, our emails are not considered spam.
We’d also checked our domains and all IPs (both dedicated and shared, from any platforms we mentioned above) using SenderScore. Everything is fine here too.
For dedicated IPs, we did the warm up too. But the issue is, even if we are sending just 1-10 emails for testing and warm up, they do end up in Spam. We aren’t even sending to anyone unsolicited. In fact, just for testing purposes, we are sending emails to multiple email addresses owned by our internal team.
I’d talked at length with support team from SendGrid. After checking everything they could think of, they also stated that they can’t find anything amiss with our configs, settings, IP and domain reputation and their own infrastucture.
So what gives? I am really out of idea. Please help!
Thanks.
I registered G-Suite free long ago for my domain. We use Google Drive for file sharing and emails under that domain. Recently Google seems forcing me to upgrade to their pay plan. They list some of our key emails to spam list so that those email can't send mail to group. It also list some of our partners emails to spam list so that they can't send mail to email group under our domain.
Google suggests that in order to manage spam list sending to a group under domain, we have to upgrade to a pay plan.
As we have many users, the pay plan will be too expensive. So I'm thinking to run my own mail server, however still want to use google drive for file sharing within users in domains.
I would like to ask if there will be any issue if I change MX records to my own email server and keep using G-Suite free for file sharing with google drive ?!
Thanks,
Klab
The answer to your question is "it depends". Your split brain approach absolutely does work. We have exactly that configuration where we have some MX records going to on-prem, some going to gmail AND THEN to on-prem and some going only to gmail. The mails flow well and users get their email. The reason that I say "it depends" is that it depends on what you mean by issue. There's no issue with mail delivery, but there are issues with management. For example ideally you will have domainA.com for your email and domainB.com for your Gsuite and keep them separate: you don't have to do this obviously, but I wish we had. If you must have only domainA.com with domainA registered as your GoogleID but not with your MX record it will work, but it will probably end up with a headache when you get a problem in two years when userX's emails don't arrive and you have to track through where they go. That may not be an issue for you, but if you end up with 100 sub domains and 100K users then it's irritating to say the least.
You have other options with GSuite Enterprise and I assume Free, you can route all your inbound emails from a mail gateway see the docs so you can have both. Your inbound mails hit your Exchange server which then forwards to GSuite, or you can set up mail routes doc to forward all your inbound emails to your Exchange server, so you keep your MX record as Google and then your forward those mails to Exchange, then you reply from Exchange and the recipient replies back to Google. We do that too. It does work, insofar that the mail is delivered but it gets confusing to debug issues. But if you must have only one domain and you have to split up users then it's one approach.
You also configure a non-Gmail mailbox see doc which routes all your messages to, say, Exchange.
However, before you do, I'd look more into the Gsuite anti-spam features. You can customise some of the Google spam filtering. See doc . You can't customise all of it: we have had hangouts with the Google spam team who (eventually) explained some of their internal workings and there are some spam messages that you simply can't get delivered because the spam filter is applied before the GSuite level. Most business-type spam, rather than the nasty malware or "adult" spam, though is managed at the Gsuite level and you can disable it by domain if you wish. Differentiating between what Google thinks is spam and what the business thinks is spam still crops up for us from time-to-time.
To address your core issue of spam emails not being delivered to your group, I do not know about the free tier: we have the Enterprise tier, but on the assumption that the Groups configuration is the same (which it may not be but if it is) you can configure message moderation docs for a group. You can set "spam messages" to "skip the moderation queues". We have done that where, as with you, legitimate mails get classed as spam because they come from, say, automated services. We have also in cases removed the "archive" ability so the group is really only a mail distribution list and that bypassed the moderation for us.
I enclose a screenshot of the Enterprise Groups moderation options page from the control panel so you can see what we get in Enterprise and if it's different from what you get in Free Tier
I'm using my own email server to send and receive my emails. Therefor I've set up a VPS at Tilaa.com which also acts as my webserver.
On the webserver I have DirectAdmin setup which takes care of my administrative things.
The problem is that I can receive and send emails but Outlook, Live and Hotmail refuse the receive any emails coming from my email server. Gmail does work f.e. ( Not even in junk folders )
When the receiver at Outlook/live or hotmail adds my email address to the safe list, emails do get through.
My domain is virtualfarmingworld.com
What I have done?
- Setup SPF record
- Setup DKIM record
- Setup A record mail.virtualfarmingworld.com to server IP 84.22.113.42
http://mxtoolbox.com/SuperTool.aspx?action=mx%3avirtualfarmingworld.com&run=toolpage#
Does anyone have any ideas?
Regards,
Ciryk Popeye
Ciryk,
Hotmail can be a bit tricky, if it's being blocked completely. Then most likely your IP is on their internal blacklist. If it's showing up in their SPAM folder it can be a number of reason. The headers from the email in Hotmail will tell you why it's in the SPAM folder.
Look for SRV:<value> PCL: <Value> and SCL: <Value>
PCL stands for Phising Confidence Level and SCL stands for Spam Confidence Level.
You should run your email through this Mail Tester, it really does point out a lot of issues. It may or may not solve the hotmail issue, but they have this inbox tester their that really awesome that will show you other places you're having issues mailing to. Keep in mind, the previous owner of the IP might of spammed from it and caused issues.
I also notice by helping a lot of people that after signing up to Microsoft Junk Mail Reporting System, wait a few days and then delivery results are better with hotmail. I did a scan on you IP and I think you did that already signed up?
You're also on this blacklist: http://www.dnsblchile.org/
Which is really easy to get off, normally takes a couple of hours after you filled out the form.
I'm making a user management system for my app, and I need to send users a "forgot my password" email with a token that lets them reset their account password. I signed up for SendGrid through Azure (to get the 25,000 emails per month free, which sounded like a great deal) and wrote some code to use it, but after testing my program a bit I was dismayed to find that only a couple of my emails actually went through.
After going onto the SG control panel, I found that 4 out of the 6 test emails I sent went through, and all of the others were rejected as being spam. I sent an email to mail-tester.com to see what it though my spam score was and it gave me a 4.3/10.
The email in question was a single sentence with a link to the password reset, without any images or other elements. I only sent those 6 emails out, so the volume of my emails definitely wasn't the issue. Still, I'm very puzzled as to why my messages are getting flagged as spam.
Without going to the trouble of making an elaborate authentication setup, are there any basic changes I can make to my system to make it get through to users?
In this case it's most likely because you are sending such a short message, with a link to 'reset your password' from a non-whitelabelled email address (the email address you're sending from cannot be verified against the actual domain), and the link may also be a different URL. It's probably getting pulled up as a potential phishing email.
You can rectify this by white labeling your domain and email links via the SendGrid dashboard, it's easy to do and should improve your deliverability.
Also check out this article from the SendGrid support team about White Labeling.
A question from 2015 which is sadly still relevant today as usage of SendGrid increases.
My organization has blocked all SendGrid mails except for those on the paid tier using fixed IP addresses with resolvable public DNS names (such as sendgrid1.sampledomain.tld) which we then whitelist.
There are now far too many domain impersonation, phishing and other spam mails coming in from SendGrid for us to allow everything from them - roughly 10 000 mails over a seven day period, which is far too many to manually report to SendGrids abuse department.
So my answer would be that switching to the paid tier of SendGrid is the better option if you like a better chance of your mails arriving intact at their destination.
I receive only Spam Mails from Sendgrid.
Goes direct to Spam folder and try to report Sendgrid everywhere I can. Maybe they get blocked by most mail servers and make them think about their policy in "hosting" all these Spammers.
In my case my emails are marked as spam because of the anchor label different to the href being actually called.
And that's because of the 'click tracking' setting of sendgrid.
So, if you have something like
yourdomain.com
sendgrid may replace the href and you end up with something like:
yourdomain.com
The sendgrid page being called tracks the click and then redirects the user to the url you originally set. But this sometimes results in your email being marked as spam.
Try to set 'click tracking' in sendgrid dashboard to off: settings | tracking | click tracking.
details here: https://sendgrid.com/docs/ui/account-and-settings/tracking/
Always start by setting up Domain Authentication, formerly known as domain whitelabel as #MartynDavies says. Found under Settings -> Sender Authentication in the UI. Should look like this:
https://sendgrid.com/docs/ui/account-and-settings/how-to-set-up-domain-authentication/
To identify problems have a look at Activity and choose to see deferred, drops, bounces, blocks and spam reports.
https://app.sendgrid.com/email_activity
Under Suppressions you can see details for Blocks and Bounces among others:
https://app.sendgrid.com/suppressions/blocks
https://app.sendgrid.com/suppressions/bounces
There you can see errors like:
550 5.7.1 SPF check failed. em1234.mydomain.com does not declare 11.222.33.44 as a valid sender
If it says Verified but you see errors like this then contact SendGrid support.
One thing that has worked is to upgrade from the Free plan to Essentials or Bronze via the Azure Portal. This made a lot of the emails marked as spam pass through.
I had a similar issue when trying to send a user verification email using SendGrid.
In my case, using a custom domain as the sender identity solved the issue.
Make sure to also verify the domain before using it.
I've searched all around, made several changes over the past two weeks, and still no luck so here I am.
We just put up a new site, and there are 3 different forms. Each form sends to a different email of theirs, a forwarder that sends to the same email of theirs (I had to make this after I figured out there was a problem with them not receiving emails from the website), and one of our emails.
Currently, they use office 365 for their email. A few days ago I figured out to change the SPF record, so I added the IP of their current website.
Here is the current SPF record:
v=spf1 include:spf.protection.outlook.com ip4:23.229.157.193 a ~all
I'm stumped. I've sent test submissions, and they receive the forward, and I receive it from my email, but the email that it's supposed to be sent to doesn't receive it.
I don't have access to their office 365 account. I tried a different option of sending the emails through swiftmailer, but GoDaddy doesn't allow me to connect to their smtp details, so that's a bust.
Has anyone encountered this problem before and know of a solution? All help is greatly appreciated.
THE SOLUTION:
After hours of calling, I was able to get the problem solved. I should have edited this earlier, but better late than never. In cPanel, there is an area for routing mail. It was set to local, rather than remote. Every email that came through went to the local emails, and since their were none, they were discarded. After changing the option to remote, the emails started flowing through. After the 3rd or 4th call, I reached someone who's actually dealt with this problem because he explained what was happening and the fix in under two minutes, unlike the others. I hope this helps anyone in the future with the same problems I encountered.
If you've configured SPF on your sending smtp server, you can configure a _dmarc
DNS record with an email address for the receiving server to send mail reports to...
Better yet, if this 'new' server is not required to be fully operational while you set up everything - you can set the _dmarc record to tell the receiving server to reject anything that doesn't pass the SPF test.
In any case, if you are setting up an email server that will send messages to any outside Internet address, and you have the ability to install software on the server - you should install and configure:
SPF, DKIM, and have a dmarc DNS record.
If you don't have these items, it's very likely much of your site's notification email will end up in the subscribers' spam box, or worse rejected by the receiving server.
Several good websites that have helped me:
unlocktheinbox.com
dmarcian.com
emailsecuritygrader
protodave.com dkim key checker
appmaildev.com domainkeys test
gettingemaildelivered.com