SendGrid's automated security automate SPF and DKIM by a CNAME record. It even allows to have our own SPF and DKIM records in place directly using Domain Registrar.
How do they do it without collusion with existing SPF and DKIM records in place of domain?
Why use Automated Security
By using CNAME records provided by Sendgrid, you delegate the ownership of Sendgrid-specific SPF and DKIM records. Whenever they need to make a change (update SPF whitelisted hostname or rotate DKIM keys), they can do it without you having to touch DNS on your end.
It's convenient.
Below is my understanding of how it works for both types of records.
DKIM
You can have multiple DKIM records (public keys) as long as they use different selectors (host names).
Ex. you may already have a DKIM record for Google Workspace (G Suite):
TXT google._domainkey.yourdomain.com "v=DKIM1; k=rsa; t=s; p=MIGM..."
When setting Sendgrid Automated Security, they will provide you with 2 CNAMEs for DKIM managed by them. E.g.
CNAME s1._domainkey.yourdomain.com s1.domainkey.uXXX.wlXXX.sendgrid.net.
CNAME s2._domainkey.yourdomain.com s2.domainkey.uXXX.wlXXX.sendgrid.net.
Selectors do not conflict with Google's and can co-exist.
If you make a DNS lookup of the CNAME you will see a proper DKIM public key:
TXT s1._domainkey.yourdomain.com "v=DKIM1; k=rsa; t=s; p=MIGM..."
SPF
You can have only one SPF TXT record per (sub)domain. Note such record can list multiple IPs/hostnames that are authenticated. With Sendgrid Automated Security, you create a CNAME for a subdomain that points to SG servers. SPF record is hosted under that subdomain.
Ex. you may already have a SPF record for Google Workspace:
TXT yourdomain.com "v=spf1 include:_spf.google.com ~all"
Sendgrid will provide you with a CNAME similar to this:
CNAME em0000.yourdomain.com u0000000.wX0X0.sendgrid.net.
It does not conflict with your existing SPF record.
If you make a DNS lookup on that CNAME you will find a SPF record:
TXT em0000.yourdomain.com "v=spf1 include:sendgrid.net ~all"
Related
Using G-suite email and DNS configuration for MX records in Route53, I'm blocked on how I can solve this error:
Multiple SPF records may cause delivery and spam classification
issues. v=spf1 include:_spf.google.com ~all v=spf1
include:transmail.net ~all
Route53 only allow a single TXT record for SPF information. Route53 does allow you to use new lines for additional SFP information.
However, when running the G-Suite check, I get the error quoted above and some clients are seeing our emails as SPAM.
Is there a solution to this?
You should have one SPF record for your domain, but you can have multiple include directives in the SPF record. You might want to try something like this:
v=spf1 include:_spf.google.com include:transmail.net ~all
The only way I was able to fix this was to proxy the DNS records in Cloudflare which allows an SPF record per line.
A domain MUST NOT have multiple SPF records, SPF fails with PermError otherwise.
An SPF record is a TXT record in the DNS starting exactly with "v=spf1", followed by an array of mechanisms and/or modifiers.
An SPF check starts by fetching all TXT records starting exactly with "v=spf1" on a domain:
if no such record is found, it returns None;
if multiple such records are found, it returns PermError.
If you have multiple services to add to SPF, you would need to combine them like mti2935.
Learn more here: https://dmarcly.com/blog/can-i-have-multiple-spf-records-on-my-domain
We have noticed that a lot of our emails are falsely flagged as spam. Upon reading online, it seems like a good way to solve this issue is to add an SPF record into the DNS, so we added a TXT record with this content:
v=spf1 a mx ip4:162.123.189.010 include:_spf.google.com include:bluehost.com ~all
Bluehost is our host provider,
162.123.189.010 is our VPS IP address from blue host,
and _spf.google.com is needed because we send/receive email using GMail.
After running a test on Google's MX tester, we got the following error:
The SPF string can not be parsed, do you have any typos in it?
Decision permanent error in processing
Explanation SPF Permanent Error: Too many DNS lookups
Record v=spf1 a mx ip4:162.123.189.010 include:_spf.google.com include:bluehost.com ~all
Does anyone have any idea what the issue is?
"SPF Permanent Error: Too many DNS lookups" is a very specific problem. Your record is too big and SPF checkers will refuse to perform enough DNS queries to determine if something passed SPF.
The SPF spec allows at most 10 DNS lookups. Your SPF record has 17.
RFC 4408 § 10.1 – Processing Limits states:
SPF implementations MUST limit the number of mechanisms and modifiers
that do DNS lookups to at most 10 per SPF check, including any
lookups caused by the use of the "include" mechanism or the
"redirect" modifier. If this number is exceeded during a check, a
PermError MUST be returned. The "include", "a", "mx", "ptr", and
"exists" mechanisms as well as the "redirect" modifier do count
against this limit. The "all", "ip4", and "ip6" mechanisms do not
require DNS lookups and therefore do not count against this limit.
The "exp" modifier does not count against this limit because the DNS
lookup to fetch the explanation string occurs after the SPF record
has been evaluated.
Your SPF record has four lookups before traversing the inclusions, including your a and mx:
v=spf1 a mx ip4:162.123.189.010 include:_spf.google.com include:bluehost.com ~all
Google's SPF
Google has three DNS lookups for three collections of CIDRs that it blesses:
_spf.google.com (+3 lookups)
v=spf1 include:_netblocks.google.com include:_netblocks2.google.com
include:_netblocks3.google.com ~all
_netblocks.google.com
v=spf1 ip4:35.190.247.0/24 ip4:64.233.160.0/19
ip4:66.102.0.0/20 ip4:66.249.80.0/20 ip4:72.14.192.0/18 ip4:74.125.0.0/16
ip4:108.177.8.0/21 ip4:173.194.0.0/16 ip4:209.85.128.0/17
ip4:216.58.192.0/19 ip4:216.239.32.0/19 ~all
_netblocks2.google.com
v=spf1 ip6:2001:4860:4000::/36
ip6:2404:6800:4000::/36 ip6:2607:f8b0:4000::/36 ip6:2800:3f0:4000::/36
ip6:2a00:1450:4000::/36 ip6:2c0f:fb50:4000::/36 ~all
_netblocks3.google.com
v=spf1 ip4:172.217.0.0/19 ip4:172.217.32.0/20
ip4:172.217.128.0/19 ip4:172.217.160.0/20 ip4:172.217.192.0/19
ip4:108.177.96.0/19 ip4:35.191.0.0/16 ip4:130.211.0.0/22 ~all"
Bluehost's SPF
The SPF record for bluehost.com is too large (its SPF record fails Google's MX tester on its own):
bluehost.com (5 lookups before further traversal)
v=spf1 include:spf2.bluehost.com include:_spf.qualtrics.com
include:_spf.google.com include:_spf.salesforce.com
include:sparkpostmail.com -all
spf2.bluehost.com (+0)
v=spf1 ip4:66.147.240.0/20 ip4:69.89.16.0/20 ip4:74.220.192.0/19
ip4:67.222.32.0/19 ip4:70.40.192.0/19 ip4:67.20.64.0/18 ip4:173.254.0.0/17
ip4:50.87.0.0/16 ip4:69.195.64.0/18 -all
_spf.qualtrics.com (+0)
v=spf1 ip4:139.60.152.0/22 ip4:162.247.216.0/22 ip4:54.186.193.102/32
ip4:52.222.73.120/32 ip4:52.222.73.83/32 ip4:52.222.62.51/32
ip4:52.222.75.85/32 ?all
(see above for _spf.google.com's +3 lookups, though redundant lookups don't add to your total)
_spf.salesforce.com (+1 using an SPF macro with the IP address)
v=spf1 exists:%{i}._spf.mta.salesforce.com -all
sparkpostmail.com is a redirection and then another exists macro and a pile of pointers (+6, wow)
v=spf1 redirect=_spf.sparkpostmail.com
v=spf1 exists:%{i}._spf.sparkpostmail.com include:_netblocks.sparkpostmail.com
ptr:sparkpostmail.com ptr:spmta.com ptr:flyingenvelope.com ~all
Danger! That sparkpost.com inclusion pulls in some ptr entries, which are arguably insecure, using a deprecated and "strongly discouraged" SPF mechanism (that's a direct quote from the spec).
_netblocks.sparkpostmail.com was pulled in by the previous record (+0)
v=spf1 ip4:147.253.208.0/20 ip4:192.174.80.0/20 ~all
Bluehost used to use SendGrid, who actually knows what they're doing (their SPF record has no additional lookups), but apparently they have traded SendGrid for SparkPost, who (based on their six extra lookups plus the insecure ptr entries) does not.
Since that totals 12 (13 with include:bluehost.com), you cannot include Bluehhost's SPF.
Bluehost's own suggested SPF record (and its default for all customers) is similarly broken (with 16 lookups, including an easily forged ptr).
Solution for Bluehost: a trimmed and safer SPF record
📢 Hello, Bluehost. I tweeted this at you. This section is just for you.
Bluehost could fix this with the following SPF records in place of their current one:
bluehost.com (7)
v=spf1 a include:spf2.bluehost.com include:_spf.google.com
include:_netblocks.sparkpostmail.com ~all
Though note that I had to downgrade include:sparkpostmail.com (7 + 6 = 13, too large, plus that includes the dangerous ptr records) to just its netblocks (7 + 0 ≤ 10). Bluehost needs to yell at SparkPost or go back to SendGrid. spf2.bluehost.com is unchanged from its current state and should be the only inclusion necessary for Bluehost customers.
(I'd use the IP for the A record to skip a lookup, but it changes so often that it looks like fast flux.)
Bluehost should suggest customers include just spf2.bluehost.com for all of the Bluehost services (assuming they're involved in sending mail). See the next section for how to advise Bluehost customers.
Solution for Bluehost customers
As noted in the previous section, start with this base (3 lookups):
v=spf1 a mx include:spf2.bluehost.com ~all
The final ~all ("soft failure") indicates mail recipients should be mildly dubious of —yet still deliver— mail that fails SPF. Set up DMARC to figure out what works and what is missed on the road to DMARC p=reject (which will block all forged mail).
You'll have to add any hosted email or Email Service Provider(s) you use, plus any other hosts that you want to authorize to send mail on behalf of your domain.
In the case of this question, I see an explicit IP address and hosted mail by Google, so you'll need:
v=spf1 ip4:162.123.189.10 a mx include:spf2.bluehost.com
include:_spf.google.com ~all
Your total DNS lookup count is now seven and therefore your SPF is valid.
"SPF Permanent Error: Too many DNS lookups" is a common type of SPF permanent error. This happens when you have more than 10 DNS lookups in your SPF record.
SPF imposes the 10-DNS-lookup limit to mitigate DDoS attacks.
You can use any online SPF checker to check your SPF record and make sure it doesn't exceed that limit.
However, if your SPF record does exceed the limit, SPF authentication returns the permanent error mentioned above, which is in turn interpreted (in DMARC or otherwise) as fail. This means that the email can fail authentication and be moved to the spam folder. If no further action is taken, this will have a negative impact on your email deliverability.
To fix the too-many-DNS-lookup issue, you can use a service like DMARCLY's Safe SPF
feature to automatically "flatten" your SPF record, so that it never exceeds the limit.
For more information on this, check out this post: Why SPF Authentication Fails
The most obvious problem is that leading 0 in your IP address, which makes it invalid. A minor issue is that it's considered best practice to put literal IPs first, as they are faster for receivers to evaluate. Give this a try:
v=spf1 ip4:162.123.189.10 a mx include:_spf.google.com include:bluehost.com ~all
Rather than using google's checker, I'd recommend Scott Kitterman's site, which is more likely to be accurate (Scott is one of the authors of the SPF spec), and spotted this exact problem.
I have been getting an error of 'too many DNS lookups' for my site's SPF record.
After I read the suggestions from the following posts, I decided to follow the advice to try and consolidate my services to reduce the number of lookups.
References:
Too many DNS lookups in an SPF record
PermError SPF Permanent Error: Too many DNS lookup
Right now my SPF record is this:
v=spf1 a include:servers.mcsv.net include:_spf.google.com include:spf.mtasv.net include:jangomail.com ~all
I want to consolidate my services so that it ends up being this:
v=spf1 a include:servers.mcsv.net include:_spf.google.com ~all
I would replace my jangomail with Mandrill (part of Mailchimp servers.mcsv.net) for my transactional emails. And mtasv.net is redundant because we don't use this service anymore.
My situation is exactly like the one posed in this question here:
SPF Record for Mandrill
Kaitlin from Mandrill suggests setting up an SPF record to include Mailchimp and Google Apps as per below.
v=spf1 a include:servers.mcsv.net include:_spf.google.com ~all
However, if I do this, does it not end up being more than 10 lookups and therefore still exceeding the limitations of the protocol?
2 - initial TXT and SPF Record
1 - include:servers.mcsv.net
1 -_spf.google.com
include:servers.mcsv.net includes
1- servers.mcsv.net
1- spf1.mcsv.net
1- spf.mandrillapp.com
_spf.google.com includes:
1- _spf.google.com TXT
1- _netblocks.google.com TXT
1- _netblocks2.google.com TXT
1- _netblocks3.google.com TXT
Total 11 lookups.
Is there any other way around this?
Consolidation using include may always end up making too many DNS look-ups as the included domains may include others.
To see my approach on solving this issue, have a look at SPF-tools which reassembles the original SPF record and outputs only the ip4 and ip6 fields into the new one. Feel free to ask me any questions, I will gladly help/extend the documentation in the repository.
What would be the correct SPF record to use for both Amazon SES and Google Apps together:
Google Apps says they want you to have the tilde "~" in it: http://support.google.com/a/bin/answer.py?hl=en&answer=178723, but most other examples have a dash "-" instead.
Amazon wants: "v=spf1 include:amazonses.com -all"
Google wants: "v=spf1 include:_spf.google.com ~all"
We currently have this, combining both together:
TXT "v=spf1 include:amazonses.com include:_spf.google.com ~all"
SPF "v=spf1 include:amazonses.com include:_spf.google.com ~all"
1) Is this the correct SPF record?
2) Are we missing anything, should this record be the exact same for both TXT & SPF DNS records? That is all we have, we don't have anything else.
We only send email from Google Apps and Amazon SES, nothing else.
Publish a TXT record:
"v=spf1 include:_spf.google.com include:amazonses.com ~all"
Amazon SES documentation says that no additional SPF configuration is required for a domain, but it turns out that adding include:amazonses.com to the record makes Sender ID pass as well. Even though Sender ID is considered obsolete, some receivers could implement it.
If Amazon SES is configured to use a custom MAIL-FROM subdomain, publish another TXT record for the subdomain:
"v=spf1 include:amazonses.com ~all"
It's good to have a custom subdomain set up for better deliverability and customer experience. For example, the domain will be displayed in the mailed-by field in Gmail.
You can use -all instead of ~all. In this case, emails sent from sources not covered in SPF record may be rejected by recipients.
According to Section 3.1 of RFC 7208:
SPF records MUST be published as a DNS TXT (type 16) Resource Record (RR) [RFC1035] only.
Thus, SPF record type is now obsolete.
Regarding your comment, here is one simple way to test whether SPF works:
Send emails to check-auth#verifier.port25.com from both Gmail and Amazon SES Test Email form.
Afterwards, search the automated reply for SPF check: pass.
The correct is to include each sender in the SPF, regarding the - or ~ it is the SPF properties
"-" = only the listed hosts are permitted
"~" = the listed hosts should send but might exist another sender.
If you are 100% sure that you have all the listed senders you can change the ~ for - BUT you should not maintain both.
TXT "v=spf1 include:amazonses.com include:_spf.google.com -all"
http://www.open-spf.org/SPF_Record_Syntax/
After many months trying to create correct SPF record and prevent my emails from Amazon classified as SPAM I've found that SPF syntax validator is exist. Using a Validator and SPF Policy Tester I was able to create SPF record that suites my requirements.
Gmail:
v=spf1 ip4:199.255.192.0/22 ip4:199.127.232.0/22 ip4:54.240.0.0/18 ip4:69.169.224.0/20 a:amazonses.com include:_spf.google.com +mx ?all
Yandex:
v=spf1 ip4:199.255.192.0/22 ip4:199.127.232.0/22 ip4:54.240.0.0/18 ip4:69.169.224.0/20 a:amazonses.com include:_spf.yandex.net +mx ?all
I email a large number of people (they all asked for the email, don't worry) and we're going to shard the email sending process across three servers.
The emails would either be sent from web1.mydomain.com, mail1.mydomain.com or mail2.mydomain.com
I want to change the SPF records for web1 to allow mail1 or mail2 to send the email, but every site I look on for advice seems to say something different.
So far, I've got
v=spf1 mx a:web1.mydomain.com a:mail1.mydomain.com a:mail2.mydomain.com -all
Is that right? And is there any way I can add a wildcard in case I add a further server, maybe something like
v=spf1 mx a:web1.mydomain.com a:mail[0-9].mydomain.com -all
You could configure a host name which resolves to several IPs. In the SPF entry you could then specify that host.
Define the A records as follows.
mail.example.com. 3600 IN A 127.0.01
mail.example.com. 3600 IN A 127.0.02
mail.example.com. 3600 IN A 127.0.03
Define the SPF records as follows.
example.com. 3600 IN TXT "v=spf1 a ~all"
Check out the domain bitcointalk.org it has a very similar configuration to this. You can check SPF configurations of any domain here:
http://spf.myisp.ch
I would avoid defining a FAIL (-). Use SOFTFAIL (~) instead because SPF entries usually cause problems with mail forwarding.