I have read through the documentation but the difference is still not very clear to me.
My understanding is:
1) include: will allow the IPs listed for the specified domain, and also any additional domains listed in that domain's own SPF records
2) a: simply allows IPs listed for the specified domain
Is that correct?
Another try at an answer for anyone else trying to put this together, and based on the syntax doc that OP is probably referring to, the open-spf.org "SPF Record Syntax" page.
Note: Previously this document lived at http://www.open-spf.org/SPF_Record_Syntax/, but that domain seems to have gone offline permanently in 2009. More details on the history and missing domain can be found here.
My answer is that yes, you seem to have it in mind.
a: Test the IP for a match in the A records for the domain.
include: Fetch the entire SPF record for the domain, evaluate IP against it, and if a PASS is found then that PASS becomes the result of the whole SPF test. If no PASS is found then it's not a fail, but your original/top-level SPF test continues (probably to the -all/~all/?all phase).
Reasons to use "a":
Because it's more predictable and straightforward.
Because you haven't set up SPF on the relevant domains.
Because you don't control those domains and the SPF isn't what you'd want (specifically if it's too lenient in accepting other servers that aren't in it's A records)
Reasons to use "include":
Because you already trust the SPF of the domain.
Because the SPF of the domain is complex, and you want to have a single source of truth for don't-repeat-yourself reasons.
Happy to be corrected!
1) include:other-domain.com just includes SPF records from other-domain.com.
If SPF entries of other-domain.com allows some ip's (for example have an ip4: or a: entry), then those ip will be also be allowed.
2) true
Related
Like many others, I have navigated the SPF/DKIM/DMARC world with some confusion.
About 4 weeks ago or so I finished setting everything (SPF/DKIM/DMARC) up correctly for a GoDaddy-hosted domain that uses Google's mailservers.
I set the _dmarc TXT record to take zero action with p=none and I used Postmark to monitor the results to see what was passing and failing over a week.
After a week or so I looked at the Postmark results and inserted the include: statements for the domains that I wanted to pass, but weren't. Then I waited another week to see the results. However, the results showed that the domains still weren't passing SPF or DKIM. Below is the SPF record, I've redacted parts of it that are revealing, but two of the domains are legit and still aren't passing.
v=spf1 include:_spf.google.com include:freshemail.io include:cherryroad.com ~all
Do I need to use the actual IP addresses in the include statements instead of the domains? Postmark lists these as well so that would be easy if so.
No, you shouldn't copy their IPs in there because they are subject to change, especially Google's.
If it's failing, presumably you have some results (usually in message headers) that tell you exactly which IP is failing, and you can track it down manually though those includes, do a reverse lookup on it, etc.
However, you're also using GoDaddy, which is mostly guaranteed not to work as they either block outbound SMTP or route it through their own servers, so you're very unlikely to get an SPF pass.
The issue was with SPF DNS lookup limits. I had no idea this was a thing and I'm amazed that this isn't mentioned anywhere on the documentation (whether that's Google's official documentation or otherwise) on setting up SPF/DKIM/DMARC, and didn't come up in Googling of this issue. This limit is designed to prevent denial of service attacks and infinite DNS loops.
For anyone else who sees this post
v=spf1 include:_spf.google.com include:freshemail.io include:cherryroad.com ~all
This SPF record actually has almost 15 DNS lookups, and the limit is 10 per domain. You can find out how many SPF DNS lookups your domain has with a service like AutoSPF or Easy DMARC
The solution, once you see your total DNS lookups, comes in four options:
Create subdomains and use those to diversify the records. For example using "email#business.mydomain.com" as the email for freshemail.io. Then on the SPF record for that subdomain, you would only have v=spf1 include:freshemail.io resulting in less than 10 DNS lookups for that domain.
As #Synchro mentioned, you don't want to use IPs because those can very well change, but the concept of using IPs instead of the domain names does essentially work because an IP address doesn't cost a DNS lookup. Check with the support/engineering of whatever service you're using, it's possible that they have an IP (or an IP range) that doesn't change often. You might be able to bring your DNS lookups under ten using this.
Note that Google takes up about 3 DNS Lookups, and you'll probably want to leave that one as the _spf.google.com value
Note that every SPF record also has a 255 character limit, so if you're using only IPs you'll need to break that up into a lot of SPF records probably
Use an SPF flattening or compressing service like AutoSPF. Essentially, these services employ method #2, but do some backend work every few hours to check and update the IP addresses associated with the domains. Then they provide you with a "compressed" record like v=spf1 include:_6359384.autospf.com ~all that references all of your records and results in far fewer DNS lookups.
Create your own method that acts kind of like #2 and #3, using GoDaddy's API and brew up something that performs updated lookups on a schedule/job and updates separate SPF records including all of the IPs.
My DNS provider works perfectly for A records.
I am having great difficulty understanding the syntax of SPF records. I have no prior experience.
The DNS provider supports SPF records and it has two control boxes for information: 'Name' and 'SPF data'.
The A record which functions fine looks like this:
Name: potsandpins.info
IPV4 Address: 45.61.228.207
The SPF record which is giving me no joy looks like this:
Name: potsandpins.info
SPF Data: "v=spf1 a -all" (including the quotation marks)
My emails are received with a red flag in Gmail which says 'Gmail couldn't verify that potsandpins.info actually sent this message'.
Can anyone suggest anything as I've tried all sensible permutations?
You don't seem to currently have an SPF record for potsandpins.info maybe you deleted it because you ran into trouble. Anyway, think of the SPF as a whitelist of any IP addresses or hosts you've given permission to send email on your behalf.
The name would be either the root domain, sometimes designated by the #, or a hostname, foo, which you'd use if you were sending email out as example#foo.example.com.
The SPF data would be the version number (v=spf1), then mechanisms (e.g., a), and then the ip addresses or hosts you'd like to authorize, then the qualifier such as -ALL, which intends a hard fail. You may want to back off from that using ~ALL for now, which intends a softfail. I think it's better to be specific in SPF records as then they're easier to follow exactly what they're authorizing.
Here's an example SPF record. Let's say you wanted to authorize 192.0.2.10 and Google.
v=spf1 ip4:192.0.2.10 include:_spf.google.com ~all
Let's say you wanted to authorize a range of IP addresses and MailChimp:
v=spf1 ip4:192.0.2.0/24 include:servers.mcsv.net ~all
Here's a good article on common mistakes in SPF records.
Then it's important to validate your SPF record using a tool such as the SPF Survey. I like this tool because it gives more detailed, actionable error messages when there's a problem.
if you post the full headers of an example email and indicate any other services you use to send email, then it would be possible to provide more specific advice. For future reference, it's best to provide more details when you post to Stack Overflow as that makes it easier to help. I tried in this post but the information you provided limited how specific the answer could be.
Also, for future reference, it's best to post using example.com rather than a real domain name and use IP addresses from an IPv4 block reserved for documentation.
The blocks 192.0.2.0/24 (TEST-NET-1), 198.51.100.0/24 (TEST-NET-2),
and 203.0.113.0/24 (TEST-NET-3) are provided for use in documentation.
Anyway, I hope this helps.
THE SITUATION:
I have ONE e-mail account per domain.
I use e-mails such as [some-alias]#[one-of-my-domains-name]. (server: mail.[mydomain]:[secure port]
My registrar (OVH) is different from my web host (Arvixe).
My hosting plan is a mutualised .NET hosting.
When I want to reply with one of my aliases, I use Mozilla Thunderbird 'Identities'. (Login = concrete domain mail account, FROM: 'the alias e-mail'.)
(And yes, this is very efficient to avoid getting spams and unwanted mailing lists.)
THE PROBLEM:
For some recipient using some spam protection services, I constantly get the error:
Remote server replied: 550 Blocked by SPF ()
HINTS/QUESTIONS AND IDEAS IN SEARCH FOR A SOLUTION
a friend said I have to configure the TXT spf record of my domain.
using different webmaster tools sites to get DIG info, I never get infos about the 'TXT' record. So I'm not sure: Should I edit this record on the side of my domain registrar or in the side of the hosting ?
Current on my registrar's side the record reads:
v=spf1 a:mail.[mydomain] include:mx.ovh.com ~all
and on my hosting's side it reads:
v=spf1 a:mail.[mydomain] ~all
THE BIG QUESTION:
How can I solve this ?
Thank you for your help
Your SPF record is a statement that the IP(s) sending the email are authorised to send email for your domain. In your case I am assuming you're actually sending through the Arvixe servers. The records should be set in the TXT record for your domain at the registrar (ovh).
So in this case you need at your registrar (ovh) to edit the TXT record for SPF to read:
v=spf1 mx a:mail.[yourdomain] include:spf.arvixe.com -all
Note: The modifiers on 'all' vary - +,-,~,? - and specify whether recipients should consider the tests conclusive and reject mail or not. There's a great (very thorough) howto on SPF here: http://www.zytrax.com/books/dns/ch9/spf.html
What is a bit unusual is that the recipients are rejecting what should be an inconclusive SPF record (e.g. default = accept). One of the core problems with email is that there will always be edge cases where recipient servers exhibit odd behaviour - I hope this helps in this case.
Solution found:
- I actually had to set the SPF or TXT (I set both at the same value) record on the hosting's side.
- I used check-auth#verifier.port25.com , which a mail service to checking if your e-mails are passing some anti-spam filter.
- The reply given by check-auth#verifier.port25.com made me understand the Source IP/HELO hostname were not mail.[mydomain] but [myhostingserversubdomain].arvixe.com here what my SPF and TXP spf records look like:
v=spf1 mx ptr:[thesubdomain].arvixe.com -all
(I think 'mx' is really not necessary in my case.)
EDIT : Slight improvement:
v=spf1 ip4:[[thesubdomain].arvixe.com IP] ptr:[thesubdomain].arvixe.com -all
mx removed since in my case the mx servers have nothing to do with it.
"ptr" requires DNS lookup, so I added the direct IP. I left "ptr" in case the IP changes (since it's managed by my host).
Where I can get a list of IP from spam blacklist database. I need something like phishtank API where I can download their blacklist and implement it with my app.
I have seen a spam website doing lookup like http://www.mxtoolbox.com/ I was hoping I could get access to spam feed like them. csv or sql would be nice.
Thank you.
That site is using DNSBL lookups, not a local resource. To perform such lookups, individual blacklists sometimes differ, but in general, to look up an IP address, reverse the octets, add the DNSBL zone, and resolve; a result indicates a match (typically 127.0.0.x where the x might be a result code of some sort). So for example, to look up 123.45.67.89 in dnsbl.example.net, you'd perform an A query like this:
nslookup 89.67.45.123.dnsbl.example.net
(The program nslookup is not particularly good or convenient, but it is widely available, even on Windows.)
Read up on DNS in general and DNSBLs in particular. I have found Wikipedia to have good overviews for these topics.
I have been asked to implement some email address validation on a web app - I'm sure we've all been there a thousand times... however, this time I have been asked to do an MX Lookup on the domain to see if it accepts emails.
Does anyone know of any potential problems with doing this? Is an mx lookup a reliable way of finding out if a domain accepts email? are there any edge cases where a valid email address could fail an MX lookup check?
Thanks for your advice!
are there any edge cases where a valid email address could fail an MX lookup check?
Yes, in that where there is no MX record, MTAs fall back to using the A record instead. So only allowing MX records would fail a class of mail servers that work in the real world.
Allowing any name with an MX or A record at least detects obvious mistypings that result in NXDOMAIN. However it will still allow mistypings that end up at squatter sites. A further step for addresses resolved by A records might be to check where port 25 is accepting connections on that address.
You can only check if there is an mail server registered for the domain.
If the server also accepts mails and if the address is valid (not syntactically but in the sense that there exists a inbox for it and so on...) you will only find out when sending the e.g. registration email
sample on how to do this in PHP
function mailserver_exists($email) {
list($user,$domain) = split('#',$email);
//included check for 'A' after [comment from bobince][1]
return checkdnsrr($domain,'MX') || checkdnsrr($domain,'A');
}
if(domain_exists('joe#foreigndomain.xx')) {...} else {...}
Yes you can use 'TinyTim#192.184.165.13' too. The PHP documentation for checkdnsrr(host, type) states
host may either be the IP address in
dotted-quad notation or the host name