I tried to find the answer from google but all result is showing why SPF is important instead of explain the working mechanism and how mail server(gmail, microsoft, smartermail, etc) implement it, generally.
Below is the criteria in came out but could find the answer:
SPF record exist, labeled sender & mail server domain aren't same, mail server domain/IP included
SPF record exist, labeled sender & mail server domain aren't same, mail server domain/IP not included
SPF record exist, labeled sender & mail server domain are same, mail server domain/IP not included
SPF record not exist, labeled sender & mail server domain aren't same
SPF record not exist, labeled sender & mail server domain are same
I would like to know, generally, which criteria will mark as junk mail by mail server.
Thank you.
Edit 1:
Lets put the other factor apart, how mail server decide to increase/decrease the level of "points" by looking at SPF only?
SPF is only responsible for identifying sources of email, and has no opinion about content.
You're asking how receiving email servers decide what to do with messages that fail SPF checks. That's a good question, because it's something that a domain owner should be concerned about, and historically this has been undefined (as others have pointed out), and so varied wildly. Fortunately there's now a mechanism whereby the domain owner can say what a receiving server should do with messages that fail SPF checks: DMARC.
DMARC includes a p parameter that tells a receiver what to do with messages that fail checks. Its value can be none (do nothing, or whatever the receiver chooses), quarantine (put in spam or similar), or reject (bounce the message).
DMARC can apply these same policies to DKIM, and it also provides additional validation of the alignment between the SMTP envelope sender and the From message header.
If a domain lacks a DMARC record, you're back to guessing the outcome, and subject to the whims of receiving mail server admins' decisions.
Related
i'm wondering: after we received a boss-scam mail that was showing the faked From in the webinterface, i read a bit about how SPF is checked, and apparently it is checked against the Return-path and not the From header. (This reddit was good summary https://www.reddit.com/r/sysadmin/comments/20rnt6/smtp_question_does_spf_only_validate_the/ )
Whats the benefit of this? As far as i can see, this renders the whole idea almost useless, as it doesnt prevent spammers from sending spam with faked From headers at all. What am i missing here?
(This is just because i'm wondering, i'am aware that DKIM + DMARC will solve this spam problem :) )
SPF validates the envelope sender (AKA SMTP MAIL FROM, Return-Path,Bounce Address henceforth sender). it's purpose is to deny the use of forged senders, by disallowing the sender to be used from unauthorized servers. stopping the generation of mail with forged senders (where SPF is supported)
BATV, (and other types of VERP) can be used to reject backscatter from those systems that do not check SPF and reject forged senders.
SRS (another type of VERP) is required if you do a mailing list - you can't retain the original sender because the list server will (most likely) not be included in the originators SPF
DKIM is the one that deals with email headers. it allows you to cryptographically sign selected email headers and full or partial content (but don't do partial signatures on MIME Multipart-Alternative messages - that will end badly)
Don't try to make SPF responsible for something it's not. SPF simply lists which servers can send mail for your domain. It checks the envelope sender (MAIL FROM) at the SMTP level, which is the value that ends up in the return-path header, but only after it's passed SPF checks. What you're saying is that (assuming you have a strict SPF policy) you're allowing someone to send fake mail from one of your own mail servers, which is a problem much further up the chain than the From header, and one that would not be solved by DKIM. Perhaps your SPF record is not strict enough? We can't tell from the information you posted.
I am creating own mail server. I am using Haraka (http://haraka.github.io/). But I am little confused about the relay thing. How to make relay my mail server so that I can send mail using other domain(DKIM and SPF verified).
I want mail in receiver inbox not in Spam. Right now mail is received in spam. What is relay in particular ? Can anyone help ?
What I've got is that you're having a problem with sending mail on behalf of "other domain".
Given that "other domain" mail reaches its destination (even in SPAM folder) I assume that you've configured your relay right.
Key thing to notice is that DKIM and SPF records not only need to be validated but also need to be aligned with your "other domain". It's a common scenario when SPF/DKIM validations 'pass' but overall DMARC policy 'fails'.
Providing both your message headers (to check how it was processed) and your other domain name (to check how SPF/DKIM/DMARC records configured) would help a lot.
I'm scoring 9/10 on mail-tester.com. My -1 comes from this,
"You do not have a DMARC record"
In my DNS (cPanel>"Advanced DNS Zone Editor") I have this DMARC record
_dmarc.mycooldomain.com. 14400 IN TXT "v=DMARC1; p=none; sp=none; ruf=mailto:myaddy#gmail.com; rf=afrf; pct=100; ri=86400"
my domain is really the correct domain in the actual DMARC record, and
myaddy#gmail.com is really the email for the cPanel/WHM account(a gmail addy), not the sender domain in the SPF record (e.g. info#mycooldomain.com). Does that matter?
otalliance.org/resources/spf-dmarc-tools-record-validator
Returns green, which I presume is good.
So is the issue with mail-tester.com, or my DMARC record?
Obviously, mycooldomain is not really your domain, so it's hard to verify what you posted, but based on what you posted, your RUF field will cause it to fail DMARC. If you send an email to mailtest#unlocktheinbox.com they have a really good DMARC tester, but unfortunately the DMARC results are not free. But I'm 100% sure that you're not following the standard on page 28 of the Dmarc Specification
Which reads
For example, if a DMARC policy query for "blue.example.com" contained
"rua=mailto:reports#red.example.net", the host extracted from the
latter ("red.example.net") does not match "blue.example.com", so this
procedure is enacted. A TXT query for
"blue.example.com._report._dmarc.red.example.net" is issued. If a
single reply comes back containing a tag of "v=DMARC1", then the
relationship between the two is confirmed. Moreover,
"red.example.net" has the opportunity to override the report
destination requested by "blue.example.com" if needed.
Since you're using a gmail account - there is no way your going to convince them to add a record on your behalf. So you need to choose a different RUF email address. Most likely one like dmarc#Mycooldomain then set up a forwarder to your gmail account if that's where you want the reports to go.
DNS changes are not available instantly. It can take hours until a new record will be visible to other servers. The DMARC entry you have posted seems to be valid except the "ruf=" email address. Here you must provide an email address assigned to your own domain.
Fix, wait and try again.
I am trying to help out a little non-profit organization, who has decided to let One.com host their domain, including website and e-mail. Now, my issue is that One.com does not add SPF-records or DKIM-keys to your domain and I believe that is the reason why a large number of mails sent from the domain, end up in spam.
I've been in touch with their support, who kindly answered:
You are already using our mail servers, there is no need to use SPF for that.
Our mail servers already have SPF installed, and if you are using our mail servers, SPF will not be question since domain is hosted here and it is using One.com's mail server. SPF will only be required if your domain is hosted here but is using a different MX record or mail server
I've tried to figure out if you can leave out SPF, but all I've been able to conclude is that proper SPF on each domain is definitely the proper way, instead of just the hosting companys main domain. I mean, if it was that simple, how come even Google Apps, Zoho, Rackspace etc. recommends adding SPF, if it worked just as well leaving it out - you'd be using their MX as well, so isn't that the same? And wouldn't leaving SPF out leave us with the same issues as before SPF, namedly that you'd have no way to validate if mail was truly being sent from the owners of the domain or just somebody imposing.
So what it comes down to: Can One.com really leave out SPF records on their clients domains, send mail on the clients behalfs and still expect mail to come through without ending up in spam more often?
Thank you very much for your time!
The short answer is "No, they can't". The longer answer is a little more complicated.
SPF uses either the EHLO domain of the sending server or the domain in the Return-Path to look up SPF records in DNS. Most systems that handle multiple domains do not use SPF records on the EHLO domains of the sending servers, so the SPF domain is taken from the email's Return-Path. You should take a look at the Return-Path for one of the emails that this non-profit has sent through One.com to determine whether the Return-Path is on a subdomain of one.com, or is using the non-profit's domain. The latter is definitely preferred.
If the Return-Path is on a subdomain of one.com, then that's the domain that will be used to look up SPF records. So adding SPF records to your non-profit's DNS won't do anything. While this may seem the easier path, it causes problems with DMARC and may cause the email to be flagged as spam even if it passes SPF, as the address in the 'From' header will have a domain that doesn't match the Return-Path
If the Return-Path is on a subdomain of your non-profit's domain, then you should definitely add an SPF record to your non-profit's DNS. Looking at one.com's current records, something like:
v=spf1 include:_spf.one.com ~all
should do it.
By the way, you should be able to see whether an email has been SPF or DKIM authorized by looking at the headers of the received email. That's the best way to understand the actual behavior.
When developing an application that sends out notification email messages, what are the best practices for
not getting flagged as a spammer by your hosting company. (Cover any of:)
best technique for not flooding a mail server
best mail server products, if you were to set up your own
sending messages as if from a specific user but still clearly from your application (to ensure complaints, etc come back to you) without breaking good email etiquette
any other lessons learned
not getting flagged as spam by the receiver's client? (Cover any of:)
configuring and using sender-id, domain-keys, SPF, reverse-dns, etc to make sure your emails are properly identified
best SMTP header techniques to avoid getting flagged as spam when sending emails for users (for example, using Sender and From headers together)
any other lessons learned
An additional requirement: this application would be sending a single message to a single recipient based upon an event. So, techniques for sending the same messages to multiple recipients will not apply.
best technique for not flooding a mail server
not a lot you can do about this beyond checking with your mail server admin (if it's a shared hosting account / not in your control). but if the requirement is one email to a single recipient per event, that shouldn't be too much of an issue. the things that tend to clog mail systems are emails with hundreds (or more) of recipients.
if you have events firing off all the time, perhaps consider consolidating them and having an email sent that summarizes them periodically.
sending messages as if from a specific user but still clearly from your application (to ensure complaints, etc come back to you) without breaking good email etiquette
you can accomplish this by using the "Reply-To" header, which will then have clients use that address instead of the From address when an email message is being composed.
you should also set the "Return-Path" header of any email, as email without this will often get filtered off.
ex.
From: me#me.com
Return-Path: me#me.com
Reply-To: auto#myapp.com
configuring and using sender-id, domain-keys, SPF, reverse-dns, etc to make sure your emails are properly identified
this is all highly dependent on how much ownership you have of your mail and DNS servers. spf/sender-id etc... are all DNS issues, so you would need to have access to DNS.
in your example this could present quite the problem. as you are setting mail to be from a specific user, that user would have to have SPF (for example) set in their DNS to allow your mail server as a valid sender. you can imagine how messy (if not outright impossible) this would get with a number of users with various domain names.
as for reverse DNS and the like, it really depends. most client ISP's, etc... will just check to see that reverse DNS is set. (ie, 1.2.3.4 resolves to host.here.domain.com, even if host.here.domain.com doesn't resolve back to 1.2.3.4). this is due to the amount of shared hosting out there (where mail servers will often report themselves as the client's domain name, and not the real mail server).
there are a few stringent networks that require matching reverse DNS, but this requires that you have control over the mail server if it doesn't match in the first place.
if you can be a bit more specific i may be able to provide a bit more advice, but generally, for people who need to send application mail, and don't have a pile of control over their environment, i'd suggest the following:
make sure to set a "Return-Path"
it's nice to add your app and abuse info as well in headers ie: "X-Mailer" and "X-Abuse-To" (these are custom headers, for informational purposes only really)
make sure reverse DNS is set for the IP address of your outgoing mail server
first a quick correction to the previous
return-path: is a header added by recieving system based on the envelope-sender of the incomming message
for spf to work the return-path/envelope-sender needs to be yourapp#yourdomain.com
and ensure the spf record for yourdomain.com {or if per-user spf} for yourapp#yourdomain.com allows mails to originate on the server that hosts the app/sends the email
this envelope-sender is the address that will recieve all bounces/errors
now sender-id is different entirely it checks the return-path/envelope-sender
and the
from: address {stored inside the message}
if sending
from: hisname yourapp#yourdomain.com
reply-to: hisname hisaddres#hisdomain.com
this will be a non-issue
if sending
from: hisname hisaddres#hisdomain.com
it will be and you must add a
Resent-From: hisname yourapp#yourdomain.com
as this specifies to ignore the from: for sender-id checks use this instead as it has been sent by you on his behalf
now for the other bits that are worthwhile
ip's mentioned are your mailservers
a have your ip's ptr point to a name that also resolves to the same ip
FQDNS
b have your server helo/ehlo with whatever.domain.com where domain.com is the same as the domain of the name in step A {not the same name for resons below}
c have that helo/ehlo servername also resolve to the ip of your server
d add the following spf record to that helo/ehlo name "v=spf1 a -all"
{meaning allow helo/ehlo with this name from ip's this name points to only}
e add the following sender-id lines to the helo/ehlo name {purely for completeness
"spf2.0/mfrom,pra -all" {ie there are no users#this-domain}
f add the following spf to the FQDNS-name and any other hostnames for your server
"v=spf1 -all" {ie no machines will ever helo/ehlo as this name and no users#this-domain}
{as the fqdns name can be determined by bots/infections its better to never allow this name to be used in helo/ehlo greetings directly it is enough that it be from the same domain as the helo/ehlo identity to prove the validity of both}