I am developing a web application.
All outgoing emails are marked as spam, or totally rejected by receivers' email providers.
This prevents us from sending welcome emails, activation codes, or anything else to registered users.
I tested my email sending with isnotspam.com, and the problem seems to be the BAYES_99 and BAYES_999 metrics. This happens regardless of email content.
Can someone tell me why this is happening?
Is there a way to make outgoing emails not marked as spam by spam filters?
Here the outcome of a isnotspam.com test result (I modified a few parameters for privacy ).
This message is an automatic response from isNOTspam's authentication verifier service. The service allows email senders to perform a simple check of various sender authentication mechanisms. It is provided free of charge, in the hope that it is useful to the email community. While it is not officially supported, we welcome any feedback you may have at .
Thank you for using isNOTspam.
The isNOTspam team
==========================================================
Summary of Results
==========================================================
SPF Check : pass
Sender-ID Check : pass
DomainKeys Check : neutral
DKIM Check : pass
SpamAssassin Check : ham (non-spam)
==========================================================
Details:
==========================================================
HELO hostname: my.hostname.com
Source IP: XX.YY.WW.ZZ
mail-from: service#mydomain.com
Anonymous To: ins-3mqqpdgw#isnotspam.com
---------------------------------------------------------
SPF check details:
----------------------------------------------------------
Result: pass
ID(s) verified: smtp.mail=service#mydomain.com
DNS record(s):
mydomain.com. 11379 IN TXT "v=spf1 +a +mx +ip4:XX.YY.WW.ZZ +ip4:AA.BB.CC.DD +include:_spf.google.com ~all"
----------------------------------------------------------
Sender-ID check details:
----------------------------------------------------------
Result: pass
ID(s) verified: smtp.mail=service#mydomain.com
DNS record(s):
mydomain.com. 11379 IN TXT "v=spf1 +a +mx +ip4:XX.YY.WW.ZZ +ip4:AA.BB.CC.DD +include:_spf.google.com ~all"
----------------------------------------------------------
DomainKeys check details:
----------------------------------------------------------
Result: neutral (message not signed)
ID(s) verified: header.From=service#mydomain.com
Selector=
domain=
DomainKeys DNS Record=
----------------------------------------------------------
DKIM check details:
----------------------------------------------------------
Result: pass
ID(s) verified: header.From=service#mydomain.com
Selector=default
domain=mydomain.com
DomainKeys DNS Record=default._domainkey.mydomain.com
----------------------------------------------------------
SpamAssassin check details:
----------------------------------------------------------
SpamAssassin 3.4.1 (2015-04-28)
Result: ham (non-spam) (03.7points, 10.0 required)
pts rule name description
---- ---------------------- -------------------------------
* 3.5 BAYES_99 BODY: Bayes spam probability is 99 to 100%
* [score: 1.0000]
* 0.0 URIBL_BLOCKED ADMINISTRATOR NOTICE: The query to URIBL was blocked.
* See http://wiki.apache.org/spamassassin/DnsBlocklists#dnsbl-block
* for more information.
* [URIs: mydomain.com]
* -0.0 SPF_HELO_PASS SPF: HELO matches SPF record
* -0.0 SPF_PASS SPF: sender matches SPF record
* 0.2 BAYES_999 BODY: Bayes spam probability is 99.9 to 100%
* [score: 1.0000]
* 0.1 HTML_MESSAGE BODY: HTML included in message
* -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from author's
* domain
* -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature
* 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily
* valid
X-Spam-Status: Yes, hits=3.7 required=-20.0 tests=BAYES_99,BAYES_999,
DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HTML_MESSAGE,SPF_HELO_PASS,SPF_PASS,
URIBL_BLOCKED autolearn=no autolearn_force=no version=3.4.1
X-Spam-Score: 3.7
I posted this question in total panic, our company was having trouble while launching a new product, customers were dissatisfied and we were struggling finding what the problem was.
In the end, our emails were not received not because of spam detection, but because of an error in our server configuration. There were conditions that caused emails to not be sent at all.
Key takeaway from this story is: if you have problems sending emails, verify that you are sending valid html (through validator.w3.org for example) and check your server configuration before thinking about spam.
Its statistical method to find a spam in spamassassin i'm not statistic man but you can find your response with the documentation official of spamassassin
and this reponse : How SpamAssassin defines BAYES_99 ? How to avoid it?
https://wiki.apache.org/spamassassin/Rules/BAYES_99
https://en.wikipedia.org/wiki/Naive_Bayes_spam_filtering
Related
I'm getting hundreds/thousands of emails on my catch all address for my domain, with the subject "Returned mail: see transcript for details".
I am trying to catch these emails with spam assassin (SA) to mark them as spam, but the SA rule only works if I email a test email to my domain with that text in the subject. The actual spam mails are not marked... only the once I send myself as a test.
I have tried (in local.cf):
header LOCAL_SUBJECT_RETURNED_MAIL Subject =~ m/Returned mail/i
score LOCAL_SUBJECT_RETURNED_MAIL 10.0
header LOCAL_SUBJECT_RETURNED Subject =~ /\bReturned\b/i
score LOCAL_SUBJECT_RETURNED 10.0
Again... the rules are working, yet it looks like the spam emails come in without being scanned by spam assassin.
Does anyone have any idea on what I could try?
Things I've tried/done
I enabled php mail logging on my system, to ensure it's not being send through a script on my domain. Log is empty, so that's good.
The reason why spamassassin is not catching these emails (backscatter), is because they don't come through spamassasin, because they come from the mailer deamon inside my domain/webserver.
Test mail I send myself:
Received: from localhost by server.myprovider.net
with SpamAssassin (version 3.2.4);
Backscatter spam:
Return-Path:
Received: from localhost (localhost)
As you can see, it doesn't come through spamassassin, hence, no rules are triggered...
The domain s****g.nl has the following DMARC record:
"v=DMARC1; p=reject; rua=mailto:postmaster#s****g.nl,
mailto:dmarc#s****g.nl"
A valid SPF record for the sending mail server and none DKIM record.
The domain fo***de.com has a valid SPF and DKIM record.
Example 1
From: from#s****g.nl
To: thetoadres#mail.com
Result:
SPF: PASS
DKIM: FAIL
DMARC: PASS
Example 2
From: from#s****g.nl
Sender: response#fo***de.com
To: thetoadres#mail.com
Result:
SPF: PASS
DKIM: PASS
DMARC: FAIL
So when I send the email using a sender (on behalf of) the DMARC fails and the mail is not delivered.
Is there a explanation for this and maybe a solution to send emails on behalf of a domain which contains a DMARC reject policy and have a valid SPF for the sending mailserver?
Edit:
[Screenshot results...][1]
I have a feeling, it's failing on your ADKIM and ASPF Tests of DMARC. If SPF and DKIM passes, then it must be failing on both alignment tests.
Read this to understand more about Identifier Alignments
I seen several cases where there DKIM Validator is coded wrong and it will fail DMARC when it fails 1 alignment test, but both must fail according to the RFC Standards.
The only alignment tester I know about is this email tester, if you post the full headers of the sent emails. It'll be much easier to understand what might be wrong. You're only sharing part of the information and it's impossible to make a 100% accurate assessment. But I'm 80% confident the issue is with the alignment.
Based on the image you linked of your headers, I added an "a" to the beginning and "1" to the end so bots don't spam you.
Return-Path = response#afo***de1.com
DKIM Signature = d=afo***de1.com
From = info#as**g1.nl
So for ADKIM alignment to Pass the "from" domain must match the "d=" domain of the dkim signature
info#as**g1.nl <> afo***de1.com
For the ASPF Alignment to pass the "return-path" domain must match the "from" domain
afo***de1.com <> as**g1.nl
One of those need to match in order for DMARC to pass.
So would username#gtld be a valid email? As a practical example google is purchasing the gTLD "gmail". Obviously they can associate A records with that permitting you to just type http://gmail/ to access the site. But, are there any specs that prohibit them from associating MX records with that as well, allowing folks to give out an alternative address username#gmail?
I ask because I want to make sure our email validator is future proof and technically correct.
I think I answered my own question. Section 3.4.1 of rfc5322 which defines a valid email address states:
addr-spec = local-part "#" domain
[...]
domain = dot-atom / domain-literal / obs-domain
[...]
The domain portion identifies the point to which the mail is delivered. In the dot-atom form, this is interpreted as an Internet domain name (either a host name or a mail exchanger name) as described in [RFC1034], [RFC1035], and [RFC1123]. In the domain-literal form, the domain is interpreted as the literal Internet address of the particular host.
"gmail" would be a valid domain and host name and thus someone#gtld is a valid email address.
When sending an email, the recipient list is given during the SMTP dialogue through RCTP TO: command. Later, in DATA command, header fields like 'To', 'Cc','bcc' are indicated. Does this RCPT TO list of recipients have to match with the headers indicated in DATA command?
Also, if the recipient is not indicated in RCPT TO, but in the To field of email header, is it going to be delivered to the recipient not in RCPT TO?
No, they don't have to match. When the message is sent, the SMTP Server (aka Message Transfer Agent or MTA) is creating a so called SMTP envelope which contains the recipients and the sender of the message (see RFC5321):
SMTP transports a mail object. A mail object contains an envelope and content. The SMTP envelope is sent as a series of SMTP protocol
units (described in Section 3). It consists of an originator
address (to which error reports should be directed), one or more
recipient addresses, and optional protocol extension material.
It is, actually, quite often that the RCPT TO: Command has more recipients that the header of the message - one common case is the usage of "blind copies" bcc: (see RFC5321):
Addresses that do not appear in the message header section may appear
in the RCPT commands to an SMTP server for a number of reasons. The
two most common involve the use of a mailing address as a "list
exploder" (a single address that resolves into multiple addresses) and
the appearance of "blind copies".
Does this RCPT TO list of recipients have to match with the headers
indicated in DATA command?
Nope.
if the recipient is not indicated in RCPT TO, but in the To field of
email header, is it going to be delivered to the recipient not in RCPT
TO ?
The RCPT. Here's a (modified) transcript from my own SMTP client where I do just what you ask:
CLIENT: MAIL FROM:<myaccount#gmail.com>
SERVER: 250 2.1.0 OK
CLIENT: RCPT TO:<myaccount#gmail.com>
SERVER: 250 2.1.5 OK
CLIENT: DATA
SERVER: 354 Go ahead
CLIENT: Subject: Test email
CLIENT: From:'John Doe'<fakeaccount#gmail.com>
CLIENT: To:'John Doe'<fakeaccount#gmail.com>
CLIENT: This is a test...
CLIENT: .
The message was successfully sent to "myaccount#gmail.com".
SMTP protocol (RFC 2821) states the following:
When RFC 822 format [7, 32] is being used, the mail data include the
memo header items such as Date, Subject, To, Cc, From. Server SMTP
systems SHOULD NOT reject messages based on perceived defects in the
RFC 822 or MIME [12] message header or message body.
And this:
The DATA command can fail at only two points in the protocol
exchange:
If there was no MAIL, or no RCPT, command, or all such commands
were rejected, the server MAY return a "command out of sequence"
(503) or "no valid recipients" (554) reply in response to the DATA
command. If one of those replies (or any other 5yz reply) is
received, the client MUST NOT send the message data; more
generally, message data MUST NOT be sent unless a 354 reply is
received.
From these statements, the headers and RCPT TO: command content does not have to match (altough they should match), and not using RCPT TO: MAY result in an error to prevent proceeding with DATA command.
I host a spread of different domains that all use my (one) mail-server to send and receive mail. When sending mails, sometimes, my mail gets rejected by the receiving end, marked to the recipient as "suspicious" or simply heads straight for the spam folder.
Also, on the inbound, I get a load of "return receipts" from random victims of spam, where one of my domain names has been used even though the mail never touched my mail server.
I have been told, that both issues stems from the fact, that my SPF record is not set properly which i have been attempting to fix for quite a while now. Unfortunately my basic knowledge of the mechanisms behind the record and the syntax itself escapes me somewhat, which is why I'm looking here for help.
For the purpose of the following example, assume the following setup:
I have two domains: mydomain.com and myotherdomain.com.
Both domains have active subdomains that send and receive mail through my mailserver.
My mail server is named mail.mydomain.com
All running on the same physical server with the IP address: 85.81.xxx.xxx.
I have a semi-static IP-address with my ISP, e.g. it never changes but is per say not mine to call my own. A whois on 85.81.xxx.xxx produces 0x39Axxxx.dslpool.isp.com
Using the tool found at http://tools.bevhost.com/spf/ i end up with the following conclusion:
Email Origin : Pass - 85.81.xx.xx
resolves to
0x39Axxxx.dslpool.isp.com which then
again resolves to 85.81.xx.xx.
Sender Details : Pass -
myname#myotherdomain.net points to a
MX-record that points to my mail sever
at mail.mydomain.net.
Host Name HELO / EHLO : Fail -
mail.mydomian.not resolves to
85.81.xxx.xxx which resolves to
0x39Axxxx.dslpool.isp.com
So, the question is: If at all possible, how would I compose the SPF entries for mydomain.com and myotherdomain.com to disregard this conflict and allow my sent mails to appear valid when spf validated by the receiver?
Hoping for a response ...
Here you should have this SPF entry in your DNS v=spf1 +ip4:85.81.xxx.xxx -all for all your domains, and nothing more in your SPF string.
Make sure that you have such a DNS entry for mail.maydomain.com as well as mydomain.com,
because the SPF entry for mydomain.com is not valid for subdomain.mydomain.com.
If you have many subdomains,you may consider to have an SPF entry for *.maydomain.com. That will take care of all the domain tree that are sub or sub.sub or sub.sub.sub etc. domains of the domain mydomain.com.