I configured exim mail server to send email from my web application (Example recovers passwords).
Tests done by mail is always considered as spam.(i use gmail web client).
How do I configure exim so that the emails they send are not considered spam?
Header with SPF
Delivered-To: spidercloudfl#gmail.com
Received: by 10.194.81.69 with SMTP id y5csp12894wjx;
Wed, 22 May 2013 10:34:21 -0700 (PDT)
X-Received: by 10.15.107.77 with SMTP id ca53mr20828718eeb.40.1369244060917;
Wed, 22 May 2013 10:34:20 -0700 (PDT)
Return-Path: <info#wizardsnc.it>
Received: from 95.110.179.201 ([95.110.179.201])
by mx.google.com with ESMTPS id u46si3055346eeg.66.2013.05.22.10.34.20
for <spidercloudfl#gmail.com>
(version=TLSv1 cipher=RC4-SHA bits=128/128);
Wed, 22 May 2013 10:34:20 -0700 (PDT)
Received-SPF: neutral (google.com: 95.110.179.201 is neither permitted nor denied by best guess record for domain of info#wizardsnc.it) client-ip=95.110.179.201;
Authentication-Results: mx.google.com;
spf=neutral (google.com: 95.110.179.201 is neither permitted nor denied by best guess record for domain of info#wizardsnc.it) smtp.mail=info#wizardsnc.it
Received: (qmail 25578 invoked from network); 22 May 2013 18:33:20 +0100
Received: from localhost (HELO 95.110.179.201) (127.0.0.1)
by localhost with ESMTPA; 22 May 2013 18:33:20 +0100
Date: Wed, 22 May 2013 18:33:20 +0100 (BST)
From: "Wizard s.n.c" <noreplay#wizardsnc.it>
To: "spidercloudfl#gmail.com" <spidercloudfl#gmail.com>
E-mail continue to be considered as spam :(.
thanks to all
Modern large webmail providers use a reputation system to score and determine placement of inbound emails. Your sending IP and domain name builds a reputation How those reputation systems work exactly is not something that we on the outside will know. But we can describe behaviors that seem to influence a sender's reputation.
If you have never sent an email from your IP, it has no reputation. Unfortunately for you, no reputation is essentially in the border of a bad reputation.
If your IP has ever sent spam that one or more Gmail users submitted as spam, it contributes negatively to your reputation.
If an IP from your /24 (or maybe from the /n BGP announcement containing your IP), it contributes negatively to your reputation. It is unknown if this is the same negativity as item #2.
If your domain does not have an SPF record, it contributes very negatively to your reputation. If it does, and it does not match, it contributes negatively. If it does have it and it does match, it contributes positively.
If your domain does not use DKIM to sign the emails, it contributes negatively to your reputation. If your domain does and the signature fails, it contributes negatively. If it does use DKIM and the signature passes, it contributes positively.
If your domain uses DMARC and it aligns, then it contributes positively to your reputation. I think it contributes LARGELY positive.
If your emails are being marked as spam due to detected URLs in various URIBL's, it contributes negatively to your reputation.
If your IP's mail volume changes drastically upward, it contributes negatively to your reputation. It tends to be changes of scale. So 0 to 5 may not be that big a deal, but 0 to 10 may be signs of abuse. Same process for 100 to 500 ok, but 100 to 1000 is signs of abuse.
If your IP's mail volume is consistently the same, it will contribute positively to your reputation.
If your IP is listed in any major RBL, it contributes negatively to your reputation.
If your emails have invalid or missing required headers, it can contribute negatively to your reputation.
If your emails don't fall afoul of any of these things, it can contribute positively to your reputation.
If your emails are marked as Not Spam by users who receive it, it will contribute positively to your reputation.
Basically there are lots of things that can be wrong that contribute negatively, and just a few things that work for positive reputation. There are more than these, but these are all I could think of off the top of my head.
Todd's answer is good, and when he's talking about reputation, he's hitting the nail on the head.
Generally, not having SPF/DKIM/DMARC records has very little impact to your IP or domains reputation, and by extension, very little or no impact upon a particular messages deliverability. Take a look at the most widely deployed mail filters (SpamAssassin, Amavis) and you'll find that having matching SPF records provides no score increase. The same goes for DKIM. The reason for this is simple, it's very easy for spammers to publish SPF records and by harnessing the power of bot nets, the CPU cost of DKIM signing messages is negligible.
Many spammers use throw-away domains (no reputation) they acquire for this very purpose, generally in TLDs that permit domain abuse like .info, .pw, .tw, and .biz. Just like Todd's points #2, and #3, if you happen to have a domain that is surrounded by spammers, it's just as bad as having an IP address or ASN with spammy neighbors.
SPF, DKIM, and DMARC are all authentication mechanisms. They serve only to validate that the domain in question is responsible for a particular message. The reason a domain owner with a good reputation should publish a SPF record and DKIM sign their messages is to prevent spammers from sending emails from their domain, and thereby sullying their good reputation.
Failing SPF, DKIM, and DMARC will definitely earn you negative points, in varying quantities.
I'm on a couple DMARC mailing lists and I run one of about 20 mail systems on the internet that do DMARC reporting. I don't know of anyone handing out extra ham points for DMARC aligned messages. That isn't to say DMARC is not helpful. If your domain has a good reputation, and you publish DMARC records and a particular message is aligned, then your good reputation extends to that message. If the message is not aligned, then we have to decide if the message really is from you, and how much of your domains reputation should be afforded it.
Another thing that's vitally important for a mail servers reputation is having properly configured DNS. Make absolutely certain your mail server's IP has matching forward and reverse DNS (FCrDNS), and that the hostname of the server is configured correctly, and that Exim is using the machines hostname.
Related
I'd like to set up custom domain authentication using DKIM and SPF for our 3rd party email marketing company (like mail chimp or constant contact). We also run MS exchange. Our Exchange guy is convinced that setting up DKIM and SPF for email marketing company will forever tie the reputation of the email marketing company to our exchange server. Is he correct? If not, how do I convince him?
I think I have enough info now to make this an answer...
Yes, if this is a permission-based list that you have sent to recently (if it's old that means likely spam traps) then I think you are correct that there's not much risk at all.
One way to convince this person would be to find out what IP address your MailChimp emails originate from (maybe send to a small list with just yourself on it but a real send). And then check out the reputation of this IP address using the tools available such as MX Toolbox and others, then show him the output. I'd be surprised if your Mailchimp assigned IP address was on any blacklists or had reputation issues
When he says exchange server is he talking about your company domain name taking a reputation hit? Or is he worried about the IP address from which you send non-marketing email? If he's worried about a separate IP that you send day-to-day email from then explain to him that your marketing emails will go out from a Mailchimp assigned IP address. If he's worried about the domain two things: 1. Your list is opt-in and you've sent recently so it's not an issue 2. If it was a bad list that would cause your domain to be blacklisted then whether you have DMARC, SPF, and DKIM doesn't matter, the originating IP that sends spam can get blocked for spamming regardless.
So I think you are right here but it's a matter of making the case.
I use dnsimple to host my DNS and have valid SPF, DKIM, and DMARC records to validate my emails sent from Zoho. However, Whenever I send emails to an #ucdavis.edu account I get an Undelivered Mail response
This message was created automatically by mail delivery software.
A message that you sent could not be delivered to one or more of its recipients. This is a permanent error.
lsmiyashita#ucdavis.edu, ERROR_CODE :550, ERROR_CODE :5.7.1 <admin#study.space>... Access denied
Received:from mail.zoho.com by mx.zohomail.com
with SMTP id 1478675695600485.6815385213283; Tue, 8 Nov 2016 23:14:55 -0800 (PST)
Message-ID:<15847f087ed.112901d8e106580.9166398699723335101#study.space>
Date:Tue, 08 Nov 2016 23:14:55 -0800
From:Jacob Bevilacqua <admin#study.space>
User-Agent:Zoho Mail
To:"lsmiyashita" <lsmiyashita#ucdavis.edu>
Subject:Here's a little test for you.
Content-Type:multipart/alternative;
boundary="----=_Part_335760_1020694757.1478675695597"
I have tried several different hosts (GSuite, MailGun, & Zoho) and I get the same issue. I checked and I am not blacklisted on any sites. I ran a test at mail-tester.com and got a 10/10. Why won't my messages deliver.
I verified that the email address you are sending to is valid: Verified Email Address
So like #Synchro says, they just don't like you. It's always a challenge to figure out the exact reason, but contacting their admins is the right way to go. I have a feeling it's because of the .space domain ending, they probably haven't updated the list of domain endings they accept.
Anyway, if you wanted to do additional mail testing, use this Mail Tester.
You are under the unfortunate illusion that it's your fault. A 5.7.1 error means that they just don't like you, and they don't have to give a reason. Welcome to the world of deliverability, or lack thereof. Well-behaved mailers are often punished for no particular reason. If it's just this domain, your best bet might be to contact their admins.
I host email for a number of different domains - lets call one of them myuser#receiver.com. Lets also pretend my domain is called myserver.com. I receive and store email in a mailbox locally for these domain on mx1.myserver.com. I am having an odd sporadic email issue which I feel must stem from a misconfiguration or setup on my part. This problem only occurs when a 3rd party acts as a forwarder (intermediary), receiving the email for myuser#receiver.com but sending it on to myuser#www.myserver.com. The exerts below are directly from the email message source.
An email is sent by someone to one of my clients. Its received by an intermediary.
Received: from email.sender.com (the.sender.com [123.123.123.123])
by the.forwarder.com (8.14.5+Sun/8.14.5) with SMTP id x12XXXxX123456
for <myuser#receiver.com>; Thu, 5 Mar 2015 13:23:22 GMT
That intermediary then forwards the email to my server, but for some odd reason readdresses it.
Received: from the.forwarder.com (the.forwarder.com [234.234.234.234])
by mx1.myserver.com (smtpd) with ESMTPS id 1234X123X1
for <myuser#www.myserver.com>; Thu, 5 Mar 2015 08:27:11 -0500 (EST)
Why is the.forwarder.com (intermediary) rewriting the domain of the recipient on this message?
Some outgoing SMTP servers are known to re-write the headers on outgoing messages, and change the sender's address in From: line to the email address of the user that is authenticating. Gmail is notorious for doing this. See How to change reply-to and return-path header with gmail smtp in django for more info.
In the end this was caused by CNAME records in DNS. There were CNAME records that were present mainly for virtual hosting of web services. This would be something like CNAME record of receiver.com points to www.myserver.com (since they share the same IP address). This had the effect of making secondary mail delivery servers address mail to www.myserver.com instead of the intended receiver.com. To fix this I removed the CNAME record and created an A record to the actual IP address.
I recently moved my transactional email sending to Mailgun
It works good so far however I am wondering about the return-path header.
Consider this email (I removed irrelevant header and replaced email/domain for privacy purposes)
Delivered-To: RECIEVER#gmail.com
Received: by 10.76.154.104 with SMTP id vn8csp478308oab;
Wed, 4 Sep 2013 05:04:44 -0700 (PDT)
X-Received: by 10.50.22.105 with SMTP id c9mr1537992igf.36.1378296283817;
Wed, 04 Sep 2013 05:04:43 -0700 (PDT)
Return-Path: <bounce+a801a1.c2b37-RECIEVER=gmail.com#my-website.com>
Received: from so254-63.mailgun.net (so254-63.mailgun.net. [198.61.254.63])
by mx.google.com with ESMTP id k5si1620852igx.55.1969.12.31.16.00.00;
Wed, 04 Sep 2013 05:04:43 -0700 (PDT)
Received-SPF: ...stripped...
Authentication-Results: ...stripped...
DKIM-Signature: ...stripped...
DomainKey-Signature: ...stripped...
Received: by luna.mailgun.net with HTTP; Wed, 04 Sep 2013 12:04:42 +0000
Mime-Version: 1.0
Content-Type: text/plain; charset="utf-8"
Subject: ...stripped...
From: my-website <support#my-website.com>
To: RECIEVER#gmail.com
Message-Id: <20130904120442.1488.88532#my-website.com>
X-Mailgun-Sid: WyI5YmI1OSIsICJqb2Vob3BmK2VlZ2VpN2lkMm9pbW9vYm9vZmFpQGdtYWlsLmNvbSIsICJjMmIzNyJd
Date: Wed, 04 Sep 2013 12:04:43 +0000
Sender: support#my-website.com
Content-Transfer-Encoding: base64
...email body...
This is the Raw email displayed from an actual mail in a gmail inbox.
As you can see the Return-Path header contains an email address that ends in #my-website.com
But I have only set up dns records for outgoing email (spf, domainkey, etc).
Not for incoming email. Meaning, my MX records still point to mailservers somewhere else (In my case google apps).
How is it possible then that the bounce email arrives at mailgun servers?
I would have expected to see an email address ending in #some-mailgun-server.com in the Return-Path header!
I have been using Amazon SES before, and there they had Return-Path header ending in amazonses.com
I asked the mailgun support and got this response:
Nick: your setup is correct, Mailgun will still automatically handle
the bounces even though your mx records are pointing elsewhere
They just assured me that everything was fine but gave me no explanation (which is okay since their job is not to teach me things I don't know but to deliver reliable email service...)
So I hope somebody can explain this to me.
I hope the point is clear, if not please ask and I will try to clarify my question.
EDIT:
One theory of me is that the bounce email is indeed sent to google mail servers where it is discarded. However that this is redundant since the error response is also sent to the sending mailserver during the process (when it opens its tcp connection to the target mail server).
To test this theory and since the Return-Path email is in the form of bounce+SOMETHING#my-website.com, and google delivers all email, regardless of what comes after the + character, to the user in front of it, I went ahead and created the account bounce#my-domain.com on google apps.
I also tried to send an email to bounce+a801a1.c2b37-RECIEVER=gmail.com#my-website.com.
It made it through to my inbox.
Now I expected to receive bounce traffic in my inbox. So I sent an email to an nonexistent hotmail address. I did not receive email on my google apps inbox, and mailgun successfully tracked the bounce.
So... It appears that it does indeed work. I just don't understand why.
One more theory I have is that the mailserver to which the bounce email is delivered is never resolved using its MX records. Rather always the delivering server, in this case luna.mailgun.net is chosen.
The domain ending in the Return-Path address is just the name of the mailbox on the server, but the domain has nothing to do with the server where the mail is actually delivered.
Then it would also make sense to make it like this since it might improve deliverability if the domains of From and Return-Path address match.
However this is only a theory. And it would also mean that a mailbox which is able to receive bounces, must be on the same server that is used for sending.
In other words it would be impossible to have a mailbox to receive bounce email addresses that is hosted somewhere else than the actual server sending the mail. But this sounds strange to me as well...
I hope somebody can enlighten me.
Turns out that there are different kinds of bounces.
When bounces occur they are generally returned to the server that is sending the email, and do not follow the MX records.
Thats why they are sent to the mailgun servers and also arrive there.
However there are also so called "Delayed Bounces" that are sent to the server declared as mailserver using MX records in the domain.
Those delayed bounces are generally difficult to handle and there are opinions out there that they violate RFC.
Those bounces are however very, very rare. Thats why mailgun does not handle them. The reason they use the clients domain in the return-path address is so that they can assign it to the right account. They just encode it that way...
In fact, as I was setting up my mailbox for bounces on my google apps mail, I recieved one such delayed bounce.
It was this email that made proper debugging possible which lead to the understanding of this issue.
So to sum up:
Yes, the address is incorrect. That is no problem for most bounces since the server does not use MX records to send them, but sends them directly to the server that has initated the conneciton.
However in case of delayed bounces, that also some times happen, the bounce will indeed go to the server behind the mx records of the domain specified in the return path address.
Those emails are not properly recognised as bounces at mailgun servers.
Since it was released I've been using Google Apps FYD for stackednotion.com. All of the email I send goes through Google's servers and I use Gmail to view my email. I haven't had any issues before, however recently I've been seeing weird bouncebacks ending up in the catch all account. It looks like somebody is using my domain to send spam. I don't really want my domain getting marked with a bad reputation, so how can I stop this?
I have setup SPF, DMARC and DKIM on the domain by following the guides on Google Apps, here is my zone file:
; stackednotion.com [9548]
$TTL 86400
# IN SOA ns1.linode.com. luca.stackednotion.com. 2012072633 7200 7200 1209600 86400
# NS ns1.linode.com.
# NS ns2.linode.com.
# NS ns3.linode.com.
# NS ns4.linode.com.
# NS ns5.linode.com.
# MX 1 ASPMX.L.GOOGLE.COM.
# MX 5 ALT1.ASPMX.L.GOOGLE.COM.
# MX 5 ALT2.ASPMX.L.GOOGLE.COM.
# MX 10 ASPMX2.GOOGLEMAIL.COM.
# MX 10 ASPMX3.GOOGLEMAIL.COM.
# MX 30 ASPMX4.GOOGLEMAIL.COM.
# MX 30 ASPMX5.GOOGLEMAIL.COM.
# TXT "v=spf1 include:_spf.google.com ~all"
google._domainkey TXT "v=DKIM1; k=rsa; p=MIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQDi19ipSdqDEpnJEWrVF7MarSLnlzXi0wPOHws2BY6oMQInbY5OHzdw9LcFr1biVvipErm4odyJfjZAIp5s8r6z50ZxQdW5Uwdy9krA1A9HMPaqVN+fm2xpntU//uXn0wD8sGc9CljYQIl+MusxQ690PfVGnAz/QeLqaZFxpHHmmQIDAQAB"
_dmarc TXT "v=DMARC1; p=quarantine; rua=mailto:dmarc#stackednotion.com"
# A 178.79.164.64
* A 178.79.164.64
_xmpp-server._tcp SRV 5 0 5269 xmpp-server.l.google.com.
_xmpp-server._tcp SRV 20 0 5269 alt1.xmpp-server.l.google.com.
Also here are the headers of a spam message (somebody tried to susbscribe me to a Zend mailing list, what kind of sick people are they?!?):
Return-Path: <F776387#stackednotion.com>
Received: (qmail 20117 invoked from network); 27 Jul 2012 06:51:01 -0000
Received: from exprod7mx200.postini.com (HELO psmtp.com) (64.18.2.92)
by rsmx2.zend.com with SMTP; 27 Jul 2012 06:51:01 -0000
Received: from source ([188.51.41.223]) by exprod7mx200.postini.com ([64.18.6.13]) with SMTP;
Fri, 27 Jul 2012 02:51:00 EDT
To: <fw-docs-subscribe#lists.zend.com>
Subject: Invoice #48469883494
From: "Order" <F776387#stackednotion.com>
Date: Sat, 28 Jul 2012 09:40:03 +0300
X-Priority: 3
X-MSMail-Priority: Normal
X-Mailer: IPS PHP Mailer
MIME-Version: 1.0
Content-type: text/plain; charset="iso-8859-1"
Content-Transfer-Encoding: 8bit
Message-ID: <20120728094003.9312B884F9D66F02CE7C#DELL-PC>
X-pstn-neptune: 500/484/0.97/100
X-pstn-levels: (S: 0.00346/89.11253 CV:99.9000 FC:95.5390 LC:95.5390 R:95.9108 P:95.9108 M:97.0282 C:98.6951 )
X-pstn-dkim: 0 skipp
At present, the way to reduce miscreants ability to send spam purportedly from your domain is to inform other mail servers what servers are allowed to send mail on your domains behalf. The mechanism is SPF and you already have a SPF record:
TXT "v=spf1 include:_spf.google.com ~all"
If blocking forgery attempts is your desire, this can be improved upon. Read the SPF Record Syntax page that describes what your SPF policy should be. If you have other mail servers sending mail on behalf of your domain, add them to the SPF record and change your policy to fail:
TXT "v=spf1 include:_spf.google.com -all"
Because SPF is so widely deployed, this will make a difference. But SPF has edge cases (forwards, email lists, etc.) where SPF policy fails, so most sites choose to be more liberal with SPF policy than you request. For example, if your policy is set to reject, and the message appears to be from an email list, most servers notate it somehow (the Authentication-Results header is defined for this purpose) and allow it to pass.
This is where DMARC comes in. You have already added a DMARC record:
_dmarc TXT "v=DMARC1; p=quarantine; rua=mailto:dmarc#stackednotion.com"
Your policy is only to quarantine failing DMARC messages. If the DMARC reports do not indicate any valid messages being blocked, and/or you are willing to live with some edge cases where valid messages are rejected, then you can improve upon this with p=reject.
Not surprisingly, getting bounces from mail servers for spam purporting to be from one of my domains is exactly what compelled me to start DKIM signing my messages, so that I could deploy DMARC. DMARC is a policy mechanism that combines SPF and DKIM, so that domain owners can assert to other mail servers that, "If it's not from this list of IPs (SPF) and it's not DKIM signed, then [reject|quarantine|allow] it."
DMARC works brilliantly. Instead of getting bounce messages, now I get DMARC reports. I use Mail::DMARC to parse the reports and put the summaries into a database.
DMARC is still an IETF draft and it's not widely deployed. However, most large email providers have implemented it and coverage is surprisingly good. After deploying DMARC for my domain, I wrote a DMARC plugin for Qpsmtpd, so I could validate incoming messages against DMARC policy. I published some of my findings as a DMARC operator in a DMARC FAQ.
I mentioned edge cases earlier, so I feel compelled to share one.
Google handles misaligned messages (those that fail both SPF and DKIM alignment) by dropping them into the users Spam folder. I have become familiar with this because emails sent from my domain are generally treated well by gmail. The exception is for messages relayed through some email lists such as dmarc-discuss#dmarc.org. Messages I sent to that list get modified by the list processing software, invalidating my DKIM signature. When the message is forwarded to gmail recipients, those messages are marked as spam because a) I have published a reject DMARC policy, and b) that email list isn't a valid SPF sender of email from tnpi.net, and c) the DKIM signature bearing my domain fails validation.
There are workarounds, besides fixing the list software, such as adding the offending mailing list server to my SPF record. Some DMARC implementations will detect messages from mailing lists and reduce the policy severity (ie, Google quarantines my list messages rather than rejecting).
At present, there is no better way to inhibit phishing and spoofing attempts using your domain than a well implemented DMARC policy.
I've noticed an increase in this kind of spoofing in the last few weeks as well.
The Google support page on this issue notes:
"Because these messages originate outside of Gmail, we aren't able to stop spammers from spoofing your address. However, Google helps protect your Gmail address's reputation by designing our systems to authenticate all the mail that really comes from you. When another domain receives an unauthenticated message from Gmail, it can tell that you didn't really send the mail, and it is unlikely that your email address will be blocked. For our part, we are concerned about spoofing and bouncebacks. We ask you to report these messages by checking the box next to the unwanted message and clicking Report Spam at the top of your inbox, or by opening the message and clicking Report Spam at the top of the message."
"You can help stop spammers by also sending the full headers of these unlawful messages to the Federal Trade Commission at spam#uce.gov."
Afaik, it's supposed that DMARC will help you achieve that. According to the Google Apps Article about DMARC, you should start using a "conservative deployment cycle" like:
Monitor all.
Quarantine 1%.
Quarantine 5%.
Quarantine 10%.
Quarantine 25%.
Quarantine 50%.
Quarantine all.
Reject 1%.
Reject 5%.
Reject 10%.
Reject 25%.
Reject 50%.
Reject all.
So, my suggestion would be stop quaranting emails, monitor for a while and then start going up.
There could be possibility that your mails are getting bounce back. So if you are receiving mails on the same account from which you send. Try changing the authority of admin of domain control in your database.