I have a few separate sites on a server with a single IP.
The sites shouldn't ever send spam, but the customers are free to send emails from their sites so I have no way to prevent them from doing so.
What I'd like to do is when sending the emails via postfix, somehow separate the sites in the headers sent out.
Previously i've setup an ip for each but i'm trying to avoid doing this.
I've also found with /etc/postfix/header_checks I can remove headers but not sure if removing specific headers will cause issues?
One thing to consider here is that blacklisting is usually based on IP addresses. Separate headers won't help much there. The reason for this is that (a) it's simple and (b) many spam sending servers have been compromised and taken over by an attacker, using custom mail sending software, so headers don't matter anymore.
Different headers might still have their merit though, as spamfilters will check those. It just won't help if your server's IP gets blacklisted.
I guess rolling out DKIM might help here, it would give you artificial separation of domains using different domain keys for each. There are some good tutorials on the net on how to set it up with OpenDKIM.
A better solution, used by big mail providers like GMX, is to send mail from a separate IP if it looks like its spam. The setup for this is a little complicated, as it requires you to scan outgoing mail with spamassassin (or something similar) and to route mail depending on the respective spam value. Not an easy task. Marking spam as such, without sending it through a separate IP, might enough to convince the other side that you try to prevent spam send from your server, but this really depends on their spam filter.
The way your server identifies itself during an SMTP conversation is through the HELO command. The smtp_helo_name parameters specifies the name used there. One could try to setup a transport mechanism to use a different name for each sender domain. I'm honestly mot sure how difficult that would be.
If you are still set on changing headers: the header_checks tables not only allow to remove headers, but also to alter them via regular expressions.
Use the REPLACE command to do so. Example:
/^(Message-ID:.*)#your-domain.example(.*)/ REPLACE ${1}#other-domain.example${2}
I'd advise against it, though. It provides to little gain for the effort of finding and setting up the right rules.
Related
I am using Mail::IMAPClient to connect to an IMAP server. Can someone advise on how to get the priority of the messages? I tried dumping the headers, but I cannot see where the importance is set.
That may depend on the client the sender used.
In my eyes, the most important importance setting is the header
Precedence: bulk
which indicates messages like delivery errors for which the sender doesn't even expect a failure notice, the mail system is simply free to drop these silently if any problems occur. (A good idea to set for outbound error messages, to avoid loops.)
Then, there is a header field called Importance which I'm not sure any mail client uses; RfC 4021 defines Priority, but it doesn't yet seem to be that common in the wild (hey, it's only been out for eight years!) and then there is the non-standard X-Priority, which may be what you are looking for.
Note that all of these are not part of the IMAP layer, but the message headers themselves. Depending on the IMAP server, IMAP flags may or may not have been set based on them, but to be sure you see what the sender thought about the mail (or their own ego), you may need to get the actual mail headers and look there.
That being said, most people I know simply ignore these values/appraisals, anyway.
I'm looking to start using an SMTP or mail relay service. I've found quite a few out there, but I'm not sure if there are advantages to one vs another. The only requirements I have are:
can send "from" more than 1 domain (possibly >20 for all the different sites I work on)
can pay for a higher limit (I may need to send as many as 15000 in 1 day, although the average is <500)
can send from PHP (although I doubt this will be a problem as most are compatible with any language)
I'm okay with an SMTP service, mail relay service or a site that uses a custom API, although an API would make the conversion more difficult.
Reasons for wanting to do this:
I don't want to host any mail services my self as they just cause head aches
I don't have to worry about being blacklisted. If they are blacklisted they will know about it and have the knowledge to get it fixed.
Reporting on if emails have gone through would be nice.
I'm not sure why you would need this. If you read the proper RFCs (822, 2822, 823, 2823), you should be able to connect to any given site directly using SMTP. You need to be a little careful with Line Endings (should always be CRLF), and should probably add mail.add_x_header = OFF to your php.ini.
However, if you need a relay, I recommend using a spam filtering provider, as then you have protection from being blacklisted due to spammers abusing email-generating forms. I would recommend Red Condor for this task, but that is only because I work there, and know that we can handle it.
I've started using Mandrill and found it to be a great, reliable service provided by MailChimp that includes enough for most sites to use for free. Easy to setup, but also has a lot more functionality available.
What are the risks, if any, of sending out massive amounts of emails over SMTP? Specifically, this question is regarding the risks of being labelled/blacklisted as spammers of spoofers.
Our mails are legitimate, however. Our system needs to send out reminders to our corporate users on a daily basis, which may number into the thousands, say. Our worry is that with such a setup, our domain might end up being blacklisted by the receiving organisation, thus rendering our reminder service useless.
Does anyone have any information on what might be a "safe" volume of emails to send out to avoid being blacklisted? Or can we just churn out emails with abandon?
You may be able to contract a third-party organization to take care of this for you. I know there's a lot of "direct marketing" companies that will let you use their API to send mass email (newsletters, etc). They can do the work of negotiating to get off blacklists - that's what you pay them for.
I haven't used Sendloop and don't know if it has the functionality you want, but it's probably a good example.
See: How to conduct legitimate email campaigns
In your reminder service, just follow some basic spam guidelines. Identify where the message came from, why the user got it, the link to "opt-out" or discontinue the reminders, and you'll be fine. Any blacklists you do get on will certainly remove you if you have this information in your messages.
Additionally, should you get blacklisted for some reason, have another server on a different network that you can use as a backup should your primary server get blacklisted temporarily for any reason.
Oh, and one final note - usually your entire "domain" (i.e. whatever.com) doesn't get blacklisted. Specific IP addresses or specific servers are usually what get blacklisted.
As long as you're mailing over clean IPs and domains you should be fine. You say your mailings are "legitimate" so there's no reason to worry about ISPs blocking you.
However, as you also mentioned, the volume can become a challenge. Broadly speaking, sending "thousands" of messages should be a non-issue. But... hundreds of thousands, say 250K messages a day on up, is when you start to qualify as a "high-volume" sender.
Once you start sending at this bulk level, you must run a tight ship. ISP filters will look for any clue that you're a black-hat mailer/spammer and will promptly block your deployment if anything looks off.
Make sure your list(s) are spic-and-span; all bounces, duplicates, typos and honey-pots have been scrubbed-out. Your IPs have been properly warmed-up, your DNS and domains are clean and properly registered and you remain responsive to your list recipients.
Basic common sense and following through on all the tiny, simple but crucial details goes a long way.
We run a large online community in the Netherlands. Because of that we send a lot of mail to the hotmail email addresses of our members.
Recently we have noticed that not all mail is reaching our members, because we have hit a certain limit or so it seems.
Google doesn't give a solution (yet) but we see a lot of others having the same problem.
Note, We have added (since long) SPF records for our domain, in TXT and SPF types.
What else can we do to tackle this problem?
// Ryan
To add to what bzlm said, hotmail probably isn't rejecting your mails, so much as trying to use rate-limiting to prevent spam. That said, there are a few potential solutions you could use here. You could contact hotmail and see about getting your mailserver exemption status from their rate-limiting. Depending on the size of your community, they may or may not respond to you or be willing to work with you. I suggest this only as the "diplomatic" solution.
Or, you could set up two mail servers; one for hotmail users, and one for everybody else. I know from some of the sites that I run that a lot of people register with hotmail accounts, mostly because everybody has one which they use as a "spam dump" for online services where they don't want to use their real email address. So, as you no doubt realize, the number of hotmail users in your database represents a fairly substantial percentage. Therefore, when you need to send an email, you could determine whether to send it to your normal SMTP server, or your hotmail-designated one. On the hotmail SMTP service, you'd need to add some type of waiting mechanism to sleep a certain amount of time after receiving a 421 response.
The problem with this idea is that the number of hotmail users you have, plus the delay you'll encounter in sending, means that the queue length might very well exceed the number of mails you must send. You could alleviate this problem by setting up secondary/tertiary servers, preferably on other networks... but I'm getting ahead of myself here. At any rate, I did a bit of googling around (as you probably have, too), and this isn't such an uncommon problem, but there is no obvious solution to it.
So likely, you'll either have to create some type of slightly-unorthodox network workaround, or try the "diplomatic" route and contact an organization unlikely to care about your problem. I'd suggest doing both in parallel. :)
421 means that the service is not currently available, and that the client should try again. This could be for any number of reasons, including trying to discourage you from too frequent mailing if Hotmail thinks you might be a spambot.
Why not simply let your outbound smtpd queue the mails and try again?
Why is "not all mail reaching your members"? Don't you try again if you get a 421 response?
EDIT: Do what sqook says.
The only real way to "get around" this is to become a good e-mail citizen. Make it easy for people to unsubscribe from your notifications, establish complaint feedback loops with the major mail providers, remove bouncing e-mails from your list automatically, don't send people e-mails they don't want to receive. Failing to adhere to these simple requirements makes you look like a spammer, and providers like Hotmail will treat you like one.
The mail server IP connecting to Outlook.com server has exceeded the rate limit allowed. Reason for rate limitation is related to IP/domain reputation. If you are not an email/network admin please contact your Email/Internet Service Provider for help.
https://mail.live.com/mail/troubleshooting.aspx
I advise you too wait some times
I'm implementing a email newsletter sender service using .NET and Windows Server technologies. Are there comprehensive guidelines which could help avoiding emails being trapped by spam filters and other mechanisms?
They should cover all aspects of (legal) bulk mail sending: SMTP configuration, DNS, HTML content, images, links within content etc. A simple example: is it better to embed images or load them from a server?
It would be great if you could provide some empirical data to show the efficiency of some measures taken.
Although I don't have a definitive answer, I think this is a very important question.
Here are few tidbits I know about it
Choose a clean hosting/smtp server. IP addresses of spamming SMTP servers are often black-listed by other ISPs.
Send a simple introductory email to every subscriber, asking them to add your sender address to their safe list.
Be very prudent in sending to only those people who are actually expecting it. You wouldn't want pattern recognizers of spam filters learning the smell of your content.
If you don't know your smtp servers in advance, its a good practice to provide configuration options in your application for controlling batch sizes and delay between batches. Some servers don't like large batches or continuous activity.
Unless you have a very specific reason to host the newsletter yourself, I think you'd be much better off using a third party service. There are lots out there, and some are very cheaply priced.
It'll save you on development work
(no point in re-inventing the
wheel).
Their system will handle all
the unsubscribe link stuff that you
need to include in email newsletters
to comply with CAN SPAM laws or
whatever.
They handle the spam
reports that you will inevitably get
if you have a list of any non-trivial size.
They keep records of who signed up,
how they signed up, and their IP
address, and can present those on
receipt of a spam report to prove
that their service wasn't sending
out spam.
You can use double-opt in
(or confirmed opt in), for extra
evidence to prove that the people
you're sending emails to actually
signed up to receive them.
If you really do need to host it yourself I'd suggest you search the web for "email deliverability". Things that are known to help include properly set up SPF records, DomainKeys/DKIM, correct DNS settings (reverse DNS especially - best to just use an online service to check your DNS settings). You can test a lot of these things by sending an email to check-auth#verifier.port25.com.
It's best to avoid using spammy words in your email - always a bit of guesswork this but you some words can trip filters.
But I'd guess that by far the most important thing is to be sending your email from a trusted server that maintains good relationships with ISPs (i.e. ensuring that ISPs don't think that the server is sending out spam). This is a big reason why it's much much easier to get a third party to handle everything for you.