email that my application sends is getting spammed: what's wrong with my headers? - email

Question

I'm sending out emails via my PHP application. However, they're getting marked as spam by Gmail. Here's how I'm sending the email (PHP):
$headers = "From: test#bookmytakeout.com\r\nReply-To: test#bookmytakeout.com";
$mail_sent = mail( 'munged#gmail.com', 'test mail', $message, $headers, '-ftest#bookmytakeout.com' );
Gmail spams this message. So I went and clicked that handy "show original message" option. Here's what I get:
Delivered-To: munged#gmail.com
Received: by 10.68.71.200 with SMTP id x8cs325812pbu;
Thu, 21 Jul 2011 01:34:52 -0700 (PDT)
Received: by 10.236.114.234 with SMTP id c70mr12483739yhh.163.1311237292052;
Thu, 21 Jul 2011 01:34:52 -0700 (PDT)
Return-Path: <test#bookmytakeout.com>
Received: from vps.bookmytakeout.com ([8.22.200.47])
by mx.google.com with ESMTPS id u61si3662037yhm.119.2011.07.21.01.34.50
(version=TLSv1/SSLv3 cipher=OTHER);
Thu, 21 Jul 2011 01:34:51 -0700 (PDT)
Received-SPF: neutral (google.com: 8.22.200.47 is neither permitted nor denied by best guess record for domain of test#bookmytakeout.com) client-ip=8.22.200.47;
DomainKey-Status: bad format
Authentication-Results: mx.google.com; spf=neutral (google.com: 8.22.200.47 is neither permitted nor denied by best guess record for domain of test#bookmytakeout.com) smtp.mail=test#bookmytakeout.com; domainkeys=neutral (bad format) header.From=test#bookmytakeout.com
DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=default; d=bookmytakeout.com;
h=Received:To:Subject:From:Reply-To:Message-Id:Date;
b=WYWQ+/9+wOAvq+OUSM5KLDAueciIoNiByXaVV29HYa0XbSwm2f+89TCj8pW24G7k1vTGCvR8n64iSwgPQuoEitz6ehbszd0+75Px0WlGsvyeZGrW3PaYEpkRFmkDoaGe;
Received: from munged by vps.bookmytakeout.com with local (Exim 4.69)
(envelope-from <test#bookmytakeout.com>)
id 1QjolW-0001Vn-Us
for munged#gmail.com; Thu, 21 Jul 2011 14:07:31 +0530
To: munged#gmail.com
Subject: test mail
From: test#bookmytakeout.com
Reply-To: test#bookmytakeout.com
Message-Id: <E1QjolW-0001Vn-Us#vps.bookmytakeout.com>
Date: Thu, 21 Jul 2011 14:07:30 +0530
X-AntiAbuse: This header was added to track abuse, please include it with any abuse report
X-AntiAbuse: Primary Hostname - vps.bookmytakeout.com
X-AntiAbuse: Original Domain - gmail.com
X-AntiAbuse: Originator/Caller UID/GID - [1005 1000] / [47 12]
X-AntiAbuse: Sender Address Domain - bookmytakeout.com
test
Now, I have no idea how to diagnose what could be setting off gmail's spam filters. Can someone please point out what part of this email is setting off the spam filter?
If possible, please post a solution as well. I'm more interested in learning what's wrong with the headers than in how to fix this for now.
PS: I have a few suspicions of my own:
the Received header says "vps.bookmytakeout.com" but the From header says "bookmytakeout.com" - but I tried sending it with From as test#vps.bookmytakeout.com - same problem, still spammed.
The headers "Received-SPF", "DomainKey-Status" and "Authentication-Results" seem to indicate some problem. I distinctly remember NOT setting up any MX records for this domain name. Could that be the issue?

Answer 1

I guess you didn't publish any SPF / DKIM authentication record
Received-SPF: neutral (google.com: 8.22.200.47 is neither permitted nor denied by best guess record for domain of test#bookmytakeout.com) client-ip=8.22.200.47;
DomainKey-Status: bad format
Most of the time, Gmail (as well as the other major ISPs) will place the non-authenticated messages in the junk folder.
An SPF record or a DKIM guarantees that you are allowed to use a certain domain as a sender. For example, if you don't own "paypal.com", you can't send an email from "contact#paypal.com". If you've published the right SPF / DKIM, the ISP will consider you as a trusted sender.
SPF & DKIM were first designed to fight against phishing.
DIY SOLUTION: Publish some records by following these instructions:
http://dkim.org/specs/rfc5585.html
http://www.openspf.org/FAQ
EASY SOLUTION: Use a service that will do everything for you. A good ESP will usually sign your emails with DKIM / SPF by default. The problem is that you will sometimes get a "sent via ESP_NAME" mention (in Gmail).
So the best thing to do is to choose an ESP which will provide you personalized DKIM & SPF. This way, it will be 100% transparent.
I work for Mailjet and we offer this service for free. Most of our competitors offer this as an option.
Here's a useful post about this "via-mention" you get when your ESP signs "by default" and how to get rid of it.
http://blog.mailjet.com/post/16922561593/personalized-spf-dkim