Nonstandard DMARC report sent by Google - email

I'm working on a system which parses DMARC reports and I figured the following issue:
Sometimes, Google sends nonstandard e-mails, as can be seen below:
MIME-Version: 1.0
X-Received: by x.x.x.x with SMTP id xxxx.xx.xxxx;
Thu, 22 Aug 2013 02:13:03 -0700 (PDT)
Message-ID:
Date: Thu, 22 Aug 2013 09:13:03 +0000
Subject: Report domain: example.com Submitter: google.com Report-ID: xxxxx
From: noreply-dmarc-support#google.com
To: postmaster#example.com
Content-Type: application/zip;
name="google.com!example.com!1377043200!1377129599.zip"
Content-Disposition: attachment;
filename="google.com!example.com!1377043200!1377129599.zip"
Content-Transfer-Encoding: base64
UEsDBAoAAAAIAEJIFkMWecIj/AEAAKkEAAAvAAAAZ29vZ2xlLmNvbSFsYW50aWFuLmV1ITEzNzcw
...
AAABAAEAXQAAAEkCAAAAAA==
Please take a look at the unusual break line between Content-Disposition and Content-Transfer-Encoding headers.
After the MIME standard, the content of the email should look like:
Content-Type: application/zip;
name="google.com!example.com!1377043200!1377129599.zip"
Content-Disposition: attachment;
filename="google.com!example.com!1377043200!1377129599.zip"
Content-Transfer-Encoding: base64
UEsDBAoAAAAIAEJIFkMWecIj/AEAAKkEAAAvAAAAZ29vZ2xlLmNvbSFsYW50aWFuLmV1ITEzNzcw
...
AAABAAEAXQAAAEkCAAAAAA==
This break line should not be there (you can see http://en.wikipedia.org/wiki/Multipurpose_Internet_Mail_Extensions ).
So, why Google do this?

If you were to join dmarc-discuss#dmarc.org and post this question there, I can assure you it would be read by a Google engineer that works on DMARC. When I wrote my DMARC implementation, I too discovered a number of variances between the reports I received and the DMARC draft spec. Not too long after reporting the variances on that list, they were all corrected.

Related

Trouble with DMARC and Google Apps / GSuite

I've had the following DMARC policy setup for over a year, but in the last two weeks I'm suddenly unable to send emails to many people. Yet I haven't changed this record. I am experienced with DNS, server administration and programming yet I cannot find any explanation for this issue.
What are the solutions to this problem?
Existing DMARC Record
v=DMARC1; p=reject; pct=100; rua=mailto:re+something#dmarc.postmarkapp.com; ruf=mailto:me#mydomain.com; sp=none; aspf=r; fo=1;
Error I get when emailing various emails (#gmail.com and custom domains).
https://gist.github.com/s3w47m88/115688a7ecd5a8c762bd3f98932756b2
Headers for Successful Email
MIME-Version: 1.0
Date: Wed, 10 Apr 2019 15:26:48 -0700
References: <BN7PR06MB4116507B5F036C4D175E082CC82E0#BN7PR06MB4116.namprd06.prod.outlook.com> <CAN9OK_OfgXw_mW2+M-=TkHLupnOdBo=VyE=wQOALykc8=EzjXA#mail.gmail.com> <BN7PR06MB411654D8A0AA5D44F92E1D1EC82E0#BN7PR06MB4116.namprd06.prod.outlook.com> <CAN9OK_PMipqHaLK9W-PAn0_dhsD876TpETq85CeVC5NBQpCPig#mail.gmail.com> <BN7PR06MB4116592A4D4E6B4EBE299EF5C82E0#BN7PR06MB4116.namprd06.prod.outlook.com>
In-Reply-To: <BN7PR06MB4116592A4D4E6B4EBE299EF5C82E0#BN7PR06MB4116.namprd06.prod.outlook.com>
Bcc: 5729491#bcc.hubspot.com
Message-ID: <CAN9OK_O-WEFqJysAr8S51LrBa1_fopy1UoFVSxq5JWNeeMuZCQ#mail.gmail.com>
Subject: Re: Your free trial
From: Me <me#mydomain.com>
To: John Doe <someone#asite.com>
Content-Type: multipart/alternative; boundary="000000000000d3115405863490ac"
--000000000000d3115405863490ac
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

Thunderbird+Lightning is rendering an ics invitation inappropriately

Using sabre/vobject library I am creating an ics file like this one:
BEGIN:VCALENDAR
VERSION:2.0
PRODID:-//Sabre//Sabre VObject 4.1.2//EN
CALSCALE:GREGORIAN
BEGIN:VEVENT
UID:sabre-vobject-d4b1ccb3-2197-4ee4-aab8-7bc2516adbf8
DTSTAMP:20170123T182612Z
SUMMARY:testev2
DESCRIPTION:
DTSTART;TZID=Europe/Athens:20170214T090000
DTEND;TZID=Europe/Athens:20170215T170000
LOCATION:
ORGANIZER;CN=Organizer Name:mailto:organizer#example.com
ATTENDEE;CN=Test User:MAILTO:test.user#somewhere.com
END:VEVENT
END:VCALENDAR
Then using phpmailer, I am attaching the file generated to an e-mail message and send it to the users who are participating on the event. Users who are using thunderbird (with Lightning extension - which is by default on) receive the e-mail message in the following format:
As you see in the picture the ics file is parsed and rendered in the table at the bottom of the e-mail. However the table header (the one marked in red) is not displaying correct information. Test user has not ever cofirmed his/her presence, and if he does, the ics file does not provide any info about it.
Am I formatting the ics file wrong?
Is it a known bug of thunderbird / Lightning ?
UPDATE
The e-mail headers of the message:
Return-Path: <XXXXXXXXXXXXXXXX>
Received: from deliver ([unix socket])
by mail (Cyrus v2.3.16-Fedora-RPM-2.3.16-13.el6_6) with LMTPA;
Tue, 24 Jan 2017 12:48:10 +0200
X-Sieve: CMU Sieve 2.3
Received: from [XXX.XXX.XXX.XXX] (XXXXXXXXXXXXXX [XXX.XXX.XXX.XXX
(using TLSv1.2 with cipher DHE-RSA-AES128-SHA (128/128 bits))
(No client certificate requested)
by XXXXXXXXXXXXXX (Postfix) with ESMTPSA id 6F18C1BE0305
for <XXXXXXXXXXXXXX>; Tue, 24 Jan 2017 12:48:10 +0200 (EET)
Subject: Fwd: Event invitation: testev2
To: "XXXXXXXX" <XXXXXXXXXXXXXX>
From: XXXXXXXXXXXXXX <XXXXXXXXXXXXXX>
X-Forwarded-Message-Id:
Message-ID: <bac7749e-9699-1b50-9de5-27a510c663a4#XXXXXXXX>
Date: Tue, 24 Jan 2017 12:48:09 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101
Thunderbird/45.6.0
MIME-Version: 1.0
Content-Type: multipart/mixed;
boundary="------------79DD2A1D49F1A57579125B45"
This is a multi-part message in MIME format.
--------------79DD2A1D49F1A57579125B45
Content-Type: multipart/alternative;
boundary="------------72E56459CD6D794D0DF5AC4B"
--------------72E56459CD6D794D0DF5AC4B
Content-Type: text/plain; charset=windows-1252; format=flowed
Content-Transfer-Encoding: 8bit
> ------- Forwarded Message --------
Forward message content
> --------------72E56459CD6D794D0DF5AC4B
> Content-Type: text/html; charset=windows-1252
> Content-Transfer-Encoding: 8bit
>
HTML Content
> --------------72E56459CD6D794D0DF5AC4B--
>
> --------------79DD2A1D49F1A57579125B45
> Content-Type: text/calendar;
> name="invitation.ics"
> Content-Transfer-Encoding: 7bit
> Content-Disposition: attachment;
> filename="invitation.ics"
>
ICS Content
> --------------79DD2A1D49F1A57579125B45--
You should make the Content-Type of the ics attachment something like:
Content-Type: text/calendar; charset="utf-8"; method=REQUEST
The method option is the magic word. I'm not completely sure this works, but it is at least closer to the spec. I'm happy to update my answer if it doesn't work.

OSB email - forcing multipart/mixed

I am sending email using a OSB (11.1.1.6) service.
Some email clients do not pick up the attachments.
We have narrowed down the problem down to MIME Content-Type.
Going through OSB it sets the Content-Type to multipart/related. In order to get it to work (we tested this using ncat) we need to set the Content-Type to multipart/mixed.
I cannot however find any way to force OSB to set it to multipart/mixed.
This message does not display the attachment on some clients:
From: <nothing#example.com>
To: nothing#example.com
Message-ID: <xxx>
Subject: Subject 123
MIME-Version: 1.0
Content-Type: multipart/related; boundary="MIME_Boundary";
start=1389578236803081255-2926c9b7.148d69bfba8.7396
Return-Path: nothing#example.com
--MIME_Boundary
Content-ID: 1389578236803081255-2926c9b7.148d69bfba8.7396
Content-Type: text/html; charset="iso-8859-1"
Content-Transfer-Encoding: 8bit
<h1>Head</h1>
<p>Paragraph <b>bold</b></p>
--MIME_Boundary
Content-Type: text/plain; name="TEST.txt"
Content-Transfer-Encoding: base64
Content-Description: TEST.txt
Content-Disposition: attachment; filename="TEST.txt"
VGVzdGluZyAxMjM=
--MIME_Boundary--
This message displays the attachment:
From: <nothing#example.com>
To: nothing#example.com
Message-ID: <xxx>
Subject: Subject 123
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="MIME_Boundary";
start=1389578236803081255-2926c9b7.148d69bfba8.7396
Return-Path: nothing#example.com
--MIME_Boundary
Content-ID: 1389578236803081255-2926c9b7.148d69bfba8.7396
Content-Type: text/html; charset="iso-8859-1"
Content-Transfer-Encoding: 8bit
<h1>Head</h1>
<p>Paragraph <b>bold</b></p>
--MIME_Boundary
Content-Type: text/plain; name="TEST.txt"
Content-Transfer-Encoding: base64
Content-Description: TEST.txt
Content-Disposition: attachment; filename="TEST.txt"
VGVzdGluZyAxMjM=
--MIME_Boundary--
As you can see the only difference is the Content-Type.
So how do I force OSB to set the Content-Type to multipart/mixed ?
You can set the Transport Header Content-Type. I assume you are using the routing to call the BS service which has email configuration. From Proxy, where you are routing, in the request actions, add Communication > Transport Headers. From the drop down, select emails >> Content-Type.
After some communication with Oracle support we were pointed to apply patch 12585136.
This was one of the bugs fixed for OSB 11.1.1.7 (link)
12585136 - The Email transport generates multipart/related emails and not mulitpart/mixed
After we have applied and tested the patch I will update this answer with more feedback.

Reply-To Email header not working anymore

First of all: My reply-to header always worked for 2 years.. Thunderbird never had a problem with it and still doesn't have any problem on my Mac.
My shop contact form sends me the email from info#webshop.com and adds the reply-to header from the customer
The source of the email is:
Return-path: <sterntau#s207.rackspeed.de>
Envelope-to: info#sterntaufe-deutschland.de
Delivery-date: Mon, 04 Nov 2013 18:00:05 +0100
Received: from sterntau by s207.rackspeed.de with local (Exim 4.80.1)
(envelope-from <sterntau#s207.rackspeed.de>)
id 1VdNVV-001tmU-Gn
for info#sterntaufe-deutschland.de; Mon, 04 Nov 2013 18:00:05 +0100
To: =?utf-8?B?aW5mbw==?= <info#sterntaufe-deutschland.de>
Subject: =?utf-8?B?S29udGFrdGZvcm11bGFy?=
Reply-To: customer#gmail.com
From: Sterntaufe-Deutschland <info#sterntaufe-deutschland.de>
Date: Mon, 04 Nov 2013 17:00:05 +0000
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
Content-Disposition: inline
MIME-Version: 1.0
Message-Id: <E1VdNVV-001tmU-Gn#s207.rackspeed.de>
=0A=0AName: Test=0AE-Mail: customer#gmail.com=0ATelefon: =0A=0AKommentar: Test
Normally it should reply to customer#gmail.com which always worked till yesterday.
From yesterday its not working anymore. On all emails! Even on those where it worked before.
But it works fine if I send a normal E-mail from my Gmail account to my info#webshop.com account.
Reply-To works like it should then. Just not for the contact form
Thunderbird recognizes the reply-to email as it shows "Reply to: customer#gmail.com" below the subject. But still uses my info#webshop.com when I click "reply"
I also tried to re-install thunderbird, install thunderbird on a clean VMware. Install Thunderbird on another laptop.. all the same.
But it still works with Microsoft outlook
Please help me
It is a BUG of Thunderbird 24.1.0 https://bugzilla.mozilla.org/show_bug.cgi?id=933555

Amazon SES and Hotmail spam

I configured an email address with google apps for my company. When I send messages from Gmail to address#hotmail.com the message is received correctly in the inbox folder.
When I moved to Amazon SES, I configured correctly SPF and DKIM so that it may work fine. The big problem is that when I send from SES emails go allways into the spam folder!!!
This is the hotmail header of a message that is falled into the spam folder:
x-store-info:CnuewmGKkJzNjuOw4Ko28wB3rXpWYbsxTq8bIGVpexou/aH5YlneZSXtbrTNbKJ4GoT+OaKU2vnoHLIPY7tpJ7yfD4ei7NGnJPMqwC1IOiYDYaHi7z9UqM7HFUFg9PvdD/GTLm1Joes=
Authentication-Results: hotmail.com; spf=pass (sender IP is 54.240.8.95) smtp.mailfrom=0000014191bce21d-5857cbb3-7185-4a04-a62d-02029457d42b-000000#amazonses.com; dkim=pass header.d=beaudience.com; x-hmca=pass header.id=support#beaudience.com
X-SID-PRA: support#beaudience.com
X-AUTH-Result: PASS
X-SID-Result: PASS
X-Message-Status: n:n
X-Message-Delivery: Vj0xLjE7dXM9MDtsPTE7YT0xO0Q9MjtHRD0xO1NDTD00
X-Message-Info: 11chDOWqoTmYiARgB8x0CqssYC30R1hAxykCxY7lMqvPXk+fm44PmUeqp2eso9uKqBo8WFDhDk3rZsgJn8uSIHpUqpn7/N+/COouobxjVl2F7FiiDMh/AjlIDYLoKhZeWqATlTzu9cdwruznM5Eh3gOw+h4szTV5OcHunEoeFZeggqKm4r8Wd97fzBr3wpj6Xji14R+Xo8C7zTF5xkQAV15Ns/IGAE0R
Received: from a8-95.smtp-out.amazonses.com ([54.240.8.95]) by COL0-MC3-F51.Col0.hotmail.com with Microsoft SMTPSVC(6.0.3790.4900);
Mon, 7 Oct 2013 00:06:18 -0700
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/simple;
s=fzsj4xlkgrzw4njd7a4n5dv47w5dmrc5; d=beaudience.com; t=1381129577;
h=Date:To:From:Reply-To:Subject:Message-ID:MIME-Version:Content-Type:Content-Transfer-Encoding;
bh=d9cLexwYe6DbP7/N2SXpl7aOUi58tQ37WMdTDDTQtvA=;
b=rR0at2KyIFuhpI6HFSd56LbiVPS2uPzECnYlscb7UliQraxznWxjRKrDCF3HVNJj
1/s3xjXaOLoCLk0H0B8xa76KzWgMwtxDulEFn39G06yRd9/r/17xTYzQ/MpMMn9lUlv
VT75xxTBO7iwm8hZ4ntQtBsMnnvybLC89tAoVXNE=
Date: Mon, 7 Oct 2013 07:06:17 +0000
Return-Path: 0000014191bce21d-5857cbb3-7185-4a04-a62d-02029457d42b-000000#amazonses.com
To: luca.pennisi#live.com
From: support BeAudience <support#beaudience.com>
Reply-To: support#beaudience.com
Subject: We remember you!
Message-ID: <0000014191bce21d-5857cbb3-7185-4a04-a62d-02029457d42b-000000#email.amazonses.com>
X-Priority: 3
X-Mailer: PHPMailer 5.2.6 (https://github.com/PHPMailer/PHPMailer/)
MIME-Version: 1.0
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: 8bit
X-SES-Outgoing: 2013.10.07-54.240.8.95
X-OriginalArrivalTime: 07 Oct 2013 07:06:18.0240 (UTC) FILETIME=[B83DF000:01CEC32B]
<html><head></head><body><b>Account
details:</b><p>username: tryHard<br />password:
porcodio</p><hr /><a
href="http://www.beaudience.com/joinus.php">Click
here to log-in</a><br /><br /><p>BeAudience
staff.</p></body></html>
I'm desperate, don't know what to do! I have configured SPF and DKIM but is useless! It not depends on the email content, I tried with different contents, html/non html but nothing to do with ses!!
It's not a problem on Amazon SES side, Microsoft filters are flagging your message based on multiple variables, like headers, content, domain age, etc.
That seems to be a Welcome Message, so the best way to go about it is to instruct the user to check the spam folder and mark the message as "not junk". Eventually and hopefully, Microsoft will learn from multiple user decisions and will start delivering your messages to the inbox.
Another advice, you are sending the message using PHPMailer. That header (X-Mailer: PHPMailer 5.2.6) may be triggering a flag on Microsoft side before even getting to your content.
I had the same problem. But I've fixed it. My steps:
add text/html version for mail;
check your html in email, make sure that is correct;
verify your domain in SES dashboard;