Why do RECEIVED email headers seem to be out of order chronologically? - email

When reviewing the headers, it appears that the 2nd server to receive the message didn't relay it until AFTER the reported delivery date in the final header.
At c9mailgw11.amadis.com, the reported time was 22:47:49 -0800 (PST). However, the final server, BY2PR0401MB0966.namprd04.prod.outlook.com, reports the time as 06:46:08 +0000.
Is the discrepancy due to something simple, like a bad clock on the amadis relay?
I've written a script that detects email delays, and when I hit an oddity like that I get a negative value from that hop to the next and I want to understand why they are ordered this way to see if I have way to lookahead.
Can anyone offer insight?
**Received: from BY2PR0401MB0966.namprd04.prod.outlook.com (10.160.64.15) by
CY1PR0401MB0971.namprd04.prod.outlook.com (10.160.160.17) with Microsoft SMTP
Server (TLS) id 15.1.403.16 via Mailbox Transport; Wed, 17 Feb 2016 06:46:08**
+0000
Received: from BN3PR0401CA0023.namprd04.prod.outlook.com (10.162.159.161) by
BY2PR0401MB0966.namprd04.prod.outlook.com (10.160.64.15) with Microsoft SMTP
Server (TLS) id 15.1.409.15; Wed, 17 Feb 2016 06:46:07 +0000
Received: from BN3NAM01FT022.eop-nam01.prod.protection.outlook.com
(2a01:111:f400:7e41::207) by BN3PR0401CA0023.outlook.office365.com
(2a01:111:e400:51d1::33) with Microsoft SMTP Server (TLS) id 15.1.409.15 via
Frontend Transport; Wed, 17 Feb 2016 06:46:07 +0000
Received: from BAY004-MC1F22.hotmail.com (10.152.66.51) by
BN3NAM01FT022.mail.protection.outlook.com (10.152.67.153) with Microsoft SMTP
Server (TLS) id 15.1.409.7 via Frontend Transport; Wed, 17 Feb 2016 06:46:06
+0000
Received: from mail2world.com ([209.67.128.125]) by BAY004-MC1F22.hotmail.com with Microsoft SMTPSVC(7.5.7601.23143);
Tue, 16 Feb 2016 22:46:06 -0800
Received: from mail pickup service by mail2world.com with Microsoft SMTPSVC;
Tue, 16 Feb 2016 22:46:04 -0800
ResentFrom: xxx#xxx.com
Return-Path: xxx#xxx.com
Received: from 216.163.188.203 unverified ([216.163.188.203]) by mwpop05oc.mail2world.com with Mail2World SMTP Server;
Tue, 16 Feb 2016 22:46:01 -0800
**Received: from sender153-mail.zoho.com (unknown [74.201.84.153])
by c9mailgw11.amadis.com (Postfix) with ESMTP id A432C5B996A81
for <xxx#xxx.com>; Tue, 16 Feb 2016 22:47:49 -0800 (PST)**
DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws;
s=zapps768; d=zoho.com;
h=content-type:mime-version:subject:to:from:date;
b=b6BP/HJmeP+ORBz043y8K0tUFE3u3c8tUZvDSHjfuH0zD84gax1wUlRdYGps/SBd7SnFHYT1Plps
wRQOJoRp6hEMPerW6MSOL9psGWfNel4lnZwdtr7ujnuh54CTTEwV
Received: from [192.168.10.1] (xxx.xxx.net [xx.xx.xx.xx]) by mx.zohomail.com
with SMTPS id 1455691501828999.0688176107503; Tue, 16 Feb 2016 22:45:01 -0800 (PST)

The answer to this question is documented in rfc5321, section 4.4 as follows:
When an SMTP server receives a message for delivery or further
processing, it MUST insert trace ("time stamp" or "Received")
information at the beginning of the message content, as discussed in
Section 4.1.1.4.
This line MUST be structured as follows:
o The FROM clause, which MUST be supplied in an SMTP environment,
SHOULD contain both (1) the name of the source host as presented
in the EHLO command and (2) an address literal containing the IP
address of the source, determined from the TCP connection.
o The ID clause MAY contain an "#" as suggested in RFC 822, but
this
is not required.
o If the FOR clause appears, it MUST contain exactly one
entry, even when multiple RCPT commands have been given. Multiple
s raise some security issues and have been deprecated, see
Section 7.2.
An Internet mail program MUST NOT change or delete a Received: line
that was previously added to the message header section. SMTP
servers MUST prepend Received lines to messages; they MUST NOT change
the order of existing lines or insert Received lines in any other
location.

Another possible cause is that Outlook modifies the headers. It does this at least when it moves messages from one IMAP folder to another.
An Outlook bug report was filed for Outlook 2013, but some of this behaviour is still occurring in the latest Outlook desktop version.
It appears that the latest version of Outlook (as of August 2022) still does the following when moving a message from one IMAP folder to another:
it groups all the Received headers together, but in a different order from the original (even though it's not strictly speaking SMTP-related, it's not quite in the spirit of RFC 5321, as quoted by #jstedfast);
it groups all the Authentication-Results headers together, also in a different order (which also makes it complete nonsense if they're not close the Received header they were related to);
it changes the Message-ID header, even if the message hasn't really changed (very much against the spirit of RFC 5322;
it removes the Return-Path and Delivered-To headers.
It's not clear whether this fully matches your use-case, but since your example involves an outlook.com domain at some point in the Received headers, it's possible that other tools in the Outlook family show a similar attitude towards headers than the IMAP-specific use-case I've just mentioned.

Related

Should the DKIM signature be inserted after the Subject: header?

My ISP (Virgin Media) is introducing DKIM in their emails.
After sending and email I had a look at the source and found that they are placing the DKIM signature between the Subject: header and the body.
Return-Path: <xxxxxxxx#blueyonder.co.uk>
Delivered-To: xxxxxxxx#blueyonder.co.uk
Received: from md17.tb.ukmail.iss.local ([212.54.57.73])
by mc8.tb.ukmail.iss.local (Dovecot) with LMTP id 5cl8E7ZoJlmNQwAAVqD7fw
for <xxxxxxxx#blueyonder.co.uk>; Thu, 25 May 2017 07:17:29 +0200
Received: from mx6.tb.ukmail.iss.as9143.net ([212.54.57.73])
by md17.tb.ukmail.iss.local (Dovecot) with LMTP id
EstZLcTm/VjmbwAAeUlFJQ
; Thu, 25 May 2017 07:17:29 +0200
Received: from know-smtprelay-omc-3.server.virginmedia.net ([80.0.253.67])
by mx6.tb.ukmail.iss.as9143.net with bizsmtp
id QVHM1v00m1U0oNg01VHVUu; Thu, 25 May 2017 07:17:29 +0200
X-SourceIP: 80.0.253.67
X-CNFS-Analysis: v=2.2 cv=K/RSJ2eI c=1 sm=1 tr=0
a=NusZbS+MIKNGTdhVDzOOtg==:117 a=o+HFlgnLNhdkn43EnkHUFQ==:17
a=KXL61P45u2kA:10 a=tJ8p9aeEuA8A:10 a=MTPcje7z3XVcjakvMuAA:9
Received: from [192.168.0.19] ([80.193.xxxx.xxxx])
by know-smtprelay-3-imp with bizsmtp
id QVHU1v0051httJ801VHU5a; Thu, 25 May 2017 06:17:29 +0100
X-Originating-IP: [80.193.182.78]
X-Authenticated-User: xxxxxxxxn#blueyonder.co.uk
X-Spam: 0
X-Authority: v=2.1 cv=XNzNMlVE c=1 sm=1 tr=0 a=o+HFlgnLNhdkn43EnkHUFQ==:117
a=o+HFlgnLNhdkn43EnkHUFQ==:17 a=L9H7d07YOLsA:10 a=9cW_t1CCXrUA:10
a=s5jvgZ67dGcA:10 a=MTPcje7z3XVcjakvMuAA:9
From: xxxxxxxx#blueyonder.co.uk
To: xxxxxxxx#timothydutton.co.uk, xxxxxxxx#blueyonder.co.uk
Subject: Sending via Python
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=blueyonder.co.uk;
s=meg.feb2017; t=1495689449;
bh=8bjk8lFzQ2833A/RBLNeoi8C+tOykYEwquKHC+57mNw=; h=From:To:Subject;
b=ZuUwY74fd1AyxgXZ1LalJmb4fUFNqrQzOPall5FXrwTWoEYMBqR6u44HNJ1VOp1Jh
+ik/ip8stBs7SXM/3ViaL/Vj4cP9CPdxO1UY/kblhojNF2Jw31f69gCpGdDQVCx6eC
1KNdN0VadqQYVXd//wERGhXo+h2tkN2Ey0fZHxItXnN4ua90wwJfg3Iq2kM16m7OYw
W+JiISTy84jqYxS7TFpgtfSLdZUG5y5OyG9r9rP9JBXxqBUIaq/QlAfcuTVJ9HXhLf
h0d0CWLqSRoNrE2Dcqe/ES5cl12tL1HR3wHN4oU/kT0ilINEzRGSzkkeT0eHAUSjC6
zfRok7vuo+GQA==
Yo this is a mail sent via Python 3.6
According to Virgin Media, this is expected behaviour. But my understanding was that SMTP servers should add the headers to the start rather than the middle of the message?
Is this allowed in the SMTP or DKIM standard, or could it cause problems?
Yes, this is allowed by the DKIM standard. The only requirement I'm aware of is the following:
The DKIM-Signature header field MUST be inserted before any other
DKIM-Signature fields in the header block.
RFC 6376 section 5.6
It is however recommended to prepended it to the message:
The DKIM-Signature header field SHOULD be treated as though it were a
trace header field as defined in Section 3.6 of [RFC5322] and hence
SHOULD NOT be reordered and SHOULD be prepended to the message.
RFC 6376 section 3.5

DKIM bad Signature. Expeted Body Hash is the same as the BH sended. Apache James

I made a lot of research about this topic. I have an Apache James 2.3.2.1 Mail Server. I'm configuring it as a production server. I got it running, and stable, but implementing jDKIM have been troublesome. I found these links to get it right.
http://www.nailedtothex.org/roller/kyle/entry/configuring-james-to-sign-dkim
https://github.com/smoradi/config/blob/master/noften/apache-james.txt
Until now, I verified the public and private keys repetitively. Mailets libraries. Java code and configuration for the Mailet.
But I'm stuck.
While testing with www.appmaildev.com/en/domainkeys/, I finally solve some errors over the themes named above. But now It's just toying with me.
In the DKIM Test result it says that the expected body hash must be
Expected Body Hash:
: frcCV1k9oG9oKj3dpUqdJg1PxRT2RSN/XKdLCPjaYaY=
But in the Message Header says that the body hash is just that.
bh : frcCV1k9oG9oKj3dpUqdJg1PxRT2RSN/XKdLCPjaYaY=
And It's just the same so this test should be OK.
Maybe I'm missing something over de DNS or in the Mailet configuration, I just dont know i have been trying to approach in different ways but this is were I get every time.
As a note, before getting this point, every time I send a mail over Outlook or Homail, it was getting to the Junk immediately, now is kind of random almost every 3 or 5 sends. While Gmail since the SPF got them always in the Inbox. And The server does not have an PTR record properly configured in the DNS.
This is the Mail from appmaildev.
I change my domain name over example.com
And change the IPs to xxx.xxx.xxx.xxx
this test in particular was made over telnet using ehlo
I marked the puzzling part of code with ** **.
================================================================
DKIM result: fail (bad signature)
Signed by: webmaster#example.com
**Expected Body Hash: frcCV1k9oG9oKj3dpUqdJg1PxRT2RSN/XKdLCPjaYaY=**
PublicKey: selector._domainkey.example.com
IN TXT = "k=rsa; p=
MIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQCtLBczKGEysTT6QuRgApyn6yvM
XOU9Sjkx6YMomuFujqPk2XgMuBbgafckRX+1F18h9G1rURmvc3EmPxegFxim6wrE
1RJGfddO+OHEjTBOsuXa7BK29P/bhCOehVeYG4o0L5DH8z3izfmvNuC3dC4N5G3R
8ZrNd7k/196TCZIaMwIDAQAB;"
---Original Message Header---
x-sender: webmaster#example.com
x-receiver: AAAA4AcCCxQA#appmaildev.com
Received: from james ([xxx.xxx.xxx.xxx]) by mail.appmaildev.com with Microsoft SMTPSVC(7.5.7600.16385);
Thu, 11 Feb 2016 22:05:19 -0500
DKIM-Signature: v=1; d=example.com; b=V9gPNU9Gjky7fFov5cYuTGN/gQEoBgaAkctwnME17pT/Uket4PDsVixMQbyiKvgQF3ADK8HTrKyd1L+9bBiesJAQaD1L1lH0ualjO/Ctf8jjzLyDUfB6cu4TEuAD+aVkIUVVbd0LNhcpxiOMsd5cADRBqZy60Mw9fQOaqyjh0QU=; s=selector; a=rsa-sha256; **bh=frcCV1k9oG9oKj3dpUqdJg1PxRT2RSN/XKdLCPjaYaY=;** h=from:to:received:received;
Message-ID: <1848510914.01455246317306.JavaMail.root#james>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit
X-UserIsAuth: true
Received: from localhost.localdomain ([127.0.0.1])
by james (JAMES SMTP Server 2.3.2.1) with SMTP ID 101
for <AAAA4AcCCxQA#appmaildev.com>;
Fri, 12 Feb 2016 03:05:17 +0000 (UTC)
Date: Fri, 12 Feb 2016 03:05:17 +0000 (UTC)
From: webmaster#example.com
subject: prueba ehlo
ehlo.:
Bcc:
Return-Path: webmaster#example.com
X-OriginalArrivalTime: 12 Feb 2016 03:05:20.0069 (UTC) FILETIME=[34ED9F50:01D16542]
After more research and testing, I made it to work.
It seems that I had missing values in the DKIM-Signature.
This was my original tag in config.xml for Apache James in the DKIMSign mailet:
<signatureTemplate>v=1; s=selector; d=example.com;
h=from:to:received:received; a=rsa-sha256; bh=;
b=;</signatureTemplate>
I was missing the "c" and I made a change over the "h"
c=relaxed/relaxed;
h=Message-ID:Date:Subject:From:To:MIME-Version:Content-Type;
After the modification I got this:
<signatureTemplate>v=1; s=selector; d=example.com;
h=Message-ID:Date:Subject:From:To:MIME-Version:Content-Type;
a=rsa-sha256; bh=; b=;c=relaxed/relaxed;</signatureTemplate>
Another Detail, that I made several test about was the DNS Record. Mine was over Windows Server , so just for caution, I checked these details.
First , The escape char before the semicolon in the "k"
k=rsa\;
Second, That the "p" value, was the last value and that does not have a semicolon or escape char.
p=MIG...QAB
So that, after a host command I got something like
selector._domainkey.example.com descriptive text "k=rsa\;p=\010...QAB"
After those changes and checkings. It pass every test :)
Another link i found usefull for the configuration was this.
http://mail-archives.apache.org/mod_mbox/james-server-user/201410.mbox/%3C544FD474.2040906%40malcolms.com%3E

Why is my email being rejected?

I am having problems sending an email from one email address to another on my domain. I am using a shared server. I use weather software which automatically constructs and sends the email message. I have omitted the host, domain and IP info and replaced them like so: host= example.co.uk, domain = mydomain.co.uk .
The situation is I run a mail group using email address (weatherinfo#) and some users in the group set auto-responders. Therefore to prevent these from going to the whole group the email is sent from a different address (reports#) to weatherinfo# with the reports# address receiving the auto-responder messages.
I am getting the following message from the mail delivery system when trying to send the email to weatherinfo# from report#:
This is the mail system at host mailauth.example.co.uk.
I'm sorry to have to inform you that your message could not
be delivered to one or more recipients. It's attached below.
For further assistance, please send mail to postmaster.
If you do so, please include this problem report. You can
delete your own text from the attached returned message.
The mail system
<weatherinfo#mydomain.co.uk>: host 127.0.0.1[127.0.0.1] said: 554 5.6.0
Reject, id=02474-18 - BAD HEADER (in reply to end of DATA command).
The error log accompanying the email shows:
Reporting-MTA: dns; mailauth.example.co.uk
X-Postfix-Queue-ID: 88002C306D
X-Postfix-Sender: rfc822; reports#mydomain.co.uk
Arrival-Date: Fri, 2 May 2014 22:00:43 +0100 (BST)
Final-Recipient: rfc822; weatherinfo#mydomain.co.uk
Original-Recipient: rfc822;weatherinfo#mydomain.co.uk
Action: failed
Status: 5.6.0
Remote-MTA: dns; 127.0.0.1
Diagnostic-Code: smtp; 554 5.6.0 Reject, id=02474-18 - BAD HEADER
The header from the original message is:
Return-Path: <reports#mydomain.co.uk>
Received: from LAPTOP (helium.example.co.uk [IP Address])
by mailauth.example.co.uk (Postfix) with ESMTPA id 88002C306D
for <weatherinfo#mydomain.co.uk>; Fri, 2 May 2014 22:00:43 +0100 (BST)
To: weatherinfo#mydomain.co.uk
From: reports#mydomain.co.uk
Subject: WEATHER REPORT
Reply-To: reports#mydomain.co.uk
Date: Fri, 02 May 2014 23:00:56 +0100
Message-Id: <43786804l.820319781l1635676l1l#mydomain.co.uk>
Message-ID: <2014.05.02.17001.wd#mail.mydomain.co.uk>
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Type: text/plain; charset="us-ascii"
Why is this message being rejected?
Is this being caused by my software or the server?
If it is the software what does the developer need do to fix it?
If it is the server what should I ask my host to do to overcome this problem?
The server says, your header is wrong, so I'd search the problem within your software.
If it isn't just a missing whitespace in your post, the header field "Recieved" isn't folded as specified in RFC 2822: 2.3.3 Long Header Fields causing two invalid headers starting with "by" and "for". Correct would be:
Received: from LAPTOP (helium.example.co.uk [IP Address])
by mailauth.example.co.uk (Postfix) with ESMTPA id 88002C306D
for <weatherinfo#mydomain.co.uk>; Fri, 2 May 2014 22:00:43 +0100 (BST)
Also I see that there are two Message-id headerfields. I do not know whether this is allowed or not.
I would always do tests with telnet (or putty/raw) in such a case and try with several heder fields skipped or modified so you can break it down to the header field your mail server is complaining about.

Solution No Such User Here error

I am battling to find a definitive answer and solution to this problem, pulling my hair out stage.
I have several Wordpress sites that have forms within these. Essentially, the form is completed and an employee receives a notification with the details, pretty simple. The employees do not receive the notification emails at all, but when I use my personal email or Gmail it works perfectly!
I am seeing the No Such User Here error via my hosts webamail as per the below. I'm told that all emails are whitelisted and shouldnt be block, but they say they dont see the emails hitting their gateway.
I'd really appreciate some help here to move on with this thing, thanks in advance.
Error message:
This message was created automatically by mail delivery software.
A message that you sent could not be delivered to one or more of its
recipients. This is a permanent error. The following address(es) failed:
rcook#csgroup.org.au
No Such User Here
------ This is a copy of the message, including all the headers. ------
Return-path: <gravityforms#ampedmediaworks.com>
Received: from amwrrc by gator41.hostgator.com with local (Exim 4.80)
(envelope-from <gravityforms#ampedmediaworks.com>)
id 1ThUzf-0007gH-00
for rcook#csgroup.org.au; Sat, 08 Dec 2012 18:43:42 -0600
Received: from 60.228.85.149 ([60.228.85.149])
(SquirrelMail authenticated user gravityforms#ampedmediaworks.com)
by gator41.hostgator.com with HTTP;
Sat, 8 Dec 2012 18:43:42 -0600
Message-ID: <ec0f41d26aad8a08452d013018cd32db.squirrel#gator41.hostgator.com>
Date: Sat, 8 Dec 2012 18:43:42 -0600
Subject: test
From: gravityforms#ampedmediaworks.com
To: rcook#csgroup.org.au
User-Agent: SquirrelMail/1.4.22
MIME-Version: 1.0
Content-Type: text/plain;charset=iso-8859-1
Content-Transfer-Encoding: 8bit
X-Priority: 3 (Normal)
Importance: Normal
test
It may have something to do with Your IP being on blacklist
http://www.mxtoolbox.com/SuperTool.aspx?action=blacklist%3a60.228.85.149

GMail messages show wrong date after import

I have created a Google Apps account and then important my old email messages from my own IMAP-based mail server. For that purpose I used imapsync.
There is some mails which show up fine when viewed in Thunderbird -- interestingly enough as well on the old IMAP server as on GMail. They correctly have the date of original sending at 2nd of December 2008. When viewing the full mail source ("original message") from GMail web interface, its headers read (XXX for anonymization in some places):
X-Account-Key: account6
X-UIDL: 10f6988675937085b6eXXX
X-Mozilla-Keys:
Return-Path: <XXX#t-online.de>
X-Flags: 0000
Delivered-To: GMX delivery to XXX#gmx.de
Received: (qmail invoked by alias); 02 Dec 2008 09:16:48 -0000
Received: from mailout09.t-online.de (EHLO mailout09.t-online.de) [194.25.134.84]
by mx0.gmx.net (mx086) with SMTP; 02 Dec 2008 10:16:48 +0100
Received: from fwd02.aul.t-online.de
by mailout09.sul.t-online.de with smtp
id 1L7RN9-XXX; Tue, 02 Dec 2008 10:16:47 +0100
Received: from localhost (Tti8HrZYYh01uPSGlzL36WA4trfZ2XXX#[172.20.101.250]) by fwd02.aul.t-online.de
with esmtp id 1L7RMr-XXX; Tue, 2 Dec 2008 10:16:29 +0100
MIME-Version: 1.0
Received: from 77.185.232.243:1675 by cmpweb16.aul.t-online.de with HTTP/1.1
(Kommunikationscenter V9-2-23 on API V3-3-19)
Date: Tue, 02 Dec 2008 10:16:29 +0100
Reply-To: "XXX#t-online.de" <XXX#t-online.de>
To: XXX#gmx.de
X-Priority: 3
X-UMS: email
X-Mailer: TOI Kommunikationscenter V9-2-23
Subject: XXX
From: "XXX#t-online.de" <XXX#t-online.de>
Content-Type: multipart/mixed;
boundary="=_71bfbbb1eca6e77241917c40d55c1078"
Message-ID: <1L7RMr-XXX#fwd02.aul.t-online.de>
X-ID: Tti8HrZYYh01uPSGlzL36WA4trfZ2Ib9MXXX#t-dialin.net
X-TOI-MSGID: 6a3e3035-aa28-XXX
X-GMX-Antivirus: 0 (no virus found)
X-GMX-Antispam: 0 (Mail was not recognized as spam)
X-GMX-UID: GcoGeJ9WIyd1gjzXXX
But in the overview lists in the web interface it shows as dated 7th of October 2011. This date might be the date of a previeous import (to my own IMAP server).
But I have no idea where Google takes that date from and how I can change this date to the date the message was really sent. Any ideas?
I found this worked for me -
Copy all mails via IMAP from GMail to a local/different account folder
Now delete all mails from GMail using their Web interface
Empty the GMail trash
Copy all mails back from local/different account to GMail
Source - http://productforums.google.com/d/msg/gmail/0PU1gQuXNi8/HGu6GRigerMJ