as I read on the official rfc2822
The "Message-ID:" field provides a unique message identifier that
refers to a particular version of a particular message. The
uniqueness of the message identifier is guaranteed by the host that
generates it (see below). This message identifier is intended to be
machine readable and not necessarily meaningful to humans. A message
identifier pertains to exactly one instantiation of a particular
message; subsequent revisions to the message each receive new message
identifiers.
Note: There are many instances when messages are "changed", but those
changes do not constitute a new instantiation of that message, and
therefore the message would not get a new message identifier. For
example, when messages are introduced into the transport system, they
are often prepended with additional header fields such as trace
fields (described in section 3.6.7) and resent fields (described in
section 3.6.6). The addition of such header fields does not change
the identity of the message and therefore the original "Message-ID:"
field is retained. In all cases, it is the meaning that the sender
of the message wishes to convey (i.e., whether this is the same
message or a different message) that determines whether or not the
"Message-ID:" field changes, not any particular syntactic difference
that appears (or does not appear) in the message.
An email message can hold the same ID in some particular cases.
For example, take a look at the two following message header:
Return-Path: <xxx.xxx#xxx.it>
Received: from relaypsm.eng.it (xxxx.xxx.it [xxx.xxx.xxx.xxx])
by xxxxx-xxx.xxxxx.it (8.14.4/8.14.4) with ESMTP id v0PJ6wOD014924
(version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=FAIL)
for <xxxxx#xxx.xxxxx.it>; Wed, 25 Jan 2017 20:06:59 +0100
X-Icontrol: Sent by Inrete Icontrol
Received: from MailAV (unknown [xxx.xxx.xxx.xxx])
by deliver (Postfix) with ESMTP id 52B743A4
for <xxxxx#xxx.xxxxx.it>; Wed, 25 Jan 2017 20:06:58 +0100 (CET)
Received: from LBURRINIW (unknown [192.168.63.9])
by xxxxx.xxx.it (Postfix) with ESMTPSA id 8D80339F
for <xxxxx#xxx.xxxxx.it>; Wed, 25 Jan 2017 20:06:51 +0100 (CET)
From: "Luca Burrini" <luca.burrini#eng.it>
To: <xxxxx#xxx.xxxxx.it>
References:
In-Reply-To:
Subject: fatture xxxxx
Date: Wed, 25 Jan 2017 20:06:51 +0100
Message-ID: <!&!AAAAAAAAAAAYAAAAAAAAAHCgtCXXdtBNl+uXnP8eDHPCgAAAEAAAAGutlHZJGkpAhmPV6f+ofh8BAAAAAA==#eng.it>
MIME-Version: 1.0
Content-Type: multipart/mixed;
boundary="----=_NextPart_000_00D6_01D27746.91533090"
X-Mailer: Microsoft Office Outlook 12.0
Thread-Index: AdCOWOCYZlK03GM4S5Cd5xo9GuRjGAAorxQwBlBUrvAFQ/QDgAew8EzACjCMyCAGU/qYMAV4vr+QBuilgWAKzjrI4AagZVDwBqrsgiAABfH4YAXsTykQBafYYmAEIBNvwBSv7j/wBgnvoHAB0wKvkAWE5BHA
Content-Language: it
X-KLMS-Rule-ID: 1
X-KLMS-Message-Action: clean
X-KLMS-AntiSpam-Status: not scanned, disabled by settings
X-KLMS-AntiPhishing: not scanned, disabled by settings
X-KLMS-AntiVirus: Kaspersky Security 8.0 for Linux Mail Server, version 8.0.1.721, bases: 2017/01/25 14:07:00 #8842754
X-KLMS-AntiVirus-Status: Clean, skipped
X-IControl-Milter-Checked: yes (IControlServer: xxxxx-NEW SrcIpType: UnTrusted SrcIpHeaderType: Undefined Authenticated: no)
X-IControl-Milter-SPF-Checked: yes (IControlServer: CEDACRIPEC1-NEW HeloSPFType: none FromSPFType: pass HeloHeaderSPFType: Undefined FromHeaderSPFType: pass)
X-IControl-Milter-MD5SUM: c803ab2c6498f91967e8cb2f5954f43a
and
Return-Path: <xxx.xxx#xxx.it>
Received: from xxxxx.xxx.it (xxxxx.xxx.it [xxx.xxx.xxx.xxx])
by xxxxx-new.xxxxx.it (8.14.4/8.14.4) with ESMTP id uBSGqOAP023446
(version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=FAIL)
for <xxxxx#xxx.xxxxx.it>; Wed, 28 Dec 2016 17:52:24 +0100
X-Icontrol: Sent by Inrete Icontrol
Received: from MailAV (unknown [161.27.15.23])
by deliver (Postfix) with ESMTP id A0FD33AC
for <xxxxx#xxx.xxxxx.it>; Wed, 28 Dec 2016 17:52:23 +0100 (CET)
Received: from LBURRINIW (unknown [xxx.xxx.xxx.xxx])
by relaypsm.eng.it (Postfix) with ESMTPSA id 7D2D83CC
for <xxxxx#xxx.xxxxx.it>; Wed, 28 Dec 2016 17:52:17 +0100 (CET)
From: "Luca Burrini" <xxx.xxxxx#xxx.it>
To: <xxxxx#xxx.xxxxx.it>
References:
In-Reply-To:
Subject: fatture xxx
Date: Wed, 28 Dec 2016 17:52:16 +0100
Message-ID: <!&!AAAAAAAAAAAYAAAAAAAAAHCgtCXXdtBNl+uXnP8eDHPCgAAAEAAAAGutlHZJGkpAhmPV6f+ofh8BAAAAAA==#eng.it>
MIME-Version: 1.0
Content-Type: multipart/mixed;
boundary="----=_NextPart_000_0092_01D26133.2135B540"
X-Mailer: Microsoft Office Outlook 12.0
Thread-Index: AdCOWOCYZlK03GM4S5Cd5xo9GuRjGAAorxQwBlBUrvAFQ/QDgAew8EzACjCMyCAGU/qYMAV4vr+QBuilgWAKzjrI4AagZVDwBqrsgiAABfH4YAXsTykQBafYYmAEIBNvwBSv7j/wBgnvoHAB0wKvkA==
Content-Language: it
X-KLMS-Rule-ID: 1
X-KLMS-Message-Action: clean
X-KLMS-AntiSpam-Status: not scanned, disabled by settings
X-KLMS-AntiPhishing: not scanned, disabled by settings
X-KLMS-AntiVirus: Kaspersky Security 8.0 for Linux Mail Server, version 8.0.1.721, bases: 2016/12/28 09:52:00 #8723401
X-KLMS-AntiVirus-Status: Clean, skipped
X-IControl-Milter-Checked: yes (IControlServer: CEDACRIPEC1-NEW SrcIpType: UnTrusted SrcIpHeaderType: Undefined Authenticated: no)
X-IControl-Milter-SPF-Checked: yes (IControlServer: CEDACRIPEC1-NEW HeloSPFType: none FromSPFType: pass HeloHeaderSPFType: Undefined FromHeaderSPFType: pass)
X-IControl-Milter-MD5SUM: 30378f3a90c4e4051ecd9eab79feaa02
They hold the same ID but the two emails, trust me, belong to different transmissions. In addition they hold different attachments.
Can you tell me why the email IDs are the same?
You can get different messages with the same message-Id. Here's how:
Message-Id 1 is generated when user A sends message1 to B and C.
B is a final destination, which accepts the message and stores it.
C is a mailing list. The mail message is held for moderation. At some later point in time, the mail list moderator accepts the message. The message is modified with the header and footer of the mailing list and sent out. B is on the mailing list and receives a copy.
Now B has two copies of message1, one without the mailing list info, one with it.
I'll take a shot at this.
RFC 822 specifies that:
"The uniqueness of the message identifier is guaranteed by the host which generates it."
It also specifies under section 4.2:
Some systems permit mail recipients to forward a message,
retaining the original headers, by adding some new fields. This
standard supports such a service, through the "Resent-" prefix to
field names.
So:
It sounds like the host email program is not changing the said ID.
message-id is optional, but ESMTP indicates that servers should add
it if missing during a transaction. (https://www.rfc-editor.org/rfc/rfc2821)
forwarding messages SHOULD add "resent-", but that's up to the code within the client/server.
The ESMTP ID change seems to indicate different email servers... and the subject's typo feels contrived, even spam-like. (isn't "fatture Engineering" a mixture of Italian and English for "Engineering Invoice"?)
If it's spam, I'd expect message IDs to be similar even if they're coming from the same server since they are, in fact, the same message.
Do you have more information about the purpose/content of this email?
Related
I use EWS GetItem endpoint in order to get MIME content of e-mail message in Outlook. As a rule several 'Received' header fields are present at the beginning of message content. For example:
Received: from AM6PR04CA0035.eurprd04.prod.outlook.com
(2603:10a6:20b:92::48) by AM6PR10MB2789.EURPRD10.PROD.OUTLOOK.COM
(2603:10a6:20b:ac::25) with Microsoft SMTP Server (version=TLS1_2,
cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3348.15; Thu,
10 Sep 2020 11:36:56 +0000 Received-SPF: Pass
(protection.outlook.com: domain of alterdomus.com designates
51.137.107.73 as permitted sender) receiver=protection.outlook.com; client-ip=51.137.107.73;
helo=smtpworker-in-9.xware-europe-west-1.o365.crossware.co.nz;
Received: from VI1PR10MB3616.EURPRD10.PROD.OUTLOOK.COM
([fe80::204a:2512:2ac2:d5ef]) by VI1PR10MB3616.EURPRD10.PROD.OUTLOOK.COM
([fe80::204a:2512:2ac2:d5ef%3]) with mapi id 15.20.3305.031; Tue, 25 Aug 2020
17:51:53 +0000
Received: from EUR02-AM5-obe.outbound.protection.outlook.com
(104.47.4.57) by
smtpworker-in-9.xware-europe-west-1.o365.crossware.co.nz with
Crossware for Office365; Thu, 10 Sep 2020 11:36:46 +0000
However in some cases those 'Received' fields are missed for some messages. As far as I understand this field is very important for spam detection. It contains information about the sender, the receiver and the received time of a message. This header is added by the receiving mail server as the top line. Depending on the number of mail servers involved, several 'Received' header fields may be included.
What can be the reason of missed 'Received' header fields inside e-mail MIME content? What does it mean?
Not all messages that are in a Office365 mailbox will have been routed via the Transport stack (most will). One example is the MyAnalytics Messages, the Office365 substrate generates these messages and they are created directly in the Mailbox by the substrate rather then being delivered. You can use something like the Message Header analyzer to check https://appsource.microsoft.com/en-us/product/office/WA104005406?src=office&corrid=f7f9ed61-d151-4fcf-b0e2-9e3dec4a92d2&omexanonuid=034f54c1-a89b-429c-adc3-526f6f024f80&referralurl=https%3a%2f%2fwww.google.com%2f to check the headers it grabs the PidtagTransportheaders property https://learn.microsoft.com/en-us/office/client-developer/outlook/mapi/pidtagtransportmessageheaders-canonical-property which is a more reliable way of getting the headers then trying to parse it back out of the Mime Message
There is a problem accessing the To alias from a o365 account IF the from account is also o365. If the from account is say, gmail, it works.
If I send an email to alias#mycompany.com which is an alias to realAccount#mycompany.com, if I examine the To header in Outlook, it will always show me the original alias. If I view the header progrmatically, it will NOT show the alias if it was sent from an o365 account. Instead, it shows the real account. If I do this same test with a gmail instead of an o365 email it works -- shows the alias in the To: header as expected.
How does Outlook access this data? The number of headers are different too. Outlook contains more data. Has anyone experienced this? Any ideas on how to access the alias like Outlook does?
Header when accessing from Outlook:
From: o365Account#somecompany.com
To: ***************** alias#mycompany.com ****************
Subject: shdaKJSDHA
Thread-Topic: shdaKJSDHA
Thread-Index: AQHUSTkz1fQhzI5SG0ie26mNIvHmmQ==
Date: Mon, 10 Sep 2018 19:05:12 +0000
Message-ID: <---#-----.prod.outlook.com>
Header when accessed programatically:
From: o365Account#company.com
To: *****************realAccount#mycompany.com ****************
Subject: shdaKJSDHA
Thread-Topic: shdaKJSDHA
Thread-Index: AQHUSTkz1fQhzI5SG0ie26mNIvHmmQ==
X-MS-Exchange-MessageSentRepresentingType: 1
Date: Mon, 10 Sep 2018 19:05:12 +0000
Message-ID: <----#-----.prod.outlook.com>
Keep in mind that Exchange always resolves the sender and recipients to their primary addresses both when sending and when receiving the messages. This is just the way it works.
Are you sending through SMTP?
You know what I ended up doing? I made a Distro list with 1 recipient. The distro list takes the place of the alias. It always shows the To: as the distro list. That way it doesn't get lost. Thank you for helping me understand Exchange Server dmitry-streblechenko.
My Rails 3 application sends out emails in both plain text and HTML formats. I have tested it locally using RoundCube and Squirrel Mail clients and they both display HTML version with images, links, etc. GMail on the other hand chooses plain text format. Any idea what's causing this?
Delivered-To: test#gmail.com
Received: by 10.42.166.2 with SMTP id m2cs16081icy;
Thu, 3 Mar 2011 17:01:48 -0800 (PST)
Received: by 10.229.211.138 with SMTP id go10mr1544841qcb.195.1299200507499;
Thu, 03 Mar 2011 17:01:47 -0800 (PST)
Return-Path: <info#example.com>
Received: from beta.example.com (testtest.test.com [69.123.123.123])
by mx.google.com with ESMTP id j14si1690118qcu.136.2011.03.03.17.01.46;
Thu, 03 Mar 2011 17:01:46 -0800 (PST)
Received-SPF: neutral (google.com: 69.123.123.123 is neither permitted nor denied by best guess record for domain of info#example.com) client-ip=69.123.123.123;
Authentication-Results: mx.google.com; spf=neutral (google.com: 69.123.123.123 is neither permitted nor denied by best guess record for domain of info#example.com) smtp.mail=info#example.com
Received: from localhost.localdomain (localhost [127.0.0.1])
by beta.example.com (Postfix) with ESMTP id F3C273A3EC
for <test#gmail.com>; Fri, 4 Mar 2011 01:01:45 +0000 (UTC)
Date: Fri, 04 Mar 2011 01:01:45 +0000
From: info#example.com
To: test#gmail.com
Message-ID: <4d7039f9e9d3e_3449482ab7831658#test.mail>
Subject: Your example account was activated.
Mime-Version: 1.0
Content-Type: multipart/alternative;
boundary="--==_mimepart_4d7039f9e6967_3449482ab7831370";
charset=UTF-8
Content-Transfer-Encoding: 7bit
----==_mimepart_4d7039f9e6967_3449482ab7831370
Date: Fri, 04 Mar 2011 01:01:45 +0000
Mime-Version: 1.0
Content-Type: text/html;
charset=UTF-8
Content-Transfer-Encoding: 7bit
Content-ID: <4d7039f9e95ed_3449482ab7831519#test.mail>
<html>
<head>
<meta content="text/html; charset=UTF-8" http-equiv="Content-Type" />
</head>
<body>
<p><img border="0" src="http://example.com/images/logo.png" alt="example logo" /></p>
<p>Congratulations, Test!</p>
<p>
Your <a style="text-decoration:none;color:#ef4923;" href="http://example.com/">example</a> account was activated.
</p>
</body>
</html>
----==_mimepart_4d7039f9e6967_3449482ab7831370
Date: Fri, 04 Mar 2011 01:01:45 +0000
Mime-Version: 1.0
Content-Type: text/plain;
charset=UTF-8
Content-Transfer-Encoding: 7bit
Content-ID: <4d7039f9e8b0e_3449482ab78314b7#test.mail>
Congratulations, Test!
Your example.com account was activated.
----==_mimepart_4d7039f9e6967_3449482ab7831370--
Try switching the order of the parts of the message, putting the HTML part after the plain-text part. It might work :).
NOTE: I cannot remember now where I read this (or if I for sure even
did), but the reason switching might help is because I think the
preferred part of the message may be the last part.
Update: I found a place where it says that parts in a multipart MIME message should be in order of increasing preference -- here, in section 7.2.3 (edit: latest version here; thanks #ALEXintlsos!), starting with the third to last paragraph.
Update: Here is a quote of section 7.2.3, (see https://stackoverflow.com/help/referencing):
7.2.3 The Multipart/alternative subtype
The multipart/alternative type is syntactically identical to multipart/mixed,
but the semantics are different. In particular, each of the parts is an
"alternative" version of the same information. User agents should recognize
that the content of the various parts are interchangeable. The user agent
should either choose the "best" type based on the user's environment and
preferences, or offer the user the available alternatives. In general, choosing
the best type means displaying only the LAST part that can be displayed. This
may be used, for example, to send mail in a fancy text format in such a way
that it can easily be displayed anywhere:
From: Nathaniel Borenstein <nsb#bellcore.com>
To: Ned Freed <ned#innosoft.com>
Subject: Formatted text mail
MIME-Version: 1.0
Content-Type: multipart/alternative; boundary=boundary42
--boundary42
Content-Type: text/plain; charset=us-ascii
...plain text version of message goes here....
--boundary42
Content-Type: text/richtext
.... richtext version of same message goes here ...
--boundary42
Content-Type: text/x-whatever
.... fanciest formatted version of same message goes here
...
--boundary42--
In this example, users whose mail system understood the "text/x-whatever"
format would see only the fancy version, while other users would see only the
richtext or plain text version, depending on the capabilities of their system.
In general, user agents that compose multipart/alternative entities should
place the body parts in increasing order of preference, that is, with the
preferred format last. For fancy text, the sending user agent should put the
plainest format first and the richest format last. Receiving user agents should
pick and display the last format they are capable of displaying. In the case
where one of the alternatives is itself of type "multipart" and contains
unrecognized sub-parts, the user agent may choose either to show that
alternative, an earlier alternative, or both.
NOTE: From an implementor's perspective, it might seem more sensible to reverse
this ordering, and have the plainest alternative last. However, placing the
plainest alternative first is the friendliest possible option when
multipart/alternative entities are viewed using a non-MIME- compliant mail
reader. While this approach does impose some burden on compliant mail readers,
interoperability with older mail readers was deemed to be more important in
this case.
It may be the case that some user agents, if they can recognize more than one
of the formats, will prefer to offer the user the choice of which format to
view. This makes sense, for example, if mail includes both a nicely-formatted
image version and an easily-edited text version. What is most critical, however,
is that the user not automatically be shown multiple versions of the same data.
Either the user should be shown the last recognized version or should
explicitly be given the choice.
I'm trying to sort out an opt in mailing list system. I understand the basic principles and design required but i'm having an issue with it being picked up as spam.
If i send a html email through outlook through email#domain.com it works fine and is not treated as spam. When i use the Zend_mail object to send mail it sends but is treated as spam on the test emails accounts i'm sending it too.
This is the code im using to send an email item.
//send an email
$mail = new Zend_Mail();
$config = array('auth' => 'login','username' => 'email#domain.com','password' => 'mypassword');
$transport = new Zend_Mail_Transport_Smtp('mail.domain.com', $config);
$mail->setSubject($item->title);
$mail->setFrom("email#domain.com");
$mail->addTo($item->email, $item->forename);
//$mail->setBodyText($item->contentPlain);
$mail->setBodyHtml($item->contentHTML);
$mail->send($transport);
As you can see im using the smtp transport object to authenticate but it still seems to treat this as spam. Anyone with pointers or tips is very much appreciated!!
Header info from the email that is treated as spam:
It seems to contain a couple of client domain names in the header info that i host for people any ideas why that would be the case? I use a shared IP address with about 10 domains on it
Received: (qmail 1436 invoked from network); 14 Aug 2009 16:02:10 +0100
Received: from clientdomain1.co.uk (HELO localhost) (91.192.***.196)
by clientdomain2.info with SMTP; 14 Aug 2009 16:02:10 +0100
Subject: Manchester 2 Day Seminar: Dealing with difficult people
From: events#domain.com
To: Andi <subscriber1#domain.com>
Date: Fri, 14 Aug 2009 15:02:10 +0000
Content-Type: text/html; charset=iso-8859-1
Content-Transfer-Encoding: quoted-printable
Content-Disposition: inline
MIME-Version: 1.0
"Roll your own mail" is often treated as spam by large hosted email systems. When you use a paid service to send out mass emails, you are paying for those company's agreements with the major email vendors to keep them white-listed.
One thing you can do, though, is to ensure that the account you are sending from exists and the email is being sent from a matching domain (e.g. #foo.com sent from foo.com's smtp server). That is a big red flag for spam filters.
Compare your email and email from outlook. Are any headers missing? Which? Do they seem significant?
Try this to get rid of the last localhost reference:
$protocol = new Zend_Mail_Protocol_Smtp('localhost');
$protocol->connect();
$protocol->helo('mail.yourserver.com'); //**DO THIS**
$transport->setConnection($protocol);
I learning Perl and doing a home made project to my family (a subscription project). The Perl application that uses Net::POP3 connect to my mailbox and save all my emails to a file (Mail.txt). When I open this file I see a lot of junk, as below. What i can do to remove this? Thanks.
Return-Path:
Received: from [unix socket] by embro.tpn.terra.com (LMTP); Sun, 11 Oct 2009 04:09:50
+0000 (UTC)
X-Abaca-Spam: 153
X-Terra-Karma: -2%
X-Terra-Hash: 2c7d32f717e807b11af5c0871edb9e93
Received-SPF: pass (embro.tpn.terra.com: domain of linuxquestions.org designates
208.101.3.244 as permitted sender) client-ip=208.101.3.244;
envelope-from=forum#linuxquestions.org; helo=sql02.linuxquestions.org;
Received: from sql02.linuxquestions.org (smtp.linuxquestions.org [208.101.3.244])
by embro.tpn.terra.com (Postfix) with ESMTP id 14EA1580000A2
for ; Sun, 11 Oct 2009 04:09:49 +0000 (UTC)
Received: from web02.linuxquestions.org (web02-be.linuxquestions.org [10.13.156.4])
by sql02.linuxquestions.org (8.13.8/8.13.8) with ESMTP id n9B49mXe005694
for ; Sun, 11 Oct 2009 00:09:48 -0400
DomainKey-Signature: a=rsa-sha1; s=smtp; d=linuxquestions.org; c=simple; q=dns;
b=Le/RccpkHMfH426hLwlLkIbCujr0LiWKM32ryuZ1fWwYU6VjCTocd30N/JAg+w77d
54VJkNnpA18iQxJ/yfKyQ==
Received: from web02.linuxquestions.org (localhost.localdomain [127.0.0.1])
by web02.linuxquestions.org (8.13.8/8.13.8) with ESMTP id n9B49m2f027957
for ; Sun, 11 Oct 2009 00:09:48 -0400
Received: (from nobody#localhost)
by web02.linuxquestions.org (8.13.8/8.13.8/Submit) id n9B49mNn027956;
Sun, 11 Oct 2009 00:09:48 -0400
Date: Sun, 11 Oct 2009 00:09:48 -0400
To: nathanpc#terra.com.br
Subject: "What programs would you like to see ported to Linux?" update
From: "LinuxQuestions.org"
Auto-Submitted: auto-generated
Message-ID:
X-Priority: 3
X-Mailer: LQ Mailer
Content-Type: text/plain; charset="ISO-8859-1"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Status: O
Dear nathanpc,
It's not junk. It's email header. Use, for example, Mail::Message to parse it. Something like this:
my $msg_obj = Mail::Message->read($rawdata); my $body = $msg_obj->body;
You know, I did recommend Mail::POP3Client which abstracts away the details:
Body( MESSAGE_NUMBER )
Get the body of the specified message, either as an array of lines or as a string, depending on context.
BodyToFile( FILE_HANDLE, MESSAGE_NUMBER )
Get the body of the specified message and write it to the given file handle.
Email headers consist of all text up to the first completely blank line. So, if you actually do want to throw them away (rather than using a good module to parse them as the earlier examples suggested), just throw away everything up to and including the first blank line.
If you're looking at an mbox-format mailbox file containing multiple messages, you can identify the start of the next message's headers by looking for a line which starts with the five characters "From " (note the trailing space - this is what distinguishes it from a "From:" header).