From what I've read, every Message-ID must be unique, however it is possible to create repeated Message-IDs if we force the header with a fixed value. So I don't understand what the point of them saying that the Message-ID should be unique, but they are very easy to create duplicates. If they can easily be generated by anyone with a little reading and basic programmatic knowledge, why do Message-IDs exist and what are they used for, which I can easily duplicate?
Short answer: For threading in email clients.
The message-id header is defined in RFC 2822:
The "Message-ID:" field contains a single unique message identifier.
The "References:" and "In-Reply-To:" field each contain one or more
unique message identifiers
The message ID is used to show which message is a reply to which other message, for example. That way mail clients can show a tree of emails with their replies even if other things like the Subject don't change. (Counting leading Re:s of the subject line would be a bad way to determine ancestors and children: not every mail client adds them, and some use language specific ones.)
https://datatracker.ietf.org/doc/html/rfc5322#section-3.6.4
in conjunction with the References and In-Reply-To fields, mail clients use Message-ID to organize multiple messages into threads.
https://en.wikipedia.org/wiki/Message-ID
and at least some clients will consider two messages with the same ID to be the same thing and discard one of them.
Related
Per RFC 2046:
The inclusion of a "References" field in the headers of the second and subsequent pieces of a fragmented message that references the Message-Id on the previous piece may be of benefit to mail readers that understand and track references. However, the generation of such "References" fields is entirely optional.
I see this. It does not explicitly mention how it is usually used in practice. Does anybody have a summary of common email clients and online email service providers (e.g., Gmail) on how this is implemented in practice?
Mail user agents usually display threads of Emails by chaining messages together according to the In-Reply-To and References header fields that contain the Message-IDs of other messages. Although a mail usually only replies to one other message, it may be the case that one message answers multiple others. The standard allows multiple entries in both fields. What can I expect when I send an email that References or is In-Reply-To multiple IDs this way?
Is it good practice to do so?
Does it confuse widespread MUAs?
Is there any common ground on how to display such a message in a
threaded view?
The "In-Reply-To:" field will contain the contents of the
"Message-ID:" field of the message to which this one is a reply (the
"parent message"). If there is more than one parent message, then
the "In-Reply-To:" field will contain the contents of all of the
parents' "Message-ID:" fields. If there is no "Message-ID:" field in
any of the parent messages, then the new message will have no "In-
Reply-To:" field.
Technically there COULD be a reason where you would reply to multiple emails and it would be valid to place multiple message ids in the In-Reply-To header. I can’t think of any program that actually supports this. As to MUAs they won’t care the delivery that the MUA cares about is the To, Cc, Bcc headers.
The In-Reply-To header and References header would control how threads are displayed. Not sure if any mail clients would have an issue handling the multiple In-Reply-To headers. 99% of the time there would only be a single message ID in the In-Reply-To header. So it’s feasible mail applications won’t support it. However they would support the additional reference entries. And this shouldn’t pose an issue.
We're considering using an external application to send e-mails where the attachments have huge filesizes.
One of our users states the requirement: it must be possible to ORDER attachments, so that I can refer to them as "the first attachment", "the second attachment" and the receiver will see the attachments in the same order I sent them.
She claims any normal email clients support this: the sender chooses the order, the recipients client always shows the attachments in this order.
I can't verify that claim. Is this true? Is it stated anywhere? How (technically) are attachments ordered?
Well... Inside the message there is order. Attachments are sent sequentially one-after-the-other inside MIME text. But RFC makes no statement on how these should be displayed in email client. And it even might depend on your email generating software to keep the order you added the attachments in (JavaMail does so)
Things you could consider:
Prefix your attachment filenames with a number 1_first_attachment.txt 2_contract.pdf and so on.
Use HTML for your emails and create a section "attachments: a, b, c" with linkd to the attachments using Content-ID (href="cid:xxx")
Actual, when u load the document to the email, and load next, they gonna be in order anyway. They not gonna be mixed.
if u afraid that particular ur email gonna mix attachments, just add numerical order)
This is example where i just adding files. I just by holding shift click on the files in that order that i wanted and that is all.
And i receive this message with attachments in the same order as it was sent.
I have recently been working with javamail. Right now, I am trying to store all mails in a file. For such a thing, one would need a unique ID, so I assumend UID would fit best here. However, I noticed something odd: A mail in the "Inbox" Folder with the subject "Hello" has the UID 10. If I fetched the same message from the "All Messages" folder, I'd get the same Message(because I am in "All Messages") with the same content, but with a different UID.
This isn't actually that much of a problem, however, is it possible that two completely different mails from different folders might have the same UID? In this case, I would have to overthink the way i store mails.
Thanks in advance.
The UIDs are not JavaMail UIDs, they're IMAP UIDs, defined by the IMAP RFC.
The UIDs are unique per-folder, based on the UIDVALIDITY value for the folder. There is no unique ID for the folder itself.
Depending on your needs, you might consider using the Message-ID for the message, although note that while it's very, very likely to be unique, there's no guarantee that it is unique, and there's no guarantee that it exists for every message.
I am writing a C# application which allows users to store emails in a MS SQL Server database. Many times, multiple users will be copied on an email from a customer. If they all try to add the same email to the database, I want to make sure that the email is only added once.
MD5 springs to mind as a way to do this. I don't need to worry about malicious tampering, only to make sure that the same email will map to the same hash and that no two emails with different content will map to the same hash.
My question really boils down to how one would combine multiple fields into one MD5 (or other) hash value. Some of these fields will have a single value per email (e.g. subject, body, sender email address) while others will have multiple values (varying numbers of attachments, recipients). I want to develop a way of uniquely identifying an email that will be platform and language independent (not based on serialization). Any advice?
What volume of emails do you plan on archiving? If you don't expect the archive require many terabytes, I think this is a premature optimization.
Since each field can be represented as a string or array of bytes, it doesn't matter how many values it contains, it all looks the same to a hash function. Just hash them all together and you will get a unique identifier.
EDIT Psuedocode example
# intialized the hash object
hash = md5()
# compute the hashes for each field
hash.update(from_str)
hash.update(to_str)
hash.update(cc_str)
hash.update(body_str)
hash.update(...) # the rest of the email fields
# compute the identifier string
id = hash.hexdigest()
You will get the same output if you replace all the update calls with
# concatenate all fields and hash
hash.update(from_str + to_str + cc_str + body_str + ...)
How you extract the strings and interface will vary based on your application, language, and api.
It doesn't matter that different email clients might produce different formatting for some of the fields when given the same input, this will give you a hash unique to the original email.
Have you looked at some other headers like (in my mail, OS X Mail):
X-Universally-Unique-Identifier: 82d00eb8-2a63-42fd-9817-a3f7f57de6fa
Message-Id: <EE7CA968-13EB-47FB-9EC8-5D6EBA9A4EB8#example.com>
At least the Message-Id is required. That field could well be the same for the same mailing (send to multiple recipients). That would be more effective than hashing.
Not the answer to the question, but maybe the answer to the problem :)
Why not just hash the raw message? It already encodes all the relevant fields except the envelope sender and recipient, and you can add those as headers yourself, before hashing. It also contains all the attachments, the entire body of the message, etc, and it's a natural and easy representation. It also doesn't suffer from the easily generated hash collisions of mikerobi's proposal.