When sending an email, the recipient list is given during the SMTP dialogue through RCTP TO: command. Later, in DATA command, header fields like 'To', 'Cc','bcc' are indicated. Does this RCPT TO list of recipients have to match with the headers indicated in DATA command?
Also, if the recipient is not indicated in RCPT TO, but in the To field of email header, is it going to be delivered to the recipient not in RCPT TO?
No, they don't have to match. When the message is sent, the SMTP Server (aka Message Transfer Agent or MTA) is creating a so called SMTP envelope which contains the recipients and the sender of the message (see RFC5321):
SMTP transports a mail object. A mail object contains an envelope and content. The SMTP envelope is sent as a series of SMTP protocol
units (described in Section 3). It consists of an originator
address (to which error reports should be directed), one or more
recipient addresses, and optional protocol extension material.
It is, actually, quite often that the RCPT TO: Command has more recipients that the header of the message - one common case is the usage of "blind copies" bcc: (see RFC5321):
Addresses that do not appear in the message header section may appear
in the RCPT commands to an SMTP server for a number of reasons. The
two most common involve the use of a mailing address as a "list
exploder" (a single address that resolves into multiple addresses) and
the appearance of "blind copies".
Does this RCPT TO list of recipients have to match with the headers
indicated in DATA command?
Nope.
if the recipient is not indicated in RCPT TO, but in the To field of
email header, is it going to be delivered to the recipient not in RCPT
TO ?
The RCPT. Here's a (modified) transcript from my own SMTP client where I do just what you ask:
CLIENT: MAIL FROM:<myaccount#gmail.com>
SERVER: 250 2.1.0 OK
CLIENT: RCPT TO:<myaccount#gmail.com>
SERVER: 250 2.1.5 OK
CLIENT: DATA
SERVER: 354 Go ahead
CLIENT: Subject: Test email
CLIENT: From:'John Doe'<fakeaccount#gmail.com>
CLIENT: To:'John Doe'<fakeaccount#gmail.com>
CLIENT: This is a test...
CLIENT: .
The message was successfully sent to "myaccount#gmail.com".
SMTP protocol (RFC 2821) states the following:
When RFC 822 format [7, 32] is being used, the mail data include the
memo header items such as Date, Subject, To, Cc, From. Server SMTP
systems SHOULD NOT reject messages based on perceived defects in the
RFC 822 or MIME [12] message header or message body.
And this:
The DATA command can fail at only two points in the protocol
exchange:
If there was no MAIL, or no RCPT, command, or all such commands
were rejected, the server MAY return a "command out of sequence"
(503) or "no valid recipients" (554) reply in response to the DATA
command. If one of those replies (or any other 5yz reply) is
received, the client MUST NOT send the message data; more
generally, message data MUST NOT be sent unless a 354 reply is
received.
From these statements, the headers and RCPT TO: command content does not have to match (altough they should match), and not using RCPT TO: MAY result in an error to prevent proceeding with DATA command.
Related
In RFC 821, it says that a reset (RSET) command can be sent after a DATA command and some mail data has been sent:
However, what distinguishes between a mail client sending an RSET command after DATA, and a mail that contains the word "RSET" on a line by itself?
I've checked RFC 5321 as well and I can't see anything that would mitigate or escape this. It does talk about escaping a mail line which starts with a ".", but not "RSET".
The client cannot terminate the mail data transfer with a period on a line by itself or the server will send the partial mail it has been given.
I imagine there's something I've missed in the RFCs, otherwise I can't help thinking that there's either an SMTP command injection attack vector in many implementations, or no-one can ever send a mail with "RSET" on a line by itself (I think people would have noticed).
The keyword here is after I believe. The DATA command is in progress until it is finished with a lone . on a line.
RFC 5321 § 4.1.1.5 (RSET) states "any stored sender, recipients, and mail data MUST be discarded." This refers to the MAIL FROM, RCPT TO, and presumably DATA commands.
However, upon receiving the . following DATA, the message "MUST" be delivered (which may result in a failure but not a partial failure, see § 4.1.1.4). This clears the buffer of everything RSET is supposed to do.
This means RSET merely elicits a 250 OK response from the receiving server (a keep-alive, much like NOOP) and confirms to the sender that there is indeed no saved sender or recipient queued for the next message.
I do not know of a way to interrupt a DATA command to issue a RSET. The only way I know of to do that is to terminate the connection and establish a new one—and, just to be safe in the case of some odd resumption capability, I'd issue an RSET right after the EHLO or HELO (which the spec says is a NOOP). If there were such a way, it should be in RFC 5321 § 4.1.1.4, § 4.1.1.5, and/or § 3.3.
Which SMTP code must the server return when the Email could not be delivered to all recipients?
I mean the case when there are multiple recipient.
Let's say there are RCPT A, RCPT B, RCPT C and the end of the DATA the server can deliver to A and B, but not to C.
As a server, I must not response with code 250. Right? Then which code must I response with?
And the other hand, I neither can give them code 5xx, because they may double the Email for A and B.
In typical implementation/scenario:
server refuses to accept "obviously" invalid recipients in reply to RCPT TO:
(e.g. non existing DNS domain or non existing user/mailbox in local email domain)
accepts message in reply to the final dot (or rejects it as spam)
tries to send accepted message to recipients
sends back "bounce messages" to envelope sender (MAIL FROM:) after repeated/multiple delivery attempts fail
Here is an example of how I have configured sieve to forward any mail sent to [nameA|nameB|nameC]#example.org to my private email address.
if address :localpart :is ["To","Cc","Bcc"]
["nameA", "nameB", "nameC"] {
redirect "<my private email address>";
stop;
}
Sometimes though, email is not forwarded because the address that it was sent to is tucked away somewhere in a "Received" header.
Received: from ###server### ([###ip_address###])
by ###server### with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA384:256)
(Exim 4.84)
(envelope-from <###email_address###>)
id 1alDM0-0000yT-60
for nameA#example.org; Wed, 30 Mar 2016 12:28:00 +0200
Is there an effective way to catch these emails in the sieve rule, too?
You have an XY Problem here. What you actually want to do here is filter based on the address being delivered to, not the address in the headers. (As unintuitive as it may be, the address in the headers may have nothing to do with the address it's being delivered to, which is how Bcc can work at all.)
The command to test against the actual SMTP envelope is envelope.
require "envelope";
if envelope :localpart :is "to" ["nameA", "nameB", "nameC"] {
redirect "<my private email address>";
stop;
}
This will handle all mail being delivered to those names, regardless of whether they show up in the mail headers or not.
With the help of the sieves' index feature you can parse the recipient address out of the Received headers.
For BCC sorting I typically do something like this:
require ["fileinto", "envelope", "variables", "mailbox", "index", "subaddress"];
...
if header :index 3 :matches "Received" "*<*#example.com>*" {
set :lower "foldername" "${2}";
fileinto :create "inbox.${foldername}";
} elsif header :index 2 :matches "Received" "*<*#example.com>*" {
set :lower "foldername" "${2}";
fileinto :create "inbox.${foldername}";
}
...
In the Received headers of the mails I receive the adress is set in angle brackets and that's why I've chosen the pattern in the example above.
Additionally, sometimes the number of Received headers varies thus I test at least for two different ones.
We used a SMTP server to receive emails, It works perfectly when I send an email to my server from gmail or hotmail. But I'm having problems with a specific company ( I will name it company x), we can read the sender, the recipient, etc, but when it comes to the DATA, the buffered reader hangs forever reading the line on the socket.
This is what happen when I receive the email from the company x:
When I receive a company x's email. they send me the EHLO command
my server returns 500 command unrecognised
the company x's server send the HELO command
my server sends 250 ok to the company x's server.
the copany x's server send me MAIL From: <sender#email.com>
my server sends 250 ok
the company x's server send me RCPT To:<recipient#email.com>
my server sends 250 ok
the company x's server sends DATA
my server sends 354 Start mail input; end with <CRLF>.<CRLF>
At this moment the server hangs forever reading the incoming data (in the in.readline()) and my server throws socket time out exception. (obviously we tried increasing the time out, but it didn't work)
what could be the difference with the company x's SMTP server and the gmail or hotmail server??, what is the problem???
We have the same error with the Java mail server and the james mail server.
The company x, is a bank so they have a high information security level.
here is the code how we send it.
private static final String MESSAGE_SEND_DATA = "354 Start mail input; end with <CRLF>.<CRLF>";
and this is the method that write the comand on the outputstream.
private void write( String message ) {
if( log.isDebugEnabled() ) { log.debug( "Writing: " + message ); }
out.print( message + "\r\n" );
out.flush();
}
we call write after we receive the data commanx from the client.
Most likely it hangs forever awaiting for line terminator symbol. BufferedReader#readLine is platform dependent in terms of line terminators, so it's a poor Reader choice for SMTP server implementation. As a matter of fact, I would advice not to use BufferedReader and PrintWriter for socket-originated streams to avoid platform-dependent issues like this one.
All SMTP server implementations in Java I'm aware of (Apache James, SubethaSMTP) use dedicated Reader implementations, focused on CRLF-terminated lines to read lines from a Socket, in full accordance with RFC 5321. So my advice would be to try one of these readers (for example this one) instead of BufferedReader.
What is SMTP Envelope and SMTP header and what is the relationship between those? How do I extract them with Perl?
An SMTP message contains a set of headers such as From, To, CC, Subject and a whole range of other stuff.
An SMTP Envelope is simply the name given to a small set of header prefixed to the standard SMTP message when the message is moved about by the Message Transport Agent (ie. the SMTP server). The most common envelope headers are X-Sender, X-Receiver and Received.
For example Microsofts SMTP Server will add the X-Sender and a series of X-Receiver headers to the top of a message when it drops the message into its Drop folder. There will be one X-Receiver for each post box that matches the domain the Drop folder is for.
Another example is SMTP servers add a Receive: header when it receives a message from another SMTP server. This header gives various details of the exchange. Hence most emails on the tinternet once arrived at the final destination will have a series of Receive headers indicating the SMTP server hops the message took to arrive. Usually servers remove the X-Sender, X-Receiver headers when the message is finally moved to a POP3 mailbox.
Accessing Headers
On the windows platform the only way I've found to access the envelope headers is to simply open and parse the eml file. Its a pretty simple format (name: value CR LF).
Again on the windows platform the main set of message headers and body parts can be accessed using the CDOSYS.dll COM based set of objects. How you would do this on other platforms I don't know. However the header format is quite straight forward as per the envelope headers, its accessing the body parts that would require more creative coding.
The envelope is the addressing information sent to the server during the initial conversation via the "MAIL FROM:" and "RCPT TO:" commands.
The SMTP header is the collection of header lines which are sent after the DATA command is issued.
How you find them is dependant on how/where you're getting the message from, and we'd need a lot more clues to attempt to answer that.
You can actually think of three different things here. There are the directives that were exchanged between the SMTP MTAs (during each hop the message took) ... the headers that were generated by the MUA and headers that were added (or modified) by MTAs along the route that a given message traversed.
The "envelope" refers to the information provided to the MTA (normally the most recent or final destination MTA). The sender includes a set of headers after the DATA directive in the SMTP connection (separated from the body of the message by a blank line ... but double check the RFC if that's specifically supposed to be a CR/LF pair). Note that the local MTA may add additonal headers and might even modify some headers before storing or forwarding the message.
(Normally it should only add Received-by: headers).
Some MTAs are configured to add X-Envelope-To: and/or X-Envelope-From: headers. Some of them will still filter the contents of these headers (for example to prevent leakage of blind copies). (Senario: the original MUA had a BCC: line directory that a number of people be copied on the message with their names all appearing to one another in the CC: headers; for each recipient domain (MX result) the MTA will only issue RCPT TO: for only the subset of addresses for which the host if the appropriate result (its own hub, smarthost, or any valid MX for the target) --- thus any subsets of recipients who share an MX with each other would see leakage in the X-Envelope-To: headers generated by MTAs that were sloppy about the handling of this detail).
Also not that an Envelope-From line would only contain a host/domain name as supplied by the HELO FROM: or EHLO FROM: directives in the SMTP exchange. It cannot be used as a return address, for replies for example.
For Perl email related stuff have a look at the Perl Email Project.