How to accurately parse smtp message status code (DSN)? - email

RFC1893 claims that status codes will come in the format below you can read more here.
But our bounce management system is having a hard time parsing error status code from bounce messages. We are able to get the raw message, but depending on the email server the code will come in different places. Is there any rule on how to parse this type of messages to obtain better results. We are not looking for the 100% solution but at least 80%.
This document defines a new set of status codes to report mail system
conditions. These status codes are intended to be used for media and
language independent status reporting. They are not intended for
system specific diagnostics.
The syntax of the new status codes
is defined as:
status-code = class "." subject "." detail
class = "2"/"4"/"5"
subject = 1*3digit
detail = 1*3digit
White-space characters and comments
are NOT allowed within a status-
code. Each numeric sub-code within
the status-code MUST be expressed
without leading zero digits.
The quote above from the RFC tells one thing but then the text below from a leading tool on bounce management says something different, where I can get a good source of standard status codes:
Return Code Description
0 UNDETERMINED - (ie. Recipient Reply)
10 HARD BOUNCE - (ie. User Unknown)
20 SOFT BOUNCE - General
21 SOFT BOUNCE - Dns Failure
22 SOFT BOUNCE - Mailbox Full
23 SOFT BOUNCE - Message Too Large
30 BOUNCE - NO EMAIL ADDRESS. VERY RARE!
40 GENERAL BOUNCE
50 MAIL BLOCK - General
51 MAIL BLOCK - Known Spammer
52 MAIL BLOCK - Spam Detected
53 MAIL BLOCK - Attachment Detected
54 MAIL BLOCK - Relay Denied
60 AUTO REPLY - (ie. Out Of Office)
70 TRANSIENT BOUNCE
80 SUBSCRIBE Request
90 UNSUBSCRIBE/REMOVE Request
100 CHALLENGE-RESPONSE

I'm not sure that it's a full answer, but this algorithm for detecting bounces might be useful.

Related

Why is the DATA in SMTP not null terminated?

I was reading the original RPC about the SMTP Protocol and came across this section:
SMTP indicates the end of the
mail data by sending a line containing only a period.
Why did Postel decide to use the period as the terminator? Would it not be easier to use the already existing null terminator?
I see, that he would not want the users content to interfere with the protocol, but I would naively assume, that a user is more likely to use a period in one line than a null terminator?
Added to that, would the implementation of the mail client not just cut of the text if the user came to use the null terminator his mail contents?
IMHO: SMTP has be designed long time ago to be human readable/writable.
It is pretty simple to test (send simple SMTP messages) typing them by hand via telnet program.
"Human readable" makes null terminator a suboptimal choice.
EMSMTP design is a fossil of pre-spam era. It is bad (by current standards) but it is so widely implemented and sufficiently good (after fixes) to make any quick revolution "not sufficiently urgent".
Extra info: Seen RFC 3030 for BDAT alternative to DATA command.

Can google apps script time-based trigger cause this problem: "554 5.4.6 Too many hops 53 (50 max):"?

Hello I have a google docs document where I installed a simple emailing code on as follows:
function sendMail() {
var file = DriveApp.getFileById('XXXXXXXX');
var recipients = "mymail#group1.com" + "mymail#group2.com" + "mymail#group3.com";
MailApp.sendEmail(recipients, 'Daily Notes', 'Please see file Attached.', {
name: file.getName(),
attachments: [file.getAs(MimeType.PDF)]
});
}
The code works fine it sends the email to multiple group emails which I want it to send. It has done this for a month now. But until recently, I keep receiving this Mail Delivery Subsystem after 20 minutes or 1 hour from the original email which is sent out and inside the error message email it says:
"The following addresses had permanent fatal errors"
and it points to only one of the emails I send daily mails to. It also says something like this: (reason: 554 5.4.6 Too many hops). and then this:
554 5.4.6 Too many hops 53 (50 max).
The Good Doc's file has Google's time-driven trigger as you can see in the image below. I wonder if somehow the trigger is running twice in the time interval I gave to it. Could it be why this is happening?
In almost all cases 554 5.4.6 Too many hops 53 (50 max) means loop along email delivery path e.g. server A forwards to server B, server B forwards to server A, server A forwards to sever B, …. It is detected by counting Received: headers in message passing via give SMTP/email server.
Suggested fixing procedure:
Inspect Reveived: headers in email bounce message to locate server(s) responsible for the loop.
contact the relevant postmaster
In many years I have seen just one "real life" case when bounce message/email like that was not caused by email delivery loop.

phpmailer error codes for outcome processing

I am building a mailout capability and it is working OK as far as it goes. However, I want to distinguish between various potential (high level) outcomes in order to determine what happens to each message after the current send attempt.
This must be a common requirement so I seem to be missing something pretty obvious, but I can't find anything that addresses it, either here or via Google or on PHPMailer site or .. . Possibly because there are so many questions about specific errors that I just can't find anything useful in all the other results.
At very high level:
Attempt send, and assess resulting error/result. Identify whether this message has been sent, must be retried later, or failed permanently.
- success -> update message status as 'SENT: OK'
- sent, but some issues (e.g. one recipient failed, others processed OK)-> 'SENT: some error'
- failed, due to temporary problem (e.g. connection problem, attachment open) -> 'TRY LATER'
- failed, due to message-specific problem that we should NOT try to resend-> 'FAILED: some error'
As I was unable to find an existing resource with e.g. a table of errors, I spent some time working through the phpmailerException code to try to build one myself, but it's not simple because a) they don't appear to have been designed in terms of this kind of grouping logic, b) it is not easy to uniquely identify a particular error: PHPMailer provides human-friendly messages, which are different in different languages, rather than an identifiable code - given that my solution will need to work across different language installations that's a problem!
Obviously SMTP itself provides a range of errorcodes which I could potentially use for this purpose, but how do I access these via PHPMailer? (This would work for me as I only use SMTP at this point - however, this would NOT work if other message transport like sendmail was used, so I would prefer a PHPMailer solution)
If you want individual result codes for individual address, you really need to send each message separately. If you do get errors on some recipients, they will be listed in the ErrorInfo property - look in the smtpSend function to see how the error string is assembled. I agree that it's not especially easy to parse that info out. The error messages in PHPMailer are generally more for the developer than the end user, so the translations are not that significant. You can get slightly more information about errors if you enable exceptions rather than relying only on return values.

how to notice if a mail is a forwarded mail?

I have a very special problem.
If we create a mail in Outlook, we add a UserProperty which contains a DataBase-ID of our System, so we can Link the mail to the representing DataBase-Item. On the service which reads the mails in each Mailbox and imports them automatically I can read this property by using ExtendedPropertyDefinitions. So far everything is fine...
If the User now forwards the message in Outlook, Olk copies the UserProperty to the new message. And now my problems beginn. Now my Service thinks the new message is also linked to our database and updates DB-Entry with the new Body and new Subject.
So does anyone now how to find out if a message is a forwarded one or how to tell Outlook not to copy the userproperty to the forwarded (new) message?
thx. Jay
What we thought about, but isnt working for our case
- a second userproperty containing a simple tag linke "fromSystem". Cause this would be copied too.
- a second userproperty containing a hashsum calculated from subject and Body. Cause both could be changed by the user. We just create the message, add all properties and Display it. from this Point on we no longer have control what is Happening to the mail until the Service handles it.
Your service consuming EWS should check the ConversationIndex and only update the database if it's 22 bytes long (original source message). Forward emails and reply emails keep appending 5 bytes (10 chars) to the ConversationIndex extending it beyond 22 bytes.
Sample ConversationIndexes
Original: 01CDD15D80E51C1D4522172840ACA96287DA28A15D97
Reply: 01CDD15D80E51C1D4522172840ACA96287DA28A15D970000018630
Forward: 01CDD15D80E51C1D4522172840ACA96287DA28A15D970000018630000000FC30
ConversationIndex represents the sequential ordering of the ConversationTopic (essentially GUID + timestamp). See Working with Conversations on MSDN. ConversationIndex is explicitly defined on MSDN here.
if (message.ConversationIndex.Length == 22)
{
// update DB body, subject, etc.
}
Also make sure you load the EmailMessageSchema.ConversationIndex before trying to access its value.

Cleaning bounce mail

I'm wondering whether cleaning the bounce mail is done by parsing the returned message text and creating rules (if contains bla bla bla) or by extracting a certain error code number.
Please clarify.
Bounces are handled using VERP, i.e. a specially formed reverse-path. The reverse-path address is the one where to bounce will be sent, and it contains the original recipient in encoded form and maybe other informations too. Something like: bounce-john+example.com#list.example.org. See Variable envelope return path.
The analysis of the bounce mail is good for retrieving the exact cause of the bounce, especially if the bounce is sent in the standard DSN format, but otherwise it is not a reliable method (for example not all mail servers use DSN).