Summary
How can SendGrid's Inbound Parse and send mail SDK be used to receive emails, respond then associate each new message in thread with the original email?
Setting the replyTo in the sending email is nice but isn't reliable as the user may respond to to other messages in the thread, such as the original message and hence miss this.
One solution we're trying is using message headers to associate emails by message ID. Not sure if this is reliable though?
This question is about best practice and finding the most reliable option.
This question doesn't apply to SendGrid alone but emails in general.
Code Snippets
shammelburg has a nice setup here that can be used to experiment.
Threads are usually something each email client can handle differently to visualize the items in your inbox. So it's not a standard you can rely on and not related to anything you can configure in the inbound parse or the client when you send an email.
Gmail and other large providers usually group emails that have the same subject line as the original email with a "RE: " prefix.
Here's an example: The subject of the original email: "Hello World" The subject line of your response: "RE: Hello World".
Related
We send an automated email whenever someone signs up for our app.
The subject is always the same for every user.
Let's say the subject is 'Welcome'.
The problem is that whenever someone replies, it always end up in a single giant thread in gmail.
So the emails from various users are interleaved with each other, which makes it very difficult to manage.
What we want is a single thread per user (i.e per recipient email address) so that we can keep the conversation with each person separate.
I'm aware that gmail has its own way of grouping messages into thread based on the subject line by default.
However, we noticed that some support tools (such as Zendesk) have somehow find a way to keep each email separate even though the same subject line is used. They seem to have a work around.
What I mean is that we have received multiple emails from the Zendesk email sender with the same subject and somehow they end up in multiple threads in our gmail.
They don't seem to use a uniquely generated email either. The from and reply-to address is simply support#domain.com
How do they do it?
We experimented with the 'Message-ID' and 'In-Reply-To' SMTP headers but were unsuccessful.
Sending emails with different Message-ID still end up in the same thread.
See example of a Zendesk email received to my gmail address below:
Since I can't post in the comment, will add to the answer here. Sample headers in an email I received from Zendesk. Note that they use the support email address + the ID of the support ticket as the 'Reply-To' address.
Like many web apps, we use Postmark to send all notifications for server side events. Many of our events are grouped and related by something simple and logical (think multiple replies to the same issue, like in GitHub).
Right now, every email sent for these related events is it's own email thread. My question is: how do I send these emails so that related ones get pushed into the same thread?
I'm not sure if this is something at the Postmark level (like include a previous message ID) or if this is something I do with SMTP (like I should format my subject better and inline previous responses), so that's why I'm seeking guidance. Also, every Google search about: "Postmark email threads" returns concerns over the thread safety of the Ruby Gem.
For more information, the app is written in PHP and right now we are znarkus/postmark-php for sending emails and jjaffeux/postmark-inbound-php for parsing inbound ones. However, I am more than willing to add any extra packages if they help me in my quest.
Thanks in advance!
You can add a few SMTP headers with the original Message-ID that most clients use to link together replies. If the original email had a Message-ID header of <123#mail.example.com> the new email you send out should keep the subject line the same and add headers of:
In-Reply-To: <123#mail.example.com>
References: <123#mail.example.com>
And that should inform clients that the two emails should be threaded.
Edit:
The value for these headers should be the SMTP Message-ID header, which is slightly confusing because it is a separate concept from the Postmark MessageID value, which is just a UUID for the email.
The SMTP Message-ID header is always in the form an email address, because that's how it's supposed to be formed, but doesn't have to be related to the from address.
I'm sending newsletter using Zend Mail. I have used setReturnPath() to put all undelivered mail notifications in one place.
And what now?
How to get the list of addresses which were unreachable?
How do I read and parse the returned notifications?
How to know whether the mail returned because of non existing email or just quota exceeded?
Which headers do I need to send and check?
Related:
Variable Envelope Return Pathwiki
Handling undelivered emails in web appso
This class may be helpful. Can determine whether the mail is a bounce and return a response code with description:
http://www.phpclasses.org/package/2691-PHP-Parse-bounced-e-mail-message-reports.html
Short Answer:
you can't do that in a simple way and not in your app.
Long Answer:
You should handle that in asynchronous way and outside your php app (at least in part). First of all you must setup the return address to something like sender+recipient=recipientdomain.com#senderdomain.com as in the TimB answer. At this point all the notification sent by receiving smtp server will go to that address.
Then you need to setup the smtp daemon at senderdomain.com mail exchanger to handle that kind of bounce messages and process them in some sort of pipe.
With a pipe you can forward the returned message to an external program which parse the message and extract the needed informations (i.e. the reason why the delivery is failed)
At that point in your program (which can be a cli script in your application) you can mark the address as failing and optionally can record why.
This is a pretty difficult task, which can't be handled in a simple application. Usually I use a dedicated software for large mailing list handling such as sympa which takes care of this task for you.
Otherwise you can use an external delivery service such as Sendgrid which will do the dirty job for you and report the failing addresses with a simple API. As a bonus with this solution, they are in the whitelists for all the major providers, so your email won't be marked as spam as far as you respect some simple rules (i.e. removing bouncing addresses and use an opt-in policy for your newsletter)
Well, the second link and especially the answer by TimB explains very well the procedure.
What may not be clear is that the return path is nothing other than a regular email account, i.e. you will get the email to that address. Zend_Mail is not waiting for a response and hence there is no list of addresses.
A forum-like app I'm working on will send an email notification to the thread starter when a new replied is received. It would be nice if the owner can just reply the email to add a new reply to the thread.
How can I implement the feature, i.e. "reply to this email to comment" like Facebook?
Option A: scan the subject line/body? I don't like it 'cause what if the user modified the subject line by mistake?
Option B: use a unique reply-to e-mail address that links to the thread ID. Is this a common function for mail server? like set up a *#addComment.domain.com ? Or does the app server needs to setup a new email account before sending the email with reply-to?
Any other options?
Thanks!
Using strings in the subject and body can be easily erased by a user of the system.
Use plus addressing (reply+UNIQUEIDENTIFIER#yourapplication.com) as the REPLY-TO address in the mail message. With CFIMAP you can retrieve the messages and parse the TO.
Wildcard domain (replyto#UNIQUEIDENTIFIER.yourapplication.com) is also an option, but if your email server supports plus addressing I would go that route.
You could stuff the thread ID or the parent message ID (the message that is being replied to) in the Msgessage-ID: header of the email, or a custom email header, and put the processing after accepting the message.
However, using custom Reply-To: addresses is quite common.
an option is to embed an identifier in both the subject and the body of the original email. something small, like bit.ly's 6-8 character code. that way, they're less likely to mess it up, and you have the safety of the email body, which most people leave in anyway.
Using a custom email header is not advised as there is no guarantee that any server along the route would not strip it off (or simply fail to pass it on). A friend who worked at a huge email data center for AT&T said the techs there warned him off that idea.
This may also be true of the Message-ID: -- don't know.
Am trying to determine the best way to persist information from an originating email, through to a reply back.
Essentially, it is to pass a GUID from the original email (c#), whereby when the receiver replies back, that GUID is also sent back for reference.
I have tried setting the MessageID, whereby using Outlook, the In-Reply-To value is set with the original ID, however using some webclient email systems, that value is not created on reply. Is there another way to sent this info through email headers?
Some variation on VERP is probably the most reliable...
http://en.wikipedia.org/wiki/Variable_envelope_return_path
Specifically, instead of having all your replies coming to the same address, encode the information you want to persist into the From address for the email.
For example, in the case of a helpdesk ticket, you could use something like:
From: Helpdesk <support-ticket-123#example.com>
To: End User <user#example.org>
Subject: Ticket #123 - problem with computer.
That way, regardless of what the user edits in the subject or text, you know what ticket is being referred to by the receiving email address.
I don't think you'll be able to do anything that is perfectly reliable by headers alone -- the number of clients that would have to cooperate is immense.
Most systems that do this work by including something in the body of the email that is sent that allows it to identify the message, and including text instructing the recipient to include that block of text in the response. You could also try including it in the subject (and including text in the body to leave the subject unchanged). That's how some mailing list managers I've seen do it.
I stumbled upon this question, and it's been very informative. This, however, leaves me with one question: Will using VERP, or a variation of editing the 'reply-to' or 'from address', cause the messages to be locked up in spam filters?
I have read that spam senders often change the bounce address to prevent their servers from getting clogged with bad email address bounces. Is it a spam risk to assume this approach?
The most reliable way is to put the ID in the subject, which should be preserved throughout replies.
(It doesn't hurt to tell your users that they should keep the subject intact.)
RT, a popular ticketing system, does this. They use a simple subject format like "[Ticket #123]" and key off of 123.