I am building a ticketing system where users can raise a support ticket and agents can reply to the particular ticket and when the agent replied to a particular ticket I will send the reply to respective user email (email service using SendGrid).
My question is when the user replied to the agent email I will fetch that email with SendGrid inbound parser but I am stuck that how to uniquely identify for which ticket user has replied. I have two ideas attaching JWT in the header or unique email address resemble ticket id (ticket-id-2654654#example.com) but I don't what will work
You could send a custom Message-ID Header (RFC 2392) with your E-Mail. Responses to your E-Mail should then contain a References and/or In-Reply-To header containing the Message-ID of the E-Mail that was answered.
Edit: Just tested it with Zendesk and Zammad. They both seem to identify responses in that way.
You mention having a unique email address, not unlike a VERP (Variable Envelope Return Path) address. VERP is most pertinent if it's dealing with bounces, since it refers to the address used for the SMTP MAIL FROM:,
MAIL FROM:<ticket-id-2654654#example.com>
in particular it is helpful to identify which emails bounce - the email likely never arrives in a mailbox at the remote end. Note that the address used during the SMTP conversation like this will be recorded at the top of the received email with a 'Return-Path:' header.
Assuming an email doesn't bounce then how to ensure an email with a recognisable identifier comes back:
MysteriousPerson's custom Message-ID: header suggestion is excellent; also specify a VERP address in a Reply-To: header - so one or both of the identifiers you put there should be found in the reply to decode which ticket/user the original message was sent to. You have the choice of using the same VERP address in the From: header or something more human-friendly, depending on what you want the recipient to see. Since Reply-To: is specified, client software will use that for replies.
The main RFCs on SMTP and message format are likely to be helpful if you're not already aware of them.
Related
Just for knowledge purposes, I want to know if it is possible to know if email spoofing has been done successful or not . Suppose if I am sending an email from one's account, What if receiver doesn't exist? Means will i get any error as a spoofer that receiver doesn't exist and you cant send email . Thanks
There is no way to know for certain, but you can make an educated guess.
First, when sending an email to a non-existent recipient and the recipient's server is configured to report this to the sender (pretty much the default behaviour), this reporting can happen either:
1) During the email transmission. The SMTP conversation with the recipient's server may very well fail at RCPT TO: (i.e. when the recipient is specified) or at the end of the DATA or BDAT commands (i.e. when the email just have been transmitted and the server either acknowledges this or rejects the email). If you receive an 5xx-type response at either stages, or, well, anytime earlier in the conversation, you can be sure the recipient did not get the message. Validation during email transmission is common.
2) After the email transmission. Servers that don't do recipient validation on SMTP level will often accept and queue emails during the transmission, then generate a bounce report (also called a Delivery Status Notification/DSN or Non-Delivery Report/NDR) later and attempt to return it to the original message sender. As you have no access to the mailbox of the original message sender when spoofing, you will have no idea if such bounce report is generated. This method of validation is still fairly common.
Second, you can embed a tracking image in your HTML email to see if the email was opened. This works by placing a HTML <IMG> element in the email that points to e.g. http://example.org/tracking-1x1.gif?uuid=<id>. The idea is that you track whether that image was downloaded. Be aware that virtually all modern email clients will disable downloading images from unknown sources and some action is required from the recipient to enable downloading images.
Third, you can spoof the MIME sender only. Emails have two distinct set of addressee information: the envelope addressees and the MIME addressees. Emails are delivered based on what's written on the envelope, but email clients render the MIME information instead (what's inside the envelope). In other words, during the SMTP transmission you may specify a different sender address (MAIL FROM: command) from that in the email (From: header). The fun part is that bounce reports are sent to the envelope sender address, a.k.a. the Return-Path, so if you have access to the envelope sender mailbox, you can receive a bounce report no matter what stage it was rejected. Note that you will reveal yourself, either through the mail server logs or through the email header, where the Return-Path header will contain the envelope sender address.
Please use the above information for "knowledge purposes" only. Be aware that email spoofing is likely a crime in just about any country.
I have a service that sends out emails and listens to replies. I need a way to reliably pair each incoming reply with the correct outgoing email. Is there any way to embed some key in the outgoing email that will persist, so that when the recipient clicks "Reply to" in his email client, his reply will retain this key?
So far, my best bet is "faking" the from address in the outgoing email, so that the username is an ID (e.g. from=41541#domain.com) and the reply comes to this email. However, the obvious downside is that I have the added configuration effort of forwarding all incoming email from those non-existent email addresses to the one real email a, e.g. incoming-mailbox#domain.com.
Every email has a unique mail ID. If you didn't give it one when you sent it the next mailserver will do it. See http://en.wikipedia.org/wiki/Message-ID
The response has that message-id in the In-reply-to header. Example - see here.
Best reference is the RFC 5322 "Internet Message Format", section 3.6.4: https://www.rfc-editor.org/rfc/rfc5322#section-3.6.4
This depends on all parties involved doing the right thing, but for the most part they'll do so these days. Of course, some software or human could send a reply without correctly set in-reply-to header, but that's the exception. When I write a reply to an email and use "New Mail" instead of "Reply", for example, my email of course won't have the in-reply-to header.
We have an application that allows users exchange messages, and delivers the message through email. But we don't reveal sender's email, instead; every time a message is sent out a new email address gets generated for the "From" field. Something similar to the way Facebook or LinkedIn send out messages.
Everything works fine, except the recipient's email client caches all these email addresses.
For instance, if Jack sends two messages to Jill, the sender's addresses can be: a123#FakeEmail.com and a456#FakeEmail.com, so next time Jill composes a new message and starts typing Jack's name, the email client suggests two addresses for Jack.
So my question is, how we can prevent email caching on client's machine? Is there something similar to HTTP header Pragma: no-cache for SMTP?
Thanks
Nivresh
OK, here is what I ended up doing. To prevent Outlook and other mail clients from suggesting several dummy email addresses generated for the same contact name, I add a special character to the beginning of the sender's name. This can be a space or if you use Unicode, you can add a zero-width space (\u200B) to the beginning of the name.
So in my example, the email is sent from "\u200BJack" which will be shown as "Jack". The dummy email address will be saved in mail client under the name of "\u200BJack" and when the recipient later on tries to send an email to Jack, email client won't suggest this address.
Hope this helps someone.
On our mailing application we are sending emails with the following header:
FROM: marketing#customer.com
TO: subscriber1#domain1.example
Return-PATH: bouncemgmt#ourcompany.example
The problem that we are facing is that some email servers will bounce back a message immediately and use the from or reverse path (marketing#customer.example) instead to our bounce mgmt server. We want to know if we modify in the header the reply-to to be the same as the return-path if we will be able to catch all bounces.
Any other ideas are welcome?
We are using the following documents as references:
VERP
RFC
Bounce Messages
SMTP Log Parsing to get Bounces
EDIT 1: A few more bits of information to see if we can get this resolve.
We want to know at what point the email server relaying the message will choose to use the reply-to versus the return-path. We have notice that when the first SMTP server relaying the message gets rejected it sends it to the reply-to, but when it happens after one hop it sends it to the return-path.
Let's start with a simple example. Let's say you have an email list, that is going to send out the following RFC2822 content.
From: <coolstuff#mymailinglist.example>
To: <you#example.com>
Subject: Super simple email
Reply-To: <coolstuff-threadId=123#mymailinglist.example>
This is a very simple body.
Now, let's say you are going to send it from a mailing list, that implements VERP (or some other bounce tracking mechanism that uses a different return-path). Lets say it will have a return-path of coolstuff-you=yourcompany.com#mymailinglist.example. The SMTP session might look like:
{S}220 workstation1 Microsoft ESMTP MAIL Service
{C}HELO workstation1
{S}250 workstation1 Hello [127.0.0.1]
{C}MAIL FROM:<coolstuff-you=yourcompany.com#mymailinglist.example>
{S}250 2.1.0 me#mycompany.com....Sender OK
{C}RCPT TO:<you#example.com>
{S}250 2.1.5 you#example.com
{C}DATA
{S}354 Start mail input; end with <CRLF>.<CRLF>
{C}From: <coolstuff#mymailinglist.example>
To: <you#example.com>
Subject: Super simple email
Reply-To: <coolstuff-threadId=123#mymailinglist.example>
This is a very simple body.
.
{S}250 Queued mail for delivery
{C}QUIT
{S}221 Service closing transmission channel
Where {C} and {S} represent Client and Server commands, respectively.
The recipient's mail would look like:
Return-Path: coolstuff-you=yourcompany.com#mymailinglist.example
From: <coolstuff#mymailinglist.example>
To: <you#example.com>
Subject: Super simple email
Reply-To: <coolstuff-threadId=123#mymailinglist.example>
This is a very simple body.
Now, let's describe the different "FROM"s.
The return path (sometimes called the reverse path, envelope sender, or envelope from — all of these terms can be used interchangeably) is the value used in the SMTP session in the MAIL FROM command. As you can see, this does not need to be the same value that is found in the message headers. Only the recipient's mail server is supposed to add a Return-Path header to the top of the email. This records the actual Return-Path sender during the SMTP session. If a Return-Path header already exists in the message, then that header is removed and replaced by the recipient's mail server.
All bounces that occur during the SMTP session should go back to the Return-Path address. Some servers may accept all email, and then queue it locally, until it has a free thread to deliver it to the recipient's mailbox. If the recipient doesn't exist, it should bounce it back to the recorded Return-Path value.
Note, not all mail servers obey this rule; Some mail servers will bounce it back to the FROM address.
The FROM address is the value found in the FROM header. This is supposed to be who the message is FROM. This is what you see as the "FROM" in most mail clients. If an email does not have a Reply-To header, then all human (mail client) replies should go back to the FROM address.
The Reply-To header is added by the sender (or the sender's software). It is where all human replies should be addressed too. Basically, when the user clicks "reply", the Reply-To value should be the value used as the recipient of the newly composed email. The Reply-To value should not be used by any server. It is meant for client-side (MUA) use only.
However, as you can tell, not all mail servers obey the RFC standards or recommendations.
Hopefully this should help clear things up. However, if I missed anything, let me know, and I'll try to answer.
Another way to think about Return-Path vs Reply-To is to compare it to snail mail.
When you send an envelope in the mail, you specify a return address. If the recipient does not exist or refuses your mail, the postmaster returns the envelope back to the return address. For email, the return address is the Return-Path.
Inside of the envelope might be a letter and inside of the letter it may direct the recipient to "Send correspondence to example address". For email, the example address is the Reply-To.
In essence, a Postage Return Address is comparable to SMTP's Return-Path header and SMTP's Reply-To header is similar to the replying instructions contained in a letter.
for those who got here because the title of the question:
I use Reply-To: address with webforms. when someone fills out the form, the webpage sends an automatic email to the page's owner. the From: is the automatic mail sender's address, so the owner knows it is from the webform. but the Reply-To: address is the one filled in in the form by the user, so the owner can just hit reply to contact them.
I had to add a Return-Path header in emails send by a Redmine instance.
I agree with greatwolf only the sender can determine a correct (non default) Return-Path.
The case is the following:
E-mails are send with the default email address: admin#example.com
But we want that the real user initiating the action receives the bounce emails, because he will be the one knowing how to fix wrong recipients emails (and not the application adminstrators that have other cats to whip :-) ).
We use this and it works perfectly well with exim on the application server and zimbra as the final company mail server.
I'm working on a web application. A user will create an email message that will be sent to another person.
I would like the e-mail that gets sent to appear from the user's name and e-mail address of the user on my system. And if they reply to the e-mail then it should go directly to the sender's email address.
However I am worried about the email message looking like spam to email filters along the way.
Is there a proper way to do this?
I noticed on a "contact" page on a WordPress blog that something very similar is done. The e-mail headers look like:
To: email#domain.com
Subject: [Test Blog] =?UTF-8?B?aGVsbA==?=
MIME-Version: 1.0
Content-Type: text/plain; charset="utf-8"
From:=?UTF-8?B?aGVsbA==?=<sender#senderdomain.com>
Message-Id: <20090207234737.39C9522802F3#web7.sat.wordpress.com>
Date: Sat, 7 Feb 2009 23:47:37 +0000 (UTC)
Return-Path: donotreply#wordpress.com
What is interesting is that the display name in the "from" tag and the name that shows up in the subject line is encoded. I do not know if this helps with the spam filters or not, but thought it was at least worth mentioning.
Also, who would receive an undeliverable notification in this example? Would it go to sender#senderdomain.com or would it go to donotreply#wordpress.com?
Basically all you need to do is set the From header to the email address of the user sending the email. The value of From is what is displayed in a recipient's email client. Most spam detection systems in place today look only at the message content, not the email headers, so you currently wouldn't have that much of a problem based on what you set the From header to.
However, there are some systems which are gaining popularity which could prevent you from sending email with somebody else's email address - most notably SPF, the Sender Policy Framework Basically, a mail server that implements SPF will check the domain of the From address of each email it receives and check with that domain directly to see if it authorizes the email. For example, if your server is mydomain.com, the email address of the user is abcdef#gmail.com, and the recipient is blah#example.com,
mydomain.com contacts example.com via SMTP to try to send the email
example.com looks up the SPF records for gmail.com
example.com checks whether mydomain.com is on the list of domains allowed to send email with the domain gmail.com
If it's not, the email is blocked
Also, I found a forum post suggesting that Return-Path is the intended destination for undeliverable notifications. Apparently that header is set based on the value of the SMTP MAIL FROM command.
Setting email fields is generally trivial, but the exact method of doing so depends on the language and framework you are using, which you don't mention. Usually it is a matter of creating a new email object, and just doing something along the lines of:
email.from = "From Name <fromemailaddress#example.com>";
You can set the specific from address to your user's, if you like, it is not at all uncommon though I personally am not a fan. You could also set the from field to something like: "Sender Name (via Your Site Name) <yourresponseaddress#example.com>".
Don't do this.
It really depends on how your mail relay is set up, but actually just don't do it. From header should contain the email address that sent the email, in your case webmaster#whatever.com. If you want people to reply to a different address, you can always use the Reply-To header. Here are some notes.
If I understand the standard (RFC 822) correctly, this is exactly what the Sender header is for (see §4.4.2. SENDER / RESENT-SENDER). Still, I'd go with a different approach and use your sites official contact address in the From header and put the user's address in the Reply-To header. Maybe add some boilerplate text that clearly states where the mail is coming from.
One further advice besides the technical stuff: don't let anonymous users use this facility, you'd become a perfect platform for spamming. Also, out of kindness, you probably want to make sure that your (registered) users know that their email addresses are exposed to the recipients.
Be aware that doing this will make spam filters more suspicious of your email. Combined with suspicious looking content your email may get filtered.
Whoever is in the "From" header will receive the undeliverable notification.
For the filtering - it really depends more on the subject and body of your message than the From address. So with that said, a couple ideas....
Use HTML encoding rather than Plain Text with simple formatting (I know there's probably an argument going the other way but in my personal experience HTML gets through more often)
Always include a footer with ("This e-mail was sent to you from ... blah blah") to identify it; if you don't want people to know it's coming from your system (i.e. really impersonated from the "From" user) then you need to find a more legit way to take care of the requirement.
If at all possible, use a real address from your system as the "From" address with the address of the user as the "Reply-To".