I'm trying to receive and process mails by querying them with IMAP from a mail server. I need guaranteed at-most-once processing. Therefore, I need to be able to detect what mails I have already processed.
The IMAP Unique ID cannot serve that purpose because it's validity might change. If that happens I have no way of relating already processed UIDs to the new UIDs that I'm seeing after the change.
How can I uniquely identify messages and achieve at-most-once processing?
In fact, how are mail clients such as Outloook doing this? If the validity changes they must be able to relate the locally stored mails to the new UIDs.
Related
I'm building a system which will send email automatically. To some of them the system will expect replies. Also a user will be able to simply send an email directly to the system, any time (not as a reply).
There's one single email which will both send and receive emails. That is, there won't be, probably, any unique email assigned to a user with +
Question: when the system receives an email, how will it be determine with 100% accuracy whether it's a reply to an earlier sent one, or simply a separate, sent out of the blue, one?
Note that "Re: <subject>" isn't a reliable way at all. Neither could it be the body, for both body and subject can be removed or altered by a user.
I think I should utilize Message-ID, In-Reply-To and References somehow. But how precisely?
Also, a user theoretically can use any email client whatsoever.
I want to learn different ways how you guys handle incoming emails in Dynamics CRM.
How you make sure that everyone is responding to their emails?
Can we filter incoming emails to get the ones which are not yet
replied? On daily basis we are getting a lot of emails and we want
to have a filtered view where we can only see the emails which are
not yet replied yet.
How you deal with incoming emails which are sent to sales#
accounting# etc.
3A. How you distribute them among different users?
Assigning to them?
Forwarding to them?
3B. And how you make sure that they are replying to those emails?
I will appreciate if you I can learn from your experience on this subject
Use the existing Customer Service module which Microsoft has built for this purpose. Incoming emails create Case records, which are actionable items with associated statuses.
How you make sure that everyone is responding to their emails?
Set up workflows so that unassigned cases cause escalation of some sort (i.e. an email sent to a manager)
Can we filter incoming emails to get the ones which are not yet replied? On daily basis we are getting a lot of emails and we want to have a filtered view where we can only see the emails which are not yet replied yet.
Filter by Case status
How you deal with incoming emails which are sent to sales# accounting# etc.
- How you distribute them among different users?
You can set up routing logic so that cases are tagged with specific attributes, or assigned to specific users based on what address the originating email targeted.
Assigning to them?
This is out of the box case functionality
Forwarding to them?
Set up a workflow to send an email to the case owner or other user
And how you make sure that they are replying to those emails?
A simple option is to review the email history associated with the case. Automating this task is certainly possible but can get complicated.
You could add each email into a queue, and have users process emails out of the queue.
There is a setting to enable this on the entity configuration with the customisations area.
Emails when tracked or synchronised then appear within Dynamics within the queue.
Users then process the queue. This also caters for the scenario where emails do not need a reply - they can be removed from the queue. In this model, the emphasis switches from checking emails are replied to, to checking queues are cleared.
Reply email are linked back to the original email via the Parent Activity Id. So you could build a filter around this.
Create a queue for these to receive email.
The email address that you enter in the Incoming Email field receives all messages sent to the queue.
A) Users can pick items from the queue, a team leader can assign queue items to others, or use bespoke customisation or development to automatically route items.
B) As above.
I need to send via agent some emails and I want to track the responses to these emails with an agent on the mailbox of the user.
Is there any field I can use to set/read to achieve this?
The flow:
1) the user create a document in a database (on the web);
2) the WebQuerySave agent creates the email, saves it in a mail-in db and sends it to the recipient;
3) the recipent sends a reply;
4) an agent on the mail-in db should read this reply and write some data back to the original document made by the user... and that's what I'm trying to do and I can't rely on the subject because it's all on the final users hands...
Is there any way I can do it?
Thanks for any advice.
Since you are dealing with outside recipients and can't control the software that they use or how they use it, it is going to be difficult to handle 100% of cases. You are going to need to build a mechanism into your solution for marking exception cases where automatic matching fails so that someone can handle them manually.
That said, you can set a special flag in your outgoing mail. Assuming that your webQuerySave agent uses mailDoc as the object name for the NotesDocument that it is going to mail, just do this:
mailDoc.ReplaceItemValue("mySpecialFlag",evaluate("#unique"))
In order to get this value to go out in the email, you are going to have to make sure that the Domino server that performs the conversion from Notes format to SMTP is configured with the outbound MIME option to "send Notes private items", which is described here. This will create a custom x-notes-item header in the outbound message, and as long as the receiving email system preserves it and copies it into the end-user's reply message, that header will be received back by Domino and converted into an ordinary NotesItem that your agent in the mail-in database can look for. I think that most ordinary mail software will preserve this header for you in replies, but you will need to be handle cases where the haeder doesn't come back either because the outside mail system's software doesn't preserve it or because the user creates a new message and copies the old one into it instead of doing a normal reply.
Given there is a "FAILTO=''" option for cfmail, triggering an email to be sent to that email if the email didn't get delivered...
Is there a way to somehow assign an ID or tracking # to an email, store it in a database with that ID... then update the status of that email if it fails?
I'd like to track bouncebacks... preferably WITHOUT sending the FAILTO to a POP3 or IMAP and then checking it with cfimap...
Is there any alternate way of handling this?
Maybe an event gateway that is triggered upon email failure?
UPDATE: I've decided to take a different approach, utilizing the sendgrid API.
I'm hoping that lends me with a few more tools than CF offers.
The short answer to your question is unfortunately no.
A longer version with a possible solution:
The failTo email address populates the return-path in the email header, this then 'should' be used by mail servers for bounce backs (however see - http://www.bennadel.com/blog/1899-GMail-Seems-To-Ignore-The-Return-Path-Header-Defined-By-The-CFMail-FailTo-Attribute.htm for an example where it doesn't)
So you are going to need to monitor an Imap or pop account to see your mails, however you can set up an event gateway to monitor this, with detailed instructions here - http://www.alagad.com/documentation/imapGateway/ImapWatcher%20Gateway%20Documentation.pdf
What you're left with is needing to identify which mail matches which bounceback, when I've done something similar in the past I used unique id's for the failTo email addresses at a domain I owned. If you set that up and then use your listener cfc to look for the id in the return-path.
So your sending code would work along the lines of:
Generate unique id
Send mail
Add row to database with unique id
Your listener.cfc would then need to inspect the email returned and if it finds the unique id update the row with whatever information that you're after.
Hope that that at least helps even if you'll need to set up some other bits.
You could use a directly watcher on the undelivr folder to log the failed emails, only really a solution if its own server and not a shared server though.
As far as I know once it leave the spool and is off to your SMTP server CF assume it's been sent correctly.
The email will trickle down the chain of SMTP servers/relays and if anything happen the only instruction they have is to bounce it back to the from address or failto address if present. CF isn't listening at this point so it can't respond.
We use an external tool called Glock email processor to handle exceptions. It's not free, but works pretty well. You can find it here: http://www.glocksoft.com/email-processor/
You need to configure it to check the failto address and from there you can take many actions. I got it setup as a three strikes system.
Email address bounce, I increment a counter in my email table, at 3 I deactivate that email from the system.
Nothing you can't do yourself with cfpop though.
Our clients sometimes don't get the emails that we send out. It's a BIG loss. How do I assure that they receive the emails so that if it's not received in the other end, the program can resend it or do something about it.
None of the suggestions above will work 100% of the time. Many email clients will (rightly so) refuse to load foreign images, negating the usefulness of "web bugs". They will also refuse (or be unable to) return Outlook-style "receipts". And many mail servers either deliberately (to curb spam) or mistakenly (due to misconfiguration) won't return bounce messages. Or possibly an over-aggressive spam filter ate your message, so it arrived but was never seen by the end user. Plus there is the little matter of mail taking hours or days to reach the end user or bounce, and how do you correlate these late notifications or bounces with the mail you sent 4 days ago?
So basically, you can catch some but not all, no matter what you do. I'd say that any design that relies on being able to know with certainty whether the end user got your mail is fatally flawed.
One thing that you can do is set up a bounceback address that receives any mail that is undeliverable. Use the bounceback address as the From address -- you may want a different one for Reply-To so that replies get directed properly.
Check the bounceback mailbox daily and contact customers to get updated email addresses for the ones that fail. You may be able to automate a couple of retries to failed addresses before resorting to the manual contact in case the failure is only intermittent.
This would take some code outside your application that scans the mailbox and keeps some state information about the number of contacts, etc. and attempts the resend.
Depending on how you generate the mails, you might be able to make this process easier: generate a unique bounce address for every single email you send out. You could use bounces+1234#example.com, for example.
Many SMTP servers will allow you to use the part after the + as a parameter to an external script, etc.
The problem is that many (broken) SMTP servers don't return enough info with a bounce to identify the original message -- sometimes, when there are forwardings involved, you don't even get back the original addressee...
With the above trick you can reliably correlate outgoing messages with incoming bounces.
There is no standard way to know whether the email reached the destination. Many email clients support different types of receipts though. You can use any of those if you want.
There are some ways to know when the user actually read the email.
There are many techniques like adding an image to your email that is to be fetched from your web server. When the user reads the email, the request for the image comes to your server and you can capture the event.
The problem is that there is no way to know that the mail did not reach the destination.
I worked on a bulk email system in a previous life. Deliverability was one of our major issues. The most common cause of undelivered emails is a spam filter.
Here are the steps we took to ensure the highest delivery rates:
We used Return Path to test emails for that spam-like smell.
If you send a lot of emails, you need to make sure your SMTP server is not blacklisted.
Remind your users to add your FROM address to their "safe senders" list.
Use a system that collects bouncebacks and use them to scrub your mailing list. This will also help keep you off the blacklists.
If the emails are critical, consider sending them return-receipt-requested. This will not really guarantee anything, but it might give you some metrics on actual deliverability.
There's not really a good way to determine if the email actually arrives in their inbox, you can only confirm that you sent it. Attach a receipt that lets you know when they open it perhaps?
Microsoft Outlook provides similar functionality, however it is based on the email client. I'm not sure if other clients, like Thunderbird, support this.
However, there is nothing in the protocols that specify receipts.
One option that may work: send a link to a generate web page and monitor that page for hits. This provides its own issues however: confidentiality, etc.