I'd like to write an application which monitors a mailbox (ideally a Gmail mailbox). In brief, my question is how do I listen to incoming emails? I could always keep pulling emails but I am afraid that this doesn't scale very well.
How does iOS do it with Gmail? I think they are using Exchange protocol, but I am not sure. Does anybody have an idea on how to do it? Is there a solution which already does something similar? I just need a hook.
Thanks!
The most universal "API" for monitoring a Gmail or Google Apps inbox is the IMAP protocol. There's some information about this here:
Gmail IMAP OAuth for desktop clients
For Google Apps inboxes (not general consumer Gmail accounts), you can also use the Gmail Inbox Feed, which provides an AtomPub XML feed of unread messages in a Google Apps inbox:
http://code.google.com/apis/gmail/docs/inbox_feed.html
The "API" to GMail is IMAP.
In IMAP the IDLE feature is used to tell the server to notify the client when there are changes to the mailbox.
IDLE is defined in RFC 2177 IDLE is an optional IMAP feature so this may or may not work on other IMAP servers.
If you just want to talk to Gmail, they have a REST API you can use and it supports push notifications (e.g. webhooks):
https://developers.google.com/gmail/api/guides/push
But again this is a Gmail-specific API, which may fit you requirements or not. If you are fine just being specific to Gmail it is likely easier and better to work with from a web/mobile type environment than something like IMAP.
Related
I'd like to send emails programmatically via SendGrid (from admin#mydomain.example.com), but have replies to that email come to my Google Workspaces inbox for admin#mydomain.example.com). I would then like to reply to the customer from Google Workspaces.
The closest question related to this I could find is Setup | G Suite + Sendgrid for transactional email, but that was a fairly different use-case.
So in my usecase, emails to the customer would come from admin#mydomain.example.com via EITHER SendGrid or Google Workspaces.
Here's a article from MailGun that talks about a similiar pattern
My questions:
Will this result in deliverability issues, having emails sometimes come with different signing keys/domains? I don't understand email signing particularly well.
Is there anyway to have replies to the email address go to BOTH SendGrid and Google Workspaces, so I'll have it in my inbox but can also get it posted to a webhook by SendGrid? I think the answer is no, but figured I'd ask.
Any other considerations I should keep in mind for this strategy? Is this a poor idea for some reason?
Thanks!
Yes, this will result in deliverability issues. You should use the same domain for both sending and receiving.
No, you can't have replies go to both. You can have replies go to a webhook, but you can't have them go to both a webhook and a mailbox.
Final consideration: This is a poor idea. You should use the same domain for both sending and receiving. If you want to use SendGrid, you should use SendGrid for both sending and receiving. If you want to use Google Workspaces, you should use Google Workspaces for both sending and receiving. You can't use both at the same time.
Another consideration: If you're sending a lot of emails, you might get flagged as a spammer if you're sending from multiple domains.
The company I work for does something similar that might be helpful for your scenario.
We have a number of systems that send emails via SendGrid e.g. Auth0, Salesforce, various bespoke systems, etc. Some of these use native integrations to SendGrid, some use SMTP relaying, and others send emails programatically via the SendGrid API.
We've implemented sender auth (i.e. DKIM) in SendGrid for our domain e.g. mydomain.example.com, but we don't handle replies in SendGrid as we want replies to go to a different system - specifically our contact center system (which is the main systems our help desk staff use). Help desk staff would then reply to emails from within the contact center system. Note: we needed to configure DKIM for both SendGrid as well as the contact center system (as both currently send emails from mydomain.example.com), but in the future the intent was to implement a SMTP relay in the contact center system (to SendGrid) so that all sending goes through SendGrid.
I'm a bit fuzzy on the fine-grained implementation details of how we handle replies, but essentially we point the (single) MX record for mydomain.example.com to a third-party email protection system which scans the emails and forwards them to Office 365 mailboxes which subsequently forwards them to our contact centre system. I believe Office 365 is really only there for email posterity. AFAIK you should be able to point the MX record to anything that can handle SMTP.
In your scenario you should be able to configure the MX record for the mydomain.example.com domain to point at your Google Workspaces. This will allow you to send emails from SendGrid and use Google Workspaces to handle the sending and receiving of replies.
Not sure if you can configure SMTP relaying in Google Workspaces (i.e. to relay through SendGrid), but this would mean you'd only need to configure DKIM for SendGrid. All emails would be sent by SendGrid and replies would be go to Google Workspaces.
I'm building a service which scans peoples email for specific PDF attachments and indexes them. I implemented this with OAuth2 for Gmail using their extensive Gmail API which works perfectly fine.
I now want to implement the same for outlook/live/hotmail. So I searched around, and I mainly read that you can "Connect to Outlook.com IMAP using OAuth 2.0" (tutorial here). The thing is that this implements a full IMAP connection. As far as I know this is more meant for aftermarket applications with which the user can view and send his email, not for applications which need to download some email in the background (like mine).
I haven't worked with IMAP from within code, but the main problems I see with this now is that:
If I read emails they will be set to "read" in the inbox of the user, which I obviously don't want (I don't want to interfere with the normal email usage of the user).
I will need to either stay connected with all email inboxes, or constantly loop through all email inboxes to get new emails.
My questions are actually;
Is there no other way than IMAP to get users outlook.com email?
Or are my problems actually not problems and should I just create an IMAP "receiver" for all the outlook email accounts?
In answer to point #1, according to Max, you can use body.peak to avoid this.
In answer to point #2, according to triplee, you can connect to all of the accounts and then use an IMAP extension called IDLE.
In answer to point #3, I found the Outlook REST API which includes options to read the email and you can call GET https://outlook.office.com/api/beta/me/messages on a regular basis to get all messages in the user's account or you can call GET https://outlook.office.com/api/v1.0/me/messages on a regular basis to get all messages in the user's inbox. You can find a full list of the REST APIs operations here.
In answer to point #4, I believe that some of your problems are problems and others aren't. I believe that you don't have any technical problems. However, I believe that you are correct that IMAP is not designed for applications like yours as is shown in the Wikipedia article on IMAP which reads (bold is mine),
In computing, the Internet Message Access Protocol (IMAP) is an Internet standard protocol used by e-mail clients to retrieve e-mail messages from a mail server over a TCP/IP connection. IMAP is defined by RFC 3501.
Also, in RFC 3501 which defines the original IMAP protocol it refers to email clients which from the absract reads (bold is mine),
The Internet Message Access Protocol, Version 4rev1 (IMAP4rev1) allows a client to access and manipulate electronic mail messages on a server.
I'm new to Amazon SES and I see that there are two ways to programmatically send emails:
SES API (http://docs.aws.amazon.com/ses/latest/DeveloperGuide/send-email-api.html)
SES SMTP Interface (http://docs.aws.amazon.com/ses/latest/DeveloperGuide/send-using-smtp-java.html)
What are the pros and cons of each method? They seem interchangeable to me, but I'd like to hear from folks who have had experience with SES.
In terms of my own requirements, I'll be sending transactional emails (i.e. receipts, account confirmation, etc.) and notification emails (i.e. "you have a new message", status change, etc.) to my users as they interact with my web and mobile app. If possible, I'd like to keep a history of all these outgoing emails.
The SES API ties you to AWS, the SMTP interface... well it's SMTP.
Do you foresee, in the future the need to move off AWS? Does your application already speak SMTP to another email server?
Depending on your current application it may be easier to go with SMTP.
If you're starting from scratch and don't foresee any need to move off AWS you should probably go with the SES API.
From Amazon's documentation on improving throughput, one advantage the API is the option of using persistent HTTP connections for increase throughput. This is not available to the SMTP option.
Apart from this, I have not been able to find any other major differences between the API and SMTP.
By using the SES API, you are using the SDK, so you can use Roles on your instances: you won't have to handle and store a password for your configuration, so you won't go through the pain of changing the password.
I released a small project https://github.com/loopingz/aws-smtp-relay
to relay from a localhost SMTP to SES API, this way you can connect legacy applications that only handles SMTP to a more normal SES API
They seem interchangeable to me
That's a fair analysis. I use both -- API for new code, SMTP for existing code that already knows how to speak SMTP. I haven't found a strong case either way.
Neither interface will preserve a history -- you'll have to do that yourself. One mechanism I'm working on for use with some legacy code is an SMTP proxy that captures the interaction between the app and SES, saving the entire transaction to S3 using the SES message ID as the S3 key for later retrieval if needed (still a work in progress, more pressing projects to do).
You, at minimum, need to preserve those message IDs returned by SES, and configure bounce, delivery, and complaint notifications so you have feedback... which also works the same with either interface.
How is Yahoo able to send push notifications for email to iPhone/iOS in general? http://en.wikipedia.org/wiki/Push-IMAP is as far as I've gotten in my research, but I'm not sure what exactly Yahoo is doing. I'm planning to setup my own mail server and I'm curious how Yahoo is able to push but others like Gmail is not.
I should mention I'm referring to the default Mail app, not Yahoo's custom mail app (not sure if they even have one).
Thanks!
Yahoo's IMAP server has a special capability 'XYMAPPLEPUSHSERVICE', which presumably is a proprietary way to negotiate use of Apple Push Notification Service
I'm using my Google Apps account (Standard version) to send single emails (automatically) to real interested customers.
The problem is that my sender email account has been suspended, because it's been considered as a "spammer" by Google.
What can I do in order to send that kind of emails from now on? I use MailChimp to send massive emails, and I've seen it has an API, called STS, for sending single emails, but it also required an Amazon SES account, so I don't know whether to invest on those services or doing something else.
What do you recommend for this case?
You should try a service that specializes in sending "transactional" emails, such as Sendgrid, Postmarkapp and Mailjet.
They have APIs and libraries for most programming languages, but in your case you just need to configure the SMTP server they rovide in your email client and you're good to go. They help you not get into the spam folder of your recipients, and help make sure you don't get flagged as a spammer
With most of them you should get some prettty powerfull analytics as well, which is a nice bonus.