I created an application which uses SendGrid's Inbound Parse Webhook. Whenever someone emails "whatever#mydomain.com", the email goes to SendGrid, and then SendGrid hits our server with a POST containing the email's contents. We can then feed that email data back into our main application.
I have it all working. But now I do not know how I am supposed to authenticate the messages SendGrid posts to our server. Does anyone know the best course of action for doing this? Verifying that our inbound emails actually come from authorized users of our main application?
Obviously we can check the "From" address in the headers, but I've read that these can be completely spoofed. Apparently "dkim" and "spf", two attributes of the incoming mail from SendGrid, have something to do with authorization. But i cannot find anything in the documentation, or really anywhere else for that matter, that tells me how I should be consuming these "dkim" and "spf" fields to verify message authenticity.
If anyone has any help, general, specific, or otherwise.. It would be greatly appreciated. Thanks in advance.
Well. Since this doesn't seem to be getting a lot of traction I thought I'd post my own janky-ass solution to the problem, on the off chance that anyone else runs across this issue in the future.
Disclaimer: this could be total garbage nonsense. But it appears to be working all right.
Basically I ended up taking some critical contextual information about the original message that initiated the inbound email. We encode that information in the local-part of the "Reply To" address that we set up with the SendGridMessage. Then I encrypt the encoded local-part.
When SendGrid POSTs to our server with the inbound email, we decrypt the "TO" local-part and validate the result. If it decrypts successfully, we check the "FROM", and verify that they are an actual authorized user of our main application. THEN, we verify that THAT user in question has the correct permissions to edit the information associated with the original encoded local-part of the "Reply To".
Related
Ok, I am working on an invite system for my work, the invite system sends an email to the recipient with a link containing a unique token allowing them to sign up.
The issue is, now that I am actually testing it, the emails are marked as "This message seems dangerous" by gmail, This looks really bad for the company. I have not been able to find any information on what triggers this.
The email is being sent via office365(SMTP) as that is what our email is set up with to try and avoid this kind of thing.
I have seen comments online that it may be due to the token link, If so, how else am I meant to do this?
I'm trying to get the basic "hello world" of sendgrid working, but have so far been unsuccessful. The response returns code 202, suggesting that it will send the email, but the email never sends out. Does anyone know what's going on?
import sendgrid
sg = sendgrid.SendGridAPIClient(apikey='**my-api-key**')
data = {
"personalizations": [
{
"to": [
{
"email": "me#gmail.com"
}
],
"subject": "Hello World from the SendGrid Python Library!"
}
],
"from": {
"email": "me#gmail.com"
},
"content": [
{
"type": "text/plain",
"value": "Hello, Email!"
}
]
}
response = sg.client.mail.send.post(request_body=data)
print(response.status_code)
I just had this issue: I created an account with SendGrid and tried to get the basic example working, the API would return a 202 response, but the email was never sent, and the SendGrid web UI's activity feed showed no activity.
I submitted a SendGrid support ticket and ~8 hours later received a response saying that they had disabled my account's ability to send emails:
Hello,
Thanks for contacting SendGrid Support!
It looks like your account has been closed by our compliance team and this is the cause of the issue.
To reactivate we’d like to know a little more about the email you’ll be sending through SendGrid. Please elaborate on the following:
The nature of your business, the services you provide, and your potential customer base
Your sending frequency and volume
How you collect your recipient addresses (link to opt in page, or sign up process)
How you will allow your recipients to opt out of your emails (whether you plan to use SendGrid’s one-click unsubscribe feature, or if you have your own method)
The types of messages you will be sending (transactional or marketing)
Please reply at your earliest convenience in order to continue the activation process. Thanks for your cooperation!
~15 hours after submitting my answers, I received a response saying my account had been "activated":
Hi nathan.wailes,
Thanks for the additional information! Your SendGrid account has been
activated, and can now be used to send email. To get started, check
out our Getting Started page.
If you hit any snags while you're getting set up, you can find
solutions to common problems and FAQS in our Knowledge Base.
Let me know if you have any more questions!
Best,
Stevin O.
Associate Support Engineer
When I then ran the basic example code with my email address set up as the recipient, the email showed up immediately in my Gmail inbox.
Debug this by going to the sendgrid api log here:
https://app.sendgrid.com/email_activity
In my case, it was a DMARC receiving domain block.
The 202 Accepted Status Code returned from SendGrid does not really indicate that the message has been successfully delivered to the recipients inbox. It indicates that the message is valid and has been "Queued For Delivery". Now, it is up to the receiving server to deliver this message to the recipients inbox, or to send it to spam, or simply drop the message.
There are several reasons as to why messages that returns 202 Accepted Status code from SendGrid does not actually get delivered to the recipients inbox.
For example:
Invalid email address: The email address may no longer be valid (example, when an employee stops working at some company, their email might be removed from the company's system removing the ability to receive any email).
Blocked Emails: If the sending IP or domain is blocked or if the recipients inbox provider has some filters set up such that some specific content of your email/campaign is considered spammy and thus gets automatically blocked.
Another thing to note is that SendGrid may send the messages to spam/junk if the domain authentication has not been properly set up. So make sure that your domain is properly authenticated.
Here is the link to the documentation from SendGrid that explains these event in details. https://sendgrid.com/blog/delivered-bounced-blocked-and-deferred-emails-what-does-it-all-mean/
I had same issue and solved it by adding new api key with full access
In case someone will come across this looking for an answer to a similar problem. I solved the same problem by confirming the domain with DNS records - after that everything worked like charm.
In Sendgrind: Setting > Senders Authentication > Domain Authentication
For us, the problem was that the used dynamic template was not yet set to "active". Unfortunately no error was shown about the in the API response or the log.
Activating the email template solved this and emails are no longer dropped.
The same issue occurred when DNS is not verified and you are not using register email in from email.
Just change from email to register email.
I recently got the same issue. My e-mails were not sent without any reason, the response status was 202, e-mail activity was empty. The possible solution is to add a company details and add an unsubscribe block to e-mail. That worked for me.
I'm implementing a password reset feature and it's a little weird to have an unsubscribe block for this. I'll try to figure out what was the real reason of such silent suppression. Probably it's because I tried to send a draft debug e-mail several times which was looked like a spam. Maybe SendGrid has some smart algorithms for detecting this.
I also had this issue. I contacted sendgrid support however while waiting I changed the dummy data to real data I was calling sendgrid API with and then it started working.
in particular I changed the from email address to show my own domain.
For me the template was the issue. When using the template engine make sure to use {{variable}} instead of {{ variable }}. Afterwards I received the emails as usual.
We have a website where we allow you to reset your password (say if you forget your password). This is standard on many websites. Basically you enter your email address which you've used to register on our website, then we send you an email containing an email reset link.
This is all standard stuff. However, the problem is: Gmail somehow thinks this email we send to the user is spam, and puts it in the Spam folder.
The specific message Gmail shows is:
Be careful with this message. Our systems couldn't verify that this message
was really sent by xyz.com. You might want to avoid clicking links or replying
with personal information.
Let me explain how we send the email. We use the company sendgrid.com to deliver
the emails. xyz.com is a domain we control. (xyz is a pseudo-name here.)
The email's from address is: do-not-reply#xyz.com
We have changed xyz.com's SPF record to include "sendgrid.com" (and "sendgrid.net" "sendgrid.me").
There's no website associated with xyz.com, however.
My question is: what else can we do to make Gmail believe the email is from the domain xyz.com? So it doesn't put the email in the spam folder?
Thank you.
Did you end up publishing DKIM with Sendgrid? Also, I have a feeling your SPF record isn't quite right as generally there's one official entry per email provider. You mention adding several. I'd recommend looking at their docs for exactly what they recommend publishing in your SPF. Do this for any provider you use for any kind of email.
Since you mentioned Sendgrid as your ESP, here are Sendgrid's instructions. Once you've done the DNS you have to ask Sendgrid to "sign" it. Since DKIM uses cryptography you'll need them to do their side.
DKIM's less complicated than it sounds. The DNS records you have to add will take a few minutes then presumably open a ticket to Sendgrid to have them do their side.
Also, as an aside, could you post what you have for your SPF record here? I don't mean your domain but what the value is? It's not directly causing the problem but it's a key component of email authentication.
Once you've completed SPF and DKIM, it is critical you validate them both. Do a search for SPF validates and DKIM validator to find online tools.
I use an email registration and confirmation in my project (yes, I know about OpenID. In my counry, a main email service lacks it).
Sometimes users misspell their email addresses. I know about this due to "message could not be delivered" letters in a mailbox. A misspelled address is absolutely correct, because I check it with a regular expression - say, jon.doe#example.com intstead of john.doe#example.com. And I do not want to duplicate a email field in a registration form (who likes it?).
Request processing routine cannot wait for email delivery - it could take an unpredictable time. So, my script will return to user a confirmation message "An email was sent". And the user will wait for it forever (of cause, not - he/she will turn to an alternative project with a more perfect registration system).
Does someone knows how it can be improved (in any programming language)?
If you cannot wait for a response back from a confirmation e-mail, I would display a yes/no message box confirming their e-mail (i.e Send e-mail to jon.doe#example.com?) before sending the e-mail. While your at it, you can check to see if the e-mail is a duplicate as well.
If no duplicate found and they click "Yes" then send the e-mail. If not, ask them to re-enter the e-mail address.
With this approach you have gone above and beyond validating correct e-mail addresses since there are no services that will explicitly check for correct/valid addresses.
Unfortunately, there's no canonical way to spell anything in an e-mail address and no way to check programmatically whether it's spelled correctly or not. The only tool that can confirm the correctness of a valid e-mail address is Eyeball 1.0, which runs client side.
You can try the SMTP VRFY on the server but most won't reply with anything valuable. Most will simply say "go ahead and try it" because returning something useful would be invaluable to spammers.
You can always check the domain name with a DNS lookup but the only way to check an email address is to send a message.
Try to have the user type his email twice with an email confirmation field (as for the password).
EDIT : Anthony's idea is good too.
For user parts of email the only thing I can think of is using:
MX lookups (if mail provider doesn't block them... since that can be abused to filter email lists)
Requiring user to enter email twice to ensure he doesn't misspell user portion of email
As for domain part of email, take a look at this:
AngularJs directive to offer suggestions for misspelled emails?
It helped me to reduce bounce for misspellings like gmai.com and gmail.con.
At the moment, we are sending an email address verification email each time someone signs up. This email has been causing a number of problems: people don't get it, they just don't click the link in the email or the email gets block by spam or some other method. We are working on resolving the spam issue, although I don't think it's possible to completely resolve it.
I'm wondering what other methods there might be for verifying and email address. Is there any other way to verify an email address without sending an email? Or is there another method of ensuring people aren't signing up with fake information?
I'm not sure if there are other good methods, but sending an email and having them click a link is definitely the simplest and most accurate.
A main feature to sending that email, is for the person to verify that it's actually them that requested it.
The only way to verify someone owns an email address is to have him use it.
As for verifying users don't enter fake information - not even sending an email can help. With so many disposable/temporary email services out there (like GuerrillaMail) , someone can fill up your form with false info, post a temp email address, log to that address and click the link in your email - manually or programatically.
You have to trust your users to come back for your content, and ignore spammers.
strikeiron.com offers a paid web service to verify if an email exists without sending a message to that email. try it out here is the link: strick
http://www.strikeiron.com/Catalog/ProductDetail.aspx?pv=5.0.0&pn=Email+Verification