My company have a website on which, at some point, users are asked to register.
Until a few weeks ago, we used to verify the mail adress by sending an email.
The user had to log into their mail account and click on a link in order to validate their account.
Problem is, we had many users having difficulties (email not received...) or simply leaving the website at this step. Moreover, our support team had to deal with many call about those difficulties.
We decided to remove this verification and it had many positive effects, the first being less ticket for our support team.
However from this point, we had many "fake" emails and I would like to know, what can I do to limit those fake emails without the verification described above? Is there any non-intrusive methods to verify an email adress ?
An exemple of (probably) fake email is hyuiuyhajsdv#gmail.com
Here are what we're already doing to prevent fake emails :
We're using PHP filter_var to validate the format
I know it's not perfect but we didn't found anything better
We're doing a DNS verification of the domain name (with gethostbyname)
This prevents something like superemail#test1245.com
Finally, we're checking if the domains name match a disposable email service.
Note: we don't have any problem with bots creating fake accounts, so a captcha won't help.
Related
My company is developing a cloud contact management service and on our iOS app we're having some problems launching a particular enterprise email client app when the user presses the "Email" button on one of their contacts.
One member of our team came up with an idea to get around the problems with this enterprise app:
We let the user specify their email address in the app's settings and create our own email composing screen. Tapping the email button on a contact would open the composing screen, they would write their message and then we would send it on their behalf from our servers (or via service like mailchimp).
Basically, this would mean we would have to create an endpoint on our api that would accept a POST request with 'from', 'to', 'subject', and 'body' fields which would send the appropriate email.
This seems like a very bad idea as it's essentially creating a free, anonymous email service that could easily be abused send spam.
A few extra notes about our setup:
We don't verify an accounts email when they sign up
Even if we did verify the account's email, the user would need to be able to specify any email, as they may have signed up with personal email, but want to email someone from their work email.
Our API doesn't currently have any kind of rate limiting
Instead of having a from field in the request, we could instead send the id of the contact they want to email. This doesn't really change anything because if someone wants to abuse the send email endpoint they can also abuse the create contact endpoint.
So exactly how much of a bad idea is this, and how can I convince my team not to do this?
A few thoughts against doing it:
This is the perfect spamming service, which could damage the reputation of your company (reputational risk).
Your email servers would very quickly make it into blacklists (RBLs), making your outgoing emails land in spam folders in very many recipients' mailboxes.
Even if your servers are not yet in RBLs, if you send a forged email like that and proper email security is set up at the recipient end, your emails will still have a good chance to get classified as spam. Have a look at things like SPF and DKIM.
This could even have legal implications. Imagine the scenario when one of your users uses this service for something like blackmail. Would you be able to prove it was not you? Probably yes with the right controls, but would you want the hassle?
Still on the legal side, many countries (the EU, mainly) have data protection regulations which strictly control how personal data like email addresses can be used, especially for commercial advertisement. You probably want to adhere to that, but that would be hard with such a service (note that I'm not a lawyer, in such a case it's probably the abuser of your service that would offend these regulations and not you, I don't know, but it's something to consider).
If anyone can just send emails, it will be fairly easy to perform a denial of service attack against your services.
A few controls you could implement to mitigate some risks:
When adding a sender (from) address, you should validate that by for example sending a (cryptographically random) token and checking if the user can send it back (eg. by clicking a link in the email). If he can, that proves to some extent that he controls the email address and is probably a valid sender.
Limit the possible recipient addresses if you can. The best would be if recipients had to opt in to receive emails. If this is not possible, at least let recipients opt out from further emails. For this, you would have to add something like a footer to emails with "never again" links, and implement a facility to maintain recipients to which you must not send anymore.
Implement rate limiting. Depending on your exact scenario and use case, only allow to send the least number of emails acceptable for your application.
Implement proper logging so that you have an audit log of who exactly sent what email to whom. For this, log metadata like IP addresses as well. For this, you will likely have to authenticate your users.
On an operational level, have monitoring in place, and be prepared to ban offending users, based on a clear ToS shared with your users.
I'm making a user management system for my app, and I need to send users a "forgot my password" email with a token that lets them reset their account password. I signed up for SendGrid through Azure (to get the 25,000 emails per month free, which sounded like a great deal) and wrote some code to use it, but after testing my program a bit I was dismayed to find that only a couple of my emails actually went through.
After going onto the SG control panel, I found that 4 out of the 6 test emails I sent went through, and all of the others were rejected as being spam. I sent an email to mail-tester.com to see what it though my spam score was and it gave me a 4.3/10.
The email in question was a single sentence with a link to the password reset, without any images or other elements. I only sent those 6 emails out, so the volume of my emails definitely wasn't the issue. Still, I'm very puzzled as to why my messages are getting flagged as spam.
Without going to the trouble of making an elaborate authentication setup, are there any basic changes I can make to my system to make it get through to users?
In this case it's most likely because you are sending such a short message, with a link to 'reset your password' from a non-whitelabelled email address (the email address you're sending from cannot be verified against the actual domain), and the link may also be a different URL. It's probably getting pulled up as a potential phishing email.
You can rectify this by white labeling your domain and email links via the SendGrid dashboard, it's easy to do and should improve your deliverability.
Also check out this article from the SendGrid support team about White Labeling.
A question from 2015 which is sadly still relevant today as usage of SendGrid increases.
My organization has blocked all SendGrid mails except for those on the paid tier using fixed IP addresses with resolvable public DNS names (such as sendgrid1.sampledomain.tld) which we then whitelist.
There are now far too many domain impersonation, phishing and other spam mails coming in from SendGrid for us to allow everything from them - roughly 10 000 mails over a seven day period, which is far too many to manually report to SendGrids abuse department.
So my answer would be that switching to the paid tier of SendGrid is the better option if you like a better chance of your mails arriving intact at their destination.
I receive only Spam Mails from Sendgrid.
Goes direct to Spam folder and try to report Sendgrid everywhere I can. Maybe they get blocked by most mail servers and make them think about their policy in "hosting" all these Spammers.
In my case my emails are marked as spam because of the anchor label different to the href being actually called.
And that's because of the 'click tracking' setting of sendgrid.
So, if you have something like
yourdomain.com
sendgrid may replace the href and you end up with something like:
yourdomain.com
The sendgrid page being called tracks the click and then redirects the user to the url you originally set. But this sometimes results in your email being marked as spam.
Try to set 'click tracking' in sendgrid dashboard to off: settings | tracking | click tracking.
details here: https://sendgrid.com/docs/ui/account-and-settings/tracking/
Always start by setting up Domain Authentication, formerly known as domain whitelabel as #MartynDavies says. Found under Settings -> Sender Authentication in the UI. Should look like this:
https://sendgrid.com/docs/ui/account-and-settings/how-to-set-up-domain-authentication/
To identify problems have a look at Activity and choose to see deferred, drops, bounces, blocks and spam reports.
https://app.sendgrid.com/email_activity
Under Suppressions you can see details for Blocks and Bounces among others:
https://app.sendgrid.com/suppressions/blocks
https://app.sendgrid.com/suppressions/bounces
There you can see errors like:
550 5.7.1 SPF check failed. em1234.mydomain.com does not declare 11.222.33.44 as a valid sender
If it says Verified but you see errors like this then contact SendGrid support.
One thing that has worked is to upgrade from the Free plan to Essentials or Bronze via the Azure Portal. This made a lot of the emails marked as spam pass through.
I had a similar issue when trying to send a user verification email using SendGrid.
In my case, using a custom domain as the sender identity solved the issue.
Make sure to also verify the domain before using it.
We have a website where we allow you to reset your password (say if you forget your password). This is standard on many websites. Basically you enter your email address which you've used to register on our website, then we send you an email containing an email reset link.
This is all standard stuff. However, the problem is: Gmail somehow thinks this email we send to the user is spam, and puts it in the Spam folder.
The specific message Gmail shows is:
Be careful with this message. Our systems couldn't verify that this message
was really sent by xyz.com. You might want to avoid clicking links or replying
with personal information.
Let me explain how we send the email. We use the company sendgrid.com to deliver
the emails. xyz.com is a domain we control. (xyz is a pseudo-name here.)
The email's from address is: do-not-reply#xyz.com
We have changed xyz.com's SPF record to include "sendgrid.com" (and "sendgrid.net" "sendgrid.me").
There's no website associated with xyz.com, however.
My question is: what else can we do to make Gmail believe the email is from the domain xyz.com? So it doesn't put the email in the spam folder?
Thank you.
Did you end up publishing DKIM with Sendgrid? Also, I have a feeling your SPF record isn't quite right as generally there's one official entry per email provider. You mention adding several. I'd recommend looking at their docs for exactly what they recommend publishing in your SPF. Do this for any provider you use for any kind of email.
Since you mentioned Sendgrid as your ESP, here are Sendgrid's instructions. Once you've done the DNS you have to ask Sendgrid to "sign" it. Since DKIM uses cryptography you'll need them to do their side.
DKIM's less complicated than it sounds. The DNS records you have to add will take a few minutes then presumably open a ticket to Sendgrid to have them do their side.
Also, as an aside, could you post what you have for your SPF record here? I don't mean your domain but what the value is? It's not directly causing the problem but it's a key component of email authentication.
Once you've completed SPF and DKIM, it is critical you validate them both. Do a search for SPF validates and DKIM validator to find online tools.
my question is about the workflow of a web registration.
1) register with email + basic data
2) activate the account with a special secret link <- is this necessary?
3) allow the user to log in to the system
EDIT: I want to make the process as simple as possible without a password to choose/remember.
In more detail:
After a user is registering on a web site I sent out a confirmation with a generated password to login on the site and proceed.
Many sites sent an activation link first and then allow logging in to the system.
Is there any reason to do this additional step when I generate the password and sent it out to the user?
Thanks for your answers.
Is there any reason to do this additional step when I generate the password and sent it out to the user?
To ensure that the provided email address exists, and belongs to the person who registered the account.
I've noticed an increasing number of websites which skip this step. It seems to be a trend.
The purpose of the activation link is to guarentee that the email address provided by the user is one to which they have access. If you are generating a password and sending it via email to the user's email address, then the link is not required (because them logging in means that they read your email).
However, email is generally not a secure way to distribute information over the internet. You are sending them the password in plaintext, and you do not know how many people have access to that email account (e.g. a shared family account). I think you would be better off allowing the user to choose their own password at registration and then send them a link (offer to generate a password for them on the registration page, if you really believe that generating it is better).
The validation of an email account is usually to help prevent someone creating numerous accounts. This helps prevents spammers and various other bad people from attacking your site from different accounts.
In general you're trying to ensure that the person is who they say they are and that you have an outside means of communicating with them.
1 - to ensure that the email is belong to the registerd user.
2 - to make it harder to the people want to create many accounts (like forums where a single person have so many accounts to use them in voting or somthing).
I remembered a funny site that gives you a 10 minutes email , just to skip the process of creating a new email or even spamming your email by the sites you've registered in.
This way you make sure that the email address is valid and it will be more difficult for a spider to generate many users than without this step. Also, you might do a lot of things in your database when a user is registered and you can do these after the user is validated, to save time by not creating extra traffic on your database server for fake users.
I'm building a system that allows people to submit text and photos via email in addition to standard access on the website. I'm trying to weight the security advantages of two strategies in particular for verifying submissions from a user. Here they are as follows:
To based auth: Create a secret email address per user and present this to the user for submission. This strategy has the advantage that people can send from multiple devices that might be setup with different mail accounts
From based auth: Only accept emails from addresses that are registered in the user database. The idea being that it is impractical/difficult to impersonate registered users based on the sending address.
Can you think of other possible solutions? Which strategy of the ones proposed makes the most sense to you?
I would suggest that you not use From based authentication, at least not without some additional credentials (a passphrase, etc)
It's way too easy to forge, and certainly not difficult if you know someone's email address.
If you echo the email back to the user for confirmation, you can make things a little more difficult, but realize that your service can end up being used as a sort of spamming relay. (I could send 100 upload requests to you, with a forged FROM address, and you'd go ahead and spam the real person with 100 confirmation requests)
The better option is to check the registered email address but add the need for a code within the email subject known to the user. This way if they forge the email from address, they would still need a key to authenticate the incoming email.
I would go with "from" + confirmation, to avoid forging.
I.e. receive the email, but send a response with auth token in the subject line (or in the body) back to the "from" address. The user either will need reply, or click a link to confirm the submission.
And you post the content only after confirmation.