Im trying to work through the architecture for a database that I need to protect the email addresses stored in it. Its a Mongo database and each document stores two email addresses and a load of other content. I want to make sure that the document is not attributable to the email addresses, however when my program needs to access the document it can access the email addresses too. It also may need searching by email address.
My understanding. Hash the email addresses before storing using AES256 (or similar symmetric encryption standard).
Store the database somewhere other than the program (and the key).
When you need to be able to search for the email address, encrypt the email query with the key and search for this in the DB. When you find it, pull all the documents.
You can then decrypt the other email address that wasn't used for search.
Seems fine, but where do you store the key? In an enviroment variable? But if the server has been hacked, then whats the point in the encryption?
On a seperate server?
What is database encryption protecting against? Just the database dump being stolen or the server getting accessed?
Please could someone walk me through how this is done as industry standard?
Thanks
Related
Please help!
I work for a telecoms company, everyday I receive stacks of emails from our ISP notifying me a customer is about to exceed there broadband data allowance. I am then required to contact each customer manually and inform them of the situation and advise a recommended solution.
The problem is this is they are coming in quicker than I can contact customers.
What I would like to do is automate this process (The email from the ISP can not be forwarded as it contains confidential information).
The system would need to extract/scrape the following data:
Telephone number, current data usage, data reset date.
This extracted data would need to be extracted into an email template.
It gets trickier - Now the system would need to establish which customer to send this email too!
The only way to identify a customer from the information contained in the email is the telephone number. I guess I would need to create a directory with each telephone number and associated customer email address.
Is this impossible to achieve? How would you guys go about achieving something like this? Any advice would be massively appreciated - In fact you would be a life saver.
Thanks,
Chris
we do something similar in an add-on to Microsoft Outlook we've made to manage projects and service calls (Invantive Vision). How it technically works is as follows:
the HTML variant of the email is sent to a webservice from within the Outlook add-in;
the webservice sends it to a stored procedure in the database;
the stored procedure searches and looks especially at the email address (since it seems more unique, many companies have a central telephone number with maybe 1.000 people behind it);
the stored procedure queries the customers and persons database;
it chooses a template in HTML with tags;
the tags are filled based upon the data found;
the results are given back for human validation.
Please note that the results do not have to be 100%; it will never be possible since people share telephone numbers and email addresses and might make typos. But 99% is reasonable and during the human validation phase the employee can correct and choose a different customer when necessary.
Sometimes 100% hits are possible in other scenarios. In that case we just archive the email on a project and let the system do everything automatically in the database.
Anyway, the crux is to bring the email to the data and let some type of stored procedure do the hard work and maybe sending and/or composing mail.
I am trying to figure out something about this article:
http://techcrunch.com/2012/09/20/facebook-crm-ads/
So, Facebook claims that they respect the privacy of your customer database and that's why they let you upload the hashed email addresses. BUT, what's the point?
It only works by virtue that they've also hashed all the email addresses of their uses to make the hits.
Presumably, the only good this does is that Facebook doesn't know the email addresses of your customers that aren't already Facebook users.
Am I missing anything else?
That's not to protect the email addresses from facebook, but to protect them the rest of the way. If there's a data leak somewhere in the process, chances are that only the hashed addresses will get leaked, making them much harder to misuse.
we usually send some verification link into email when users registered to verify users email.
The link may look similar to
http://www.example.com/register.php?id=12832&&unique_number=ij86435232as
it means that we have to store this unique number in our database to verify users identity.
I'm thinking that if we send user password into his email and tell them that
check your email , we sent your username and password
Because now we don't have to save additional unique number in our database , we are saving memory
so my question is that, is that anything wrong for this approach .
If you want your application to be considered secure, it's not wise to send plain-text passwords out via e-mail.
The amount of storage required to hold your unique number is trivial in the extreme, so I'd continue to do that.
Closed. This question is off-topic. It is not currently accepting answers.
Want to improve this question? Update the question so it's on-topic for Stack Overflow.
Closed 9 years ago.
Improve this question
I have to send sensitive information (name, address, social security number etc.) collected from a website, that has been entered by a user, to an e-mail address.
What is the best course of action to make the information secure and easy to extract on the receiver side?
Edit: I will be using ASP.NET for the website, not sure what it has for capabilities on this matter.
Edit: If I decide to store the information in a database and just send a mail when a new entry has been made, would this be better? And create some secure way to dump the information instead.
The best course of action would be to run the other way, fast. Redesign your application so that it doesn't enable identity theft.
You can use S/MIME or PGP to send secure email to most non-Web email clients, but it takes a lot of set up either way: the recipient has to have a certificate, and you have to get the right certificate for each recipient.
As an example of a better design, consider one where the recipient is mailed a notification, and then returns to the web site to view the information after authenticating securely over SSL.
While it helps to reduce the complexity of the system needed by the recipient, the bigger win is that it strengthens control over the distribution and retention of the sensitive information, and aids in auditing the access to that information. Sending someone an email makes it that much easier for them to store it unsafely, forever, or forward it to unauthorized recipients.
Encryption, Encryption, Encryption!!! Nothing else :)
Put the data into an encrypted attachment of the mail. The attachment can be any format you like. This will make parsing on the other side reliable. You can use any codec which ASP.NET supports, so you can choose anything that the other side can read.
Sadly, I don't think there is a practical solution for you here if you are taking about a mailto: URL.
The only secure way to send things over email is encryption. You can use PGP or S/MIME, but neither of those are easy to deploy for arbitrary users on web forms, since you can't easily set the user up with a PGP key or an X.509 certificate.
If you are deploying a web site from a hosted environment, it is possible that the code you control could encrypt the data and then send it. You could, for example, use GPG to encrypt with a public key of your own, and then send the results out in an email message. But that requires code, not just a mailto: url.
Unless you can find a public key encryption library in Javascript.
In short no. Email by default is un-protected. You use encryption programs to protect the information inside, but that requires the end user to be able to decrypt it.
The easiest way would probably be to create an ssl encrypted site where the user can log in and access the information.
You may want to look at http://ecocrypt.com/SecureMessage/
This will allow you to create an encrypted message that can be sent over standard email.
I think they have an API you can use for purchase/license( very low cost...)
But users can encrypt/decrypt in the cloud without having any set up requirements.
This will create a message like:
Encrypted Message Using http://ecocrypt.com/SecureMessage
To read this secure message do the following:
1. go to http://ecocrypt.com/SecureMessage:
2. Copy this ENTIRE message into the encrypted message area:
3. Enter the message pass phrase:
4. Press the Decrypt button:
NOTE: the message pass phrase was sent in a separate message, or the sender assumed you already have it.
NOTE: If you need or lost the pass phrase contact the sender at:test#ecocrypt.com
------------------HEAD--------------------------------
ECOCRYPT:67570:95992fad87a1165c100a0b915f86ce7f:
------------------START--------------------------------
GyuyjmHF68edfoSGM0YqtICXPrA6P69Pf7pXdgZ22g1PjzoANDOVy+0UJ/P0Pb+B09O+IsXqWPus
BO1gsVOA1BnMEE5r68A2fa02nRC9F3anVV8rvZDiZdfu9up2uDWrtsGhlLcHI6iKau4z7dAxq6qV
k7C/o2l3
------------------END--------------------------------
It can be decrypted if the user knows the correct pass phrase.
Same address in the cloud: http://ecocrypt.com/SecureMessage/
The message Decrypted:
Greetings,
This is a test Message.
Sensitive Information Goes Here:
name ss and other sensitive info.
Thanks.
I use Gmail SMTP which uses TLS. I'm not a .NET person but it's very easy to connect to through a Rails app for instance. And it's free!! Of course I think you only get about 200 msgs a month, but you can pay for more
If you have "control" of the receiver side (by this, I mean if the receiver is not a lambda user), I'll go for public/private key encryption with PGP. This article has a fully working example using GnuPG with .NET.
Let me rephrase the information flow to see if I got it right.
a) User U puts information into form at website W,
b) W then sends information to U via eMail.
That makes two communication processes that have to be protected. As you are only asking for protection of b) I assume you are already using TLS/SSL during a).
To protect b) you need some keys for encryption. I see two solutions: either some symmetrical secret that is exchanged during a) or some asymmetric public key that is exchanged during a). The latter one doesn't need any secrecy during a), which is an advantage, but you still need to authenticate that this key is indeed coming from U.
Both types of keys may be transferred during a), you probably just have to insert a new field into your form. In both cases U needs to have the necessary software to decipher any encrypted communication received in b). Having x509-Certificates or PGP-Keys as the two standard encryption algorithms these are probably the best guesses to start with. I am not sure how much is supported by ASP.NET, but even if I assume eMail encryption is not supported directly, some encryption should be supported.
Problem still is, that you will not find a solution that does not assumes some features of U's mailclient (above reading eMails, for some of us encryption is a fairly common feature).
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 6 years ago.
Improve this question
From security reasons, is it worth encrypting user emails before putting them into the database?
I know we hash and salt passwords but that's another story as we do not really need password originals. With emails it is different.
Knowing that the decryption key will anyway be somewhere close to the database, does it make sense to encrypt emails? I suppose if someone gets into the system, they will find the key as well, if not immediately then eventually.
What are the best-practices? Are there any other options available if I run my own servers and not on a shared/virtual hosting?
EDIT: I intend to use SQL Server. And no, it is no corporate software with security requirements, just some entertainment site I have in mind.
If you're going to need the email address in the future, then you'll have to store them in plain text.
You could encrypt them, of course, however, this is effectively security through obscurity in this case. Basically, if your application's perimeter is secure, your data within it can be plain text. Encrypting here adds complexity to you working with the data, but doesn't really stop an attacker from getting your raw data.
As you say, if he gets through your perimeter defenses, he's likely to easily get your decryption key to decrypt the email data. Encryption may slow down the determined attacker slightly, but will not add any real security to your data.
The best scenario is to hash the email address (with salt!) and store that. This allows you to check the email address against an input value (for example) and verify that the email address input is the same as what you have stored, of course, the major downside for this is that you can't know what the email address is without that additional value, so if you're wanting to (for example) regularly email your users, you'll be out of luck.
I suspect you're storing the email address because it's useful data, and you will want to do something with it (like send an email :) in which case, encrypting just adds overhead to working with that data, whilst gaining very little in return.
In this case, I would focus on securing access the database itself (i.e. your "perimeter" defenses) and ensure they are as strong as can be, whilst leaving the data in the database in plain text.
Hopefully this answer will answer your question as well.
Is it worth encrypting email addresses in the database?
In short, no, it is not worth encrypting user email addresses. You're right in thinking that a database compromise will likely result in somebody also gaining access to the keys required to break your encryption.
In general I agree with others saying it's not worth the effort. However, I disagree that anyone who can access your database can probably also get your keys. That's certainly not true for SQL Injection, and may not be true for backup copies that are somehow lost or forgotten about. And I feel an email address is a personal detail, so I wouldn't care about spam but about the personal consequences when the addresses are revealed.
Of course, when you're afraid of SQL Injection then you should make sure such injection is prohibited. And backup copies should be encrypted themselves.
Still, for some online communities the members might definitely not want others to know that they are a member (like related to mental healthcare, financial help, medical and sexual advice, adult entertainment, politics, ...). In those cases, storing as few personal details as possible and encrypting those that are required (note that database-level encryption does not prevent the details from showing using SQL Injection), might not be such a bad idea. Again: treat an email address as such personal detail.
For your entertainment site this is probably not the case, and you should focus on prohibiting SELECT * FROM through SQL Injection, and making sure visitors cannot somehow get to someone else's personal profile or order information by changing the URL.
One of the most often-cited truisms in computer security is that the
only truly secure computer is one buried in concrete, with the power
turned off and the network cable cut.
With that in mind the best way to securely store email addresses? Dont store them at all!
tl;dr Do you need their email address, or a way of sending them emails? Either trust someone who will do a better job than you or don't use the email address at all.
Why do you need to keep a record of a customer's email address? The only reasons I have run into are:
Account confirmation & authentication
Transaction & Marketing emails
Confirmation & Authentication
The core of what we want is two step authentication: Something they know and something they have. Something they know is a password, and is easy to prove since they will be the only one who knows it. Something they have is harder to prove and traditionally we use an email address since it is easy to verify. These days though there are other things we can use:
Mobile phone
An account with a trusted website (Facebook, Google, Twitter)
Mobile phone verification is simple. Send them a sms using a service like twilio.com and ask them to text back a confirmation code. We now know that the mobile belongs to the customer who wanted to register. With OpenID you can verify existing accounts with other trusted sites, and the confirmation process is handled by them.
For the customer to authenticate then all they provide is either their mobile number and password, or an OpenID authentication token. Neither require a email address (well the OpenID provider might but thats not your responsibility).
If these are not an option then you can still confirm an email address and then use it for authentication. Confirmation only requires a unique token to be stored and a link to be sent to the email address. Store a salted hash of the email address, and use that to match the account in the same way we do passwords.
Transaction & Marketing Emails
The real reason why we want to store the email address! So we can send them offers of stuff we think they need so they can delete it without reading it. Seriously though is email the best medium for this? If we have an OpenID account then why not use that for notifications? Send a Facebook message or write on their wall, #mention them on Twitter, send a text message to their mobile, build an app and push notifications at them. There are so many channels much more effective than email.
If you want to use email then use a email platform like Mandrill and MailChimp. When they register create a subscriber in a mailing list on MailChimp. Store the subscriber id with the account. For transaction emails ( reset password, account updates ) fetch the subscriber and pass the stored email to Mandrill to send the email. For mass marketing just send to the mailing list in MailChimp.
The only thing stored in the database is the subscriber id. It also gives all the benefits of using a email platform, unsubscribes, open and click through rates, e-commerce tracking etc. Email platforms will do a better job of delivering emails that you. They will also do a better job at protecting the privacy of their data than you. Let them do the hard work of database security so you can focus on getting more customers.
I think that when people can come in your database you are anyway screwed :)
It doesn't make a lot of sense to just encrypt your email addresses. Beside that there will be a lot of other information in your database that you would not like to be gathered, the decryption key will be indeed within reach at the same time your database is open.
I would like to suggest to find your layer of security and data integrity on a higher level. So the prevention of people entering your database.
And why would email addresses be so important? Most people will anyway get spam or their email addresses will otherwise be available somewhere on the web.
Depends on how often you access the addresses. If you read them once in a while, it might make sense, but this would be one of the last security issues I would spend time on.
I do not encrypt user e-mails. The point is to protect the database; the keys are accessible anyway if you actually want to use the e-mails once they are stored.
Do check the address for validity and possible SQL injection, though.
If the application server and database are on separate servers, it would generally increase security to have all or parts of the database encrypted.
Even if they are on the same machine, a hacker may not figure out where your password is stored (although I wouldn't rely on that).
I generally wouldn't encrypt the emails at the application level, instead relying on database-wide encryption offered by most enterprise databases.
Of course if you're using something like MySQL, then you have no choice but to do it at the application level.
I normally tell my clients it isn't worth the trouble encrypting a database, however if you have stricter privacy requirements it may make sense to do so.
Encrypting database content is always a tricky consideration. Clearly the content is useless unless it can be unencrypted, and if that has to happen without human intervention, then you're storing both the cyphertext AND the key somewhere. If that somewhere is on the same machine, then one might wonder why you even bothered.
Well, there's a few reasons why you might want to do this. One is because you're required to do so because of some company policy. Another is that perhaps your database is housed in a more hostile environment than that machine that accesses it.
In general, encrypting database content isn't going to win you any awards, but if you can justify it, then you clearly have at least some motivation to do so.
yeah could be helpful for the user if you hash it with salt. I had a code before which i used that I use salt and hash then I can decrypt it. Flow is that once user will register you then hash and salt (encryption process) it. Then if you need to fetch the encrypted data there will be decryption.