I am working on a project that has to store users passwords. With that password you can gain access to a user achievements and stuff so it's really important that you can not get the password even if you hacked into the database. My problem is which hashing function to choose in security and efficiency level.
Right now I am using sha256 with salt and pepper but I read that using a slow hashing function like Bcrypt with cost factor of 12 can be superior. And if it does, how much more security do I gain from using a pepper as well because it's really time consuming for hash function like Bcrypt.
My question is what hash function should I use base on the assumption that I do not expect to get hacked by a global hacking organization with supercomputers?
Plain crypto hash functions like sha2 or sha3 don't cut it anymore for password hashing, because it's too efficient to compute them, which means an attacker that controls many cpu cores can compute large tables relatively quickly. In practice the feasibility of this still depends on the length of passwords mostly (and the character set used in those password to some extent), but you should still not use these unless you really know why that will be acceptable.
Instead, you should be using a hash or key derivation function more suitable for password hashing. Pbkdf2, bcrypt, scrypt and Argon2 are all acceptable candidates, with somewhat different strengths and weaknesses. For a regular web app, it almost doesn't matter which of these you choose, and all will be more secure than plain hashes.
The difference will be very real if your database is compromised, properly hashed passwords will likely not be revealed, while at least some passwords with sha likely will, despite salt and pepper.
Related
I am using Scrypt to get a hash for my input and I didn't use SHA3 because I found out that it can be bruteforced with a dictionary attack to find the SHA3 output. Later I was told not to use Scrypt because it's unnecessary and just hash the output of SHA3 a million times, as it would be simpler but also more secure.
Is that true? or is using Scrypt still a fine choice?
No, just hashing the password a million times is not more secure than scrypt.
There are at least two things that are missing:
the use of a salt, which differentiates the hash when users use the same password and prevent rainbow tables;
the memory usage of scrypt which can make it harder to crack passwords using specialized hardware.
What you are trying to re-implement is a password hash or PBKDF (Password Based Key Derivation Function, the same thing but to derive keys instead of hashes). There has been a password hashing competition not too long ago which Argon2 won. Baloon hashing is a later password hash created by a team of cryptographers.
I don't know which of your co-workers or acquaintances think that they could do better, but I think that they should learn about the Dunning-Kruger effect.
Hearing about all the recent hacks at big tech firms, it made me wonder their use of password storage.
I know salting + hashing is accepted as being generally secure but ever example I've seen of salting has the salt key hard-coded into the password script which is generally stored on the same server.
So is it a logical solution to hash the user's password initially, pass that hash to a "salting server" or some function stored off-site, then pass back the salted hash?
The way I I'm looking at it is, if an intruder gains access to the server or database containing the stored passwords, they won't immediately have access to the salt key.
No -- salt remains effective even if known to the attacker.
The idea of salt is that it makes a dictionary attack on a large number of users more difficult. Without salt, the attacker hashes all the words in a dictionary, and sees which match with your users' hashed paswords. With salt, he has to hash each word in the dictionary many times over (once for each possible hash value) to be certain of having one that fits each user.
This multiplication by several thousand (or possibly several million, depending on how large a salt you use) increases the time to hash all the values, and the storage need to store the results -- the point that (you hope) it's impractical.
I should add, however, that in many (most?) cases, a very large salt doesn't really add a lot of security. The problem is that if you use, say, a 24 bit salt (~16 million possible values) but have only, say, a few hundred users, the attacker can collect the salt values you're actually using ahead of time, then do his dictionary attack for only those values instead of the full ~16 million potential values. In short, your 24-bit salt adds only a tiny bit of difficulty beyond what a ~8 bit salt would have provided.
OTOH, for a large server (Google, Facebook, etc.) the story is entirely different -- a large salt becomes quite beneficial.
Salting is useful even if intruder knows the salt.
If passwords are NOT salted, it makes possible to use widely available precomputed rainbow tables to quickly attack your passwords.
If your password table was salted, it makes it very difficult to precompute rainbow tables - it is impractical to create rainbow table for every possible salt.
If you use random salt that is different for every password entry, and put it in plaintext right next to it, it makes very difficult for intruder to attack your passwords, short of brute force attack.
Salting passwords protects passwords against attacks where the attacker has a list of hashed passwords. There are some common hashing algorithms that hackers have tables for that allow them to look up a hash and retrieve the password. For this to work, the hacker has to have broken into the password storage and stolen the hashes.
If the passwords are salted, then the attacker must re-generate their hash tables, using the hashing algorithm and the salt. Depending on the hashing algorithm, this can take some time. To speed things up, hackers also use lists of the most common passwords and dictionary words. The idea of the salt is to slow an attacker down.
The best approach to use a different salt for each password, make it long and random, and it's ok to store the salt next to each password. This really slows an attacker down, because they would have to run their hash table generation for each individual password, for every combination of common passwords and dictionary words. This would make it implausible for an attacker to deduce strong passwords.
I had read a good article on this, which I can't find now. But Googling 'password salt' gives some good results. Have a look at this article.
I would like to point out, that the scheme you described with the hard-coded salt, is actually not a salt, instead it works like a key or a pepper. Salt and pepper solve different problems.
A salt should be generated randomly for every password, and can be stored together with the hashed password in the database. It can be stored plain text, and fullfills it's purpose even when known to the attacker.
A pepper is a secret key, that will be used for all passwords. It will not be stored in the database, instead it should be deposited in a safe place. If the pepper is known to the attacker, it becomes useless.
I tried to explain the differences in a small tutorial, maybe you want to have a look there.
Makes sense. Seems like more effort than worth (unless its a site of significant worth or importance) for an attacker.
all sites small or large, important or not, should take password hashing as high importance
as long as each hash has its own large random salt then yes it does become mostly impracticable, if each hash uses an static salt you can use Rainbow tables to weed out the users hashs who used password1 for example
using an good hashing algorithm is also important as well (using MD5 or SHA1 is nearly like using plaintext with the mutli gpu setups these days) use scrypt if not then bcrypt or if you have to use PBKDF2 then (you need the rounds to be very high)
I'm a little confused about password-safe-keeping.
Let's say I've got database with user-account table.
And this is the place where i keep passwords.
At this time i'm using salted sha1.
I read Blowfish based function are better then sha1 because they need more time to process request.
Is there any reason why not to use salted sha1 and just limit login attempt count to some reasonable number (for example 50times per hour) as a 'firewall' to bruteforce attacks?
person who is working with this database has no need to bruteforce anything because
he can change records by queries.
With blowfish based function, you surely mean the BCrypt hash function. As you already stated BCrypt is designed to be slow (need some computing time), that's the only advantage over other fast hash functions, but this is crucial.
With an off-the-shelf GPU, you are able to calculate about 3 Giga hash values per second, so you can brute-force a whole english dictionary with 5'000'000 words in less than 2 milliseconds. Even if SHA-1 is a safe hash function, that makes it inappropriate for hashing passwords.
BCrypt has a cost factor, which can be adapted to future, and therefore faster, hardware. The cost factor determines how many iterations of hashing are performed. Recently i wrote a tutorial about hashing passwords, i would invite you to have a look at it.
Your point about restricting login attempts makes sense, but the hashing should protect the passwords in case the attacker has access to the database (SQL-injection). Of course you can limit the login attempts, but that has nothing to do with hashing, you could even store the passwords plaintext in this scenario.
Storing passwords in Blowfish is more secure than SHA-1 because, as of now, there has been no reported method of obtaining the value of a Blowfish-encrypted string. SHA-1, on the other hand, does have reported methods of obtaining data from encrypted strings. You cannot trust SHA-1 to prevent someone from obtaining its data.
If you are open to suggestion, I don't see a need to work with two-way encryption at all as you are storing passwords. Hashing your users passwords with a salted SHA-256 method may be an option. Allowing your users to reset their own passwords via Email is generally considered a good policy, and it results in a data set that cannot be easily cracked.
If you do require two-way encryption for any reason, aside from Blowfish, AES-256 (Rijndael) or Twofish are also currently secure enough to handle sensitive data. Don't forget that you are free to use multiple algorithms to store encrypted data.
On the note of brute forcing, it has little to do with encrypted database storage. You are looking at a full security model when you refer to methods of attack. Using a deprecated algorithm and "making up for it" by implementing policies to prevent ease of attack is not considered a mature approach to security.
In Short
Use one way hashing for storing passwords, allow users to reset via email
Don't be afraid use multiple methods to store encrypted data
If you must use an encryption/decryption scheme, keep your keys safe and only use proven algorithms
Preventing brute force attacks is a good mindset, but it will only slow someone down or encourage them to search for other points of entry
Don't take this as gospel: when it comes to security everyone has different requirements, the more research you do the better your methods will become. If you don't completely encapsulate your sensitive data with a full-on security policy, you may get a nasty surprise down the track.
Source: Wikipedia, http://eprint.iacr.org/2005/010
Is there any reason why not to use salted sha1 and just limit login
attempt count to some reasonable number (for example 50times per hour)
as a 'firewall' to bruteforce attacks?
If you don't encrypt your passwords with any decent algorithm you are failing basic security precautions.
Why isn't 'just' blocking login attempts safe?
Well beside the fact you would need to block EVERY possible entrance, eg:
ssh
webservices (your webapp, phpmyadmin, openpanel, etcetera)
ftp
lots more
You would also need to trust every user that has access to the database and server, I wouldn't like people to read my password, but what I dislike even more, is you deciding for me, metaforically speaking :-)
Maybe someone else can shed light on the Blowfish vs SHA discussion, although I doubt that part is a stackworthy formatted question
I am currently using SHA256 with a salt to hash my passwords. Is it better to continue using SHA256 or should I change to SHA512?
Switching to SHA512 will hardly make your website more secure. You should not write your own password hashing function. Instead, use an existing implementation.
SHA256 and SHA512 are message digests, they were never meant to be password-hashing (or key-derivation) functions. (Although a message digest could be used a building block for a KDF, such as in PBKDF2 with HMAC-SHA256.)
A password-hashing function should defend against dictionary attacks and rainbow tables. In order to defend against dictionary attacks, a password hashing scheme must include a work factor to make it as slow as is workable.
Currently, the best choice is probably Argon2. This family of password hashing functions won the Password Hashing Competition in 2015.
If Argon2 is not available, the only other standardized password-hashing or key-derivation function is PBKDF2, which is an oldish NIST standard. Other choices, if using a standard is not required, include bcrypt and scrypt.
Wikipedia has pages for these functions:
https://en.wikipedia.org/wiki/Argon2
https://en.wikipedia.org/wiki/Bcrypt
https://en.wikipedia.org/wiki/Scrypt
https://en.wikipedia.org/wiki/PBKDF2
EDIT: NIST does not recommend using message digests such as SHA2 or SHA3 directly to hash passwords! Here is what NIST recommends:
Memorized secrets SHALL be salted and hashed using a suitable one-way
key derivation function. Key derivation functions take a password, a
salt, and a cost factor as inputs then generate a password hash. Their
purpose is to make each password guessing trial by an attacker who has
obtained a password hash file expensive and therefore the cost of a
guessing attack high or prohibitive. Examples of suitable key
derivation functions include Password-based Key Derivation Function 2
(PBKDF2) [SP 800-132] and Balloon [BALLOON].
SHA256 is still NIST Approved, but it would be good to change to SHA512, or bcrypt, if you can.
The list of NIST approved hash functions, at time of writing, is: SHA-1, SHA-224, SHA-256, SHA-384, SHA-512, SHA-512/224, SHA-512/256, and SHA3-224, SHA3-256, SHA3-384, and SHA3-512, SHAKE128 and SHAKE256.
See https://csrc.nist.gov/projects/hash-functions
Depending on what operating system you are running, you probably don't have access to the SHA3 or SHAKE hash functions.
Many people prefer bcrypt to SHA512, but bcrypt is also only available on some operating systems.
SHA512 will be available on your system, or if not, you probably have such an old system that choice of hashing algorithm is the least of your problems.
One reason commonly given for preferring bcrypt is that bcrypt is tuneable - you can increase the number of rounds (work factor) to increase the time it takes to crack bcrypt hashes.
But SHA256 and SHA512 are also tuneable. While the default is 5000 rounds, you can specify more if you wish. 500000 takes my current pc about 0.45 seconds to calculate, which feels tolerable.
e.g.:
password required pam_unix.so sha512 shadow rounds=500000 ...
The reason to change from SHA256 to SHA512 is that SHA256 needs a lot more rounds to be as secure as SHA512, so while it's not insecure, it's less secure.
See, for example: https://medium.com/#davidtstrauss/stop-using-sha-256-6adbb55c608
Crypto changes quickly, so any answer you get might be proved wrong tomorrow, but current state of the art is that while bcrypt is possibly better than SHA512, SHA512 is fine.
If SHA512 is what you have available 'out of the box', use it (not SHA256), and don't worry about bcrypt or any of the SHA3 family until they become standard for your distribution.
As an aside, the current top rated answer has a number of claims that are either wrong or misleading.
"Switching to SHA512 will hardly make your website more secure."
This is misleading. Switching to SHA512 will make your site slightly more secure. SHA256 isn't as good as SHA512, but it isn't dreadful either. There's nothing that is clearly better than SHA512 that is likely to be available on your system yet. Bcrypt might be better, but this isn't clear, and bcrypt isn't available on a lot of systems. The SHA3 family is probably better, but it isn't widely available either.
"SHA256 and SHA512 were never meant to be password-hashing"
This is wrong. Both SHA256 and SHA512 are approved NIST hash algorithms.
"to defend against dictionary attacks, a password hashing scheme must include a work factor to make it as slow as is workable."
This is wrong. A high work factor will protect against brute force hash cracking, but not against a dictionary attack. There is no work factor that is low enough to be usable but high enough to protect against a dictionary attack. If your password is a dictionary word, it will fall to a dictionary attack. The protection against a dictionary attack to not use passwords that can be found in dictionaries.
On my current PC, the limit on rounds seems to be 10 million, which produces a delay of 8.74 seconds for each password entered. That's long enough to be extremely painful, longer than you'd want to use. It's long enough to prevent a brute force attack - but a determined adversary with a good cracking rig and a bit of patience could still iterate through a dictionary if they wanted to.
"A password-hashing function should defend against ... rainbow tables"
This is, at best, misleading. The defence against rainbow tables is to make sure that each password has their own 'salt'. That's pretty much standard practice these days, and it happens before the hash function is called. (Salting means adding a random string to the password before hashing it. The salt is stored with the password, so it's not a secret, but it does mean that even if a user picks a well-known password, the attacker can't just recognise that {this hash} belongs to {that password}, they still need to crack the hash.)
"Currently, the best choice is probably Argon2. This family of password hashing functions won the Password Hashing Competition in 2015."
This is unclear. Any 'new' cryptographic function can have unobvious ways of being broken, which is why most people prefer functions that have been widely used. Besides which, Argon2 is probably not available to you.
"Other choices, if using a standard is not required, include bcrypt and scrypt."
This is unclear. At one point, scrypt was seen as a better bcrypt. However, for various reasons, sentiment has moved away from scrypt towards bcrypt. See, for example: https://blog.ircmaxell.com/2014/03/why-i-dont-recommend-scrypt.html
To repeat, at this point in time, SHA512 appears to be a good choice and so does bcrypt.
SHA512 is NIST approved and bcrypt is not.
SHA512 will almost certainly be available on your system. Bcrypt may or may not be.
If both are on your system, I'd probably recommend bcrypt, but it's a close call. Either is fine.
This has already been answered reasonably well, if you ask me: https://stackoverflow.com/questions/3897434/password-security-sha1-sha256-or-sha512
Jeff had an interesting post on hashing, too: http://www.codinghorror.com/blog/2012/04/speed-hashing.html
Note that SHA512 is a lot slower to compute than SHA256. In the context of secure hashing, this is an asset. Slower to compute hashes mean it takes more compute time to crack, so if you can afford the compute cost SHA512 will be more secure for this reason.
SHA512 may be significantly faster when calculated on most 64-bit processors as SHA256ses 32-bit math, an operation that is often slightly slower.
Outside of the really good and more practical/accurate answers regarding passwords, I have another perspective (one that I think is complementary to the others).
We use tools and companies to perform vulnerability assessments. One red flag we've had in code is use of MD5. This was not anything related to passwords... it was simply to generate a digest for a string. MD5 is nice and short, and really not a security issue for this specific scenario.
The problem is, it takes time to configure scanners to ignore these false-positives. And it is much more difficult to modify a security report written by an external vendor, in order to change the "high risk" finding to "low risk" or removed.
So my view is, why not use a better algorithm? In my case, I'm starting to use SHA512 in place of MD5. The length is a bit obscene compared to MD5, but for me it doesn't matter. Obviously, one's own performance needs in either calculation or storage would need to be considered.
As an aside for my situation, switching from MD5 to SHA256 would probably also be okay and not raise any red flags... but that leads me to my "why not use a better algorithm" comment.
In a web application written in Perl and using PostgreSQL the users have username and password. What would be the recommended way to store the passwords?
Encrypting them using the crypt() function of Perl and a random salt? That would limit the useful length of passswords to 8 characters and will require fetching the stored password in order to compare to the one given by the user when authenticating (to fetch the salt that was attached to it).
Is there a built-in way in PostgreSQL to do this?
Should I use Digest::MD5?
Don't use SHA1 or SHA256, as most other people are suggesting. Definitely don't use MD5.
SHA1/256 and MD5 are both designed to create checksums of files and strings (and other datatypes, if necessary). Because of this, they're designed to be as fast as possible, so that the checksum is quick to generate.
This fast speed makes it much easier to bruteforce passwords, as a well-written program easily can generate thousands of hashes every second.
Instead, use a slow algorithm that is specifically designed for passwords. They're designed to take a little bit longer to generate, with the upside being that bruteforce attacks become much harder. Because of this, the passwords will be much more secure.
You won't experience any significant performance disadvantages if you're only looking at encrypting individual passwords one at a time, which is the normal implementation of storing and checking passwords. It's only in bulk where the real difference is.
I personally like bcrypt. There should be a Perl version of it available, as a quick Google search yielded several possible matches.
MD5 is commonly used, but SHA1/SHA256 is better. Still not the best, but better.
The problem with all of these general-purpose hashing algorithms is that they're optimized to be fast. When you're hashing your passwords for storage, though, fast is just what you don't want - if you can hash the password in a microsecond, then that means an attacker can try a million passwords every second if they get their hands on your password database.
But you want to slow an attacker down as much as possible, don't you? Wouldn't it be better to use an algorithm which takes a tenth of a second to hash the password instead? A tenth of a second is still fast enough that users won't generally notice, but an attacker who has a copy of your database will only be able to make 10 attempts per second - it will take them 100,000 times longer to find a working set of login credentials. Every hour that it would take them at a microsecond per attempt becomes 11 years at a tenth of a second per attempt.
So, how do you accomplish this? Some folks fake it by running several rounds of MD5/SHA digesting, but the bcrypt algorithm is designed specifically to address this issue. I don't fully understand the math behind it, but I'm told that it's based on the creation of Blowfish frames, which is inherently slow (unlike MD5 operations which can be heavily streamlined on properly-configured hardware), and it has a tunable "cost" parameter so that, as Moore's Law advances, all you need to do is adjust that "cost" to keep your password hashing just as slow in ten years as it is today.
I like bcrypt the best, with SHA2(256) a close second. I've never seen MD5 used for passwords but maybe some apps/libraries use that. Keep in mind that you should always use a salt as well. The salt itself should be completely unique for each user and, in my opinion, as long as possible. I would never, ever use just a hash against a string without a salt added to it. Mainly because I'm a bit paranoid and also so that it's a little more future-proof.
Having a delay before a user can try again and auto-lockouts (with auto-admin notifications) is a good idea as well.
The pgcrypto module in PostgreSQL has builtin suppotr for password hashing, that is pretty smart about storage, generation, multi-algorithm etc. See http://www.postgresql.org/docs/current/static/pgcrypto.html, the section on Password Hashing Functions. You can also see the pgcrypto section of http://www.hagander.net/talks/hidden%20gems%20of%20postgresql.pdf.
Use SHA1 or SHA256 hashing with salting. Thats the way to go for storing passwords.
If you don't use a password recovery mechanism (Not password reset) I think using a hashing mechanism is better than trying to encrypt the password. You can just check the hashes without any security risk. Even you don't know the password of the user.
I would suggest storing it as a salted md5 hash.
INSERT INTO user (password) VALUES (md5('some_salt'||'the_password'));
You could calculate the md5 hash in perl if you wish, it doesn't make much difference unless you are micro-optimizing.
You could also use sha1 as an alternative, but I'm unsure if Postgres has a native implementation of this.
I usually discourage the use of a dynamic random salt, as it is yet another field that must be stored in the database. Plus, if your tables were ever compromised, the salt becomes useless.
I always go with a one-time randomly generated salt and store this in the application source, or a config file.
Another benefit of using a md5 or sha1 hash for the password is you can define the password column as a fixed width CHAR(32) or CHAR(40) for md5 and sha1 respectively.