I'm studying a migration from an old LDAP to a new database.
Today I just found that the passwords are all saved as {crypt} passwords and the new system does not support that.
Is it possible to decode the {crypt} passwords (in PERL if possible) ?
No, crypt implements a one way hash.
However, you have the hash values and (hopefully) the salt. Whenever a user submits a password, it's encrypted with the same salt, and if the hashes match, the user is authenticated.
In general, it's a very bad practice to have human readable passwords, but you don't need to read a raw input password to authenticate a user.
The answer should be no. The whole point of hashes such as crypt is that you can't do that. However, if it's using the old-fashion unix crypt(), it's often actually quite brute-forceable (it's rather weak cryptography). Question is, should you really want that.
This is sounding like an XY problem though. What LDAP servers are we talking about, and what formats do they support? There's a reasonable chance there's some kind of overlap that can be used for migrational purposes.
If implemented correctly a password stored as crypt can not be decoded into cleartext because it is not encrypted but hashed.
Depending on your target system you might be able to migrate the hashed passwords "as-is", but you have to determine which hash was used and if the same is supported by your target system.
You need to disclose more about the LDAP server and the database you use.
Related
I'm creating new login system for my single page app. This system will require administrator to create account for the users. Once they setup account for the user I will send them an email where they have to enter their Information like Security Question and Password. So I have done some research and looked over our existing system. There is hash function that is used together with salt. I read few articles and there is a lot of debate on hash being vulnerable. Also I see that in this case hashed password is stored as well as salt. They are in separate columns. Is this good practice to store salt in DB? Also is there better way to store password in database? Here is example of the logic that I found:
<cfset password = trim(FORM.password)>
<cfset salt = randomSalt()> //This is function that generates random salt.
<cfset totPW = password & salt>
<cfset hashedPW = hash(totPW,"SHA-256")>
I'm currently using Cold Fusion 2016. I'm not sure if there is some better way to encrypt the password in CF. If anyone can provide some useful resource or example please let me know. Thanks.
Generally speaking, hashing is still fine these days. The thing that matters here is what hashing algorithm you use (and how many iterations). In case of database leaks, you want to make it as difficult as possible to match inputs (commmon password/dictionary based attack). Salting does help a little bit, so does having an irregular pattern salt or a hidden number of iterations based on the username etc. Having different hashing strategies helps as long as the attacker doesn't know how your hashing is implemented. (Accessing the database server is one thing, accessing your source code another.) It's about causing effort to the attacker.
Now about hashing algorithms: SHA-2 is easier to attack than for example bcrypt due to being targetable by GPUs. The number of iterations on the hash will take the attacker more time for each password. bcrypt isn't supported by hash(), but at least SHA-512 is. hash() supports iterations (see docs). Rule of thumb is having an iteration count that takes at least a second to process on your server hardware.
On a side note: Don't trim the password input, people might intend to use leading/trailing whitespaces.
Why is it not a security hole that PostgreSQL by default stores user passwords in an MD5 hash? I am studying the internals of PostgreSQL and have gotten to the system catelog pg_authid and when I read about the MD5 hash encryption it appears that it is regarded as antiquated. In my thinking if an admin or a user is able to access the underlying file store then they could hypothetically crack the passwords and do whatever said credentials would enable.
I ask why it is not a security hole because apparently PostgreSQL has been "Common Criteria Certified" which seems to be military grade secure according to it's wiki which notes it's provenance from western defense organizations.
Thanks!
First, PostgreSQL 10 adds SCRAM-SHA256 based on SASL, making this a moot point.
For older versions: It's a weakness, but it's not a large security hole for a number of reasons:
Internet-deployed PostgreSQL instances should be using SSL, preventing eavesdropping on the protocol. This greatly reduces the chances of successful password theft.
Passwords are twice-salted. The password stored on disk is hashed with a salt and the md5 digest is taken. But the password sent on the wire is re-salted and re-hashed with an authentication-exchange specific salt, so if you capture a hashed password on the wire you cannot simply replay it in a later authentication.
If you manage to get a few copies of the same twice-salted password by eavesdropping a plaintext connection, you could potentially exploit weaknesses in MD5 to find the once-salted version that's stored on disk, and use that to authenticate with the DB.
But it's a lot of work, and it's pretty much totally prevented by using SSL.
Personally speaking, I think the "Common Criteria" is close to a pile bureaucratic nonsense. It applies only to one specific install with very narrow, specific versions of everything from hardware on up. It should help exclude total snake-oil, but it sure doesn't prove anything is secure. (Hell, look at Government systems...)
I have oracle database to move on to new postgresql server.
Some tables are having field sesitive and those are all encryted through DBMS_OBFUSCATION_TOOLKIT.DESENCRYPT/DESDECRYPT.
The problem is here. The size of postgresql's encrypted data size(bytea type) should be the same with oracle's.
I tried to get it done with aes(encrypt/decrypt) which takes almost three times larger than raw data.(oracle takes 16byte with des algorithm, postgresql takes 33byte with aes and the raw data is of 13byte.)
I tried the postgresql crypt also, but the manual doesn't metion the way of decrypting it back limiting 8byte of raw data size.
Now i really need encrypt method which takes as small encryted data size as possible and provides decrypt method also.
Is there a good way or the other options for me???
Thanks in advance.
Crypt and DES are old cyphers and should not be used
Plain old DES is an obsolete algorithm. You can't really usefully compare it to AES128; it's like complaining that a SHA256 hash is bigger than an MD5 hash - yep, it is, but only one of them might slow the attacker down for a while. DES was widely considered weak even in 1999 and should never be used in new applications. Do not use it.
I don't think it's a good idea to seek an encryption method that "provides the smallest data size possible" - because it's basically a waste of time to encrypt data using DES. Why not use ROT13 (caesar cypher)? The "encrypted" result is the same size as the input, pity the encryption can be broken by a 3-year-old.
crypt is of a similar vintage. The old UNIX crypt hashing algorithm is ... elderly ... and totally unsuitable for any new application. Hashes should be SHA256 at minimum, really.
Crypt is a one-way hash
As for not being able to figure out how to decrypt crypted data: crypt isn't an encryption algorithm, it's a cryptographic hash function or "one way hash". One way hashes are suitable for verifying that data is unmodified, comparing to a stored salted hash for password authentication, for use in challenge-response authentication, etc. You cannot decrypt crypted data.
Deal with the size
Use a decent cryptographic function and live with the size increase. bf or aes128 are about the weakest you can reasonably use.
Personally I prefer to do my encryption/decryption in the app, not in the DB. If it's done in the DB the keys can be revealed by pg_stat_statements, in the logs by log_statement or errors, etc. Better that the key never be in the same place as the stored data at all.
Most programming languages have good cryptographic routines you can use.
It's hard to offer any more advice as you haven't really explained what you're encrypting, why, what your requirements are, what the threat(s) are, etc.
Passwords?
If you're storing passwords, you're probably doing it wrong.
If possible, let someone else do the authentication:
OAuth or OpenID for Internet
SSPI, Kerberos/GSSAPI, Active Directory, LDAP bind, SASL, HTTP DIGEST, etc for intranet
If you really must do the auth yourself, add a salt to the passwords and hash the result. Store the hash and the salt. When you must compare passwords, salt the new plaintext from the user with the same salt you used for the stored hash, hash the new password+salt, and see if the hash is the same as what you stored. If it is, they gave the right password.
You almost certainly don't need to recover cleartext passwords. Implement a secure password reset instead. If you really, really must, use a decently secure algorithm like aes to encrypt them and think carefully about key storage and management. See other posts on SO about key storage/management with pgcrypto.
See also:
Secure method for storing/retrieving a PGP private key and passphrase?
Depending on how your postgresql was built, it may have DES support in the pgcrypto module. It depends on if Postgres was configured with OpenSSL support as it relies on OpenSSL to do DES (while with other more modern algorithms it implements them itself).
PGCrypto Algorithms
If openssl support was included and you specify DES as the algorithm to encrypt and decrypt, the data should be the same as you get from Oracle (although you may need to specify padding options).
As Craig says though, the DES algorithm is weak and one of the reasons it is weak is because the output ciphertext is so small.
I have a database with roughly 4,000 members. I need to decrypt all their passwords so that I may re-encrypt them using MD5 and put into a new system. I have one account with before encryption and after encryption password strings so with this I assume I should be able to figure out a method and key range. I do not have access to the original source code or I would not be here.
I should mention that existing passwords vary dramatically.
User1 - qwDyP1db9iOPI
User2 - $1$.WXtDyLg$yxBPAWxzd.srEWJZfqZAY/
User3 - do8sLMoVVqxKQ
User4 - TKmnAlPe
How do I approach such a problem?
Typically, you can't decrypt the passwords, because the passwords are not stored. What you are seeing is hashes of the password - there's no efficient way to find a password that matches a particular hash.
You can either force everyone to change their password once you have the configuration changed to use MD5 hashing, or you can generate new temporary passwords for everyone to get all the hashes replaced at once, but then they should all change them again anyway.
Those look like they probably came from Linux (the one that starts with $1$ is already an MD5 hash, the others look like the older crypt() DES variant style).
That's not always the case when dealing with application passwords - for example, some databases use considerably weaker credential storage - but Unix/Linux flavors have not stored actual passwords for a very long time, if ever.
let's think. Suppose I can hack my way to your computer / DB (not an impossible task). So I have what you have - a list of all users' passwords, but encrypted. But most probably I can register to your system myself, and I know my own password. I can find its encrypted form in the DB, either by my username, or by the latest entry, so now I have the clear + encrypted pair! If I could do what you are asking, I'd have all passwords and full access to all accounts. Meaning that encrypting passwords, or data in general has virtually no additional security effect, and all I should care about is preventing unauthorized access to the data. But this is not true, so you can't :)
BTW, MD5 is not an encryption but a hash, or more correctly a digest.
Looks like twalberg is right for the most part, but I'd like to go a step further to give you some practical advice:
1) Look at http://www.openwall.com/john/ (John the Ripper) for help getting the original passwords out of the database you have. Note that the passwords generated may not be the original passwords (it is likely, but not certain).
Once you've gotten all of the passwords using this tool, think to yourself: Anyone could do this. Your users might use these passwords somewhere else. If someone broke into your site, they could expose the people who trust you. Your users could blame you if someone used this information to break into their other accounts. Having the cleartext passwords around is risky for you.
I recommend you take this opportunity to have your users change their passwords. Those that no longer use their accounts should probably have to go through a small verification step to regain access, so you can disable their accounts now.
2) When you generate the new password database, I recommend you at a minimum use a secure hash (E.g. SHA1 or SHA2) along with a per-user salt (random value prepended to the generated hash output in a standard way) to help secure the database from accidental exposure.
While this answer isn't efficient, it is probably practical for the most part for your current database if your actual password database has even approximately the proportions you represent.
If you're switching away from your current system don't switch to plain MD5 - there are "rainbow tables" to crack a lot of common passwords; instead use salted/double MD5 or SHA
Thanks all for your input. I will just have everyone change their passwords. I have no intention of keeping anyone's password. The goal was to not have all 4000 users have to change their passwords after migrating to the new system but it seems unavoidable. As davka pointed out, I have an account in this database and know my password and can see the encrypted version so thought that would be sufficient to develop a script that decrypts and renters the password into the new database. Thanks again.
In a web application written in Perl and using PostgreSQL the users have username and password. What would be the recommended way to store the passwords?
Encrypting them using the crypt() function of Perl and a random salt? That would limit the useful length of passswords to 8 characters and will require fetching the stored password in order to compare to the one given by the user when authenticating (to fetch the salt that was attached to it).
Is there a built-in way in PostgreSQL to do this?
Should I use Digest::MD5?
Don't use SHA1 or SHA256, as most other people are suggesting. Definitely don't use MD5.
SHA1/256 and MD5 are both designed to create checksums of files and strings (and other datatypes, if necessary). Because of this, they're designed to be as fast as possible, so that the checksum is quick to generate.
This fast speed makes it much easier to bruteforce passwords, as a well-written program easily can generate thousands of hashes every second.
Instead, use a slow algorithm that is specifically designed for passwords. They're designed to take a little bit longer to generate, with the upside being that bruteforce attacks become much harder. Because of this, the passwords will be much more secure.
You won't experience any significant performance disadvantages if you're only looking at encrypting individual passwords one at a time, which is the normal implementation of storing and checking passwords. It's only in bulk where the real difference is.
I personally like bcrypt. There should be a Perl version of it available, as a quick Google search yielded several possible matches.
MD5 is commonly used, but SHA1/SHA256 is better. Still not the best, but better.
The problem with all of these general-purpose hashing algorithms is that they're optimized to be fast. When you're hashing your passwords for storage, though, fast is just what you don't want - if you can hash the password in a microsecond, then that means an attacker can try a million passwords every second if they get their hands on your password database.
But you want to slow an attacker down as much as possible, don't you? Wouldn't it be better to use an algorithm which takes a tenth of a second to hash the password instead? A tenth of a second is still fast enough that users won't generally notice, but an attacker who has a copy of your database will only be able to make 10 attempts per second - it will take them 100,000 times longer to find a working set of login credentials. Every hour that it would take them at a microsecond per attempt becomes 11 years at a tenth of a second per attempt.
So, how do you accomplish this? Some folks fake it by running several rounds of MD5/SHA digesting, but the bcrypt algorithm is designed specifically to address this issue. I don't fully understand the math behind it, but I'm told that it's based on the creation of Blowfish frames, which is inherently slow (unlike MD5 operations which can be heavily streamlined on properly-configured hardware), and it has a tunable "cost" parameter so that, as Moore's Law advances, all you need to do is adjust that "cost" to keep your password hashing just as slow in ten years as it is today.
I like bcrypt the best, with SHA2(256) a close second. I've never seen MD5 used for passwords but maybe some apps/libraries use that. Keep in mind that you should always use a salt as well. The salt itself should be completely unique for each user and, in my opinion, as long as possible. I would never, ever use just a hash against a string without a salt added to it. Mainly because I'm a bit paranoid and also so that it's a little more future-proof.
Having a delay before a user can try again and auto-lockouts (with auto-admin notifications) is a good idea as well.
The pgcrypto module in PostgreSQL has builtin suppotr for password hashing, that is pretty smart about storage, generation, multi-algorithm etc. See http://www.postgresql.org/docs/current/static/pgcrypto.html, the section on Password Hashing Functions. You can also see the pgcrypto section of http://www.hagander.net/talks/hidden%20gems%20of%20postgresql.pdf.
Use SHA1 or SHA256 hashing with salting. Thats the way to go for storing passwords.
If you don't use a password recovery mechanism (Not password reset) I think using a hashing mechanism is better than trying to encrypt the password. You can just check the hashes without any security risk. Even you don't know the password of the user.
I would suggest storing it as a salted md5 hash.
INSERT INTO user (password) VALUES (md5('some_salt'||'the_password'));
You could calculate the md5 hash in perl if you wish, it doesn't make much difference unless you are micro-optimizing.
You could also use sha1 as an alternative, but I'm unsure if Postgres has a native implementation of this.
I usually discourage the use of a dynamic random salt, as it is yet another field that must be stored in the database. Plus, if your tables were ever compromised, the salt becomes useless.
I always go with a one-time randomly generated salt and store this in the application source, or a config file.
Another benefit of using a md5 or sha1 hash for the password is you can define the password column as a fixed width CHAR(32) or CHAR(40) for md5 and sha1 respectively.