I have oracle database to move on to new postgresql server.
Some tables are having field sesitive and those are all encryted through DBMS_OBFUSCATION_TOOLKIT.DESENCRYPT/DESDECRYPT.
The problem is here. The size of postgresql's encrypted data size(bytea type) should be the same with oracle's.
I tried to get it done with aes(encrypt/decrypt) which takes almost three times larger than raw data.(oracle takes 16byte with des algorithm, postgresql takes 33byte with aes and the raw data is of 13byte.)
I tried the postgresql crypt also, but the manual doesn't metion the way of decrypting it back limiting 8byte of raw data size.
Now i really need encrypt method which takes as small encryted data size as possible and provides decrypt method also.
Is there a good way or the other options for me???
Thanks in advance.
Crypt and DES are old cyphers and should not be used
Plain old DES is an obsolete algorithm. You can't really usefully compare it to AES128; it's like complaining that a SHA256 hash is bigger than an MD5 hash - yep, it is, but only one of them might slow the attacker down for a while. DES was widely considered weak even in 1999 and should never be used in new applications. Do not use it.
I don't think it's a good idea to seek an encryption method that "provides the smallest data size possible" - because it's basically a waste of time to encrypt data using DES. Why not use ROT13 (caesar cypher)? The "encrypted" result is the same size as the input, pity the encryption can be broken by a 3-year-old.
crypt is of a similar vintage. The old UNIX crypt hashing algorithm is ... elderly ... and totally unsuitable for any new application. Hashes should be SHA256 at minimum, really.
Crypt is a one-way hash
As for not being able to figure out how to decrypt crypted data: crypt isn't an encryption algorithm, it's a cryptographic hash function or "one way hash". One way hashes are suitable for verifying that data is unmodified, comparing to a stored salted hash for password authentication, for use in challenge-response authentication, etc. You cannot decrypt crypted data.
Deal with the size
Use a decent cryptographic function and live with the size increase. bf or aes128 are about the weakest you can reasonably use.
Personally I prefer to do my encryption/decryption in the app, not in the DB. If it's done in the DB the keys can be revealed by pg_stat_statements, in the logs by log_statement or errors, etc. Better that the key never be in the same place as the stored data at all.
Most programming languages have good cryptographic routines you can use.
It's hard to offer any more advice as you haven't really explained what you're encrypting, why, what your requirements are, what the threat(s) are, etc.
Passwords?
If you're storing passwords, you're probably doing it wrong.
If possible, let someone else do the authentication:
OAuth or OpenID for Internet
SSPI, Kerberos/GSSAPI, Active Directory, LDAP bind, SASL, HTTP DIGEST, etc for intranet
If you really must do the auth yourself, add a salt to the passwords and hash the result. Store the hash and the salt. When you must compare passwords, salt the new plaintext from the user with the same salt you used for the stored hash, hash the new password+salt, and see if the hash is the same as what you stored. If it is, they gave the right password.
You almost certainly don't need to recover cleartext passwords. Implement a secure password reset instead. If you really, really must, use a decently secure algorithm like aes to encrypt them and think carefully about key storage and management. See other posts on SO about key storage/management with pgcrypto.
See also:
Secure method for storing/retrieving a PGP private key and passphrase?
Depending on how your postgresql was built, it may have DES support in the pgcrypto module. It depends on if Postgres was configured with OpenSSL support as it relies on OpenSSL to do DES (while with other more modern algorithms it implements them itself).
PGCrypto Algorithms
If openssl support was included and you specify DES as the algorithm to encrypt and decrypt, the data should be the same as you get from Oracle (although you may need to specify padding options).
As Craig says though, the DES algorithm is weak and one of the reasons it is weak is because the output ciphertext is so small.
Related
I am working on a project that has to store users passwords. With that password you can gain access to a user achievements and stuff so it's really important that you can not get the password even if you hacked into the database. My problem is which hashing function to choose in security and efficiency level.
Right now I am using sha256 with salt and pepper but I read that using a slow hashing function like Bcrypt with cost factor of 12 can be superior. And if it does, how much more security do I gain from using a pepper as well because it's really time consuming for hash function like Bcrypt.
My question is what hash function should I use base on the assumption that I do not expect to get hacked by a global hacking organization with supercomputers?
Plain crypto hash functions like sha2 or sha3 don't cut it anymore for password hashing, because it's too efficient to compute them, which means an attacker that controls many cpu cores can compute large tables relatively quickly. In practice the feasibility of this still depends on the length of passwords mostly (and the character set used in those password to some extent), but you should still not use these unless you really know why that will be acceptable.
Instead, you should be using a hash or key derivation function more suitable for password hashing. Pbkdf2, bcrypt, scrypt and Argon2 are all acceptable candidates, with somewhat different strengths and weaknesses. For a regular web app, it almost doesn't matter which of these you choose, and all will be more secure than plain hashes.
The difference will be very real if your database is compromised, properly hashed passwords will likely not be revealed, while at least some passwords with sha likely will, despite salt and pepper.
I know about storing passwords as salted hashes and I know it is even safe enough for Linux.
But even before I knew this, I was wondering if it is safe to store a password in an AES container encrypted with the password itself.
In case my question got incomprehensible, some pythonish pseudo code:
AES(data=password, key=password)
No, that is not as safe as using a Password Based Key Derivation Function. The most important issue with passwords are dictionary and brute force attacks - trying passwords, in other words. Now the outcome of AES(data=password, key=password) is always the same value (as the calculation does not contain any salt). This means that building a rainbow table is possible. Furthermore, AES is a very fast, so it is very easy for attackers to check many passwords.
So you are much better off using a PBKDF such as PBKDF2, bcrypt or scrypt, with a high iteration count and at least 64 bits of random salt.
I'm a little confused about password-safe-keeping.
Let's say I've got database with user-account table.
And this is the place where i keep passwords.
At this time i'm using salted sha1.
I read Blowfish based function are better then sha1 because they need more time to process request.
Is there any reason why not to use salted sha1 and just limit login attempt count to some reasonable number (for example 50times per hour) as a 'firewall' to bruteforce attacks?
person who is working with this database has no need to bruteforce anything because
he can change records by queries.
With blowfish based function, you surely mean the BCrypt hash function. As you already stated BCrypt is designed to be slow (need some computing time), that's the only advantage over other fast hash functions, but this is crucial.
With an off-the-shelf GPU, you are able to calculate about 3 Giga hash values per second, so you can brute-force a whole english dictionary with 5'000'000 words in less than 2 milliseconds. Even if SHA-1 is a safe hash function, that makes it inappropriate for hashing passwords.
BCrypt has a cost factor, which can be adapted to future, and therefore faster, hardware. The cost factor determines how many iterations of hashing are performed. Recently i wrote a tutorial about hashing passwords, i would invite you to have a look at it.
Your point about restricting login attempts makes sense, but the hashing should protect the passwords in case the attacker has access to the database (SQL-injection). Of course you can limit the login attempts, but that has nothing to do with hashing, you could even store the passwords plaintext in this scenario.
Storing passwords in Blowfish is more secure than SHA-1 because, as of now, there has been no reported method of obtaining the value of a Blowfish-encrypted string. SHA-1, on the other hand, does have reported methods of obtaining data from encrypted strings. You cannot trust SHA-1 to prevent someone from obtaining its data.
If you are open to suggestion, I don't see a need to work with two-way encryption at all as you are storing passwords. Hashing your users passwords with a salted SHA-256 method may be an option. Allowing your users to reset their own passwords via Email is generally considered a good policy, and it results in a data set that cannot be easily cracked.
If you do require two-way encryption for any reason, aside from Blowfish, AES-256 (Rijndael) or Twofish are also currently secure enough to handle sensitive data. Don't forget that you are free to use multiple algorithms to store encrypted data.
On the note of brute forcing, it has little to do with encrypted database storage. You are looking at a full security model when you refer to methods of attack. Using a deprecated algorithm and "making up for it" by implementing policies to prevent ease of attack is not considered a mature approach to security.
In Short
Use one way hashing for storing passwords, allow users to reset via email
Don't be afraid use multiple methods to store encrypted data
If you must use an encryption/decryption scheme, keep your keys safe and only use proven algorithms
Preventing brute force attacks is a good mindset, but it will only slow someone down or encourage them to search for other points of entry
Don't take this as gospel: when it comes to security everyone has different requirements, the more research you do the better your methods will become. If you don't completely encapsulate your sensitive data with a full-on security policy, you may get a nasty surprise down the track.
Source: Wikipedia, http://eprint.iacr.org/2005/010
Is there any reason why not to use salted sha1 and just limit login
attempt count to some reasonable number (for example 50times per hour)
as a 'firewall' to bruteforce attacks?
If you don't encrypt your passwords with any decent algorithm you are failing basic security precautions.
Why isn't 'just' blocking login attempts safe?
Well beside the fact you would need to block EVERY possible entrance, eg:
ssh
webservices (your webapp, phpmyadmin, openpanel, etcetera)
ftp
lots more
You would also need to trust every user that has access to the database and server, I wouldn't like people to read my password, but what I dislike even more, is you deciding for me, metaforically speaking :-)
Maybe someone else can shed light on the Blowfish vs SHA discussion, although I doubt that part is a stackworthy formatted question
In a web application written in Perl and using PostgreSQL the users have username and password. What would be the recommended way to store the passwords?
Encrypting them using the crypt() function of Perl and a random salt? That would limit the useful length of passswords to 8 characters and will require fetching the stored password in order to compare to the one given by the user when authenticating (to fetch the salt that was attached to it).
Is there a built-in way in PostgreSQL to do this?
Should I use Digest::MD5?
Don't use SHA1 or SHA256, as most other people are suggesting. Definitely don't use MD5.
SHA1/256 and MD5 are both designed to create checksums of files and strings (and other datatypes, if necessary). Because of this, they're designed to be as fast as possible, so that the checksum is quick to generate.
This fast speed makes it much easier to bruteforce passwords, as a well-written program easily can generate thousands of hashes every second.
Instead, use a slow algorithm that is specifically designed for passwords. They're designed to take a little bit longer to generate, with the upside being that bruteforce attacks become much harder. Because of this, the passwords will be much more secure.
You won't experience any significant performance disadvantages if you're only looking at encrypting individual passwords one at a time, which is the normal implementation of storing and checking passwords. It's only in bulk where the real difference is.
I personally like bcrypt. There should be a Perl version of it available, as a quick Google search yielded several possible matches.
MD5 is commonly used, but SHA1/SHA256 is better. Still not the best, but better.
The problem with all of these general-purpose hashing algorithms is that they're optimized to be fast. When you're hashing your passwords for storage, though, fast is just what you don't want - if you can hash the password in a microsecond, then that means an attacker can try a million passwords every second if they get their hands on your password database.
But you want to slow an attacker down as much as possible, don't you? Wouldn't it be better to use an algorithm which takes a tenth of a second to hash the password instead? A tenth of a second is still fast enough that users won't generally notice, but an attacker who has a copy of your database will only be able to make 10 attempts per second - it will take them 100,000 times longer to find a working set of login credentials. Every hour that it would take them at a microsecond per attempt becomes 11 years at a tenth of a second per attempt.
So, how do you accomplish this? Some folks fake it by running several rounds of MD5/SHA digesting, but the bcrypt algorithm is designed specifically to address this issue. I don't fully understand the math behind it, but I'm told that it's based on the creation of Blowfish frames, which is inherently slow (unlike MD5 operations which can be heavily streamlined on properly-configured hardware), and it has a tunable "cost" parameter so that, as Moore's Law advances, all you need to do is adjust that "cost" to keep your password hashing just as slow in ten years as it is today.
I like bcrypt the best, with SHA2(256) a close second. I've never seen MD5 used for passwords but maybe some apps/libraries use that. Keep in mind that you should always use a salt as well. The salt itself should be completely unique for each user and, in my opinion, as long as possible. I would never, ever use just a hash against a string without a salt added to it. Mainly because I'm a bit paranoid and also so that it's a little more future-proof.
Having a delay before a user can try again and auto-lockouts (with auto-admin notifications) is a good idea as well.
The pgcrypto module in PostgreSQL has builtin suppotr for password hashing, that is pretty smart about storage, generation, multi-algorithm etc. See http://www.postgresql.org/docs/current/static/pgcrypto.html, the section on Password Hashing Functions. You can also see the pgcrypto section of http://www.hagander.net/talks/hidden%20gems%20of%20postgresql.pdf.
Use SHA1 or SHA256 hashing with salting. Thats the way to go for storing passwords.
If you don't use a password recovery mechanism (Not password reset) I think using a hashing mechanism is better than trying to encrypt the password. You can just check the hashes without any security risk. Even you don't know the password of the user.
I would suggest storing it as a salted md5 hash.
INSERT INTO user (password) VALUES (md5('some_salt'||'the_password'));
You could calculate the md5 hash in perl if you wish, it doesn't make much difference unless you are micro-optimizing.
You could also use sha1 as an alternative, but I'm unsure if Postgres has a native implementation of this.
I usually discourage the use of a dynamic random salt, as it is yet another field that must be stored in the database. Plus, if your tables were ever compromised, the salt becomes useless.
I always go with a one-time randomly generated salt and store this in the application source, or a config file.
Another benefit of using a md5 or sha1 hash for the password is you can define the password column as a fixed width CHAR(32) or CHAR(40) for md5 and sha1 respectively.
For my iPhone application, Apple wants to know if my password encryption (md5) is greater then 64-bit symmetric or greater then 1024-bit symmetric. I have not been able to find it online, so I am wondering if anyone knows the answer. In addition, is this considered an appropriate encryption technology for passwords, or should I use something different?
Thanks for any help!
MD5 is a hashing function, thus by definition it is not reversible. This is not the case for encryption (either symmetric or asymmetric), which has to be reversible to be useful.
To be more precise, hashes are one-way functions, in that an infinite number of inputs can map to a single output, thus it is impossible to obtain the exact input, with certainty, that resulted in a given output.
However, it may be possible to find a different input that hashes to the same output. This is called a collision.
Generally, hashing passwords instead of storing the plain text (even encrypted) is a good idea. (Even better if using a salt) However, MD5 has known weaknesses (and large collections of rainbow tables that aid in finding collisions), thus it would be a good idea to switch to something like SHA-1 or one of the SHA-2 family of hashes.
However, to answer your original question, there is really is no way to compare MD5 or any hash against any type of encryption; they have no equivalents because it's like comparing apples and oranges.
md5 isn't really symmetric or asymmetric encryption because it isn't reversible either symmetrically or asymmetrically. It's a Message Digest (secure hash) algorithm.
It's not encryption, it's a digest. If you didn't salt it, it's not particularly secure, but they're asking you the wrong question.
What exactly are you doing with MD5 and passwords? There are standard ways of doing things here, and it's always better to use one, but without knowing what you want to do it's hard to point you at a relevant standard.
It is NOT encryption at all.
Apple asks the question about the use of MD5 for hashing passwords to see if it requires authorization for export from the Department of Commerce/Bureau of Industry and Security.
The answer for that purpose is that using MD5 for password protection is not controlled as strong encryption (like symmetric algorithms in excess of 64 bits) in accord with the Technical Note to 15 CFR part 774, Supplement 1, ECCN 5A002, paragraph a.1, which describes using encryption for password protection. However, it may still be controlled under ECCN 5A992.
http://www.bis.doc.gov/encryption/ccl5pt2.pdf
The other answers are not helpful in the context of why the question was asked.
Also, you may want to call the Department of Commerce/Bureau of Industry and Security at 202-482-0707 and ask about your specific application.
Hash function most of times is a way to compress your data. They are one-way hash functions, meaning that are difficult to reversed(having the hash function=digest of a message it is difficult to find the original message that is converted to the specific hash value). On the other hand, are very easy to implemented because there is no need of any type of key.
It is not a symmetric or asymmetric algorithm. These kind of algorithms are used to encrypt and not to hash data. Encryption is used for confidentiality reasons, to protect data from attackers where they try to read someone's.
Encryption or cipher algorithms need keys to perform their tasks in contrast to hashes where they do not need any kind of key. Hashes are not used for confidentiality reasons but for integrity reasons even if they do not have enough strength. MD5 is one type of a hash function where exists many others because MD5 is not strong enough
I think MD5 is used for better security.... if we tell about any encryption or decryption algorithm, they are just for converting any plain text into cipher text... but on the other hand MD5 provides an uniqueness on that plain text that would be sent by any source(Alice)...so we can say that for better security or for providing envelop on plain text MD5 should be used before using any encryption algothim(symmetric or asymmetric).
As the numerous other guys on here have mentioned, MD5 is not a symmetric or an asymmetric algorithm.
Instead it comes under a different branch in cryptography all together. It's one of the smallest hashing algorithms available in the .Net framework. At a mere 16bytes for its keysizes, which should be 128 bit. Something that you learn your bread and butter with.
So yes it is greater than 64bit which is only 8bytes in size.
The maximum key size the common symm' enc' algs use is 256bit (Rijndael Managed).
If you want to be looking at keysizes greater than that, then you can use the RC2 symm' enc' algs which supports variable key sizes. Something that you can experiment with?
If you want higher than 1024bit, then you need to be looking at Asymm' Enc' Algs like the RSACryptoServiceProvider class which supports key sizes going upto 16K in Bits I think?
If you want to use passwords, then you need to use Keyed Hashing Algs, like anything HMAC' something, they should be Keyed Hashing Algorithms or MacTripleDes. These all use secret keyes to encrypt the hash that is generated from the data you supply. The keys are created by using passwords and salt values via the RFC2898DerivesBytes class. <-- Don't forget that RC2, Rijndael, AES, DES and etc all can be set-up to use passwords to help derive the secret keys. In case you are thinking that the opening sentence of this paragraph is a little misleading. So i added this just to be sure in the event that hashing is not what you need altogether.
*REMEMBER THAT THERE ARE UNIQUE INHERITANCE HIERARCHIES IN .net's Cryptography NameSpace.
So MD5 is the base Abstract class all MD5 Derived classes are to derive from. .Net provides one such derived class that is called MD5CryptoServiceProvider class. Which is essentially a managed wrapper class that makes call to windows unmanaged Crypto-Libraries API. MD5 is known in MS official textbooks under the umbrella term as a Non-Keyed Hashing Algorithm. *
There are plenty of options available to you.
: ) Enjoy !