flask_bcrypt does not allow encoded (Byte) password with '\x00' in the middle - hash

I have some code that accepts a password in string format, and hash it with SHA3-512 before passing it to flask_bcrypt for hashing. However, by some coincidence, I found a test case that produces a hash that contains '\x00' in the middle of the hash result, as seen below:
password_str = 'tes15!tes15!tes15!tes15!tes15!tes15!tes15!tes15!tes15!tes15!tes15!tes15!.'
password_bytes = password_str.encode('utf-8')
hashed = hashlib.sha3_512(password_bytes).digest()
The result of the hash is:
b'u~"\x98\xac\xc8E2eV\xbb\x8e#}\x92R\xdc\xa2\xab\xab\xcb\x8d.~\x9f\x82a\xbf\xec]k\xdb\xc55\x1d\xa4\x00\xe8\x03\x94\xb0\x91\x14\xf0\x9ec\x9a\x9ay\xfeP\xe3\x07J\x00\xb5\xbd\xba\xcb(\xf5\xdb\xab\x1a'
As we can see, there is an '\x00' found in the middle of the hash, but not at the end. flask_bcrypt detects this a ValueError exception:
if b"\x00" in password:
raise ValueError("password may not contain NUL bytes")
I understand that '\x00' is a reserved character and possible attack vector due to hash-length extension attacks. However, I do not think that I have done anything outside the ordinary. Is there a best practice that sanitizes the hash prior to passing it to bcrypt or flask_bcrypt that I am missing, or is there another type of byte-encoding that should be used with passwords?
It would be weird to reject a normal password just because it's derived hash conflicts with the auth library used.. Any help or advice is greatly appreciated. Thank you!

BCrypt was designed to hash user passwords, this is not the same as e.g. SHA which is meant to hash binary input like a file. A user entered password actually never contains a \0 character, after all it is the user who desides about his own password, not a hacker trying to exploit a security vulnerability.
In other words, you should pass in the password as string directly, and avoid the UTF-8 encoding which converts it to a binary value.

Related

Are MD5 hashes always either capital or lowercase?

I'm passing an HMAC-MD5 encoded parameter into a form and the vendor is returning it as invalid. However, it matches what their hash generator gives me, with the exception of capitalization on the letters. What I did to get around this was use an lcase command. I'm wondering if this will cause me trouble later. Coldfusion generates the hashed string in capital letters, the vendor always seems to use lowercase; is it always one or the other or will they ever be mixed?
MD5 as every other hash function will produce binary output, in case of MD5 it is 16 bytes.
Because those bytes are difficult to handle, they are encoded to a string. In case of MD5 they are usually encoded to 32 lowercase hexadecimal digits, so every byte is represented by 2 characters.
Whether the target system accepts upper- or lowercase encodings or both is up to the system, it is unrelated to the hash function, both are different representations of a the same MD5 hash. So to answer your question, format the output as the target system requires it.
While RFC-1321 MD5 Message-Digest Algorithm doesn't discuss hexadecimal string encoding, the test suite does show results in lowercase.
The MD5 test suite (driver option "-x") should print the following results:
MD5 test suite:
MD5 ("") = d41d8cd98f00b204e9800998ecf8427e
MD5 ("a") = 0cc175b9c0f1b6a831c399e269772661
MD5 ("abc") = 900150983cd24fb0d6963f7d28e17f72
MD5 ("message digest") = f96b697d7cb7938d525a2f31aaf161d0
MD5 ("abcdefghijklmnopqrstuvwxyz") = c3fcd3d76192e4007dfb496cca67e13b
MD5 ("ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789") =
d174ab98d277d9f5a5611c2c9f419d9f
MD5 ("123456789012345678901234567890123456789012345678901234567890123456
78901234567890") = 57edf4a22be3c955ac49da2e2107b67a
Lowercase is simply the outcome of C/C++ printf() format specifier %02x, not a requirement: "should print", not "must print".
Ref: RFC-1321 Appendix A.5 Test suite
A hex string can contain anything in the 0-9 and a-f, A-F range, so you should anticipate both upper and lower-case versions.
If you're really stuck trying to interface between two highly opinionated systems, force upper or lower case depending on your requirements.

Dovecot password hashing

Can anyone tell me how the Dovecot administration tool (doveadm pw) hashes passwords when using SHA-512. $6$ indicates SHA-512, followed by a salt, then the hash. How exactly does Dovecot generate the salt? Does it use an own algorithm? As far as I can see it uses /dev/random or /dev/urandom, but how does it deal with non-ASCII characters?
Nevermind, found out in password-scheme.c.
It reads data from /dev/urandom and has an array with allowed characters (static const char salt_chars[] = "./0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz";).
The salt is generated by using one of the characters from that array; precisely, it takes the byte from /dev/urandom modulo the length of salt_chars - 1 and uses that as index to pick a char from salt_chars.

Play! hash password returns bad result

I'm using Play 1.2.1. I want to hash my users password. I thought that Crypto.passwordHash will be good, but it isn't. passwordHash documentation says it returns MD5 password hash. I created some user accounts in fixture, where I put md5 password hash:
...
User(admin):
login: admin
password: f1682b54de57d202ba947a0af26399fd
fullName: Administrator
...
The problem is, when I try to log in, with something like this:
user.password.equals(Crypto.passwordHash(password))
and it doesn't work. So I put a log statement in my autentify method:
Logger.info("\nUser hashed password is %s " +
"\nPassed password is %s " +
"\nHashed passed password is %s",
user.password, password, Crypto.passwordHash(password));
And the password hashes are indeed different, but hey! The output of passwordHash method isn't even an MD5 hash:
15:02:16,164 INFO ~
User hashed password is f1682b54de57d202ba947a0af26399fd
Passed password is <you don't have to know this :P>
Hashed passed password is 8WgrVN5X0gK6lHoK8mOZ/Q==
How about that? How to fix it? Or maybe I have to implement my own solution?
Crypto.passwordHash returns base64-encoded password hash, while you are comparing to hex-encoded.
MD5 outputs a sequence of 16 bytes, each byte having (potentially) any value between 0 and 255 (inclusive). When you want to print the value, you need to convert the bytes to a sequence of "printable characters". There are several possible conventions, the two main being hexadecimal and Base64.
In hexadecimal notation, each byte value is represented as two "hexadecimal digits": such a digit is either a decimal digit ('0' to '9') or a letter (from 'a' to 'f', case is irrelevant). The 16 bytes thus become 32 characters.
In Base64 encoding, each group of three successive bytes is encoded as four characters, taken in a list of 64 possible characters (digits, lowercase letters, uppercase letters, '+' and '/'). One or two final '=' signs may be added so that the encoded string consists in a number of characters which is multiple of 4.
Here, '8WgrVN5X0gK6lHoK8mOZ/Q==' is the Base64 encoding of a sequence of 16 bytes, the first one having value 241, the second one 104, then 43, and so on. In hexadecimal notation, the first byte would be represented by 'f1', the second by '68', the third by '2b'... and the hexadecimal notation of the complete sequence of 16 bytes is then 'f1682b54de57d202ba947a0af26399fd', the value that you expected.
The play.libs.Codec class contains methods for decoding and encoding Base64 and hexadecimal notations. It also contains Codec.hexMD5() which performs MD5 hashing and returns the value in hexadecimal notation instead of Base64.
as Nickolay said you are comparing Hex vs Base-64 strings. Also, I would recommend using BCrypt for that, not the Crypto tool of Play.

Verifying salted hashes with Perls unpack()

I'm trying to verify salted passwords with Perl and am stuck with unpack.
I've got a salted hashed password, e.g. for SHA256: SSHA256 = SHA256('password' + 'salt') + 'salt'
Base64 encoded that gets '
{SSHA256}eje4XIkY6sGakInA+loqtNzj+QUo3N7sEIsj3fNge5lzYWx0'.
I store this string in my user database. When a user logs in need to separate the salt from the hash to hash the supplied password with the salt and compare the result to the one retrieved from the db. This is where I'm stuck. I don't seem to have the right unpack template separate the hash (8-bit binary, fixed length, in this case 32 byte) from the salt (8-bit binary, variable length).
I have tried something like
my ($hash, $salt) = unpack('N32 N*', $data);
but that doesn't seem to work out.
My question is: How can I unpack this hash (after it has been Base64 decoded) to get the fixed length hash in one and the variable length salt in another variable?
I think you're needlessly re-inventing the wheel.
You could use e.g. Crypt::SaltedHash to easily verify it, for instance:
my $password_entered = $cgi->param('password');
my $valid = Crypt::SaltedHash->validate($salted, $password_entered);
A longer example, showing using Crypt::SaltedHash to generate the salted password in the first instance, too:
my $csh = Crypt::SaltedHash->new(algorithm => 'SHA-256');
$csh->add('secretpassword');
my $salted = $csh->generate;
# $salted will contain the salted hash (Crypt::SaltedHash picks random
# salt for you automatically)
# for example:
DB x $salted = $csh->generate;
0 '{SSHA256}H1WaxHcyAB81iyIPwib/cCUtjqCm2sxQNA1QvGeh/iT3m51w'
# validating that against the plaintext 'secretpassword' shows it's right:
DB x Crypt::SaltedHash->validate($salted, 'secretpassword');
0 1
# and trying it with an incorrect password:
DB x Crypt::SaltedHash->validate($salted, 'wrongpassword');
0 ''
No reason to re-invent all of this yourself.
You seem to be doing RFC2307 the hard way and also manage to introduce bugs. Those + do not mean what you think.
Subclass Authen::Passphrase::SaltedDigest instead.
Not sure the whole picture is present, but the unpack template you have specified -'N32 N*'- is for 32 unsigned long (32-bit) (big-endian) integers (see pack docs).
Looks like you may instead need unsigned chars: '32C C*'

How should I generate a random alphanumeric initial password for new users?

We have to automatically import a large list of users with some data into a running system.
For an initial password I want to update the list (csv format at the moment) with a random alphanumeric key (8 digits).
When inserting it with a special routine (which needs a csv file), the password (in this case the alphanumeric key) is stored as a md5 hash.
i.e. I generate a random alphanumeric key:
H2A5D39A -> MD5: 1642fccf791f15d137cf31282af79752
This way I want to create a list where authenticated users can ask me for their initial password (the alphanumeric key).
Do you have a better idea for a "secret" initial password?
How would you create the alphanumeric key in Perl?
P.S.: The "running system", not programmed by us, just allowes alphanumeric passwords (no special chars,...)
How would you create the alphanumeric key in Perl?
join'', map +(0..9,'a'..'z','A'..'Z')[rand(10+26*2)], 1..8
I would probably use pwgen. It is great as it allows easy customization, and has the switch not to use ambiguous characters (think: I, l, 1, O, 0).
for example:
=> pwgen -c -n -B 8 50
shuFak9o peiCh3Oo ohPieng9 Vohh7zuu os3Theep aeV9nuo9 aexeik4B aeChoh9s
uth3eePu baePhu3o aiS3pahn iPie4itu We9zuphi xie3Chi3 yeiRoo7c fai3ITai
aCh9ohco Echuab7v Fu9ahCho Aevae4no Peethai9 AiJio3Pa aeNge9Fo baePh7Uy
Nai7shei eeMoh9en Zeibai4n eGe7yuch Jaek7nai aeZah7sh Chei4ua4 shoo9oG9
iu7Wohho aep7De4U Fong9fo3 AhneeP7U oxae7Yoh ahF4eim3 fahm9Aiw naoNg4ie
Chie4xua jix3Uvot aChei7ai diey4Shi Yur7ee4j eeJeo9ee Bou3ahmu kaeb4Cah
Eh4Eemae oD4phoo9
Anonymous's answer is very good, but, if you need a random string that conforms to some rules (such as at least one uppercase, one lowercase, and one number), you may want to look into String::Random.
Another module to consider is Data::Random
I just completed a review of the 12 modules on CPAN that can be used to generate random passwords:
http://blogs.perl.org/users/neilb/2011/08/random-password-generation.html
In short: if you want a pronounceable password, look at Crypt::YAPassGen,
otherwise go for App::Genpass.
App::Genpass avoids confusable characters and gives you better control (and defaults) than Data::Random or String::Random