This question is unlikely to help any future visitors; it is only relevant to a small geographic area, a specific moment in time, or an extraordinarily narrow situation that is not generally applicable to the worldwide audience of the internet. For help making this question more broadly applicable, visit the help center.
Closed 9 years ago.
i had googled a lot for decryption but it was useless can anyone please tell me which type of encryption is this and how to decrypt
0x010056049b0e92e4e85487c8a63385cdb89bdd66cb7f28cab34e
for more info:
password hash: 0x010056049b0e92e4e85487c8a63385cdb89bdd66cb7f28cab34e
header: 0x0100
salt: 56049b0e
mixedcase: 92e4e85487c8a63385cdb89bdd66cb7f28cab34e
can anyone teach me how to survive this i m totally new to this
If the password owner doesn't work for your company then you may be running afoul of the law if you continue down this path. Be very careful to ensure that you know if you are violating the law and make an appropriate decision for whether or not to do so. I do not advocate breaking the law, and provide the publicly available information, below, for your general educational purposes.
Not wanting to change the password but wanting to learn it is generally very suspicious. If you have business accessing the data that the password protects, then you probably have access to change the password, or you probably already know the password. Wanting to keep the password unchanged, but learn what it is and gain access to the underlying assets begs the question: why leave it unchanged? One obvious answer is to prevent the rightful owner from knowing that you have access. Another obvious answer is because you forgot the password and are trying to regain access to your own data - though it is very unusual to have access to the hash but not the ability to just change the password if one is truly the rightful owner.
As commenters have mentioned, you cannot decrypt a hash. Hashing and encryption/decryption are two separate operations. Encryption and decryption are opposites, while hashing has no opposite function.
For simplicity's sake, consider encryption to be like adding 1 to each character, so "a" in the original text becomes "b", "b" in the original text becomes "c", and so on. Decryption would be subtracting 1 from the number, so "b" in the cipher text becomes "a". In reality, much harder math problems are used, but this is the general idea.
Hashing doesn't have an inverse. Consider hashing like finding the mod 2 result. For example, "a" could be represented as the value "97", and 97 mod 2 = 1. "b" could be represented as the value "98" and 98 mod 2 = 0. Shown as a table below...
letter numeric mod 2
representation (the 'hash')
a 97 1
b 98 0
c 99 1
d 100 0
e 101 1
As you can see from the table, there's no way to get back to the original letter knowing only the hash value. In reality, hashes are much more complex, based on much harder math, and often are designed to prevent collisions - avoiding the scenario in my description where mod2 every "odd" letter results in a "1" and every "even" letter results in a "0".
Nonetheless, it should be clear that you cannot reverse a hash like you can reverse (decrypt) encryption.
That said, there sometimes ways to find out the password. You might "brute force" every possible input until you get the expected result. That is, try your salt with the password "a", "aa", "aaa", "aaa", ..., "b", "bb", ..., "ab", ... With a good hash algorithm, this is the fastest way to figure out a particular hash. However, with a good hash algorithm and a sufficiently long password, you won't be able to learn the password this way before the sun dies out.
Sometimes hash algorithms will have weaknesses. Sometimes, there will be a method that is faster than brute forcing. You'd need to look up the particular algorithm in use and learn its weaknesses, but even then this doesn't mean it will be possible to break in reasonable time. I'll leave this as an exercise for the reader, and will not be responding to any requests for pointers :)
Finally, often easier than attempting to reverse a hash is to learn the password another way. Realistically, this is exactly what a strong hash algorithm is intended to force you to do - it's supposed to be so hard to learn the original password that you have to resort to another technique. For example, if the person who uses the password works for your company, the CEO can ask them for their password. More often, the real goal is to gain control over a valid account, so you can just change the current password.
That's about as much of an answer as I can really give, given the vagueness of your question. If you are interested in more, talk to your lawyer and start reading Wikipedia articles. They do a very good job of explaining encryption and hashing and related algorithms.
Related
Is it possible to reverse a SHA-1?
I'm thinking about using a SHA-1 to create a simple lightweight system to authenticate a small embedded system that communicates over an unencrypted connection.
Let's say that I create a sha1 like this with input from a "secret key" and spice it with a timestamp so that the SHA-1 will change all the time.
sha1("My Secret Key"+"a timestamp")
Then I include this SHA-1 in the communication and the server, which can do the same calculation. And hopefully, nobody would be able to figure out the "secret key".
But is this really true?
If you know that this is how I did it, you would know that I did put a timestamp in there and you would see the SHA-1.
Can you then use those two and figure out the "secret key"?
secret_key = bruteforce_sha1(sha1, timestamp)
Note1:
I guess you could brute force in some way, but how much work would that actually be?
Note2:
I don't plan to encrypt any data, I just would like to know who sent it.
No, you cannot reverse SHA-1, that is exactly why it is called a Secure Hash Algorithm.
What you should definitely be doing though, is include the message that is being transmitted into the hash calculation. Otherwise a man-in-the-middle could intercept the message, and use the signature (which only contains the sender's key and the timestamp) to attach it to a fake message (where it would still be valid).
And you should probably be using SHA-256 for new systems now.
sha("My Secret Key"+"a timestamp" + the whole message to be signed)
You also need to additionally transmit the timestamp in the clear, because otherwise you have no way to verify the digest (other than trying a lot of plausible timestamps).
If a brute force attack is feasible depends on the length of your secret key.
The security of your whole system would rely on this shared secret (because both sender and receiver need to know, but no one else). An attacker would try to go after the key (either but brute-force guessing or by trying to get it from your device) rather than trying to break SHA-1.
SHA-1 is a hash function that was designed to make it impractically difficult to reverse the operation. Such hash functions are often called one-way functions or cryptographic hash functions for this reason.
However, SHA-1's collision resistance was theoretically broken in 2005. This allows finding two different input that has the same hash value faster than the generic birthday attack that has 280 cost with 50% probability. In 2017, the collision attack become practicable as known as shattered.
As of 2015, NIST dropped SHA-1 for signatures. You should consider using something stronger like SHA-256 for new applications.
Jon Callas on SHA-1:
It's time to walk, but not run, to the fire exits. You don't see smoke, but the fire alarms have gone off.
The question is actually how to authenticate over an insecure session.
The standard why to do this is to use a message digest, e.g. HMAC.
You send the message plaintext as well as an accompanying hash of that message where your secret has been mixed in.
So instead of your:
sha1("My Secret Key"+"a timestamp")
You have:
msg,hmac("My Secret Key",sha(msg+msg_sequence_id))
The message sequence id is a simple counter to keep track by both parties to the number of messages they have exchanged in this 'session' - this prevents an attacker from simply replaying previous-seen messages.
This the industry standard and secure way of authenticating messages, whether they are encrypted or not.
(this is why you can't brute the hash:)
A hash is a one-way function, meaning that many inputs all produce the same output.
As you know the secret, and you can make a sensible guess as to the range of the timestamp, then you could iterate over all those timestamps, compute the hash and compare it.
Of course two or more timestamps within the range you examine might 'collide' i.e. although the timestamps are different, they generate the same hash.
So there is, fundamentally, no way to reverse the hash with any certainty.
In mathematical terms, only bijective functions have an inverse function. But hash functions are not injective as there are multiple input values that result in the same output value (collision).
So, no, hash functions can not be reversed. But you can look for such collisions.
Edit
As you want to authenticate the communication between your systems, I would suggest to use HMAC. This construct to calculate message authenticate codes can use different hash functions. You can use SHA-1, SHA-256 or whatever hash function you want.
And to authenticate the response to a specific request, I would send a nonce along with the request that needs to be used as salt to authenticate the response.
It is not entirely true that you cannot reverse SHA-1 encrypted string.
You cannot directly reverse one, but it can be done with rainbow tables.
Wikipedia:
A rainbow table is a precomputed table for reversing cryptographic hash functions, usually for cracking password hashes. Tables are usually used in recovering a plaintext password up to a certain length consisting of a limited set of characters.
Essentially, SHA-1 is only as safe as the strength of the password used. If users have long passwords with obscure combinations of characters, it is very unlikely that existing rainbow tables will have a key for the encrypted string.
You can test your encrypted SHA-1 strings here:
http://sha1.gromweb.com/
There are other rainbow tables on the internet that you can use so Google reverse SHA1.
Note that the best attacks against MD5 and SHA-1 have been about finding any two arbitrary messages m1 and m2 where h(m1) = h(m2) or finding m2 such that h(m1) = h(m2) and m1 != m2. Finding m1, given h(m1) is still computationally infeasible.
Also, you are using a MAC (message authentication code), so an attacker can't forget a message without knowing secret with one caveat - the general MAC construction that you used is susceptible to length extension attack - an attacker can in some circumstances forge a message m2|m3, h(secret, m2|m3) given m2, h(secret, m2). This is not an issue with just timestamp but it is an issue when you compute MAC over messages of arbitrary length. You could append the secret to timestamp instead of pre-pending but in general you are better off using HMAC with SHA1 digest (HMAC is just construction and can use MD5 or SHA as digest algorithms).
Finally, you are signing just the timestamp and the not the full request. An active attacker can easily attack the system especially if you have no replay protection (although even with replay protection, this flaw exists). For example, I can capture timestamp, HMAC(timestamp with secret) from one message and then use it in my own message and the server will accept it.
Best to send message, HMAC(message) with sufficiently long secret. The server can be assured of the integrity of the message and authenticity of the client.
You can depending on your threat scenario either add replay protection or note that it is not necessary since a message when replayed in entirety does not cause any problems.
Hashes are dependent on the input, and for the same input will give the same output.
So, in addition to the other answers, please keep the following in mind:
If you start the hash with the password, it is possible to pre-compute rainbow tables, and quickly add plausible timestamp values, which is much harder if you start with the timestamp.
So, rather than use
sha1("My Secret Key"+"a timestamp")
go for
sha1("a timestamp"+"My Secret Key")
I believe the accepted answer is technically right but wrong as it applies to the use case: to create & transmit tamper evident data over public/non-trusted mediums.
Because although it is technically highly-difficult to brute-force or reverse a SHA hash, when you are sending plain text "data & a hash of the data + secret" over the internet, as noted above, it is possible to intelligently get the secret after capturing enough samples of your data. Think about it - your data may be changing, but the secret key remains the same. So every time you send a new data blob out, it's a new sample to run basic cracking algorithms on. With 2 or more samples that contain different data & a hash of the data+secret, you can verify that the secret you determine is correct and not a false positive.
This scenario is similar to how Wifi crackers can crack wifi passwords after they capture enough data packets. After you gather enough data it's trivial to generate the secret key, even though you aren't technically reversing SHA1 or even SHA256. The ONLY way to ensure that your data has not been tampered with, or to verify who you are talking to on the other end, is to encrypt the entire data blob using GPG or the like (public & private keys). Hashing is, by nature, ALWAYS insecure when the data you are hashing is visible.
Practically speaking it really depends on the application and purpose of why you are hashing in the first place. If the level of security required is trivial or say you are inside of a 100% completely trusted network, then perhaps hashing would be a viable option. Hope no one on the network, or any intruder, is interested in your data. Otherwise, as far as I can determine at this time, the only other reliably viable option is key-based encryption. You can either encrypt the entire data blob or just sign it.
Note: This was one of the ways the British were able to crack the Enigma code during WW2, leading to favor the Allies.
Any thoughts on this?
SHA1 was designed to prevent recovery of the original text from the hash. However, SHA1 databases exists, that allow to lookup the common passwords by their SHA hash.
Is it possible to reverse a SHA-1?
SHA-1 was meant to be a collision-resistant hash, whose purpose is to make it hard to find distinct messages that have the same hash. It is also designed to have preimage-resistant, that is it should be hard to find a message having a prescribed hash, and second-preimage-resistant, so that it is hard to find a second message having the same hash as a prescribed message.
SHA-1's collision resistance is broken practically in 2017 by Google's team and NIST already removed the SHA-1 for signature purposes in 2015.
SHA-1 pre-image resistance, on the other hand, still exists. One should be careful about the pre-image resistance, if the input space is short, then finding the pre-image is easy. So, your secret should be at least 128-bit.
SHA-1("My Secret Key"+"a timestamp")
This is the pre-fix secret construction has an attack case known as the length extension attack on the Merkle-Damgard based hash function like SHA-1. Applied to the Flicker. One should not use this with SHA-1 or SHA-2. One can use
HMAC-SHA-256 (HMAC doesn't require the collision resistance of the hash function therefore SHA-1 and MD5 are still fine for HMAC, however, forgot about them) to achieve a better security system. HMAC has a cost of double call of the hash function. That is a weakness for time demanded systems. A note; HMAC is a beast in cryptography.
KMAC is the pre-fix secret construction from SHA-3, since SHA-3 has resistance to length extension attack, this is secure.
Use BLAKE2 with pre-fix construction and this is also secure since it has also resistance to length extension attacks. BLAKE is a really fast hash function, and now it has a parallel version BLAKE3, too (need some time for security analysis). Wireguard uses BLAKE2 as MAC.
Then I include this SHA-1 in the communication and the server, which can do the same calculation. And hopefully, nobody would be able to figure out the "secret key".
But is this really true?
If you know that this is how I did it, you would know that I did put a timestamp in there and you would see the SHA-1. Can you then use those two and figure out the "secret key"?
secret_key = bruteforce_sha1(sha1, timestamp)
You did not define the size of your secret. If your attacker knows the timestamp, then they try to look for it by searching. If we consider the collective power of the Bitcoin miners, as of 2022, they reach around ~293 double SHA-256 in a year. Therefore, you must adjust your security according to your risk. As of 2022, NIST's minimum security is 112-bit. One should consider the above 128-bit for the secret size.
Note1: I guess you could brute force in some way, but how much work would that actually be?
Given the answer above. As a special case, against the possible implementation of Grover's algorithm ( a Quantum algorithm for finding pre-images), one should use hash functions larger than 256 output size.
Note2: I don't plan to encrypt any data, I just would like to know who sent it.
This is not the way. Your construction can only work if the secret is mutually shared like a DHKE. That is the secret only known to party the sender and you. Instead of managing this, a better way is to use digital signatures to solve this issue. Besides, one will get non-repudiation, too.
Any hashing algorithm is reversible, if applied to strings of max length L. The only matter is the value of L. To assess it exactly, you could run the state of art dehashing utility, hashcat. It is optimized to get best performance of your hardware.
That's why you need long passwords, like 12 characters. Here they say for length 8 the password is dehashed (using brute force) in 24 hours (1 GPU involved). For each extra character multiply it by alphabet length (say 50). So for 9 characters you have 50 days, for 10 you have 6 years, and so on. It's definitely inaccurate, but can give us an idea, what the numbers could be.
I am generating random OTP-style strings that serve as a short-term identifier to link two otherwise unrelated systems (which have authentication at each end). These need to be read and re-entered by users, so in order to reduce the error rate and reduce the opportunities for forgery, I'd like to make one of the digits a check digit. At present my random string conforms to the pattern (removing I and O to avoid confusion):
^[ABCDEFGHJKLMNPQRSTUVWXYZ][0-9]{4}$
I want to append one extra decimal digit for the check. So far I've implemented this as a BLAKE2 hash (from libsodium) that's converted to decimal and truncated to 1 char. This gives only 10 possibilities for the check digit, which isn't much. My primary objective is to detect single character errors in the input.
This approach kind of works, but it seems that one digit is not enough to detect single char errors, and undetected errors are quite easy to find, for example K37705 and K36705 are both considered valid.
I do not have a time value baked into this OTP; instead it's purely random and I'm relying on keeping a record of the OTPs that have been generated recently for each user, which are deleted periodically, and I'm reducing opportunities for brute-forcing by rate and attempt-count limiting.
I'm guessing that BLAKE2 isn't a good choice here, but given there are only 10 possibilities for the result, I don't know that others will be better. What would be a better algorithm/approach to use?
Frame challenge
Why do you need a check digit?
It doesn't improve security, and a five digits is trivial for most humans to get correct. Check if server side and return an error message if it's wrong.
Normal TOTP tokens are commonly 6 digits, and actors such as google has determined that people in general manage to get them orrect.
I'm trying to store user passwords in my DB using Argon2 algorithm.
This is what I obtain by using it:
$echo -n "password" | argon2 "smallsalt" -id -t 4 -m 18 -p 4
Type: Argon2id
Iterations: 4
Memory: 262144 KiB
Parallelism: 4
Hash: cb4447d91dd62b085a555e13ebcc6f04f4c666388606b2c401ddf803055f54ac
Encoded: $argon2id$v=19$m=262144,t=4,p=4$c21hbGxzYWx0$y0RH2R3WKwhaVV4T68xvBPTGZjiGBrLEAd34AwVfVKw
1.486 seconds
Verification ok
In this case, what should I store in the DB?
The "encoded" value as shown above?
The "hash" value as shown above?
Neither, but another solution?
Please, could you help me? I'm a newbie with this and I'm a little bit lost.
I'm a bit late to the party, but I disagree with the previous answers.
You should store the field: Encoded
The $argon2id$.... value.
(At least if you are using normal Argon2 libraries having the verify() function.
It does not look like the man-page for argon2 command does this, however.
Only if you are stuck with the command line, you should consider storing each field individually.)
The $argon2id$ encoded hash
The argon2 encoded hash follows the same as its older cousin bcrypt's syntax.
The encoded hash includes all you ever need to verify the hash when the user logs in.
It is most likely more future proof. When a newer and better argon2 comes along: You can upgrade your one column hashed passwords. Just like you could detect bcrypt's $2a$-hashes, and re-hash them as $argon2id$-hashes, next time the user logs in. (If you were moving from bcrypt to agron2.)
TL;DR
Store the $-encoded value encoded_hash in your database.
Use argon2.verify(password, encoded_hash) to verify that the password is correct.
Don't bother about all the values inside the hash. Let the library do that for you. :)
Neither. Save following as a single value:
algorithm ID (e.g. argon2id)
salt
number of iterations (4)
memory usage factor (18)
parallelism (4)
The output of the field "encoded" is misleading because you cannot use it as is for password check (i.e. for hash generation), e.g. m=262144 where as for password check you need the original factor m=18.
Are you going to launch an OS process each time you check password? I would discourage you from doing this. I'd suggest you use a library (C++, Java, ...). They produce a string that contains all these data concatenated and separated with "$".
I'd put the type, iterations, memory, parallelism, hash, salt and corresponding user id into separate columns and leave the encoded bit out, because it's just all the attributes joined together. If they're in separate columns then you can reference the attributes more easily than having to split and index the encoded string.
The other option is to just store the encoded string in 1 column, but as I said its more tedious to look at certain attributes, as you'd have to split the encoded string and then index it.
I had the same question and read this post while gathering some information. Now after some days and thoughts about all this, I'll personally take a different route than the accepted answer and therefore slightly disagree with it. I thought I would share my perspective so that it might help others as well.
I suppose it will depend on everyone's context. I don't think there is a one size fits all answer here. I'm sure there are situations where it is perfectly valid and even better/simpler to store the encoded string ($argon2...).
However, I would argue that depending on the context, storing the encoded string doesn't seem to be the right approach.
First of all, it makes the hashing method very obvious. It is probably not that important but for some reasons it makes me a bit more comfortable not having it ^^. But, more importantly, it means that implementation details are stored in your persistence layer (db or else). At the time of writing, argon2id is the recommended hashing mechanism by OWASP but these things can change (eventually do change...). Some day, it might be considered unsecure, or another function will be considered more secure.
As a result, I would suggest this more function "agnostic" starting point:
The hash (for argon2 -> the hex string)
The salt
The last_modified date
A string with hashing parameters (for argon2, you could put the parameters here in the form of your choosing)
The last_modified allows to know if the hash needs updating or not and the parameters allows to support the verification and update of "old" hashes.
Of course this means that you have to work a bit more in the code and can't simply use every libraries shortcuts straight away. But, I would say that this increased complexity offer more flexibility in other circumstances (like moving away from a given hashing function). As always there are no free lunch.
That's why I suppose it depends on your context and why personally I wouldn't go with the accepted answer in my situation.
PS: I'm no cryptography expert nor some devsecop guru. So feel free to contradict, enrich, agree or disagree. I just like to keep my implementation details contained ;)
I'm currently experimenting with both public-key and personal file encryption. The programs I use have 2048 bit RSA and 256 bit AES level encryption respectively. As a newbie to this stuff (I've only been a cypherpunk for about a month now - and am a little new to information systems) I'm not familiar with RSA algorithms, but that's not relevant here.
I know that unless some secret lab or NSA program happens to have a quantum computer, it is currently impossible to brute force hack the level of security these programs provide, but I was wondering how much more secure it would be to encrypt a file over and over again.
In a nutshell, what I would like to know is this:
When I encrypt a file using 256-bit AES, and then encrypt the already encrypted file once more (using 256 again), do I now have the equivalent of 512-bit AES security? This is pretty much a question of whether or not the the number of possible keys a brute force method would potentially have to test would be 2 x 2 to the 256th power or 2 to the 256th power squared. Being pessimistic, I think it is the former but I was wondering if 512-AES really is achievable by simply encrypting with 256-AES twice?
Once a file is encrypted several times so that you must keep using different keys or keep putting in passwords at each level of encryption, would someone** even recognize if they have gotten through the first level of encryption? I was thinking that perhaps - if one were to encrypt a file several times requiring several different passwords - a cracker would not have any way of knowing if they have even broken through the first level of encryption since all they would have would still be an encrypted file.
Here's an example:
Decrypted file
DKE$jptid UiWe
oxfialehv u%uk
Pretend for a moment that the last sequence is what a cracker had to work with - to brute-force their way back to the original file, the result they would have to get (prior to cracking through the next level of encryption) would still appear to be a totally useless file (the second line) once they break through the first level of encryption. Does this mean that anyone attempting to use brute-force would have no way of getting back to the original file since they presumably would still see nothing but encrypted files?
These are basically two questions that deal with the same thing: the effect of encrypting the same file over and over again. I have searched the web to find out what effect repeated encryption has on making a file secure, but aside from reading an anecdote somewhere that the answer to the first question is no, I have found nothing that pertains to the second spin on the same topic. I am especially curious about that last question.
**Assuming hypothetically that they somehow brute-forced their way through weak passwords - since this appears to be a technological possibility with 256-AES right now if you know how to make secure ones...
In general, if you encrypt a file with k-bit AES then again with k-bit AES, you only get (k+1) bits of security, rather than 2k bits of security, with a man-in-the-middle attack. The same holds for most types of encryption, like DES. (Note that triple-DES is not simply three rounds of encryption for this reason.)
Further, encrypting a file with method A and then with method B need not be even as strong as encrypting with method B alone! (This would rarely be the case unless method A is seriously flawed, though.) In contrast, you are guaranteed to be at least as strong as method A. (Anyone remembering the name of this theorem is encouraged to leave a comment; I've forgotten.)
Usually you're much better off simply choosing a single method as strong as possible.
For your second question: Yes, with most methods, an attacker would know that the first layer had been compromised.
More an opinion here...
First, when computer are strong enough to do a brute-force attack on AES-256 for example, it will be also for iterations of the same... doubling or tripling the time or effort is insignificant at that level.
Next, such considerations can be void depending on the application you are trying to use this encryption in... The "secrets" you will need to carry become bigger (number of iterations and all the different keys you will need, if in fact they are different), the time to do the encryption and the decryption will also need to increase.
My hunch is that iterating the encryption does not help much. Either the algorithm is strong enough to sustain a brute-force attach or it is not. The rest is all in the protection of the keys.
More practically, do you think your house is more protected if you have three identical or similar locks on your front door ? (and that includes number of keys for you to carry around, don't loose those keys, make sure windows and back door are secured also...)
Question 1:
The size of the solution space is going to be the same for two passes of the 256-bit key as the 512-bit key, since 2^(256+256) = 2^512
The actual running time of each decrypt() may increase non-linearly as the key-size grows (it would depend on the algorithm), in this case I think brute forcing the 256+256 would run faster than the 2^512, but would still be infeasible.
Question 2:
There are probably ways to identify certain ciphertext. I wouldn't be surprised if many algorithms leave some signature or artifacts that could be used for identification.
We have a system where we want to prevent the same credit card number being registered for two different accounts. As we don't store the credit card number internally - just the last four digits and expiration date - we cannot simply compare credit card numbers and expiration dates.
Our current idea is to store a hash (SHA-1) in our system of the credit card information when the card is registered, and to compare hashes to determine if a card has been used before.
Usually, a salt is used to avoid dictionary attacks. I assume we are vulnerable in this case, so we should probably store a salt along with the hash.
Do you guys see any flaws in this method? Is this a standard way of solving this problem?
Let's do a little math: Credit card numbers are 16 digits long. The first seven digits are 'major industry' and issuer numbers, and the last digit is the luhn checksum. That leaves 8 digits 'free', for a total of 100,000,000 account numbers, multiplied by the number of potential issuer numbers (which is not likely to be very high). There are implementations that can do millions of hashes per second on everyday hardware, so no matter what salting you do, this is not going to be a big deal to brute force.
By sheer coincidence, when looking for something giving hash algorithm benchmarks, I found this article about storing credit card hashes, which says:
Storing credit cards using a simple single pass of a hash algorithm, even when salted, is fool-hardy. It is just too easy to brute force the credit card numbers if the hashes are compromised.
...
When hashing credit card number, the hashing must be carefully designed to protect against brute forcing by using strongest available cryptographic hash functions, large salt values, and multiple iterations.
The full article is well worth a thorough read. Unfortunately, the upshot seems to be that any circumstance that makes it 'safe' to store hashed credit card numbers will also make it prohibitively expensive to search for duplicates.
People are over thinking the design of this, I think. Use a salted, highly secure (e.g. "computationally expensive") hash like sha-256, with a per-record unique salt.
You should do a low-cost, high accuracy check first, then do the high-cost definitive check only if that check hits.
Step 1:
Look for matches to the last 4 digits (and possibly also the exp. date, though there's some subtleties there that may need addressing).
Step 2:
If the simple check hits, use the salt, get the hash value, do the in depth check.
The last 4 digits of the cc# are the most unique (partly because it includes the LUHN check digit as well) so the percentage of in depth checks you will do that won't ultimately match (the false positive rate) will be very, very low (a fraction of a percent), which saves you a tremendous amount of overhead relative to the naive "do the hash check every time" design.
Do not store a simple SHA-1 of the credit card number, it would be way too easy to crack (especially since the last 4 digits are known). We had the same problem in my company: here is how we solved it.
First solution
For each credit card, we store the last 4 digits, the expiration date, a long random salt (50 bytes long), and the salted hash of the CC number. We use the bcrypt hash algorithm because it is very secure and can be tuned to be as CPU-intensive as you wish. We tuned it to be very expensive (about 1 second per hash on our server!). But I guess you could use SHA-256 instead and iterate as many times as needed.
When a new CC number is entered, we start by finding all the existing CC numbers that end with the same 4 digits and have the same expiration date. Then, for each matching CC, we check whether its stored salted hash matches the salted hash calculated from its salt and the new CC number. In other words, we check whether or not hash(stored_CC1_salt+CC2)==stored_CC1_hash.
Since we have roughly 100k credit cards in our database, we need to calculate about 10 hashes, so we get the result in about 10 seconds. In our case, this is fine, but you may want to tune bcrypt down a bit. Unfortunately, if you do, this solution will be less secure. On the other hand, if you tune bcrypt to be even more CPU-intensive, it will take more time to match CC numbers.
Even though I believe that this solution is way better than simply storing an unsalted hash of the CC number, it will not prevent a very motivated pirate (who manages to get a copy of the database) to break one credit card in an average time of 2 to 5 years. So if you have 100k credit cards in your database, and if the pirate has a lot of CPU, then he can can recover a few credit card numbers every day!
This leads me to the belief that you should not calculate the hash yourself: you have to delegate that to someone else. This is the second solution (we are in the process of migrating to this second solution).
Second solution
Simply have your payment provider generate an alias for your credit card.
for each credit card, you simply store whatever you want to store (for example the last 4 digits & the expiration date) plus a credit card number alias.
when a new credit card number is entered, you contact your payment provider and give it the CC number (or you redirect the client to the payment provider, and he enters the CC number directly on the payment provider's web site). In return, you get the credit card alias! That's it. Of course you should make sure that your payment provider offers this option, and that the generated alias is actually secure (for example, make sure they don't simply calculate a SHA-1 on the credit card number!). Now the pirate has to break your system plus your payment provider's system if he wants to recover the credit card numbers.
It's simple, it's fast, it's secure (well, at least if your payment provider is). The only problem I see is that it ties you to your payment provider.
Hope this helps.
PCI DSS states that you can store PANs (credit card numbers) using a strong one-way hash. They don't even require that it be salted. That said you should salt it with a unique per card value. The expiry date is a good start but perhaps a bit too short. You could add in other pieces of information from the card, such as the issuer. You should not use the CVV/security number as you are not allowed to store it. If you do use the expiry date then when the cardholder gets issued a new card with the same number it will count as a different card. This could be a good or bad thing depending on your requirements.
An approach to make your data more secure is to make each operation computationally expensive. For instance if you md5 twice it will take an attacker longer to crack the codes.
Its fairly trivial to generate valid credit card numbers and to attempt a charge through for each possible expiry date. However, it is computationally expensive. If you make it more expensive to crack your hashes then it wouldn't be worthwhile for anyone to bother; even if they had the salts, hashes and the method you used.
#Cory R. King
SHA 1 isn't broken, per se. What the article shows is that it's possible to generate 2 strings which have the same hash value in less than brute force time. You still aren't able to generate a string that equates to a SPECIFIC hash in a reasonable amount of time. There is a big difference between the two.
I believe I have found a fool-proof way to solve this problem. Someone please correct me if there is a flaw in my solution.
Create a secure server on EC2, Heroku, etc. This server will serve ONE purpose and ONLY one purpose: hashing your credit card.
Install a secure web server (Node.js, Rails, etc) on that server and set up the REST API call.
On that server, use a unique salt (1000 characters) and SHA512 it 1000 times.
That way, even if hackers get your hashes, they would need to break into your server to find your formula.
Comparing hashes is a good solution. Make sure that you don't just salt all the credit card numbers with the same constant salt, though. Use a different salt (like the expiration date) on each card. This should make you fairly impervious to dictionary attacks.
From this Coding Horror article:
Add a long, unique random salt to each password you store. The point of a salt (or nonce, if you prefer) is to make each password unique and long enough that brute force attacks are a waste of time. So, the user's password, instead of being stored as the hash of "myspace1", ends up being stored as the hash of 128 characters of random unicode string + "myspace1". You're now completely immune to rainbow table attack.
Almost a good idea.
Storing just the hash is a good idea, it has served in the password world for decades.
Adding a salt seems like a fair idea, and indeed makes a brute force attack that much harder for the attacker. But that salt is going to cost you a lot of extra effort when you actually check to ensure that a new CC is unique: You'll have to SHA-1 your new CC number N times, where N is the number of salts you have already used for all of the CCs you are comparing it to. If indeed you choose good random salts you'll have to do the hash for every other card in your system. So now it is you doing the brute force. So I would say this is not a scalable solution.
You see, in the password world a salt adds no cost because we just want to know if the clear text + salt hashes to what we have stored for this particular user. Your requirement is actually pretty different.
You'll have to weigh the trade off yourself. Adding salt doesn't make your database secure if it does get stolen, it just makes decoding it harder. How much harder? If it changes the attack from requiring 30 seconds to requiring one day you have achieved nothing -- it will still be decoded. If it changes it from one day to 30 years you have achived someting worth considering.
Yes, comparing hashes should work fine in this case.
A salted hash should work just fine. Having a salt-per-user system should be plenty of security.
SHA1 is broken. Course, there isn't much information out on what a good replacement is. SHA2?
If you combine the last 4 digits of the card number with the card holder's name (or just last name) and the expiration date you should have enough information to make the record unique. Hashing is nice for security, but wouldn't you need to store/recall the salt in order to replicate the hash for a duplicate check?
I think a good solution as hinted to above, would be to store a hash value of say Card Number, Expiration date, and name. That way you can still do quick comparisons...
Sha1 broken is not a problem here.
All broken means is that it's possible to calculate collisions (2 data sets that have the same sha1) more easily than you would expect.
This might be a problem for accepting arbitrary files based on their sha1 but it has no relevence for an internal hashing application.
If you are using a payment processor like Stripe / Braintree, let them do the "heavy lifting".
They both offer card fingerprinting that you can safely store in your db and compare later to see if a card already exists:
Stripe returns fingerprint string - see doc
Braintree returns unique_number_identifier string - see doc