I was searching through the internet for the meaning of this prefix.
Solidity docs do not have a clear explanation.
So what is this and why should I use it?
"\x19Ethereum Signed Message:\n32",
The prefix was introduced in EIP-191 as a way to distinguish signatures meant for Ethereum smart contracts, and for other cryptographic platforms. By adding the Ethereum-specific prefix, the message (with the prefix) results in a different signature, specific just for Ethereum. But if there were no platform-specific prefixes, the resulting signature would be the same for all platforms, which could potentially lead to signature replay attacks.
Note: Back in 2016, when this standard was introduced, there were no other EVM networks (e.g. Binance Smart Chain and Polygon) - or at least they were not widely used. Nowdays it's common to use the "Ethereum" prefix even if the message is intended for other EVM networks such as Polygon.
This specific byte 0x19 was standardized simply because an already existing implementation (in the Go Ethereum client software) was using it before the standard was finalized.
And the last number (32 in this case) is the byte length of the message (excluding the prefix) that is being signed.
Related
I am writing a parser (in C++) and I have a small list of strings (less than 100) where each one represents a valid parser tag. I need to map each such known tag to an enum value for further processing.
As all strings are known at compile time, I have been looking into using a perfect hash function for this purpose.
I am aware of existing tools and algorithms for perfect hash function generation s.a. gperf, mph, cmph. However, all such tools/implementations are under some restrictive license (such as GPL, LGPL, MPL), while due to my limitations I am looking for some code which is under a relaxed license for reuse (such as MIT license) and preferably in C/C++ or C#.
Are you aware of any such tool or code ?
Yes, here's one that seems to fit your parameters:
https://www.codeproject.com/Articles/989340/Practical-Perfect-Hashing-in-Csharp
Note it's using a license agreement that I'm not particularly familiar with. But it doesn't look like its GPL related.
Is there a standard, canonical method for creating a fingerprint (aka thumbprint) for a JWK?
From what I was reading it seems that the standard doesn't define how a kid should be specified, which I find odd. To me it makes the most since to have it be a deterministic value rather than one that requires a lookup table such that others could easily recreate the key id in by virtue of possessing the public key.
I am aware that SSH fingerprints and X.509 thumbprints are standardized, but those don't seem like a suitable solution for all environments where JWKs are used (especially browsers) because they are too complex for naive implementations and including the libraries capable of manipulating such (i.e. forge) would waste a lot of memory, bandwidth, and vm compile time.
Update
Officially it's called a "thumbprint" not a "fingerprint".
I think the RFC7638 will answer your question.
This RFC describes a way to compute a hash value over a JWK.
It is really easy to implement:
Keep the required parameters only. For a RSA key: kty, n and e and for an EC key: crv, kty, x and y.
Sort those parameters in lexicographic order: e,kty and n
Compute the parameters and values into Json: {"e":"AQAB","kty":"RSA","n":"0vx7agoebGcQSuuPiLJXZptN9nndrQmbXEps2 aiAFbWhM78LhWx4cbbfAAtVT86zwu1RK7aPFFxuhDR1L6tSoc_BJECPebWKRXjBZCi FV4n3oknjhMstn64tZ_2W-5JsGY4Hc5n9yBXArwl93lqt7_RN5w6Cf0h4QyQ5v-65Y GjQR0_FDW2QvzqY368QQMicAtaSqzs8KJZgnYb9c7d0zgdAZHzu6qMQvRL5hajrn1n 91CbOpbISD08qNLyrdkt-bFTWhAI4vMQFh6WeZu0fM4lFd2NcRwr3XPksINHaQ-G_x BniIqbw0Ls1jF44-csFCur-kEgU8awapJzKnqDKgw"}
Hash it using SHA-256 and encode it into Base64 Url Safe: NzbLsXh8uDCcd-6MNwXF4W_7noWXFZAfHkxZsRGC9Xs
I don't believe there is a true standard, but this topic has been discussed in the IETF mailing archives. While the conversation seemed to get a little side-tracked by whether or not canonical JSON was a good idea in general, there was one method that seems reasonable as a standard fingerprinting method.
Remove all "metadata" fields from the JWK (where in this case "metadata" is defined as any non-required key, ie anything but "kty" and the parameters for the encryption algorithm defined by the JWA RFC-7518).
Convert stripped JWK into "canonical" JSON (sort keys lexicographically, no leading or trailing whitespace, and no whitespace between tokens).
Compute digest over created JSON string.
There is also no true standard that I am aware of for canonical JSON, but all the sources I've seen agree on at least the rules listed above (which are the only rules that should be relevant for the types of objects used for JWK's).
I am designing a protocol to exchange IOUs (digital promissory notes).
These should be digitally signed, but the signature should be independent from the data representation (whether its XML, JSON, binary, little or big endian numbers).
Is there any standard on how to sign a list of strings and primitive types (like integers, floating points, booleans)?
There isn't one standard encoding, but you can specify canonical forms for particular encodings.
For json you could specify that there is no whitespace outside strings and that keys should be sorted in a particular way.
For ASN.1 there is DER encoding, which is the canonical form of BER.
There is Cryptographic Message Syntax (CMS), but I don't know much about it.
The better question is what is the best format for verifying digitally signed Data primitives.
The answer is xml formatted and signed according to the XAdES standard. XAdES is harmonized with the related standards and many implementations participate in interoperability tests hosted by etsi.
Unless it is easy to verify a digitally signed format, the signature has limited value.
You can sign any bit stream and store/maintain the signature as a detached signature. But then you and the relying parties (the recipients) need to deal with two files. One for the data and one for the signature.
The advantage of xml with XAdES is that the format enables the signed xml file to include the digital signature.
You can create an equivalent of XAdES for another data format such as json. But a new format has limited use unless it becomes popular and standardized. XAdES has already accomplished this, so it is the way to go.
Added
Re: comment--
I want to provide non-repudiation. I understand that I have to save the information I signed. But I was hoping that I don't have to save it as XML but could rather save all values included in the signature in a database (less verbosely) and uniquely reconstruct the signed string from them before verifying.
Technically, you can do that. You'll need to watch out for spacing issues within the xml. But practically, not a good idea. Why:
Proving non-repudiation requires that you meet the applicable burden of proof that the alleged signer really did sign the data.
You may be trying to convince the original signer of this, an expert third party (an auditor) or non-experts (lawyers and juries). You want to make it easy and simple to convince these people. Schemes such as "re-creating" the signed file are not simple to understand compared with "here is the original signed file. Its signature verifies and it was signed with the digital certificate belonging to Susan Signer."
To keep it simple, I'd suggest signing an XAdES XML file. Then extract the data from the file and use it in your dbms. Hang on to the original signed file in your dbms or elsewhere. In case of a dispute, produce the original file and show that it verifies. A second part of the audit would be to show that your dbms has the same data values as the signed XML.
The programming and storage costs of hanging on to the original, signed, xml file are de minimis, when compared with your goal of proving non-repudiation of the data.
By the way, how is the signer's certificate managed? If it is anything less than a QSCD (Qualified Signature Creation Device), such as storing the cert in the file system, then you have another problem: no way to conclusively prove that the certificate wasn't used by an imposter. Use a secure system for signing such as CoSign (my company) or an equivalent system.
I am kinda playing with the SHA-1 algorithm. I want to find out differences and variations in the results if I change few values in the SHA-1 algorithm for a college report. I have found a piece of java code to generate hash of a text. Its done by importing
java.security.MessageDigest
class. However, I want to change the h0-4 values and edit them but I don't know where can I find them? I had a look inside the MessageDigest class but couldn't find it there. Please help me out!
Thanx in advance.
I don't believe you can do that. Java doesn't provide any API for its MessageDigest Class, which can allow you change the values.
However, there are some workarounds (none of which I've ever tried). Take a look at this answer to the question "How to edit Java Platform Package (Built-in API) source code?"
If you're playing around with tweaks to an algorithm, you shouldn't be using a built-in class implementing that algorithm. The class you mention is designed to implement standard algorithms for people who just want to use them in production; if you're using SHA-1 (or any cryptographic algorithm) instead of playing around and tweaking it, it's never a good idea to change the algorithm yourself (e.g. by changing the initial hash value), so the class does not support modifying those constants.
Just implement the algorithm yourself; from Wikipedia's pseudocode, it doesn't look like it's all that complicated. I know that "don't implement your own crypto, use a standard and well-tested implementation" is a common mantra here, but that only applies to production-type code -- if you're playing around with an algorithm to see what effect tweaking it has, you should implement it yourself, so you have more flexibility in modifying it and seeing the effect of the modifications.
Basically adding to #Rahil's answer but too much for comments:
Even without API access, if MessageDigest were the implementation you could use reflection. But it's not.
Most of the java standard library is just commonly-useful classes in the usual way, e.g. java.util.ArrayList contains the implementation of ArrayList (or ArrayList<?> since 6), java.io.FileInputStream contains the implementation of FileInputStream (although it may use other classes in that implementation), etc. Java Cryptography uses a more complicated scheme where the implementations are not in the API classes but instead in "providers" that are mostly in their own jars (in JRE/lib and JRE/lib/ext) not rt.jar and mostly(?) don't have source in src.zip.
Thus the java.security.MessageDigest class does not have the code to implement SHA1, or SHA256, or MD5, etc etc. Instead it has code to search the JVM's current list of crypto providers to find an implementation of whatever algorithm is asked for, and instantiate and use that. Normally the list of providers used is set to (the list of) those included in the JRE distribution, although an admin or program can change it.
With the normal JRE7 providers, SHA1 is implemented by sun.security.provider.SHA.
In effect the API classes like MessageDigest Signature Cipher KeyGenerator etc function more like interfaces or facades by presenting the behavior that is common to possibly multiple underlying implementations, although in Java code terms they are actual classes and not interfaces.
This was designed back in 1990 or so to cope with legal restrictions on crypto in effect then, especially on export from the US. It allowed the base Java platform to be distributed easily because by itself it did no crypto. To use it -- and even if you don't do "real" crypto on user data in Java you still need things like verification of signed code -- you need to add some providers; you might have one set of providers, with complete and strong algorithms, used in US installations, and a different set, with fewer and weaker algorithms, used elsewhere. This capability is now much less needed since the US officially relaxed and in practice basically dropped enforcement about 2000, although there are periodically calls to bring it back. There is still one residual bit, however: JCE (in Oracle JREs) contains a policy that does not allow symmetric keys over 128 bits; to enable that you must download from the Oracle website and install an additional (tiny) file "JCE Unlimited Strength Policy".
TLDR: don't try to alter the JCE implementation. As #cpast says, in this case where you want to play with something different from the standard algorithm, do write your own code.
Since SHA-3 seems to be an already known function (Keccak as the finalist of NIST hash function competition) I have several questions related to this topic:
NIST site says that NIST is closed due to a lapse in government funding. Is there any chance that SHA-3 will ever be finally accepted?
BouncyCastle library has an implementation of SHA-3 which digest results are the same as examples posted in wikipedia article (I tested this). Since the final standard is not approved, can this be trusted? Wikipedia says this is likely to be changed but how can it change as the final algorithm does not seem to be a subject to change (or else it would be another algorithm).
Here someone noted that usage of PBKDF2 with SHA-3 for key strengthening and password hashing should be avoided. But I cannot understand why? (how can it give attacker an advantage if the algorithm is not fast?)
I could not find test vectors anywhere to test my implementation of PBKDF2-HMAC-SHA3 in scala based on BouncyCastle java api. I can post my test spec with some results. But first can anybody post any/spec test vectors?
Here is my implementation in scala:
package my.crypto
import org.bouncycastle.crypto.digests.SHA3Digest
import org.bouncycastle.crypto.generators.PKCS5S2ParametersGenerator
import org.bouncycastle.crypto.PBEParametersGenerator
import org.bouncycastle.crypto.params.KeyParameter
object PBKDF2WithHmacSHA3 {
def apply(password: String, salt: Array[Byte], iterations: Int = 65536, keyLen: Int = 256): Array[Byte] = {
val generator = new PKCS5S2ParametersGenerator(new SHA3Digest(256))
generator.init(
PBEParametersGenerator.PKCS5PasswordToUTF8Bytes(password.toCharArray),
salt,
iterations
)
val key = generator.generateDerivedMacParameters(keyLen).asInstanceOf[KeyParameter]
key.getKey
}
}
One questionable thing for me is new SHA3Digest(256), the 256 bit length in the constructor, should it be same as provided key length or some fixed one as I did? I decided to use a fixed length because only some fixed values can be used and object API user can provide any value as key length parameter, but most of uncommon ones would result in exception thrown from inside SHA3Digest constructor. Also the default value seem to be 288 (when no key length is provided) which looks strange.
Thanks in advance!
Shutdown is temporary. SHA-3 will most likely be standardized at some point in 2014.
No, those values are probably for Final Round Keccak, not for SHA-3. There is no SHA-3 spec yet and it's quite likely that SHA-3 will be tweaked before standardization.
=> it's impossible to implement SHA-3 now, you can only implement Keccak.
Password hashes should be as expensive as possible for the attacker. The attacker uses different hardware from the defender, at minimum a GPU, but possible even custom chips.
The defender has a limited time budged for a hash (e.g. 100ms) and wants a function that's as expensive as possible for the attacker given that constraint. This means that custom hardware shouldn't gain a big advantage over a standard computer. So it's preferable to use a software friendly hash, but Keccak is relatively hardware friendly.
SHA-1 and SHA-2 are decent in hardware as well, so in practice the difference is small compared to the advantage other password hashes have over PBKDF2-HMAC-SHA-x. If you care about security instead of standard conformance, I recommend scrypt.