KRL: Signing requests with HMAC_SHA1 - krl

I made a test suite for math:hmac_* KRL functions. I compare the KRL results with Python results. KRL gives me different results.
code: https://gist.github.com/980788 results: http://ktest.heroku.com/a421x68
How can I get valid signatures from KRL? I'm assuming that they Python results are correct.
UPDATE: It works fine unless you want newline characters in the message. How do I sign a string that includes newline characters?

I suspect that your python SHA library returns a different encoding than is expected by the b64encode library. My library does both the SHA and base64 in one call so I to do some extra work to check the results.
As you show in your KRL, the correct syntax is:
math:hmac_sha1_base64(raw_string,key);
math:hmac_sha256_base64(raw_string,key);
These use the same libraries that I use for the Amazon module which is testing fine right now.
To test those routines specifically, I used the test vectors from the RFC (sha1, sha256). We don't support Hexadecimal natively, so I wasn't able to use all of the test vectors, but I was able to use a simple one:
HMAC SHA1
test_case = 2
key = "Jefe"
key_len = 4
data = "what do ya want for nothing?"
data_len = 28
digest = 0xeffcdf6ae5eb2fa2d27416d5f184df9c259a7c79
HMAC SHA256
Key = 4a656665 ("Jefe")
Data = 7768617420646f2079612077616e7420666f72206e6f7468696e673f ("what do ya want for nothing?")
HMAC-SHA-256 = 5bdcc146bf60754e6a042426089575c75a003f089d2739839dec58b964ec3843
Here is my code:
global {
raw_string = "what do ya want for nothing?";
mkey = "Jefe";
}
rule first_rule {
select when pageview ".*" setting ()
pre {
hmac_sha1 = math:hmac_sha1_hex(raw_string,mkey);
hmac_sha1_64 = math:hmac_sha1_base64(raw_string,mkey);
bhs256c = math:hmac_sha256_hex(raw_string,mkey);
bhs256c64 = math:hmac_sha256_base64(raw_string,mkey);
}
{
notify("HMAC sha1", "#{hmac_sha1}") with sticky = true;
notify("hmac sha1 base 64", "#{hmac_sha1_64}") with sticky = true;
notify("hmac sha256", "#{bhs256c}") with sticky = true;
notify("hmac sha256 base 64", "#{bhs256c64}") with sticky = true;
}
}
var hmac_sha1 = 'effcdf6ae5eb2fa2d27416d5f184df9c259a7c79';
var hmac_sha1_64 = '7/zfauXrL6LSdBbV8YTfnCWafHk';
var bhs256c = '5bdcc146bf60754e6a042426089575c75a003f089d2739839dec58b964ec3843';
var bhs256c64 = 'W9zBRr9gdU5qBCQmCJV1x1oAPwidJzmDnexYuWTsOEM';
The HEX results for SHA1 and SHA256 match the test vectors of the simple case.
I tested the base64 results by decoding the HEX results and putting them through the base64 encoder here
My results were:
7/zfauXrL6LSdBbV8YTfnCWafHk=
W9zBRr9gdU5qBCQmCJV1x1oAPwidJzmDnexYuWTsOEM=
Which match my calculations for HMAC SHA1 base64 and HMAC SHA256 base64 respectively.
If you are still having problems, could you provide me the base64 and SHA results from python separately so I can identify the disconnect?

Related

Multiple JWT's match hashed JWT

I'm using bcryptjs to hash a user's refresh_token before storing it in my database.
It seems that the following always evaluates to true when comparing a hashed string with a JWT, I've also gotten the same behavior on https://bcrypt-generator.com/
for example the hash $2a$10$z4rwnyg.cVtP2SHt3lYj7.aGeAzonmmzbxqCzi2UW3SQj6famGaqW is a match with the following two JWTs
eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJ1c2VySWQiOiI2NTZlODdkNi1jMmVkLTRmN2ItOTU2Zi00NDFhMWU1NjA2MmQiLCJpYXQiOjE2Mzk1OTg2MDIsImV4cCI6MTY0MjE5MDYwMn0.aJlzFHhBMGO4J7vlOudqOrOFnL1P-yEGrREgdaCXlxU
eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJ1c2VySWQiOiI2NTZlODdkNi1jMmVkLTRmN2ItOTU2Zi00NDFhMWU1NjA2MmQiLCJpYXQiOjE2Mzk2MDY4ODgsImV4cCI6MTY0MjE5ODg4OH0.vo4HKLXuQbT0Yb0j21M4xl-rakxyE5wINjuGdkPuSJY
You can verify these on the site as well that they both result in a 'match'
Go to https://bcrypt-generator.com/ and open your browser console.
Enter these lines into the console:
> var jwt1 = "eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJ1c2VySWQiOiI2NTZlODdkNi1jMmVkLTRmN2ItOTU2Zi00NDFhMWU1NjA2MmQiLCJpYXQiOjE2Mzk1OTg2MDIsImV4cCI6MTY0MjE5MDYwMn0.aJlzFHhBMGO4J7vlOudqOrOFnL1P-yEGrREgdaCXlxU"
< undefined
> var jwt2 = "eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJ1c2VySWQiOiI2NTZlODdkNi1jMmVkLTRmN2ItOTU2Zi00NDFhMWU1NjA2MmQiLCJpYXQiOjE2Mzk2MDY4ODgsImV4cCI6MTY0MjE5ODg4OH0.vo4HKLXuQbT0Yb0j21M4xl-rakxyE5wINjuGdkPuSJY"
< undefined
> var h = "$2a$10$z4rwnyg.cVtP2SHt3lYj7.aGeAzonmmzbxqCzi2UW3SQj6famGaqW"
< undefined
Then enter these lines into the console, observe how they return true:
> bcrypt.compareSync(jwt1, h)
< true
> bcrypt.compareSync(jwt2, h)
< true
This is my own JS code that also reproduces the hash match:
// Login Logic
const refresh_token: string = jwt.sign({ userId }, authSecrets.refresh_secret, { expiresIn: '30d' });
const hash_refresh = bcrypt.hashSync(refresh_token);
await UserModel.update({
id: user.id,
refresh_token: hash_refresh,
});
// Refresh logic
// 'value' is the payload after using joi to validate it
const claims: any = jwt.verify(value.refresh_token, authSecrets.refresh_secret);
user = await UserModel.get(claims.userId);
if (!bcrypt.compareSync(value.refresh_token, user.refresh_token)) {
// This never happens with any JWT!
return response(401, 'Refresh Token is incorrect');
}
Why is this happening? the strings are clearly different (although not by a lot).
The hash collisions are because bcrypt only hashes the first 72 bytes of input (in most implementations).
This is documented in the README for both the bcryptjs and bcrypt npm packages:
bcryptjs:
The maximum input length is 72 bytes (note that UTF8 encoded characters use up to 4 bytes) and the length of generated hashes is 60 characters.
bcrypt:
Per bcrypt implementation, only the first 72 bytes of a string are used. Any extra bytes are ignored when matching passwords. Note that this is not the first 72 characters. It is possible for a string to contain less than 72 characters, while taking up more than 72 bytes (e.g. a UTF-8 encoded string containing emojis).
(That's an objectively terrible design considering this is for user-security... The bcryptjs library really should always throw an exception if the input exceeds 72 bytes IMO)
I note that bcrypt is design for human-supplied (i.e. non-random) passwords, not as a general-purpose message-digest algorithm. Given you don't need to add a salt to randomly-generated passwords (like your refresh_token value) you probably should use something like a SHA-2 family algorithm (e.g. SHA-256, but not SHA-1) for this.

Spotify PKCE in Dart/Flutter: "code_verifier was incorrect"

Using the Authorization Code Flow with PKCE of the Spotify-API I am getting the error that my code_verifier is incorrect, which has to be an encoding problem from what I know by now.
{"error":"invalid_grant","error_description":"code_verifier was incorrect"}
This is the original code I wrote:
String getAuthUrl() {
code = getRandomString(128);
// saveToPrefs("verfifier_code", code);
var hash = sha256.convert(ascii.encode(code));
String code_challenge = base64Url.encode(hash.bytes);
return Uri.parse(
"https://accounts.spotify.com/authorize?response_type=code&client_id=${widget.client_id}&redirect_uri=http%3A%2F%2Flocalhost%2Fauth&scope=user-top-read&code_challenge=$code_challenge&code_challenge_method=S256")
.toString();
}
This is how I understand the Spotify-Authorisation-Guide (https://developer.spotify.com/documentation/general/guides/authorization-guide/).
After finding this post (https://stackoverflow.com/a/63174909/14266484) I tried porting the fix to Dart but failed. As far as I understand it the code hashes the in ascii encoded code_verifier and then uses btoa() to convert it into ascii again. This function also seems to do base64 (but not base64Url, maybe this is why certain parts have to be replaced manually?).
String getAuthUrl() {
// also tried static verifier_codes for debugging, so the getRandomString() function is working properly
code = getRandomString(128);
// saveToPrefs("verfifier_code", code);
var hash = sha256.convert(ascii.encode(code));
// this does not work with either base64 or base64Url
String code_challenge = base64.encode(hash.bytes).replaceAll(RegExp(r"/\+/g"), '-').replaceAll(RegExp(r"/\//g"), '_').replaceAll(RegExp(r"/=+$/"), '');
return Uri.parse(
"https://accounts.spotify.com/authorize?response_type=code&client_id=${widget.client_id}&redirect_uri=http%3A%2F%2Flocalhost%2Fauth&scope=user-top-read&code_challenge=$code_challenge&code_challenge_method=S256")
.toString();
}
I also tried different ways of encoding:
-Using String.codeUnits (But this is using UTF-16)
-Getting a String for the sha256-function aswell the base64(-Url)-function with String.fromCharCodes() (which should be using ASCII?)
-Switching between ASCII and UTF-8 (Which should not make a difference in that case as my verifier_code is made up of ASCII-Characters only)
EDIT:
To make the requests I use:
var res = await http.post(endpoint, body: {"client_id":widget.client_id, "grant_type":"authorization_code", "code":value, "redirect_uri":"http://localhost/auth", "code_verifier":code});
After some more research I found out that the important thing that is happening is that the "=" at the end of the challenge has to be removed (Shouldn't base64Url do that?). Anyways, that is the working code:
EDITED CODE:
String getAuthUrl() {
code = getRandomString(128);
var hash = sha256.convert(ascii.encode(code));
String code_challenge = base64Url.encode(hash.bytes).replaceAll("=", "").replaceAll("+", "-").replaceAll("/", "_");
return Uri.parse(
"https://accounts.spotify.com/authorize?response_type=code&client_id=${widget.client_id}&redirect_uri=http%3A%2F%2Flocalhost%2Fauth&scope=user-top-read&code_challenge=$code_challenge&code_challenge_method=S256")
.toString();
}
EDIT:
Further "+" has to be replaced with "-" and "/" with "_"!

crypto-js decrypt from Hex

I am trying to make a JavaScript function with package crypto-js to decode AES (CBC mode).
I input the data in an online decoding tool and it decrypted correctly, so I am sure the following data is correct, but I just can't reproduce it by JavaScript.
Here is the online decrypting (so I'm sure the data, key, iv are correct): http://aes.online-domain-tools.com/link/deb718giF4dUxZylq/
My code with crypto-js#3.1.8:
// data, key, iv are all Hex
var data = "bd6e0a73147a2c224c7c20346d0e9a138b744a5d94463cdff6dbb965055f974f097104399d2c40af2f0ac667f3857e70e9703bf27f6411f7e97c3449e8921f3c98e665914689b4b77b5bbcc8d8bc319e680eb89eedb1c25178923ae57fb3fb476755d6009f1aed88fffcb9b2ed3b4cf6f23d9c4c56da1dde6619e45a8d6f06412853ae1941cf554b6824112a913750a7485ed67fb38b950411310410de998f2597c2fcc81a305b0df369f54b75426176";
var key = 'befce5c6da98837ea421811c832817ae';
var iv = "a884a7edd5d06a48d6da9ad11fd36a75";
// transfer Hex to WordArray
var _data = CryptoJS.enc.Hex.parse(data);
var base64_data = _data.toString(CryptoJS.enc.Base64);
var _key = CryptoJS.enc.Hex.parse(key);
var _iv = CryptoJS.enc.Hex.parse(iv);
decrypted = CryptoJS.AES.decrypt(
base64_data, // pass base64
_key, // pass WordArray
{iv: _iv, // pass WordArray
mode: CryptoJS.mode.CBC,
padding: CryptoJS.pad.ZeroPadding
})
console.log(decrypted.toString(CryptoJS.enc.Utf8));
// out put fail to match Utf8
It output Error: Malformed UTF-8 data
The decoded string should be: (the link is not important)
https://emogo-media-testing.s3.amazonaws.com/1503342403787_blob?AWSAccessKeyId=AKIAI5MUDCK6XYWKGAKA&Expires=1534882403&Signature=t1PFesQuOpOlIMKoOqje%2Bs7I%2Fhg
Any hint is appreciated. Thank you!
I know it has been a while since you asked the question but I will respond just so the next person does not stumble upon an unanswered question.
Your code works fine, it decrypts AES.CBC encrypted data correct, the problem lies with your input data.
Your encrypted data string should have looked like:
80b7c4881334675693ef9c95259e70b24d0736e98f8424233d5e37f353261c2a589287bc3f675449f7d8ed4e2289a4c06b22d7f83efc09cfb72abe3a76e193a8efbdc968232d29b9b58135bfa24d51e60e34791f652a0aa806d0be7734dd61a930a30c99f31f08740cdb182af07b19d5b4274deb958d984b3ccb9d6e2be0cfa3a026dd6b734dbf1dd3635bc7bcceface9c55dfb9455ca834a6dbd1aa0f3c23923ce6aeba59acbc80d681fee73487b9004496540830d44102b94e35eac291c4e3b8c9ac168ae799e46cde45ee652415ae69992d0f7527045fd42b82e9e6946cfb2dbcc3b93f19ff0e5035ab12250f7a917975b2f7c069cbd8a0ba0d94b318634a
for this example to work correctly.
The key you used is not a hex string but a text string. Your online example is no longer valid but I figured it out after a couple of tries.
If change the following line:
var _key = CryptoJS.enc.Hex.parse(key);
to:
var _key = CryptoJS.enc.Utf8.parse(key);
Your code example will work fine with your original data string.
When you decrypted the text on http://aes.online-domain-tools.com/ you probably had the plaintext textbox selected instead of hex for your key input.

Kraken API MATLAB client invalid signature error

I'm trying to do some authenticated calls to Kraken private endpoints but without success. I'm still getting an error EAPI:Invalid signature.
Does anybody know what's wrong?
Here's the code:
function [response,status]=kraken_authenticated(uri,postdata)
% test uri='0/private/AddOrder'
% test postdata='&pair=XBTEUR&type=buy&ordertype=limit&price=345.214&volume=0.65412&leverage=1.5&oflags=post'
url=['https://api.kraken.com/',uri];
% nonce
nonce = num2str(floor((now-datenum('1970', 'yyyy'))*8640000000));
[key,secret]=key_secret('kraken');
% 1st hash
Opt.Method = 'SHA-256';
Opt.Input = 'ascii';
sha256string = DataHash(['nonce=',nonce,postdata],Opt);
% 2nd hash
sign = crypto([uri,sha256string], secret, 'HmacSHA512');
header_1=http_createHeader('API-Key',key);
header_2=http_createHeader('API-Sign',char(sign));
header=[header_1 header_2];
[response,status] = urlread2(url,'POST',['nonce=',nonce,postdata],header);
end
Crypto function is in another file:
function signStr = crypto(str, key, algorithm)
import java.net.*;
import javax.crypto.*;
import javax.crypto.spec.*;
import org.apache.commons.codec.binary.*
keyStr = java.lang.String(key);
key = SecretKeySpec(keyStr.getBytes('UTF-8'), algorithm);
mac = Mac.getInstance(algorithm);
mac.init(key);
toSignStr = java.lang.String(str);
signStr = java.lang.String(Hex.encodeHex( mac.doFinal( toSignStr.getBytes('UTF-8'))));
end
I've also tried
sign = crypto([uri,sha256string], base64decode(secret), 'HmacSHA512');
but without success.
This is guide for authenticated call HTTPS Header:
API-Key = API key
API-Sign = Message signature using HMAC-SHA512 of (URI path + SHA256(nonce + POST data)) and base64 decoded secret API key
This is guide for authenticated call POST Data:
nonce = always increasing unsigned 64 bit integer
otp = two-factor password (if two-factor enabled, otherwise not required)
I've tried to pass "nonce" parameter or all parameters in "postdata" to POST data but without success.
Thanks for help.
The problem is in function crypto here:
keyStr = java.lang.String(key);
key = SecretKeySpec(keyStr.getBytes('UTF-8'), algorithm);
As the base64 encoded private key from kraken is not necessarily UTF-8 encoded, you cannot use UTF-8 encoding to extract the key and pass UTF-8 string to the SecretKeySpec function. You need to use byte array instead.
Similar issues
https://code.google.com/p/google-apps-script-issues/issues/detail?id=5113
https://code.google.com/p/google-apps-script-issues/issues/detail?id=3121
Solution for javascript
github.com/Caligatio/jsSHA

Storing Special Characters in Windows Azure Blob Metadata

I have an app that is storing images in a Windows Azure Block Blob. I'm adding meta data to each blob that gets uploaded. The metadata may include some special characters. For instance, the registered trademark symbol (®). How do I add this value to meta data in Windows Azure?
Currently, when I try, I get a 400 (Bad Request) error anytime I try to upload a file that uses a special character like this.
Thank you!
You might use HttpUtility to encode/decode the string:
blob.Metadata["Description"] = HttpUtility.HtmlEncode(model.Description);
Description = HttpUtility.HtmlDecode(blob.Metadata["Description"]);
http://lvbernal.blogspot.com/2013/02/metadatos-de-azure-vs-caracteres.html
The supported characters in the blob metadata must be ASCII characters. To work around this you can either escape the string ( percent encode), base64 encode etc.
joe
HttpUtility.HtmlEncode may not work; if Unicode characters are in your string (i.e. &#8217), it will fail. So far, I have found Uri.EscapeDataString does handle this edge case and others. However, there are a number of characters that get encoded unnecessarily, such as space (' '=chr(32)=%20).
I mapped the illegal ascii characters metadata will not accept and built this to restore the characters:
static List<string> illegals = new List<string> { "%1", "%2", "%3", "%4", "%5", "%6", "%7", "%8", "%A", "%B", "%C", "%D", "%E", "%F", "%10", "%11", "%12", "%13", "%14", "%15", "%16", "%17", "%18", "%19", "%1A", "%1B", "%1C", "%1D", "%1E", "%1F", "%7F", "%80", "%81", "%82", "%83", "%84", "%85", "%86", "%87", "%88", "%89", "%8A", "%8B", "%8C", "%8D", "%8E", "%8F", "%90", "%91", "%92", "%93", "%94", "%95", "%96", "%97", "%98", "%99", "%9A", "%9B", "%9C", "%9D", "%9E", "%9F", "%A0", "%A1", "%A2", "%A3", "%A4", "%A5", "%A6", "%A7", "%A8", "%A9", "%AA", "%AB", "%AC", "%AD", "%AE", "%AF", "%B0", "%B1", "%B2", "%B3", "%B4", "%B5", "%B6", "%B7", "%B8", "%B9", "%BA", "%BB", "%BC", "%BD", "%BE", "%BF", "%C0", "%C1", "%C2", "%C3", "%C4", "%C5", "%C6", "%C7", "%C8", "%C9", "%CA", "%CB", "%CC", "%CD", "%CE", "%CF", "%D0", "%D1", "%D2", "%D3", "%D4", "%D5", "%D6", "%D7", "%D8", "%D9", "%DA", "%DB", "%DC", "%DD", "%DE", "%DF", "%E0", "%E1", "%E2", "%E3", "%E4", "%E5", "%E6", "%E7", "%E8", "%E9", "%EA", "%EB", "%EC", "%ED", "%EE", "%EF", "%F0", "%F1", "%F2", "%F3", "%F4", "%F5", "%F6", "%F7", "%F8", "%F9", "%FA", "%FB", "%FC", "%FD", "%FE" };
private static string MetaDataEscape(string value)
{
//CDC%20Guideline%20for%20Prescribing%20Opioids%20Module%206%3A%20%0Ahttps%3A%2F%2Fwww.cdc.gov%2Fdrugoverdose%2Ftraining%2Fdosing%2F
var x = HttpUtility.HtmlEncode(value);
var sz = value.Trim();
sz = Uri.EscapeDataString(sz);
for (int i = 1; i < 255; i++)
{
var hex = "%" + i.ToString("X");
if (!illegals.Contains(hex))
{
sz = sz.Replace(hex, Uri.UnescapeDataString(hex));
}
}
return sz;
}
The result is:
Before ==> "1080x1080 Facebook Images"
Uri.EscapeDataString =>
"1080x1080%20Facebook%20Images"
After => "1080x1080 Facebook
Images"
I am sure there is a more efficient way, but the hit seems negligible for my needs.