I have an task to generate an BASE64 string from HTTP response which will be used by the external program.Before I was doing this,I had known that if you use BASE64,the length of the string will be extend like 57 * 4/3 = 76.
But I've no idea which need to be handle if the HTTP respone longer than 57 bytes! So I had not any special handle for the HTTP response,just direct covert "$response->content" to which I want.(Actually length of the response more than 57 bytes).
3.The unexpected thing is that the length of the encode string not excatly follow 4/3 rule!!! When the len of input string is 57,then len of encode string is 77. When the len of input string is 114, the encode string is 154,why?
4.When I try to use the BASE64 output from an external c# arg(),seems it can only receive first 57 bytes.
#Sample Code
my $cont = $response->content;
$cont = substr ($cont, 0, 57);
my $encode = encode_base64($cont);
printf("Length Before Decode = %d.\n",length($cont));
printf("Length After Decode = %d.\n",length($encode));
#
According to this note, encode_base64() adds a newline. Are you counting that newline in the encoded string length? It seems to account for your extra character.
Related
Is there any reason why when I run the following
var name = "A"
withUnsafeBytes(of: &name, { bytes in
print (bytes)
for byte in bytes.enumerated() {
print (byte.element, byte.offset)
}
})
The last byte is 255?
I expected the bytes to just contain 65 as that is the ASCII code!
That is, byte 0 is 65 (as expected) and byte 15 is 255 (all the rest are zeroed)
Why is byte 15 255?
struct String is a (opaque) structure, containing pointers to the actual character storage. Short strings are stored directly in those pointers, which is why in your particular case the byte 65 is printed first.
If you run your code with
var name = "A really looooong string"
then you'll notice that there is no direct connection between the output of the program and the characters of the string.
If the intention is to enumerate the bytes of the UTF-8 representation of the string then
for byte in name.utf8 { print(byte) }
is the correct way.
How can I treat a string which is a hexadecimal number as a hexadecimal number? For example, I am loading in a file of hexadecimal numbers but Python is loading the file in as a string. Is it possible to get Powershell to treat the hexadecimal numbers in the file as hexadecimal numbers?
Background: I'm asking the user for a Mac Address and I need to add x numbers to the last block to get the next x Mac addresses. I can split on ":" and get the last block but this is a string containing for example "13", which is not 13 decimal but 13 hex.
Try converting it first to an int by casting the variable with the base 16 as the second argument
hex_str = "0xAD4"
hex_int = int(hex_str, 16)
then just add 1 to the result and use
hex(hex_int)
to get the result back to hex format
Just the whole MAC address in once by using the PhysicalAddress.Parse(String) Method:
$MAC = 'F0:E1:D2:C3:B4:A5' -Replace '[^0-9a-fA-F]' # remove all the non-Hex characters
[System.Net.NetworkInformation.PhysicalAddress]::Parse($MAC).GetAddressBytes()
240
225
210
195
180
165
Solved it by doing it like this:
$MacNr = ([int64]"0x$($MacAddress.Split(":")[5])")
$MacNr ++1
$NewMac = ('{0:X2}' -f $MacNr)
I understand that you have a hex string and perform SHA256 on it twice and then byte-swap the final hex string. The goal of this code is to find a Merkle Root by concatenating two transactions. I would like to understand what's going on in the background a bit more. What exactly are you decoding and encoding?
import hashlib
transaction_hex = "93a05cac6ae03dd55172534c53be0738a50257bb3be69fff2c7595d677ad53666e344634584d07b8d8bc017680f342bc6aad523da31bc2b19e1ec0921078e872"
transaction_bin = transaction_hex.decode('hex')
hash = hashlib.sha256(hashlib.sha256(transaction_bin).digest()).digest()
hash.encode('hex_codec')
'38805219c8ac7e9a96416d706dc1d8f638b12f46b94dfd1362b5d16cf62e68ff'
hash[::-1].encode('hex_codec')
'ff682ef66cd1b56213fd4db9462fb138f6d8c16d706d41969a7eacc819528038'
header_hex is a regular string of lower case ASCII characters and the decode() method with 'hex' argument changes it to a (binary) string (or bytes object in Python 3) with bytes 0x93 0xa0 etc. In C it would be an array of unsigned char of length 64 in this case.
This array/byte string of length 64 is then hashed with SHA256 and its result (another binary string of size 32) is again hashed. So hash is a string of length 32, or a bytes object of that length in Python 3. Then encode('hex_codec') is a synomym for encode('hex') (in Python 2); in Python 3, it replaces it (so maybe this code is meant to work in both versions). It outputs an ASCII (lower hex) string again that replaces each raw byte (which is just a small integer) with a two character string that is its hexadecimal representation. So the final bit reverses the double hash and outputs it as hexadecimal, to a form which I usually call "lowercase hex ASCII".
I am using ColdFusion 9
Referencing Ben Nadel's good works on his blog, I tried
ucase(digestUtils.sha512(imageBinary))
For SHA-512 hashing I get that dreaded:
The sha512 method was not found. Either there are no methods with the
specified method name and argument types or the sha512 method is
overloaded with argument types that ColdFusion cannot decipher
reliably. ColdFusion found 0 methods that match the provided
arguments. If this is a Java object and you verified that the method
exists, use the javacast function to reduce ambiguity.
Now I know that sha512 does indeed exist as a method, because I saw it here, but when I perform a
cfdump var="#digestUtils#"
I only get:
md5(byte[]) byte[]
md5(java.lang.String) byte[]
md5Hex(byte[]) java.lang.String
md5Hex(java.lang.String) java.lang.String
sha(java.lang.String) byte[]
sha(byte[]) byte[]
shaHex(java.lang.String) java.lang.String
shaHex(byte[]) java.lang.String
What happened to the other methods? I guess I have to try something else.
Please advise with a ColdFusion solution. A ColdFusion/Java solution would be ok too.
I'm trying to write a SSO application where the 3rd party guys feeds me URL parameters. I have successfully decoded the 1st parameter to get my XML Post. I now need to take the 2nd parameter which is the hash payload and go through the algorithm to ensure my 1st parameter hasn't been tampered with.
=========
Editing begins here: Okay,I tried writing the code again to no avail.
The algorithm sounds simple enough. But trying to implement it is killing me.
1. compute the hash string value of the XMLPost string above:
a. convert the base64 salt string to a UTF-8 byte array.
b. convert the base64 XML payload string to a UTF-8 byte array.
c. create a new byte array consisting of the XML payload bytes from step b, appended with the salt bytes from step a.
d. perform a SHA512 hash on the concatenated byte array from step c, which results in a hashed byte array.
e. create a new byte array consisting of the hashed bytes from step d, appended with the salt bytes from step a.
f. convert the result of step e to a base64-encoded string and should be the value of query string parameter "h" payload hash.
xmlPost was created by my third party guys as such:
This XML payload string was converted to a UTF-8 byte array, which was then converted to a base-64 string. The resulting base-64 string is the value of my xmlPost below.
So I do this:
<code>
<cfset xmlPost = urlDecode("PD94bWwgdmVyc2lvbj0iMS4wIj8%2bPEVzdG9yZVNzb0N1c3RvbWVyIHhtbG5zOnhzaT0iaHR0cDovL3d3dy53My5vcmcvMjAwMS9YTUxTY2hlbWEtaW5zdGFuY2UiIHhtbG5zOnhzZD0iaHR0cDovL3d3dy53My5vcmcvMjAwMS9YTUxTY2hlbWEiPjxDdXN0b21lcklkPjExMjk0MDwvQ3VzdG9tZXJJZD48RGVhbGVyQ29kZT5OODg4ODg8L0RlYWxlckNvZGU%2bPFBvaW50QmFsYW5jZT4yODA8L1BvaW50QmFsYW5jZT48Rmlyc3ROYW1lPkZhaXRoPC9GaXJzdE5hbWU%2bPExhc3ROYW1lPkh1dHVsYTwvTGFzdE5hbWU%2bPC9Fc3RvcmVTc29DdXN0b21lcj4%3d") />
<cfset salt = "3dfjh674!MujErf98344#090" />
<cfset payload_hash = urlDecode("EtLDRJfcRESFKpY4OGZZnRSN2THqT%2bEelzOuXVU06jotd2kE4yKnlYay7BqyAdcUSATRgSMaHxZa6uBqKKd9rjNkZmpoNjc0IU11akVyZjk4MzQ0QDA5MA%3d%3d") />
<cfset strXML = ToString( ToBinary( xmlpost ) ) /> <!--- to get actual XML --->
<!--- base64 encoding returns a byte array --->
<cfset saltByteArray = toBase64( salt, "utf-8" ) />
<cfset xmlpostByteArray = toBase64( xmlPost, "utf-8" ) />
<!--- append salt to xmlpost --->
<cfset xmlpostsaltByteArray = xmlpostByteArray & saltByteArray />
<!--- now let us perform a sha512 hash on this concatenated byte array --->
<cfscript>
// Create an instance of our DigestUtils class
digestUtils = createObject("java","org.apache.commons.codec.digest.DigestUtils");
// I hash a byte array using the given algorithm and return a
// 32-character Hexadecimal string. Home-made hash function for CF9 and earlier
function hashBytes( bytes, algorithm = "SHA-512" ){
// Get our instance of the digest algorithm that we'll use
// to hash the byte array.
var messageDigest = createObject( "java", "java.security.MessageDigest" ).getInstance( javaCast( "string", algorithm ) );
// Get the digest for the given byte array. This returns the
// digest (i.e., hash) in byte-array format.
var digest = messageDigest.digest( bytes );
// Now that we have our digested byte array (i.e., our hash as another byte
// array), we have to convert that into a HEX string. So, we'll need a HEX buffer.
var hexBuffer = [];
// Each integer in the byte digest needs to be converted into
// a HEX character (with possible leading zero).
for (byte =1 ;byte LTE ArrayLen(digest);byte = byte + 1) {
//for ( var byte in digest){
// Get the hex value for this byte. When converting the
// byte, only use the right-most 8 bits (last 8 bits of the integer)
// otherwise the sign of the byte can create oddities
var tail = bitAnd( 255, byte );
// Get the hex-encoding of the byte.
var hex = ucase( formatBaseN( tail, 16 ) );
// In order to make sure that all of the HEX characters
// are two-digits, we have to prepend a zero for any
// value that was originally LTE to 16 (the largest value
// that won't result in two HEX characters).
arrayAppend( hexBuffer, (tail <= 16 ? ("0" & hex) : hex) );
}
// Return the flattened character buffer.
return( arrayToList( hexBuffer, "" ) );
}
// Get the hash of the byte array using our hashBytes() function
hashByteArray = hashBytes( xmlpostsaltByteArray );
</cfscript>
<!--- The hashByteArray is in HEX format now. Convert to binary --->
<!--- You must binary decode the hashed string before converting it to binary --->
<cfset hashByteArray = toBase64( BinaryDecode( hashByteArray, 'HEX' ) ) />
<!--- The final step is to append this new hashbytearray with the salt byte array --->
<cfset hashByteArray = hashByteArray & saltByteArray />
<!--- now convert this value to a base64 encoded string --->
<cfset hashByteArray2 = toBase64( hashByteArray )/>
Here is what I get for my strXML variable:
Actual xml structure converted from base 64 to string:
<?xml version="1.0"?><EstoreSsoCustomer xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema"><CustomerId>112940</CustomerId><DealerCode>N88888</DealerCode><PointBalance>280</PointBalance><FirstName>Faith</FirstName><LastName>Hutula</LastName></EstoreSsoCustomer>
The final value, hasByteArray2 is not even remotely similar to payload_hash
This is my first time doing this and my understanding of hashing, byte arrays and character conversions flew out of the window decades ago.
What am I doing wrong?
Thank you
Faith Sloan
DigestUtils.sha512 was added in version 1.4. ColdFusion 9 uses an older version, 1.3. That is why the method is not found.
Use the other function based on MessageDigest. Just be sure to pass in the correct algorithm ie:
imageHash = hashBytes( imageBinary, "SHA-512" );
UPDATE: Based on the updated code, some of the instructions may be a bit misleading. I believe they just mean decode the xml and salt strings from their given encoding (base64 and utf-8) into byte arrays, not strings:
// note: salt value has invalid characters for base64
// assuming it is a plain utf-8 string
saltArray = charsetDecode(salt, "utf-8");
xmlByteArray = binaryDecode(xmlPost, "base64");
Then merge the two binary arrays (see custom function)
mergedBytes = mergeArrays( xmlByteArray, saltArray );
Calculate the hash of the new byte array:
messageDigest = createObject( "java", "java.security.MessageDigest" );
messageDigest = messageDigest.getInstance( javaCast( "string", "SHA-512") );
hashedByteArray = messageDigest.digest( javacast("byte[]", mergedBytes) );
Merge the arrays again:
mergedBytes = mergeArrays( hashedByteArray, saltArray);
Finally convert the binary to base64 and compare:
calculatedPayload = binaryEncode( javacast("byte[]", mergedBytes), "base64");
// check results
arePayloadsEqual = compare(calculatedPayload, payload_hash) eq 0;
WriteDump("arePayloadsEqual="& arePayloadsEqual);
WriteDump("calculatedPayload="& calculatedPayload);
WriteDump("payload_hash="& payload_hash);
Note: BinaryDecode/CharsetDecode return java arrays. Unlike CF arrays, they are immutable (ie cannot be changed). So the handy addAll(..) trick will not work here.
// merge immutable arrays the long way
function mergeArrays( array1, array2 ){
var i = 0;
var newArray = [];
for (i = 1; i <= arrayLen(arguments.array1); i++) {
arrayAppend(newArray, arguments.array1[i]);
}
for (i = 1; i <= arrayLen(arguments.array2); i++) {
arrayAppend(newArray, arguments.array2[i]);
}
return newArray;
}
I am working on a small task which requires some base64 encoding. I am trying to do it in head but getting lost .
I have a 13 digit number in java long format say: 1294705313608 , 1294705313594 , 1294705313573
I do some processing with it, bascially I take this number append it with stuff put it in a byte array and then convert it to base64 using:
String b64String = new sun.misc.BASE64Encoder().encodeBuffer(bArray);
Now , I know that for my original number, the first 3 digits would never change. So 129 is constant in above numbers. I want to find out how many chars corresponding to those digits would not change in the resultant base64 string.
Code to serialize long to the byte array. I ignore the first 2 bytes since they are always 0:
bArray[0] = (byte) (time >>> 40);
bArray[1] = (byte) (time >>> 32);
bArray[2] = (byte) (time >>> 24);
bArray[3] = (byte) (time >>> 16);
bArray[4] = (byte) (time >>> 8);
bArray[5] = (byte) (time >>> 0);
Thanks.
Notes:
I know that base64 would take 6 bits and make one character out of it. So if first 3 digits do not change in long how many chars would not change in base64.
This in NOT a HW assignment, but I am not very familiar with encoding.
1290000000000 is 10010110001011001111111011110010000000000 in binary.
1299999999999 is 10010111010101110000010011100011111111111 in binary.
Both are 41 bits long, and they differ after the first 7 bits. Your shift places bits 41-48 in the first byte, which will always be 00000001. The following byte will always be 00101110, 00101101, or 00101110. So you've got the leading 14 bits in common across all your possible array values, which (at 6 bits per encoded base64 char) means 2 characters in common in the encoded string.
Appears you're on the right track. I think what you want to do is convert a long to a byte array, then convert the byte array to Base64.
How do I convert Long to byte[] and back in java shows you how to convert it to bytes.