I'm trying to calculate the RIPEMD160 hash in matlab for some data represented by a hex string. I found the following java class and compiled it for jvm 1.6
http://developer.nokia.com/Community/Wiki/RIPEMD160_encryption_in_JavaME
the following code works perfectly in matlab for hashing strings:
clear all
% add folder with class file to java path
functions_folder = strcat(pwd,filesep,'functions');
javaaddpath(functions_folder)
% string to hash
string_to_hash = 'test12345';
% convert to java String
str_to_hash_java = javaObject('java.lang.String',uint8(string_to_hash));
% pass in string and convert output to char array
mystr = char(RIPEMD160.RIPEMD160String(str_to_hash_java))
Now my problem comes about when I try to hash some binary data represented by a hex string. The hash output is correct for hex values of 7f or smaller, but once I have 8 bits (>= 80) it no longer gives the correct answer. I can't seem to find the problem. Here is my code:
clear all
% add folder with class file to java path
functions_folder = strcat(pwd,filesep,'functions');
javaaddpath(functions_folder)
% data to hash in hex format
hex_string_in = '80';
hex_string_in_length = length(hex_string_in);
% split every to characters and calculate the data in each byte
for i=1:hex_string_in_length/2
data_uint8_array(1,i) = uint8(hex2dec(hex_string_in(2*i-1:2*i)));
end
% constructor
x = RIPEMD160;
% pass in binary data
x.update(data_uint8_array)
% get hash in binary format
hash_out_bin = x.digestBin();
% typecast binary data into unit8 primitive
hash_out_unit8=typecast(hash_out_bin,'uint8');
% convert to hex
hash_out_hex = dec2hex(hash_out_unit8)';
% pad with zeros if bytes all smaller than hex(80)
if(size(hash_out_hex,1))==1
hash_out_hex=[repmat('0',[1 size(hash_out_hex,2)]);hash_out_hex];
end
% final formatting, convert to lowercase
hash_out_hex = lower(hash_out_hex(:)')
for an input of '7f' it produces the correct hash of c8297aad716979548921b2e8e26ca8f20061dbef
but for '80' is gives e633ca40d977e24a1ffd56b7a992e99b48d13359 instead of the correct result b436441e6bb882fe0a0fa0320cb2d97d96b4d1bc
Thanks.
You are passing strings to your Java code instead of regular byte arrays. Not all bytes are representations of valid character encodings. Therefore you are likely to loose information. Only hash bytes, without any conversion. If strings are required use base 64 encoding/decoding.
str_to_hash_java should not be needed.
Related
How can I convert a message into a hash value using SHA/MD5 hashing in MATLAB? is there any builtin function or any fixed code?
There are no functions in matlab to calculate hashes. However, you can call Java (any OS) or .Net (Windows only) functions directly from matlab and either of these implement what you want.
Note that you haven't specified the encoding of the string. The hash is different if you consider the string in ASCII, UTF8, UTF16, etc.
Also note that matlab does not have 160-bit or 256-bit integer, so the hash can't obviously be a single integer.
Anyway, using .Net:
SHA256
string = 'some string';
sha256hasher = System.Security.Cryptography.SHA256Managed;
sha256 = uint8(sha256hasher.ComputeHash(uint8(string)));
dec2hex(sha256)
SHA1
sha1hasher = System.Security.Cryptography.SHA1Managed;
sha1= uint8(sha1hasher.ComputeHash(uint8(string)));
dec2hex(sha1)
Java based solution can be found in the following link
https://www.mathworks.com/matlabcentral/answers/45323-how-to-calculate-hash-sum-of-a-string-using-java
MATLAB's .NET classes appear to be a more recent creation than the JAVA hashing.
However, these classes don't have much/any public documentation available. After playing with it a bit, I found a way to specify one of several hash algorithms, as desired.
The "System.Security.Cryptography.HashAlgorithm" constructor accepts a hash algorithm name (string). Based on the string name you pass in, it returns different hasher classes (.SHA256Managed is only one type). See the example below for a complete string input ==> hash string output generation.
% Available options are 'SHA1', 'SHA256', 'SHA384', 'SHA512', 'MD5'
algorithm = 'SHA1';
% SHA1 category
hasher = System.Security.Cryptography.HashAlgorithm.Create('SHA1'); % DEFAULT
% SHA2 category
hasher = System.Security.Cryptography.HashAlgorithm.Create('SHA256');
hasher = System.Security.Cryptography.HashAlgorithm.Create('SHA384');
hasher = System.Security.Cryptography.HashAlgorithm.Create('SHA512');
% SHA3 category: Does not appear to be supported
% MD5 category
hasher = System.Security.Cryptography.HashAlgorithm.Create('MD5');
% GENERATING THE HASH:
str = 'Now is the time for all good men to come to the aid of their country';
hash_byte = hasher.ComputeHash( uint8(str) ); % System.Byte class
hash_uint8 = uint8( hash_byte ); % Array of uint8
hash_hex = dec2hex(hash_uint8); % Array of 2-char hex codes
% Generate the hex codes as 1 long series of characters
hashStr = str([]);
nBytes = length(hash_hex);
for k=1:nBytes
hashStr(end+1:end+2) = hash_hex(k,:);
end
fprintf(1, '\n\tThe %s hash is: "%s" [%d bytes]\n\n', algorithm, hashStr, nBytes);
% SIZE OF THE DIFFERENT HASHES:
% SHA1: 20 bytes = 20 hex codes = 40 char hash string
% SHA256: 32 bytes = 32 hex codes = 64 char hash string
% SHA384: 48 bytes = 48 hex codes = 96 char hash string
% SHA512: 64 bytes = 64 hex codes = 128 char hash string
% MD5: 16 bytes = 16 hex codes = 32 char hash string
REFERENCES:
1) https://en.wikipedia.org/wiki/SHA-1
2) https://defuse.ca/checksums.htm#checksums
I just used this and it works well.
Works on strings, files, different data types.
For a file I compared against CRC SHA through file explorer and got the same answer.
https://www.mathworks.com/matlabcentral/fileexchange/31272-datahash
I am generating 2500 values in Matlab in format (time,heart_rate, resp_rate) by using below code
numberOfSeconds = 2500;
time = 1:numberOfSeconds;
newTime = transpose(time);
number0 = size(newTime, 1)
% generating heart rates
heart_rate = 50 +(70-50) * rand (numberOfSeconds,1);
intHeartRate = int64(heart_rate);
number1 = size(intHeartRate, 1)
% hist(heart_rate)
% generating resp rates
resp_rate = 50 +(70-50) * rand (numberOfSeconds,1);
intRespRate = int64(resp_rate);
number2 = size(intRespRate, 1)
% hist(heart_rate)
% joining time and sensor data
joinedStream = strcat(num2str(newTime),{','},num2str(intHeartRate),{','},num2str(intRespRate))
dlmwrite('/Users/amar/Desktop/geenrated/rate.txt', joinedStream,'delimiter','');
The data shown in the console is alright, but when I save this data to a .txt file, it contains extra spaces in beginning. Hence I am not able to parse the .txt file to generate input stream. Please help
Replace the last two lines of your code with the following. No need to use strcat if you want a CSV output file.
dlmwrite('/Users/amar/Desktop/geenrated/rate.txt', [newTime intHeartRate intRespRate]);
๐โ๐ ๐ ๐๐๐ข๐ก๐๐๐ ๐ ๐ข๐๐๐๐ ๐ก๐๐ ๐๐ฆ ๐๐พ๐ ๐๐ ๐กโ๐ ๐ ๐๐๐๐๐๐ ๐ก ๐๐๐ ๐ฆ๐๐ข๐ ๐๐๐ ๐. ๐โ๐๐ ๐๐๐ ๐ค๐๐ ๐๐ฅ๐๐๐๐๐๐ ๐คโ๐ฆ ๐ฆ๐๐ข ๐๐๐ก ๐กโ๐ ๐ข๐๐๐ฅ๐๐๐๐ก๐๐ ๐๐ข๐ก๐๐ข๐ก.
The data written in the file is exactly what is shown in the console.
>> joinedStream(1) %The exact output will differ since 'rand' is used
ans =
cell
' 1,60,63'
num2str basically converts a matrix into a character array. Hence number of characters in its each row must be same. So for each column of the original matrix, the row with the maximum number of characters is set as a standard for all the rows with less characters and the deficiency is filled by spaces. Columns are separated by 2 spaces. Take a look at the following smaller example to understand:
>> num2str([44, 42314; 4, 1212421])
ans =
2ร11 char array
'44 42314'
' 4 1212421'
What is the difference between string and character class in MATLAB?
a = 'AX'; % This is a character.
b = string(a) % This is a string.
The documentation suggests:
There are two ways to represent text in MATLABยฎ. You can store text in character arrays. A typical use is to store short pieces of text as character vectors. And starting in Release 2016b, you can also store multiple pieces of text in string arrays. String arrays provide a set of functions for working with text as data.
This is how the two representations differ:
Element access. To represent char vectors of different length, one had to use cell arrays, e.g. ch = {'a', 'ab', 'abc'}. With strings, they can be created in actual arrays: str = [string('a'), string('ab'), string('abc')].
However, to index characters in a string array directly, the curly bracket notation has to be used:
str{3}(2) % == 'b'
Memory use. Chars use exactly two bytes per character. strings have overhead:
a = 'abc'
b = string('abc')
whos a b
returns
Name Size Bytes Class Attributes
a 1x3 6 char
b 1x1 132 string
The best place to start for understanding the difference is the documentation. The key difference, as stated there:
A character array is a sequence of characters, just as a numeric array is a sequence of numbers. A typical use is to store short pieces of text as character vectors, such as c = 'Hello World';.
A string array is a container for pieces of text. String arrays provide a set of functions for working with text as data. To convert text to string arrays, use the string function.
Here are a few more key points about their differences:
They are different classes (i.e. types): char versus string. As such they will have different sets of methods defined for each. Think about what sort of operations you want to do on your text, then choose the one that best supports those.
Since a string is a container class, be mindful of how its size differs from an equivalent character array representation. Using your example:
>> a = 'AX'; % This is a character.
>> b = string(a) % This is a string.
>> whos
Name Size Bytes Class Attributes
a 1x2 4 char
b 1x1 134 string
Notice that the string container lists its size as 1x1 (and takes up more bytes in memory) while the character array is, as its name implies, a 1x2 array of characters.
They can't always be used interchangeably, and you may need to convert between the two for certain operations. For example, string objects can't be used as dynamic field names for structure indexing:
>> s = struct('a', 1);
>> name = string('a');
>> s.(name)
Argument to dynamic structure reference must evaluate to a valid field name.
>> s.(char(name))
ans =
1
Strings do have a bit of overhead, but still increase by 2 bytes per character. After every 8 characters it increases the size of the variable. The red line is y=2x+127.
figure is created using:
v=[];N=100;
for ct = 1:N
s=char(randi([0 255],[1,ct]));
s=string(s);
a=whos('s');v(ct)=a.bytes;
end
figure(1);clf
plot(v)
xlabel('# characters')
ylabel('# bytes')
p=polyfit(1:N,v,1);
hold on
plot([0,N],[127,2*N+127],'r')
hold off
One important practical thing to note is, that strings and chars behave differently when interacting with square brackets. This can be especially confusing when coming from python. consider following example:
>>['asdf' '123']
ans =
'asdf123'
>> ["asdf" "123"]
ans =
1ร2 string array
"asdf" "123"
I have a .txt file that contains a data like this:
0000000011111000
0000001110001110
0000011000011111
0001110000000001
0011000000000001
0011000000000001
0110000000000001
0100000000000001
1100000000000001
1100000000000001
1000000000000001
1100000000000010
1100000000000110
0100000000001100
0110000000011000
0011111111110000
0
//repeats like this
The 0 at the end is a label that describes the 16x16 matrix of 0's and 1's. As you can see it is actually a binary image of 0.
I need to load this file as a 16x16 matrix. I have tried importdata, textscanand fscanf but none worked for me.
The file continues in this format.
My initial tought was to use '' as a delimiter for importdata, but that did not worked.
Is there a way to achieve this?
This is one way to read the file (see here for some documentation):
fid=fopen(textfile);
dat = textscan(fid,'%s',-1); % <-- read into cell array of strings
fclose(fid);
dat=char(dat); % <-- concatenate the strings into one char array
dat = double(dat)- '0'; % <-- convert to numeric 0/1 (48 = '0'+0)
The last row will contain the number represented ("0") and superfluous stuff, you can delete with e.g. dat(end,:)=[];
Happy trails!
Edit: Although the posted answer works with the input text file and input method I used, for the OP the code requires modification (probably due to a difference in input format):
i = 1 : length (dat{1,1})
result(i,:) = double(char(dat{1,1}{i,1})) - '0';
end
I have a matrix, say:
M = [1000 1350;2000 2040;3000 1400];
I wish to write this matrix onto a text file in the hex format, like this:
0x000003e8 0x00000bb8
0x000007d0 0x000007f8
0x00000bb8 0x00000578
I considered using the function dec2hex but it's very slow and inefficient. It also gives me the output as a string, which I don't know how to restructure for my above required format.
MATlab directly converts hex numbers to decimal when reading from a text file, ex. when using the function fscanf(fid,'%x').
Can we do the exact same thing while writing a matrix?
You can use the %x format string. For the sake of demonstration, see an example with sprintf below. If you want to write to a file, you will have to use fprintf.
M = [1000 1350;2000 2040;3000 1400];
str = sprintf('0x%08x\t0x%08x\n', M')
this results in
str =
0x000003e8 0x00000546
0x000007d0 0x000007f8
0x00000bb8 0x00000578
You may use num2str with a format string:
str = num2str(M, '0x%08x ');
which returns
str =
0x000003e8 0x00000546
0x000007d0 0x000007f8
0x00000bb8 0x00000578
Using this instead of sprintf you do not need to repeat the format string for each column.