Hex Encoding and Decoding - encoding

I have two modules in some 3rd party application(it does not have any documentation and I cannot reveal application name due to confidentiality). One module outputs only integers and other outputs only floating point numbers.
The module that outputs integers has very simple data format as it was HEX representation of numbers in reverse byte order. So, I am able to decode it successfully. But having issues in decoding HEX representation of floating point numbers.
The data below shows the data dump in HEX followed by the expected converted value. I have a little information about its representation that the last two bytes are some sort of CRC, so, it is like 8 byte number with two CRC bytes.
I have highlighted the 8 bytes that needs to be converted and their expected value is given below :
Dataset 1: 02 B5 E6 7B 15 C8 0C 00 0A F9 = 999359.533
Dataset 2: 7C 4C 3A 00 00 00 00 00 B7 4C = 0.001
Can anyone suggest something here, I have tried many encoding schemes including IEEE formats also. I do not have any other relevant information that I can share(I know it will be a hit and trial technique to solve this).

Not sure if this helps but:
02 B5 E6 7B 15 C8 0C 00 = 0x000CC8157BE6B502 = 3597694319113474
7C 4C 3A 00 00 00 00 00 = 0x00000000003A4C7C = 3820668
and
3597694319113474 / 3600000000 = 999359.5331
3820668 / 3600000000 = 0.001061297
So within a certain amount of rounding maybe they are fixed point numbers in fractions of 3600000000?
Can you get some more data points?

Related

How to decide if FF instruction is a call instruction or a jmp instruction in X86-64 ELF under linux? [duplicate]

This question already has answers here:
How to read the Intel Opcode notation
(3 answers)
How does one opcode byte decode to different instructions depending on the "register/opcode" field? What is that? [duplicate]
(3 answers)
Closed last month.
I have such ELF in linux and I disassemble it to do some research. However, I get confused "How to decide if FF instruction is a call instruction or a jmp instruction?
For example, the ff instruction occurs like
`
8405de: 48 8b 04 c5 00 81 98 mov 0x988100(,%rax,8),%rax
8405e5: 00
8405e6: ff e0 jmpq *%rax
8405e8: 48 8b 45 e8 mov -0x18(%rbp),%rax
and like
84167f: 48 8b 45 90 mov -0x70(%rbp),%rax
841683: ff d0 callq *%rax
841685: 48 89 45 f0 mov %rax,-0x10(%rbp)
841689: 48 8b 45 f0 mov -0x10(%rbp),%rax
`
I read Intel® 64 and IA-32 Architectures Software Developer’s Manual Volume.
CALL
FF /2 CALL r/m16 M N.E. Valid Call near, absolute indirect, address given in r/m16.
FF /2 CALL r/m32 M N.E. Valid Call near, absolute indirect, address given in r/m32.
FF /2 CALL r/m64 M Valid N.E. Call near, absolute indirect, address given in r/m64
and
JMP
FF /4 JMP r/m16 M N.S. Valid Jump near, absolute indirect, address = zeroextended r/m16. Not supported in 64-bit
mode.
FF /4 JMP r/m32 M N.S. Valid Jump near, absolute indirect, address given in
r/m32. Not supported in 64-bit mode.
FF /4 JMP r/m64 M Valid N.E. Jump near, absolute indirect, RIP = 64-Bit
offset from register or memory
FF /5 JMP m16:16 D Valid Valid Jump far, absolute indirect, address given in
m16:16
FF /5 JMP m16:32 D Valid Valid Jump far, absolute indirect, address given in
m16:32.
Since I'm a Rookie, I could tell difference even I read this Manual, could you give me some advice?

Is it safe to replace CP850 with UTF-8 encoding

I have an old project reading files with CP850 encoding. But it handles accent characters wrong (e.g., Montréal becomes MontrÚal).
I want to replace CP850 with UTF-8. The question is:
Is it safe? In other word, can we assume UTF-8 is a super set and Encoding the same way as CP850 encoding characters?
Thanks
I tried hexdump, below is the sample of my csv file, is it UTF-8?
000000d0 76 20 64 65 20 4d 61 72 6c 6f 77 65 2c 2c 4d 6f |v de Marlowe,,Mo|
000000e0 6e 74 72 c3 a9 61 6c 2c 51 43 2c 48 34 41 20 20 |ntr..al,QC,H4A |
If by superset you mean does UTF-8 include all the characters of CP850, then trivially yes, since UTF-8 can encode all valid Unicode code points using a variable-length encoding (1–4 bytes).
If you mean are characters encoded the same way, then as you've seen this is not the case, since é (U+00E9) is encoded as 82 in CP850 and C3 A9 in UTF-8.
I cannot see a character set / code page that encodes Ú as 82, but Ú is encoded as E9 in CP850, which is the ISO-8859-1 representation of é, so it's possible you've got your conversion the wrong way around (i.e. you're converting your file from ISO-8859-1 to CP850, and you want to convert from CP850 to UTF-8).
Here's an example using hd and iconv:
hd test.cp850.txt
00000000 4d 6f 6e 74 72 82 61 6c |Montr.al|
00000008
iconv --from cp850 --to utf8 test.cp850.txt > test.utf8.txt
hd test.utf8.txt
00000000 4d 6f 6e 74 72 c3 a9 61 6c |Montr..al|
00000009

Why do you wrap around in 16 bit checksum (hex used)?

I have the question:
Compute the 16-bit checksum for the data block E3 4F 23 96 44 27 99
F3. Then perform the verification calculation.
I can perform the addition and I get the overflow like:
E3 4F
23 96
44 27
99 F3
``````````
1 E4 FF (overflow)
The solution then takes the overflow and adds it causing E4 FF to become E5 00. Can someone explain to me why this occurs?

CRC-16 in MATLAB

I have a working program in LabVIEW that I want to port to MATLAB. It takes an input number, converts it to hex, appends it to a constant string (0110 0001 0002 0400 03), calculates a CRC-16, and sends it all to a COM port. Here are two examples for 1500 and 2000 respectively.
0110 0001 0002 0400 0305 DCC0 AA
0110 0001 0002 0400 0307 D0C1 CF
I can see that dec2hex(1500) produces the 5DC, and dec2hex(2000) produces the 7D0. The AA and the CF are produced by a CRC-16 LabVIEW program, which are 170 and 207 respectively. I understand these are some sort of check-sums, but I can't find a way to reproduce it in MATLAB.
Solution found via: FEX submission
A = ['01';'10';'00';'01';'00';'02';'04';'00';'03';'07';'D0']
dec2hex(append_crc(hex2dec(A)'))
Returns:
01
10
00
01
00
02
04
00
03
07
D0
C1
CF

Manual Wilcoxon Rank-Sum Test

My statistics professor wants us to perform a manual Wilcoxon Rank-Sum Test using Matlab. Unfortunately, I have no experience with Matlab whatsoever, and I have been discovering as I go along. In short, we are given a list of 24 paired observations:
33 53 54 84 69 34 60 34 50 56 64 50 76 47 58 63 55 66 58 43 28 80 45
55
66 62 54 58 60 74 54 68 64 60 53 59 61 49 63 55 61 64 54 59 64 46 70
82
I've gotten to the point where I have a matrix with the absolute differences in the first column, the sign of the difference (indicated by a 1 for positive and -1 for negative) in the second column and the rank of the difference (1 through 24) in the third column.
I am struggling with finding a quick and efficient way to "break the ties" between the differences of equal size and allocating the average rank to each of these differences. I expect that some loops and logical statements may be required, but I am having a hard time with them as I have no prior experience.
Any suggestions on how to do this would be much appreciated.
One way to average over the ranks for entries with matching differences is as follows:
irankavg=zeros(length(dp),1);
[dpu,ix,iclass]=unique(dp);
for ii=1:length(dpu)
irankavg(iclass(ii)==iclass) = mean(irank(iclass(ii)==iclass));
end
where dp is a column array that contains the differences