i got a problem in a project to uncompress a date.
(No documentation is available)
I have to convert a date, shown by this 6 Bytes:
0xFD 0x77 0x59 0x51 0x10 0x00
Did anyone know, how to uncompress ?
The date is from today, ~ 10:30 GMT
Programm language doesn´t matters.
(It is just a question of understanding. Not a question by programming)
Christian
added Again some examples
11:09 -->
0x fd 77 59 fd 10 00
11:09 -->
0x fd 77 79 05 28 00
11:05 -->
0x fd 77 59 fd 28 00
The solution was, to convert the Data byte array not via Java to Hex. In several cases (Byte is > 127) the result is a ..FD Hex Value. If i convert it otherwise, the Log Result is i.e.:
0x DD7719b33A00
dd7 -> 7dd -> 2013
7 -> 7 -> Month
19 -> 25 (dec.) --> Day
b -> 11 -> Hour
33 -> 51 -> Min.
A -> 10 -> seconds
0 -> 0 -> ms
Related
From a file that I'm trying to decode, I have the following data and his corresponding timestamps:
05/21/2022 12:30:00.000 PM
62 d9 d0 58 31 44 89 c4 00 00 00 00
8/24/2022 12:15:00.000 PM
62 fd 6f 58 31 83 27 42 00 00 00 00
First 4 bytes are close to the unix timestamp representation but just close
I think byte 31 is a separator but not sure
Please help! :)
If you take some of the bytes and multiply by 2, you can recreate the Unix timestamp.
For example,
05/21/2022 12:30:00 PM => 31 44 89 c4 => 826575300 x 2 => 1653150600 => 2022-05-21 16:30:00
8/24/2022 12:15:00 PM => 31 83 27 42 => 830678850 x 2 => 1661357700 => 2022-08-24 16:15:00
Note: data seems to be shifted 4 hours (Chile = GMT-4)
Did you tried the INT96 format?
that's an 12 byte representation
i have 3 parameters with date/time and status of phases of electricity meter:
28.11.2019 4:18 - all phases have power
28.11.2019 4:18 - all phases don't have power
28.11.2019 4:23 - all phases don't have power
But these data coded to hex like this:
FA 22 7C 27 07 FF E1
49 22 7C 27 07 08 F3
4E 22 7C 27 00 08 1D
And i don't know how it was did? I mean maybe there is some method to keep and transfer date/time or what?
34 [48 83 27] 07 C6 EA - 3.12.2019 09:01, </b>
38 [48 83 27] 00 E7 74 - 3.12.2019 09:01, </b>
89 [49 83 27] 07 10 38 - 3.12.2019 09:12, </b>
8E [49 83 27] 00 10 D6 - 3.12.2019 09:12, </b>
0E [4B 83 27] 07 21 EA - 3.12.2019 09:24, </b>
13 [4B 83 27] 00 42 74 - 3.12.2019 09:24,</b>
19 [4B 83 27] 07 63 2A - 3.12.2019 09:24,</b>
1D [4B 83 27] 00 63 A4 - 3.12.2019 09:24,</b>
21 [4B 83 27] 07 84 5A - 3.12.2019 09:25,</b>
25 [4B 83 27] 00 84 D4 - 3.12.2019 09:25,</b>
square brackets - i guess it's date. 4th byte is state of phases. I mean low halfbyte(or semibyte?) of 4th byte. 0x7(0b0111) - all phases have power, 0x0(0b0000) - al phases doesn't have power.
TL;DR: The first 4 bytes of each line is a little-endian count of time units since some epoch that is either January 1 or January 6, 1980.
Each timestamp is 4 bytes. When we look at the third last and the second last line, they both have 4B 83 27 within the square brackets that you have put, yet the time changes fro 9:24 to 9:25. The times are obviously truncated, there needs not be more than a second or two between them, but they are not the same. The solution I found was to include the byte before the square bracket in the timestamp.
Timestamps are little endian. The byte before the square bracket varies the fastest as time moves. The next byte varies a little. The remaining two bytes stay constant within the 24 minutes range you are giving. The obvious explanation is that 34 48 83 27 really means 0x27834834, that is, the bytes are reversed.
It took some experimentation, but two common epochs would fit:
January 1, 1980. In this case the time unit used for counting the time since the epoch is around 1 900 440 700 nanoseconds.
January 6 the same year. In this case the unit must be a little smaller, around 1 899 789 030 nanoseconds.
Both January 1 and January 6, 1980 are commonly used epochs. See the link at the bottom for documentation.
I have searched somewhat for an explanation for the size of the unit. Could it be some nice fraction of a second or of a day, for example? I haven’t found any reasonable explanation. I am probably missing something.
I have ignored the issue of time zone and UTC offset. The epoch is likely defined in UTC, and if the times given in the question are in your local time zone, for example, this adds a slight inaccuracy to my analysis.
In code
To convert from hex number to date and time we may use a simple method like the following in Java.
private static LocalDateTime epoch = LocalDateTime.of(1980, Month.JANUARY, 6, 0, 0);
private static int nanosPerUnit = 1_899_789_030;
private static LocalDateTime convert(int n) {
return epoch.plusNanos((long) n * (long) nanosPerUnit);
}
Demonstration:
int[] numbers = {
0x27834834,
0x27834838,
0x27834989,
0x2783498e,
0x27834b0e,
0x27834b13,
0x27834b19,
0x27834b1d,
0x27834b21,
0x27834b25
};
for (int n : numbers) {
System.out.format("%x %s%n", n, convert(n));
}
Output:
27834834 2019-12-03T09:01:20.396289720
27834838 2019-12-03T09:01:27.995445840
27834989 2019-12-03T09:12:08.224348950
2783498e 2019-12-03T09:12:17.723294100
27834b0e 2019-12-03T09:24:27.242281620
27834b13 2019-12-03T09:24:36.741226770
27834b19 2019-12-03T09:24:48.139960950
27834b1d 2019-12-03T09:24:55.739117070
27834b21 2019-12-03T09:25:03.338273190
27834b25 2019-12-03T09:25:10.937429310
The dates and times agree with those from the question. I leave it to you to insert January 1 and 1_900_440_700 nanoseconds and see that this too gives results that agree with the question.
Links
Endianness on Wikipedia
Epoch (computing) on Wikipedia
I have a database filled with a lot of logged IPV4 messages. It is used to get queries like: "give me all messages from MacAddress ... that were logged in the period ... to ... that have ..."
Some queries will result in a huge amount of logged messages. Therefore we decided to make a PCAP file if such a request was made.
"Please create a PCAP file containing all logged messages from your
database that ..."
So upon request, my service should fetch the requested data from the database (in pages) and create a PCAP file filled with the data fetched from the database. Later callers can ask for a read-only OWIN stream to this file
The service can create such a file. The problem is that it is not recognized as a proper WireShark file.
I've read Libcap File Format. Whenever I have to create a file filled with LoggedMessages I fill a binary file as follows.
Global Header
Per logged message:
A packet header
Packet data with:
Ethernet Frame: Destination Mac, Source Mac, EtherType (0x800)
IPV4 header
Logged Data
Wireshark starts complaining about the file when it attempts to read the Ethertype. It says this is a Length. Definition of Ethernet Frame with EtherType
So below I show the start of my file. Hexadecimal format per byte + my interpretation of it. After that the comments from wireshark
The created stream starts with the Global Header: a 32 bytes structure. First the hexadecimal values then the interpretation:
=== Global Header ====
D4 C3 B2 A1 02 00 04 00
00 00 00 00 00 00 00 00
FF FF 00 00 01 00 00 00
Magic number A1B2C3D4 (Original Time Precision)
Version: 2 - 4
ThisZone 0
sigFigs 0
snapLen 0000FFFF
datalinkType 1
Note that the magic number has the LSB first, indicating that every multi-byte number will have the least significant byte first. So a 2 byte value of 0x1234 will have in memory first 34 then 12.
After that the Packets should come. Every time one Packet Header, followed by one Packet Data
=== Packet header ===
09 89 58 5A C8 85 0B 00
6B 00 00 00 6B 00 00 00
Timestamp: 1515751689.7551446 (usec precision)
Number of saved bytes (incl_len) 107 bytes (0x006b)
Actual packet length (orig_len) 107 bytes (0x006b)
=== Packet Data ===
CF 31 59 D3 E7 98 53 39 - 17 F0 A9 9C 00 08 45 00
5D 00 00 00 00 00 FF 00 - E0 0D 8A 84 77 44 E0 2B
9C FB 4D 43 D5 8A 00 00 - 00 00 41 41 41 41 41 41
41 41 41 41 41 41 41 41 - 41 41 41 41 41 41 41 41
// etc, until total 107 bytes
The packet data consists of a Mac Header, IPV4 header and a couple of 0x41 as data
=== Mac Header ===
Destination Mac: CF:31:59:D3:E7:98
Source Mac: 53:39:17:F0:A9:9C
Ether type: 0800
Note that the magic number showed that every multi-byte number has the LSB first, so the two bytes 00 08 will have a 16-bit meaning of 0x0800
If you look at the PCAP file interpretation I show below, then the problem starts here: the Ether Type is not interpreted as Ether Type, but as length.
After remark in one of the answers, I tried to reverse the two byte ether type from 00 08 into 08 00 (MSB first), but that made the problems worse.
=== IPV4 header ===
- 45 00 5D 00
- 00 00 00 00
- FF 00 E0 0D
- 8A 84 77 44
- E0 2B 9C FB
Specification of the IPV4 header structure
DWORD 0
- bits 00..04: version; bits 04..07 IP Header Length: 04 05
- bits 08..13 DSCP; bits 14..15 ECN: 00
- bits 16..31 Total Length (header + Payload): 93 (005D)
DWORD 1
- bits 00..15 Identification: 0000
- bits 16..18 Flags; bits 19..31 offset: 0000
DWORD 2
- bits 00..07 Time to Live FF
- bits 08..15 Protocol; used protocol 00
- bits 16..31 Header Checksum 3552 (0DE0)
DWORD 3 and 4
Source IP: 138.132.119.68
Destination IP: 224.43.156.251
Bacause wireshark complains about checksum, I verify as follows:
Verify checksum:
Header: 0045 005D 0000 0000 00FF 0DE0 848A 4477 2BE0 FB9C
69 + 93 + 0 + 0 + 255 + 3552 + 33930 + 17527 + 11232 + 64412 = 131070 (01FFFE)
0001 + FFFE = FFFF
1's complement: 0000 (checksum ok)
This is what WireShark (version 2.4.4) makes of it:
The following seems normal:
Frame 1: 107 bytes on wire (856 bits), 107 bytes captured (856 bits)
Encapsulation type: Ethernet (1)
Arrival Time: Jan 12, 2018 11:08:09.755144000 W. Europe Standard Time
[Time shift for this packet: 0.000000000 seconds]
Epoch Time: 1515751689.755144000 seconds
[Time delta from previous captured frame: 0.000000000 seconds]
[Time delta from previous displayed frame: 0.000000000 seconds]
[Time since reference or first frame: 0.000000000 seconds]
Frame Number: 1
Frame Length: 107 bytes (856 bits)
Capture Length: 107 bytes (856 bits)
[Frame is marked: False]
[Frame is ignored: False]
[Protocols in frame: eth:llc:data]
[Coloring Rule Name: Checksum Errors]
[Coloring Rule String [truncated]: eth.fcs.status=="Bad" ||
ip.checksum.status=="Bad" || tcp.checksum.status=="Bad" ||
udp.checksum.status=="Bad" || sctp.checksum.status=="Bad" ||
mstp.checksum.status=="Bad" || cdp.checksum.status=="Bad" ||]
Here comes the first problem: EtherType is interpreted as Length
IEEE 802.3 Ethernet
Destination: cf:31:59:d3:e7:98 (cf:31:59:d3:e7:98)
Source: 53:39:17:f0:a9:9c (53:39:17:f0:a9:9c)
Length: 8
Padding: ff00e00d8a847744e02b9cfb4d43d58a0000000041414141...
Trailer: 414141414141414141414141414141414141414141414141...
Frame check sequence: 0x41414141 incorrect, should be 0xe19cae36
[FCS Status: Bad]
After the length, which I meant as an EtherType, comes a lot of padding, instead of interpretation of my 5 DWORDs.
The link to the Ethernet Frame in wikipedia I showed says:
The EtherType field is two octets long and it can be used for two
different purposes. Values of 1500 and below mean that it is used to
indicate the size of the payload in octets, while values of 1536 and
above indicate that it is used as an EtherType, to indicate which
protocol is encapsulated in the payload of the frame.
My value if 0x0800 = 2048. This certainly is above 1536
For example, an EtherType value of 0x0800 signals that the frame
contains an IPv4 datagram.
If value 0x0800 the incorrect value? Or is my error somewhere else?
Looks like your ethertype has the wrong byte order. It should be:
=== Packet Data ===
CF 31 59 D3 E7 98 53 39 - 17 F0 A9 9C 08 00 XX XX
I've been working on Midi file for some time and I stuck on some kind of status byte of thing. According to the standard Midi file format there is no such a things. So, Can someone tell what is this 3 bytes information "00 a040". I know that "00" is the byte stands for delta time and 0xa0 should be status byte, If only I understood it correctly. Last 3 bytes located at line 18 is the only part I don't understand so far. After those 3 bytes, then comes the text meta event bytes lead by "00 ff01".
Midi File Line 18th to 19th:
ff 51 03 09 cc 90 00 c0 00 00 b0 07 64 00 0a 40
00 ff 01 20 62 64 63 61 34 32 36 64 31 30 34 61
The SMF specification says:
Running status is used: status bytes of MIDI channel messages may be omitted if the preceding event is a MIDI channel message with the same status.
So these bytes can be decoded as follows:
ff 51 03 09 cc 90: meta event: set tempo, 9CC90h = 642192 µs per quarter note
00: delta time
c0 00: set program 0 (piano) on channel 0
00: delta time
b0 07 64: set controller 7 (volumn) to value 100
00: delta time
0a 40: running status (repeat B0h); set controller 10 (expression) to value 64
00: delta time
ff 01 20 ...: meta event: text: "bdca426d104a..."
Closed. This question needs details or clarity. It is not currently accepting answers.
Want to improve this question? Add details and clarify the problem by editing this post.
Closed 8 years ago.
Improve this question
I've boon looking up the Perl pack function and came across this tutorial.
All looked good, except when I got to the section about how the information was held in memory.
The tutorial does state that the first line is in decimal, second in hex, and the third as characters where applicable, and that pipe characters indicate field boundaries.
The data to be packed was derived from the following C structure:
struct SupplyRequest {
time_t request_time; // time request was entered
int employee_id; // employee making request
char item[32]; // item requested
short quantity; // quantity needed
short urgent; // request is urgent
};
with the input data (quoted exactly from the tutorial):
"If monk number 217641 (hey! that's me!) placed an urgent order for two boxes of paperclips on January 1, 2003 at 1pm EST, $rec would contain the following (first line in decimal, second in hex, third as characters where applicable). Pipe characters indicate field boundaries."
When packed, the resultant data would resemble the following:
Offset Contents (increasing addresses left to right)
0 160 44 19 62| 41 82 3 0| 98 111 120 101 115 32 111 102
A0 2C 13 3E| 29 52 03 00| 62 6f 78 65 73 20 6f 66
| b o x e s o f
16 32 112 97 112 101 114 99 108 105 112 115 0 0 0 0 0
20 70 61 70 65 72 63 6c 69 70 73 00 00 00 00 00
p a p e r c l i p s
32 0 0 0 0 0 0 0 0| 2 0| 1 0
00 00 00 00 00 00 00 00| 02 00| 01 00
I'd like some help trying to understand what's going on here, what the above fields represent; i.e. what is meant by the offset etc.
The string represents the representation of a instance of the following structure in memory.
struct SupplyRequest {
time_t request_time; // time request was entered
int employee_id; // employee making request
char item[32]; // item requested
short quantity; // quantity needed
short urgent; // request is urgent
};
"Fields" refers to the fields of the structure (request_time, employee_id, ...).
"Offset" refers to the offset (position) into the string at which the byte is located.