I need to work with a DBF file (dBASE III PLUSE, with memo) which contains weirdly formatted date values. The values are saved as HEX numbers counting up for each day.
Now comes the weird part, the counter jumps (compare e.g. integer value) every month one and every year two values. Basically, it counts a 00 month and a 00 day.
EDIT: Further it counts for every month 31 days, e.g. end of February.
EDIT 2: Another strange implementation I didn't see before is, that each byte/hex value count's only until 7F and then the next byte count's up e.g. 2001-02-15 -> 2001-02-16...
HEX Integer Date * not a valid date
====================================================
2E 22 30 3023408 1970-00-00 *
2E 22 31 3023409 1970-01-00 *
2E 22 32 3023410 1970-01-01
2E 22 33 3023411 1970-01-02
2E 22 34 3023412 1970-01-02
[...]
2E 22 50 3023440 1970-01-31
2E 22 51 3023441 1970-02-00 *
2E 22 52 3023442 1970-02-01
[...]
2E 22 6D 3023469 1970-02-28
2E 22 6E 3023470 1970-02-29 *
2E 22 6F 3023471 1970-02-30 *
2E 22 70 3023472 1970-02-31 *
2E 22 71 3023473 1970-03-00 *
2E 22 72 3023474 1970-03-01
[...]
2E 25 30 3024176 1970-12-31
2E 25 31 3024177 1971-00-00 *
2E 25 32 3024178 1971-01-00 *
2E 25 33 3024179 1971-01-01
[...]
2E 7F 7F 3047295 2001-02-15
2F 00 00 3080192 2001-02-16
[...]
With this knowledge, I could generate a list from 1900-01-01 until 2038-01-18, see example.txt.
Is this a known pattern, I just never heard about it?
Is there a simple way to convert date back and forward?
As #js2010 pointed out it's probably the 3 byte Date of last update, in YYMMDD format structure, see dbase.com/Knowledgebase/INT/db7_file_fmt.htm . Though there isn't really a further explanation.
I need some help to understand the avcC atom structure of a particular mp4 sample I am trying to analyze.
Hex dump:
00 00 00 38 61 76 63 43 01 64 00 1F FF E1 00 1C 67 64 00 1F AC D9 80
50 05 BB 01 6A 02 02 02 80 00 00 03 00 80 00 00 1E 07 8C 18 CD 01 00
05 68 E9 7B 2C 8B FD F8 F8 00 00 00 00 13 63 6F 6C 72
This is what I understand from the above:
00 00 00 38 Size of avcC atom
61 76 63 43 avcC signature
01 configurationVersion
64 AVCProfileIndication
00 profile_compatibility
1F AVCLevelIndication
FF 111111b + lengthSizeMinusOne
E1 111b + numOfSequenceParameterSets (in this case, 1 SPS)
00 1C SPS length (in this case, 28 bytes)
67 64 00 1F AC D9 80 50 05 BB 01 6A 02 02 02 80 00 00 03 00 80 00 00 1E 07 8C 18 CD SPS data (28 bytes as per above)
01 numOfPictureParameterSets (in this case, 1 PPS)
00 05 PPS length
This is where the problem begins. Based on the PPS length given by the previous bytes, the next 5 bytes should be the PPS data: 68 E9 7B 2C 8B
However according to the avcC header, the total length of the atom is 56 bytes (0x38), which means that the following 4 bytes should be included: FD F8 F8 00
But the problem is that the PPS length is given as 5 bytes (0x05). So what exactly are these final 4 bytes?
Then follows the header of the colr atom:
00 00 00 13 size of colr atom
63 6F 6C 72 colr signature
Which I have checked and is indeed 19 bytes in length (0x13).
The problem is with the avcC atom and with that particular mp4 sample I am analyzing (I've checked other samples too and they didn't have this peculiarity).
You can find the sample here.
EDIT
mp4info tool from the bento4 suite reports the following as the avcC atom's size: 8+48
And mp4dump reports:
AVC SPS: [6764001facd9805005bb016a02020280000003008000001e078c18cd]
AVC PPS: [68e97b2c8b]
So it correctly reports the total size of the atom as 56 bytes (0x38) based on what is found in the avcC header, but the SPS/PPS data are analyzed the same way as above. I still don't understand what the final 4 bytes are or where do they belong.
I dind't get any answer but fortunately a bit more careful reading of ISO 14496-15 solved this issue:
if( profile_idc == 100 || profile_idc == 110 ||
profile_idc == 122 || profile_idc == 144 )
{
bit(6) reserved = ‘111111’b;
unsigned int(2) chroma_format;
bit(5) reserved = ‘11111’b;
unsigned int(3) bit_depth_luma_minus8;
bit(5) reserved = ‘11111’b;
unsigned int(3) bit_depth_chroma_minus8;
unsigned int(8) numOfSequenceParameterSetExt;
for (i=0; i< numOfSequenceParameterSetExt; i++) {
unsigned int(16) sequenceParameterSetExtLength;
bit(8*sequenceParameterSetExtLength) sequenceParameterSetExtNALUnit;
}
}
Apparently a sequence of 4+ bytes may exist at the end of an avcC atom depending on the profile used. In my sample above the profile is 100 (0x64), hence it meets the criteria. So the last 4 bytes are:
FD = bits 111111 are reserved, remaining 01 means chroma subsampling 4:2:0
F8 = bits 11111 are reserved, remaining 000 means luma bit depth is 8
F8 = bits 11111 are reserved, remaining 000 means chroma bit depth is 8
00 = zero SPS extensions
I'm currently working with NFC/NDEF and I'm running into an issue where I'm unable to understand the data coming in. I have a general understanding of the NDEF standard and have looked over the MIFARE datasheet, so I'm able to pick out a few things, but there are a few bytes that are seemingly out of place and are puzzling me.
Here is the hexdump of the data on the tag, collected via nfc-mfultralight r:
00000000 04 02 2f a1 d2 11 5f 81 1d 48 00 00 e1 10 12 00 |../..._..H......|
00000010 01 03 a0 0c 34 03 1b 91 01 05 54 02 65 6e 68 69 |....4.....T.enhi|
00000020 11 01 05 54 02 65 6e 68 69 51 01 05 54 02 65 6e |...T.enhiQ..T.en|
00000030 68 69 fe 00 00 00 00 00 00 00 00 00 00 00 00 00 |hi..............|
I know the first 16 bytes (04 02 2f a1 d2 11 5f 81 1d 48 00 00 e1 10 12 00) are the NFC/MIFARE header (first 9 being the serial number/check bytes, 1 byte for internal, 2 for lock, and then final 4 are OTP bytes.)
Starting at byte 21 I can see the start of a TLV record with the Terminator TLV flag at the end (03 1b ... fe), indicating a record of NDEF type with length 27. This matches the length of the expected NDEF record.
However, I'm confused by bytes 16..20 (01 03 a0 0c 34). What are these?
It appears these are a part of the Lock Control TLV, a part of the NFC Type 2 Tag standard (pages 10-11).
The bytes are laid out as such:
0x01 - Lock Control TLV block name
0x03 - Length is 3 bytes
0xa0 - Encodes the position within the tag the lock area is at, composed of two nibbles:
0b0000 - Higher 4 bits represent the number of pages, while the lower 4 bits are the number of bytes
0b1100 - The number of bits used in the lock area.
0x0c - Indicates size in bits of the lock area
0x34 - Provides number of bytes in a page and the number of bytes each dynamic lock bit is able to lock.
I can read UDP packet using
void my_callback(u_char *useless, const struct pcap_pkthdr* pkthdr, const u_char* packet)
I have hexa output of my packet:
08 00 27 E5 B5 3B 52 54 00 12 35 02 08 00 45 00 00 4C 7C E7 00 00 40 11 3C 28 5B BD 59 C6 0A 00 02 0F 00 7B 00 7B 00 38 B7 9D 24 02 03 E8 00 00 04 A8 00 00 07 51 83 BC 03 DC DC C5 CC 47 F1 F1 69 C3 DC C5 CF 37 D2 5F A7 F5 DC C5 CF 38 3C 2D C2 CF DC C5 CF 38 3C 32 0B 9A
I know, that it is NTP packet.
How can I extrath data? Cut ethernet frames, etc..
Thank you for your help.
I am using pcap c++.
If you read pcap you get raw packet from the network device. Several options may be there:
Packet is read from ethernet device
Packet is read from vlan device
Packet is read from some other device
What kind of device is used during pcap defines what protocol header is first in your packet. To know it you can look at link layer type field of global pcap header.
Once you defined first protocol header you need to open protocol specification and find:
Size of header (in your case it looks like regular ethenet header - 14 bytes 08 00 27 E5 B5 3B 52 54 00 12 35 02 08 00)
How to find encapsulated packet type (in your case last 08 00 means IP)
Once you found IP header (45 00 00 4C 7C E7 00 00 40 11 ...) you can determine IP header length:
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|Version| IHL |Type of Service| Total Length |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Identification |Flags| Fragment Offset |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Time to Live | Protocol | Header Checksum |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Source Address |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Destination Address |
Here you need:
IHL defines sizeof IP header. This is lower 4 bits of first byte of IP header. In your case it is 0x5. This means 5 words or 20 bytes.
Protocol defines what data is encapsulated in IP header. In your case 0x11 (IPPROTO_UDP)
After that you can get UDP header (8 bytes) check ports if you need it and parse NTP header in according to NTP specification.
In your example total shift of the NTP header will be 14+20+8 bytes.
I had used the following command to generate a GTP packet using sendp
>>>sendp(Ether()/IP(dst="1.1.1.1", proto=17,
len=124)/UDP(sport=2152,dport=2152,len=104)/Raw(load=('32 ff 00 58 00 00 00 01 '
'28 db 00 00 45 00 00 54 00 00 40 00 40 00 5e a5 ca 0b 28 9e c0 a8 28 b2 08 00 '
'be e7 00 00 28 7b 04 11 20 4b f4 3d 0d 00 08 09 0a 0b 0c 0d 0e 0f 10 11 12 13 '
'14 15 16 17 18 19 1a 1b 1c 1d 1e 1f 20 21 22 23 24 25 26 27 28 29 2a 2b 2c 2d '
'2e 2f 30 31 32 33 34 35 36 37')), iface="eth1", loop=1, inter=1.0002)
In Wireshark,
Click on the following link for wireshark view :
http://imgur.com/M1Hpl7P
Expecting the Data of UDP packet as -
32 ff 00 58 00 00 00 01 28 db 00 00 45 00 00 54 00 00 40 00 40 00 5e a5 ca 0b 28
9e c0 a8 28 b2 08 00 be e7 00 00 28 7b 04 11 20 4b f4 3d 0d 00 08 09 0a 0b 0c 0d
0e 0f 10 11 12 13 14 15 16 17 18 19 1a 1b 1c 1d 1e 1f 20 21 22 23 24 25 26 27 28
29 2a 2b 2c 2d 2e 2f 30 31 32 33 34 35 36 37
But I found this data at "right most side of the packet description", instead of "Middle portion of the packet description".
Could you please let me know the command to be used to correct this.
I assume you want the above hex values to be the data in the UDP packet.
What you provided to Raw was a string of characters, spaces included, not hex code in Python. We will first convert your string of characters into a valid hex string in Python, then provide that to Scapy so the data will go on the wire as you want it.
I'll also show you some nice functions to preview what you will see in Wireshark.
First we'll put your data into its own variable.
>>> data = ('32 ff 00 58 00 00 00 01 '
... '28 db 00 00 45 00 00 54 00 00 40 00 40 00 5e a5 ca 0b 28 9e c0 a8 28 b2 08 00 '
... 'be e7 00 00 28 7b 04 11 20 4b f4 3d 0d 00 08 09 0a 0b 0c 0d 0e 0f 10 11 12 13 '
... '14 15 16 17 18 19 1a 1b 1c 1d 1e 1f 20 21 22 23 24 25 26 27 28 29 2a 2b 2c 2d '
... '2e 2f 30 31 32 33 34 35 36 37')
Then split that up into a nice list.
>>> data_list = data.split(" ")
>>> data_list
['32', 'ff', '00', '58', '00', '00', '00', '01', '28', 'db', '00', '00', '45',
'00', '00', '54', '00', '00', '40', '00', '40', '00', '5e', 'a5', 'ca', '0b',
'28', '9e', 'c0', 'a8', '28', 'b2', '08', '00', 'be', 'e7', '00', '00', '28',
'7b', '04', '11', '20', '4b', 'f4', '3d', '0d', '00', '08', '09', '0a', '0b',
'0c', '0d', '0e', '0f', '10', '11', '12', '13', '14', '15', '16', '17', '18',
'19', '1a', '1b', '1c', '1d', '1e', '1f', '20', '21', '22', '23', '24', '25',
'26', '27', '28', '29', '2a', '2b', '2c', '2d', '2e', '2f', '30', '31', '32',
'33', '34', '35', '36', '37']
Generate the string which can be passed to Raw as the binary data you want to appear in the packet.
>>>data_s = ''.join(data_list).decode('hex')
>>>data_s
'2\xff\x00X\x00\x00\x00\x01(\xdb\x00\x00E\x00\x00T\x00\x00#\x00#\x00^\xa5\xca
\x0b(\x9e\xc0\xa8(\xb2\x08\x00\xbe\xe7\x00\x00({\x04\x11K\xf4=\r\x00\x08\t\n\x0b
\x0c\r\x0e\x0f\x10\x11\x12\x13\x14\x15\x16\x17\x18\x19\x1a\x1b\x1c\x1d\x1e\x1f
!"#$%&\'()*+,-./01234567'
Use this string for your UDP Payload and build your packet!
>>> packet = IP(dst="1.1.1.1", proto=17,
... len=124)/UDP(sport=2152,dport=2152,len=104)/Raw(load=data_s)
>>> packet.show()
###[ IP ]###
version= 4
ihl= None
tos= 0x0
len= 124
id= 1
flags=
frag= 0
ttl= 64
proto= udp
chksum= None
src= 0.0.0.0
dst= 1.1.1.1
\options\
###[ UDP ]###
sport= gtp_user
dport= gtp_user
len= 104
chksum= None
###[ Raw ]###
load= '2\xff\x00X\x00\x00\x00\x01(\xdb\x00\x00E\x00\x00T\x00\x00#\x00#
\x00^\xa5\xca\x0b(\x9e\xc0\xa8(\xb2\x08\x00\xbe\xe7\x00\x00({\x04
\x11K\xf4=\r\x00\x08\t\n\x0b\x0c\r\x0e\x0f\x10\x11\x12\x13\x14
\x15\x16\x17\x18\x19\x1a\x1b\x1c\x1d\x1e\x1f !"#$%&\'()*+,-./0
1234567'
This should produce the expected data in Wireshark. Here is a hexdump of what you should see. I excluded the Ethernet layer, was giving Scapy on my machine some trouble for an unknown reason.
>>> hexdump(p)
0000 45 00 00 7C 00 01 00 00 40 11 78 6F 00 00 00 00 E..|....#.xo....
0010 01 01 01 01 08 68 08 68 00 68 90 1A 32 FF 00 58 .....h.h.h..2..X
0020 00 00 00 01 28 DB 00 00 45 00 00 54 00 00 40 00 ....(...E..T..#.
0030 40 00 5E A5 CA 0B 28 9E C0 A8 28 B2 08 00 BE E7 #.^...(...(.....
0040 00 00 28 7B 04 11 20 4B F4 3D 0D 00 08 09 0A 0B ..({.. K.=......
0050 0C 0D 0E 0F 10 11 12 13 14 15 16 17 18 19 1A 1B ................
0060 1C 1D 1E 1F 20 21 22 23 24 25 26 27 28 29 2A 2B .... !"#$%&'()*+
0070 2C 2D 2E 2F 30 31 32 33 34 35 36 37 ,-./01234567