I need to work with a DBF file (dBASE III PLUSE, with memo) which contains weirdly formatted date values. The values are saved as HEX numbers counting up for each day.
Now comes the weird part, the counter jumps (compare e.g. integer value) every month one and every year two values. Basically, it counts a 00 month and a 00 day.
EDIT: Further it counts for every month 31 days, e.g. end of February.
EDIT 2: Another strange implementation I didn't see before is, that each byte/hex value count's only until 7F and then the next byte count's up e.g. 2001-02-15 -> 2001-02-16...
HEX Integer Date * not a valid date
====================================================
2E 22 30 3023408 1970-00-00 *
2E 22 31 3023409 1970-01-00 *
2E 22 32 3023410 1970-01-01
2E 22 33 3023411 1970-01-02
2E 22 34 3023412 1970-01-02
[...]
2E 22 50 3023440 1970-01-31
2E 22 51 3023441 1970-02-00 *
2E 22 52 3023442 1970-02-01
[...]
2E 22 6D 3023469 1970-02-28
2E 22 6E 3023470 1970-02-29 *
2E 22 6F 3023471 1970-02-30 *
2E 22 70 3023472 1970-02-31 *
2E 22 71 3023473 1970-03-00 *
2E 22 72 3023474 1970-03-01
[...]
2E 25 30 3024176 1970-12-31
2E 25 31 3024177 1971-00-00 *
2E 25 32 3024178 1971-01-00 *
2E 25 33 3024179 1971-01-01
[...]
2E 7F 7F 3047295 2001-02-15
2F 00 00 3080192 2001-02-16
[...]
With this knowledge, I could generate a list from 1900-01-01 until 2038-01-18, see example.txt.
Is this a known pattern, I just never heard about it?
Is there a simple way to convert date back and forward?
As #js2010 pointed out it's probably the 3 byte Date of last update, in YYMMDD format structure, see dbase.com/Knowledgebase/INT/db7_file_fmt.htm . Though there isn't really a further explanation.
Related
From a file that I'm trying to decode, I have the following data and his corresponding timestamps:
05/21/2022 12:30:00.000 PM
62 d9 d0 58 31 44 89 c4 00 00 00 00
8/24/2022 12:15:00.000 PM
62 fd 6f 58 31 83 27 42 00 00 00 00
First 4 bytes are close to the unix timestamp representation but just close
I think byte 31 is a separator but not sure
Please help! :)
If you take some of the bytes and multiply by 2, you can recreate the Unix timestamp.
For example,
05/21/2022 12:30:00 PM => 31 44 89 c4 => 826575300 x 2 => 1653150600 => 2022-05-21 16:30:00
8/24/2022 12:15:00 PM => 31 83 27 42 => 830678850 x 2 => 1661357700 => 2022-08-24 16:15:00
Note: data seems to be shifted 4 hours (Chile = GMT-4)
Did you tried the INT96 format?
that's an 12 byte representation
Hi I'm trying to use power shell command Format-Hex and it works well .
In normal hex editor there is option to hide and not include the decoded text characters on the right .
I couldnt figure out how to use Format-Hex in power shell to only show :
00000030 1A 00 00 00 00 00 00 00 68 74 74 70 3A 2F 2F 66
Is there a parameter to do that with PS C:\Users\YourName\Desktop>Format-Hex file.dll instead of the below ? I am just confirming if any such parameter or cli tool that exists to allow when using Format-Hex command not looking for truncation or file parsing solutions .
00000030 1A 00 00 00 00 00 00 00 68 74 74 70 3A 2F 2F 66 ........http://f
00000040 72 61 6E 6B 69 65 2E 70 68 6F 74 6F 00 00 FF ED rankie.photo...í
00000050 00 C2 50 68 6F 74 6F 73 68 6F 70 20 33 2E 30 00 .APhotoshop 3.0.
00000060 38 42 49 4D 04 04 00 00 00 00 00 A6 1C 02 28 00 8BIM.......▌..(.
00000070 62 46 42 4D 44 30 31 30 30 30 61 66 63 30 64 30 bFBMD01000afc0d0
I read the help on Format-Hex
Format-Hex has no parameter that would omit the last column from its output.
If you're running on macOS or Linux, you can use the standard utilities available there, such as hexdump.
However, at the expense of performance, you can write a simple wrapper function around Format-Hex:
function Format-HexBare {
Format-Hex #args |
Out-String -Stream |
ForEach-Object `
-Begin { $maxLen = if ($IsCoreCLR) { 64 } else { 58 } } `
-Process {
$line = $_
try { $line.Substring(0, $maxLen) } catch { $line }
}
}
Note: The - PowerShell edition-specific - max. line length excluding the last column is hard-coded above, for the following reasons:
A robust regex solution would be challenging.
The .Substring() solution also eliminates the header of the unwanted column (which is only displayed in PowerShell (Core) 7+).
While there is no guarantee that output formats won't change over time,[1] it is unlikely to in this case. Should it happen nonetheless, the function is easy to adapt.
[1] The specifics of the for-display formatting - meant for the human observer only - aren't part of PowerShell's public contract with respect to backward compatibility.
I need some help to understand the avcC atom structure of a particular mp4 sample I am trying to analyze.
Hex dump:
00 00 00 38 61 76 63 43 01 64 00 1F FF E1 00 1C 67 64 00 1F AC D9 80
50 05 BB 01 6A 02 02 02 80 00 00 03 00 80 00 00 1E 07 8C 18 CD 01 00
05 68 E9 7B 2C 8B FD F8 F8 00 00 00 00 13 63 6F 6C 72
This is what I understand from the above:
00 00 00 38 Size of avcC atom
61 76 63 43 avcC signature
01 configurationVersion
64 AVCProfileIndication
00 profile_compatibility
1F AVCLevelIndication
FF 111111b + lengthSizeMinusOne
E1 111b + numOfSequenceParameterSets (in this case, 1 SPS)
00 1C SPS length (in this case, 28 bytes)
67 64 00 1F AC D9 80 50 05 BB 01 6A 02 02 02 80 00 00 03 00 80 00 00 1E 07 8C 18 CD SPS data (28 bytes as per above)
01 numOfPictureParameterSets (in this case, 1 PPS)
00 05 PPS length
This is where the problem begins. Based on the PPS length given by the previous bytes, the next 5 bytes should be the PPS data: 68 E9 7B 2C 8B
However according to the avcC header, the total length of the atom is 56 bytes (0x38), which means that the following 4 bytes should be included: FD F8 F8 00
But the problem is that the PPS length is given as 5 bytes (0x05). So what exactly are these final 4 bytes?
Then follows the header of the colr atom:
00 00 00 13 size of colr atom
63 6F 6C 72 colr signature
Which I have checked and is indeed 19 bytes in length (0x13).
The problem is with the avcC atom and with that particular mp4 sample I am analyzing (I've checked other samples too and they didn't have this peculiarity).
You can find the sample here.
EDIT
mp4info tool from the bento4 suite reports the following as the avcC atom's size: 8+48
And mp4dump reports:
AVC SPS: [6764001facd9805005bb016a02020280000003008000001e078c18cd]
AVC PPS: [68e97b2c8b]
So it correctly reports the total size of the atom as 56 bytes (0x38) based on what is found in the avcC header, but the SPS/PPS data are analyzed the same way as above. I still don't understand what the final 4 bytes are or where do they belong.
I dind't get any answer but fortunately a bit more careful reading of ISO 14496-15 solved this issue:
if( profile_idc == 100 || profile_idc == 110 ||
profile_idc == 122 || profile_idc == 144 )
{
bit(6) reserved = ‘111111’b;
unsigned int(2) chroma_format;
bit(5) reserved = ‘11111’b;
unsigned int(3) bit_depth_luma_minus8;
bit(5) reserved = ‘11111’b;
unsigned int(3) bit_depth_chroma_minus8;
unsigned int(8) numOfSequenceParameterSetExt;
for (i=0; i< numOfSequenceParameterSetExt; i++) {
unsigned int(16) sequenceParameterSetExtLength;
bit(8*sequenceParameterSetExtLength) sequenceParameterSetExtNALUnit;
}
}
Apparently a sequence of 4+ bytes may exist at the end of an avcC atom depending on the profile used. In my sample above the profile is 100 (0x64), hence it meets the criteria. So the last 4 bytes are:
FD = bits 111111 are reserved, remaining 01 means chroma subsampling 4:2:0
F8 = bits 11111 are reserved, remaining 000 means luma bit depth is 8
F8 = bits 11111 are reserved, remaining 000 means chroma bit depth is 8
00 = zero SPS extensions
This question already has answers here:
PowerShell hex to string conversion
(7 answers)
Closed 6 years ago.
I have a series of hex values which looks like this:
68 65 6c 6c 6f 57 6f 72 6c 64 7c 31 2f 30 38 31 35 7c 41 42 43 2d 31 35 02 08
I now need to convert this hex value to ASCII so that the result looks like:
helloWorld|1/0815|ABC-15
I tried so many things but I never came to the final code. I tried to use the convert-function in every imaginable way without any success.
At the moment I use this website to convert, but I need to do this in my PowerShell script.
Much like Phil P.'s approach, but using the -split and -join operators instead (also, integers not needed, ASCII chars will fit into a [byte]):
$hexString = "68 65 6c 6c 6f 57 6f 72 6c 64 7c 31 2f 30 38 31 35 7c 41 42 43 2d 31 35 02 08"
$asciiChars = $hexString -split ' ' |ForEach-Object {[char][byte]"0x$_"}
$asciiString = $asciiChars -join ''
$asciiString
Well, we can do something terrible and treat the HEX as a string... And then convert it to int16, which then can be converted to a char.
$hexString = "68 65 6c 6c 6f 57 6f 72 6c 64 7c 31 2f 30 38 31 35 7c 41 42 43 2d 31 35 02 08"
We have the string, now we can use the spaces to split and get each value separately. These values can be converted to an int16, which is a ascii code representation for the character
$hexString.Split(" ") | forEach {[char]([convert]::toint16($_,16))}
The only problem is that it returns an array of single characters. Which we can iterate through and concatenate into a string
$hexString.Split(" ") | forEach {[char]([convert]::toint16($_,16))} | forEach {$result = $result + $_}
$result
OS Ubuntu 12.04
Shell bash
Why does this sed command
sed -e 's/$/,/g' test.in
replace the 5 in test.in?
Here are the contents of test.in
52147398,9480,12/31/2011 23:22,101049000,LNAM,FNAM,80512725,43,0,75,1/1/2012 6:45,101049000
Here are the results of running sed
,2147398,9480,12/31/2011 23:22,101049000,LNAM,FNAM,80512725,43,0,75,1/1/2012 6:45,101049000
I want to put a comma after 1010489000
After replacing the date-time format so that it's in yyyy-mm-dd hh:mm:ss format, now
s/$/,/g
works as expected. Is this because sed got hung up on the mm/dd/yyyy hh:mm format?
Probably your file is in DOS format. The carriage return just before the newline would cause the comma to be written as the first character on the line overwriting an existing character (in this case the 5).
Convert to Unix format first:
tr -d '\r' < file > newfile
$ cat test.txt | sed 's/$/,/g'
52147398,9480,12/31/2011 23:22,101049000,LNAM,FNAM,80512725,43,0,75,1/1/2012 6:45,101049000,
Maybe the file is corrupt? Try copy/pasting into a new file...
Also try,
$ hexdump test.txt
hexdump test.txt
0000000 35 32 31 34 37 33 39 38 2c 39 34 38 30 2c 31 32
0000010 2f 33 31 2f 32 30 31 31 20 32 33 3a 32 32 2c 31
0000020 30 31 30 34 39 30 30 30 2c 4c 4e 41 4d 2c 46 4e
0000030 41 4d 2c 38 30 35 31 32 37 32 35 2c 34 33 2c 30
0000040 2c 37 35 2c 31 2f 31 2f 32 30 31 32 20 36 3a 34
0000050 35 2c 31 30 31 30 34 39 30 30 30 0a
000005c
The first character should be 0x35.