Extract data from a binary file - perl

Question

I have a binary file. I want extract all data from $marker + $step to $marker (or end of file).
Data example:
23 40 92 34 32 09 84 39 02 89 30 fe 90 38 01 02 03 f1 f2 00 00 00 22 33 44 56 77 22 aa bb cc dd ee ff 00 11 ff dd cc cc cc 22 80 ee 01 02 03 f1 f2 00 00 00 22 33 44 56 23 40 92 34 32 dd cc cc 22 33 44 22 33 44 01 02 03 f1 f2 00 00 00 22 33 44 56 77 22 FF FF FF 52 FF FF 52 00 00 00 00 00 00 00
It contains three blocks. I need:
1
00 00 00 22 33 44 56 77 22 aa bb cc dd ee ff 00 11 ff dd cc cc cc 22 80 ee
2
00 00 00 22 33 44 56 23 40 92 34 32 dd cc cc 22 33 44 22 33 44
3
00 00 00 22 33 44 56 77 22 FF FF FF 52 FF FF 52 00 00 00 00 00 00 00
I never worked with binary files with Perl.
$filename = $ARGV[0];
$marker = \x01\x02\x03\xf1\xf2;
$step = 3;
$count = 0;
open $file
while <$file> {
seek $marker;
Go to forward +$step bytes;
$count++
print EXTFILE_.$count.'.dat' $_
# Until do not seek new $marker or EOF
}
close file
As a result, I have to get three .dat files.
How can I realize this pseudocode? What would be some simple example?

Answer 1

Perl regular expressions are just as happy with binary data as with readable text, and binary files can be opened with a mode of raw to avoid translating line endings.
Here's a solution that reads the whole file into memory and scans it for the marker string.
use strict;
use warnings;
my $filename = shift;
my $binary = do {
open my $fh, '<:raw', $filename or die $!;
local $/;
<$fh>;
};
my $marker = "\x01\x02\x03\xf1\xf2";
while ( $binary =~ /$marker(.*?)(?=$marker|\z)/sg ) {
my #hex = map { sprintf '%02X', $_ } unpack 'C*', $1;
print "#hex\n";
}
Output
00 00 00 22 33 44 56 77 22 AA BB CC DD EE FF 00 11 FF DD CC CC CC 22 80 EE
00 00 00 22 33 44 56 23 40 92 34 32 DD CC CC 22 33 44 22 33 44
00 00 00 22 33 44 56 77 22 FF FF FF 52 FF FF 52 00 00 00 00 00 00 00
If the file is huge, or if you simply prefer the idea, you could set the input record separator to the marker string. Then a readline operation on the file would fetch up to and including the next occurrence of the marker pattern in the file. It means that each record is being read along with the marker from the beginning of the next record, but as it's going to be removed anyway it doesn't matter.
use strict;
use warnings;
my $filename = shift;
my $marker = "\x01\x02\x03\xf1\xf2";
open my $fh, '<:raw', $filename or die $!;
local $/ = $marker;
<$fh>; # Drop the data up to and including the first marker
while (<$fh>) {
chomp; # Remove the marker string from the end, if any
my #hex = map { sprintf '%02X', $_ } unpack 'C*';
print "#hex\n";
}
The output is identical to that of the program above.
Though that doesn't work for the required output of the program. This program uses the second technique above but writes to a series of EXTFILE.dat files instead of dumping the hex data. Note that an open mode of raw is necessary again.
use strict;
use warnings;
my $filename = shift // 'file.bin';
my $marker = "\x01\x02\x03\xf1\xf2";
open my $fh, '<:raw', $filename or die $!;
local $/ = $marker;
<$fh>; # Drop the data up to and including the first marker
my $count;
while (my $record = <$fh>) {
chomp $record;
my $outfile = sprintf 'EXTFILE_%d.dat', ++$count;
open my $out_fh, '>:raw', $outfile or die $!;
print $out_fh $record;
close $out_fh or die $!;
}