I am reading a file, containing integers using the "33441122" byte order. How can I convert the file to the "11223344" (big endian) byte order? I have tried a few things, but I am really lost.
I have read a lot about Perl, but when it comes to swapping bytes, I'm in the dark. How can I convert this:
33 44 11 22
into this:
11 22 33 44
using Perl.
Any input would be greatly appreciated :)
You can read 4 bytes at a time, split it into individual bytes, swap them and write them out again
#! /usr/bin/perl
use strict;
use warnings;
open(my $fin, '<', $ARGV[0]) or die "Cannot open $ARGV[0]: $!";
binmode($fin);
open(my $fout, '>', $ARGV[1]) or die "Cannot create $ARGV[1]: $!";
binmode($fout);
my $hexin;
my $n;
while (($n = read($fin, $bytes_in, 4)) == 4) {
my #c = split('', $bytes_in);
my $bytes_out = join('', $c[2], $c[3], $c[0], $c[1]);
print $fout $bytes_out;
}
if ($n > 0) {
print $fout $bytes_in;
}
close($fout);
close($fin);
This will be called on the command line as
perl script.pl infile.bin outfile.bin
outfile.bin will be overwritten.
I think the best way is to read two bytes at a time and dwap them before outputting them.
This program creates a data file test.bin, and then reads it in, swapping the bytes as described.
use strict;
use warnings;
use autodie;
open my $fh, '>:raw', 'test.bin';
print $fh "\x34\x12\x78\x56";
open my $out, '>:raw', 'new.bin';
open $fh, '<:raw', 'test.bin';
while (my $n = read $fh, my $buff, 2) {
$buff = reverse $buff if $n == 2;
print $out $buff;
}
Related
I have a file tmp.txt which contains only an integer 0.
I need to read this file and increment it by 1.
open (my $tmp, '+<', $tmp_file) or die "failed to open";
flock($tmp, LOCK_EX);
$t_int = int(<$tmp>);
$t_count = $t_int + 1;
print "***********t_count in file $t_int , $t_count**********\n";
print $tmp $t_count;
print "===========t_count in file $t_int , $t_count==========\n";
close($tmp) or die "failed to close";
I get the output as follows:
Runnning script for the first time:
***********t_count in file 0 , 1**********
===========t_count in file 0 , 1==========
Second time:
***********t_count in file 1 , 2**********
===========t_count in file 1 , 2==========
Third time:
***********t_count in file 12 , 13**********
===========t_count in file 12 , 13==========
Fourth time:
***********t_count in file 1213 , 1214**********
===========t_count in file 1213 , 1214==========
Fifth time:
***********t_count in file 12131214 , 12131215**********
===========t_count in file 12131214 , 12131215==========
and so on...
I don't understand why it is concatenating the previous numbers to the results. Why is this script not correct?
seek() allows you to move position in the filehandle.
seek($fh, 0, SEEK_SET) means move to the position zero bytes away from the start of the file. You have to import the constant from Fcntl.
#!/usr/bin/perl
use strict;
use warnings;
use Fcntl qw/ LOCK_EX SEEK_SET /;
my $tmp_file = 'test.dat';
open (my $tmp, '+<', $tmp_file) or die $!;
flock($tmp, LOCK_EX);
my $t_int = (int(<$tmp>));
my $t_count = $t_int + 1;
seek ($tmp, 0, SEEK_SET);
print $tmp $t_count;
close $tmp or die $!;
__END__
You need to use seek to go back to the start of the file before you print to the file. Try adding this before you print:
seek($tmp,0,0);
This is much more convenient using the core module Tie::File
Here's an example of how it might work
use Tie::File;
use Fcntl 'LOCK_EX';
my $tmp_file = 'inc.txt';
{
my $tied = tie my #file, 'Tie::File', $tmp_file or die $!;
$tied->flock(LOCK_EX);
++$file[0];
}
Update
Because of the flock I assume this is to be run in parallel using something like Parallel::ForkManager. This program shows how to increment the number in the file 1,000 times using thirty parallel processes
use strict;
use warnings;
use Tie::File;
use Fcntl 'LOCK_EX';
use Parallel::ForkManager;
my $tmp_file = 'inc.txt';
my $pm = Parallel::ForkManager->new(30);
for (1 .. 1000) {
next if $pm->start;
{
my $tied = tie my #file, 'Tie::File', $tmp_file or die $!;
$tied->flock(LOCK_EX);
++$file[0];
}
$pm->finish;
}
$pm->wait_all_children;
I have a file perl_script_2_out_2.txt and I want to put all strings separated by space in an array #arr.
I wrote this code but isnt working .
open my $FILE4, '<', 'perl_script_2_out_2.txt' or die $!;
my #array4 = <FILE4>;
close($FILE4);
open my $FILE5, '>', 'perl_script_2_out_2.txt' or die $!;
foreach $_ (#array4) {
s/\s+/\n/g;
print $FILE5 "$_";
}
close($FILE5);
open my $FILE6, '<', 'perl_script_2_out_2.txt' or die $!;
#arr = <$FILE6>;
You must always use strict and use warnings at the top of every Perl program that you write. In this case you would have seen the message
Name "main::FILE4" used only once: possible typo
which points to the statement
my #array4 = <FILE4>
and helps you to see that you have opened the file handle $FILE4 but tried to read from FILE4, which is different.
If you fix that then your code will work, but it's a strange way to do things and it's much better like this. I have used Data::Dump to display the final contents of the array; it's not necessary for the program to work.
use strict;
use warnings;
open my $fh, '<', 'perl_script_2_out_2.txt' or die $!;
my #arr;
while (<$fh>) {
push #arr, split;
}
use Data::Dump;
dd \#arr;
output
[
"uart_1_baddress",
2211,
"uart_2_baddress",
3344,
"uart_3_baddress",
2572,
]
I have a binary file that contain 3 files, a PNG, a PHP and a TGA file.
Here the file to give you the idea : container.bin
the file is build this way:
first 6 bytes are a pointer to the index, in this case 211794
Then you have all 3 files stacked one after the other
and at the ofset 211794, you have the index, that tell you where the file start and end
in this example you have:
[offset start] [offset end] [random data] [offset start] [name]
6 15149 asdf 6 Capture.PNG
15149 15168 4584 15149 index.php
15168 211794 12 15168 untilted.tga
meaning that capture.png start at offset 6, finish at offset 15149, then asdf is a random data, and the start offset is repeated again.
Now what I want to do is a perl to separate the file on this binary files.
The perl need to check the first 6 offset of the file (header), then jump to the index location, and use the list to extract the file out.
A mix of seek and read can be used to achieve the task:
#!/usr/bin/env perl
use strict;
use warnings;
use Fcntl 'SEEK_SET';
sub get_files_info {
my ( $fh, $offset ) = #_;
my %file;
while (<$fh>) {
chomp;
my $split_count = my ( $offset_start, $offset_end, $random_data, $offset_start_copy,
$file_name ) = split /\s/;
next if $split_count != 5;
if ( $offset_start != $offset_start_copy ) {
warn "Start of offset mismatch: $file_name\n";
next;
}
$file{$file_name} = {
'offset_start' => $offset_start,
'offset_end' => $offset_end,
'random_data' => $random_data,
};
}
return %file;
}
sub write_file {
my ( $fh, $file_name, $file_info ) = #_;
seek $fh, $file_info->{'offset_start'}, SEEK_SET;
read $fh, my $contents,
$file_info->{'offset_end'} - $file_info->{'offset_start'};
open my $fh_out, '>', $file_name or die 'Error opening file: $!';
binmode $fh_out;
print $fh_out $contents;
print "Wrote file: $file_name\n";
}
open my $fh, '<', 'container.bin' or die "Error opening file: $!";
binmode $fh;
read $fh, my $offset, 6;
seek $fh, $offset, SEEK_SET;
my %file = get_files_info $fh, $offset;
for my $file_name ( keys %file ) {
write_file $fh, $file_name, $file{$file_name};
}
The only real difficulty here is to make sure that both input and output files are read in binary mode. This can be achieved by using the :raw PerlIO layer when the files are opened.
This program seems to do what you want. It first locates and reads the index block into a string, and then opens that string for input and reads the start and end position and name of each of the constituent files. Thereafter processing each file is simple.
Be aware that unless the formatting of the index block is more strict than you say, you can rely only on the first, second, and last whitespace-separated fields on each line since random text could contain spaces. There is also no way to specify a file name containing spaces.
The output, using Data::Dump, is there to demonstrate correct functionality and is not necessary for the functioning of the program.
use v5.10;
use warnings;
use Fcntl ':seek';
use autodie qw/ open read seek close /;
open my $fh, '<:raw', 'container.bin';
read $fh, my $index_loc, 6;
seek $fh, $index_loc, SEEK_SET;
read $fh, my ($index), 1024;
my %contents;
open my $idx, '<', \$index;
while (<$idx>) {
my #fields = split;
next unless #fields;
$contents{$fields[-1]} = [ $fields[0], $fields[1] ];
}
use Data::Dump;
dd \%contents;
for my $file (keys %contents) {
my ($start, $end) = #{ $contents{$file} };
my $size = $end - $start;
seek $fh, $start, SEEK_SET;
my $nbytes = read $fh, my ($data), $size;
die "Premature EOF" unless $nbytes == $size;
open my $out, '>:raw', $file;
print { $out } $data;
close $out;
}
output
{
"Capture.PNG" => [6, 15149],
"index.php" => [15149, 15168],
"untilted.tga" => [15168, 211794],
}
this
is just
an example.
Lets assume the above is out.txt. I want to read out.txt and write onto the same file.
<Hi >
<this>
<is just>
<an example.>
Modified out.txt.
I want to add tags in the beginning and end of some lines.
As I will be reading the file several times I cannot keep writing it onto a different file each time.
EDIT 1
I tried using "+<" but its giving an output like this :
Hi
this
is just
an example.
<Hi >
<this>
<is just>
<an example.>
**out.txt**
EDIT 2
Code for reference :
open(my $fh, "+<", "out.txt");# or die "cannot open < C:\Users\daanishs\workspace\CCoverage\out.txt: $!";
while(<$fh>)
{
$s1 = "<";
$s2 = $_;
$s3 = ">";
$str = $s1 . $s2 . $s3;
print $fh "$str";
}
The very idea of what you are trying to do is flawed. The file starts as
H i / t h i s / ...
If you were to change it in place, it would look as follows after processing the first line:
< H i > / i s / ...
Notice how you clobbered "th"? You need to make a copy of the file, modify the copy, the replace the original with the copy.
The simplest way is to make this copy in memory.
my $file;
{ # Read the file
open(my $fh, '<', $qfn)
or die "Can't open \"$qfn\": $!\n";
local $/;
$file = <$fh>;
}
# Change the file
$file =~ s/^(.*)\n/<$1>\n/mg;
{ # Save the changes
open(my $fh, '>', $qfn)
or die "Can't create \"$qfn\": $!\n";
print($fh $file);
}
If you wanted to use the disk instead:
rename($qfn, "$qfn.old")
or die "Can't rename \"$qfn\": $!\n";
open(my $fh_in, '<', "$qfn.old")
or die "Can't open \"$qfn\": $!\n";
open(my $fh_out, '>', $qfn)
or die "Can't create \"$qfn\": $!\n";
while (<$fh_in>) {
chomp;
$_ = "<$_>";
print($fh_out "$_\n");
}
unlink("$qfn.old");
Using a trick, the above can be simplified to
local #ARGV = $qfn;
local $^I = '';
while (<>) {
chomp;
$_ = "<$_>";
print(ARGV "$_\n");
}
Or as a one-liner:
perl -i -pe'$_ = "<$_>"' file
Read contents in memory and then prepare required string as you write to your file. (SEEK_SET to zero't byte is required.
#!/usr/bin/perl
open(INFILE, "+<in.txt");
#a=<INFILE>;
seek INFILE, 0, SEEK_SET ;
foreach $i(#a)
{
chomp $i;
print INFILE "<".$i.">"."\n";
}
If you are worried about amount of data being read in memory, you will have to create a temporary result file and finally copy the result file to original file.
You could use Tie::File for easy random access to the lines in your file:
use Tie::File;
use strict;
use warnings;
my $filename = "out.txt";
my #array;
tie #array, 'Tie::File', $filename or die "can't tie file \"$filename\": $!";
for my $line (#array) {
$line = "<$line>";
# or $line =~ s/^(.*)$/<$1>/g; # -- whatever modifications you need to do
}
untie #array;
Disclaimer: Of course, this option is only viable if the file is not shared with other processes. Otherwise you could use flock to prevent shared access while you modify the file.
Disclaimer-2 (thanks to ikegami): Don't use this solution if you have to edit big files and are concerned about performance. Most of the performance loss is mitigated for small files (less than 2MB, though this is configurable using the memory arg).
One option is to open the file twice: Open it once read-only, read the data, close it, process it, open it again read-write (no append), write the data, and close it. This is good practice because it minimizes the time you have the file open, in case someone else needs it.
If you only want to open it once, then you can use the +< file type - just use the seek call between reading and writing to return to the beginning of the file. Otherwise, you finish reading, are at the end of the file, and start writing there, which is why you get the behavior you're seeing.
Need to specify
use Fcntl qw(SEEK_SET);
in order to use
seek INFILE, 0, SEEK_SET;
Thanks user1703205 for the example.
I have written the two program. One program is write the content to the text file simultaneously. Another program is read that content simultaneously.
But both the program should run at the same time. For me the program is write the file is correctly. But another program is not read the file.
I know that once the write process is completed than only the data will be stored in the hard disk. Then another process can read the data.
But I want both read and write same time with different process in the single file. How can I do that?
Please help me.
The following code write the content in the file
sub generate_random_string
{
my $length_of_randomstring=shift;# the length of
# the random string to generate
my #chars=('a'..'z','A'..'Z','0'..'9','_');
my $random_string;
foreach (1..$length_of_randomstring)
{
# rand #chars will generate a random
# number between 0 and scalar #chars
$random_string.=$chars[rand #chars];
}
return $random_string;
}
#Generate the random string
open (FH,">>file.txt")or die "Can't Open";
while(1)
{
my $random_string=&generate_random_string(20);
sleep(1);
#print $random_string."\n";
print FH $random_string."\n";
}
The following code is read the content. This is another process
open (FH,"<file.txt") or die "Can't Open";
print "Open the file Successfully\n\n";
while(<FH>)
{
print "$_\n";
}
You might use an elaborate cooperation protocol such as in the following. Both ends, reader and writer, use common code in the TakeTurns module that handles fussy details such as locking and where the lock file lives. The clients need only specify what they want to do when they have exclusive access to the file.
reader
#! /usr/bin/perl
use warnings;
use strict;
use TakeTurns;
my $runs = 0;
reader "file.txt" =>
sub {
my($fh) = #_;
my #lines = <$fh>;
print map "got: $_", #lines;
++$runs <= 10;
};
writer
#! /usr/bin/perl
use warnings;
use strict;
use TakeTurns;
my $n = 10;
my #chars = ('a'..'z','A'..'Z','0'..'9','_');
writer "file.txt" =>
sub { my($fh) = #_;
print $fh join("" => map $chars[rand #chars], 1..$n), "\n"
or warn "$0: print: $!";
};
The TakeTurns module is execute-around at work:
package TakeTurns;
use warnings;
use strict;
use Exporter 'import';
use Fcntl qw/ :DEFAULT :flock /;
our #EXPORT = qw/ reader writer /;
my $LOCKFILE = "/tmp/taketurns.lock";
sub _loop ($&) {
my($path,$action) = #_;
while (1) {
sysopen my $lock, $LOCKFILE, O_RDWR|O_CREAT
or die "sysopen: $!";
flock $lock, LOCK_EX or die "flock: $!";
my $continue = $action->();
close $lock or die "close: $!";
return unless $continue;
sleep 0;
}
}
sub writer {
my($path,$w) = #_;
_loop $path =>
sub {
open my $fh, ">", $path or die "open $path: $!";
my $continue = $w->($fh);
close $fh or die "close $path: $!";
$continue;
};
}
sub reader {
my($path,$r) = #_;
_loop $path =>
sub {
open my $fh, "<", $path or die "open $path: $!";
my $continue = $r->($fh);
close $fh or die "close $path: $!";
$continue;
};
}
1;
Sample output:
got: 1Upem0iSfY
got: qAALqegWS5
got: 88RayL3XZw
got: NRB7POLdu6
got: IfqC8XeWN6
got: mgeA6sNEpY
got: 2TeiF5sDqy
got: S2ksYEkXsJ
got: zToPYkGPJ5
got: 6VXu6ut1Tq
got: ex0wYvp9Y8
Even though you went to so much trouble, there are still issues. The protocol is unreliable, so reader has no guarantee of seeing all messages that writer sends. With no writer active, reader is content to read the same message over and over.
You could add all this, but a more sensible approach would be using abstractions the operating system provides already.
For example, Unix named pipes seem to be a pretty close match to what you want, and note how simple the code is:
pread
#! /usr/bin/perl
use warnings;
use strict;
my $pipe = "/tmp/mypipe";
system "mknod $pipe p 2>/dev/null";
open my $fh, "<", $pipe or die "$0: open $pipe: $!";
while (<$fh>) {
print "got: $_";
sleep 0;
}
pwrite
#! /usr/bin/perl
use warnings;
use strict;
my $pipe = "/tmp/mypipe";
system "mknod $pipe p 2>/dev/null";
open my $fh, ">", $pipe or die "$0: open $pipe: $!";
my $n = 10;
my #chars = ('a'..'z','A'..'Z','0'..'9','_');
while (1) {
print $fh join("" => map $chars[rand #chars], 1..$n), "\n"
or warn "$0: print: $!";
}
Both ends attempt to create the pipe using mknod because they have no other method of synchronization. At least one will fail, but we don't care as long as the pipe exists.
As you can see, all the waiting machinery is handled by the system, so you do what you care about: reading and writing messages.
This works.
The writer:
use IO::File ();
sub generate_random_string {...}; # same as above
my $file_name = 'file.txt';
my $handle = IO::File->new($file_name, 'a');
die "Could not append to $file_name: $!" unless $handle;
$handle->autoflush(1);
while (1) {
$handle->say(generate_random_string(20));
}
The reader:
use IO::File qw();
my $file_name = 'file.txt';
my $handle = IO::File->new($file_name, 'r');
die "Could not read $file_name: $!" unless $handle;
STDOUT->autoflush(1);
while (defined (my $line = $handle->getline)) {
STDOUT->print($line);
}
are you on windows or *nix? you might be able to string something like this together on *nix by using tail to get the output as it is written to the file. On windows you can call CreateFile() with FILE_SHARE_READ and/or FILE_SHARE_WRITE in order to allow others to access the file while you have it opened for read/write. you may have to periodically check to see if the file size has changed in order to know when to read (i'm not 100% certain here.)
another option is a memory mapped file.