Read all files in a directory in perl - perl

in the first part of my code, I read a file and store different parts of it in different files in a directory and in the following I wanna read all the files in that directory that I build it in the first part of code:
while(<file>){
#making files in directory Dir
}
opendir(Dir, $indirname) or die "cannot open directory $indirname";
#docs = grep(/\.txt$/,readdir(Dir));
foreach $d (#Dir) {
$rdir="$indirname/$d";
open (res,$rdir) or die "could not open $rdir";
while(<res>){
}
but with this code, the last line of the last file wont be read

As I don't know what you are doing in the line reading loop and don't understand #docs and #Dir, I'll show code that 'works' for me:
use strict;
use warnings;
use English;
my $dir = './_tmp/readFID';
foreach my $fp (glob("$dir/*.txt")) {
printf "%s\n", $fp;
open my $fh, "<", $fp or die "can't read open '$fp': $OS_ERROR";
while (<$fh>) {
printf " %s", $_;
}
close $fh or die "can't read close '$fp': $OS_ERROR";
}
output:
./_tmp/readFID/123.txt
1
2
3
./_tmp/readFID/45.txt
4
5
./_tmp/readFID/678.txt
6
7
8
Perhaps you can spot a relevant difference to your script.

I modified the code slightly to just test the basic idea in a directory containing my perl programs and it does seem to work. You should be iterating through #docs instead of #dir though (and I highly recommend using both the strict and warnings pragmas).
opendir(DIR, ".") or die "cannot open directory";
#docs = grep(/\.pl$/,readdir(DIR));
foreach $file (#docs) {
open (RES, $file) or die "could not open $file\n";
while(<RES>){
print "$_";
}
}

glob does what you want, without the open/close stuff. And once you stick a group of files into #ARGV the "diamond" operator works as normal.
#ARGV = <$indirname/*.txt>;
while ( <> ) {
...
}

Related

Perl-Copying file from one location to other but content not copying

I am writing a script in perl where I am creating a file and getting input from user for file but when I am copying that file to other location the file is copying but it is empty only. My code is
# !/usr/bin/perl -w
for($i = 1;$i<5;$i++)
{
open(file1,"</u/man/fr$i.txt");
print "Enter text for file $i";
$txt = <STDIN>;
print file1 $txt;
open(file2,">/u/man/result/fr$i.txt");
while(<file1>)
{
print file2 $_;
}
close(file1);
close(file2);
}
fr1 to fr4 are creating but these are empty. like when I run my code it is asking for input i provide the input and code run without error but still the files are empty. Please help.
in line number 4 I changed < to > also as I thought for creating new file it might need that but still it is not working
You need to close the filehandle that was written to in order to be able to read from that file.
use warnings;
use strict;
use feature 'say';
for my $i (1..4)
{
my $file = "file_$i.txt";
open my $fh, '>', $file or die "Can't open $file: $!";
say $fh "Written to $file";
# Opening the same filehandle first *closes* it if already open
open $fh, '<', $file or die "Can't open $file: $!";
my $copy = "copy_$i.txt";
open my $fh_cp, '>', $copy or die "Can't open $copy: $!";
while (<$fh>) {
print $fh_cp $_;
}
close $fh_cp; # in case of early errors in later iterations
close $fh;
}
This creates the four files, file_1.txt etc, and their copies, copy_1.txt etc.
Please note the compulsory checking whether open worked.
You can't write to a filehandle that's not open for writing. You can't read from a filehandle that's not open for reading. Never ignore the return value of open.
# !/usr/bin/perl
use warnings; # Be warned about mistakes.
use strict; # Prohibit stupid things.
for my $i (1 .. 4) { # lexical variable, range
open my $FH1, '>', "/u/man/fr$i.txt" # 3 argument open, lexical filehandle, open for writing
or die "$i: $!"; # Checking the return value of open
print "Enter text for file $i: ";
my $txt = <STDIN>;
print {$FH1} $txt;
open my $FH2, '<', "/u/man/fr$i.txt" # Reopen for reading.
or die "$i: $!";
open my $FH3, '>', "/u/man/result/fr$i.txt" or die "$i: $!";
while (<$FH2>) {
print {$FH3} $_;
}
close $FH3;
}
I opened the file in write mode using filehandler1 Then i again opened the file in read mode using same filehandler1 then I opened filehandler2 for destiantion So it is working fine for me then.
system("cp myfile1.txt /somedir/myfile2.txt")
`cp myfile1.txt /somedir/myfile2.txt`

Perl Script: sorting through log files.

Trying to write a script which opens a directory and reads bunch of multiple log files line by line and search for information such as example:
"Attendance = 0 " previously I have used grep "Attendance =" * to search my information but trying to write a script to search for my information.
Need your help to finish this task.
#!/usr/bin/perl
use strict;
use warnings;
my $dir = '/path/';
opendir (DIR, $dir) or die $!;
while (my $file = readdir(DIR))
{
print "$file\n";
}
closedir(DIR);
exit 0;
What's your perl experience?
I'm assuming each file is a text file. I'll give you a hint. Try to figure out where to put this code.
# Now to open and read a text file.
my $fn='file.log';
# $! is a variable which holds a possible error msg.
open(my $INFILE, '<', $fn) or die "ERROR: could not open $fn. $!";
my #filearr=<$INFILE>; # Read the whole file into an array.
close($INFILE);
# Now look in #filearr, which has one entry per line of the original file.
exit; # Normal exit
I prefer to use File::Find::Rule for things like this. It preserves path information, and it's easy to use. Here's an example that does what you want.
use strict;
use warnings;
use File::Find::Rule;
my $dir = '/path/';
my $type = '*';
my #files = File::Find::Rule->file()
->name($type)
->in(($dir));
for my $file (#files){
print "$file\n\n";
open my $fh, '<', $file or die "can't open $file: $!";
while (my $line = <$fh>){
if ($line =~ /Attendance =/){
print $line;
}
}
}

Search string with multiple words in the pattern

My program is trying to search a string from multiple files in a directory. The code searches for single patterns like perl but fails to search a long string like Status Code 1.
Can you please let me know how to search for strings with multiple words?
#!/usr/bin/perl
my #list = `find /home/ad -type f -mtime -1`;
# printf("Lsit is $list[1]\n");
foreach (#list) {
# print("Now is : $_");
open(FILE, $_);
$_ = <FILE>;
close(FILE);
unless ($_ =~ /perl/) { # works, but fails to find string "Status Code 1"
print "found\n";
my $filename = 'report.txt';
open(my $fh, '>>', $filename) or die "Could not open file '$filename' $!";
say $fh "My first report generated by perl";
close $fh;
} # end unless
} # end For
There are a number of problems with your code
You must always use strict and use warnings at the top of every Perl program. There is little point in delcaring anything with my without strict in place
The lines returned by the find command will have a newline at the end which must be removed before Perl can find the files
You should use lexical file handles (my $fh instead of FILE) and the three-parameter form of open as you do with your output file
$_ = <FILE> reads only the first line of the file into $_
unless ($_ =~ /perl/) is inverted logic, and there's no need to specify $_ as it is the default. You should write if ( /perl/ )
You can't use say unless you have use feature 'say' at the top of your program (or use 5.010, which adds all features available in Perl v5.10)
It is also best to avoid using shell commands as Perl is more than able to do anything that you can using command line utilities. In this case -f $file is a test that returns true if the file is a plain file, and -M $file returns the (floating point) number of days since the file's modification time
This is how I would write your program
#!/usr/bin/perl
use strict;
use warnings;
use 5.010;
for my $file ( glob '/home/ad/*' ) {
next unless -f $file and int(-M $file) == 1;
open my $fh, '<', $file or die $!;
while ( <$fh> ) {
if ( /perl/ ) {
print "found\n";
my $filename = 'report.txt';
open my $out_fh, '>>', $filename or die "Could not open file '$filename': $!";
say $fh "My first report generated by perl";
close $out_fh;
last;
}
}
}
it should have matched unless $_ contains text in different case.
try this.
unless($_ =~ /Status\s+Code\s+1/i) {
Change
unless ($_ =~ /perl/) {
to:
unless ($_ =~ /(Status Code 1)/) {
I am certain the above works, except it's case sensitive.
Since you question it, I rewrote your script to make more sense of what you're trying to accomplish and implement the above suggestion. Correct me if I am wrong, but you're trying to make a script which matches "Status Code 1" in a bunch of files where last modified within 1 day and print the filename to a text file.
Anyways, below is what I recommend:
#!/usr/bin/perl
use strict;
use warnings;
my $output_file = 'report.txt';
my #list = `find /home/ad -type f -mtime -1`;
foreach my $filename (#list) {
print "PROCESSING: $filename";
open (INCOMING, "<$filename") || die "FATAL: Could not open '$filename' $!";
foreach my $line (<INCOMING>) {
if ($line =~ /(Status Code 1)/) {
open( FILE, ">>$output_file") or die "FATAL: Could not open '$output_file' $!";
print FILE sprintf ("%s\n", $filename);
close(FILE) || die "FATAL: Could not CLOSE '$output_file' $!";
# Bail when we get the first match
last;
}
}
close(INCOMING) || die "FATAL: Could not close '$filename' $!";
}

Script to remove first line from all the text files in a directory

I'm trying to write a Perl script which reads all the text files in a directory and writes all the lines except first to a separate file. If there are 3 files, I want the script to read all those 3 files and write 3 new files with same lines except the first. This is what I wrote.. but when I try to run the script, it executes fine with no errors but doesn't do the work it is supposed to. Can someone please look into it?
opendir (DIR, "dir\\") or die "$!";
my #files = grep {/*?\.txt/} readdir DIR;
close DIR;
my $count=0;
my $lc;
foreach my $file (#files) {
$count++;
open(FH,"dir\\$file") or die "$!";
$str="dir\\example_".$count.".txt";
open(FH2,">$str");
$lc=0;
while($line = <FH>){
if($lc!=0){
print FH2 $line;
}
$lc++;
}
close(FH);
close(FH2);
}
And the second file doesn't exists, it is supposed to be created by script.
Try changing these lines
opendir (DIR, "dir\\") or die "$!";
...
close DIR;
to
opendir (DIR, "dir") or die "$!";
...
closedir DIR;
I tried running your code locally and the only two issues I had were with the directory name containing the trailing slash and trying to use the filehandle close() function on a dirhandle.
If you have the list of files ...
foreach my $file ( #files ) {
open my $infile , '<' , "dir/$file" or die "$!" ;
open my $outfile , '>' , "dir/example_" . ++${counter} . '.txt' or die "$!" ;
<$infile>; # Skip first line.
while( <$infile> ) {
print $outfile $_ ;
}
}
The lexical filehandles will be closed automatically when going out of scope.
Not sure why you're using $count here, as that's going to just turn a list of files like:
01.txt
bob.txt
alice.txt
02.txt
into:
01_1.txt
bob_2.txt
alice_3.txt
02_4.txt
Keep in mind, #files isn't being sorted, so it will return in the order the files exist in the directory table. If you were to delete and re-create the file 01.txt, it would be moved to the end of the list, re-ordering the whole set:
bob_1.txt
alice_2.txt
02_3.txt
01_4.txt
Since that wasn't really part of your original question, this does exactly what you asked to do:
#!/usr/bin/perl
while(<*.txt>) { # for every file in the *.txt glob from the current directory
open(IN, $_) or die ("Cannot open $_: $!"); # open file for reading
my #in = <IN>; # read the contents into an array
close(IN); # close the file handle
shift #in; # remove the first element from the array
open(OUT, ">$_.new") or die ("Cannot open $_.new: $!"); # open file for writing
print OUT #in; # write the contents of the array to the file
close(OUT); # close the file handle
}

perl: Writing file at Nth position

I am trying to write in to file at Nth POSITION. I have tried with below example but it writes at the end. Please help to achieve this.
#!/usr/bin/perl
open(FILE,"+>>try.txt")
or
die ("Cant open file try.txt");
$POS=5;
seek(FILE,$POS,0);
print FILE "CP1";
You are opening the file in read-write appending mode. Try opening the file in read-write mode:
my $file = "try.txt";
open my $fh, "+<", $file
or die "could not open $file: $!";
Also, note the use of the three argument open, the lexical filehandle, and $!.
#!/usr/bin/perl
use strict;
use warnings;
#create an in-memory file
my $fakefile = "1234567890\n";
open my $fh, "+<", \$fakefile
or die "Cant open file: $!";
my $offset = 5;
seek $fh, $offset, 0
or die "could not seek: $!";
print $fh "CP1";
print $fakefile;
The code above prints:
12345CP190
If I understand you correctly, if the file contents are
123456789
you want to change that to
1234CP157689
You cannot achieve that using modes supplied to open (regardless of programming language).
You need to open the source file and another temporary file (see File::Temp. Read up to the insertion point from the source and write the contents to the temporary file, write what you want to insert, then write the remainder of the source file to the temporary file, close the source and rename the temporary to the source.
If you are going to do this using seek, both files must be opened in binary mode.
Here is an example using line oriented input and text mode:
#!/usr/bin/perl
use strict; use warnings;
use File::Temp qw( :POSIX );
my $source = 'test.test';
my $temp = tmpnam;
open my $source_h, '<', $source
or die "Failed to open '$source': $!";
open my $temp_h, '>', $temp
or die "Failed to open '$temp' for writing: $!";
while ( my $line = <$source_h> ) {
if ( $line =~ /^[0-9]+$/ ) {
$line = substr($line, 0, 5) . "CP1" . substr($line, 5);
}
print $temp_h $line;
}
close $temp_h
or die "Failed to close '$temp': $!";
close $source_h
or die "Failed to close '$source': $!";
rename $temp => $source
or die "Failed to rename '$temp' to '$source': $!";
this works for me
use strict;
use warnings;
open( my $fh, '+<', 'foo.txt' ) or die $!;
seek( $fh, 3, 0 );
print $fh "WH00t?";
this is also a more "modern" use of open(), see http://perldoc.perl.org/functions/open.html
The file will be closed when $fh goes out of scope ..
"Inserting" a string into a function can (mostly) be done in place. See the lightly used truncate built-in function.
open my $fh, '+<', $file or die $!;
seek $fh, 5, 0;
$/ = undef;
$x = <$fh>; # read everything after the 5th byte into $x
truncate $fh, 5;
print $fh "CPI";
print $fh $x;
close $fh;
If your file is line or record oriented, you can insert lines or modify individual lines easily with the core module Tie::File This will allow the file to be treated as an array and Perl string and array manipulation to be used to modify the file in memory. You can safely operate on huge files larger than your RAM with this method.
Here is an example:
use strict; use warnings;
use Tie::File;
#create the default .txt file:
open (my $out, '>', "nums.txt") or die $!;
while(<DATA>) { print $out "$_"; }
close $out or die $!;
tie my #data, 'Tie::File', "nums.txt" or die $!;
my $offset=5;
my $insert="INSERTED";
#insert in a string:
$data[0]=substr($data[0],0,$offset).$insert.substr($data[0],$offset)
if (length($data[0])>$offset);
#insert a new array element that becomes a new file line:
splice #data,$offset,0,join(':',split(//,$insert));
#insert vertically:
$data[$_]=substr($data[$_],0,$offset) .
substr(lc $insert,$_,1) .
substr($data[$_],$offset) for (0..length($insert));
untie #data; #close the file too...
__DATA__
123456789
234567891
345678912
456789123
567891234
678912345
789123456
891234567
912345678
Output:
12345iINSERTED6789
23456n7891
34567s8912
45678e9123
56789r1234
I:N:St:E:R:T:E:D
67891e2345
78912d3456
891234567
912345678
The file modifications with Tie::File are made in place and as the array is modified. You could use Tie::File just on the first line of you file to modify and insert as you requested. You can put sleep between the array mods and use tail -n +0 -f on the file and watch the file change if you wish...
Alternatively, if your file is reasonable size and you want to treat it like characters, you can read the entire file into memory, do string operations on the data, then write the modified data back out. Consider:
use strict; use warnings;
#creat the default .txt file:
open (my $out, '>', "nums.txt") or die $!;
while(<DATA>) { print $out "$_"; }
close $out or die $!;
my $data;
open (my $in, '<', "nums.txt") or die $!;
{ local $/=undef; $data=<$in>; }
close $in or die $!;
my $offset=5;
my $insert="INSERTED";
open (my $out, '>', "nums.txt") or die $!;
print $out substr($data,0,$offset).$insert.substr($data,$offset);
close $out or die $!;
__DATA__
123456789
2
3
4
5
6
7
8
9
Output:
12345INSERTED6789
2
3
4
5
6
7
8
9
If you treat files as characters, beware that under Windows, files in text mode have a \r\n for a new line. That is two characters if opened in binary mode.