To parse multiple files in Perl - perl

Please correct my code, I cannot seem to open my file to parse.
The error is this line open(my $fh, $file) or die "Cannot open file, $!";
Cannot open file, No such file or directory at ./sample.pl line 28.
use strict;
my $dir = $ARGV[0];
my $dp_dpd = $ENV{'DP_DPD'};
my $log_dir = $ENV{'DP_LOG'};
my $xmlFlag = 0;
my #fileList = "";
my #not_proc_dir = `find $dp_dpd -type d -name "NotProcessed"`;
#print "#not_proc_dir\n";
foreach my $dir (#not_proc_dir) {
chomp ($dir);
#print "$dir\n";
opendir (DIR, $dir) or die "Couldn't open directory, $!";
while ( my $file = readdir DIR) {
next if $file =~ /^\.\.?$/;
next if (-d $file);
# print "$file\n";
next if $file eq "." or $file eq "..";
if ($file =~ /.xml$/ig) {
$xmlFlag = 1;
print "$file\n";
open(my $fh, $file) or die "Cannot open file, $!";
#fileList = <$fh>;
close $file;
}
}
closedir DIR;
}

Quoting readdir's documentation:
If you're planning to filetest the return values out of a readdir, you'd better prepend the directory in question. Otherwise, because we didn't chdir there, it would have been testing the wrong file.
Your open(my $fh, $file) should therefore be open my $fh, '<', "$dir/$file" (note how I also added '<' as well: you should always use 3-argument open).
Your next if (-d $file); is also wrong and should be next if -d "$dir/$file";
Some additional remarks on your code:
always add use warnings to your script (in addition to use strict, which you already have)
use lexical file/directory handle rather than global ones. That is, do opendir my $DH, $dir, rather than opendir DH, $dir.
properly indent your code (if ($file =~ /.xml$/ig) { is one level too deep; it makes it harder to read you code)
next if $file =~ /^\.\.?$/; and next if $file eq "." or $file eq ".."; are redundant (even though not technically equivalent); I'd suggest using only the latter.
the variable $dir defined in my $dir = $ARGV[0]; is never used.

Related

Renaming files with perl

I'm very new to perl, and I want to rename a group of files so that they are lowercase instead of uppercase (so from SBC005.wav -> sbc005.wav).
use strict;
use warnings;
my $dirnam = "/Users/.../SoundFiles";
opendir(DIR, $dirnam) or die "Cannot open directory: $!";
my #files = readdir(DIR);
foreach my $oldfile (#files) {
my $newfile = lc($oldfile);
#print $newfile;
#print $oldfile;
rename $oldfile, $newfile or die "Cannot rename file: $!";
}
closedir(DIR);
I checked that the variables are working right with the commented out print statements, but when I run the program I get a message that says "Cannot rename file: Invalid argument at rename.pl line 13." I'm not sure what I'm doing wrong here.
Thank you so much!
edit:
Thank you so much to the answer below! I found that this code using glob works too, but the code below works better because it does not have to be in the same directory as the sound files (as the glob code does)
use strict;
use warnings;
my #files = glob("*.wav");
foreach my $oldfile (#files) {
my $newfile = lc($oldfile);
#print $newfile;
#print $oldfile;
rename $oldfile, $newfile or die "Cannot rename file: $!";
}
exit 0;
Try this. you forgot to add the path of directory while renaming.
use strict;
use warnings;
my $dirnam = "/Users/.../SoundFiles";
opendir(DIR, $dirnam) or die "Cannot open directory: $!";
my #files = readdir(DIR);
foreach my $oldfile (#files)
{
unless($oldfile eq "." || $oldfile eq ".." )
{
my $newfile = lc($oldfile);
rename "$dirnam/$oldfile", "$dirnam/$newfile" or die "Cannot rename file: $!";
}
}

Perl - search and replace across multiple lines across multiple files in specified directory

At the moment this code replaces all occurences of my matching string with my replacement string, but only for the file I specify on the command line. Is there a way to change this so that all .txt files for example, in the same directory (the directory I specify) are processed without having to run this 100s of times on individual files?
#!/usr/bin/perl
use warnings;
my $filename = $ARGV[0];
open(INFILE, "<", $filename) or die "Cannot open $ARGV[0]";
my(#fcont) = <INFILE>;
close INFILE;
open(FOUT,">$filename") || die("Cannot Open File");
foreach $line (#fcont) {
$line =~ s/\<br\/\>\n([[:space:]][[:space:]][[:space:]][[:space:]][A-Z])/\n$1/gm;
print FOUT $line;
}
close INFILE;
I have also tried this:
perl -p0007i -e 's/\<br\/\>\n([[:space:]][[:space:]][[:space:]][[:space:]][A-Z])/\n$1/m' *.txt
But have noticed that is only changes the first occurence of the matched pattern and ignores all the rest in the file.
I also have tried this, but it doesn't work in the sense that it just creates a blank file:
use v5.14;
use strict;
use warnings;
use DBI;
my $source_dir = "C:/Testing2";
# Store the handle in a variable.
opendir my $dirh, $source_dir or die "Unable to open directory: $!";
my #files = grep /\.txt$/i, readdir $dirh;
closedir $dirh;
# Stop script if there aren't any files in the list
die "No files found in $source_dir" unless #files;
foreach my $file (#files) {
say "Processing $source_dir/$file";
open my $in, '<', "$source_dir/$file" or die "Unable to open $source_dir/$file: $!\n";
open(FOUT,">$source_dir/$file") || die("Cannot Open File");
foreach my $line (#files) {
$line =~ s/\<br\/\>\n([[:space:]][[:space:]][[:space:]][[:space:]][A-Z])/\n$1/gm;
print FOUT $line;
}
close $in;
}
say "Status: Processing of complete";
Just wondering what am I missing from my code above? Thanks.
You could try the following:
opendir(DIR,"your_directory");
my #all_files = readdir(DIR);
closedir(DIR);
for (#all_files) .....

Perl Open: No such file or directory

I'm trying to read every text file in a directory into a variable then print the first 100 characters, including line breaks. However, Perl says that the files don't exist even though they really do exist.
use strict;
use warnings;
my $dir = "C:\\SomeFiles";
my #flist;
open(my $fh, "dir /a:-d /b $dir |") || die "$!";
while (<$fh>) {
if ($_ =~ /.*(.txt)$/i) {
push(#flist, $_);
}
}
foreach my $f (#flist) {
print "$dir\\$f";
my $txt = do {
local $/ = undef;
open(my $ff, "<", "$dir\\$f") || die "$!";
<$ff>;
};
print substr($txt, 0, 100);
}
When I run the script, the following is written to the console:
C:\SomeFiles\file1.txt
No such file or directory at script.pl line 19, <$fh> chunk 10.
It's looking at the right file and I'm certain that the file exists. When I try using this method to open a single file rather than getting each file via an array with foreach, it works just fine. Is there something obvious that I've overlooked here?
A better solution is to use readdir() instead (or File::Find if you ever want to do it recursively):
my $dir = "C:\\SomeFiles";
opendir(my $dh, $dir) || die "$!";
while (my $file = readdir($dh)) {
if ($file =~ /\\.txt$/i) {
print $file . "\n";
my $txt = do {
local $/ = undef;
open(my $ff, "<", "$dir\\$file") || die "$!";
<$ff>;
};
print substr($txt, 0, 100) . "\n";
}
}
closedir($dh);

Find a specific word in a file

First, I want to search for a particular file in the directory and then in the file I need to search for a specific word. Here's what I have so far:
$show_bp = 'ShowBuildProcess';
$get_bs = 'GetBuildStatus';
opendir (DIR, $my_dir) or die $!;
while(my $file = readdir(DIR))
{
if($file=~/\.log/)
{
if($file=~/GetBuildStatus/)
{
Filenames will be like GetStatus.<number>.log, e.g. GetStatus.123456.log. I need to:
Find all .log files in the directory
Search for a file with filename starting with GetStatus
Search for filename with the lower numeric part
Search for a particular word in that file
Here is a possible solution for you:
First we look at the file pattern and also extract $1,which is the first regex match )in the brackets). If the file fits we open it and look through it line by line and look for a match to your /YourSearchPattern/:
#!/usr/bin/perl
use warnings;
use strict;
my $mydir = './test/';
opendir (DIR, $mydir) or die $!;
while(my $file = readdir(DIR)){
if ($file =~ /^GetStatus\.(\d+)\.log$/){
if ($1 >= 123456 || $1 < 345678){
open(my $fh,'<', $mydir . $file) or die "Cannot open file $file: $!\n";
while (<$fh>){
if ($_ =~ /YourSearchPattern/){
print $_;
}
}
close($fh);
}
}
}
When you look for the smallest sequence number of the files from your dir you can simply store them in an array and then sort them after those numbers:
...
opendir (DIR, $mydir) or die $!;
my #files;
while(my $file = readdir(DIR)){
if ($file =~ /^GetStatus\.(\d+)\.log$/){
push #files $file;
}
}
my #sortedfiles = sort { my ($anum,$bnum); $a =~ /^GetStatus\.(\d+)\.log$/; $anum = $1; $b =~ /^GetStatus\.(\d+)\.log$/; $bnum = $1; $anum <=> $bnum } #files;
print $sortedfiles[0] . " has the smallest sequence number!\n";

What am I doing wrong in my Perl script written to parse a CSV file?

I have two scripts in which I'm experimenting with CSV_XS. In the first, I hard-coded everything: source directory, filename, and the csv delimiter I wanted to look for. The script works great. In the second, however, I try to dynamically discover as much as possible. That script seems to run, but it outputs nothing.
I'm having trouble figuring out why, and I was hoping you fine Perl folks wouldn't mind lending a second set of eyes to the problem:
First, the successful script:
#!/usr/bin/perl -w
use Text::CSV_XS;
my #records;
my $file = 'Data/space.txt';
my $csv=Text::CSV_XS->new({ sep_char => " " });
open(FILE,$file) || die "Couldn't open $file: $!\n";
while (<FILE>){
$csv->parse($_);
push(#records,[$csv->fields]);
}
close FILE;
foreach (#records){
print $_->[0], ",", $_->[1], ",", $_->[2], ",", $_->[3], ",", $_->[4], "\n";
}
And second, the "failing" script:
#!/usr/bin/perl -w
use Text::CSV_XS;
$input_dir = $ARGV[0]; #I pass "Data" on the command line
my #records;
opendir(DIR, $input_dir) || die "cannot open dir $input_dir: $!";
my #filelist = grep {$_ ne '.' && $_ ne '..'} readdir DIR;
closedir DIR;
foreach $file (#filelist){
print "Input file='",$input_dir,"/",$file,"'\n";
if ($file =~ /comma/) {$sep=','}
elsif ($file =~ /pipe/) {$sep='|'}
elsif ($file =~ /space/) {$sep=' '}
else {die "Cannot identify separator in $file: $!";}
print "Delimiter='",$sep,"'\n";
open(FILE,$input_dir||"/"||$file) || die "Couldn't open $file: $!\n";
my $csv=Text::CSV_XS->new({ sep_char => $sep });
while (<FILE>){
$csv->parse( $_ );
push(#records,[$csv->fields]);
print "File Input Line:'", $_ ,$csv->fields,"'\n";
};
close FILE;
}
foreach $record (#records){
print $record->[0], ",", $record->[1], ",", $record->[2], ",", $record->[3], ",", $record->[4], "\n";
}
This line looks kind of suspect:
open(FILE,$input_dir||"/"||$file) || die "Couldn't open $file: $!\n";
I don't think you want to put those || in there. What that does is check to see if $input_dir is true, then if it isn't, it check to see if "/" is true (which it always is). Your $input_dir is likely always true, so you're just opening the $input_dir.
You should be using File::Spec to create your fully-qualified files:
my $fullfile = File::Spec->catfile( $input_dir, $file );
open( FILE, $fullfile ) || die "Couldn't open $fullfile: $!\n";
This will "do the right thing" in putting a / where appropriate (or, if you're on Windows, \). Then pass that in to your open() command.
Further, you should be using lexical filehandles and directory handles, along with the three-option open():
open my $fh, '<', $fullfile or die "Could not open file $fullfile: $!\n";
Lexical filehandles are much safer, as they can't get overridden by some other module defining a FILE filehandle. Three-option open() is easier to understand and isn't prone to error when you have a filename that has a > or < or | in it.
If you want to get really crazy, put use autodie; at the top, so you don't even have to check for the return value of open() or opendir():
use autodie;
open my $fh, '<', $fullfile;