Perl cannot stat $_ - perl

I want the last modified time for each file in the directory. To make sure my loop is working I print $_ and I see the file names of the directory:
for ( #Files ) {
opendir( D, $path . '\/' . $_ ) or die "$!";
my #textfiles = grep { ! /^\.{1,2}$/ } readdir( D );
for ( #textfiles ) {
# print "$_\n"; <----the file names.
my $epoch_timestamp = ( stat( $_ ) )[9];
print "$epoch_timestamp\n";
}
I get this error
Use of uninitialized value $epoch_timestamp in concatenation (.) or string
What am I doing wrong?

readdir returns only the names of the files. If your current working directory is different then you must build the full path as you did with the parameter to opendir. The easiest way is to use map in the list for the for loop
I'm concerned about your statement
opendir( D, $path . '\/' . $_ ) or die "$!";
which will put, literally, \/ between $path and $_. I think you need just /, but it is simplest to interpolate the variables with
opendir( D, "$path/$_" ) or die "$!";
But $_ comes from the array #Files. If these are indeed file names then your opendir will fail. They need to be directory names
In my solution I've built the variable $dir as
my $dir = "$path/$_"
so that it can be used in the call to opendir as well as to build the full path to the files in the following for loop
Note that I have also used a lexical directory handle my $dh, which are far superior to global handles D
for ( #Files ) {
my $dir = "$path/$_";
opendir my $dh, $dir or die $!;
my #textfiles = grep { ! /^\.{1,2}$/ } readdir $dh;
for ( map { "$dir/$_" } #textfiles ) {
# print "$_\n"; <----the file names.
my $epoch_timestamp = ( stat( $_ ) )[9];
print "$epoch_timestamp\n";
}

Or alternatively to above perfect answers, you could use some modules and make your life more easy. :) Like: Path::Tiny[1]
use 5.014;
use warnings;
use Path::Tiny;
my $path = path('/etc');
my #Files = qw(defaults cups ssl);
for my $dir (#Files) {
my #textfiles = $path->child($dir)->children;
for my $file (#textfiles) {
say "$file: ", $file->stat->mtime;
}
}
Of course, the above the nested loop could be written as
for my $dir (#Files) {
my #textfiles = $path->child($dir)->children;
say "$_: ", $_->stat->mtime for (#textfiles);
}
and also storing the list of files into #textfiles isn't necessary, so it could be reduced to:
for my $dir (#Files) {
say "$_: ", $_->stat->mtime for ( $path->child($dir)->children );
}
Path::Tiny conveniently throws a clean exception message on error.

readdir only returns the name of the file in the directory. You need to provide a qualified path to the file to stat.
my $dir_qfn = ...;
opendir(my $dh, $dir_qfn)
or do {
warn("Can't read dir \"$dir_qfn\": $!\n");
next;
};
while (defined( my $fn = readdir($dh) )) {
next if $fn =~ /^\.\.?\z/;
my $qfn = "$dir_qfn/$fn";
my $mtime = ( stat($qfn) )[9];
defined($mtime)
or do {
warn("Can't stat file \"$file_qfn\": $!\n");
next;
};
...
}

Using glob instead
my $dir = ...;
my %ts =
map { $_ => (stat $_)[9] }
grep { !m{/\.\.?\z} } #/
glob "\Q$dir\E/{*,.*}";
say "ts{$_} => $_" for sort keys %ts;
I use a hash name => timestamp to collect both in a data structure. The pattern $dir/{*,.*} is there to catch dot files as well, or it would be just $dir/*.
The grep filters out . and .. filenames, found in path by m{..} match. Its pattern needs \Q..\E to prevent an injection bug with particular directory names. It also escapes spaces so File::Glob with its :bsd_globoption isn't needed. Thanks to ikegami for comments.
If you'd rather process files one at a time, retrieve the list with glob and then iterate through it.

Related

Create an array of hashes describing files obtained from a given directory

I need to
Read a list of the files from in a specified directory
Create an array of hashes describing those files
My program fetches the path to a directory from the command line, opens it and reads its contents. Directories and "dot" files are skipped, and the absolute path to every other entry is printed.
use strict;
use warnings;
use Data::Dumper qw(Dumper);
use File::Spec;
use Digest::SHA qw(sha256_hex);
my $dir = $ARGV[0];
opendir DIR, $dir or die "cannot open dir $dir: $!";
while ( my $file = readdir DIR ) {
next unless ( -f "${ \File::Spec->catfile($dir, $file) }" );
next if ( "$file" =~ m/^\./) ;
print "${ \File::Spec->rel2abs($file) }\n";
}
closedir DIR;
Here I take a single file and create a hash with the path, size, and sha256sum.
my $file = "file1.txt";
my #fileref = (
{
path => "Full path: " . File::Spec->rel2abs($file),
size => "Size (bytes): " . -s $file,
id => "SHA256SUM: " . sha256_hex($file),
},
);
print "\n";
print "$fileref[0]{path}\n";
print "$fileref[0]{size}\n";
print "$fileref[0]{id}\n";
All of this works, but I cannot figure out how to iterate over each file and add it to the array.
This is what I planned
for each file
push file into array
add the path, size, and id key:value pairs to file
repeat
How can I generate the necessary array?
Thanks to Wumpus Q. Wumbley's push suggestion, I have solved my problem:
my #array;
opendir DIR, $dir or die "cannot open dir $dir: $!";
while(my $file = readdir DIR) {
next unless(-f "${\File::Spec->catfile($dir, $file)}");
next if("$file" =~ m/^\./);
#print "${\File::Spec->rel2abs($file)}\n";
my %hash = (
path => File::Spec->rel2abs($file),
size => -s $file,
id => sha256_hex($file),
);
push(#array, \%hash);
#print Dumper sort \#array;
}
closedir DIR;
print Dumper \#array;
I create the "frame" for the hash, and then pass it to the array via reference and the push function.
Here's my approach to a solution. I've saved a lot of code by using File::Spec::Functions instead of File::Spec, and calling rel2abs only once
I've also removed the labels, like "Full path: ", from the values in the hashes. There's no reason to put presentation strings in there: that's for the output code to do
use strict;
use warnings;
use File::Spec::Functions qw/ catfile rel2abs /;
use Digest::SHA qw/ sha256_hex /;
use Data::Dumper qw/ Dumper /;
my ( $root ) = #ARGV;
$root = rel2abs( $root ); # Allow for relative path input
my #file_data;
{
opendir my $dh, $root or die qq{Cannot open directory "$root": $!};
while ( readdir $dh ) {
next if /^\./;
my $file = catfile( $root, $_ );
next unless -f $file;
push #file_data, {
path => $file,
size => -s $file,
id => sha256_hex( $file ),
};
}
}
print Dumper \#file_data;
You have all the code, you are creating an array with the first element. Your way to dereference that hints at an easy way to add others:
my $i = 0;
while(my $file = readdir DIR) {
next unless(-f "${\File::Spec->catfile($dir, $file)}");
next if("$file" =~ m/^\./);
print "${\File::Spec->rel2abs($file)}\n";
$fileref[$i]{path} = File::Spec->rel2abs($file);
$fileref[$i]{size} = -s $fileref[0]{path};
$fileref[$i]{id} = sha256_hex(fileref[0]{path});
$i++;
}

Counting the number of files in a directory of special type in perl

I would like to know if there is anyway to find the number of files exsiting in a folder with special type. For example I have a folder with 30 files with *.txt, *.doc and html extension. I want to know the number of say html file inthis directory.
Update: Here is what I have as a number os files in the directory. But I am not sure how I could use glob(). Of course, instead of getcwd one could give another parameter.
use Cwd;
my $dir = getcwd;
my $count = 0;
opendir (DIR, $dir) or die $!;
my #dir = readdir DIR;
my #file_list;
if (#file_list eq glob "*.pl"){
print "$item\n";
$count = $count + 1;
}
closedir DIR;
$count = $count - 2;
print "There are $count files in this directory.";
I found out how to do it without glob():
#!/usr/bin/perl
use strict;
use warnings;
use Cwd;
my $dir = getcwd;
my $count = 0;
opendir(my $dh, $dir) or die "$0: $dir: $!\n";
while (my $file = readdir($dh)) {
# We only want files
next unless (-f "$dir/$file");
# Use a regular expression to find files ending in .txt
next unless ($file =~ m/\.html$/);
print "$file\n";
$count = $count + 1;
}
closedir($dh);
print "There are $count files in this directory.";
exit 0;
Thanks a lot for the comments!!
The problem you've got in your question is that glob is a bit magic. You can do this:
foreach my $file ( glob ("*.txt") ) {
print $file,"\n";
}
and
while ( my $file = glob ("*.txt" )) {
print $file,"\n";
}
Glob is detecting whether you're expecting a scalar (single value) return - in which case it works as an iterator - or an array (multiple scalars) - in which case it returns the whole lot.
You can make it do what you want like this:
my #stuff = glob ( "*.txt" );
print "There are: ", scalar #stuff," files matching the pattern\n";
print join ( "\n", #stuff );
Note that readdir works the same way - you can either slurp the whole lot doing it in a list context, or one line at a time with a scalar context:
opendir ( my $dirh, "some_directory");
my #stuff = readdir ( $dirh );
#etc.
Or
opendir ( my $dirh, "." ) or die $!;
while ( my $dir_entry = readdir ( $dirh ) ) {
#etc.
}
If you do want to do readdir-and-filter you can also do it like this:
my #matches = grep { m/\.txt$/ } readdir ( $dirh );
For example (this doesn't save you any efficiency - grep just hides the loop. It might make it more readable - that's a matter of taste).

Perl program help on opendir and readdir

So I have a program that I want to clean some text files. The program asks for the user to enter the full pathway of a directory containing these text files. From there I want to read the files in the directory, print them to a new file (that is specified by the user), and then clean them in the way I need. I have already written the script to clean the text files.
I ask the user for the directory to use:
chomp ($user_supplied_directory = <STDIN>);
opendir (DIR, $user_supplied_directory);
Then I need to read the directory.
my #dir = readdir DIR;
foreach (#dir) {
Now I am lost.
Any help please?
I'm not certain of what do you want. So, I made some assumptions:
When you say clean the text file, you meant delete the text file
The names of the files you want to write into are formed by a pattern.
So, if I'm right, try something like this:
chomp ($user_supplied_directory = <STDIN>);
opendir (DIR, $user_supplied_directory);
my #dir = readdir DIR;
foreach (#dir) {
next if (($_ eq '.') || ($_ eq '..'));
# Reads the content of the original file
open FILE, $_;
$contents = <FILE>;
close FILE;
# Here you supply the new filename
$new_filename = $_ . ".new";
# Writes the content to the new file
open FILE, '>'.$new_filename;
print FILE $content;
close FILE;
# Deletes the old file
unlink $_;
}
I would suggest that you switch to File::Find. It can be a bit of a challenge in the beginning but it is powerful and cross-platform.
But, to answer your question, try something like:
my #files = readdir DIR;
foreach $file (#files) {
foo($user_supplied_directory/$file);
}
where "foo" is whatever you need to do to the files. A few notes might help:
using "#dir" as the array of files was a bit misleading
the folder name needs to be prepended to the file name to get the right file
it might be convenient to use grep to throw out unwanted files and subfolders, especially ".."
I wrote something today that used readdir. Maybe you can learn something from it. This is just a part of a (somewhat) larger program:
our #Perls = ();
{
my $perl_rx = qr { ^ perl [\d.] + $ }x;
for my $dir (split(/:/, $ENV{PATH})) {
### scanning: $dir
my $relative = ($dir =~ m{^/});
my $dirpath = $relative ? $dir : "$cwd/$dir";
unless (chdir($dirpath)) {
warn "can't cd to $dirpath: $!\n";
next;
}
opendir(my $dot, ".") || next;
while ($_ = readdir($dot)) {
next unless /$perl_rx/o;
### considering: $_
next unless -f;
next unless -x _;
### saving: $_
push #Perls, "$dir/$_";
}
}
}
{
my $two_dots = qr{ [.] .* [.] }x;
if (grep /$two_dots/, #Perls) {
#Perls = grep /$two_dots/, #Perls;
}
}
{
my (%seen, $dev, $ino);
#Perls = grep {
($dev, $ino) = stat $_;
! $seen{$dev, $ino}++;
} #Perls;
}
The crux is push(#Perls, "$dir/$_"): filenames read by readdir are basenames only; they are not full pathnames.
You can do the following, which allows the user to supply their own directory or, if no directory is specified by the user, it defaults to a designated location.
The example shows the use of opendir, readdir, stores all files in the directory in the #files array, and only files that end with '.txt' in the #keys array. The while loop ensures that the full path to the files are stored in the arrays.
This assumes that your "text files" end with the ".txt" suffix. I hope that helps, as I'm not quite sure what's meant by "cleaning the files".
use feature ':5.24';
use File::Copy;
my $dir = shift || "/some/default/directory";
opendir(my $dh, $dir) || die "Can't open $dir: $!";
while ( readdir $dh ) {
push( #files, "$dir/$_");
}
# store ".txt" files in new array
foreach $file ( #files ) {
push( #keys, $file ) if $file =~ /(\S+\.txt\z)/g;
}
# Move files to new location, even if it's across different devices
for ( #keys ) {
move $_, "/some/other/directory/"; || die "Couldn't move files: $!\n";
}
See the perldoc of File::Copy for more info.

How can I get a list of all files with a certain extension from a specific directory?

I'm using this code to get a list of all the files in a specific directory:
opendir DIR, $dir or die "cannot open dir $dir: $!";
my #files= readdir DIR;
closedir DIR;
How can I modify this code or append something to it so that it only looks for text files and only loads the array with the prefix of the filename?
Example directory contents:
.
..
923847.txt
98398523.txt
198.txt
deisi.jpg
oisoifs.gif
lksdjl.exe
Example array contents:
files[0]=923847
files[1]=98398523
files[2]=198
my #files = glob "$dir/*.txt";
for (0..$#files){
$files[$_] =~ s/\.txt$//;
}
it is enough to change one line:
my #files= map{s/\.[^.]+$//;$_}grep {/\.txt$/} readdir DIR;
If you can use the new features of Perl 5.10, this is how I would write it.
use strict;
use warnings;
use 5.10.1;
use autodie; # don't need to check the output of opendir now
my $dir = ".";
{
opendir my($dirhandle), $dir;
for( readdir $dirhandle ){ # sets $_
when(-d $_ ){ next } # skip directories
when(/^[.]/){ next } # skip dot-files
when(/(.+)[.]txt$/){ say "text file: ", $1 }
default{
say "other file: ", $_;
}
}
# $dirhandle is automatically closed here
}
Or if you have very large directories, you could use a while loop.
{
opendir my($dirhandle), $dir;
while( my $elem = readdir $dirhandle ){
given( $elem ){ # sets $_
when(-d $_ ){ next } # skip directories
when(/^[.]/){ next } # skip dot-files
when(/(.+)[.]txt$/){ say "text file: ", $1 }
default{
say "other file: ", $_;
}
}
}
}
This is the simplest way I've found (as in human readable) using the glob function:
# Store only TXT-files in the #files array using glob
my #files = grep ( -f ,<*.txt>);
# Write them out
foreach $file (#files) {
print "$file\n";
}
Additionally the "-f" ensures that only actual files (and not directories) are stored in the array.
To get just the ".txt" files, you can use a file test operator (-f : regular file) and a regex.
my #files = grep { -f && /\.txt$/ } readdir $dir;
Otherwise, you can look for just text files, using perl's -T (ascii-text file test operator)
my #files = grep { -T } readdir $dir;
Just use this:
my #files = map {-f && s{\.txt\z}{} ? $_ : ()} readdir DIR;

How do I read in the contents of a directory in Perl?

How do I get Perl to read the contents of a given directory into an array?
Backticks can do it, but is there some method using 'scandir' or a similar term?
opendir(D, "/path/to/directory") || die "Can't open directory: $!\n";
while (my $f = readdir(D)) {
print "\$f = $f\n";
}
closedir(D);
EDIT: Oh, sorry, missed the "into an array" part:
my $d = shift;
opendir(D, "$d") || die "Can't open directory $d: $!\n";
my #list = readdir(D);
closedir(D);
foreach my $f (#list) {
print "\$f = $f\n";
}
EDIT2: Most of the other answers are valid, but I wanted to comment on this answer specifically, in which this solution is offered:
opendir(DIR, $somedir) || die "Can't open directory $somedir: $!";
#dots = grep { (!/^\./) && -f "$somedir/$_" } readdir(DIR);
closedir DIR;
First, to document what it's doing since the poster didn't: it's passing the returned list from readdir() through a grep() that only returns those values that are files (as opposed to directories, devices, named pipes, etc.) and that do not begin with a dot (which makes the list name #dots misleading, but that's due to the change he made when copying it over from the readdir() documentation). Since it limits the contents of the directory it returns, I don't think it's technically a correct answer to this question, but it illustrates a common idiom used to filter filenames in Perl, and I thought it would be valuable to document. Another example seen a lot is:
#list = grep !/^\.\.?$/, readdir(D);
This snippet reads all contents from the directory handle D except '.' and '..', since those are very rarely desired to be used in the listing.
A quick and dirty solution is to use glob
#files = glob ('/path/to/dir/*');
This will do it, in one line (note the '*' wildcard at the end)
#files = </path/to/directory/*>;
# To demonstrate:
print join(", ", #files);
IO::Dir is nice and provides a tied hash interface as well.
From the perldoc:
use IO::Dir;
$d = IO::Dir->new(".");
if (defined $d) {
while (defined($_ = $d->read)) { something($_); }
$d->rewind;
while (defined($_ = $d->read)) { something_else($_); }
undef $d;
}
tie %dir, 'IO::Dir', ".";
foreach (keys %dir) {
print $_, " " , $dir{$_}->size,"\n";
}
So you could do something like:
tie %dir, 'IO::Dir', $directory_name;
my #dirs = keys %dir;
You could use DirHandle:
use DirHandle;
$d = new DirHandle ".";
if (defined $d)
{
while (defined($_ = $d->read)) { something($_); }
$d->rewind;
while (defined($_ = $d->read)) { something_else($_); }
undef $d;
}
DirHandle provides an alternative, cleaner interface to the opendir(), closedir(), readdir(), and rewinddir() functions.
Similar to the above, but I think the best version is (slightly modified) from "perldoc -f readdir":
opendir(DIR, $somedir) || die "can't opendir $somedir: $!";
#dots = grep { (!/^\./) && -f "$somedir/$_" } readdir(DIR);
closedir DIR;
You can also use the children method from the popular Path::Tiny module:
use Path::Tiny;
my #files = path("/path/to/dir")->children;
This creates an array of Path::Tiny objects, which are often more useful than just filenames if you want to do things to the files, but if you want just the names:
my #files = map { $_->stringify } path("/path/to/dir")->children;
Here's an example of recursing through a directory structure and copying files from a backup script I wrote.
sub copy_directory {
my ($source, $dest) = #_;
my $start = time;
# get the contents of the directory.
opendir(D, $source);
my #f = readdir(D);
closedir(D);
# recurse through the directory structure and copy files.
foreach my $file (#f) {
# Setup the full path to the source and dest files.
my $filename = $source . "\\" . $file;
my $destfile = $dest . "\\" . $file;
# get the file info for the 2 files.
my $sourceInfo = stat( $filename );
my $destInfo = stat( $destfile );
# make sure the destinatin directory exists.
mkdir( $dest, 0777 );
if ($file eq '.' || $file eq '..') {
} elsif (-d $filename) { # if it's a directory then recurse into it.
#print "entering $filename\n";
copy_directory($filename, $destfile);
} else {
# Only backup the file if it has been created/modified since the last backup
if( (not -e $destfile) || ($sourceInfo->mtime > $destInfo->mtime ) ) {
#print $filename . " -> " . $destfile . "\n";
copy( $filename, $destfile ) or print "Error copying $filename: $!\n";
}
}
}
print "$source copied in " . (time - $start) . " seconds.\n";
}
from: http://perlmeme.org/faqs/file_io/directory_listing.html
#!/usr/bin/perl
use strict;
use warnings;
my $directory = '/tmp';
opendir (DIR, $directory) or die $!;
while (my $file = readdir(DIR)) {
next if ($file =~ m/^\./);
print "$file\n";
}
The following example (based on a code sample from perldoc -f readdir) gets all the files (not directories) beginning with a period from the open directory. The filenames are found in the array #dots.
#!/usr/bin/perl
use strict;
use warnings;
my $dir = '/tmp';
opendir(DIR, $dir) or die $!;
my #dots
= grep {
/^\./ # Begins with a period
&& -f "$dir/$_" # and is a file
} readdir(DIR);
# Loop through the array printing out the filenames
foreach my $file (#dots) {
print "$file\n";
}
closedir(DIR);
exit 0;
closedir(DIR);
exit 0;