I have a perl script which I have written to search files present in my windows folders, recursively. I enter the search text as the perl script runtime argument to find a file having this text in it's name. The perl script is as below:
use Cwd;
$file1 = #ARGV[0];
##res1 = glob "*test*";
##res1 = glob "$file1*";
#res1 = map { Cwd::abs_path($_) } glob "$file1*";
foreach (#res1)
{
print "$_\n";
}
But this is not searching all the sub-directories recursively. I know glob doesn't match recursively.
So tried using module File::Find and the function find(\&wanted, #directories);
But I got a error saying find() undefined. From what I read from help, I thought find() function is defined by default in Perl installation, with some basic code to find folders/files. Isn't it correct?
Questions is, in the above perl script, how do I search for files/folders recursively?
Second questions, I found that perldoc <module> help does not have examples about using a certain function in that module, which would make it clear.
Can you point to some good help/document/book for using various perl functions from different perl modules with clear examples of usage of those module functions.
Another excellent module to use is File::Find::Rule which hides some of the complexity of File::Find while exposing the same rich functionality.
use File::Find::Rule;
use Cwd;
my $cwd = getcwd();
my $filelist;
sub buildFileIndex {
open ($filelist, ">", "filelist.txt") || die $!;
# File find rule
my $excludeDirs = File::Find::Rule->directory
->name('demo', 'test', 'sample', '3rdParty') # Provide specific list of directories to *not* scan
->prune # don't go into it
->discard; # don't report it
my $includeFiles = File::Find::Rule->file
->name('*.txt', '*.csv'); # search by file extensions
my #files = File::Find::Rule->or( $excludeDirs, $includeFiles )
->in($cwd);
print $filelist map { "$_\n" } #files;
return \$filelist;
}
These two pages are all you need to study:
File::Find documentation
Beginners guide to File::Find
If you don't mind using cpan module, Path::Class can do the work for you:
use Path::Class;
my #files;
dir('.')->recurse(callback => sub {
my $file = shift;
if($file =~ /some text/) {
push #files, $file->absolute->stringify;
}
});
for my $file (#files) {
# ...
}
An alternative would be to use find2perl to create the start of the script for you. It can turn a find command like,
find . -type f -name "*test*" -print
To an equivalent perl script. You just put find2perl instead of find. It uses File::Find under the hood but gets you going quickly.
use 5.010; # Enable 'say' feature
use strict;
use warnings;
use File::Find; # The module for 'find'
find(\&wanted, #ARGV); # #ARGV is the array of directories to find.
sub wanted {
# Do something...
# Some useful variables:
say $_; # File name in each directory
say $File::Find::dir; # the current directory name
say $File::Find::name; # the complete pathname to the file
}
Example for listing driver modules on Linux (Fedora):
use 5.022;
use strict;
use warnings;
use POSIX qw(uname);
use File::Find;
my $kernel_ver = (uname())[2];
my #dir = (
"/lib/modules/$kernel_ver/kernel/drivers"
);
find(\&wanted, #dir);
sub wanted {
say if /.*\.ko\.xz/;
}
Related
I have a scenario where I need to loop through multiple level directories to find text files.
Say, I start from the folder C:\A and then I want to read all the text files inside it. But they are not placed at same level.
Some text files are arranged at C:\A\A1\sample1.txt
Some as C:\A\sample2.txt
Some as C:\A\A2\A3\sample.txt
I am able to loop inside the folder A which returns me A1 and A2, but I wanted to know is there a way it would iterate automatically to every level and return me the text files along with its path.
ance!
I'd use File::Find::Rule:
#!/usr/bin/env perl
use strict;
use warnings;
use File::Find::Rule;
my #files = File::Find::Rule -> file()
-> name('sample*.txt')
-> in ( 'C:\\A' );
foreach my $file ( #files ) {
print "Found: $file\n";
#process it here.
}
Use the File::Find module to do it.
use strict;
use warnings 'all';
use File::Find 'find';
my #files;
find(sub {
push #files, $File::Find::name if -f and /\.txt$/i;
}, 'C:\A');
print "$_\n" for #files;
how can i list all files in parent and sub directories for multilpe dirs?
$dir="/home/httpd/cgi-bin/r/met";
opendir(DIR,"/home/httpd/cgi-bin/r/met")||die"error";
while($line=readdir DIR)
{
print"$line\n";
opendir DIR1,"$dir/$line"||die"error";
while($line1=readdir DIR1)
{
print"$line1\n";
}
}
closedir DIR;
closedir DIR1;
Don't do it this way, use File::Find instead.
use strict;
use warnings;
use File::Find;
my $search = "/home/httpd/cgi-bin/r/met";
sub print_file_names {
print $_,"\n";
}
find ( \&print_file_names, $search );
File::Find effectively walks through list of directories and executes a subroutine defined by you for each file or directory found recursively below the starting directory. Before calling your subroutine find (a function exported by File::Find module) by default changes to the directory being scanned and sets the following (global) variables:
$File::Find::dir -- visited directory path relative to the starting directory
$File::Find::name -- full path of the file being visited relative to the starting directory
$_ -- basename of the file being visited (used in my example)
One way to solve your problem would be:
#!/usr/bin/perl
# Usage: ffind [dir1 ...]
use strict; use warnings;
use 5.010; # to be able to use say
use File::Find;
# use current working directory if no command line argument
#ARGV = qw(.) unless #ARGV;
find( sub { say if -f }, #ARGV );
#!/usr/bin/perl -w
use File::Copy;
use strict;
my $i= "0";
my $j= "1";
my $source_directory = $ARGV[$i];
my $target_directory = $ARGV[$j];
#print $source_directory,"\n";
#print $target_directory,"\n";
my#list=process_files ($source_directory);
print "remaninign files\n";
print #list;
# Accepts one argument: the full path to a directory.
# Returns: A list of files that reside in that path.
sub process_files {
my $path = shift;
opendir (DIR, $path)
or die "Unable to open $path: $!";
# We are just chaining the grep and map from
# the previous example.
# You'll see this often, so pay attention ;)
# This is the same as:
# LIST = map(EXP, grep(EXP, readdir()))
my #files =
# Third: Prepend the full path
map { $path . '/' . $_}
# Second: take out '.' and '..'
grep { !/^\.{1,2}$/ }
# First: get all files
readdir (DIR);
closedir (DIR);
for (#files) {
if (-d $_) {
# Add all of the new files from this directory
# (and its subdirectories, and so on... if any)
push #files, process_files ($_);
} else { #print #files,"\n";
# for(#files)
while(#files)
{
my $input= pop #files;
print $input,"\n";
copy($input,$target_directory);
}
}
# NOTE: we're returning the list of files
return #files;
}
}
This basically copies files from source to destination but I need some guidance on how to
copy the directory as well. The main thing to note here is no CPAN modules are allowed except copy, move, and path
Instead of rolling your own directory processing adventure, why not simply use File::Find to go through the directory structure for you.
#! /usr/bin/env perl
use :5.10;
use warnings;
use File::Find;
use File::Path qw(make_path);
use File::Copy;
use Cwd;
# The first two arguments are source and dest
# 'shift' pops those arguments off the front of
# the #ARGV list, and returns what was removed
# I use "cwd" to get the current working directory
# and prepend that to $dest_dir. That way, $dest_dir
# is in correct relationship to my input parameter.
my $source_dir = shift;
my $dest_dir = cwd . "/" . shift;
# I change into my $source_dir, so the $source_dir
# directory isn't in the file name when I find them.
chdir $source_dir
or die qq(Cannot change into "$source_dir");;
find ( sub {
return unless -f; #We want files only
make_path "$dest_dir/$File::Find::dir"
unless -d "$dest_dir/$File::Find::dir";
copy "$_", "$dest_dir/$File::Find::dir"
or die qq(Can't copy "$File::Find::name" to "$dest_dir/$File::Find::dir");
}, ".");
Now, you don't need a process_files subroutine. You let File::Find::find handle recursing the directory for you.
By the way, you could rewrite the find like this which is how you usually see it in the documentation:
find ( \&wanted, ".");
sub wanted {
return unless -f; #We want files only
make_path "$dest_dir/$File::Find::dir"
unless -d "$dest_dir/$File::Find::dir";
copy "$_", "$dest_dir/$File::Find::dir"
or die qq(Can't copy "$File::Find::name" to "$dest_dir/$File::Find::dir");
}
I prefer to embed my wanted subroutine into my find command instead because I think it just looks better. It first of all guarantees that the wanted subroutine is kept with the find command. You don't have to look at two different places to see what's going on.
Also, the find command has a tendency to swallow up your entire program. Imagine where I get a list of files and do some complex processing on them. The entire program can end up in the wanted subroutine. To avoid this, you simply create an array of the files you want to operate on, and then operate on them inside your program:
...
my #file_list;
find ( \&wanted, "$source_dir" );
for my $file ( #file_list ) {
...
}
sub wanted {
return unless -f;
push #file_list, $File::Find::name;
}
I find this a programming abomination. First of all, what is going on with find? It's modifying my #file_list, but how? No where in the find command is #file_list mentioned. What is it doing?
Then at the end of my program is this sub wanted function that is using a variable, #file_list in a global manner. That's bad programming practice.
Embedding my subroutine directly into my find command solves many of these issues:
my #file_list;
find ( sub {
return unless -f;
push #file_list;
}, $source_dir );
for my $file ( #file_list ) {
...
}
This just looks better. I can see that #file_list is being manipulated directly by my find command. Plus, that pesky wanted subroutine has disappeared from the end of my program. Its' the exact same code. It just looks better.
Let's get to what that find command is doing and how it works with the wanted subroutine:
The find command finds each and every file, directory, link, or whatnot located in the directory list you pass to it. With each item it finds in that directory, it passes it to your wanted subroutine for processing. A return leaves the wanted subroutine and allows find to fetch the next item.
Each time the wanted subroutine is called, find sets three variables:
$File::Find::name: The name of the item found with the full path attached to it.
$File::Find::dir: The name of the directory where the item was found.
$_: The name of the item without the directory name.
In Perl, that $_ variable is very special. It's sort of a default variable for many commands. That is, you you execute a command, and don't give it a variable to use, that command will use $_. For example:
print
prints out $_
return if -f;
Is the same as saying this:
if ( -f $_ ) {
return;
}
This for loop:
for ( #file_list ) {
...
}
Is the same as this:
for $_ ( #file_list ) {
...
}
Normally, I avoid the default variable. It's global in scope and it's not always obvious what is being acted upon. However, there are a few circumstances where I'll use it because it really clarifies the program's meaning:
return unless -f;
in my wanted function is very obvious. I exit the wanted subroutine unless I was handed a file. Here's another:
return unless /\.txt$/;
This will exit my wanted function unless the item ends with '.txt'.
I hope this clarifies what my program is doing. Plus, I eliminated a few bugs while I was at it. I miscopied $File::Find::dir to $File::Find::name which is why you got the error.
I was wondering how to import a Perl file to a script. I experimented with use, require and do, but nothing seems to work for me. This is how I did it with require:
#!/usr/bin/perl
require {
(equations)
}
print "$x1\n";
Is it possible to code for substituting a value (I get in my script) into equations.pl, then have my script use an equation defined in equations.pl to calculate another value? How do I do this?
You can require a .pl file, which will then execute the code in it, but in order to access variables, you need a package, and either "use" instead of require (the easy way) or via Exporter.
http://perldoc.perl.org/perlmod.html
Simple example: here's the stuff you want to import, name it Example.pm:
package Example;
our $X = 666;
1; # packages need to return true.
And here's how to use it:
#!/usr/bin/perl -w
use strict;
use Example;
print $Example::X;
This presumes Example.pm is in the same directory, or the top level of an #INC directory.
equations.pm file:
package equations;
sub add_numbers {
my #num = #_;
my $total = 0;
$total += $_ for #num;
$total;
}
1;
test.pl file:
#!/usr/bin/perl -w
use strict;
use equations;
print equations::add_numbers(1, 2), "\n";
output:
3
You can't import a file. You can execute a file and import symbols (variables and subs) from it. See Perl Modules in perlmod.
You've given very few details about equations.pl, but if the input can be given via a command line argument, then you can open a pipe:
use strict;
use warnings;
my $variable; #the variable that you will get from equations.pl
my $input=5; #the input into equations.pl
open (my $fh,"-|","perl equations.pl $input") or die $!;
while(my $output=<$fh>)
{
chomp($output); #remove trailing newline
$variable=$output;
}
if(defined($variable))
{
print "It worked! \$variable=$variable\n";
}
else
{
print "Nope, \$variable is still undefined...\n";
}
If this is the body of equations.pl:
use strict;
use warnings;
my $foo=$ARGV[0];
$foo++;
print "$foo\n";
Then the code above outputs:
It worked! $variable=6
I have one folder. There are 32 files and 3 directories in that folder. I want to add some lines of text on each file at top. How can I do that?
Use File::Find to find the files. Use Tie::File and unshift to add lines to the top of the file.
TLP already told you some hints how to solve the Problem. But there is always more then one way to do it. Instead of File::Find and Tie::File i would use some more "modern" modules. In these full example i use Path::Class::Rule with an iterative interface instead of an recursive interface that i like more.
#!/usr/bin/env perl
use strict;
use warnings;
use utf8;
use open ':encoding(UTF-8)';
use open ':std';
use Path::Class;
use Path::Class::Rule;
my $rule = Path::Class::Rule->new->file;
my $iter = $rule->iter(dir('test'));
while ( my $file = $iter->() ) {
print $file->stringify, "\n";
add_line_to_file($file, "Sid was here.\n");
}
# 1: Path::Class::File Object
# 2: The Line
sub add_line_to_file {
my ( $file, $line ) = #_;
# Open File - return IO::File object
my $fh = $file->open('>>') or die "Cannot open $file: $!\n";
# Seek to end
$fh->seek(0, 2);
# Add line
$fh->print($line);
$fh->close;
return;
}
This could work:
perl -pi -e 's/^/my text\n/' * */*
Please try this on copy of your directory to make sure it does what you want.