I have a directory which contains multiple levels of sub dirs. I want to print path for each and every directory.
Currently, I am using
use File::Find;
find(
{
wanted => \&findfiles,
}, $maindirectory);
sub findfiles
{
if (-d) {
push #arrayofdirs,$File::Find::dir;
}
}
But each subdirectory contains thousands of files at each level. The above code takes lot of time to provide the result as it compares each file for directory. Is there a way to get subdirectories path without comparing files to save time or any other optimized method?
Edit: This issue got partially resolved but a new issue came up because of this solution. I have listed it here: Multiple File search in varying level of directories in perl
If you are on a UNIX/Linux platform then you can try reading output of find $maindirectory -type d command into your program (see this answer for a safe way to do that.). This command prints the names of directories in $maindirectory. It is faster because a compiled C program (find) does all the hard work. The following script should print all directory paths found.
Sample script:
use strict;
use warnings;
my $maindirectory = '.';
open my $fh, '-|', 'find', $maindirectory, '-type', 'd' or die "Can't open pipe: $!";
while( my $dir = <$fh>) {
print $dir;
}
close $fh or warn "can't close pipe: $!";
Note that there is no point in calling find through perl and then just printing its output without any processing. You can just as well run find $maindirectory -type d in shell itself.
I am beginner to Perl and I have to create a .pl file and I have folder containing near about 30 exe files(inside Folder1 in G:\Folder1). All of them must be executed by click to the .pl file.
My try is :
use strict; use warnings;
use autodie; # automatic error handling
while (defined(my $file = glob 'C:\shekhar_Axestrack_Intern*.exe'))
{
open my $fh, "<", $file; # lexical file handles, automatic error handling
while (defined( my $line = <$fh> )) {
do system $fh ;
}
close $fh;
}
Please let me know if my logic correct ? Could some one please correct me if i am wrong ?
Use system to execute an exe:
while (my $file = glob 'C:\shekhar_Axestrack_Intern\*.exe') {
system $file;
}
In addition, I have the feeling that you meant to write 'C:\shekhar_Axestrack_Intern*.exe'
instead of 'C:\shekhar_Axestrack_Intern*.exe'.
I think pl2bat may help you. It allows you to wrap Perl code into a batch file.
BTW why are you using echo in your Perl script? You should use print.
Edit: You have edited your question and now you want to know how to run all exe files from a folder using Perl?
Use the system command to run the exe files providing the full path.
See: How to run an executable file using Perl on Windows XP?
Edit 2: do system $fh ; This is not how you do it, please get a book (I'd suggest Beginning Perl by Ovid) and start learning Perl.
I am new to Perl and trying to write text files. I can write text files to an existing directory no problem, but ultimately I would like to be able to create my own directories.
I am going to download files from my course works website and I want to put the files in a folder named after the course. I don't want to make a folder for each course manually beforehand, and I would also like to eventually share the script with others, so I need a way to make the directories and name them based on the course names from the HTML.
So far, I have been able to get this to work:
use strict;
my $content = "Hello world";
open MYFILE, ">C:/PerlFiles/test.txt";
print MYFILE $content;
close (MYFILE);
test.txt doesn't exist, but C:/PerlFiles/ does and supposedly typing > allows me to create files, great.
The following, however does not work:
use strict;
my $content = "area = pi*r^2";
open MYFILE, ">C:/PerlFiles/math_class/circle.txt";
print MYFILE $content;
close (MYFILE);
The directory C:/PerlFiles/math_class/ does not exist.
I also tried sysopen but I get an error when adding the flags:
use strict;
my $content = "area = pi*r^2";
sysopen (MYFILE, ">C:/PerlFiles/math_class/circle.txt", O_CREAT);
print MYFILE $content;
close (MYFILE);
I got this idea from the Perl Cookbook chapter 7.1. Opening a File. It doesn't work, and I get the error message Bareword "O_CREAT" not allowed while "strict subs" in use. Then again the book is from 1998, so perhaps O_CREAT is obsolete. At some point I think I will need to fork over the dough for an up-to-date version.
But still, what am I missing here? Or do the directories have to be created manually before creating a file in it?
Right, directories have to be created manually.
Use mkdir function.
You can check if directory already exists with -d $dir (see perldoc -f -X).
Use File::Path to create arbitrarily deep paths. Use dirname to find out a file's containing directory.
Also, use lexical file handles and three-argument open:
open(my $fd, ">", $name) or die "Can't open $name: $!";
Trying to troubleshoot a port of some perl code from CentOS to Windows.
Really know nothing about Perl, and the code I'm porting is around 700-1000 lines. 100% sure one of the issues I'm seeing is related to how the code is being rendered as a result of being on the OS it's running on.
So, I'm looking for a way to troubleshoot debugging how the OS's are rendering filepath apart from the legacy code; which I can not post to SO due to "IP" reasons.
So, I looking for some perl that I can set a directory to open within the script (for example, C:\data\ or /home/data), then script attempts to load the directory, prints if it failed or succeeded, and then prints the string it attempted to load, regardless if the code failed to open the directory or not.
Open to suggestions, but that's the issue, and the solution I'm seeing.
Questions, feedback, requests - just comment, thanks!!
use IO::Dir;
my $dir = IO::Dir->new($dir_path) or
die "Could not open directory $dir_path: $!\n";
of course, where $dir_path is some path to a directory on your system that you want, either as a var or hard coded. The more 'old school' way would look like:
opendir my $dir, $dir_path or die "Could not open directory $dir_path: $!\n";
That won't print of the directory is opened, but the program will fail if it doesn't open it then print the precise error as to why, which is what the $! variable holds.
Is this what you're looking for?
use DirHandle;
my $dir = "test";
my $dh = new DirHandle($dir);
if($dh) {
print "open directory succeeded\n";
}
else {
print "open directory failed\n";
}
print $dir, "\n";
new DirHandle opens the directory and returns a handle to it. The handle will be undef if the open failed.
Recently there have been a few attackers trying malicious things on my server so I've decided to somewhat "track" them even though I know they won't get very far.
Now, I have an entire directory containing the server logs and I need a way to search through every file in the directory, and return a filename if a string is found. So I thought to myself, what better of a language to use for text & file operations than Perl? So my friend is helping me with a script to scan all files for a certain IP, and return the filenames that contain the IP so I don't have to search for the attacker through every log manually. (I have hundreds)
#!/usr/bin/perl
$dir = ".";
opendir(DIR, "$dir");
#files = grep(/\.*$/,readdir(DIR));
closedir(DIR);
foreach $file(#files) {
open FILE, "$file" or die "Unable to open files";
while(<FILE>) {
print if /12.211.23.200/;
}
}
although it is giving me directory read errors. Any assistance is greatly appreciated.
EDIT: Code edited, still saying permission denied cannot open directory on line 10. I am just going to run the script from within the logs directory if you are questioning the directory change to "."
Mike.
Can you use grep instead?
To get all the lines with the IP, I would directly use grep, no need to show a list of files, it's a simple command:
grep 12\.211\.23\.200 *
I like to pipe it to another file and then open that file in an editor...
If you insist on wanting the filenames, it's also easy
grep -l 12\.211\.23\.200 *
grep is available on all Unix//Linux with the GNU tools, or on windows using one of the many implementations (unxutils, cygwin, ...etc.)
You have to concatenate $dirname with $filname when using files found through readdir, remember you haven't chdir'ed into the directory where those files resides.
open FH, "<", "$dirname/$filname" or die "Cannot open $filname:$!";
Incidentally, why not just use grep -r to recursively search all subdirectories under your log dir for your string?
EDIT: I see your edits, and two things. First, this line:
#files = grep(/\.*$/,readdir(DIR));
Is not effective, because you are searching for zero or more . characters at the end of the string. Since it's zero or more, it'll match everything in the directory. If you're trying to exclude files ending in ., try this:
#files = grep(!/\.$/,readdir(DIR));
Note the ! sign for negation if you're trying to exclude those files. Otherwise (if you only want those files and I'm misunderstanding your intent), leave the ! out.
In any case, if you're getting your die message on line 10, most likely you're hitting a file that has permissions such that you can't read it. Try putting the filename in the die output so you can see which file it's failing on:
open FILE, "$file" or die "Unable to open file: $file";
But as with other answers, and to reiterate: Why not use grep? The unix command, not the Perl function.
This will get the file names you are looking for in perl, and probably do it much faster than running and doing a perl regex.
#files = `find ~/ServerLogs -name "*.log" | xargs grep -l "<ip address>"`'
Although, this will require a *nix compliant system, or Cygwin on Windows.
Firstly get a list of files within your source directory:
opendir(DIR, "$dir");
#files = grep(/\.log$/,readdir(DIR));
closedir(DIR);
And then loop through those files
foreach $file(#files)
{
// file processing code
}
My first suggest would be to use grep instead. The right tool for the job, they say...
But to answer your question:
readdir just returns the filenames from the directory. You'll need to concatenate the directory name and filename together.
$path = "$dirname/$filname";
open FH, $path or die ...
Then you should ignore files that are actually directories, such as "." and "..". After getting the $path, check to see if it's a file.
if (-f $path) {
open FH, $path or die ...
while (<FH>)
BTW, I thought I would throw in a mention for File::Next. To iterate over all files in a directory (recursively):
use Path::Class; # always useful.
use File::Next;
my $files = File::Next::files( dir(qw/path to files/) ); # look in path/to/files
while( defined ( my $file = $files->() ) ){
$file = file( $file );
say "Examining $file";
say "found foo" if $file->slurp =~ /foo/;
}
File::Next is taint-safe.
~ doesn't auto-expand in Perl.
opendir my $fh, '~/' or die("Doin It Wrong"); # Doing It Wrong.
opendir my $fh, glob('~/') and die( "Thats right!" );
Also, if you must use readdir(), make sure you guard the expression thus:
while (defined(my $filename = readdir(DH))) {
...
}
If you don't do the defined() test, the loop will terminate if it finds a file called '0'.
Have you looked on CPAN for log parsers? I searched with 'log parse' and it yielded over 200 hits. Some (probably many) won't be relevant - some may be. It depends, in part, on which web server you are using.
Am I reading this right? Your line 10 that gives you the error is
open FILE, "$file" or die "Unable to open files";
And the $file you are trying to read, according to line 6,
#files = grep(/\.*$/,readdir(DIR));
is a file that ends with zero or more dot. Is this what you really wanted? This basically matches every file in the directory, including "." and "..". Maybe you don't have enough permission to open the parent directory for reading?
EDIT: if you only want to read all files (including hidden ones), you might want to use something like the following:
opendir(DIR, ".");
#files = readdir(DIR);
closedir(DIR);
foreach $file (#files) {
if ($file ne "." and $file ne "..") {
open FILE, "$file" or die "cannot open $file\n";
# do stuff with FILE
}
}
Note that this doesn't take care of sub directories.
I know I am way late to this discussion (ran across it while searching for grep related posts) but I am going to answer anyway:
It isn't specified clearly if these are web server logs (Apache, IIS, W3SVC, etc.) but the best tool for mining those for data is the LogParser tool from Microsoft. See logparser.com for more info.
LogParser will allow you to write SQL-like statements against the log files. It is very flexible and very fast.
Use perl from the command line, like a better grep
perl -wnl -e '/12.211.23.200/ and print;' *.log > output.txt
the benefit here is that you can chain logic far easier
perl -wnl -e '(/12.211.23.20[1-11]/ or /denied/i ) and print;' *.log
if you are feeling wacky you can also use more advanced command line options to feed perl one liner result into other perl one liners.
You really need to read "Minimal Perl: For UNIX and Linux People", awesome book on this very sort of thing.
First, use grep.
But if you don't want to, here are two small improvements you can make that I haven't seen mentioned yet:
1) Change:
#files = grep(/\.*$/,readdir(DIR));
to
#files = grep({ !-d "$dir/$_" } readdir(DIR));
This way you will exclude not just "." and ".." but also any other subdirectories that may exist in the server log directory (which the open downstream would otherwise choke on).
2) Change:
print if /12.211.23.200/;
to
print if /12\.211\.23\.200/;
"." is a regex wildcard meaning "any character". Changing it to "\." will reduce the number of false positives (unlikely to change your results in practice but it's more correct anyway).