My code involves going to a host(doing with openSSH), getting the list of files matching a pattern(doing using remote find command-OpenSSH) and then opening each file I have got and processing each(grepping etc).
I am done with getting the file names and passing to a function. Now, I have to pass these filename to a function where I open each and process it. I am trying to do it using File::Remote as follows:
sub processFiles{
my $fileList =shift;
#Iterate over the file and try to find start and stop time stamps from the file
for my $file ( #{$fileList}) {
#finding start time of file:its found in lines of file
my $HEAD;
open $HEAD, "host:head -1000 $File|" or die "Unable to check file for starting time";
while ( <$HEAD> ) {
#process...
But I am unable to open the file on the host an I am getting an error.
When performing remote actions in batch, the first principle is to reduce the number of ssh connections (ideally only one). Using shell or perl one liner, you can do very complex actions. Here is my understanding of your need:
ssh hostname 'find dirname -name \*.pl| xargs grep -l pattern /dev/null'
You can pipe further processing in the same line. If you want to process the files locally, you can transfer all the files in a single command. Here is a full example:
use Archive::Tar;
open my $file, '-|', 'ssh -C hostname "find dirname -name \*.pl | xargs grep -l pattern /dev/null | xargs tar c"'
or die "Can't pipe: $!";
my $tar = Archive::Tar->new($file);
foreach my $file ($tar->get_files()) {
print $file->name, ', ', $file->size, "\n";
}
The File::Remote module is not part of a standard Perl installation. If you want to use this module, then you need to install it.
How you install it will depend on what platform you're using and how you have installed Perl. On a Linux installation where you are using the system Perl you could try sudo yum install perl-File-Remote (on a RedHat-based system) or sudo apt-get install libfile-remote-perl (on a Debian/Ubuntu-based system). You could also try using cpan or cpanm to install it.
I'd also suggest that as this module was last updated in 2005, it's quite likely that it is unmaintained, so I would be wary of using it.
Related
Below command in perl script is failing if there are many file in #file_to_tar, but it is working fine if we have less files in the array (#file_to_tar).
my $tar_command = "cd $ProcVars->{dropbox_dir}; tar -cvzf SmartMiles.$ProcVars->{batch_nb}.tar.gz -P #file_to_tar --remove-files";
Can some one please help me to fix the issue.
The best way is pass a very long list of files to tar is using the --from-file option:
tar -czf myarchive.tar.gz -P --from-file=$filelist --remove-files
You can also make it read the list of files from standard input by using --from-file=-
See https://www.gnu.org/software/tar/manual/html_node/files.html for more information.
You might consider, rather than calling a shell command, using the Archive::Tar Perl module instead. In particular, the "create_archive" method. This also would sidestep any potential problems arising from special characters or whitespace in the file names being interpreted by the shell your Perl code is invoking.
it is working now.
my $file_location = "$ProcVars->{dropbox_dir}/Archive_Files.csv"; open(DATA, ">$file_location") or die $!; foreach $a (#file_to_tar) { print DATA "$a\n"; } close DATA;
my $tar_command = "cd $ProcVars->{\dropbox_dir};tar -czf ABC.$ProcVars->{batch_nb}.tar.gz -P --files-from $file_location --remove-files";
I want to rename *.DIF files to *.SUC files
But the following script is giving "sh: bad substitution" error
I cannot use "rename" becuase my OS is not Linux.
$com="for i in *.DIF; do mv \$i \${i/DIF/SUC}; done;";
print $com."\n";
print `$com`;
Output :
for i in *.DIF; do mv $i ${i/DIF/SUC}; done;
sh: bad substitution
If you are having problems with Perl's rename, use File::Copy, a platform-independent module for moving and copying files. It is a core module, so it should be already part of your Perl installation.
If the system command works when you enter it in the shell, but not in Perl, the most likely reason is that Perl isn't using the shell you expect. We would need more information about your system to be sure, though.
There's no need to shell out for file operations that you can easily do within Perl.
The following renames all of your .dif extension files as .suc.
use strict;
use warnings;
use File::Copy qw(move);
move($_, s/dif$/suc/r) for glob('*.dif');
be default perl was using sh, instead of bash, which allows {//}
this question helped.
Finally I used :
open my $bash_handle, '| bash' or die "Cannot open bash: $!";
print $bash_handle 'for i in *.SUC; do mv $i ${i/SUC/DIF}; done;';
hi i have written a perl script which copies all the entire directory structure from source to destination and then i had to create a restore script from the perl script which will undo what the perl script has done that is create a script(shell) which can use bash features to restore the contents from destination back to source i m struggling to find the correct function or command which can copy recursively (not an requirement) but i want exactly the same structure as it was before
Below is the way i m trying to create a file called restore to do the restoration process
i m particularly looking for algorithm.
Also restore will restore the structure to a command line directory input if it is supplied if not You can assume the default input supplied to perl script
$source
$target
in this case we would wanna copy from target to source
So we have two different parts in one script.
1 which will copy from source to destination.
2 it will create a script file which will undo what part 1 has done
i hope this makes it very clear
unless(open FILE, '>'."$source/$file")
{
# Die with error message
# if we can't open it.
die "\nUnable to create $file\n";
}
# Write some text to the file.
print FILE "#!/bin/sh\n";
print FILE "$1=$target;\n";
print FILE "cp -r \n";
# close the file.
close FILE;
# here we change the permissions of the file
chmod 0755, "$source/$file";
The last problem i have is i couldn't get $1 in my restore file as it refers to a some variable in perl
but i need this for getting command line input when i run restore as $0 = ./restore $1=/home/xubuntu/User
First off, the standard way in Perl for doing this:
unless(open FILE, '>'."$source/$file") {
die "\nUnable to create $file\n";
}
is to use the or statement:
open my $file_fh, ">", "$source/$file"
or die "Unable to create "$file"";
It's just easier to understand.
A more modern way would be use autodie; which will handle all IO problems when opening or writing to files.
use strict;
use warnings;
use autodie;
open my $file_fh, '>', "$source/$file";
You should look at the Perl Modules File::Find, File::Basename, and File::Copy for copying files and directories:
use File::Find;
use File::Basename;
my #file_list;
find ( sub {
return unless -f;
push #file_list, $File::Find::name;
},
$directory );
Now, #file_list will contain all the files in $directory.
for my $file ( #file_list ) {
my $directory = dirname $file;
mkdir $directory unless -d $directory;
copy $file, ...;
}
Note that autodie will also terminate your program if the mkdir or copy commands fail.
I didn't fill in the copy command because where you want to copy and how may differ. Also you might prefer use File::Copy qw(cp); and then use cp instead of copy in your program. The copy command will create a file with default permissions while the cp command will copy the permissions.
You didn't explain why you wanted a bash shell command. I suspect you wanted to use it for the directory copy, but you can do that in Perl anyway. If you still need to create a shell script, the easiest way is via the :
print {$file_fh} << END_OF_SHELL_SCRIPT;
Your shell script goes here
and it can contain as many lines as you need.
Since there are no quotes around `END_OF_SHELL_SCRIPT`,
Perl variables will be interpolated
This is the last line. The END_OF_SHELL_SCRIPT marks the end
END_OF_SHELL_SCRIPT
close $file_fh;
See Here-docs in Perldoc.
First, I see that you want to make a copy-script - because if you only need to copy files, you can use:
system("cp -r /sourcepath /targetpath");
Second, if you need to copy subfolders, you can use -r switch, can't you?
I'm very new to Perl, and I would like to make a program that creates a directory and moves a file into that directory using the Unix command like:
mkdir test
Which I know would make a directory called "test". From there I would like to give more options like:
mv *.jpg test
That would move all .jpg files into my new directory.
So far I have this:
#!/usr/bin/perl
print "Folder Name:";
$fileName = <STDIN>;
chomp($fileType);
$result=`mkdir $fileName`;
print"Your folder was created \n";
Can anyone help me out with this?
Try doing this :
#!/usr/bin/perl
use strict; use warnings;
print "Folder Name:";
$dirName = <STDIN>;
chomp($dirName);
mkdir($dirName) && print "Your folder was created \n";
rename $_, "$dirName/$_" for <*.jpg>;
You will have a better control when using built-in perl functions than using Unix commands. That's the point of my snippet.
Most (if not all) Unix commands have a corresponding version as a function
e.g
mkdir - see here
mv - See here
Etc. either get a print out of the various manual pages (or probably have a trip down to the book shop - O'Reilly nut shell book is quite good along with others).
In perl you can use bash commands in backticks. However, what happens when the directory isn't created by the mkdir command? Your program doesn't get notified of this and continues on its merry way thinking that everything is fine.
You should use built in command in perl that do the same thing.
http://perldoc.perl.org/functions/mkdir.html
http://perldoc.perl.org/functions/rename.html
It is much easier to trap errors with those functions and fail gracefully. In addition, they run faster because you don't have to fork a new process for each command you run.
Perl has some functions similar to those of the shell. You can just use
mkdir $filename;
You can use backquotes to run a shell command, but it is only usefull if the command returns anything to its standard output, which mkdir does not. For commands without output, use system:
0 == system "mv *.jpg $folder" or die "Cannot move: $?";
I have been given a few Perl scripts to deploy.
What is the easiest way to find and install all modules used by these scripts?
EDIT:
From what I can find there are no conditional includes or includes in evals.
Does my Module::Extract::Use help? There's an extract_modules program in the examples directory:
$ examples/extract_modules -l some_program
File::Spec
File::Spec::Functions
strict
warning
You can pipe that list to cpan.
$ examples/extract_modules -l some_program | xargs cpan
Once you have that list, which is only the first level of dependencies, you can make a script distribution that allows people to use the normal CPAN toolchain to install everything.
If there's something that doesn't work for you, modify the program to handle that. If you think it would be useful to other people, send a pull request. :)
I was hoping Module::ScanDeps which provides the command line utility scandeps.pl would be useful here but, to my dismay, Module::ScanDeps is apparently not intended for this particular purpose as scandeps.pl either ignores missing modules or (with -c or -x) croaks when the script uses a module that is not installed.
Here is a quick'n'dirty Perl script that tries to execute the script using do until it succeeds:
#!/usr/bin/perl
use strict;
use warnings;
use Term::Prompt;
my ($script) = #ARGV;
die "Provide script file name on the command line\n"
unless defined $script;
until ( do $script ) {
my $ex = $#;
if ( my ($file) = $ex =~ /^Can't locate (.+?) in/ ) {
my $module = $file;
$module =~ s/\.(\w+)$//;
$module = join('::', split '/', $module);
print "Attempting to install '$module' via cpan\n";
system(cpan => $module);
last unless prompt(y => 'Try Again?', '', 'n');
}
else {
die $ex;
}
}
If you do not want the script to be run, you can run perl -c $script, capture stderr output of that and parse for missing module messages and call cpan for each such module found until perl -c $script outputs "Syntax OK". That gives you a cleaner loop too. I'll look at this later.
You might miss dependencies loaded at run time using this technique.
Well this is the very simplistic way I solved this.
In a bash shell:
cat *.pl | grep "^use " | tr ';' ' ' | while read a b c; do echo $b; done | sort -iu > modules.txt
This gave me a file with only the module names, one on each line.
I then used this
cat modules.txt | while read a; do cpan $a; done
to invoke cpan to each module name in the file. And then sat there answering yes to CPAN's questions to install dependencies as appropriate.
Not pretty but this time it got the job done.
Or let PAR's pp do the work for you in collecting together everything you need in a single executable.