Below command in perl script is failing if there are many file in #file_to_tar, but it is working fine if we have less files in the array (#file_to_tar).
my $tar_command = "cd $ProcVars->{dropbox_dir}; tar -cvzf SmartMiles.$ProcVars->{batch_nb}.tar.gz -P #file_to_tar --remove-files";
Can some one please help me to fix the issue.
The best way is pass a very long list of files to tar is using the --from-file option:
tar -czf myarchive.tar.gz -P --from-file=$filelist --remove-files
You can also make it read the list of files from standard input by using --from-file=-
See https://www.gnu.org/software/tar/manual/html_node/files.html for more information.
You might consider, rather than calling a shell command, using the Archive::Tar Perl module instead. In particular, the "create_archive" method. This also would sidestep any potential problems arising from special characters or whitespace in the file names being interpreted by the shell your Perl code is invoking.
it is working now.
my $file_location = "$ProcVars->{dropbox_dir}/Archive_Files.csv"; open(DATA, ">$file_location") or die $!; foreach $a (#file_to_tar) { print DATA "$a\n"; } close DATA;
my $tar_command = "cd $ProcVars->{\dropbox_dir};tar -czf ABC.$ProcVars->{batch_nb}.tar.gz -P --files-from $file_location --remove-files";
Related
What would be an example of how I can call a shell command, say 'ls -a' in a Perl script and the way to retrieve the output of the command as well?
How to run a shell script from a Perl program
1. Using system system($command, #arguments);
For example:
system("sh", "script.sh", "--help" );
system("sh script.sh --help");
System will execute the $command with
#arguments and return to your script when finished. You may check $!
for certain errors passed to the OS by the external application. Read
the documentation for system for the nuances of how various
invocations are slightly different.
2. Using exec
This is very similar to the use of system, but it will
terminate your script upon execution. Again, read the documentation
for exec for more.
3. Using backticks or qx//
my $output = `script.sh --option`;
my $output = qx/script.sh --option/;
The backtick operator and it's equivalent qx//, excute the command and options inside the operator and return that commands output to STDOUT when it finishes.
There are also ways to run external applications through creative use of open, but this is advanced use; read the documentation for more.
From Perl HowTo, the most common ways to execute external commands from Perl are:
my $files = `ls -la` — captures the output of the command in $files
system "touch ~/foo" — if you don't want to capture the command's output
exec "vim ~/foo" — if you don't want to return to the script after executing the command
open(my $file, '|-', "grep foo"); print $file "foo\nbar" — if you want to pipe input into the command
Examples
`ls -l`;
system("ls -l");
exec("ls -l");
Look at the open function in Perl - especially the variants using a '|' (pipe) in the arguments. Done correctly, you'll get a file handle that you can use to read the output of the command. The back tick operators also do this.
You might also want to review whether Perl has access to the C functions that the command itself uses. For example, for ls -a, you could use the opendir function, and then read the file names with the readdir function, and finally close the directory with (surprise) the closedir function. This has a number of benefits - precision probably being more important than speed. Using these functions, you can get the correct data even if the file names contain odd characters like newline.
As you become more experienced with using Perl, you'll find that there are fewer and fewer occasions when you need to run shell commands. For example, one way to get a list of files is to use Perl's built-in glob function. If you want the list in sorted order you could combine it with the built-in sort function. If you want details about each file, you can use the stat function. Here's an example:
#!/usr/bin/perl
use strict;
use warnings;
foreach my $file ( sort glob('/home/grant/*') ) {
my($dev,$ino,$mode,$nlink,$uid,$gid,$rdev,$size,$atime,$mtime,$ctime,$blksize,$blocks)
= stat($file);
printf("%-40s %8u bytes\n", $file, $size);
}
There are a lot of ways you can call a shell command from a Perl script, such as:
back tick
ls which captures the output and gives back to you.
system
system('ls');
open
Refer #17 here: Perl programming tips
You might want to look into open2 and open3 in case you need bidirectional communication.
I have been using system and qq to run linux programs inside perl. And it has worked well.
#!/usr/bin/perl # A hashbang line in perl
use strict; # It can save you a lot of time and headache
use warnings; # It helps you find typing mistakes
# my keyword in Perl declares the listed variable
my $adduser = '/usr/sbin/adduser';
my $edquota = '/usr/sbin/edquota';
my $chage = '/usr/bin/chage';
my $quota = '/usr/bin/quota';
my $nomeinteiro;
my $username;
my $home;
# system() function executes a system shell command
# qq() can be used in place of double quotes
system qq($adduser --home $home --gecos "$fullname" $username);
system qq($edquota -p john $username);
system qq($chage -E \$(date -d +180days +%Y-%m-%d) $username);
system qq($chage -l $username);
system qq($quota -s $username);
I'm very new to Perl, and I would like to make a program that creates a directory and moves a file into that directory using the Unix command like:
mkdir test
Which I know would make a directory called "test". From there I would like to give more options like:
mv *.jpg test
That would move all .jpg files into my new directory.
So far I have this:
#!/usr/bin/perl
print "Folder Name:";
$fileName = <STDIN>;
chomp($fileType);
$result=`mkdir $fileName`;
print"Your folder was created \n";
Can anyone help me out with this?
Try doing this :
#!/usr/bin/perl
use strict; use warnings;
print "Folder Name:";
$dirName = <STDIN>;
chomp($dirName);
mkdir($dirName) && print "Your folder was created \n";
rename $_, "$dirName/$_" for <*.jpg>;
You will have a better control when using built-in perl functions than using Unix commands. That's the point of my snippet.
Most (if not all) Unix commands have a corresponding version as a function
e.g
mkdir - see here
mv - See here
Etc. either get a print out of the various manual pages (or probably have a trip down to the book shop - O'Reilly nut shell book is quite good along with others).
In perl you can use bash commands in backticks. However, what happens when the directory isn't created by the mkdir command? Your program doesn't get notified of this and continues on its merry way thinking that everything is fine.
You should use built in command in perl that do the same thing.
http://perldoc.perl.org/functions/mkdir.html
http://perldoc.perl.org/functions/rename.html
It is much easier to trap errors with those functions and fail gracefully. In addition, they run faster because you don't have to fork a new process for each command you run.
Perl has some functions similar to those of the shell. You can just use
mkdir $filename;
You can use backquotes to run a shell command, but it is only usefull if the command returns anything to its standard output, which mkdir does not. For commands without output, use system:
0 == system "mv *.jpg $folder" or die "Cannot move: $?";
I have this directory structure:
$ ls -F
analyze/
data.pl
input.pl
logminer.txt
logSearch.pl
readFormat.pl
Version Notes.txt
datadump.pl
format/
logminer.pl
logs/
properties.txt
unzip.exe
I run:
perl -e 'if (!(-d analyze)){ print "poo\n"}'
and it prints poo.
What is missing here? I have done tests like this earlier and it would correctly identify that the directory exists. Why not this directory?
perl -e 'if (!(-d "analyze")){ print "poo\n"}'
^-- ^---
missing quotes?
edit: changed to double quotes - forgot this was for command-line perl
First,
-d analyze
means "check if the file handle anaylyze is a directory handle". You want
-d 'analyze'
Now, you say you still get the problem by doing that, so check what error you're getting.
my $rv = -d 'analyze';
die "stat: $!" if !defined($rv);
die "Not a dir" if !$rv;
-d is just a thin wrapper around stat(2), so it's not Perl that "can't see", it's the system.
The most common errors:
The current work directory isn't what you think it is. (Many people assume it's always the directory in which the script resides.)
The file name has trailing whitespace, especially a newline. (That's not likely to be the case here.)
I need to get an example file file from a find command in a Perl script to create another system call afterwards. For some reason, the find command gets stuck when I call it from the script. Here is what I need to do:
my $search_dir = "/something/like/this/??/??/??";
# the triple '??' are needed here
my $cmd = "find $search_dir -name \"\*.$var1.token1.$var2.ext\" | head -n 1";
my $first_example_file = `$cmd`; chomp $first_example_file;
This gets stuck when I run it through Perl, it never finishes executing the command, whereas the constructed $cmd runs in no time if I copy+paste it and run in in my bash terminal. Any ideas?
Try using the File::Find perl module for finding files. If you would like to use bash's find in your perl then you might have to use $(..) in your command.
I am not in to perl … just trying to help out.
Update:
As stated in the comments by Rohaq you can also use File::Find::Rule
I'd wager globbing (shell metacharacter expansion) is involved. But regardless, try and chop the command up. Does it work without the pipe? What about without the ?? in the pathname? What happens if you prepend 'echo' ("echo find ...")? Still hanging? Then you can try it under perl -d - the debugger; perldoc perldebug is your friend.
How do I run a Perl script on multiple input files with the same extension?
perl scriptname.pl file.aspx
I'm looking to have it run for all aspx files in the current directory
Thanks!
In your Perl file,
my #files = <*.aspx>;
for $file (#files) {
# do something.
}
The <*.aspx> is called a glob.
you can pass those files to perl with wildcard
in your script
foreach (#ARGV){
print "file: $_\n";
# open your file here...
#..do something
# close your file
}
on command line
$ perl myscript.pl *.aspx
You can use glob explicitly, to use shell parameters without depending to much on the shell behaviour.
for my $file ( map {glob($_)} #ARGV ) {
print $file, "\n";
};
You may need to control the possibility of a filename duplicate with more than one parameter expanded.
For a simple one-liner with -n or -p, you want
perl -i~ -pe 's/foo/bar/' *.aspx
The -i~ says to modify each target file in place, and leave the original as a backup with an ~ suffix added to the file name. (Omit the suffix to not leave a backup. But if you are still learning or experimenting, that's a bad idea; removing the backups when you're done is a much smaller hassle than restoring the originals from a backup if you mess something up.)
If your Perl code is too complex for a one-liner (or just useful enough to be reusable) obviously replace -e '# your code here' with scriptname.pl ... though then maybe refactor scriptname.pl so that it accepts a list of file name arguments, and simply use scriptname.pl *.aspx to run it on all *.aspx files in the current directory.
If you need to recurse a directory structure and find all files with a particular naming pattern, the find utility is useful.
find . -name '*.aspx' -exec perl -pi~ -e 's/foo/bar/' {} +
If your find does not support -exec ... + try with -exec ... \; though it will be slower and launch more processes (one per file you find instead of as few as possible to process all the files).
To only scan some directories, replace . (which names the current directory) with a space-separated list of the directories to examine, or even use find to find the directories themselves (and then perhaps explore -execdir for doing something in each directory that find selects with your complex, intricate, business-critical, maybe secret list of find option predicates).
Maybe also explore find2perl to do this directory recursion natively in Perl.
If you are on Linux machine, you could try something like this.
for i in `ls /tmp/*.aspx`; do perl scriptname.pl $i; done
For example to handle perl scriptname.pl *.aspx *.asp
In linux: The shell expands wildcards, so the perl can simply be
for (#ARGV) {
operation($_); # do something with each file
}
Windows doesn't expand wildcards so expand the wildcards in each argument in perl as follows. The for loop then processes each file in the same way as above
for (map {glob} #ARGV) {
operation($_); # do something with each file
}
For example, this will print the expanded list under Windows
print "$_\n" for(map {glob} #ARGV);
You can also pass the path where you have your aspx files and read them one by one.
#!/usr/bin/perl -w
use strict;
my $path = shift;
my #files = split/\n/, `ls *.aspx`;
foreach my $file (#files) {
do something...
}