I am trying to run an executable perl file that copies a directory to another location and then removes every file in that new location except for those ending with .faa and .tsv. Here's the code:
#!/usr/bin/perl
use strict;
use warnings;
my $folder = $ARGV[0];
system ("cp -r ~/directoryA/$folder/ ~/directoryB/");
chdir "~/directoryB/$folder";
# Remove everything except for .faa and .tsv files
system ("rm !\(*.faa|*.tsv\)");
Regardless of whether or not I escape the parenthesis, I get the error:
sh: 1: Syntax error: "(" unexpected
and it didn't remove any files. The location of the perl file is ~/bin, and I'd like to avoid changing the #!/usr/bin/perl line since several computers will be using this script.
This is a little beyond my knowledge, as I only know basic scripting, but does anyone know why this is happening?
This entire program is much simpler without the use of shell commands
I would write this, which copies only the wanted file types in the first place. I assume there are no nested directories to be copied
use strict;
use warnings;
use File::Copy 'copy';
my ($folder) = #ARGV;
while ( my $file = glob "~/directoryA/$folder/*.{faa,tsv}" ) {
copy $file, '~/directoryB';
}
It's complaining about this line:
system ("rm !\(*.faa|*.tsv\)");
as even if you get the quoting of the shell metacharacters right, is pretty obtuse and does not, I believe, erase all files that don't end in .faa or .tsv.
Perl is up to the latter task.
unlink grep { -f $_ && !/[.]faa$/ && !/[.]tsv$/ } glob("*")
is one of several ways.
Related
I want to rename *.DIF files to *.SUC files
But the following script is giving "sh: bad substitution" error
I cannot use "rename" becuase my OS is not Linux.
$com="for i in *.DIF; do mv \$i \${i/DIF/SUC}; done;";
print $com."\n";
print `$com`;
Output :
for i in *.DIF; do mv $i ${i/DIF/SUC}; done;
sh: bad substitution
If you are having problems with Perl's rename, use File::Copy, a platform-independent module for moving and copying files. It is a core module, so it should be already part of your Perl installation.
If the system command works when you enter it in the shell, but not in Perl, the most likely reason is that Perl isn't using the shell you expect. We would need more information about your system to be sure, though.
There's no need to shell out for file operations that you can easily do within Perl.
The following renames all of your .dif extension files as .suc.
use strict;
use warnings;
use File::Copy qw(move);
move($_, s/dif$/suc/r) for glob('*.dif');
be default perl was using sh, instead of bash, which allows {//}
this question helped.
Finally I used :
open my $bash_handle, '| bash' or die "Cannot open bash: $!";
print $bash_handle 'for i in *.SUC; do mv $i ${i/SUC/DIF}; done;';
So I'm taking a Perl programming class and our teacher gave us our first assignment with very little talk about how to actually program perl. Here's exactly what our teacher assigned:
"You should write a script (you may name you script whatever you deem appropriate) that accepts 3 filenames as arguments. The first filename corresponds to the program source code written in C++. The second filename corresponds to an input file that is to be used by the C++ program listed as the first filename. The third filename corresponds to a text file that contains the expected, correct output for the program in question. A directory path may be provided with any of the filenames.
If the program doesn’t require an input file, the second parameter on the command line should be the filename "/dev/null".
All of the files - the program source, the input file, and expected output file - should be copied to a scratch test directory before any of them are used by the script so that there is little chance that the originals will be modified by the test procedure. If the scratch directory doesn’t exist, your script should create one as a subdirectory of the current working directory.
Your script should then compile and link the scratch copy of the source using the GNU g++ compiler. The script should then run the program – saving the output to a temporary file stored in the scratch test directory.
After running the program, your script should then use the UNIX command diff to compare the actual output generated in the previous step with the expected output file and report either that the output conforms to specifications or report any differences as reported by the diff program.
After completion, your script should remove all of the temporary copies and scratch files. Do not remove the original program, the original input file, the original expected output file, or the scratch directory itself."
I have this so far:
#!/usr/bin/perl -w
use strict;
my ($line, $program, $input, $output);
print "Give the program, input, and standart output for testing. ";
$line = <>;
chomp $line;
($program, $input, $output) = split/\s+/, $line; # split/\s+/ is to separate spaces from the input
my($o_test) = $output + "_test";
print "$program ";
print "$input ";
print "$output ";
system("mkdir test_scratch") == 0
or die "failed to create test_scratch. exiting...."
system("cp $program, /test_scratch/"); # error
system("cp $input, /test_scratch/");
system("cp $output, /test_scratch/");
system("cd test_scratch");
system("g++ $program");
system("chmod +x a.out");
system("./a.out < $input > $o_test");
my($DIFF) = system("diff $output $o_test") # error
if[ $DIFF != ""]
print ("Output conforms to specifications."); # error
then
print ("$DIFF");
system("cd ..");
I'm getting errors at the # in code. I don't even know how to do the "/dev/null". I've spent a lot of time looking things up online and searching stackoverflow, but I just don't know what else to do. I realize this is an extremely long question but I don't know what else to do. Thank you for ANY help you can give me.
There are several modules which can help you here. The first one I would recommend is ExtUtils::CBuilder which can manage a build process for you. Then you might also use File::Copy for moving things intot the temporary folder, and even File::chdir for managing the working directory. Since the prof specifies that you should use diff perhaps you should, but there are modules which do that task, or you could use Test::More to check that the output is what is expected.
Just for future reference, this is how I would accomplish a similar task (don't keep temp dir, don't need diff):
#!/usr/bin/env perl
use strict;
use warnings;
use File::chdir;
use File::Temp;
use File::Copy;
use File::Basename;
use File::Slurp;
use ExtUtils::CBuilder;
use Test::More tests => 1;
die "Not enough inputs\n" unless #ARGV >= 3;
# create a temporary directory
my $temp = File::Temp->newdir;
# copy all arguments to that temporary directory
copy $_, $temp for #ARGV;
# store only the filename (not path) of each argument
my ($cpp_file, $in_file, $expected_file) = map { scalar basename $_ } #ARGV;
# change working directory to temporary one (via File::chdir)
local $CWD = $temp;
# build the executable
my $builder = ExtUtils::CBuilder->new(config => {cc => 'g++'});
my $obj = $builder->compile( source => $cpp_file, 'C++' => 1 );
my $exe = $builder->link_executable( objects => 'hello.o' );
# run the executable
my $output = `./$exe $in_file`;
# read in the expected file
my $expected = read_file $expected_file;
# test the resulting output
is $output, $expected, 'Got expected output from executable';
Note that you may need to be careful with newlines on the output and expected files.
I'm getting errors at the # in code.
The easy fix for this error is to remove the comma. The standard Linux cp command doesn't take a comma between the source and destination. You might also consider File::Copy to copy files in Perl.
I don't even know how to do the "/dev/null".
/dev/null is a standard Linux file(*), which points to emptiness. You can write bytes to this file, and they disappear. You can try to read bytes from this file, and nothing's ever there.
This leaves you with two options:
Just use /dev/null directly, and the program will be unable to read anything from an empty byte stream.
Check if the input file is equal to /dev/null, and don't give the C++ program an input value if the input is /dev/null.
Either method would work. If you copy /dev/null to a directory, you get a new zero-length file called 'null', which is a perfectly valid empty input file for the C++ program.
(*) No, /dev/null is not a file in the sense you're used to if you come from the Windows world. It doesn't exist on disk - never has and never will. But the classic Unix philosophy is to make every data source a file, even if that file doesn't exist on disk. As an example, see the Linux /proc filesystem, which allows you to see information about CPUs, processes, etc. in a directory tree structure. That philosophy held up somewhat until the sockets API took a completely different route. If you're looking for an operating system that does make basically everything into a file, including network connections and the screen, look up Plan 9. I would not recommend doing your homework assignments on Plan 9 and expecting them to work on Linux, though.
N.B. Don't forget to check return codes from system, since if g++ fails to compile the C++ program there will be no way to run a compiled program that doesn't exist.
Say I have this perl "program" called simple.pl:
#!/usr/bin/perl
use xyz; # xyz is bogus and doesn't exist
And I also have this "program", called simple2.pl:
#!/usr/bin/perl
system("simple.pl");
my $abc = `simple.pl`;
printf("abc %s\n", $abc);
for both system and backtick, I get this message:
Can't exec "simple.pl": No such file or directory at scripts/perl/simple2.pl line 7.
Can't exec "simple.pl": No such file or directory at scripts/perl/simple2.pl line 9.
Not very useful for the user calling simple2.pl. Is there a way to get a more useful message?
Note. simple.pl does exist in the current directory. The real problem is that simple.pl doesn't compile. simple2 responds by saying simple doesn't exist. it's a misleading message.
If I had a way to even capture the compile message that would be a start.
This means system couldn't find an executable named "simple.pl" on your PATH. If your simple.pl is in the current directory, you could try to change "simple.pl" to "./simple.pl".
Actually, I don't see how to make this message more descriptive. If you were perl, how would you report this error?
BTW, I wouldn't try to run "simple2.pl" from inside of simple2.pl :)
Yes, check to see if the file exists and is executable, and if it isn't, print a more descriptive message.
unless (-ex $filename) {
print "I am unable to execute file $filename.";
}
If perl say it can't find the file, then it can't find the file. And the problem is more your code. Look at this example.
sidburn#sid:~/perl$ cat test.pl
#!/usr/bin/env perl
use strict;
use warnings;
use xyz;
sidburn#sid:~/perl$ cat test2.pl
#!/usr/bin/env perl
use strict;
use warnings;
system('test.pl');
sidburn#sid:~/perl$ cat test3.pl
#!/usr/bin/env perl
use strict;
use warnings;
system('./test.pl');
If you execute test2.pl you get:
sidburn#sid:~/perl$ ./test2.pl
Can't exec "test.pl": No such file or directory at ./test2.pl line 4.
If you execute test3.pl you get:
sidburn#sid:~/perl$ ./test3.pl
Can't locate xyz.pm in #INC (#INC contains: /home/sidburn/perl510/lib/5.10.1/i686-linux /home/sidburn/perl510/lib/5.10.1 /home/sidburn/perl510/lib/site_perl/5.10.1/i686-linux /home/sidburn/perl510/lib/site_perl/5.10.1 .) at ./test.pl line 4.
BEGIN failed--compilation aborted at ./test.pl line 4.
If you don't provide a relative or absolute path then perl lookup the command in your $PATH environment variable. If it is not there it can't find the file.
You need to provide "./" if it is in the current directory. But note "current directory" doesn't mean the directory where your script relies.
If you want the later then you probably want to do a
use FindBin;
with this you can do something like this:
#!/usr/bin/env perl
use strict;
use warnings;
use FindBin;
use File::Spec::Functions;
my $exe = catfile($FindBin::RealBin, 'test.pl');
print $exe, "\n";
system($exe);
if you want to check if system returns correctly, you need to check the return value from the system() command or $? later that holds the value.
if ( $? != 0 ) {
die "Cannot execute $exe.\n";
}
if you want to suppress messages from your program you need to redirect STDOUT, STDERR before starting your program with system().
Or use something like IPC::System::Simple
Or IPC::Open3 (in the core).
Bonus points for enabling the warnings pragma! Have an upvote!
You want to use backticks or qx// to capture the output of an external program, not system. To substitute your own error message that will make sense to your users (more points for you!), then you might do something as in
#! /usr/bin/perl
use strict;
use warnings;
no warnings 'exec';
chomp(my $abc = `simple2.pl`);
if ($? == 0) {
printf("abc %s\n", $abc);
}
else {
die "$0: unable to calculate abc\n";
}
In case you're unfamiliar, $? is
$CHILD_ERROR
$?
The status returned by the last pipe close, backtick command, successful call to wait or waitpid, or from the system operator.
When $? is zero, it indicates success.
Remember that the warnings pragma is lexical, so rather than disabling the warning for the whole program, you might do it for just one sub:
sub calculate_abc {
no warnings 'exec';
# ...
}
If you are trying to execute something you know is a Perl script, why not invoke the interpreter directly rather than dealing with the system knowing how to execute the file?
my $file = 'simple.pl';
-e $file or die "file '$file' not found";
system "perl $file";
# or
print `perl $file`;
to run with the same installation of perl that is running your current script:
system "$^X $file"; # or `$^X $file`
$^X is a special Perl variable that contains the file name of the running interpreter.
I had the exact same issue and figured out that perl wasn't installed. So the bash script was trying to execute the perl without an interpreter.
ls /usr/bin/perl
Try specifying the full path to the "simple.pl" file.
How do I run a Perl script on multiple input files with the same extension?
perl scriptname.pl file.aspx
I'm looking to have it run for all aspx files in the current directory
Thanks!
In your Perl file,
my #files = <*.aspx>;
for $file (#files) {
# do something.
}
The <*.aspx> is called a glob.
you can pass those files to perl with wildcard
in your script
foreach (#ARGV){
print "file: $_\n";
# open your file here...
#..do something
# close your file
}
on command line
$ perl myscript.pl *.aspx
You can use glob explicitly, to use shell parameters without depending to much on the shell behaviour.
for my $file ( map {glob($_)} #ARGV ) {
print $file, "\n";
};
You may need to control the possibility of a filename duplicate with more than one parameter expanded.
For a simple one-liner with -n or -p, you want
perl -i~ -pe 's/foo/bar/' *.aspx
The -i~ says to modify each target file in place, and leave the original as a backup with an ~ suffix added to the file name. (Omit the suffix to not leave a backup. But if you are still learning or experimenting, that's a bad idea; removing the backups when you're done is a much smaller hassle than restoring the originals from a backup if you mess something up.)
If your Perl code is too complex for a one-liner (or just useful enough to be reusable) obviously replace -e '# your code here' with scriptname.pl ... though then maybe refactor scriptname.pl so that it accepts a list of file name arguments, and simply use scriptname.pl *.aspx to run it on all *.aspx files in the current directory.
If you need to recurse a directory structure and find all files with a particular naming pattern, the find utility is useful.
find . -name '*.aspx' -exec perl -pi~ -e 's/foo/bar/' {} +
If your find does not support -exec ... + try with -exec ... \; though it will be slower and launch more processes (one per file you find instead of as few as possible to process all the files).
To only scan some directories, replace . (which names the current directory) with a space-separated list of the directories to examine, or even use find to find the directories themselves (and then perhaps explore -execdir for doing something in each directory that find selects with your complex, intricate, business-critical, maybe secret list of find option predicates).
Maybe also explore find2perl to do this directory recursion natively in Perl.
If you are on Linux machine, you could try something like this.
for i in `ls /tmp/*.aspx`; do perl scriptname.pl $i; done
For example to handle perl scriptname.pl *.aspx *.asp
In linux: The shell expands wildcards, so the perl can simply be
for (#ARGV) {
operation($_); # do something with each file
}
Windows doesn't expand wildcards so expand the wildcards in each argument in perl as follows. The for loop then processes each file in the same way as above
for (map {glob} #ARGV) {
operation($_); # do something with each file
}
For example, this will print the expanded list under Windows
print "$_\n" for(map {glob} #ARGV);
You can also pass the path where you have your aspx files and read them one by one.
#!/usr/bin/perl -w
use strict;
my $path = shift;
my #files = split/\n/, `ls *.aspx`;
foreach my $file (#files) {
do something...
}
This question already has answers here:
How do I read in the contents of a directory in Perl?
(9 answers)
Closed 7 years ago.
I would like to execute ls in a Perl program as part of a CGI script. For this I used exec(ls), but this does not return from the exec call.
Is there a better way to get a listing of a directory in Perl?
Exec doesn't return at all. If you wanted that, use system.
If you just want to read a directory, open/read/close-dir may be more appropriate.
opendir my($dh), $dirname or die "Couldn't open dir '$dirname': $!";
my #files = readdir $dh;
closedir $dh;
#print files...
Everyone else seems stuck on the exec portion of the question.
If you want a directory listing, use Perl's built-in glob or opendir. You don't need a separate process.
exec does not give control back to the perl program.
system will, but it does not return the results of an ls, it returns a status code.
tick marks `` will give you the output of our command, but is considered by some as unsafe.
Use the built in dir functions.
opendir, readdir, and so on.
http://perldoc.perl.org/functions/opendir.html
http://perldoc.perl.org/functions/readdir.html
In order to get the output of a system command you need to use backticks.
$listing = `ls`;
However, Perl is good in dealing with directories for itself. I'd recommend using File::Find::Rule.
Yet another example:
chdir $dir or die "Cannot chroot to $dir: $!\n";
my #files = glob("*.txt");
Use Perl Globbing:
my $dir = </dir/path/*>
EDIT: Whoops! I thought you just wanted a listing of the directories... remove the 'directory' call to make this script do what you want it to...
Playing with filehandles is the wrong way to go in my opinion. The following is an example of using File::Find::Rule to find all the directories in a specified directory. It may seem like over kill for what you're doing, but later down the line it may be worth it.
First, my one line solution:
File::Find::Rule->maxdepth(1)->directory->in($base_dir);
Now a more drawn out version with comments. If you have File::Find::Rule installed you should be able to run this no problem. Don't fear the CPAN.
#!/usr/bin/perl
use strict;
use warnings;
# See http://search.cpan.org/~rclamp/File-Find-Rule-0.32/README
use File::Find::Rule;
# If a base directory was not past to the script, assume current working director
my $base_dir = shift // '.';
my $find_rule = File::Find::Rule->new;
# Do not descend past the first level
$find_rule->maxdepth(1);
# Only return directories
$find_rule->directory;
# Apply the rule and retrieve the subdirectories
my #sub_dirs = $find_rule->in($base_dir);
# Print out the name of each directory on its own line
print join("\n", #sub_dirs);
I would recommend you have a look at IPC::Open3. It allows for far more control over the spawned process than system or the backticks do.
On Linux, I prefer find:
my #files = map { chomp; $_ } `find`;