How do I make Perl’s system() format arguments using scientific notation? - perl

From my Perl program, I am trying to run another command written in C using system. The command requires several arguments: a string, a float, two integers, a pair of floats, and another string. I am running this as
my $arg1="electron";
my $arg2=0.511;
# more definitions
system("./fermions $arg1 $arg2 $arg3 $arg4 " .
"$arg5 $arg6 \"string\" > outfile.out");
I need to vary $arg5 to be several different float values. I made it work by setting $arg5="1.0e5" and then running an if-statement in the for-loop to change the value as a string. I would like to be able to do this as floats, and tried
system("./fermions $arg1 $arg2 $arg3 $arg4 " .
"%e $arg6 \"string\" >outfile.out",
$arg5);
but that did not work. Is there another alternative, or is my if-statement option the only one?

If you want to use printf notation (like "%e"), you need to use the Perl sprintf builtin. Otherwise you just end up passing "%e" as a literal argument.

Important: You should always check the return value from Perl’s system function to determine whether the command failed.
Use Perl’s sprintf to format the float values as in the code below. Yes, you may be able to get away with using the command as your format specifier, but you’re likely to get surprising results if the command has stray % characters elsewhere. Using two steps is safer.
#! /usr/bin/env perl
use strict;
use warnings;
my #float_values = (1.0e5, 3.14159, 2.71828);
for my $f (#float_values) {
my $arg5 = sprintf "%e", $f;
system(qq[./fermions $arg5 "string" >> outfile.out]) == 0
or warn "$0: fermions failed";
}
In case you aren’t familiar with the syntax, qq[...] works like a double-quoted string, but the different delimiter means you don’t have to escape double-quotes in your command.
Note that I elided the other arguments for typographical purposes, but you can interpolate them along with the value of $arg5. Another subtle change is the switch to >> for appending rather than > for clobbering.
Using a stand-in for fermions
#! /usr/bin/env perl
$" = "][";
warn "[#ARGV]\n";
the output of the two programs running together is
[1.000000e+05][string]
[3.141590e+00][string]
[2.718280e+00][string]
With respect to terminology, a system call refers to a low-level request for service from the operating system, e.g., open, close, unlink, and so on. Although Perl’s system function makes use of system calls, the two concepts are distinct.
To be really safe about the shell not fudging your command-line arguments, use the techniques described in the “Safe Pipe Opens” section of perlipc. Perl’s system and exec functions bypass the shell when given a list of arguments rather than a single string containing the entire command.
Your situation is a little trickier because you want to redirect the standard output. The code below forks a child, sets the child’s STDOUT to append to outfile.out, and then runs fermion with exec in the child. The parent waits for the child to exit and reports any failure.
#! /usr/bin/env perl
use strict;
use warnings;
my #float_values = (1.0e5, 3.14159, 2.71828);
for my $f (#float_values) {
my $arg5 = sprintf "%e", $f;
my $pid = fork;
if (defined $pid) {
if ($pid == 0) {
my $path = "outfile.out";
open STDOUT, ">>", $path or die "$0: open $path: $!";
exec "./fermions", $arg5, "string" or die "$0: exec: $!";
}
else {
local $!;
my $pid = waitpid $pid, 0;
warn "$0: waitpid: $!" if $pid == -1 && $!;
warn "$0: fermion exited " . ($? >> 8) if $?;
}
}
else {
die "$0: fork: $!";
}
}

Related

How to run shell command in Perl, like Raku?

I have excellent code in Raku:
#!/usr/bin/env perl6
CONTROL {
when CX::Warn {
note $_;
die
}
}
use fatal;
role KeyRequired {
method AT-KEY (\key) {
die "Key {key} not found" unless self.EXISTS-KEY(key);
nextsame
}
}
sub execute ($cmd) {
put $cmd;
my $proc = shell $cmd, :err, :out;
if $proc.exitcode != 0 {
put 'exit code = ' ~ $proc.exitcode;
put 'stderr ' ~ $proc.err.slurp;
put 'stdout ' ~ $proc.out.slurp;
die
}
}
execute "ls *.p6"
I say "excellent" because the Raku version runs a command, returns an exit code, and prints stdout/stderr if needed, and all in an easily-read and easily-understood manner.
Reading through the Perl5 manual for IPC::Run https://metacpan.org/pod/IPC::Run I've come across what appears to be the best Perl5 way of doing this, but I find the methods used there to be much less easily readable and understood than the Raku way of doing things.
Reading through the manual for IPC::Run the best that I can find is:
#!/usr/bin/env perl
use strict;
use warnings FATAL => 'all';
use feature 'say';
use autodie qw(:all);
use IPC::Run qw(run timeout);
sub execute {
my $cmd = shift;
my #cat = ('cat', __FILE__); # Raku doesn't need to split the string into an array
run \#cat, \undef, \my $out, \my $err, timeout( 10 ) or die "cat: $?";
if ($out ne '') {
say "\$out = $out";
}
if ($err ne '') {
say "\$err = $err";
}
}
execute("cat " . __FILE__);
execute("cat __Fle"); #intentionally wrong to produce an error
How can I re-write the Perl5 so that it is as easily read and used as the Raku code?
You've unfairly loaded the Perl 5 example with a lot of extra fluff, and you haven't handled many things in the Raku code. For instance, you output the results in Raku despite what's in the variables, but test the variables in Perl 5.
Your Perl 5 would look more like this:
use v5.30;
use IPC::Run qw(run timeout);
sub execute {
my #command = #_;
run \#command, \undef, \my $out, \my $err, timeout( 10 )
or die "cat: $?";
say "\$out = $out";
say "\$err = $err";
}
execute("cat ", __FILE__);
ikegami offered this version in his pastebin link:
sub execute {
my ($command) = #_;
if (! run $command, \undef, \my $out, \my $err, timeout( 10 ) ) {
say "exit code = $?";
say "stderr $err";
put "stdout $out";
die "Died";
}
}
There's an interesting thing to note in both of those cases. You are assuming an error if the exit code is not zero (and Raku assumes that, which is why you have to worry about not sinking the result). However, many useful programs don't follow that convention. For instance, git merge base uses exit value 1 to mean "not an ancestor" and all exit values higher than 1 to mean an error. The command-line grep is similar. sendmail had exit code 75 to mean that something didn't work out, but it would try again later.
Raku, having an opinion on that, ignores this sort of thing and does not allow you to tell the Proc which exit values it should accept as successful exits. Perl 5 is not so opinionated. Using or die or ! ... is really saying "exit code is not zero", but that's not really a good enough description. In many cases you get away with it, but at least Perl 5 isn't deciding for you. If you expanded the Raku example to check the literal value and decide if that's successful, it will look messy.
But, notice that Raku's shell documentation notes that it's unsafe and that you should use run instead.
For what it's worth, I don't find Raku's interprocess communication all that trustworthy. In many cases, I think its IPC design was neglected. See, for instance, Does changing Perl 6's $*OUT change standard output for child processes? . I have several other IPC questions spread out in bug reports and in Stackoverflow, and almost none of them received a satisfactory answer. Mostly, I think that's because nobody thought about it that much. Granted, Raku is developed by a small team and its a big project, but when it comes to production programming, that's no factor.
Some more Raku shell weirdness:
Which shell does Perl 6's shell() use?

Passing arguments containing spaces from one script to another in Perl

I am trying to pass arguments from one Perl script to another. Some of the arguments contain spaces.
I am reading in a comma-delimited text file and splitting each line on the comma.
my ($jockey, $racecourse, $racenum, $hnamenum, $trainer, $TDRating, $PRO) = split(/,/, $line);
The data in the comma-delimited text file look as follows:
AARON LYNCH,WARRNAMBOOL,RACE 1,DAREBIN (8),ERIC MUSGROVE,B,1
When I print out each variable, from the parent script, they look fine (as above).
print "$jockey\n";
print "$racecourse\n";
print "$racenum\n";
print "$hnamenum\n";
print "$trainer\n";
print "$TDRating\n";
print "$PRO\n";
AARON LYNCH
WARRNAMBOOL
RACE 1
DAREBIN (8)
ERIC MUSGROVE
B
1
When I pass the arguments to the child script (as follows), the arguments are passed incorrectly.
system("perl \"$bindir\\narrative4.pl\" $jockey $racecourse $racenum $hnamenum $trainer $TDRating $PRO");
AARON
LYNCH
WARRNAMBOOL
RACE
1
DAREBIN
(8)
As you can see, $ARGV[0] becomes AARON, $ARGV[1] becomes LYNCH, $ARGV[2] becomes WARRNAMBOOL, and so on.
I have investigated adding quotes to the arguments using qq, quotemeta and Win32::ShellQuote, unfortunately, even if I pass qq{"$jockey"}, the quotes are still stripped before they reach the child script, so they must be protected in some way.
I not sure if either of the aforementioned solutions is the correct but I'm happy to be corrected.
I'd appreciate any suggestions. Thanks in advance.
Note: I am running this using Strawberry Perl on a Windows 10 PC.
Note2: I purposely left out use strict; & use warnings; in these examples.
Parent Script
use Cwd;
$dir = getcwd;
$bin = "bin"; $bindir = "$dir/$bin";
$infile = "FINAL-SORTED-JOCKEY-RIDES-FILE.list";
open (INFILE, "<$infile") or die "Could not open $infile $!\n";
while (<INFILE>)
{
$line = $_;
chomp($line);
my ($jockey, $racecourse, $racenum, $hnamenum, $trainer, $TDRating, $PRO) = split(/,/, $line);
print "$jockey\n";
print "$racecourse\n";
print "$racenum\n";
print "$hnamenum\n";
print "$trainer\n";
print "$TDRating\n";
print "$PRO\n";
system("perl \"$bindir\\narrative4.pl\" $jockey $racecourse $racenum $hnamenum $trainer $TDRating $PRO");
sleep (1);
}
close INFILE;
exit;
Child Script
$passedjockey = $ARGV[0];
$passedracecourse = $ARGV[1];
$passedracenum = $ARGV[2];
$passedhnamenum = $ARGV[3];
$passedtrainer = $ARGV[4];
$passedTDRating = $ARGV[5];
$passedPRO = $ARGV[6];
print "$passedjockey\n";
print "$passedracecourse\n";
print "$passedracenum\n";
print "$passedhnamenum\n";
print "$passedtrainer\n";
print "$passedTDRating\n";
print "$passedPRO\n\n";
That whole double-quoted string that is passed to system is first evaluated and thus all variables are interpolated -- so the intended multi-word arguments become merely words in a list. So in the end the string has a command to run with individual words as arguments.
Then, even if you figure out how to stick which quotes in there just right, so to keep those multi-word arguments "together," there's still a chance of a shell being invoked, in which case those arguments again get broken up into words before being passed to the program.
Instead of all this use the LIST form of system. The first argument is then the name of the program that will be directly executed without a shell (see docs for some details on that), and the remaining arguments are passed as they are to that program.
parent
use warnings;
use strict;
use feature 'say';
my #args = ('first words', 'another', 'two more', 'final');
my $prog = 'print_args.pl';
system($prog, #args) == 0
or die "Error w/ system($prog, #args): $!";
and the invoked print_args.pl
use warnings;
use strict;
use feature 'say';
say for #ARGV;
The #ARGV contains arguments passed to the program at invocation. There's more that can be done to inspect the error, see docs and links in them.†
By what you show you indeed don't need a shell and the LIST form is generally easy to recommend as a basic way to use system, when the shell isn't needed. If you were to need shell's capabilities for something in that command then you'd have to figure out how to protect those spaces.
† And then there are modules for running external programs that are far better than system & Co. From ease-of-use to features and power:
IPC::System::Simple, Capture::Tiny, IPC::Run3, IPC::Run.

How to make perl to keep perform action until the match is found

I am new to Perl and trying to write a code to keep executing an action until the match is found and else give an error.
I am trying to execute a command ps -ef and check if it has got any process running in the name of "box", if there is no process named "box" found, I want to repeat ps -ef command execution until it gets the "box" process and then proceed to next action.
#!/usr/bin/perl -w
open (FH, "ps -ef |") or die "Cannot run the command:$!\n";
$line = "box";
while (<FH>) {
if (/$line/i) { next; }
else {
print ("ps -ef |") or die "Cannot run the command:$!\n");
}
}
close (FH);
You need to use an infinite loop and an exit-condition. Your condition is that the ps -ef command contains the word box. There is no need to open a pipe to that command explicitly, you can just run it as a system call with the qx operator (same as backticks).
use strict;
use warnings;
my $ps;
PS: while (1) {
$ps = qx/ps -ef/;
last PS if $ps =~ m/box/i;
print '.'; # do something in every run
}
print $ps;
As this has come up in the comments as well as in in AdrianHHH's answer: it might make sense to sleep after every run to make sure you don't hog the CPU. Depending on the nature of the process you are looking for, either the sleep builtin or usleep from Time::HiRes might be appropriate. The latter let's your program rest for milliseconds, while the builtin only works with full seconds. These might be too long if the target box process is very quick.
Explanation of your code:
Note that you have some issues in your implementation. I'll explain what your code does. This is taken from the question, comments are mine.
#!/usr/bin/perl -w
# open a filehandle to the ps command
open (FH, "ps -ef |") or die "Cannot run the command:$!\n";
$line = "box";
# read the output of one run line by line, for each line execute
# the block
while (<FH>) {
# if there is 'box' case-insensitive, skip the line
if (/$line/i) { next; }
else {
# else output (not run!) the command
print ("ps -ef |") or die "Cannot run the command:$!\n");
}
}
close (FH);
After it went through all the lines of the output of your command once it will stop.
I would recommend using pgrep(1) instead of ps because it lets you do a more granular search. With ps -ef, you potentially have to deal with cases like:
boxford 6254 6211 0 08:23 pts/1 00:00:00 /home/boxford/box --bounding-box=123
It's hard to tell if you're matching a process being run by a user with box in their username, a process that has box somewhere in its path, a process named box, or a process with box somewhere in its argument list.
pgrep, on the other hand, lets you match against just the process name or the full path, a specific user or users, and more. The following prints a message when a process named box appears (this looks for an exact match, so it will not match processes named dropbox, for example):
use strict;
use warnings;
use 5.010;
use String::ShellQuote qw(shell_quote);
sub is_running {
my ($proc) = #_;
my $cmd = 'pgrep -x ' . shell_quote($proc) . ' >/dev/null 2>&1';
system($cmd);
if ($? == -1) {
die "failed to execute pgrep: $!";
}
elsif ($? & 127) {
die "pgrep died with signal ", $? & 127;
}
else {
my $status = $? >> 8;
die "pgrep exited with error: exit status $status" if $status > 1;
return $status == 0;
}
}
my $proc = 'box';
until ( is_running($proc) ) {
sleep 1;
}
say "Process '$proc' is running";
Note that pgrep doesn't have a case-insensitive flag, probably because process names in *nix are almost always lowercase. If you really need to do a case-insensitive match, you can pass [Bb][Oo][Xx] to the is_running function.
The ps command outputs the current list of processes, then it completes. The code in the question reads that output. Suppose that the first ps command that is executed does not contain the wanted line, then there is nothing in the code in the question to run the ps command again.
The next statement in the question makes the script move on to the next line in the output from ps, not to rerun the command. The else print ... after the next will probably be executed for the first line of the output from ps. The outcome is that the print is run for each line in the ps output that does not have the wanted text and that the next command has no significant effect. In the code print ... or die "..." the or die "..." part is not very useful, the print is unlikely to fail and even if it did the die message would be wrong.
Perhaps you should write some code in the following style. Here the ps is run repeatedly until the wanted text is found. Note the sleep call, without that the script will keep running without pause, possibly preventing real work or at least slowing it down.
# This code is not tested.
use strict;
use warnings;
my $found_wanted_line = 0; # Boolean, set to false
my $line = "box";
while ( ! $found_wanted_line ) {
open (my $FH, "ps -ef |") or die "Cannot run the command:$!\n";
while (<$FH>) {
if (/$line/i) {
$found_wanted_line = 1; # Boolean, set to true
last;
}
}
close ($FH);
if ( ! $found_wanted_line )
sleep 2; # Pause for 2 seconds, to prevent this script hogging the CPU.
}
}

Capture the output of Perl's 'system()'

I need to run a shell command with system() in Perl. For example,
system('ls')
The system call will print to STDOUT, but I want to capture the output into a variable so that I can do future processing with my Perl code.
That's what backticks are for. From perldoc perlfaq8:
Why can't I get the output of a command with system()?
You're confusing the purpose of system() and backticks (``). system()
runs a command and returns exit status information (as a 16 bit value:
the low 7 bits are the signal the process died from, if any, and the
high 8 bits are the actual exit value). Backticks (``) run a command
and return what it sent to STDOUT.
my $exit_status = system("mail-users");
my $output_string = `ls`;
See perldoc perlop for more details.
IPC::Run is my favourite module for this kind of task. Very powerful and flexible, and also trivially simple for small cases.
use IPC::Run 'run';
run [ "command", "arguments", "here" ], ">", \my $stdout;
# Now $stdout contains output
Simply use similar to the Bash example:
$variable=`some_command some args`;
That's all. Notice, you will not see any printings to STDOUT on the output because this is redirected to a variable.
This example is unusable for a command that interact with the user, except when you have prepared answers. For that, you can use something like this using a stack of shell commands:
$variable=`cat answers.txt|some_command some args`;
Inside the answers.txt file you should prepare all answers for some_command to work properly.
I know this isn't the best way for programming :) But this is the simplest way how to achieve the goal, specially for Bash programmers.
Of course, if the output is bigger (ls with subdirectory), you shouldn't get all output at once. Read the command by the same way as you read a regular file:
open CMD,'-|','your_command some args' or die $#;
my $line;
while (defined($line=<CMD>)) {
print $line; # Or push #table,$line or do whatever what you want processing line by line
}
close CMD;
An additional extended solution for processing a long command output without extra Bash calling:
my #CommandCall=qw(find / -type d); # Some example single command
my $commandSTDOUT; # File handler
my $pid=open($commandSTDOUT),'-|'); # There will be an implicit fork!
if ($pid) {
#parent side
my $singleLine;
while(defined($singleline=<$commandSTDOUT>)) {
chomp $line; # Typically we don't need EOL
do_some_processing_with($line);
};
close $commandSTDOUT; # In this place $? will be set for capture
$exitcode=$? >> 8;
do_something_with_exit_code($exitcode);
} else {
# Child side, there you really calls a command
open STDERR, '>>&', 'STDOUT'; # Redirect stderr to stdout if needed. It works only for child - remember about fork
exec(#CommandCall); # At this point the child code is overloaded by an external command with parameters
die "Cannot call #CommandCall"; # Error procedure if the call will fail
}
If you use a procedure like that, you will capture all procedure output, and you can do everything processing line by line. Good luck :)
I wanted to run system() instead of backticks because I wanted to see the output of rsync --progress. However, I also wanted to capture the output in case something goes wrong depending on the return value. (This is for a backup script). This is what I am using now:
use File::Temp qw(tempfile);
use Term::ANSIColor qw(colored colorstrip);
sub mysystem {
my $cmd = shift; # "rsync -avz --progress -h $fullfile $copyfile";
my ($fh, $filename) = tempfile();
# http://stackoverflow.com/a/6872163/2923406
# I want to have rsync progress output on the terminal AND capture it in case of error.
# Need to use pipefail because 'tee' would be the last cmd otherwise and hence $? would be wrong.
my #cmd = ("bash", "-c", "set -o pipefail && $cmd 2>&1 | tee $filename");
my $ret = system(#cmd);
my $outerr = join('', <$fh>);
if ($ret != 0) {
logit(colored("ERROR: Could not execute command: $cmd", "red"));
logit(colored("ERROR: stdout+stderr = $outerr", "red"));
logit(colored("ERROR: \$? = $?, \$! = $!", "red"));
}
close $fh;
unlink($filename);
return $ret;
}
# And logit() is something like:
sub logit {
my $s = shift;
my ($logsec, $logmin, $loghour, $logmday, $logmon, $logyear, $logwday, $logyday, $logisdst) = localtime(time);
$logyear += 1900;
my $logtimestamp = sprintf("%4d-%02d-%02d %02d:%02d:%02d", $logyear, $logmon+1, $logmday, $loghour, $logmin, $logsec);
my $msg = "$logtimestamp $s\n";
print $msg;
open LOG, ">>$LOGFILE";
print LOG colorstrip($msg);
close LOG;
}

how to source a shell script [environment variables] in perl script without forking a subshell?

I want to call "env.sh " from "my_perl.pl" without forking a subshell. I tried with backtics and system like this --> system (. env.sh) [dot space env.sh] , however wont work.
Child environments cannot change parent environments. Your best bet is to parse env.sh from inside the Perl code and set the variables in %ENV:
#!/usr/bin/perl
use strict;
use warnings;
sub source {
my $name = shift;
open my $fh, "<", $name
or die "could not open $name: $!";
while (<$fh>) {
chomp;
my ($k, $v) = split /=/, $_, 2;
$v =~ s/^(['"])(.*)\1/$2/; #' fix highlighter
$v =~ s/\$([a-zA-Z]\w*)/$ENV{$1}/g;
$v =~ s/`(.*?)`/`$1`/ge; #dangerous
$ENV{$k} = $v;
}
}
source "env.sh";
for my $k (qw/foo bar baz quux/) {
print "$k => $ENV{$k}\n";
}
Given
foo=5
bar=10
baz="$foo$bar"
quux=`date +%Y%m%d`
it prints
foo => 5
bar => 10
baz => 510
quux => 20110726
The code can only handle simple files (for instance, it doesn't handle if statements or foo=$(date)). If you need something more complex, then writing a wrapper for your Perl script that sources env.sh first is the right way to go (it is also probably the right way to go in the first place).
Another reason to source env.sh before executing the Perl script is that setting the environment variables in Perl may happen too late for modules that are expecting to see them.
In the file foo:
#!/bin/bash
source env.sh
exec foo.real
where foo.real is your Perl script.
You can use arbitrarily complex shell scripts by executing them with the relevant shell, dumping their environment to standard output in the same process, and parsing that in perl. Feeding the output into something other than %ENV or filtering for specific values of interest is prudent so you don't change things like PATH that may have interesting side effects elsewhere. I've discarded standard output and error from the spawned shell script although they could be redirected to temporary files and used for diagnostic output in the perl script.
foo.pl:
#!/usr/bin/perl
open SOURCE, "bash -c '. foo.sh >& /dev/null; env'|" or
die "Can't fork: $!";
while(<SOURCE>) {
if (/^(BAR|BAZ)=(.*)/) {
$ENV{$1} = ${2} ;
}
}
close SOURCE;
print $ENV{'BAR'} . "\n";
foo.sh:
export BAR=baz
Try this (unix code sample):
cd /tmp
vi s
#!/bin/bash
export blah=test
vi t
#!/usr/bin/perl
if ($ARGV[0]) {
print "ENV second call is : $ENV{blah}\n";
} else {
print "ENV first call is : $ENV{blah}\n";
exec(". /tmp/s; /tmp/t 1");
}
chmod 777 s t
./t
ENV first call is :
ENV second call is : test
The trick is using the exec to source your bash script first and then calling your perl script again with an argument so u know that you are being called for a second time.