I thought this must be simple, but I haven't found a viable solution.
The problem is as simple as this: I want to execute a system command and capture the output in Perl variable. The command is specified in Perl array (containing command and parameters, e.g. #cmd = ('mycmd', '-opt1', 'arg1', 'val1')).
I don't want to use forking, i.e. open(FROM_KID, '-|') is not an option. I know that if I had the command in a string I can achieve this with backticks. So perhaps this problem reduces to converting #cmd array into a string. In my case, the command arguments can contain spaces.
Is there a simple way to convert #cmd array into string that can be used with backticks, but such that all arguments are properly quoted? Also ideally without using any external libraries.
Thanks!!
You may be looking for String::ShellQuote. Note that only Bourne shell quoting is supported.
But backticks also perform an implicit fork, and if they in any way differ from the implicit fork of pipe open, I for one never noticed. :-\
Related
Sorry, this is pretty basic, and I suspect a duplicate, but after some searching I'm coming up empty:
Given the following script:
#!/usr/bin/perl
use strict;
use warnings;
use IPC::Run3;
my $stdout2;
print $ARGV[0];
print "\n";
my #cmd1 = split /\s+/, $ARGV[0] ;
run3 (\#cmd1, \undef, \$stdout2, \$stdout2);
print $stdout2
And running it like so:
£ perl comp.pl "md5sum *(.)"
md5sum *(.)
md5sum: '*(.)': No such file or directory
Fair enough. The *(.) isn't being intrepreted by the shell and probably most would consider this a feature. But I would like it to be intepreted by the current shell (or zsh specifically would be fine).
The question is how I can do this without complicating the shell command to run the perl script.
Prepending "zsh" and "-c" to cmd1 is ok if that's a reasonable way to do it. It just seems like...it isn't.
My intention is also to pass slightly more complex commands to this script eventually, like so:
perl comp.pl 'md5sum *(.)' 'ssh remoteHost "md5sum *(.)"'
I have no objection to non-perl answers to the problem you can probably infer I'm trying to solve (I suspect rsync could do this) but I'm primarily interested in solving this through Perl as there'll eventually be business-specific logic in this comparison.
EDIT
I tried various forms of:
my $cmd = $ARGV[0];
run3 (\$cmd, \undef, \$stdout2, \$stdout2);
the documentation seems to think this would be ok, but I get:
Not an ARRAY reference at /usr/local/share/perl/5.22.1/IPC/Run3.pm line 320.
The IPC::Run3 docs say that one can pass a string instead of an arrayref for the command
run3($cmd, $stdin, $stdout, $stderr, \%options)
...
$cmd
Usually $cmd will be an ARRAY reference and the child is invoked via
system #$cmd;
But $cmd may also be a string in which case the child is invoked via
system $cmd;
In this case the string $cmd is passed to the shell if it contains shell metacharacters. So take input without splitting it, $cmd = $ARGV[0], or join it after validation, $cmd = join ' ', #cmd;
Even in general this is not the preferred way, and the docs warn to see system for "pitfalls" of it.
Things are yet much worse here since you'd be passing user input directly for execution! Never mind possible nefarious intents, just think of what a good typo can do. Even without that, there is simply a difference between typing a command at the terminal and passing it to a script, which may edit it, may get modified, pick up bugs, etc.
If nothing else, I'd urge to add code for substantial checks of submitted input. An analysis may involve identifying the known and accepted metacharacters while suitably quoting parts of input that shouldn't be interpreted, for example using String::ShellQuote.
But I'd really suggest to reconsider the design, so to not submit complete commands to the script. Rather, specify with keywords what should happen. Things like globbing (assembling a file list) are done from Perl really nicely and with a lot of control. Do outside only what is necessary; generally there'll be no need for the shell then.
I am writing a large Perl script, which needs to utilize other existing Perl scripts. The problem is the main script needs to reference many different scripts from different folders. For example the main script would be contained in:
/perl/programs/io
It may need to run a script which is stored in:
/perl/programs/tools
Note that there are other orthogonal folders besides tools so I need to be able to access any of them on the fly.
Currently this is what I got:
my $mynumber = '../tools/convert.pl bin2dec 1011';
In theory it should move back from the io directory then enter the appropriate tool directory and call the convert.pl script while passing it the parameters.
All this does is store the string in the single quotes to $myNumber.
I like to assign the output of a command to an array so I can loop through the array to find error or other messages. For example if I'm making a zip file to email to someone I want to check to see if the zip program had any errors before I continue to make and send the email.
#msgs = `zip -f myfile.zip *.pl`; # Use backticks
You can also assign the output to a scalar:
$msg = `ls -al *.pl`; # Use backticks
To run any system command or script all you have to do is use `backticks`. From observing another programer's perl code, I misread these strange quotes for 'single quotes'.
backticks are also nice because they return the text in STDOUT to your perl script so that the output can be assigned to a variable, something I have found impossible if using system("");
The similar question answer does not work with my version of perl. The line
use IPC::System::Simple qw(system capture);
throws some errors. However just using system works, like this:
my $mynumber = system($^X, "../tools/convert.pl", 'bin2dec', '1011');
I can use the above without setting equal to something to execute scripts which return no value and are only sent arguments.
This seems to be the easiest way to do what I need to and the entire programs folder can be moved anywhere and it will still work as no parent directories above programs are used.
I am working on transcribing an outdated file from perl to python and got caught up with some perl syntax.
my $jobID = `echo \$JOBID`;
chomp($jobID);
unless ($jobID) {
print "Please specify a job\n";
exit;
}
Thus far, I have been able to transcribe all of the command-line parsing extremely easily but am very fairly stuck here with what exactly this code is doing (specifically the echo within the declaration on line 1).
Within the perl script cmd-line parsing options - that enables one to set the jobID - it states that "default = $JOBID". So my assumption is that the first line in this code simply sets this default jobID until one is specified.
If this is the case why would you need to use echo within the variable default declaration? Is this good practice in perl or am I missing a larger picture?
I have tried searching low and high for this but can't seem to google ninja my way to anything useful.
Any help on the 'echo' would be greatly appreciated (or any good reads on this as well)!
This is one way to get a value from a shell variable. The backticks (`) run the shell command and give you the output. So the echo is running inside of a shell and in this case it just returns the one shell variable. A cleaner way to get this in Perl is to use %ENV like so:
my $jobID = $ENV{'JOBID'};
This also removes the need for chomp, avoids creating an extra process, and is much more efficient.
It is evaluating an environment variable named $JOBID and storing the result in $jobID, which (as duskwuff points out) is better accomplished using $ENV{JOBID}.
The backticks around the echo \$JOBID tell Perl to invoke the specified command in a subshell and return the output of the invoked command.
How can i call the subroutine of the shell script and pass certain parameters to it? something like below ?
#!/usr/bin/perl
### need something like this
source file.sh
routine; # <-- this is supposed to be part of file.sh which is called
# from perl script and some parameter are passed to it
No. They are two different languages. All you can do is call a shell script from Perl as a child process, using, for example, system() or qx().
Write your program in one language, Perl or shell, don't try to mix them.
OK, it is possible to export a function from a shell and then parse and execute it in Perl, but it is a lot of work, insecure, and usually not worth the effort.
A hack around this problem would be to isolate the content of routine out of file.sh into subfile.sh - then you can do this in Perl:
#cmdargs=('subfile.sh', $arg1, "arg2");
system(#cmdargs);
The first list element is the command, the second the value of a Perl variable being passed as an argument to subfile.sh; the third is a literal being passed as an argument to subfile.sh.
Avoid maintenance issues between the copy of routine's contents in subfile.sh and file.sh by writing a wrapper for subfile.sh in file.sh and simply call it with the appropriate arguments, like any other shell command, wherever you're calling routine in file.sh.
I think that would work.
When using system() calls in Perl, do you have to escape the shell args, or is that done automatically?
The arguments will be user input, so I want to make sure this isn't exploitable.
If you use system $cmd, #args rather than system "$cmd #args" (an array rather than a string), then you do not have to escape the arguments because no shell is invoked (see system). system {$cmd} $cmd, #args will not invoke a shell either even if $cmd contains metacharacters and #args is empty (this is documented as part of exec). If the args are coming from user input (or other untrusted source), you will still want to untaint them. See -T in the perlrun docs, and the perlsec docs.
If you need to read the output or send input to the command, qx and readpipe have no equivalent. Instead, use open my $output, "-|", $cmd, #args or open my $input, "|-", $cmd, #args although this is not portable as it requires a real fork which means Unix only... I think. Maybe it'll work on Windows with its simulated fork. A better option is something like IPC::Run, which will also handle the case of piping commands to other commands, which neither the multi-arg form of system nor the 4 arg form of open will handle.
On Windows, the situation is a bit nastier. Basically, all Win32 programs receive one long command-line string -- the shell (usually cmd.exe) may do some interpretation first, removing < and > redirections for example, but it does not split it up at word boundaries for the program. Each program must do this parsing themselves (if they wish -- some programs don't bother). In C and C++ programs, routines provided by the runtime libraries supplied with the compiler toolchain will generally perform this parsing step before main() is called.
The problem is, in general, you don't know how a given program will parse its command line. Many programs are compiled with some version of MSVC++, whose quirky parsing rules are described here, but many others are compiled with different compilers that use different conventions.
This is compounded by the fact that cmd.exe has its own quirky parsing rules. The caret (^) is treated as an escape character that quotes the following character, and text inside double quotes is treated as quoted if a list of tricky criteria are met (see cmd /? for the full gory details). If your command contains any strange characters, it's very easy for cmd.exe's idea of which parts of text are "quoted" and which aren't to get out of sync with your target program's, and all hell breaks loose.
So, the safest approach for escaping arguments on Windows is:
Escape arguments in the manner expected by the command-line parsing logic of the program you're calling. (Hopefully you know what that logic is; if not, try a few examples and guess.)
Join the escaped arguments with spaces.
Prefix every single non-alphanumeric character of the resulting string with ^.
Append any redirections or other shell trickery (e.g. joining commands with &&).
Run the command with system() or backticks.
sub esc_chars {
# will change, for example, a!!a to a\!\!a
#_ =~ s/([;<>\*\|`&\$!#\(\)\[\]\{\}:'"])/\\$1/g;
return #_;
}
http://www.slac.stanford.edu/slac/www/resource/how-to-use/cgi-rexx/cgi-esc.html
If you use system "$cmd #args" (a string), then you have to escape the arguments because a shell is invoked.
Fortunately, for double quoted strings, only four characters need escaping:
" - double quote
$ - dollar
# - at symbol
\ - backslash
The answers on your question were very useful. In the end I followed #runrig's advice but then used the core module open3() command so I could capture the output from STDERR as well as STDOUT.
For sample code of open3() in use with #runrig's solution, see my related question and answer:
Calling system commands from Perl