The user is going to enter input string such as Tom's Toy.
However the perl script complains saying "unmatched '."
This is my code.
my $commandline="";
while (#ARGV) {
$_ = shift #ARGV;
{$commandline .= $_ . ' ';}
}
print " Running $commandline\n";
system ($commandline);
Now if the user input is Tom's Toy. I just want to print back Tom's Toy.
However perl complains "unmatched '.".
IF I dont user quote it works fine. (for eg: Tom Toy is good)
How do I fix this issue.
Any help is greatly appreciated.
Thanks in advance
If you switch things around a little to use the system $cmd, #args version of the function, no shell will be invoked, so no escaping will be necessary.
my $cmd = shift #ARGV;
my #args = #ARGV;
print " Running $cmd\n";
system $cmd, #args;
I tested with ./test.pl echo Tom\'s Toy and it gives the expected output:
Running echo
Tom's Toy
system(#ARGV) is probably all you need.
If you give system() a single argument, and if that argument contains any shell metacharacters (including spaces, quotation marks, etc), then the argument will be passed to the shell. jwodder is quite correct: the error message is from the shell, not from Perl.
If you pass system() multiple arguments, it's done without invoking a shell -- which is usually better. The approach you're using takes your program's command-line arguments, joins them together into a single string, then passes that string to the shell, which splits it back into multiple arguments for execution.
On the other hand, sometimes you might want to invoke the shell, for example if you're building up a complex command using pipes, I/O redirection, and so forth, and you don't want to set it all up in Perl. But you have to be careful about metacharacters, as you've seen.
"perldoc -f system" explains this more fully.
If all you want to do is print back the user input, use print, not system. system will try to pass the supplied string to the shell for execution as a command, and it's the shell that's complaining about the unmatched quote.
(Also, instead of manually concatenating #ARGV, may I direct your attention to the join function?)
Related
Sorry, this is pretty basic, and I suspect a duplicate, but after some searching I'm coming up empty:
Given the following script:
#!/usr/bin/perl
use strict;
use warnings;
use IPC::Run3;
my $stdout2;
print $ARGV[0];
print "\n";
my #cmd1 = split /\s+/, $ARGV[0] ;
run3 (\#cmd1, \undef, \$stdout2, \$stdout2);
print $stdout2
And running it like so:
£ perl comp.pl "md5sum *(.)"
md5sum *(.)
md5sum: '*(.)': No such file or directory
Fair enough. The *(.) isn't being intrepreted by the shell and probably most would consider this a feature. But I would like it to be intepreted by the current shell (or zsh specifically would be fine).
The question is how I can do this without complicating the shell command to run the perl script.
Prepending "zsh" and "-c" to cmd1 is ok if that's a reasonable way to do it. It just seems like...it isn't.
My intention is also to pass slightly more complex commands to this script eventually, like so:
perl comp.pl 'md5sum *(.)' 'ssh remoteHost "md5sum *(.)"'
I have no objection to non-perl answers to the problem you can probably infer I'm trying to solve (I suspect rsync could do this) but I'm primarily interested in solving this through Perl as there'll eventually be business-specific logic in this comparison.
EDIT
I tried various forms of:
my $cmd = $ARGV[0];
run3 (\$cmd, \undef, \$stdout2, \$stdout2);
the documentation seems to think this would be ok, but I get:
Not an ARRAY reference at /usr/local/share/perl/5.22.1/IPC/Run3.pm line 320.
The IPC::Run3 docs say that one can pass a string instead of an arrayref for the command
run3($cmd, $stdin, $stdout, $stderr, \%options)
...
$cmd
Usually $cmd will be an ARRAY reference and the child is invoked via
system #$cmd;
But $cmd may also be a string in which case the child is invoked via
system $cmd;
In this case the string $cmd is passed to the shell if it contains shell metacharacters. So take input without splitting it, $cmd = $ARGV[0], or join it after validation, $cmd = join ' ', #cmd;
Even in general this is not the preferred way, and the docs warn to see system for "pitfalls" of it.
Things are yet much worse here since you'd be passing user input directly for execution! Never mind possible nefarious intents, just think of what a good typo can do. Even without that, there is simply a difference between typing a command at the terminal and passing it to a script, which may edit it, may get modified, pick up bugs, etc.
If nothing else, I'd urge to add code for substantial checks of submitted input. An analysis may involve identifying the known and accepted metacharacters while suitably quoting parts of input that shouldn't be interpreted, for example using String::ShellQuote.
But I'd really suggest to reconsider the design, so to not submit complete commands to the script. Rather, specify with keywords what should happen. Things like globbing (assembling a file list) are done from Perl really nicely and with a lot of control. Do outside only what is necessary; generally there'll be no need for the shell then.
I want to take date as input from User & pass that input to shell command below.
for e.g.
$date = ARGV[0];
`cd xyz $date`
will this variable be interpolated in perl?
You have a couple of problems; first of all, cd only takes one parameter. Perhaps you meant something like cd xyz$date? Second, backticks start a shell that executes the command you give, which will do the change directory command and then immediately exit, having no effect. (The parent perl process's current directory is left unchanged.) You might be looking for chdir.
But yes, the interpolation will work; to disable interpolation in backticks, you either escape special characters (echo xyz \$date) or use qx with single quote delimiters (qx'echo xyz $date').
This is trivial to test:
Input:
use v5.16;
use strict;
use warnings;
my $foo = 123;
say `echo $foo`;
Output:
123
So yes.
Beware of variables containing characters with special meaning in shell though.
Yes, it will interpolate the variable. However, I highly recommend two things:
Use qx/../ instead of back ticks.
Set the command you're running first in a Perl variable, and then interpolate that variable.
The qx/../ is nice because it makes it more obvious what you're doing:
my $variable = qx(ls $directory);
You could use a wide variety of characters for qx:
my $variable = qx(ls $directory);
my $variable = qx/ls $directory/;
my $variable = qx^ls $directory^;
my $variable = qx#ls $directory#;
The character after the qx will be used as quotes. If you use parentheses, square brackets, or curly braces, you use them in pairs with the opening one and closing one.
This means that you can avoid issues where a particular character in your command might confuse things.
The other thing is to build your command into a Perl variable and then execute it when you try to do interpolations. This gives you more control over what you're doing:
my $HOME;
my $command;
$HOME = "bin";
$command = "ls $HOME";
print qx($command); #Lists the directory bin
$command = 'ls $HOME';
print qx($command); #List user's home directory
In both of these examples, I'm doing qx($command). However, in the first example, I allow Perl to substitute the value of $HOME. In the second example, I use single quotes, so Perl doesn't substitute the value of$HOME. Instead, the string$HOME` is just part of my command I'm using. I'm letting the shell interpolates it.
I usualy am leery of any program that uses qx/.../. In most cases, it's used to run a command that could be done in Perl itself. For example, in early Perl programs, you'd see things like this:
$date = `date +%M/%Y/%D`
chop $date; #Yeah, I said EARLY Perl programs
Because it was simply a lot easier to run the Unix command rather than trying to do it in a pure Perl way. However, doing it the Perl (i.e. the correct) way means you're no longer dependent upon the OS's behavior which is not entirely under your control.
If you need the output of the command for use in your Perl script, you should use open to execute your command, and treat the output of the command as a file.
my $command = "ls $HOME";
open my command_fh, "|-", $command or die qq(Couldn't execute "$command");
If you simply need to execute the command, use the system command:
my $command = "xyz $date";
my $error = system $command;
if ( $error ) {
#Something terrible happened...
}
Note that if you send only a single scalar argument to the system command, and it contains possible shell meta characters, it will execute the command via the OS shell. If you send the system command a list to execute, or there are no shell metacharacters, Perl will call the command executor to execute the command directly without any shell interpolations:
my #command = qw(ls $HOME);
system #command; #Will print out "No such directory '$HOME'
Perldoc has the answers, as ever.
Specifically, http://perldoc.perl.org/perlop.html#Quote-and-Quote-like-Operators has "Yes" in the Interpolates? column (with a proviso you don't need to worry about here), so yep, it's safe to assume your variable will be interpolated.
You can get the same document by running 'perldoc perlop' at your local commandline, often.
I'm trying to modify a script that someone else has written and I wanted to keep my script separate from his.
The script I wrote ends with a print line that outputs all relevant data separated by spaces.
Ex: print "$sap $stuff $more_stuff";
I want to use this data in the middle of another perl script and I'm not sure if it's possible using a system call to the script I wrote.
Ex: system("./sap_calc.pl $id"); #obtain printed data from sap_calc.pl here
Can this be done? If not, how should I go about this?
Somewhat related, but not using system():
How can I get one Perl script to see variables in another Perl script?
How can I pass arguments from one Perl script to another?
You're looking for the "backtick operator."
Have a look at perlop, Section "Quote-like operators".
Generally, capturing a program's output goes like this:
my $output = `/bin/cmd ...`;
Mind that the backtick operator captures STDOUT only. So in order to capture everything (STDERR, too) the commands needs to be appended with the usual shell redirection "2>&1".
If you want to use the data printed to stdout from the other script, you'd need to use backticks or qx().
system will only return the return value of the shell command, not the actual output.
Although the proper way to do this would be to import the actual code into your other script, by building a module, or simply by using do.
As a general rule, it is better to use all perl solutions, than relying on system/shell as a way of "simplifying".
myfile.pl:
sub foo {
print "Foo";
}
1;
main.pl:
do 'myfile.pl';
foo();
perldoc perlipc
Backquotes, like in shell, will yield the standard output of the command as a string (or array, depending on context). They can more clearly be written as the quote-like qx operator.
#lines = `./sap_calc.pl $id`;
#lines = qx(./sap_calc.pl $id);
$all = `./sap_calc.pl $id`;
$all = qx(./sap_calc.pl $id);
open can also be used for streaming instead of reading into memory all at once (as qx does). This can also bypass the shell, which avoids all sorts of quoting issues.
open my $fh, '-|', './sap_calc.pl', $id;
while (readline $fh) {
print "read line: $_";
}
I was using Perl to read through each line of a file. I used a command line tool to call a service, and I noticed some interesting functionality that I can't figure out how to search for. To the variable $cmd I assigned the command that invokes the service. If I refer to $cmd later in the code it prints out the command line argument, but if I refer to it as `$cmd`, however, it gives the output from running the service.
What is the explanation for this?
It works just like backquotes in the shell, which is why it is called that. See sh(1) for details. It captures the standard output alone, and nothing else. It sets the $? variable to the 16-bit wait status word.
This is all explained in the perlop(1) manpage:
qx/STRING/
`STRING`
A string which is (possibly) interpolated and then
executed as a system command with /bin/sh or its
equivalent. Shell wildcards, pipes, and redirections
will be honored. The collected standard output of the
command is returned; standard error is unaffected. In
scalar context, it comes back as a single (potentially
multi-line) string, or undef if the command failed.
In list context, returns a list of lines (however
you’ve defined lines with $/ or
$INPUT_RECORD_SEPARATOR), or the empty list if the
command failed.
Because backticks do not affect standard error: use
shell file descriptor syntax (assuming the shell
supports this) if you care to address this. To
capture a command’s STDERR and STDOUT merged together:
$output = `cmd 2>&1`;
To capture a command’s STDOUT but discard its STDERR:
$output = `cmd 2>/dev/null`;
To capture a command’s STDERR but discard its STDOUT
(ordering is important here):
$output = `cmd 2>&1 1>/dev/null`;
To exchange a command’s STDOUT and STDERR in order to
capture the STDERR but leave its STDOUT to come out
the old STDERR:
$output = `cmd 3>&1 1>&2 2>&3 3>&-`;
To read both a command’s STDOUT and its STDERR
separately, it’s easiest to redirect them separately
to files, and then read from those files when the
program is done:
system("program args 1>program.stdout 2>program.stderr");
The STDIN filehandle used by the command is inherited
from Perl’s STDIN. For example:
open(BLAM, "blam") || die "$0: can't open blam: $!";
open (STDIN, "<&BLAM") || die "$0: can't dup BLAM: $!";
print `sort`;
will print the sorted contents of the file blam.
Using single-quote as the delimiter protects the command
from Perl’s double-quote interpolation, passing the contents on
to the shell instead:
$perl_info = qx(ps $$); # that's Perl's $$
$shell_info = qx'ps $$'; # that's the new shell's $$
How that string gets evaluated is entirely subject to
the command interpreter on your system. On most
platforms, you will have to protect shell
metacharacters if you want them treated literally.
This is in practice difficult to do, as it’s unclear
which characters need escaping, or how. See perlsec for a
clean and safe example of a manual fork and exec
to emulate backticks safely.
On some platforms (notably DOS-like ones), the shell
may not be capable of dealing with multiline commands,
so putting newlines in the string may not get you what
you want. You may be able to evaluate multiple
commands in a single line by separating them with the
command separator character, if your shell supports
that (e.g. ; on many Unix shells; & on the Windows
NT CMD.COM shell).
Beginning with v5.6.0, Perl attempts to flush all
files opened for output before starting the child
process, but this may not be supported on some
platforms (see perlport(1)). To be safe, you may need to
set $| ($AUTOFLUSH in English) or call the
autoflush method of IO::Handle on any open
handles.
Beware that some command shells may place restrictions
on the length of the command line. You must ensure
your strings don’t exceed this limit after any
necessary interpolations. See the platform-specific
release notes for more details about your particular
environment.
Using this operator can lead to programs that are
difficult to port, because the shell commands called
vary between systems, and may in fact not be present
at all. As one example, the type command under the
POSIX shell is very different from the type command
under DOS. That doesn't mean you should go out of
your way to avoid backticks when they’re the right way
to get something done. Perl was made to be a glue
language, and one of the things it glues together is
commands. Just understand what you’re getting
yourself into.
See I/O Operators for more discussion.
Here’s a simple example of using backticks to get the exit status of the first element in a pipeline:
$device = q(/dev/rmt8);
$dd_noise = q(^[0-9]+\+[0-9]+ records (in|out)$);
$status = `exec 3>&1; ((dd if=$device ibs=64k 2>&1 1>&3 3>&- 4>&-; echo $? >&4) | egrep -v "$dd_noise" 1>&2 3>&- 4>&-) 4>&1`;
EDIT
Well ok then, so maybe that wasn’t that simple an example. :) But this one is.
I’d like to recommend the Capture::Tiny CPAN module as a simpler way to manage the output from external commands that you would normally run using backquotes. It has advantages and disadvantages, but I feel that for many people, the advantages outweigh any arguable disadvantageL
The advantage is that you get to do all this without requiring deep knowledge of arcane mysteries of file-descriptor redirection the way the previous example did.
The disadvantage is it’s yet another non-core dependency — something else you have to install from CPAN.
That’s really not bad for what you get.
Here’s an example of how easy it is:
NAME
Capture::Tiny - Capture STDOUT and STDERR from Perl, XS, or external programs
SYNOPSIS
use Capture::Tiny qw/capture tee capture_merged tee_merged/;
($stdout, $stderr) = capture {
# your code here
};
($stdout, $stderr) = tee {
# your code here
};
$merged = capture_merged {
# your code here
};
$merged = tee_merged {
# your code here
};
DESCRIPTION
Capture::Tiny provides a simple, portable way to capture anything sent to STDOUT or STDERR, regardless of whether it comes from Perl, from XS code
or from an external program. Optionally, output can be teed so that it is captured while being passed through to the original handles. Yes, it
even works on Windows. Stop guessing which of a dozen capturing modules to use in any particular situation and just use this one.
There, isn’t that a whole lot easier?
The back-quote in Perl does much the same as the back-quote in shell - it runs a command and captures the standard output.
See also qx//.
I think the backtick lets you run commands and store their output in a variable:
$listing=`ls -1 /tmp/`;
When using system() calls in Perl, do you have to escape the shell args, or is that done automatically?
The arguments will be user input, so I want to make sure this isn't exploitable.
If you use system $cmd, #args rather than system "$cmd #args" (an array rather than a string), then you do not have to escape the arguments because no shell is invoked (see system). system {$cmd} $cmd, #args will not invoke a shell either even if $cmd contains metacharacters and #args is empty (this is documented as part of exec). If the args are coming from user input (or other untrusted source), you will still want to untaint them. See -T in the perlrun docs, and the perlsec docs.
If you need to read the output or send input to the command, qx and readpipe have no equivalent. Instead, use open my $output, "-|", $cmd, #args or open my $input, "|-", $cmd, #args although this is not portable as it requires a real fork which means Unix only... I think. Maybe it'll work on Windows with its simulated fork. A better option is something like IPC::Run, which will also handle the case of piping commands to other commands, which neither the multi-arg form of system nor the 4 arg form of open will handle.
On Windows, the situation is a bit nastier. Basically, all Win32 programs receive one long command-line string -- the shell (usually cmd.exe) may do some interpretation first, removing < and > redirections for example, but it does not split it up at word boundaries for the program. Each program must do this parsing themselves (if they wish -- some programs don't bother). In C and C++ programs, routines provided by the runtime libraries supplied with the compiler toolchain will generally perform this parsing step before main() is called.
The problem is, in general, you don't know how a given program will parse its command line. Many programs are compiled with some version of MSVC++, whose quirky parsing rules are described here, but many others are compiled with different compilers that use different conventions.
This is compounded by the fact that cmd.exe has its own quirky parsing rules. The caret (^) is treated as an escape character that quotes the following character, and text inside double quotes is treated as quoted if a list of tricky criteria are met (see cmd /? for the full gory details). If your command contains any strange characters, it's very easy for cmd.exe's idea of which parts of text are "quoted" and which aren't to get out of sync with your target program's, and all hell breaks loose.
So, the safest approach for escaping arguments on Windows is:
Escape arguments in the manner expected by the command-line parsing logic of the program you're calling. (Hopefully you know what that logic is; if not, try a few examples and guess.)
Join the escaped arguments with spaces.
Prefix every single non-alphanumeric character of the resulting string with ^.
Append any redirections or other shell trickery (e.g. joining commands with &&).
Run the command with system() or backticks.
sub esc_chars {
# will change, for example, a!!a to a\!\!a
#_ =~ s/([;<>\*\|`&\$!#\(\)\[\]\{\}:'"])/\\$1/g;
return #_;
}
http://www.slac.stanford.edu/slac/www/resource/how-to-use/cgi-rexx/cgi-esc.html
If you use system "$cmd #args" (a string), then you have to escape the arguments because a shell is invoked.
Fortunately, for double quoted strings, only four characters need escaping:
" - double quote
$ - dollar
# - at symbol
\ - backslash
The answers on your question were very useful. In the end I followed #runrig's advice but then used the core module open3() command so I could capture the output from STDERR as well as STDOUT.
For sample code of open3() in use with #runrig's solution, see my related question and answer:
Calling system commands from Perl