I'm having an issue with some code and I'm wondering if anyone can assist.
Basically I'm trying to execute an isql query against a database and assign it to a scalar variable. The isql command makes use of the column seperator which is defined as the pipe symbol.
So I have it set-up as such:
my $command = "isql -S -U -s| -i";
my $isql_output = `$command`;
The isql command works in isolation but when issued as a backtick it stops at the pipe. I've tried concatenating the $command string using sub-strings, using single quotes and backslash escaping items such as -s\"\|\" to no avail. I've also tried using qx instead of backticks.
Unfortunately I'm currently using an older version of perl (v5.6.1) with limited scope for upgrade so I'm not sure if I can resolve this.
You have to quote the | in a way that the shell does not recognize it as a special character. Two ways:
Put the -s| into single quotes: '-s|'. Perl will leave single quotes inside double quoted strings alone and pass them to the shell unmodified.
Escape the | with two backslashes: -s\\|. Why two? The first one is seen by Perl telling it to pass the next character through unmodified. Therefore the shell sees -s\| and does something very similar: it sees the single \ and knows not to treat the next char, |, special.
The problem is that the command is being executed through a shell.
You can avoid this by passing the command and arguments in a list rather than a single string.
The backtick construct does not support that, so you would need to use the open() function instead.
I haven't tested the following code but it gives the idea:
my #command = (qw(isql -Sserver -Uuser -Ppassword -s| -w4096), '–i' . $file);
print join(' ', #command), "\n";
open(my $fh, '-|', #command)
or die "failed to run isql command: $#\n";
my #isql_output = <$fh>;
close($fh);
my $isql_output = $isql_output[0]; chomp($isql_output);
If you're working with a 15 year old version of Perl (which Oracle users tend to do) I'm not sure this will all be supported. For instance, you may need to write chop instead of chomp.
UPDATE: the problem is not the Perl version, but this construct not being supported on Windows, according to the documentation. This must be qualified: I use Perl on Cygwin and it works fine there, but I don't know whether you can use Cygwin.
Single quotes should work. Try to run test perl script:
my $cmd = "./test.sh -S -U -s '|' -i";
print `$cmd`;
With test.sh:
#!/bin/sh
echo $#
Output should be -S -U -s | -i
Related
I would like to inplace edit ssd_config file where i need to replace the #Port to a custom port.
Before:
#Port <portnum>
ex: #Port 22
After:
Port <customport>
ex: Port 2022
Here the custom port is coming in a variable $port.
I tried the below script but does nothing.
my $prt = "Port 2022";
my $cmd = "sed -i \'s/#Port [0-9]\+/$prt/g\' sshd_config";
system($cmd);
Tried even with tick operator.
`sed -i \"s/#Port [0-9]\+/\$prt/g\" sshd_config`;
I'd suggest to do all that in Perl, once you are running a Perl program. That way one doesn't have to run external programs, not to mention the benefits of not having to mess with all the quoting and escaping if you actually need a shell (not in the shown example though).
Then we need a few more lines of code, to read the file and and edit its content and write it back out. That shouldn't matter, given all the benefits -- but there are libraries that cut out even that as well, for example the very handy Path::Tiny
use Path::Tiny;
...
path($filename)->edit_lines( sub { s/#Port [0-9]+/Port $prt/ } );
I take it that $filename and $prt have been introduced earlier in the program.
There is also a edit method, which slurps the whole file.
Anything sed can do, Perl can do.
If this is your entire Perl program:
my $prt = "Port 2022";
my $cmd = "sed -i \'s/#Port [0-9]\+/$prt/g\' sshd_config";
system($cmd);
Then you can do it all in Perl itself from the command line.
perl -i -p -e's/#Port [0-9]+/Port 2022/g' sshd_config
system("sed ....") is invoking a shell to parse the command line, which means that everything needs to be properly escaped according to the rules of the shell. This together with string escaping inside Perl makes it hard to get right. A much simpler way is to skip the shell and call sed directly and also use string concatenation to add the $prt at the right place (and don't forget to also include the "Port" string itself since you want to have it in the output):
system('sed','-i','s/#Port [0-9]+/Port ' . $prt . '/', 'sshd_config');
Alternatively one could do it in Perl directly, i.e. something like this:
open(my $fh,'<','sshd_config') or die $!;
my #lines = map { s/#Port \d+/Port $prt/ } <$fh>;
open($fh,'>','sshd_config') or die $!;
print $fh #lines;
close($fh);
This is a bit longer but does not rely on starting an external program and is thus faster. And if there is more to do than a simple replacement it is also more flexible.
I am trying to test the code snippet below for a bigger script that I am writing. However, I can't get the search working with parentheses and variables.
Appreciate any help someone can give me.
Code snippet:
#!/usr/bin/perl
$file="test4.html";
$Search="Help (Test)";
$Replace="Testing";
print "/usr/bin/sed -i cb 's/$Search/$Replace/g' $file\n";
`/usr/bin/sed -i cb 's/$Search/$Replace/g' $file`;
Thanks,
Ash
The syntax to run a command in a child process and wait for its termination in perl is system "cmd", "arg1", "arg2",...:
#!/usr/bin/perl
$file="test4.html";
$Search="Help (Test)";
$Replace="Testing";
print "/usr/bin/sed -icb -e 's/$Search/$Replace/g' -- $file\n";
system "/usr/bin/sed", "-icb", "-e", "s/$Search/$Replace/g", "--", $file;
(error checking left as an exercise, see perldoc -f system for details)
Note that -i is not a standard sed option. The few implementations that support it (yours must be the FreeBSD one as you've separated the cb backup extension from -i) have actually copied it from perl! It does feel a bit silly to be calling sed from perl here.
Looking at your approach:
The `...` operator itself is reminiscent of the equivalent `...` shell operator. In perl, what's inside is evaluated as if inside double quoted, in that $var, #var... perl variables are expanded, and a shell is started with -c and the resulting string as arguments and with its stdout redirected to a pipe.
The shell interprets that argument as code in the shell syntax. Perl reads the output of that inline shell script from the other end of the pipe and that makes up the expansion of `...`. Same as in shell command substitution except that there's is no stripping of zero bytes or of trailing newlines.
sed -i produces no output, so it's pointless to try and capture its output with `...` here.
Now in your case, the code that sh is asked to interpret is:
/usr/bin/sed -i cb 's/Help (Test)/Testing/g' test4.html
That should work fine on FreeBSD or macOS at least. If $file had been test$(reboot).html, that would have been worse though.
Here, because you have the contents of variables that end up interpreted as code in an interpreter (here sh), you have a potential arbitrary command injection vulnerability.
In the system approach, we remove sh, so that particular vulnerability is removed. However sed is also an interpreter of some language. That language is not as omnipotent as that of sh, but for instance sed can write to arbitrary files with its w command. The GNU implementation (which you don't seem to be using) can run arbitrary commands as well.
So you still potentially have a code injection vulnerability in the case of $Search or $Replace coming from an external source.
If that's the case, you'd need to make sure your properly sanitise those values before running sed. See for instance: How to ensure that string interpolated into `sed` substitution escapes all metachars
I have a Perl script which runs fine under Perl 5.005 on HPUX 11i v3 and that causes a small problem under Perl 5.10 on my Ubuntu 11.04 box.
It boils down to a line like:
open (CMD, "do something | grep -E 'sometext$' |");
(it's actually capturing ps -ef output to see if a process is running but I don't think that's important (a)).
Now this runs fine on the HPUX environment but, when I try it out under Ubuntu, I get:
sh: Syntax error: Unterminated quoted string
By inserting copious debug statements, I tracked it down to that offending line then started removing characters one-by-one until it stopped complaining. Luckily, $ was the first one I tried and it stopped giving me the error, so I then changed the line to:
open (CMD, "do something | grep -E 'sometext\$' |");
and it worked fine (under Linux anyway - I haven't tested that on HPUX since I don't have access to that machine today - if it does work, I'll just use that approach but I'd still like to know why it's a problem).
So it seems obvious that the $ is "swallowing" the closing single quote under my Linux environment but not apparently on the HPUX one.
My question is simply, why? Surely there weren't any massive changes between 5.005 and 5.10. Or is there some sort of configuration item I'm missing?
(a) But, if you know a better way to do this without external CPAN modules (ie, with just the baseline Perl 5.005 installation), I'd be happy to know about it.
$' is a special variable (see perldoc perlvar). 5.005 was many versions ago, so it's possible that something has changed in the regexp engine to make this variable different (although it appears to be in 5.005 also)
As for the better way, you could at least only run the 'ps -ef' in a pipeline and do the 'grep' in perl.
Use the following!!!
use strict;
use warnings;
You would have gotten
Use of uninitialized value $' in concatenation (.) or string
A sigil followed by any punctuation symbol (on a standard keyboard) is a variable in Perl, regardless of if it is defined or not. So in a double quoted string [$#][symbol] will always be read as one token and interpolated unless the sigil is escaped.
I have a feeling that the difference you are seeing has to do with different system shells rather than different versions of perl.
Consider your line:
open (CMD, "do something | grep -E 'sometext$' |");
When perl sees that, it will interpolate the empty $' variable into the double quoted string, so the string becomes:
open (CMD, "do something | grep -E 'sometext |");
At that point, your shell gets to process a line that looks like:
do something | grep -E 'sometext
And if it succeeds or fails will depend on the shell's rules regarding unterminated strings (some shells will complain loudly, others will automatically terminate the string at eof).
If you were using the warnings pragma in your script, you probably would have gotten a warning about interpolating an undefined variable.
A shorter and cleaner way to read in the output of ps would be:
my #lines = grep /sometext\$/, `ps -ef`;
Or using an explicit open:
my #lines = grep /sometext\$/, do {
open my $fh, '|-', 'ps -ef' or die $!;
<$fh>
};
Because $' is a special variable in recent versions of Perl.
From the official documentation (perlvar):
$':
The string following whatever was matched by the last successful
pattern match (not counting any matches hidden within a BLOCK or
eval() enclosed by the current BLOCK).
If there were no successful pattern matches, $' is empty and your statement essentially interpolates to
open (CMD, "do something | grep -E 'sometext |");
Escaping the dollar sign (the solution that works for you on Linux) should work on HPUX too.
I'm not sure when was this variable added, but I can confirm that it exists in Perl 5.8.5. What's New for Perl 5.005 mentions $' (not as a new feature), so I think it was already there before then.
You should probably use single-quotes rather than double quotes around the string since there is nothing in the string that should be interpolated:
open (CMD, q{do something | grep -E 'sometext$' |});
Better would be to use the 3-argument form of open with a lexical file handle:
open my $cmd, '-|', q{do something | grep -E 'sometext$'} or die 'a horrible death';
I don't have a good explanation for why $' is being recognized as the special variable in 5.10 and yet it was not in 5.005. That is unexpected.
Is there a really good reason you can't upgrade to something like 5.14.1? Even if you don't change the system-provided Perl, there's no obvious reason you can't install a recent version in some other location and use that for all your scripting work.
$' is a special variable.
If you want to avoid variable expansion, just use q():
open (CMD, q(do something | grep -E 'sometext$' |));
When I run the ls command this runs fine. But echo $PATH does not give me any output from perl. When I run echo from the shell prompt it gives me output. Can you explain this behavior?
#!usr/bin/perl
$\="\n";
$out = `ls`;
print $out;
$out=`echo $PATH`;
print $out;
Please note that while the technically correct answer to your question is the $ interpolation, you should also note that you should not treat Perl like a shell script and call external commands via backticks instead of using Perl built-in or library functions designed for the purpose:
$out = join("\n", glob("*")); # same as `ls`
$out = $ENV{PATH}; # same as `echo $PATH`
This has several significant advantages:
speed (no call to system)
portability
More security (no shell attack vector)
Most built ins cover proper error handling for you better than your own system call implementation
Nearly always a better, cleaner, shorter and easier to read/maintain code
Backticks interpolate like double quotes, so you need to escape the $.
$out=`echo \$PATH`;
$PATH is shell variable, from perl you should use it as perl variable $ENV{PATH}
Still try to read some basic docs too, like perldoc perlintro. No need for executing echo at all.
Perl is interpolating $PATH in the backticks as a Perl variable, and you've not set a $PATH anywhere in your script, so the command is coming out as
$out = `echo `
which is basically a null-op. Try
$out = `echo \$PATH`
instead, which would force Perl to ignore the $ and pass it intact to the shell.
You need to escape $ in $PATH because the backticks operator interpolates any variables.
$out=`echo \$PATH`;
You could also use qx// with single quotes. From perldoc perlop:
Using single-quote as a delimiter
protects the command from Perl's
double-quote interpolation, passing it
on to the shell instead:
$perl_info = qx(ps $$); # that's Perl's $$
$shell_info = qx'ps $$'; # that's the new shell's $$
Others have already explained the reason - variables inside backticks are interpolated, so your echo $PATH is actually becoming echo since there's no $PATH variable declared.
However, always put use strict; at the top of every Perl script you write.
Had you done so, Perl would have told you what was happening, e.g.:
Global symbol "$PATH" requires explicit package name at myscript.pl line 9
To stop variables being interpolated, either escape them (e.g. \$PATH), or, more cleanly, use e.g. qx'echo $PATH'.
Also, as others have pointed out, calling echo $PATH makes no real-world sense; if you're trying to get the contents of the PATH environment variable, just use $ENV{PATH} - however, you may have just been using it as a simple reduced demonstration case.
As I understand (Perl is new to me) Perl can be used to script against a Unix command line. What I want to do is run (hardcoded) command line calls, and search the output of these calls for RegEx matches. Is there a way to do this simply in Perl? How?
EDIT: Sequence here is:
-Call another program.
-Run a regex against its output.
my $command = "ls -l /";
my #output = `$command`;
for (#output) {
print if /^d/;
}
The qx// quasi-quoting operator (for which backticks are a shortcut) is stolen from shell syntax: run the string as a command in a new shell, and return its output (as a string or a list, depending on context). See perlop for details.
You can also open a pipe:
open my $pipe, "$command |";
while (<$pipe>) {
# do stuff
}
close $pipe;
This allows you to (a) avoid gathering the entire command's output into memory at once, and (b) gives you finer control over running the command. For example, you can avoid having the command be parsed by the shell:
open my $pipe, '-|', #command, '< single argument not mangled by shell >';
See perlipc for more details on that.
You might be able to get away without Perl, as others have mentioned. However, if there is some Perl feature you need, such as extended regex features or additional text manipulation, you can pipe your output to perl then do what you need. Perl's -e switch let's you specify the Perl program on the command line:
command | perl -ne 'print if /.../'
There are several other switches you can pass to perl to make it very powerful on the command line. These are documented in perlrun. Also check out some of the articles in Randal Schwartz's Unix Review column, especially his first article for them. You can also google for Perl one liners to find lots of examples.
Do you need Perl at all? How about
command -I use | grep "myregexp" && dosomething
right in the shell?
#!/usr/bin/perl
sub my_action() {
print "Implement some action here\n";
}
open PROG, "/path/to/your/command|" or die $!;
while (<PROG>) {
/your_regexp_here/ and my_action();
print $_;
}
close PROG;
This will scan output from your command, match regexps and do some action (which now is printing the line)
In Perl you can use backticks to execute commands on the shell. Here is a document on using backticks. I'm not sure about how to capture the output, but I'm sure there's more than a way to do it.
You indeed use a one-liner in a case like this. I recently coded up one that I use, among other ways, to produce output which lists the directory structure present in a .zip archive (one dir entry per line). So using that output as an example of command output that we'd like to filter, we could put a pipe in and then use perl with the -n -e flags to filter the incoming data (and/or do other things with it):
[command_producing_text_output] | perl -MFile::Path -n -e \
"BEGIN{#PTM=()} if (m{^perl/(bin|lib(?!/site))}) {chomp;push #PTM,$_}" ^
-e "END{#WDD=mkpath (\#PTM,1);" ^
-e "printf qq/Created %u dirs to reflect part of structure present in the .ZIP file\n/, scalar(#WDD);}"
the shell syntax used, including: quoting of perl code and escaping of newlines, reflects CMD.exe usage in Windows NT-like consoles. If you need to, mentally replace
"^" with "\" and " with ' in the appropriate places.
The one-liner above adds only the directory names that start with "perl/bin" or
"perl/lib (not followed by "/site"); it then creates those directories. You wind
up with a (empty) tree that you can use for whatever evil purposes you desire.
The main point is to illustrate that there are flags available (-n, -p) to
allow perl to loop over each input record (line), and that what you can do is unlimited in terms of complexity.