In Perl, one way to read the STDOUT of a subprocess is to use open:
open(PIPE, "ls -l |");
I was looking for a more object-oriented way to do this, though, and I've been using IO::Pipe with some success. I want to detect errors, though, specifically if the command is not executable. I can't figure out how to do that via IO::Pipe, though. Here's what I have:
use strict;
use warnings;
use IO::Pipe;
my($cmd) = join (" ", #ARGV);
open(PIPE, "$cmd |") || die qq(error opening PIPE);
while (<PIPE>) {
chomp;
print "DBG1: $_\n";
}
close PIPE;
my($pipe) = IO::Pipe->new();
$pipe->reader($cmd);
die qq(error opening IO::Pipe) if $pipe->eof();
while (<$pipe>) {
chomp;
print "DBG2: $_\n";
}
$pipe->close();
If the sub-process command is invalid, both checks will correctly die. If the sub-process produces no output, though, eof() will report an error, even if the command itself is fine:
$ perl pipe.pl "ls -l >/dev/null"
error opening IO::Pipe at pipe.pl line 20.
A bunch of questions, then:
Is there a reasonable OO way to read from a sub-process in Perl? Is IO::Pipe the correct tool to use? If so, how do I check to make sure the sub-process command is created successfully? If not, what should I be using? I don't want to write to the sub-process, so I don't think I want IPC::Open2 or IPC::Open3. I would prefer to use a core module, if possible.
The issue is not IO::Pipe. The problem is eof is the wrong way to check for a pipe error. It doesn't mean there's no pipe, it means there's nothing to read from that pipe. You'd have the same problem with eof PIPE. It's perfectly fine for a sub-process to not print anything.
If you want to check the sub-process successfully ran, it turns out IO::Pipe already does that for you.
# IO::Pipe: Cannot exec: No such file or directory
$pipe->reader("hajlalglagl");
Backticks is not a core module but seems to do what your looking for.
use strict;
use warnings;
use Backticks;
my $cmd = Backticks->new(join (" ", #ARGV));
$cmd->run();
if ($cmd->success){
print $cmd->stdout
} else {
print "Command failed\n";
}
Running this with a valid command then invalid command gives the below results
io_pipe.pl "uname -o"
GNU/Linux
io_pipe.pl "uname -z"
Command failed
Update
As pointed out by #thisSuitIsNotBlack, this module changes the deafult behaviour of backticks in perl. You should read the Notes section of the POD. However the main one to be aware of is:
The overriding of backticks is provided by Filter::Simple. Source
filtering can be weird sometimes... if you want to use this module in
a purely traditional Perl OO style, simply turn off the source
filtering as soon as you load the module:
use Backticks;
no Backticks;
Related
I just created a text test.conf file with some information. How can I read it on Perl?
I am new to Perl and I am not sue would will I need to do.
I tried the following:
C:\Perl\Perl_Project>perl
#!/usr/local/bin/perl
open (MYFILE, 'test.conf');
while (<MYFILE>)
{ chomp; print "$_\n"; }
close (MYFILE);
I tried installing Perl on my laptop that has Windows 7 OS, and using command line.
Instead of using command line, write your program in a file (you can use any editor to write your program, I would suggest use Notepad++) and save as myprogram.pl in the same directory where you have your .conf file.
use warnings;
use strict;
open my $fh, "<", "test.conf" or die $!;
while (<$fh>)
{
chomp;
print "$_\n";
}
close $fh;
Now open a command prompt and go to the same path where you have your both file myprogram.pl and test.conf file and execute your program by typing this:
perl myprogram.pl
You can give full path of your input file inside program and can run your program from any path from command prompt by giving full path of your program:
perl path\to\myprogram.pl
Side note: Always use use warnings; and use strict; at the top of your program and to open file always use lexical filehandle with three arguments with error handling.
This is an extended comment more than an answer, as I believe #serenesat has given you everything you need to execute your program.
When you do "command line" Perl, it's typically stuff that is relatively brief or trivial, such as:
perl -e "print 2 ** 16"
Anything that goes beyond a few lines, and you're probably better off putting that in a file and having Perl run the file. You certainly can put larger programs on the command line, but when it comes to going back in and editing lines, it becomes more of a hassle than a shortcut.
Also, for what it's worth the -n and -p parameters allow you to process the contents of a stream, meaning you could do something like this:
perl -ne "print if /oracle/i" test.conf
I have perl version v5.8.3 installed on my windows machine.
While running a perl script having the below code, failing.
if(-e $file1)
I knew that this checks whether file1 is present or not.
The error just shown "perl command failed". Nothing else.
Could you please help me on this
You're using a version of Perl from 2004. You should seriously consider upgrading.
The file test operators like -e have been part of Perl for a very long time. They are certainly supported by Perl 5.8.3.
You say that your error is "perl command failed". That is not an error that is generated by Perl, so I suspect there is something else going on here that you're not telling us about (presumably because you think it isn't important).
If I had to guess why your -e test is failing, I'd say that it's because $file1 doesn't contain any information about the directory where the file can be found, and therefore Perl is looking in the wrong place. Perhaps you can get more information with code like this:
use Cwd;
if (-e $file1) {
...
} else {
die "Can't find file: " . cwd() . '/' . $file1;
}
This will show you where Perl is looking for the file.
I'm fairly confused. I just got a new development machine, and perl appears to be outputting nothing for print commands.
#!/usr/bin/perl
use warnings;
use strict;
print "A";
print STDOUT "B";
print STDERR "C";
open FH, ">", "testprint';
print FH "D";
close FH;
Produces nothing in the console, and testprint becomes a 1-bye (empty) file.
Even this produces nothing:
perl -e "print 'a';"
This occurs for both perl binaries that happen to be on my machine. I'm stumped about where to start debugging this problem. Any ideas?
EDIT:
perl -v
This is perl, v5.8.8 built for x86_64-linx-thread-multi
and
which perl
/usr/bin/perl
I believe the problem exists outside of Perl. Either
the terminal in some unusual state when you ran the script,
perl's parent process redirected perl's output away from the terminal, or
perl's parent process did not provide a STDOUT and STDERR for perl.
You might be able to gather more information by actually checking if print returned an error. (It always baffles me why people don't check for errors when something doesn't work they way they expect it to work.)
perl -we'print("a") or die("Can'\''t print: $!\n");'
You might be able to gather more information by using strace or whatever it's called on your system. (Look for write(1 and write(2.)
strace perl -we'print("a") or die("Can'\''t print: $!\n");'
But those should print nothing at all if the problem is outside of Perl, which is why it might be wise to try redirecting the output to a file and then examining the file and its size.
perl -we'print("a") or die("Can'\''t print: $!\n");' 1>out 2>err
The problem was not STDOUT missing or redirected from the shell, but rather that the shell was set to send a carriage return without a newline when writing a prompt, thus overwriting all output sent to the same line.
Specifically, my old version of zsh had promptcr set. See question 3.23 here for more information.
I want to run shell commands in a CGI program (written in Perl). My program doesn’t have root permission. It runs as nobody. I want to use this code:
use strict;
system <<'EEE';
awk '{a[$1]+=$2;b[$1]+=$3}END{for(i in a)print i, a[i], b[i]|"sort -nk 3"}' s.txt
EEE
I can run my code successfully with perl from the command line but not as a CGI program.
Based on the code in your question, there are at least four possibilities for failure.
The nobody user does not have permission to execute your program.
The Perl code in your question has no shebang (#!) line. You are trying to run awk, so I assume you are running on some form of Unix. If your code is missing this line, then your operating system does not know how to run your program.
The file s.txt is either not in the executing program’s working directory, or it is not readable by the nobody user.
For whatever reason, awk is not reachable via the PATH of your executing program’s environment.
To quickly diagnose such low-level problems, try to have all error output to show up in the browser. One way to do this is adding the following just after the shebang line in your code.
BEGIN {
print "Content-type: text/plain\n\n";
open STDERR, ">&", \*STDOUT or print "$0: dup: $!";
}
The output will render as plain text rather than HTML, but this is a temporary measure to see your program’s output. By wrapping it in a BEGIN block, the code executes as soon as it parses. Redirecting STDERR means your browser also gets anything written to the standard output.
Another way to do this is with the CGI::Carp module.
use CGI::Carp 'fatalsToBrowser';
This way, errors go to the browser and also to the web server’s error log.
If you still see 500-series errors from your server, the problem is happening at a lower level: probably some failure to start perl. Go examine your server’s error log. Once your program is executing, you can remove this temporary redirection of error output.
Finally, I recommend changing your program to
#! /usr/bin/perl -T
BEGIN { print "Content-type: text/plain\n\n"; }
use strict;
use warnings;
$ENV{PATH} = "/bin:/usr/bin";
my $input = "/path/to/your/s.txt";
my $buckets = <<'EOProgram'
{ a[$1] += $2; b[$1] += $3 }
END { for (i in a) print i, a[i], b[i] }
EOProgram
open STDIN, "-|", "awk", $buckets, $input or die "$0: open: $!";
exec "sort", "-nk", 3 or die "$0: exec: $!";
The -T switch enables a security dataflow analysis called taint mode that prevents you from using unsanitized input on system operations such as open, exec, and so on that an attacker (or benign user supplying unexpected input) could use to harm your system. You should always add -T to CGI programs and any other code that runs on behalf of another user.
Given the nature of your awk program, a content type of text/plain seems reasonable. Output it as soon as possible.
With taint mode enabled, be explicit about the value of your PATH environment variable. If instead you stick with whatever untrusted PATH your program inherits, attempting to run external programs will fail.
Nail down the full path of your input. This will eliminate surprises.
Using the multi-argument forms of open and exec eliminates the shell and its argument parsing. (For completeness, system also has a similar multi-argument form.) Yes, writing it this way can mean being a little more deliberate (such as breaking out the arguments and setting up the pipeline yourself), but it also avoids nasty surprises.
I'm sure nobody is allowed to run shell commands. The problem is that nobody doesn't have permission to open the file s.txt. Add read permission for everyone to s.txt, and add execute permission to everyone on every directory up to s.txt.
I would suggest finding out the full qualified path for awk and specifying it directly. Likely the nobody that launched httpd had a very minimal path in its $ENV{PATH}. Displaying the $ENV{PATH} I am guessing will show this.
This is a good thing, I wouldn't modify the path, but just specify the path /usr/bin/awk or what not.
If you have shell access and it works, type 'which awk' to find this out.
i can run my codes successfully in
perl file but not in cgi file.
What web server are you running under? For instance, apache requires printing a CGI header i.e. print "Content-type: text/plain; charset=utf-8\n\n", or
use CGI;
my $q = CGI->new();
print $q->header('text/html');
(See CGI)
Apache will conplain in the log (error.log) about "premature end of script headers" IF what I said is the case.
You could just do it inline without having to fork out to another process...
if ( open my $fh, '<', 's.txt' ) {
my %data;
while (<$fh>) {
my ($c1,$c2,$c3) = split;
$data{a}{$c1} += $c2;
$data{b}{$c1} += $c3;
}
foreach ( sort { $data{b}{$a} <=> $data{b}{$b} } keys %{ $data{b} } ) {
print "$_ $data{a}{$_} $data{b}{$_}\n";
}
} else {
warn "Unable to open s.txt: $!\n";
}
I'm writing a perl script that mimics gcc. This my script needs to process some stdout from gcc. The part for processing is done, but I can't get the simple part working: how can I forward all the command line parameters as is to the next process (gcc in my case). Command lines sent to gcc tend to be very long and can potentially contain lots of escape sequences and I don't want now to play that game with escaping and I know that it's tricky to get it right on windows in complicated cases.
Basically,
gcc.pl some crazies\ t\\ "command line\"" and that gcc.pl has to forward that same command line to real gcc.exe (I use windows).
I do it like that: open("gcc.exe $cmdline 2>&1 |") so that stderr from gcc is fed to stdout and I my perl script processes stdout. The problem is that I can't find anywhere how to construct that $cmdline.
I would use AnyEvent::Subprocess:
use AnyEvent::Subprocess;
my $process_line = sub { say "got line: $_[0]" };
my $gcc = AnyEvent::Subprocess->new(
code => ['gcc.exe', #ARGV],
delegates => [ 'CompletionCondvar', 'StandardHandles', {
MonitorHandle => {
handle => 'stdout',
callback => $process_line,
}}, {
MonitorHandle => {
handle => 'stderr',
callback => $process_line,
}},
],
);
my $running = $gcc->run;
my $done = $running->recv;
$done->is_success or die "OH NOES";
say "it worked";
The MonitorHandle delegate works like redirection, except you have the option of using a separate filter for each of stdout and stderr. The "code" arg is an arrayref representing a command to run.
"Safe Pipe Opens" in the perlipc documentation describes how to get another command's output without having to worry about how the shell will parse it. The technique is typically used for securely handling untrusted inputs, but it also spares you the error-prone task of correctly escaping all the arguments.
Because it sidesteps the shell, you'll need to create the effect of 2>&1 yourself, but as you'll see below, it's straightforward to do.
#! /usr/bin/perl
use warnings;
use strict;
my $pid = open my $fromgcc, "-|";
die "$0: fork: $!" unless defined $pid;
if ($pid) {
while (<$fromgcc>) {
print "got: $_";
}
}
else {
# 2>&1
open STDERR, ">&STDOUT" or warn "$0: dup STDERR: $!";
no warnings "exec"; # so we can write our own message
exec "gcc", #ARGV or die "$0: exec: $!";
}
Windows proper does not support open FH, "-|", but Cygwin does so happily:
$ ./gcc.pl foo.c
got: gcc: foo.c: No such file or directory
got: gcc: no input files
Read up on the exec function and the system function in Perl.
If you provide either of these with an array of arguments (rather than a single string), it invokes the Unix execve() function or a close relative directly, without letting the shell interpret anything, exactly as you need it to do.
Thanks for answers, I came to conclusion that I made a big mistake that I touched perl again: hours of time wasted to find out that it can't be done properly.
Perl uses different way to split command line parameters than all other apps that use MS stdlib (which is standard on win32).
Because of that some commandline parameters that were meant to be interpreted as a signle commandline argument, by perl can be interpreted as more than one argument. That means that all what I'm trying to do is waste of time because of that buggy behavior in perl. It's impossible to get this task done correctly if I 1) can't access original command line as is and 2) perl doesn't split command line arguments correctly.
as a simple test:
script.pl """test |test"
on win32 will incorrectly interpret command line as:
ARGV=['"test', '|test']
Whereas, the correct "answer" on windows has to be
ARGV=['"test |test']
I used activestate perl, I tried also latest version of strawberry perl: both suck. It appears that perl that comes with msys works properly, most likely because it was built against mingw instead of cygwin runtime?..
The problem and reason with perl is that it has buggy cmd line parser and it won't work on windows NO MATTER WHAT cygwin supports or not.
I have a simple case where an environment variable (which I cannot control) expands to
perl gcc.pl -c "-IC:\ffmpeg\lib_avutil\" rest of args
Perl sees that I have two args only: -c and '-IC:\ffmpeg\lib_avutil" rest of args'
whereas any conforming windows implementation receives second cmd line arg as: '-IC:\ffmpeg\lib_avutil\', that mean that perl is a huge pile of junk for my simple case, because it doesn't provide adequate means to access cmd line arguments. I'm better off using boost::regex and do all my parsing in c++ directly, at least I won't ever make dumb mistakes like ne and != for comparing strings etc. Windows's escaping rules for command line arguments are quite strange, but they are standard on windows and perl for some strange reason doesn't want to follow OS's rules.