I'm fairly confused. I just got a new development machine, and perl appears to be outputting nothing for print commands.
#!/usr/bin/perl
use warnings;
use strict;
print "A";
print STDOUT "B";
print STDERR "C";
open FH, ">", "testprint';
print FH "D";
close FH;
Produces nothing in the console, and testprint becomes a 1-bye (empty) file.
Even this produces nothing:
perl -e "print 'a';"
This occurs for both perl binaries that happen to be on my machine. I'm stumped about where to start debugging this problem. Any ideas?
EDIT:
perl -v
This is perl, v5.8.8 built for x86_64-linx-thread-multi
and
which perl
/usr/bin/perl
I believe the problem exists outside of Perl. Either
the terminal in some unusual state when you ran the script,
perl's parent process redirected perl's output away from the terminal, or
perl's parent process did not provide a STDOUT and STDERR for perl.
You might be able to gather more information by actually checking if print returned an error. (It always baffles me why people don't check for errors when something doesn't work they way they expect it to work.)
perl -we'print("a") or die("Can'\''t print: $!\n");'
You might be able to gather more information by using strace or whatever it's called on your system. (Look for write(1 and write(2.)
strace perl -we'print("a") or die("Can'\''t print: $!\n");'
But those should print nothing at all if the problem is outside of Perl, which is why it might be wise to try redirecting the output to a file and then examining the file and its size.
perl -we'print("a") or die("Can'\''t print: $!\n");' 1>out 2>err
The problem was not STDOUT missing or redirected from the shell, but rather that the shell was set to send a carriage return without a newline when writing a prompt, thus overwriting all output sent to the same line.
Specifically, my old version of zsh had promptcr set. See question 3.23 here for more information.
Related
In Perl, one way to read the STDOUT of a subprocess is to use open:
open(PIPE, "ls -l |");
I was looking for a more object-oriented way to do this, though, and I've been using IO::Pipe with some success. I want to detect errors, though, specifically if the command is not executable. I can't figure out how to do that via IO::Pipe, though. Here's what I have:
use strict;
use warnings;
use IO::Pipe;
my($cmd) = join (" ", #ARGV);
open(PIPE, "$cmd |") || die qq(error opening PIPE);
while (<PIPE>) {
chomp;
print "DBG1: $_\n";
}
close PIPE;
my($pipe) = IO::Pipe->new();
$pipe->reader($cmd);
die qq(error opening IO::Pipe) if $pipe->eof();
while (<$pipe>) {
chomp;
print "DBG2: $_\n";
}
$pipe->close();
If the sub-process command is invalid, both checks will correctly die. If the sub-process produces no output, though, eof() will report an error, even if the command itself is fine:
$ perl pipe.pl "ls -l >/dev/null"
error opening IO::Pipe at pipe.pl line 20.
A bunch of questions, then:
Is there a reasonable OO way to read from a sub-process in Perl? Is IO::Pipe the correct tool to use? If so, how do I check to make sure the sub-process command is created successfully? If not, what should I be using? I don't want to write to the sub-process, so I don't think I want IPC::Open2 or IPC::Open3. I would prefer to use a core module, if possible.
The issue is not IO::Pipe. The problem is eof is the wrong way to check for a pipe error. It doesn't mean there's no pipe, it means there's nothing to read from that pipe. You'd have the same problem with eof PIPE. It's perfectly fine for a sub-process to not print anything.
If you want to check the sub-process successfully ran, it turns out IO::Pipe already does that for you.
# IO::Pipe: Cannot exec: No such file or directory
$pipe->reader("hajlalglagl");
Backticks is not a core module but seems to do what your looking for.
use strict;
use warnings;
use Backticks;
my $cmd = Backticks->new(join (" ", #ARGV));
$cmd->run();
if ($cmd->success){
print $cmd->stdout
} else {
print "Command failed\n";
}
Running this with a valid command then invalid command gives the below results
io_pipe.pl "uname -o"
GNU/Linux
io_pipe.pl "uname -z"
Command failed
Update
As pointed out by #thisSuitIsNotBlack, this module changes the deafult behaviour of backticks in perl. You should read the Notes section of the POD. However the main one to be aware of is:
The overriding of backticks is provided by Filter::Simple. Source
filtering can be weird sometimes... if you want to use this module in
a purely traditional Perl OO style, simply turn off the source
filtering as soon as you load the module:
use Backticks;
no Backticks;
context: I'm a beginner in Perl and struggling, please be patient, thanks.
the question: there is a one-liner that seems to do the job I want (in a cygwin console it does fine on my test file). So now I would need to turn it into a script, but I can't manage that unfortunately.
The one-liner in question is provided in the answer by Aki here Delete lines in perl
perl -ne 'print unless /HELLO/../GOODBYE/' <file_name>
Namely I would like to have a script that opens my file "test.dat" and removes the lines between some strings HELLO and GOODBYE. Here is what I tried and which fails (the path is fine for cygwin):
#!/bin/perl
use strict;
use warnings;
open (THEFILE, "+<test.dat") || die "error opening";
my $line;
while ($line =<THEFILE>){
next if /hello/../goodbye/;
print THEFILE $line;
}
close (THEFILE);
Many thanks in advance!
Your one-liner is equivalent to the following
while (<>) {
print unless /HELLO/../GOODBYE/;
}
Your code does something quite different. You should not attempt to read and write to the same file handle, that usually does not do what you think. When you want to quickly edit a file, you can use the -i "in-place edit" switch:
perl -ni -e 'print unless /HELLO/../GOODBYE/' file
Do note that changes to the file are irreversible, so you should make backups. You can use the backup option for that switch, e.g. -i.bak, but be aware that it is not flawless, as running the same command twice will still overwrite your backup (by saving to the same file name twice).
The simplest and safest way to do it, IMO, is to simply use shell redirection
perl script.pl file.txt > newfile.txt
While using the script file I showed at the top.
I want to run shell commands in a CGI program (written in Perl). My program doesn’t have root permission. It runs as nobody. I want to use this code:
use strict;
system <<'EEE';
awk '{a[$1]+=$2;b[$1]+=$3}END{for(i in a)print i, a[i], b[i]|"sort -nk 3"}' s.txt
EEE
I can run my code successfully with perl from the command line but not as a CGI program.
Based on the code in your question, there are at least four possibilities for failure.
The nobody user does not have permission to execute your program.
The Perl code in your question has no shebang (#!) line. You are trying to run awk, so I assume you are running on some form of Unix. If your code is missing this line, then your operating system does not know how to run your program.
The file s.txt is either not in the executing program’s working directory, or it is not readable by the nobody user.
For whatever reason, awk is not reachable via the PATH of your executing program’s environment.
To quickly diagnose such low-level problems, try to have all error output to show up in the browser. One way to do this is adding the following just after the shebang line in your code.
BEGIN {
print "Content-type: text/plain\n\n";
open STDERR, ">&", \*STDOUT or print "$0: dup: $!";
}
The output will render as plain text rather than HTML, but this is a temporary measure to see your program’s output. By wrapping it in a BEGIN block, the code executes as soon as it parses. Redirecting STDERR means your browser also gets anything written to the standard output.
Another way to do this is with the CGI::Carp module.
use CGI::Carp 'fatalsToBrowser';
This way, errors go to the browser and also to the web server’s error log.
If you still see 500-series errors from your server, the problem is happening at a lower level: probably some failure to start perl. Go examine your server’s error log. Once your program is executing, you can remove this temporary redirection of error output.
Finally, I recommend changing your program to
#! /usr/bin/perl -T
BEGIN { print "Content-type: text/plain\n\n"; }
use strict;
use warnings;
$ENV{PATH} = "/bin:/usr/bin";
my $input = "/path/to/your/s.txt";
my $buckets = <<'EOProgram'
{ a[$1] += $2; b[$1] += $3 }
END { for (i in a) print i, a[i], b[i] }
EOProgram
open STDIN, "-|", "awk", $buckets, $input or die "$0: open: $!";
exec "sort", "-nk", 3 or die "$0: exec: $!";
The -T switch enables a security dataflow analysis called taint mode that prevents you from using unsanitized input on system operations such as open, exec, and so on that an attacker (or benign user supplying unexpected input) could use to harm your system. You should always add -T to CGI programs and any other code that runs on behalf of another user.
Given the nature of your awk program, a content type of text/plain seems reasonable. Output it as soon as possible.
With taint mode enabled, be explicit about the value of your PATH environment variable. If instead you stick with whatever untrusted PATH your program inherits, attempting to run external programs will fail.
Nail down the full path of your input. This will eliminate surprises.
Using the multi-argument forms of open and exec eliminates the shell and its argument parsing. (For completeness, system also has a similar multi-argument form.) Yes, writing it this way can mean being a little more deliberate (such as breaking out the arguments and setting up the pipeline yourself), but it also avoids nasty surprises.
I'm sure nobody is allowed to run shell commands. The problem is that nobody doesn't have permission to open the file s.txt. Add read permission for everyone to s.txt, and add execute permission to everyone on every directory up to s.txt.
I would suggest finding out the full qualified path for awk and specifying it directly. Likely the nobody that launched httpd had a very minimal path in its $ENV{PATH}. Displaying the $ENV{PATH} I am guessing will show this.
This is a good thing, I wouldn't modify the path, but just specify the path /usr/bin/awk or what not.
If you have shell access and it works, type 'which awk' to find this out.
i can run my codes successfully in
perl file but not in cgi file.
What web server are you running under? For instance, apache requires printing a CGI header i.e. print "Content-type: text/plain; charset=utf-8\n\n", or
use CGI;
my $q = CGI->new();
print $q->header('text/html');
(See CGI)
Apache will conplain in the log (error.log) about "premature end of script headers" IF what I said is the case.
You could just do it inline without having to fork out to another process...
if ( open my $fh, '<', 's.txt' ) {
my %data;
while (<$fh>) {
my ($c1,$c2,$c3) = split;
$data{a}{$c1} += $c2;
$data{b}{$c1} += $c3;
}
foreach ( sort { $data{b}{$a} <=> $data{b}{$b} } keys %{ $data{b} } ) {
print "$_ $data{a}{$_} $data{b}{$_}\n";
}
} else {
warn "Unable to open s.txt: $!\n";
}
I have a Perl script that runs a different utility (called Radmind, for those interested) that has the capability to edit the filesystem. The Perl script monitors output from this process, so it would be running throughout this whole situation.
What would happen if the utility being run by the script tried to edit the script file itself, that is, replace it with a newer version? Does Perl load the script and any linked libraries at the start of its execution and then ignore the script file itself unless told specifically to mess with it? Or perhaps, would all hell break loose, and executions might or might not fail depending on how the new file differed from the one being run?
Or maybe something else entirely? Apologies if this belongs on SuperUser—seems like a gray area to me.
It's not quite as simple as pavel's answer states, because Perl doesn't actually have a clean division of "first you compile the source, then you run the compiled code"[1], but the basic point stands: Each source file is read from disk in its entirety before any code in that file is compiled or executed and any subsequent changes to the source file will have no effect on the running program unless you specifically instruct perl to re-load the file and execute the new version's code[2].
[1] BEGIN blocks will run code during compilation, while commands such as eval and require will compile additional code at run-time
[2] Most likely by using eval or do, since require and use check whether the file has been loaded already and ignore it if it has.
For a fun demonstration, consider
#! /usr/bin/perl
die "$0: where am I?\n" unless -e $0;
unlink $0 or die "$0: unlink $0: $!\n";
print "$0: deleted!\n";
for (1 .. 5) {
sleep 1;
print "$0: still running!\n";
}
Sample run:
$ ./prog.pl
./prog.pl: deleted!
./prog.pl: still running!
./prog.pl: still running!
./prog.pl: still running!
./prog.pl: still running!
./prog.pl: still running!
Your Perl script will be compiled first, then run; so changing your script while it runs won't change the running compiled code.
Consider this example:
#!/usr/bin/perl
use strict;
use warnings;
push #ARGV, $0;
$^I = '';
my $foo = 42;
my $bar = 56;
my %switch = (
foo => 'bar',
bar => 'foo',
);
while (<ARGV>) {
s/my \$(foo|bar)/my \$$switch{$1}/;
print;
}
print "\$foo: $foo, \$bar: $bar\n";
and watch the result when run multiple times.
The script file is read once into memory. You can edit the file from another utility after that -- or from the Perl script itself -- if you wish.
As the others said, the script is read into memory, compiled and run. GBacon shows that you can delete the file and it will run. This code below shows that you can change the file and do it and get the new behavior.
use strict;
use warnings;
use English qw<$PROGRAM_NAME>;
open my $ph, '>', $PROGRAM_NAME;
print $ph q[print "!!!!!!\n";];
close $ph;
do $PROGRAM_NAME;
... DON'T DO THIS!!!
Perl scripts are simple text files that are read into memory, compiled in memory, and the text file script is not read again. (Exceptions are modules that come into lexical scope after compilation and do and eval statements in some cases...)
There is a well known utility that exploits this behavior. Look at CPAN and its many versions which is probably in your /usr/bin directory. There is a CPAN version for each version of Perl on your system. CPAN will sense when a new version of CPAN itself is available, ask if you want to install it, and if you say "y" it will download the newer version and respawn itself right where you left off without loosing any data.
The logic of this is not hard to follow. Read /usr/bin/CPAN and then follow the individualized versions related to what $Config::Config{version} would generate on your system.
Cheers.
I'm writing a perl script that mimics gcc. This my script needs to process some stdout from gcc. The part for processing is done, but I can't get the simple part working: how can I forward all the command line parameters as is to the next process (gcc in my case). Command lines sent to gcc tend to be very long and can potentially contain lots of escape sequences and I don't want now to play that game with escaping and I know that it's tricky to get it right on windows in complicated cases.
Basically,
gcc.pl some crazies\ t\\ "command line\"" and that gcc.pl has to forward that same command line to real gcc.exe (I use windows).
I do it like that: open("gcc.exe $cmdline 2>&1 |") so that stderr from gcc is fed to stdout and I my perl script processes stdout. The problem is that I can't find anywhere how to construct that $cmdline.
I would use AnyEvent::Subprocess:
use AnyEvent::Subprocess;
my $process_line = sub { say "got line: $_[0]" };
my $gcc = AnyEvent::Subprocess->new(
code => ['gcc.exe', #ARGV],
delegates => [ 'CompletionCondvar', 'StandardHandles', {
MonitorHandle => {
handle => 'stdout',
callback => $process_line,
}}, {
MonitorHandle => {
handle => 'stderr',
callback => $process_line,
}},
],
);
my $running = $gcc->run;
my $done = $running->recv;
$done->is_success or die "OH NOES";
say "it worked";
The MonitorHandle delegate works like redirection, except you have the option of using a separate filter for each of stdout and stderr. The "code" arg is an arrayref representing a command to run.
"Safe Pipe Opens" in the perlipc documentation describes how to get another command's output without having to worry about how the shell will parse it. The technique is typically used for securely handling untrusted inputs, but it also spares you the error-prone task of correctly escaping all the arguments.
Because it sidesteps the shell, you'll need to create the effect of 2>&1 yourself, but as you'll see below, it's straightforward to do.
#! /usr/bin/perl
use warnings;
use strict;
my $pid = open my $fromgcc, "-|";
die "$0: fork: $!" unless defined $pid;
if ($pid) {
while (<$fromgcc>) {
print "got: $_";
}
}
else {
# 2>&1
open STDERR, ">&STDOUT" or warn "$0: dup STDERR: $!";
no warnings "exec"; # so we can write our own message
exec "gcc", #ARGV or die "$0: exec: $!";
}
Windows proper does not support open FH, "-|", but Cygwin does so happily:
$ ./gcc.pl foo.c
got: gcc: foo.c: No such file or directory
got: gcc: no input files
Read up on the exec function and the system function in Perl.
If you provide either of these with an array of arguments (rather than a single string), it invokes the Unix execve() function or a close relative directly, without letting the shell interpret anything, exactly as you need it to do.
Thanks for answers, I came to conclusion that I made a big mistake that I touched perl again: hours of time wasted to find out that it can't be done properly.
Perl uses different way to split command line parameters than all other apps that use MS stdlib (which is standard on win32).
Because of that some commandline parameters that were meant to be interpreted as a signle commandline argument, by perl can be interpreted as more than one argument. That means that all what I'm trying to do is waste of time because of that buggy behavior in perl. It's impossible to get this task done correctly if I 1) can't access original command line as is and 2) perl doesn't split command line arguments correctly.
as a simple test:
script.pl """test |test"
on win32 will incorrectly interpret command line as:
ARGV=['"test', '|test']
Whereas, the correct "answer" on windows has to be
ARGV=['"test |test']
I used activestate perl, I tried also latest version of strawberry perl: both suck. It appears that perl that comes with msys works properly, most likely because it was built against mingw instead of cygwin runtime?..
The problem and reason with perl is that it has buggy cmd line parser and it won't work on windows NO MATTER WHAT cygwin supports or not.
I have a simple case where an environment variable (which I cannot control) expands to
perl gcc.pl -c "-IC:\ffmpeg\lib_avutil\" rest of args
Perl sees that I have two args only: -c and '-IC:\ffmpeg\lib_avutil" rest of args'
whereas any conforming windows implementation receives second cmd line arg as: '-IC:\ffmpeg\lib_avutil\', that mean that perl is a huge pile of junk for my simple case, because it doesn't provide adequate means to access cmd line arguments. I'm better off using boost::regex and do all my parsing in c++ directly, at least I won't ever make dumb mistakes like ne and != for comparing strings etc. Windows's escaping rules for command line arguments are quite strange, but they are standard on windows and perl for some strange reason doesn't want to follow OS's rules.