Redirect STDOUT to a file while process is waiting for user input - perl

I am trying to redirect the STDOUT and STDERR into a log file, but I also want to print those streams to the console. I am using Perl, and my code looks like this:
use Capture::Tiny ':all';
my $stderr, $stdout;
($stdout, $stderr) = capture {
system($command);
};
print $stdout;
print $stderr;
It works, but if the command waits for a user input, the program doesn't print $stdout to STDOUT until a key is pressed. Is there any way to print $stdout to STDOUT before it needs user input? Line by line approach would be fine.
Thank you in advance!

Well, I'm not familiar with Capture::Tiny so this may not be entirely relevant - generally though, if I'm looking to handle STDIN, STDOUT and/or STDERR then I look towards either open (if it's just one), or IPC::Open2 and [IPC::Open3][1] which open multiple file descriptors attached to a process.
use IPC::Open3;
$pid = open3(\*CHLD_IN, \*CHLD_OUT, \*CHLD_ERR,
'some cmd and args', 'optarg', ...);
use IPC::Open2;
$pid = open2(\*CHLD_OUT, \*CHLD_IN, 'some', 'cmd', 'and', 'args');
Although I would suggest rather than the examples - you can use lexical filehandles:
my($chld_out, $chld_in);
$pid = open2($chld_out, $chld_in, 'some cmd and args');
You can then read and write from your filehandles (bear in mind though - by default a read will be blocking).
You do need to close and then (ideally) waitpid to clear up the process when you're done though.

You need to use Capture::Tiny's tee instead of capture.
The tee function works just like capture, except that output is captured as well as passed on to the original STDOUT and STDERR.
Just replace the function call and your output will end up both in the variables and on the screen.
use Capture::Tiny ':all';
my $stderr, $stdout;
($stdout, $stderr) = tee {
system($command);
};

Simple approach that I could think of:
#! /usr/bin/perl -w
# Using perl one liner as a command here
# which prints a string to STDOUT and STDERR
my $cmd = "perl -e 'print STDOUT \"stdout\n\"; print STDERR \"stderr\n\";'";
my $log = "./test.log";
# using "2>&1" we are redirecting stderr also to stdout
system("$cmd 2>&1 | tee $log");
# Sample run results in both the strings getting printed to console as well as to log file
> perl test.pl
stderr
stdout
> cat test.log
stderr
stdout

Related

How to get STDERR in Perl from a command executed in pipe with su -c

I'm trying to capture the output of the command executed as a different user using:
my $command = qq(sudo su - <username> -c '/usr/bin/whatever');
my $pid = open $cmdOutput, "-|", $command;
How can I capture the STDERR of /usr/bin/whatever?
I tried
$pid = open $cmdOutput, "-|", $command || die " something went wrong: $!";
but it looks like this is capturing the possible errors of "open" itself.
I also tried
my $command = qq(sudo su - <username> -c '/usr/bin/whatever' 2>/tmp/error.message);
which will redirect the STDERR to the file, which I can parse later, but I wanted some more straightforward solution.
Also, I only want to use core modules.
This is covered thoroughly in perlfaq8. Since you are using a piped open, the relevant examples are those that go by open3 from the core IPC::Open3 module.
Another option is to use IPC::Run for managing your processes, and the pump function will do what you need. The IPC::Open3 documentation says for IPC::Run
This is a CPAN module that has better error handling and more facilities than Open3.
With either of these you can manipulate STDOUT and STDERR separately or together, as needed. For convenient and complete output capture also see Capture::Tiny.
Other than 2>output redirection, there are no more elementary methods for the piped open.
If you don't mind mixing the streams or losing STDOUT altogether, another option is
my $command = 'cmd 2>&1 1>/dev/null' # Remove 1>/dev/null to have both
my $pid = open my $cmdOutput, "-|", $command;
while (<$cmdOutput>) { print } # STDERR only
The first redirection merges STDERR stream with STDOUT so you get them both, and mixed (with STDOUT subject to buffering, thus things may well come out of order). The second redirect sends the STDOUT away so with it in place you read only the command's STDERR from the handle.
The question is about running an external command using open but I'd like to mention that the canonical and simple qx (backticks) can be used in the same way. It returns the STDOUT so redirection just like above is needed to get STDERR. For completeness:
my $cmd = 'cmd_to_execute';
my $allout = qx($cmd 2>&1); # Both STDOUT and STDERR in $out, or
my $stderr = qx($cmd 2>&1 1>/dev/null); # Only STDERR
my $exit_status = $?;
The qx puts the child process exit code (status) in $?. This can then be inspected for failure modes; see a summary in the qx page or a very thorough discussion in I/O operators in perlop.
Note that the STDERR returned this way is from the command, if it ran. If the command itself couldn't be run (for a typo in command name, or fork failed for some reason) then $? will be -1 and the error will be in $!.
As suggested by zdim I used the IPC::Open3 module for the matter and I've got something like this doing the job for me
$instanceCommand = qq(sudo su - <username> -c '<command>');
my ($infh,$outfh,$errfh,$pid);
$errfh = gensym();
$pid = open3($infh, $outfh, $errfh, $instanceCommand);
my $sel = new IO::Select;
$sel->add($outfh,$errfh);
while (my #ready = $sel->can_read){
foreach my $fh (#ready){
my $line =<$fh>;
if (not defined $line){
$sel->remove($fh);
next;
}
if ($fh == $outfh){
chomp($line);
#<----- command output processing ----->
}
elsif ($fh == $errfh){
chomp $line;
#<----- command error processing ----->
}
else {
die "Reading from something else\n";
}
}
}
waitpid($pid, 0);
Maybe not completely bullet proof, but its working fine for me. Even whilst executing funny cascaded script as < command > .
The desired destination, opened for writing, could be dup()'ed to FD #2

From inside a perl script can you know the name of the file you are redirecting output to?

So I have:
test.pl > test.log
is there a way to know inside test.pl that I am outputing to 'test.log'? At the end of my script I want to do some manipulation of test.log without hardcoding the name.
Maybe. The following works on Linux, but will not be very portable to other systems...
#!/usr/bin/env perl
use strict;
use warnings;
my $out = readlink("/proc/$$/fd/1");
print STDERR "I am being output to $out\n";
Naturally, this is probably a bad idea. Better to explicitly open the file and write to it in Perl, rather than having the shell set up redirections.
You can redirect standard output from perl, with minimal changes to your script,
test.pl test.log
my ($file) = #ARGV;
if (#ARGV) {
open STDOUT, ">", $file or die $!;
}
print "output is redirected to $file\n";
# use $file at the end

open(HANDLE, "-|", $cmd, #args) : how to read stderr?

I want to read the output of a command including the stderr into HANDLE:
open(HANDLE, "-|", $cmd, #args);
But the above command just reads the stdin?
How can I also read the stderr?
The IPC::Run module provides a run function that works like a supercharged system. It allows us to collect the output of STDERR and STDOUT combined:
run [$cmd, #args], '&>', \my $output;
after that, the $output variable holds the combined output as a string.
Example:
use IPC::Run qw/ run /;
run ['perl', '-E say "stdout"; say STDERR "stderr"'], '&>', \my $output;
print uc $output;
Output:
STDOUT
STDERR
I don't know how to use a filehandle in place of the scalar reference so that the output can be read normally in a while(<$fh>) loop.
You're going to want to look at IPC::Open3 which starts a process and provide separate file handles for writing to the child, and reading the child's STDOUT and STDERR.
I use Bash redirection as below in my perl code:
open (CMDOUT, "df -h 2>&1 |");

How to capture both STDOUT and STDERR in two different variables using Backticks in Perl

Let's say I want to run an external program from my script with backticks and at the same time I want to capture both STDOUT and STDERR but in two different variables. How can I do that? For istance if I run this script...
my $cmd = `snmpwalk -v $version -c $community $hostname $oid`;
...if there is no error everything works just fine BUT if the command raise an error this error will be printed on the command line and I don't want that to happen. I want to capture the error as well. Nothing has to be printed on the screen. Any ideas?
You needn't go all the way to open3, which IIRC is only for when you need to read and write to an external command, and even then there are other methods.
For your problem I suggest using Capture::Tiny, which can capture (or even tee) the STDOUT and STDERR from anything run inside its block. For example, per your question:
#!/usr/bin/env perl
use strict;
use warnings;
use Capture::Tiny qw/capture/;
...
my ($stdout, $stderr) = capture {
system ( "snmpwalk -v $version -c $community $hostname $oid" );
};
For another example consider this functioning code:
#!/usr/bin/env perl
use strict;
use warnings;
use Capture::Tiny qw/capture/;
my ($stdout, $stderr) = capture {
system ( "echo 'hello'" );
system ( "date" );
warn "Arg1!";
};
print "STDOUT:\n$stdout";
print "STDERR:\n$stderr";
which just gave me:
STDOUT:
hello
Mon Dec 19 23:59:06 CST 2011
STDERR:
Arg1! at ./test.pl line 11.
The only way to do this with backticks is to redirect to a file inside the shell command:
my $cmd = `snmpwalk -v $version -c $community $hostname $oid 2>error.dat`;
If you want to capture the STDERR inside your script, you need IPC::Open3 instead of backticks
In the Perl FAQ you have different options depending how do you want to proceed:
http://perldoc.perl.org/perlfaq8.html#How-can-I-capture-STDERR-from-an-external-command%3f
IO::CaptureOutput
is a very convenient wrapper for what you want to do.

command output is directed to the screen

novice Perl programmer here!.
I'm using the system() function to get the return code for an external program (in this case - php), however, the command's output is still printed to the screen.
How do I prevent it from doing so?
My code is:
use strict; use warnings;
print 'Return code:', system('php -l wrong.php'), "\n";
This code does print the return code, but it also print the the output of the command executed.
Any help will be appreciated!
EDIT: further testing shown that this happens while only using the php lint command.. using it with other commands doesnt print anything...
What you want is IPC::Open3:
use IPC::Open3;
use Symbol qw(gensym);
my $err = gensym; #work around IPC::Open3's interface
my $pid = open3 my $wtr, my $rdr, $err,
'some cmd', 'arg1', 'arg2', ...;
The separates out stdin into $wtr, stdout into $rdr, and stderr into $err
This also gives you maximum control over communicating with the process
If you are on a UNIX-like OS, you should be able to redirect the output in the command:
Try:
system('php -l wrong.php >> /dev/null') to get rid of what's being sent to stdout.
You could also open the command with a pipe to handle the output directly, which should be more portable.