Here is a little Perl server. It displays (1), accepts a line of input, then displays (2), etc. If you type "error" or "commit", it gives a custom message. If you type "exit", it quits. Otherwise, it just endlessly takes lines of input.
use strict;
use warnings;
$|++;
my $counter = 1;
print "($counter) ";
while (<STDIN>) {
chomp;
if ($_ eq "error") {print "Error on command #$counter\n";}
if ($_ eq "commit") {print "Committing data\n";}
if ($_ eq "exit") {print "Exiting program...\n"; exit;}
$counter++;
print "($counter) ";
}
Now, here is an Expect.pm client script to interact with the server script by typing in various lines.
use strict;
use warnings;
use Expect;
$|++;
my $exp = new Expect;
$exp->raw_pty(1);
$exp->log_file("/tmp/expect.out");
$exp->log_stdout(1);
my #commands = (
"This is the first command",
"Here is the second command",
"error",
"commit",
"This is the last command",
"exit",
);
$exp->spawn("./expecttest_server.pl");
foreach my $command (#commands) {
print "$command\n";
$exp->send("$command\n");
$exp->expect(1, '-re','\(\d+\)');
}
$exp->soft_close();
What I want is to be able to store the entire session from start to finish, including everything the server script generated, and everything the Expect.pm script sent.
That is, I want my client script to be able to return output like this, which is what you would see if you ran and interacted with the server script manually:
(1) This is the first command
(2) Here is the second command
(3) error
Error on command #3
(4) commit
Committing data
(5) This is the last command
(6) exit
Exiting program...
But the STDOUT display that comes from running the client script looks like this:
This is the first command
(1) (2) Here is the second command
error
(3) Error on command #3
(4) commit
This is the last command
Committing data
(5) (6) exit
Exiting program...
and the file specified by $exp->log_file (tmp/expect.out) shows this:
(1) (2) (3) Error on command #3
(4) Committing data
(5) (6) Exiting program...
I've tried experimenting by logging various combinations of the command itself + the before_match and after_match variables returned by $exp->expect(). But so far I haven't gotten the right combination. And it seems like an awfully clunky way to get what I'm looking for.
So, what's the best practice for capturing the entirety of an Expect.pm session?
Thanks to anyone who can help!
When run on the command line, your server prints
(1)
to stdout immediately and waits for input.
However, when you create an Expect object, you are actually setting up a PTY (a pseudo terminal). Any processes you spawn will have their stdin and stdout connected to this PTY, not to the TTY that your shell is connected to. This means that it's up to your Expect object whether output from the spawned process is displayed or not; it will not be displayed automatically.
When you spawn a process, the Expect object holds onto any output in an input buffer. When you send a string to the process, any additional output that is generated will be read into the buffer. If the PTY has echoing enabled (the default), the string you send will be echoed back, but the contents of the Expect object's buffer will not.
When you call the expect method, Expect waits until a matching string appears in the input buffer. If a match is found before the timeout expires, expect returns and prints the matching string.
So, all you need to do is call expect before sending your first command, something like this:
Server
use strict;
use warnings;
$| = 1;
my $counter = 1;
do {
print "($counter) ";
$counter++;
} while (<>);
Client
use strict;
use warnings;
use Expect;
$| = 1;
my $exp = Expect->new;
my $server = './expect_server';
$exp->spawn($server);
my #commands = qw(foo bar baz);
foreach my $command (#commands) {
$exp->expect(1, '-re', '\(\d+\)');
$exp->send("$command\r");
}
$exp->soft_close;
Output
(1) foo
(2) bar
(3) baz
(4)
Note that this is the exact same process you would use when interacting with a process manually: wait for the prompt, then type your command.
Related
I have been running some commands with the IPC::Run module and everything is fine, except that I can't access the output (STDOUT, STDERR), the process produced and were redirected into variables. Is there a way to retrieve those in the error handling?
#commands = ();
foreach my $id (1..3) {
push #commands, ["perl", "script" . $id . ".pl"];
}
foreach my $cmd (#commands) {
my $out = "";
my $err = "";
my $h = harness $cmd, \undef, \$out, \$err, timeout(12,exception => {name => 'timeout'});
eval {
run $h;
};
if ($#) {
my $err_msg = $#; # save in case another error happens
print "$out\n";
print "$err\n";
$h->kill_kill;
}
}
I don't need any input for now, I just need to execute it and get the output.
EDIT
I have been testing it with running perl scripts which look like this:
for (my $i = 0; $i < 10; $i++) {
sleep 1;
print "Hello from script 1 " . localtime() . "\n";
}
I have 3 such scripts with different times and the 3rd takes 20 seconds to complete, which is more than the 12 I have in the timer.
As noted by #ysth, the reason you do not get any output, is that the STDOUT and STDERR of the process corresponding to the command $cmd, is not line buffered, but rather block buffered. So all output is collected in a buffer which is not shown (printed) until the buffer is full or it is explicitly flushed. However, when your command times out, all the output is still in the buffer and has not yet been flushed and hence collected into the variable $out in the parent process (script).
Also note that since your $cmd script is a Perl script, this behavior is documented in perlvar:
$|
If set to nonzero, forces a flush right away and after every write
or print on the currently selected output channel. Default is 0
(regardless of whether the channel is really buffered by the system or
not; $| tells you only whether you've asked Perl explicitly to flush
after each write). STDOUT will typically be line buffered if output is
to the terminal and block buffered otherwise.
The problem (that the program is not connected to a terminal or a tty) is also noted in the documentation page for IPC::Run :
Interactive applications are usually optimized for human use. This can
help or hinder trying to interact with them through modules like
IPC::Run. Frequently, programs alter their behavior when they detect
that stdin, stdout, or stderr are not connected to a tty, assuming
that they are being run in batch mode. Whether this helps or hurts
depends on which optimizations change. And there's often no way of
telling what a program does in these areas other than trial and error
and occasionally, reading the source. This includes different versions
and implementations of the same program.
The documentation also lists a set of possible workarounds, including using pseudo terminals.
One solution for your specific case is then to explicitly make STDOUT line buffered at the beginning of your script:
STDOUT->autoflush(1); # Make STDOUT line buffered
# Alternatively use: $| = 1;
for (my $i = 0; $i < 10; $i++) {
sleep 1;
print "Hello from script 1 " . localtime() . "\n";
}
Edit:
If you cannot modify the scripts you are running for some reason, you could try connect the script to a pseudo terminal. So instead of inserting statements like STDOUT->autoflush(1) in the source code of the script, you can fool the script to believe it is connected to a terminal, and hence that it should use line buffering. For your case, we just add a >pty> argument before the \$out argument in the call to harness:
my $h = harness $cmd, \undef, '>pty>', \$out,
timeout(12, exception => {name => 'timeout'});
eval {
run $h;
};
I have a perl script that now works from the windows command line (ActivePerl installed). It sends commands to turn ports on/off on a power controller (smart power strip). I have it setup so that I take a couple of command line arguments. The IP address of the device, and the power operation I want to take on that device. i.e. 10.0.40.1 on should turn the device with the IP 10.0.40.1 on, or rather turn the appropriate port on in the power controller.
I have an if elseif statement that will evaluate the IP address passed to the script and determine which power controller and what port is appropriate. The only other operation is the port and the operation are joined so if the device is on port 3 of the controller and the operation passed was 'on' then the new variable becomes 3on. This is what the remainder of the script understands and being a novice, it was easier to leave that unchanged.
THE ISSUE: When I run this command from windows either by
perl Lpower.pl 10.0.30.15 on
or
Lpower.pl 10.0.30.15 on
The issue is the point of this was to allow another application to invoke the script on my client. The other app is a piece of network management software. Through supported custom GUI extension I was able to add items to the menu for each device being managed. I have one for "Power On" and another for "Power Off". I point it to the Lpower script with the path local to my client machine, and it is invoked and runs, it also passes the arguments as expected, but my if statement does not work. It will not match on any IP, nor will it match on the operation "on" "off". I don't need to evaluate the latter, I just put a statement in their to test.
My script prints the variables, and they do show up, but the if statement doesn't work. It does not appear that it is adding leading whitespace or anything.
Here is a code snip of the relevant parts:
# $language = "PerlScript"
# $interface = "1.0"
#!c:\perl64\bin\perl -w -CA
...
print STDERR "UserUtil $version\n\n";
my ($ipaddr, $onoff) = #ARGV;
print "$ipaddr $onoff\n";
if ($onoff eq "off") {print "it matched off"}
if ($ipaddr eq "10.0.40.1") {
$epc='10.0.30.92';
$port='1';
}
elsif ($ipaddr eq "10.0.40.105") {
$epc='10.0.30.92';
$port='2';
}
elsif (($ipaddr eq "10.0.40.100") || ($ipaddr eq "10.0.40.101")) {
$epc='10.0.30.92';
$port='3';
}
...
else {print "no matches found"}
$oper="$port$onoff";
$base='http://'.$auth.'#'.$epc.'/';
print "$epc $oper $ipaddr";
cmd($oper) && die "Unknown Command $_\n";
There are some print statements in there for debugging. The output from CLI looks like this (and it works, ports power on/off as expected):
10.0.30.15 off
it matched off10.0.30.93 1off 10.0.30.15
Invoked from the web-app (which launches the script on my client)
10.0.30.15 on
Unknown Command
no matches found 10.0.30.15 off
Why would it be different? Anything I can do to force the correct handling of the arguments passed? If I read correctly they are expected to be UTF-8 encoded strings, could that app be sending something different and therefore the if statements break?
One more note. The app calls the script directly and passes the arguments, it does not call the perl executable, my environment knows what to do with a pl script. Not sure if that makes a difference.
Any Help is much appreciated. I do have some coding in my distant back-ground, but never for a profession, and completely new to perl.
The relevant part of your script
my ($ipaddr, $onoff) = #ARGV;
print "$ipaddr $onoff\n";
if ($ipaddr eq "10.0.40.1") { print "matched\n"; }
else {print "no matches found\n"}
run it as
perl power.pl 10.0.40.1 off
prints:
10.0.40.1 off
matched
run it as
perl power.pl "10.0.40.1 off" #two words as one arg
prints
10.0.40.1 off
no matches found
when you add
use strict;
use warnings;
my ($ipaddr, $onoff) = #ARGV;
print "$ipaddr $onoff\n";
if ($ipaddr eq "10.0.40.1") { print "matched\n"; }
else {print "no matches found\n"}
will print:
Use of uninitialized value $onoff in concatenation (.) or string at power.pl line 5.
10.0.40.1 off
no matches found
I want to display an iterative progress bar during the execution of a particular command in my Perl-CGI program. I use the CGI::ProgressBar module to achieve this. For example, if I want to show the progress bar during the execution of an RKHunter scan, this is the code I wrote:
use CGI::ProgressBar qw/:standard/;
$| = 1;
print progress_bar( -from=>1, -to=>100 );
open(my $quik_rk, '-|', 'rkhunter', '--enable', '"known_rkts"') or print "ERROR RUNNING BASIC ROOTKIT CHECK!!";
# print the progress bar while the command executes
while(<$quik_rk>)
{
print update_progress_bar;
#print "<img src=\"ajax-loader.gif\"></img>";
}
close($quik_rk);
This works fine. However, I try the same on another command(this one's to scan using Linux Maldet) immediate after the code above:
open(my $quik_lmd, '-|', 'maldet', '-a', '/home?/?/public_html') or print "ERROR RUNNING BASIC MALWARE CHECK!!";
my $this_ctr = 0;
while(<$quik_lmd>)
{ $this_ctr++;
print update_progress_bar;
}
close($quik_lmd);
The progress bar doesn't execute but te command itself runs in the background.
What am I doing wrong?
Is there a better way to show a progress bar on a browser in Perl-CGI?
I am not familiar with RKHunter, but based on your results my guess is that it outputs a line of text for each test it runs, while the other command does not.
Each line of text output by RKHunter will trigger the next iteration of <$quik_rk>.
The second command, <$quik_lmd>, it is likely silent, so it never triggers the loop. Once the command terminates, execution continues after your while.
The key bit here is "line of text". The <$filehandle> operator returns a line of text each time it sees a newline character. In order to do what you want using this construct, you would need to coerce the second command into being verbose about it's activities, and most importantly, to be verbose with a lot of newlines.
Alternatively, you can open a background process and use sleep to manage your loop, e.g.,
use strict;
use POSIX qw(WNOHANG);
my $pid = open(my $quik_rk, '-|', 'sleep', '5'); # replace with your command
do {
print "waiting\n"; # update_progress_bar;
sleep 1; # seconds between update
} while (waitpid($pid, WNOHANG)==0);
I want to execute an external command from within my Perl script, putting the output of both stdout and stderr into a $variable of my choice, and to get the command's exit code into the $? variable.
I went through solutions in perlfaq8 and their forums, but they're not working for me. The strange thing is that I don't get the output of sdterr in any case, as long as the exit code is correct.
I'm using Perl version 5.8.8, on Red Hat Linux 5.
Here's an example of what I'm trying:
my $cmd="less";
my $out=`$cmd 2>&1`;
or
my $out=qx($cmd 2>&1);
or
open(PIPE, "$cmd 2>&1|");
When the command runs successfully, I can capture stdout.
I don't want to use additional capture modules. How can I capture the full results of the external command?
This was exactly the challenge that David Golden faced when he wrote Capture::Tiny. I think it will help you do exactly what you need.
Basic example:
#!/usr/bin/env perl
use strict;
use warnings;
use Capture::Tiny 'capture';
my ($stdout, $stderr, $return) = capture {
system( 'echo Hello' );
};
print "STDOUT: $stdout\n";
print "STDERR: $stderr\n";
print "Return: $return\n";
After rereading you might actually want capture_merged to join STDOUT and STDERR into one variable, but the example I gave is nice and general, so I will leave it.
Actually, the proper way to write this is:
#!/usr/bin/perl
$cmd = 'lsss';
my $out=qx($cmd 2>&1);
my $r_c=$?;
print "output was $out\n";
print "return code = ", $r_c, "\n";
You will get a '0' if no error and '-1' if error.
STDERR is intended to be used for errors or messages that might need to be separated from the STDOUT output stream. Hence, I would not expect any STDERR from the output of a command like less.
If you want both (or either) stream and the return code, you could do:
my $out=qx($cmd 2>&1);
my $r_c=$?
print "output was $out\n";
print "return code = ", $r_c == -1 ? $r_c : $r_c>>8, "\n";
If the command isn't executable (perhaps because you meant to use less but wrote lsss instead), the return code will be -1. Otherwise, the correct exit value is the high 8-bits. See system.
A frequently given answer to this question is to use a command line containing shell type redirection. However, suppose you want to avoid that, and use open() with a command and argument list, so you have to worry less about how a shell might interpret the input (which might be partly made up of user-supplied values). Then without resorting to packages such as IPC::Open3, the following will read both stdout and stderr:
my ($child_pid, $child_rc);
unless ($child_pid = open(OUTPUT, '-|')) {
open(STDERR, ">&STDOUT");
exec('program', 'with', 'arguments');
die "ERROR: Could not execute program: $!";
}
waitpid($child_pid, 0);
$child_rc = $? >> 8;
while (<OUTPUT>) {
# Do something with it
}
close(OUTPUT);
I am trying to communicate with an interactive process. I want my perl script to be a "moddle man" between the user and the process. The process puts text to stdout, prompts the user for a command, puts more text to stdout, prompts the user for a command, ....... A primitive graphic is provided:
User <----STDOUT---- interface.pl <-----STDOUT--- Process
User -----STDIN----> interface.pl ------STDIN---> Process
User <----STDOUT---- interface.pl <-----STDOUT--- Process
User -----STDIN----> interface.pl ------STDIN---> Process
User <----STDOUT---- interface.pl <-----STDOUT--- Process
User -----STDIN----> interface.pl ------STDIN---> Process
The following simulates what I'm trying to do:
#!/usr/bin/perl
use strict;
use warnings;
use FileHandle;
use IPC::Open2;
my $pid = open2( \*READER, \*WRITER, "cat -n" );
WRITER->autoflush(); # default here, actually
my $got = "";
my $input = " ";
while ($input ne "") {
chomp($input = <STDIN>);
print WRITER "$input \n";
$got = <READER>;
print $got;
}
DUe to output buffering the above example does not work. No matter what text is typed in, or how many enters are pressed the program just sits there. The way to fix it is to issue:
my $pid = open2( \*READER, \*WRITER, "cat -un" );
Notice "cat -un" as opposed to just "cat -n". -u turns off output buffering on cat. When output buffering is turned off this works. The process I am trying to interact with most likely buffers output as I am facing the same issues with "cat -n". Unfortunately I can not turn off output buffering on the process I am communicating with, so how do I handle this issue?
UPDATE1 (using ptty):
#!/usr/bin/perl
use strict;
use warnings;
use IO::Pty;
use IPC::Open2;
my $reader = new IO::Pty;
my $writer = new IO::Pty;
my $pid = open2( $reader, $writer, "cat -n" );
my $got = "";
my $input = " ";
$writer->autoflush(1);
while ($input ne "") {
chomp($input = <STDIN>);
$writer->print("$input \n");
$got = $reader->getline;
print $got;
}
~
There are three kinds of buffering:
Block buffering: Output is placed into a fixed-sized buffer. The buffer is flushed when it becomes full. You'll see the output come out in chunks.
Line buffering: Output is placed into a fixed-sized buffer. The buffer is flushed when a newline is added to the buffer and when it becomes full.
No buffering: Output is passed directly to the OS.
In Perl, buffering works as follows:
File handles are buffered by default. One exception: STDERR is not buffered by default.
Block buffering is used. One exception: STDOUT is line buffered if and only if it's connected to a terminal.
Reading from STDIN flushes the buffer for STDOUT.
Until recently, Perl used 4KB buffers. Now, the default is 8KB, but that can be changed when Perl is built.
This first two are surprisingly standard across all applications. That means:
User -------> interface.pl
User is a person. He doesn't buffer per say, though it's a very slow source of data. OK
interface.pl ----> Process
interface.pl's output is block buffered. BAD
Fixed by adding the following to interface.pl:
use IO::Handle qw( );
WRITER->autoflush(1);
Process ----> interface.pl
Process's output is block buffered. BAD
Fixed by adding the following to Process:
use IO::Handle qw( );
STDOUT->autoflush(1);
Now, you're probably going to tell me you can't change Process. If so, that leaves you three options:
Use a command line or configuration option provided by tool to change its buffering behaviour. I don't know of any tools that provide such an option.
Fool the child to use line buffering instead of block buffering by using a pseudo tty instead of a pipe.
Quitting.
interface.pl -------> User
interface.pl's output is line buffered. OK (right?)