If I have this perl app:
print `someshellscript.sh`;
that prints bunch of stuff and takes a long time to complete, how can I print that output in the middle of execution of the shell script?
Looks like Perl will only print the someshellscript.sh result when it completes, is there a way to make output flush in the middle of execution?
What you probably want to do is something like this:
open(F, "someshellscript.sh|");
while (<F>) {
print;
}
close(F);
This runs someshellscript.sh and opens a pipe that reads its output. The while loop reads each line of output generated by the script and prints it. See the open documentation page for more information.
The problem here is that escaping with backticks stores your script to a string, which you then print. For this reason, there would be no way to "flush" with print.
Using the system() command should print output continuously, but you won't be able to capture the output:
system "someshellscript.sh";
Related
I have a Perl script
for my $i (1..10) {
print "How $i\n";
sleep(1);
}
I want to run it from another perl script and capture its output. I know that I can use qx() or backticks, but they will return all the output simultaneously when the program exits.
But, what I instead want is to print the output of the first script from the second script as soon as they are available ie. for the example code the output of the first script is printed in ten steps from the second script and not in one go.
I looked at this question and wrote my second script as
my $cmd = "perl a.pl";
open my $cmd_fh, "$cmd |";
while(<$cmd_fh>) {
print "Received: $_\n";
STDOUT->flush();
}
close $cmd_fh;
However, the output is still being printed simultaneously. I wanted to know a way to get this done ?
The child sends output in chunks of 4 or KiB or 8 KiB rather than a line at a time. Perl programs, like most programs, flush STDOUT on linefeed, but only when connected to a terminal; they fully buffer the output otherwise. You can add $| = 1; to the child to disable buffering of STDOUT.
If you can't modify the child, such programs can be fooled by using pseudo-ttys instead of pipes. See IPC::Run. (Search its documentation for "pty".)
I basically want to reopen STDERR/STDOUT so they write to one logfile with both the stream and the timestamp included on every line. So print STDERR "Hello World" prints STDERR: 20130215123456: Hello World. I don't want to rewrite all my print statements into function calls, also some of the output will be coming from external processes via system() calls anyway which I won't be able to rewrite.
I also need for the output to be placed in the file "live", i.e. not only written when the process completes.
(p.s. I'm not asking particularly for details of how to generate timestamps, just how to redirect to a file and prepend a string)
I've worked out the following code, but it's messy:
my $mode = ">>";
my $file = "outerr.txt";
open(STDOUT, "|-", qq(perl -e 'open(FILE, "$mode", "$file"); while (<>) { print FILE "STDOUT: \$\_"; }'));
open(STDERR, "|-", qq(perl -e 'open(FILE, "$mode", "$file"); while (<>) { print FILE "STDERR: \$\_"; }'));
(The above doesn't add dates, but that should be trivial to add)
I'm looking for a cleaner solution, one that doesn't require quoting perl code and passing it on the command line, or at least module that hides some of the complexity. Looking at the code for Capture::Tiny it doesn't look like it can handle writing a part of output, though I'm not sure about that. annotate-output only works on an external command sadly, I need this to work on both external commands and ordinary perl printing.
The child launched via system doesn't write to STDOUT because it does not have access to variables in your program. Therefore, means having code run on a Perl file handle write (e.g. tie) won't work.
Write another script that runs your script with STDOUT and STDERR replaced with pipes. Read from those pipes and print out the modified output. I suggest using IPC::Run to do this, because it'll save you from using select. You can get away without it if you combine STDOUT and STDERR in one stream.
I have the following command line that outputs vmstat every second with a time stamp on each line via the perl script:
vmstat 15 | /home/Beer/addtimestamp.pl > File_1
the contents of the addtimestamp.pl:
!/usr/bin/perl
while (<>) { print localtime() . ": $_"; }
So why doesn't the output get redirected to the "File_1" file?
It works perfectly when I don't, it prints out the output perfectly every second with no issues at all.
When outputting to the terminal, perl's output is line buffered, so you will see every line as it is output. When its output is a file, it will be block buffered so you will not see any output until a full block is ready to write (4k I think, but variable and system-defined).
You need to set stdout to use line buffering:
$|=1;
Search for [perl line buffered output] and you'll see plenty of results about this.
I have a Perl script that has to wrap a PHP script that produces a lot of output, and takes about half an hour to run.
At moment I'm shelling out with:
print `$command`;
This works in the sense that the PHP script is called, and it does it's job, but, there is no output rendered by Perl until the PHP script finishes half an hour later.
Is there a way I could shell out so that the output from PHP is printed by perl as soon as it receives it?
The problem is that Perl's not going to finish reading until the PHP script terminates, and only when it finishes reading will it write. The backticks operator blocks until the child process exits, and there's no magic to make a read/write loop implicitly.
So you need to write one. Try a piped open:
open my $fh, '-|', $command or die 'Unable to open';
while (<$fh>) {
print;
}
close $fh;
This should then read each line as the PHP script writes it, and immediately output it. If the PHP script doesn't output in convenient lines and you want to do it with individual characters, you'll need to look into using read to get data from the file handle, and disable output buffering ($| = 1) on stdout for writing it.
See also http://perldoc.perl.org/perlipc.html#Using-open()-for-IPC
Are you really doing print `$command`?
If you are only running a command and not capturing any of its output, simply use system $command. It will write to stdout directly without passing through Perl.
You might want to investigate Capture::Tiny. IIRC something like this should work:
use strict;
use warnings;
use Capture::Tiny qw/tee/;
my ($stdout, $stderr, #result) = tee { system $command };
Actually, just using system might be good enough, YMMV.
I'm trying to modify a script that someone else has written and I wanted to keep my script separate from his.
The script I wrote ends with a print line that outputs all relevant data separated by spaces.
Ex: print "$sap $stuff $more_stuff";
I want to use this data in the middle of another perl script and I'm not sure if it's possible using a system call to the script I wrote.
Ex: system("./sap_calc.pl $id"); #obtain printed data from sap_calc.pl here
Can this be done? If not, how should I go about this?
Somewhat related, but not using system():
How can I get one Perl script to see variables in another Perl script?
How can I pass arguments from one Perl script to another?
You're looking for the "backtick operator."
Have a look at perlop, Section "Quote-like operators".
Generally, capturing a program's output goes like this:
my $output = `/bin/cmd ...`;
Mind that the backtick operator captures STDOUT only. So in order to capture everything (STDERR, too) the commands needs to be appended with the usual shell redirection "2>&1".
If you want to use the data printed to stdout from the other script, you'd need to use backticks or qx().
system will only return the return value of the shell command, not the actual output.
Although the proper way to do this would be to import the actual code into your other script, by building a module, or simply by using do.
As a general rule, it is better to use all perl solutions, than relying on system/shell as a way of "simplifying".
myfile.pl:
sub foo {
print "Foo";
}
1;
main.pl:
do 'myfile.pl';
foo();
perldoc perlipc
Backquotes, like in shell, will yield the standard output of the command as a string (or array, depending on context). They can more clearly be written as the quote-like qx operator.
#lines = `./sap_calc.pl $id`;
#lines = qx(./sap_calc.pl $id);
$all = `./sap_calc.pl $id`;
$all = qx(./sap_calc.pl $id);
open can also be used for streaming instead of reading into memory all at once (as qx does). This can also bypass the shell, which avoids all sorts of quoting issues.
open my $fh, '-|', './sap_calc.pl', $id;
while (readline $fh) {
print "read line: $_";
}