I need to run a string through a Java program and then retrieve the output. The Java program accepts the string through standard input. The following works:
my $output = `echo $string | java -jar java_program.jar`;
There is one problem: $string could be just about anything. Any thoughts on a good solution to this problem?
I suggest you to look at IPC::Run3 module. It uses very simple interface and allow to get STDERR and STDOUT. Here is small example:
use IPC::Run3;
## store command output here
my ($cmd_out, $cmd_err);
my $cmd_input = "put your input string here";
run3([ 'java', '-jar', 'java_program.jar'], \$cmd_input, \$cmd_out, \$cmd_err);
print "command output [$cmd_out] error [$cmd_err]\n";
See IPC::Run3 comparation with other modules.
If you can use CPAN modules (and I'm assuming most people can), look at Ivan's answer on using IPC::Run3. It should handle everything you need.
If you can't use modules, here's how to do things the plain vanilla way.
You can use a pipe to do your input, and it will avoid all those command line quoting issues:
open PIPE, "| java -jar java_program.jar";
print PIPE "$string";
close(PIPE);
It looks like you actually need the output of the command, though. You could open two pipes with something like IPC::Open2 (to and from the java process) but you risk putting yourself in deadlock trying to deal with both pipes at the same time.
You can avoid that by having java output to a file, then reading from that file:
open PIPE, "| java -jar java_program.jar > output.txt";
print PIPE "$string";
close(PIPE);
open OUTPUT, "output.txt";
while (my $line = <OUTPUT>) {
# do something with $line
}
close(OUTPUT);
The other option is to do things the other way around. Put $string in a temporary file, then use it as input to java:
open INPUT, "input.txt";
print INPUT "$string";
close(INPUT);
open OUTPUT, "java -jar java_program.jar < input.txt |";
while (my $line = <OUTPUT>) {
# do something with the output
}
close(OUTPUT);
Note that this isn't the greatest way to do temporary files; I've just used output.txt and input.txt for simplicity. Look at the File::Temp docs for various cleaner ways to create temporary files more cleanly.
Have you looked into IPC::Run?
Syntax similar to this might be what you are looking for:
use IPC::Run qw( run );
my $input = $string;
my ($out, $err);
run ["java -jar java_program.jar"], \$input, \$out, \$err;
Create a pipeline just like your shell would.
Here's our scary string:
my $str = "foo * ~ bar \0 baz *";
We'll build our pipeline backwards, so first we gather the output from the Java program:
my $pid1 = open my $fh1, "-|";
die "$0: fork: $!" unless defined $pid1;
if ($pid1) {
# grab output from Java program
while (<$fh1>) {
chomp;
my #c = unpack "C*" => $_;
print "$_\n => #c\n";
}
}
Note the special "-|" argument to Perl's open operator.
If you open a pipe on the command '-' , i.e., either '|-' or '-|' with 2-arguments (or 1-argument) form of open(), then there is an implicit fork done, and the return value of open is the pid of the child within the parent process, and 0 within the child process … The filehandle behaves normally for the parent, but i/o to that filehandle is piped from/to the STDOUT/STDIN of the child process.
The unpack is there to peek into the contents of the data read from the pipe.
In your program, you'll want to run the Java program, but the code below uses a reasonable facsimile:
else {
my $pid2 = open my $fh2, "-|";
die "$0: fork: $!" unless defined $pid2;
if ($pid2) {
$| = 1;
open STDIN, "<&=" . fileno($fh2)
or die "$0: dup: $!";
# exec "java", "-jar", "java_program.jar";
# simulate Java program
exec "perl", "-pe", q(
BEGIN { $" = "][" }
my #a = split " ", scalar reverse $_;
$_ = "[#a]\n";
);
die "$0: exec failed";
}
Finally, the humble grandchild simply prints the scary string (which arrives on the standard input of the Java program) and exits. Setting $| to a true value flushes the currently selected filehandle and puts it in unbuffered mode.
else {
print $str;
$| = 1;
exit 0;
}
}
Its output:
$ ./try
[*][zab][][rab][~][*][oof]
=> 91 42 93 91 122 97 98 93 91 0 93 91 114 97 98 93 91 126 93 91 42 93 91 111 111 102 93
Note that the NUL survives the trip.
The builtin IPC::Open2 module provides a function to handle bidirectional-piping without an external file.
Related
I want to write perl scripts that can read the STDIN that is given at invocation of the script, finish reading it, and then interactively prompt the user for a one-line STDIN. This one-line STDIN will tell the script how to proceed.
In a practical application, I would like the script to create a temporary file, report on the size of temporary file, and then ask the user if they really want to print the entire temporary file to STDOUT, or do they want to give a filename that will be clobbered by the temporary file's contents.
The following script behaves as desired if I give STDIN as a filename but does not work if I pipe STDIN to the script.
#! /usr/bin/perl
use strict; use warnings;
my $count = 0;
while(<>)
{
$count++;
}
print "you counted $count lines. Now do you want to proceed?";
my $answer = <STDIN>;
chomp $answer;
print STDERR "answer=$answer\n";
if ( $answer eq "yes" )
{
print STDERR "you said $answer so we do something affirmative\n";
}
else
{
print STDERR "you said $answer which is not \"yes\" so we do NOT proceed\n";
}
for instance
> wc junk
193 1042 11312 junk
> junk.pl junk
you counted 193 lines. Now do you want to proceed?yes
answer=yes
you said yes so we do something affirmative
> junk.pl junk
you counted 193 lines. Now do you want to proceed?no
answer=no
you said no which is not "yes" so we do NOT proceed
> cat junk | junk.pl
Use of uninitialized value $answer in scalar chomp at /Users/BNW/u/kh/bin/junk.pl line 10.
Use of uninitialized value $answer in concatenation (.) or string at /Users/BNW/u/kh/bin/junk.pl line 11.
answer=
Use of uninitialized value $answer in string eq at /Users/BNW/u/kh/bin/junk.pl line 12.
Use of uninitialized value $answer in concatenation (.) or string at /Users/BNW/u/kh/bin/junk.pl line 18.
you said which is not "yes" so we do NOT proceed
you counted 193 lines. Now do you want to proceed?>
Sort of. Maybe.
First off, in your first example, it's not true that you "gave STDIN as a filename". STDIN is the terminal throughout. <> is reading from the ARGV handle, not STDIN, so STDIN is available later when you need it.
The problem with the second example example is that the pipe from cat is STDIN. Closing it and reopening it to what it was initially doesn't do anything for you, because it will still be an exhausted pipe.
Many systems, though, have a special device /dev/tty which points to the controlling terminal of whichever process asks for it. On such a system, you could reopen STDIN from /dev/tty after it gives EOF, and you would get the console that the user invoked your program from, instead of whatever file or pipe they initially gave you as STDIN.
Thanks to #hobbs. Note that this works either way: piping the file junk into the script or passing junk as ARGV.
> printf "line 1 \nline 222 \n" > junk
> perl -e 'use strict; use warnings; while(<>) { print; } my $stuff = "/dev/tty"; my $h; open $h, "<", $stuff or die "waah $stuff"; print "give answer:"; my $answer=<$h>; print "answer=$answer\n";' junk
line 1
line 222
give answer:This is an answer!
answer=This is an answer!
> cat junk | perl -e 'use strict; use warnings; while(<>) { print; } my $stuff = "/dev/tty"; my $h; open $h, "<", $stuff or die "waah $stuff"; print "give answer:"; my $answer=<$h>; print "answer=$answer\n";'
line 1
line 222
give answer:So, what was it she was saying?? ??
answer=So, what was it she was saying?? ??
>
following setup:
linux debian based 4.4.14-v7+ armv7l GNU/Linux
perl version is v5.20.2 built for arm-linux-gnueabihf-thread-multi-64int
A Perl script which should read a stream of data (hex chars, different length per line)
example stream output:
00 AA BB 11 22 33 44 ...
00 AB BB 11 22 33 44 ...
Depending on specific values the script should do specific actions.
Works fine however when the stream stops sending data, i.e. stream is finished sending data, the while loop does not get stopped. and the script waits for more lines.
The stream itself sends f.e. 5 sec of data, and then the analyse script should do_stuff;
once the analyse script is finished with the calculations, it will start the stream again.
However i am unable to figure out the reason why either the "next command" is not executed, or the while loop does not terminate correctly on no further lines.
If i manually start the stream process again the analyse script continues just fine until there are no more lines again.
simplified Code
#!/usr/bin/perl
#script to analyse data
use warnings;
use strict;
use POSIX ();
sub stop_stream($){
my $pid = shift;
my $cmd = "/bin/kill -9 $pid";
system($cmd);
}
while (1){
my $steampid = open( STREAM, "/usr/local/bin/stream |" );
STREAM: while ( my $row = <STREAM> ) {
chomp $row;
my #o = split /\s+/, $row;
#do_stuff;
#..
#this is the row on which the script hangs until it get's new lines in the filehandle.
next STREAM if $o[1] ne "AA";
#...
#do_other_stuff;
#...
}
stop_stream( $steampid );
close STREAM;
}
Resources i tried to figure out the issue:
http://perldoc.perl.org/perlsyn.html#Compound-Statements
http://www.perlmonks.org/?node_id=1065701
and numerous others.
I tried stackoverflow with some combination of "perl while loop close filehandle" to no success.
OK, the root of the problem here is that a while ( <FILEHANDLE> ) { loop will block if the file handle is open but there is no data to be read.
So it's likely that /usr/local/bin/stream continues piping data until killed - and so your reads block.
The easy solution is to use something like IO::Select which has can_read as an option:
use IO::Select;
open ( my $stream, '-|', '/usr/local/bin/stream' ) or die $!;
#register this filehandle with a new select object
my $select = IO::Select -> new ( $stream );
#2s stream timeout.
#works because can_read returns a list of filehandles ready for reading
#with only one filehandle registered, it can be used as a simple logic test.
while ( $select -> can_read(2) ) {
my $row = <$stream>;
#etc.
}
#etc.
This post demonstrates how one can read from STDIN or from a file, without using the null filehandle (i.e., while(<>)). However, I'd like to know how can one address situations where input may come from files, STDIN, or both simultaneously.
For instance, the <> syntax can handle such a situation, as demonstrated by the following minimal example:
$ echo -e 'a\nb\nc' | \
while read x; do echo $x > ${x}".txt"; done; echo "d" | \
perl -e "while(<>) {print;}" {a,b,c}.txt -
a
b
c
d
How can I do this without using while(<>)?
I want to avoid using <> because I want to handle each file independently, rather than aggregating all input as a single stream of text. Moreover, I want to do this without testing for eof on every line of input.
If you want to handle each file independently of the others, you should loop over the arguments that have been given and open each file in turn:
for (#ARGV) {
open(my $fh, '<', $_) || die "cannot open $_";
while (<$fh>) {
... process the file here ...
}
}
# handle standard input
while (<STDIN>) {
...
}
Here is an idea based on Tim's that checks if STDIN has something to read (NON BLOCKING STDIN). This is useful if you don't really care about a user entering input manually from STDIN yet still want to be able to pipe and redirect data to the script.
File: script.pl
#!/usr/bin/env perl
use IO::Select;
$s = IO::Select->new();
$s->add(\*STDIN);
if ($s->can_read(0)) { push #ARGV, "/dev/stdin"; }
for (#ARGV) {
open(IN, "<$_") || die "** Error opening \"$_\": $!\n";
while (<IN>) {
print $_
}
}
$> echo "hello world" | script.pl
hello world
$> script.pl < <(echo "hello world")
hello world
$> script.pl <(echo "hello world")
hello world
$> script.pl <<< "hello world"
hello world
$> script.pl
$>
This was already answered by the Answer to which the question links.
#ARGS = '-' if !#ARGV;
for my $qfn (#ARGV) {
open($fh, $qfn);
while (<$fh>) {
...
}
}
I need to run a shell command with system() in Perl. For example,
system('ls')
The system call will print to STDOUT, but I want to capture the output into a variable so that I can do future processing with my Perl code.
That's what backticks are for. From perldoc perlfaq8:
Why can't I get the output of a command with system()?
You're confusing the purpose of system() and backticks (``). system()
runs a command and returns exit status information (as a 16 bit value:
the low 7 bits are the signal the process died from, if any, and the
high 8 bits are the actual exit value). Backticks (``) run a command
and return what it sent to STDOUT.
my $exit_status = system("mail-users");
my $output_string = `ls`;
See perldoc perlop for more details.
IPC::Run is my favourite module for this kind of task. Very powerful and flexible, and also trivially simple for small cases.
use IPC::Run 'run';
run [ "command", "arguments", "here" ], ">", \my $stdout;
# Now $stdout contains output
Simply use similar to the Bash example:
$variable=`some_command some args`;
That's all. Notice, you will not see any printings to STDOUT on the output because this is redirected to a variable.
This example is unusable for a command that interact with the user, except when you have prepared answers. For that, you can use something like this using a stack of shell commands:
$variable=`cat answers.txt|some_command some args`;
Inside the answers.txt file you should prepare all answers for some_command to work properly.
I know this isn't the best way for programming :) But this is the simplest way how to achieve the goal, specially for Bash programmers.
Of course, if the output is bigger (ls with subdirectory), you shouldn't get all output at once. Read the command by the same way as you read a regular file:
open CMD,'-|','your_command some args' or die $#;
my $line;
while (defined($line=<CMD>)) {
print $line; # Or push #table,$line or do whatever what you want processing line by line
}
close CMD;
An additional extended solution for processing a long command output without extra Bash calling:
my #CommandCall=qw(find / -type d); # Some example single command
my $commandSTDOUT; # File handler
my $pid=open($commandSTDOUT),'-|'); # There will be an implicit fork!
if ($pid) {
#parent side
my $singleLine;
while(defined($singleline=<$commandSTDOUT>)) {
chomp $line; # Typically we don't need EOL
do_some_processing_with($line);
};
close $commandSTDOUT; # In this place $? will be set for capture
$exitcode=$? >> 8;
do_something_with_exit_code($exitcode);
} else {
# Child side, there you really calls a command
open STDERR, '>>&', 'STDOUT'; # Redirect stderr to stdout if needed. It works only for child - remember about fork
exec(#CommandCall); # At this point the child code is overloaded by an external command with parameters
die "Cannot call #CommandCall"; # Error procedure if the call will fail
}
If you use a procedure like that, you will capture all procedure output, and you can do everything processing line by line. Good luck :)
I wanted to run system() instead of backticks because I wanted to see the output of rsync --progress. However, I also wanted to capture the output in case something goes wrong depending on the return value. (This is for a backup script). This is what I am using now:
use File::Temp qw(tempfile);
use Term::ANSIColor qw(colored colorstrip);
sub mysystem {
my $cmd = shift; # "rsync -avz --progress -h $fullfile $copyfile";
my ($fh, $filename) = tempfile();
# http://stackoverflow.com/a/6872163/2923406
# I want to have rsync progress output on the terminal AND capture it in case of error.
# Need to use pipefail because 'tee' would be the last cmd otherwise and hence $? would be wrong.
my #cmd = ("bash", "-c", "set -o pipefail && $cmd 2>&1 | tee $filename");
my $ret = system(#cmd);
my $outerr = join('', <$fh>);
if ($ret != 0) {
logit(colored("ERROR: Could not execute command: $cmd", "red"));
logit(colored("ERROR: stdout+stderr = $outerr", "red"));
logit(colored("ERROR: \$? = $?, \$! = $!", "red"));
}
close $fh;
unlink($filename);
return $ret;
}
# And logit() is something like:
sub logit {
my $s = shift;
my ($logsec, $logmin, $loghour, $logmday, $logmon, $logyear, $logwday, $logyday, $logisdst) = localtime(time);
$logyear += 1900;
my $logtimestamp = sprintf("%4d-%02d-%02d %02d:%02d:%02d", $logyear, $logmon+1, $logmday, $loghour, $logmin, $logsec);
my $msg = "$logtimestamp $s\n";
print $msg;
open LOG, ">>$LOGFILE";
print LOG colorstrip($msg);
close LOG;
}
I want to run and executable ./runnable on argument input.afa. The standard input to this executable is through a file finalfile. I was earlier trying to do the same using a bash script, but that does not seem to work out. So I was wondering whether Perl provides such functionality. I know I can run the executable with its argument using backticks or system() call. Any suggestions on how to give standard input through file.
_ UPDATE _
As I said I had written a bash script for the same. I not sure how to go about doing it in Perl. The bash script I wrote was:
#!/bin/bash
OUTFILE=outfile
(
while read line
do
./runnable input.afa
echo $line
done<finalfile
) >$OUTFILE
The data in standard input file is as follows, where each line correspond to one time input. So if there are 10 lines then the executable should run 10 times.
__DATA__
2,9,2,9,10,0,38
2,9,2,10,11,0,0
2,9,2,11,12,0,0
2,9,2,12,13,0,0
2,9,2,13,0,1,4
2,9,2,13,3,2,2
2,9,2,12,14,1,2
If I understood your question correctly, then you are perhaps looking for something like this:
# The command to run.
my $command = "./runnable input.afa";
# $command will be run for each line in $command_stdin
my $command_stdin = "finalfile";
# Open the file pointed to by $command_stdin
open my $inputfh, '<', $command_stdin or die "$command_input: $!";
# For each line
while (my $input = <$inputfh>) {
chomp($input); # optional, removes line separator
# Run the command that is pointed to by $command,
# and open $write_stdin as the write end of the command's
# stdin.
open my $write_stdin, '|-', $command or die "$command: $!";
# Write the arguments to the command's stdin.
print $write_stdin $input;
}
More info about opening commands in the documentation.
Perl code:
$stdout_result = `exescript argument1 argument2 < stdinfile`;
Where stdinfile holds the data you want to be passed through stdin.
edit
The clever method would be to open stdinfile, tie it via select to stdin, and then execute repeatedly. The easy method would be to put the data you want to pass through in a temp file.
Example:
open $fh, "<", "datafile" or die($!);
#data = <$fh>; #sucks all the lines in datafile into the array #data
close $fh;
foreach $datum (#data) #foreach singluar datum in the array
{
#create a temp file
open $fh, ">", "tempfile" or die($!);
print $fh $datum;
close $fh;
$result = `exe arg1 arg2 arg3 < tempfile`; #run the command. Presumably you'd want to store it somewhere as well...
#store $result
}
unlink("tempfile"); #remove the tempfile