I'm trying to run a tail command from within a perl script using the usual backticks.
The section in my perl script is as follows:
$nexusTime += nexusUploadTime(`tail $log -n 5`);
So I'm trying to get the last 5 lines of this file but I'm getting the following error when the perl script finishes:
sh: line 1: -n: command not found
Even though when I run the command on the command line it is indeed successful and I can see the 5 lines from that particular.
Not sure what is going on here. Why it works from command line but through perl it won't recognize the -n option.
Anybody have any suggestions?
$log has an extraneous trailing newline, so you are executing
tail file.log
-n 5 # Tries to execute a program named "-n"
Fix:
chomp($log);
Note that you will run into problems if log $log contains shell meta characters (such as spaces). Fix:
use String::ShellQuote qw( shell_quote );
my $tail_cmd = shell_quote('tail', '-n', '5', '--', $log);
$nexusTime += nexusUploadTime(`$tail_cmd`);
ikegami pointed out your error, but I would recommend avoiding external commands whenever possible. They aren't portable and debugging them can be a pain, among other things. You can simulate tail with pure Perl code like this:
use strict;
use warnings;
use File::ReadBackwards;
sub tail {
my ($file, $num_lines) = #_;
my $bw = File::ReadBackwards->new($file) or die "Can't read '$file': $!";
my ($lines, $count);
while (defined(my $line = $bw->readline) && $num_lines > $count++) {
$lines .= $line;
}
$bw->close;
return $lines;
}
print tail('/usr/share/dict/words', 5);
Output
ZZZ
zZt
Zz
ZZ
zyzzyvas
Note that if you pass a file name containing a newline, this will fail with
Can't read 'foo
': No such file or directory at tail.pl line 10.
instead of the more cryptic
sh: line 1: -n: command not found
that you got from running the tail utility in backticks.
The answer to this question is to place the option -n 5 before the target file
Related
I'm having this issue with my perl program that is reading from a file (which I open on STDIN and read each line one at a time using $line = <>). After I execute a `backtick` command, and then I go to read the next line from STDIN, I get an undef, signaling EOF. I isolated it to the backtick command using debugging code as follows:
my $dir = dirname(__FILE__);
say STDERR "before: tell(STDIN)=" . tell(STDIN) . ", eof(STDIN)=" . eof(STDIN);
say STDERR "\#export_info = `echo nostdin | perl $dir/pythonizer_importer.pl $fullfile`;";
#export_info = `echo nostdin | perl $dir/pythonizer_importer.pl $fullfile`;
say STDERR "after: tell(STDIN)=" . tell(STDIN) . ", eof(STDIN)=" . eof(STDIN);
The output is:
before: tell(STDIN)=15146, eof(STDIN)=
#export_info = `echo nostdin | perl ../pythonizer_importer.pl ./Pscan.pm`;
after: tell(STDIN)=15146, eof(STDIN)=1
I recently added the echo nostdin | to the perl command which had no effect. How do I run this command and get the STDOUT without messing up my STDIN? BTW, this is all running on Windows. I fire off the main program from a git bash if that matters.
Try locally undefining STDIN before running the backticks command, like this example script does. Note that any subroutines called from the sub that calls local will see the new value. You can also do open STDIN, "<", "file for child process to read"; after the local *STDIN but before the backticks but remember to close() the file before restoring STDIN to its old value.
The child process is affecting your STDIN because "the STDIN filehandle used by the command is inherited from Perl's STDIN." – perlop manual
This is just an example; in your actual script, replace the sed command with your actual command to run.
use strict;
use warnings;
#Run a command and get its output
sub get_output {
# Prevent passing our STDIN to child process
local *STDIN = undef;
print "Running sed\n";
#replace the sed command with the actual command you want to run
return `sed 's/a/b/'`;
}
my $output = get_output();
print $output;
#We can still read STDIN even after running a child process
print "Waiting for input\n";
print "Readline is " . scalar readline;
Input:
a
b
c
^D
line
Output:
Running sed
b
b
c
Waiting for input
Readline is line
Below is the code I'm trying to execute. I have mentioned the line 266 in the code. I have added that code to remove the blank lines in the log file. I'm not sure whether we can run the perl command inside a Perl script. Is there another way that I can run this so that I can remove the blank lines in the log file?
Below is the error I'm getting while running through the Perl script:
syntax error at ./reportJBossErrors.pl line 266, near "n -e "
Execution of ./reportJBossErrors.pl aborted due to compilation errors.
Here is a portion of the code, showing line 266:
sub main {
readConfiguration($config_file);
$short_hostname = `hostname | cut -f 1 -d.`;
chomp $short_hostname;
getFileandInstance($short_hostname);
$yesterday = getYesterday();
validateEnvironment();
$log_file = getLogFile($FMASK,$yesterday);
perl -i -n -e "print if /\S/" $log_file; # 266 line. This is where I'm getting the compilation error
processFile($log_file);
$html_out = writeEmail();
sendEmail($CONFIG{"FROMADDR"},$CONFIG{"TOADDR"},"Normal",
"JBOSS",$short_hostname,$log_file,$CONFIG{ENVTYPE},$html_out);
}
You can not call the perl command inside a Perl program as if it were a Perl builtin function. You can use system to run an external command:
my $cmd = 'perl -i -n -e "print if /\S/"';
system "$cmd $log_file";
You need to be careful of quoting. Since you have a file name/path in the Perl variable $logfile, which you want to interpolate, that can go inside double quotes. Since you do not want to interpolate \S, that should go in single quotes.
You cannot invoke the perl executable inside a Perl program as if it were a Perl builtin function. Instead, use the list form of system to run an external command. Don't forget to check if the command succeeded:
my #cmd = (perl => '-i', '-n', '-e', 'print if /\S/', $log_file);
system(#cmd) == 0
or die "system #cmd failed: $?";
In general, I would recommend using the full path to perl rather than relying on $PATH.
Also, if you need to keep track of status etc, use Capture::Tiny to get both STDOUT and STDERR of the command you are running so that you can log error information.
I am trying to execute below awk command inside a perl script, but it is failing.
#!/usr/bin/perl
print `awk -F, '{print $NF}' f1.txt > f2.txt`
This is the error:
syntax error at ./MO.pl line 3, near "print"
Execution of ./MO.pl aborted due to compilation errors.
Can anyone please help what I am doing wrong here?
This is a Perl error and has nothing to do with your awk script itself. The error is usually seen when the previous statement doesn't have a semicolon at the end.
Here's a very simple program (which should include use strict; and use warnings;, but I wanted to emulate what you have).
#! /usr/bin/env perl
#
print "Hello, World\n" # Line 4
print "Hello, Back\n"; # Line 5
And the error message is:
syntax error at test.pl line 5, near "print"
Execution of test.pl aborted due to compilation errors.
Note the error is near the print in Line #5, but the error is actually at the end of Line #4 where I'm missing a semicolon.
Running your exact program works on my system (although doesn't quite produce the results you want). I am assuming this isn't your exact program, but instead a simplification of your program. Is there a statement before that print?
Several other things:
You're redirecting your awk output, so there's nothing to print.
Use strict and warnings.
Better to use qx(....) than backticks (grave accent). It's more readable and allows you to do quoted executable in quoted executable.
Watch for Perlisms in your code. The $NF is interpreted by Perl, and without the use strict;, doesn't give you an error. Instead, the print in your Awk statement is a null print which prints the entire line.
Why do you use print if nothing is printing out? You're better off in this position to use system which allows you to put single quotes around your entire statement:
system q(awk -F, '{print $NF}' f1.txt > f2.txt);
This way, $NF doesn't have to be quoted.
Why are you doing Awk in a Perl program? Perl will do anything Awk will do and do it better:
Here's a version of your program using plain ol' Perl:
#! /usr/bin/env perl
use strict;
use warnings;
use autodie;
while ( my $line = <> ) {
my #array = split /\s*,\s*/, $line;
print $array[-1];
}
To run this program:
$ test.pl f1.txt > f2.txt
Yes, it's longer, but half of the program is taken up by pragmas that all Perl programs should use.
I'm sure people smarter than me can turn this into a Perl one-liner.
Since you're redirecting the awk output, there's nothing for perl to print. You might as well use system and the quoting operator q():
system q(awk -F, '{print $NF}' f1.txt > f2.txt)
Or, of course, do it in perl, which saves you from having to spawn a shell and then spawn awk:
open my $in, '<', 'f1.txt';
open my $out, '>', 'f1.txt';
while (<$in>) {
print $out (split " ")[-1], "\n";
}
close $in;
close $out;
If there are more lines in the script, you need a semi-colon at the end of the print statement.
I found this perl script here which seems will work for my purposes. It opens a Unicode text file and reads each line so that a command can be run. But I cannot figure out how to run a certain ICU command on each line. Can someone help me out? The error I get is (largefile is the script name):
syntax error at ./largefile line 11, near "/ ."
Search pattern not terminated at ./largefile line 11.
#!/usr/bin/perl
use strict;
use warnings;
my $file = 'test.txt';
open my $info, $file or die "Could not open $file: $!";
while( my $line = <$info>) {
do
LD_LIBRARY_PATH=icu/source/lib/ ./a.out "$line" >> newtext.txt
done
}
close $info;
Basically I want to open a large text file and run the command (which normally runs from the command line...I think how I call this in the perl script is the problem, but I don't know how to fix it) "LD_LIBRARY_PATH=icu/source/lib/ ./a.out "$line" >> newtext.txt" on each line so that "newtext.txt" is then populated with all the lines after they have been processed by the script. The ICU part is breaking words for Khmer.
Any help would be much appreciated! I'm not much of a programmer... Thanks!
For executing terminal commands, the command needs to be in system(), hence change to
system("LD_LIBRARY_PATH=icu/source/lib/ ./a.out $line >> newtext.txt");
Have you tried backticks:
while (my $line = <$info>) {
`LD_LIBRARY_PATH=icu/source/lib/ ./a.out "$line" >> newtext.txt`
last if $. == 2;
}
When executing a command on the command-line from Perl, is there a way to store that result as a variable in Perl?
my $command = "cat $input_file | uniq -d | wc -l";
my $result = system($command);
$result always turns out to be 0.
Use "backticks":
my $command = "cat $input_file | uniq -d | wc -l";
my $result = `$command`;
And if interested in the exit code you can capture it with:
my $retcode = $?;
right after making the external call.
From perlfaq8:
Why can't I get the output of a command with system()?
You're confusing the purpose of system() and backticks (````). system() runs a command and returns exit status information (as a 16 bit value: the low 7 bits are the signal the process died from, if any, and the high 8 bits are the actual exit value). Backticks (``) run a command and return what it sent to STDOUT.
$exit_status = system("mail-users");
$output_string = `ls`;
You can use the Perl back-ticks to run a shell command, and save the result in an array.
my #results = `$command`;
To get just a single result from the shell command, you can store it in a scalar variable:
my $result = `$command`;
If you are expecting back multiple lines of output, it's easier to use an array, but if you're just expecting back one line, it's better to use scalar.
(something like that, my perl is rusty)
You can use backticks, as others have suggested. That's fine if you trust whatever variables you're using to build your command.
For more flexibility, you can open the command as a pipe and read from that as you would a file. This is particularly useful when you want to pass variables as command line arguments to the program and you don't trust their source to be free of shell escape characters, as open in recent Perl (>= 5.8) has the capacity to invoke a program from an argument list. So you can do the following:
open(FILEHANDLE, '-|', 'uniq', 'some-file.txt') or die "Cannot fork: $!\n";
while (<FILEHANDLE>) {
# process a line from $_
}
close FILEHANDLE or die "child error: $!\n";
IPC::System::Simple provides the 'capture' command which provides a safe, portable alternative to backticks. It (and other commands from this module) are highly recommended.
I'm assuming this a 'contrived' example, because this is
equivalent: 'uniq -d $input_file | wc -l'.
In almost all my experience, the only reason for putting the results
in to a perl variable, is to parse the later. In that case, I use
the following pattern:
$last = undef;
$lc = 0;
open(FN, "$input_file");
while (<FN>) {
# any other parsing of the current line
$lc++ if ($last eq $_);
$last = $_;
}
close(FN);
print "$lc\n";
This also has the added advantages:
no fork for shell, cat, uniq, and wc
faster
parse and collect the desired input