As per solution provided in perldoc, I am trying to emulate tail -f but it's not working as expected. The below code could print all lines first time but not the newly lines appended. Could you please elaborate if I am missing any thing here.
#!/usr/bin/perl
open (LOGFILE, "aa") or die "could not open file reason $! \n";
for (;;)
{
seek(LOGFILE,0,1); ### clear OF condition
for ($curpos = tell(LOGFILE); <LOGFILE>; $curpos = tell(LOGFILE))
{
print "$_ \n";
}
sleep 1;
seek(LOGFILE,$curpos,0); ### Setting cursor at the EOF
}
works fine for me. How are you updating "aa" ?
You wont see the data immediately if it is a buffered write to "aa".
can you try the following in a different terminal and check whether you are seeing any update.
while ( 1 )
echo "test" >> aa
end
If you are using perl to update aa, check this section on buffering and how to disable.
Related
I'm trying to write a perl script where I'm trying to save whole contents of those files which contain a specific string 'PYAG_GENERATED', in a single .txt/.tmp file one after another. These file names are in a specific pattern and this pattern is 'output_nnnn.txt' where nnnn is 0001,0002 and so on. But I don't know how many number of files are present with this 'output_nnnn.txt' name.
I'm new in perl and I don't know how I can resolve this issue to get the output correctly. Can anyone help me. Thanks in advance.
I've tried to write perl script in different ways but nothing is coming in output file. I'm giving here one of those I've tried. 'new_1.txt' is the new file where I want to save the expected output and "PYAG_GENERATED" is that specific string I'm finding for in the files.
open(NEW,">>new_1.txt") or die "could not open:$!";
$find2="PYAG_GENERATED";
$n='0001';
while('output_$n.txt'){
if(/find2/){
print NEW;
}
$n++;
}
close NEW;
I expect that the output file 'new_1.txt' will save the whole contents of the the files(with filename pattern 'output_nnnn.txt') which have 'PYAG_GENERATED' string at least once inside.
Well, you tried I guess.
Welcome to the wonderful world of Perl where there are always a dozen ways of doing X :-) One possible way to achieve what you want. I put in a lot of comments I hope are helpful. It's also a bit verbose for the sake of clarity. I'm sure it could be golfed down to 5 lines of code.
use warnings; # Always start your Perl code with these two lines,
use strict; # and Perl will tell you about possible mistakes
use experimental 'signatures';
use File::Slurp;
# this is a subroutine/function, a block of code that can be called from
# somewhere else. it takes to arguments, that the caller must provide
sub find_in_file( $filename, $what_to_look_for )
{
# the open function opens $filename for reading
# (that's what the "<" means, ">" stands for writing)
# if successfull open will return we will have a "file handle" in the variable $in
# if not open will return false ...
open( my $in, "<", $filename )
or die $!; # ... and the program will exit here. The variable $! will contain the error message
# now we read the file using a loop
# readline will give us the next line in the file
# or something false when there is nothing left to read
while ( my $line = readline($in) )
{
# now we test wether the current line contains what
# we are looking for.
# the index function gives us the index of a string within another string.
# for example index("abc", "c") will give us 3
if ( index( $line, $what_to_look_for ) > 0 )
{
# we found what we were looking for
# so we don't need to keep looking in this file anymore
# so we must first close the file
close( $in );
# and then we indicate to the caller the search was a successfull
# this will immedeatly end the subroutine
return 1;
}
}
# If we arrive here the search was unsuccessful
# so we tell that to the caller
return 0;
}
# Here starts the main program
# First we get a list of files
# we want to look at
my #possible_files = glob( "where/your/files/are/output_*.txt" );
# Here we will store the files that we are interested in, aka that contain PYAG_GENERATED
my #wanted_files;
# and now we can loop over the files and see if they contain what we are looking for
foreach my $filename ( #possible_files )
{
# here we use the function we defined earlier
if ( find_in_file( $filename, "PYAG_GENERATED" ) )
{
# with push we can add things to the end of an array
push #wanted_files, $filename;
}
}
# We are finished searching, now we can start adding the files together
# if we found any
if ( scalar #wanted_files > 0 )
{
# Now we could code that us ourselves, open the files, loop trough them and write out
# line by line. But we make life easy for us and just
# use two functions from the module File::Slurp, which comes with Perl I believe
# If not you have to install it
foreach my $filename ( #wanted_files )
{
append_file( "new_1.txt", read_file( $filename ) );
}
print "Output created from " . (scalar #wanted_files) . " files\n";
}
else
{
print "No input files\n";
}
use strict;
use warnings;
my #a;
my $i=1;
my $find1="PYAG_GENERATED";
my $n=1;
my $total_files=47276; #got this no. of files by writing 'ls' command in the terminal
while($n<=$total_files){
open(NEW,"<output_$n.txt") or die "could not open:$!";
my $join=join('',<NEW>);
$a[$i]=$join;
#print "$a[10]";
$n++;
$i++;
}
close NEW;
for($i=1;$i<=$total_files;$i++){
if($a[$i]=~m/$find1/){
open(NEW1,">>new_1.tmp") or die "could not open:$!";
print NEW1 $a[$i];
}
}
close NEW1;
I am new to Perl and trying to write a code to keep executing an action until the match is found and else give an error.
I am trying to execute a command ps -ef and check if it has got any process running in the name of "box", if there is no process named "box" found, I want to repeat ps -ef command execution until it gets the "box" process and then proceed to next action.
#!/usr/bin/perl -w
open (FH, "ps -ef |") or die "Cannot run the command:$!\n";
$line = "box";
while (<FH>) {
if (/$line/i) { next; }
else {
print ("ps -ef |") or die "Cannot run the command:$!\n");
}
}
close (FH);
You need to use an infinite loop and an exit-condition. Your condition is that the ps -ef command contains the word box. There is no need to open a pipe to that command explicitly, you can just run it as a system call with the qx operator (same as backticks).
use strict;
use warnings;
my $ps;
PS: while (1) {
$ps = qx/ps -ef/;
last PS if $ps =~ m/box/i;
print '.'; # do something in every run
}
print $ps;
As this has come up in the comments as well as in in AdrianHHH's answer: it might make sense to sleep after every run to make sure you don't hog the CPU. Depending on the nature of the process you are looking for, either the sleep builtin or usleep from Time::HiRes might be appropriate. The latter let's your program rest for milliseconds, while the builtin only works with full seconds. These might be too long if the target box process is very quick.
Explanation of your code:
Note that you have some issues in your implementation. I'll explain what your code does. This is taken from the question, comments are mine.
#!/usr/bin/perl -w
# open a filehandle to the ps command
open (FH, "ps -ef |") or die "Cannot run the command:$!\n";
$line = "box";
# read the output of one run line by line, for each line execute
# the block
while (<FH>) {
# if there is 'box' case-insensitive, skip the line
if (/$line/i) { next; }
else {
# else output (not run!) the command
print ("ps -ef |") or die "Cannot run the command:$!\n");
}
}
close (FH);
After it went through all the lines of the output of your command once it will stop.
I would recommend using pgrep(1) instead of ps because it lets you do a more granular search. With ps -ef, you potentially have to deal with cases like:
boxford 6254 6211 0 08:23 pts/1 00:00:00 /home/boxford/box --bounding-box=123
It's hard to tell if you're matching a process being run by a user with box in their username, a process that has box somewhere in its path, a process named box, or a process with box somewhere in its argument list.
pgrep, on the other hand, lets you match against just the process name or the full path, a specific user or users, and more. The following prints a message when a process named box appears (this looks for an exact match, so it will not match processes named dropbox, for example):
use strict;
use warnings;
use 5.010;
use String::ShellQuote qw(shell_quote);
sub is_running {
my ($proc) = #_;
my $cmd = 'pgrep -x ' . shell_quote($proc) . ' >/dev/null 2>&1';
system($cmd);
if ($? == -1) {
die "failed to execute pgrep: $!";
}
elsif ($? & 127) {
die "pgrep died with signal ", $? & 127;
}
else {
my $status = $? >> 8;
die "pgrep exited with error: exit status $status" if $status > 1;
return $status == 0;
}
}
my $proc = 'box';
until ( is_running($proc) ) {
sleep 1;
}
say "Process '$proc' is running";
Note that pgrep doesn't have a case-insensitive flag, probably because process names in *nix are almost always lowercase. If you really need to do a case-insensitive match, you can pass [Bb][Oo][Xx] to the is_running function.
The ps command outputs the current list of processes, then it completes. The code in the question reads that output. Suppose that the first ps command that is executed does not contain the wanted line, then there is nothing in the code in the question to run the ps command again.
The next statement in the question makes the script move on to the next line in the output from ps, not to rerun the command. The else print ... after the next will probably be executed for the first line of the output from ps. The outcome is that the print is run for each line in the ps output that does not have the wanted text and that the next command has no significant effect. In the code print ... or die "..." the or die "..." part is not very useful, the print is unlikely to fail and even if it did the die message would be wrong.
Perhaps you should write some code in the following style. Here the ps is run repeatedly until the wanted text is found. Note the sleep call, without that the script will keep running without pause, possibly preventing real work or at least slowing it down.
# This code is not tested.
use strict;
use warnings;
my $found_wanted_line = 0; # Boolean, set to false
my $line = "box";
while ( ! $found_wanted_line ) {
open (my $FH, "ps -ef |") or die "Cannot run the command:$!\n";
while (<$FH>) {
if (/$line/i) {
$found_wanted_line = 1; # Boolean, set to true
last;
}
}
close ($FH);
if ( ! $found_wanted_line )
sleep 2; # Pause for 2 seconds, to prevent this script hogging the CPU.
}
}
I'm having an issue with a Perl script relating to the Weather Research Forecast (WRF) model configuration. The script in question is a part of the download located here (login required, simple signup). If you download the most recent WRF-NMM core, in the unzipped directory is arch/Config_new.pl. The error that I'm having lies somewhere within lines 262-303:
until ( $validresponse ) {
print "------------------------------------------------------------------------\n" ;
print "Please select from among the following supported platforms.\n\n" ;
$opt = 1 ;
open CONFIGURE_DEFAULTS, "< ./arch/configure_new.defaults"
or die "Cannot open ./arch/configure_new.defaults for reading" ;
while ( <CONFIGURE_DEFAULTS> )
{
for $paropt ( #platforms )
{
if ( substr( $_, 0, 5 ) eq "#ARCH"
&& ( index( $_, $sw_os ) >= 0 ) && ( index( $_, $sw_mach ) >= 0 )
&& ( index($_, $paropt) >= 0 ) )
{
$optstr[$opt] = substr($_,6) ;
$optstr[$opt] =~ s/^[ ]*// ;
$optstr[$opt] =~ s/#.*$//g ;
chomp($optstr[$opt]) ;
$optstr[$opt] = $optstr[$opt]." (".$paropt.")" ;
if ( substr( $optstr[$opt], 0,4 ) ne "NULL" )
{
print " %2d. %s\n",$opt,$optstr[$opt] ;
$opt++ ;
}
}
}
}
close CONFIGURE_DEFAULTS ;
$opt -- ;
print "\nEnter selection [%d-%d] : ",1,$opt ;
$response = <STDIN> ;
if ( $response == -1 ) { exit ; }
if ( $response >= 1 && $response <= $opt )
{ $validresponse = 1 ; }
else
{ print("\nInvalid response (%d)\n",$response);}
}
Specifically, I am sent to an input line without any kind of prompting or list of what my options are. Only after I select a valid choice am I presented with the previous options. This is repeated a second time with another chunk of code further down (lines 478-528). What's got me confused is that, when I entered debugging mode, I inserted a break before the start of this portion of code. I ran p $validresponse and got the following:
0
If you REALLY want Grib2 output from WRF, modify the arch/Config_new.pl script.
Right now you are not getting the Jasper lib, from the environment, compiled into WRF.
This intrigues me, as the paragraph is from a printf from several lines before. In this particular script, it is the only printf that has run so far, but why the output was saved to the next created variable is beyond me. Any suggestions?
EDIT: After looking at choroba's suggestion, the same problem occurs with any type of redirection, whether piping, using tee, or stderr/stdout redirection. As such, I'm thinking it may be a problem with bash? That is, the only way I can run it is without any kind of logging (at least to my knowledge, which is admittedly quite limited).
You want to enable autoflushing, so that the Perl print buffer is flushed automatically after something is printed.
This is the default behavior when a Perl script outputs to a terminal window, but when the output is redirected in any way, the default is to buffer the output. Enabling autoflushing disables the buffering.
You can enable autoflushing by adding the following two lines to the top of the Perl script (below the Perl hashbang line, of course):
use IO::Handle qw();
STDOUT->autoflush(1);
When you redirect with pipes or similar you are (normally) redirecting STDOUT. All of the print statements go to STDOUT, so when redirecting the will be sent to whatever process you are piping to. Without seeing the full command you are using I can't say exactly why you aren't seeing the STDOUT messages, but they are obviously being swallowed by the redirection. Whether or not that is actually a problem if for you to decide.
the line
$response = <STDIN> ;
causes the script to wait for input from STDIN which is why you see the prompt. You are not piping anything in to STDIN so it waits.
I run a simple file test in perl with the code below:
my $f1 = "$pre_file";
unless (-e $1) {
print "\n Pre_check file does not exists! \n";
die;
}
It prints the following output:
Pre_check file does not exists!
Died at ./huawei-postcheck line 81.
However I do not want the last line "Died at ./huawei-postcheck line 81.".
I want to to "die" with no error message.
Is it possible?
See the documentation for die.
If the last element of LIST does not end in a newline, the current
script line number and input line number (if any) are also printed,
and a newline is supplied.
So you can get die to work without printing anything by just using die "\n". But given that you have an error message, I can't see why you don't use that.
unless (-e $f1) {
die "\n Pre_check file does not exist!\n";
}
Of course, the difference is that the message will now go to STDERR rather than STDOUT. But that's probably the right place for it to go.
use exit instead of die.
You could just say
die "\n";
to suppress the message.
You probably want to exit 1 instead of dying then.
my $f1 = "$pre_file";
unless (-e $1) {
print "\n Pre_check file does not exists! \n";
exit 1;
}
I have written a script which is reading some data from log file and transform the data to simpler form and writing it back to another file. the reading is done line by line with a delay of 5 seconds i.e. sleep(5).
Meanwhile, on the command line if a user enters 'suspend' (through STDIN) then the program went on to sleep unless 'resume' is not entered and then read the next line.
Since, with every iteration in the loop I am checking STDIN whether 'suspend' is entered or not by the user.
if not then read the next line from the file. but when my programs runs I have to at least hit a ENTER key, otherwise it does not picks the next line from the input log file albeit i put an if statement to check if STDIN is undefined or not.
I am not a perl expert and this the first time I am writing a code in PERL. infact i have never done this file parsing thing before :'-(
my code implementation is like this;
#!/usr/local/bin/perl
my $line_no = 0;my $cancel = 0; my $line = "";
my $curpos = 0; my $whence = 0;
my $log_file = "/var/log/tmp_nagios.log";
#open(LOGFILE, "+< $log_file")or die "Failed to open $log_file, $!";
my $inp_file = "/var/log/sec_input";
my $logbuffer="";
#open(LOGFILE, "+< $log_file")or die "Failed to open $log_file, $!";
my $in;
while(1){
print "in While (1) Pos: $curpos and Whence:$whence\n";
open(LOGFILE, "+< $log_file")or die "Failed to open $log_file, $!";
seek(LOGFILE, $curpos, $whence);
next if(eof(LOGFILE));
print "Beginning\n";
while(<LOGFILE>){
#chomp($in = <STDIN>);
#if(defined($in) && $in =~ /^suspend$/i){
### Problem here ###
if(defined(<STDIN>) && <STDIN> =~ /^suspend\n$/i){ ## checking if 'suspend' is entered
print "Suspend Mode";
do{
sleep(5);
}while(!(<STDIN> =~ /^resume\n$/i));
print "Resume now\n";
close(LOGFILE);
last;
}
else{
$line = $_;
if($line =~ m/^\[(\d+)\]\sCURRENT\sSERVICE\sSTATE:\s(\w+);(\w+|\_|\d+)+;(CRITICAL|OK);.+$/){
$logbuffer = "$1,$2-$3,$4\n";
print $logbuffer;
open(INPFILE, ">> $inp_file")or die "Failed! to open $inp_file, $!";
#print INPFILE $logbuffer;
close(INPUTFILE);
sleep(5);
$curpos = tell(LOGFILE);
$whence = 1;
}
}
}
print "\nRe openning the file from Pos=$curpos and Whence=$whence\n";
}
close(LOGFILE);
here is the sample log file (/var/log/tmp_nagios.log) data;
[1284336000] CURRENT SERVICE STATE: host1;event1;CRITICAL; s
[1284336000] CURRENT SERVICE STATE: host2;event1;CRITICAL; f
[1284336000] CURRENT SERVICE STATE: host3;event3;CRITICAL; g
[1284336000] CURRENT SERVICE STATE: host4;event4;CRITICAL; j
[1284336000] CURRENT SERVICE STATE: host5;event1;CRITICAL; s
[1284336000] CURRENT SERVICE STATE: host6;event1;CRITICAL; f
[1284336000] CURRENT SERVICE STATE: host7;event7;CRITICAL; s
Sorry guys! Typo mistake
In the beginning I said, 'my script is reading data from log file with a delay of 5 seconds i.e. sleep(5)'
but actually i forget to mention it in my code, therefore, uncomment this line: #sleep(3); and make 'sleep(5);'
thanks
If I have understood your question correctly: check out the Term::ReadKey CPAN Module.
You can use it to do non-blocking buffer reads. (If there is nothing in the buffer, your script does not pause for user input)
https://metacpan.org/pod/Term::ReadKey
You may also like to approach this problem slightly differently - using signals:
http://perldoc.perl.org/perlipc.html. You can have your program run normally, but capture interrupts (e.g. CTRL-C)
Alternatively, you could just use CTRL-Z and fg to make your script sleep and wake.
Another option is POE::Wheel::FollowTail for the log and POE::Wheel::ReadLine or Term::Visual for user input. Though this might be a little overkill