Perl not able to tail rotated log file - perl

I am using property based filters of rsyslog to send specific logs to seperate file where those logs will be parsed using perl.
This is my rsyslog entry
$template SPLIT,"/home/shivam/hello-%$YEAR%%$MONTH%%$DAY%%$HOUR%%$MINUTE%"
:msg, contains, "hello" -?SPLIT
So rsyslog will create separate files for logs coming after every minute. Files will be created like this hello-201505281139.
My perl script to parse these files is
use strict;
use warnings;
use POSIX qw(strftime);
my $date = strftime "%Y%m%d%H%M", localtime;
my $file = '/root/defer-'.$date;
open(my $fh,'<',$file) or die "unable to open file $!\n";
while(1) {
while(my $line = <$fh>){
print "$date\n";
print "$line";
}
sleep(2);
unless ($date == strftime "%Y%m%d%H%M", localtime) {
close($fh);
$date = strftime "%Y%m%d%H%M", localtime;
$file = '/root/defer-'. $date;
system("touch $file");
open(my $fh,'<',$file) or die "unable to open file $!\n";
}
}
In unless block i am checking that if minute has changed then i close previous file and open new file.
The reason i am creating new file from script and not waiting for rsyslog to create file is that the frequency of logs coming is not that much. So i create file and just start reading on it in hope that when any new log will come i will be able to read that.
But what is happening is that i am able to create new file but not able to read anything from that file.
This is what i am getting as warning
readline() on closed filehandle $fh at test.pl line 14.
Line 14 in my code is this line
while(my $line = <$fh>){
I am not able to see anything wrong in my code. Please suggest what is the mistake.

You have two different $fh lexical (my) variables,
So instead declaring new one
open(my $fh,'<',$file) or die "unable to open file $!\n";
keep using previously declared one,
open($fh,'<',$file) or die "unable to open file $!\n";

Related

How Can I output the response to text file and open the text file when done

I want to modify a Perl script to save the result to text file output.txt and open the results output.txt when task is done
I have tried used "use strict;
use warnings;" and other methods but I keep getting an errors' The Last error was "Couldn't open output.txt"
#!/usr/bin/perl
use HTTP::Request;
use LWP::UserAgent;
system('cls');
system "color c";
print"\n";
print" |||||||||||||||||Robots scanner|||||||||||||||||||||";
print "\n";
print " Scan Your site Site\n\n Example: www.test.com \n\n-> ";
$site=<STDIN>;
chomp $site;
if($site !~ /http:\/\//) { $site = "http://$site/"; };
print "\n";
#path = ('robotx.txt','robots.txt','robot.txt','search',);
foreach $robots(#path){
$url = $site.$robots;
$req = HTTP::Request->new(GET=>$url);
$useragent = LWP::UserAgent->new();
$response = $useragent->request($req);
my $filename = 'report.txt';
open(my $fh, '>', $filename) or die "Could not open file '$filename' $!";
if ($response->is_success){
print ">>>>>>Robots Found !!!>>>: $url\n";
print $fh "out put has been generated by perl\n"; # THIS DOESN'T WORK THE report.txt is empty
print "done\n";
}else{
print "NotFound : $robots\n";
print "done\n";
# I want to open the file from windows explorer automatically here when its done
close $fh;
}
}
after running cmd as Admin , as you can see the report.txt file shows empty
I expect to see the out put of the Response
I also want perl to open the report.txt ( without going to windows explorer and opening it manually by the user ) I want it automatically open when its done but I wasn't able to achieve that.
Thanks!
Seems you're always overwriting your report file.
Your open is inside the foreach loop. The loop is done 4 times. You will only receive the output of the last run as the first 3 file creations will be overwritten by the last one.
You should put the open and close outside the loop.
If you only need the first result, you can leave, the loop by putting a last statement at the end of the "then-part" of your if.
Try changing open(my $fh, '>', $filename) to open(my $fh, '>>', $filename) (two arrows instead of one. This will add each new entry instead of delete the last one.
Here are some answers on opening Windows files with Perl:
How to run an executable file using Perl on Windows XP?

How to modify content of a file using single file handle

I'm trying to modify content of a file using Perl.
The following script works fine.
#!/usr/bin/perl
use strict;
use warnings;
open(FH,"test.txt") || die "not able to open test.txt $!";
open(FH2,">","test_new.txt")|| die "not able to opne test_new.txt $!";
while(my $line = <FH>)
{
$line =~ s/perl/python/i;
print FH2 $line;
}
close(FH);
close(FH2);
The content of test.txt:
im learning perl
im in File handlers chapter
The output in test_new.txt:
im learning python
im in File handlers chapter
If I try to use same file handle for modifying the content of file, then I'm not getting expected output. The following is the script that attempts to do this:
#!/usr/bin/perl
use strict;
use warnings;
open(FH,"+<","test.txt") || die "not able to open test.txt $!";
while(my $line = <FH>)
{
$line =~ s/perl/python/i;
print FH $line;
}
close(FH);
Incorrect output in test.txt:
im learning perl
im learning python
chapter
chapter
How do I modify the file contents using single file handle?
You can't delete from a file (except at the end).
You can't insert characters into a file (except at the end).
You can replace a character in a file.
You can append to a file.
You can shorten a file.
That's it.
You're imagining you can simply replace "Perl" with "Python" in the file. Those aren't of the same length, so it would require inserting characters into the file, and you can't do that.
You can effectively insert characters into a file by loading the rest of the file into memory and writing it back out two characters further. But doing this gets tricky for very large files. It's also very slow since you end up copying a (possibly very large) portion of the file every time you want to insert characters.
The other problem with in-place modifications is that you can't recover from an error. If something happens, you'll be left with an incomplete or corrupted file.
If the file is small and you're ok with losing the data if something goes wrong, the simplest approach is to load the entire file into memory.
open(my $fh, '<+', $qfn)
or die("Can't open \"$qfn\": $!\n");
my $file = do { local $/; <$fh> };
$file =~ s/Perl/Python/g;
seek($fh, 0, SEEK_SET)
or die $!;
print($fh $file)
or die $!;
truncate($fh)
or die $!;
A safer approach is to write the data to a new file, then rename the file when you're done.
my $new_qfn = $qfn . ".tmp";
open(my $fh_in, '<', $qfn)
or die("Can't open \"$qfn\": $!\n");
open(my $fh_out, '<', $new_qfn)
or die("Can't create \"$new_qfn\": $!\n");
while (<$fh_in>) {
s/Perl/Python/g;
print($fh_out $_);
}
close($fh_in);
close($fh_out);
rename($qfn_new, $qfn)
or die $!;
The downside of this approach is it might change the file's permissions, and hardlinks will point to the old content instead of the new file. You also need permissions to create a file.
As #Сухой27 answered
it's typical situation that perl onliner pleasingly used.
perl -i -pe 's/perl/python/i'
perl takes below options
-p : make line by line loop(every line assign into $_ and print after evaluated $_)
-e : evaluate code block in above loop ( regex take $_ as default operand )
-i : in plcae file edit (if you pass arguments for -i, perl preserve original files with that extention)
if you run below script
perl -i.bak -pe 's/perl/python/i' test.txt
you will get modified test.txt
im learning python
im in File handlers chapter
and get original text files named in test.txt.bak
im learning perl
im in File handlers chapter

Perl multiple files read replacing string, write to multiple files

I have this code working, but this is only for one file with specific name, how can I let it does all .vb file in current folder and output with file name plus _1 in the back
#!/usr/bin/perl
use strict;
use warnings;
open my $fhIn, '<', 'file.vb' or die $!;
open my $fhOut, '>', 'file_1.vb' or die $!;
while (<$fhIn>) {
print $fhOut "'01/20/2016 Added \ngpFrmPosition = Me.Location\n" if /MessageBox/ and !/'/;
print $fhOut $_;
}
close $fhOut;
close $fhIn;
I might approach it like this. (This assumes the script is running in the same directory as the .vb files).
#!/usr/bin/perl
use strict;
use warnings;
# script running in same directory as the .vb files
for my $file (glob "*.vb") {
my $outfile = $file =~ s/(?=\.vb$)/_1/r;
print "$file $outfile\n"; # DEBUG
# open input and output files
# do the while loop
}
The print statement in the loop is for debug purposes - to see if you are creating the new file names correctly. You can delete it or comment it out when you are satisfied you have got the files you want.
Update: Put the glob in the for loop instead of reading it to an array.

Open file in perl and STDERR contents problems

I'm trying to write some unit tests for a perl file uploading script. I'm still pretty new to perl so I'm having some issues achieving the outcome I expect from my code.
Basically my thought process is that I can pass a test_only attribute along with the request that will tell the script to just grab a file already on the system rather than try to use an uploaded file.
I created a test file and put it in my output/tmp directory. I made sure to set its permissions to 775. Its just a simple .txt file that says "I am a test file".
What I expect to happen currently is that when I run my test script I should see the contents of the file printed out to the error log as well as some reference to the buffer(so I can verify the file is being opened properly). However, this is not happening, nothing is being put in the error log. I'm wondering if the file is being opened properly?
I'm sure I'm just missing something fundamental about how perl opens files. Any help will be appreciated. Thanks :)
This is the appropriate snippet of my code:
my $test_only = 1;
my $tmp_uploads_path = "/home/my_instance/output/tmp/";
if($test_only)
{
#put simulated file handle and file name here
$file = "";
$file_name = "test_file.txt";
}
else
{
$file = $q->upload('file')
|| die "No file data sent\n $!";
$file_name = $q->param('file_name')
|| die "No file_name sent\n $!";
}
########
#SAVE THE UPLOAD
########
my $bufsize = 1024;
my $buffer = '';
open(my $TMPFILE, ">".$tmp_uploads_path.$file_name);
binmode $TMPFILE;
print STDERR "=> ".Dumper($TMPFILE)."\n";
while(read ($TMPFILE, $buffer, $bufsize)){
print STDERR "=> ".Dumper($TMPFILE)."\n";
print STDERR "=> ".Dumper($buffer)."\n";
print $TMPFILE $buffer;
}
close($TMPFILE);
You opened the $TMPFILE for writing, due to the > mode. Therefore, you cannot read from it.
You should always put use strict; use warnings; at the top of your scripts, this would have alerted you to this problem!
You should open files like
my $name = ...;
open my $fh, "<", $name or die "Can't open $name: $!";
or
use autodie;
open my $fh, "<", $name;
That is, do proper error handling, and use the three-arg variant of open: handle, mode and name (don't concat mode and name, except on ancient perls).
I am also suprised that you are using read. You can get a similar effect by
local $/ = \$bufsize;
while (defined(my $buffer = <$TMPFILE>)) { ... }

perl unable to copy contents of file and print it

I need to read/copy the contents of a file(test.pl) just as the way it is formatted and email it.
I am using the following code but I am unable to print anything out.
I get this error even though the file exists in the same directory.
Failed: No such file or directory
Code:
#!/usr/bin/perl
use strict;
use warnings;
use DBI;
open my $fh, '<', 'test.pl '
or die "Failed: $!\n";
my $text = do {
local $/;
<$fh>
};
close $fh
or die "Failed again: $!\n";
print $text, "\n";
It looks like there is an extra space in the filename you are trying to open. In your open statement, try changing 'test.pl ' to 'test.pl'.
if you are going to read files names from STDIN (user's input), you may want to trim them either by using regex (s/^\s+//....) or Text::Trim among other validations.