I'm currently using Perl to read a logfile backwards using the aptly named File::ReadBackwards module. However, the first line it returns only contains
CLOG� ��
Am I doing something wrong by trying to read the last line of a cyclic logfile?
This is the relevant code:
use File::ReadBackwards;
my $filePath = '/var/log/';
my #fileNames = ('gateways.log', 'system.log');
for(my $i = 0; $i <= #fileNames; $i++){
scanFile($filePath,$fileNames[$i]);
sleep(60);
}
sub scanFile(){
my $handle = File::ReadBackwards->new($_[0] . $_[1]);
my $line = $handle->readline;
return $line
}
EDIT: I just read this - https://doc.pfsense.org/index.php/Why_can't_I_view_view_log_files_with_cat/grep/etc%3F_(clog)
I can't seem to find a module or the likes that already reads these, can anyone point me the right way to extract lines from this CLog?
Related
New to Perl and quite new to coding in general so I apologise if this is formatted terribly and an easy question! Trying simply to input somebody's elses code as a step in a larger project involving PRAAT. The code is designed to distinguish beats in speech rhythm, I've followed their nomenclature in file naming (on line 2) but the code won't move past line 13. Could anyone tell me why? Is it trying to open a directory called "intensities"? Additionally, is there anywhere else I may have to change the code, it is quite possibly reasonably old! Thank you very much!
#!/usr/local/bin/perl -w
scalar(#ARGV) == 1 or scalar(#ARGV) == 2 or die "Usage: getBeatsOneShot.pl someSoundFile <threshold>";
$stem = shift;
# Parameters to fiddle with
if (scalar(#ARGV) == 0) {
$threshold = 0.2;
} else {
$threshold = shift;
print "Threshold is $threshold\n";
}
open(IN, "intensities/$stem.intensity") or die "badly";
open(OUT, ">beats/$stem.beats") or die "eek";
# File type = "ooTextFile short"
$_ = <IN>; print OUT $_;
# replace "Intensity" with "TextGrid"
$_ = <IN>; print OUT "\"TextGrid\"\n\n";
# skip a line
$_ = <IN>;
chomp($xmin = <IN>);
chomp($xmax = <IN>);
chomp($nx = <IN>); $nx = 0; #(just suprress a arning here)
chomp($dx = <IN>);
chomp($x1 = <IN>);
# Read in intensity contour into #e (envelope)
#e = ();
while($_ = <IN>) { chomp; last unless $_ eq "1";}
push #e, $_;
while($_ = <IN>) {
chomp($_);
push #e, $_;
}
# (1) Find max and min
$max = 0; $min = 1000000;
foreach $ival (#e) {
if($ival > $max) {
$max = $ival;
}
if($ival < $min) {
$min = $ival;
}
}
# (2) look for beats
#beats = ();
print "Thresh: $threshold\n";
open doesn't create the path to the file. Directories intensities/ and beats/ therefore must exist in the current working directory before the script is run.
When open fails, it sets $! to the reason of the failure. Instead of eek or badly, use die $! so Perl can tell you what went wrong.
Moreover, you should turn strict and warnings on. They prevent many common mistakes. As a newbie, you might like to enable diagnostics, too, to get detailed explanations of all the errors and warnings.
I'm working with xml-files which I need to manipulate in my script. My first approach on this was:
qx(perl export_xml.pl $export_params > $path$prefix\investment.xml); # Create the xml-file
open DERI, '+<'.$path.$prefix.'investment.xml' or die 'Can\'t open investment.xml: '.$!;
my #derivative_xml = <DERI>;
seek(DERI, 0, 0);
foreach (#derivative_xml) {
$_ =~ s/^\s*$//g;
$_ =~ s/^.*detected on Server.*$//g;
$_ = encode('utf8', $_);
}
print DERI join('', #derivative_xml);
This is working for testing purposes, but unfortunately the real files are just too big for that (up to 6GB).
Is there a way to read the file line by line and then modify the input through the filehandle? Something like
foreach (<DERI>) { ##derivative_xml) {
$_ =~ s/^\s*$//g;
$_ =~ s/^.*detected on Server.*$//g;
$_ = encode('utf8', $_);
}
I can't really test that in a non-ridiculous amount of time, so it would be pretty nice, if I didn't have to trial and error here.
Thanks in advance!
This should work. No need of another script file.
perl -pi -e 's/^\s*$//g;s/^.*detected on Server.*$//g;$_ = encode('utf8', $_)' investment.xml
Did not test though with huge file upto 6GB. Test this and check how much time it takes.
Basically, I'm looping through html files and looking for a couple of regexes. They match which is fine, but I don't expect every file to contain matches, but when the loop runs, every iteration contains the same match (despite it not being in that file). I assume that by using $1 it is persisting through each iteration.
I've tried using an arbitary regex straight after each real match to reset it, but that doesn't seem to work. The thread I got that idea from seemed to have a lot of argument etc on best practice and the original questions problem, so I thought it would be worth asking for specific advice to my code. It's likely not written in a great way either:
# array of diff filenames
opendir(TDIR, "$folder/diff/$today") || die "can't opendir $today: $!";
#diffList = grep !/^\.\.?$/, readdir(TDIR);
closedir TDIR;
# List of diff files
print "List of Diff files:\n" . join("\n", #diffList) . "\n\n";
for($counter = 0; $counter < scalar(#diffList); $counter++) {
# Open diff file, read in to string
$filename = $diffList[$counter];
open FILE, "<", "$folder/diff/$today/$filename";
while(<FILE>) {
$lines .= $_;
}
close FILE or warn "$0: close today/$filename: $!";
# Use regular expressions to extract the found differences
if($lines =~ m/$plus1(.*?)$span/s) {
$plus = $1;
"a" =~ m/a/;
} else {$plus = "0";}
if($lines =~ m/$minus1(.*?)$span/s) {
$minus = $1;
"a" =~ m/.*/;
} else {$minus = "0";}
# If changes were found, send them to the database
if($plus ne "0" && $minus ne "0") {
# Do stuff
}
$plus = "0";
$minus = "0";
}
If I put a print inside the "do stuff" if, it's always true and always shows the same two values that are found in one of the files.
Hopefully I've explained my situation well enough. Any advice is appreciated, thanks.
It may be that your code appends lines from newly-read files onto $lines. Have you tried explicitly clearing it after each iteration?
It's already been answered, but you could also consider a different syntax for reading the file. It can be noticeably quicker and helps you avoid little bugs like this.
Just add this to read the file between the open/close:
local $/ = undef;
$lines = <FILE>;
That'll temporarily unset the line separator so it reads the whole file at once. Just enclose it in a { } block if you need to read another file in the same scope.
I have a piece of code which opens up a file and parses it. This text document has a redundant structure and has multiple entries. I need to peek ahead within my loop to see if there is a new entry, if there is, I will be able to parse all of the data my program extracts. Let me first show my implementation so far
use strict;
my $doc = open(my $fileHandler, "<", "test.txt");
while(my $line = <$fileHandler>) {
## right here I want to look at the next line to see if
## $line =~ m/>/ where > denotes a new entry
}
Try handling the iteration yourself:
my $line = <$fileHandler>;
while(1) { # keep looping until I say so
my $nextLine = <$fileHandler>;
if ($line =~ m/>/ || !defined $nextLine) {
### Do the stuff
}
### Do any other stuff;
last unless defined $nextLine;
$line = $nextLine;
}
I added the extra check in the if statement under the assumption that you will also want to process what you have when you reach the end of the file.
Alternatively, as suggested by friedo, if the file can fit into memory, you can load the whole thing into an array at once:
my #lines = <$fileHandler>;
for (my $i = 0; $i <= $#lines; $i++) {
if ($i == $#lines || $lines[$i+1] =~ />/) {
### Do the stuff
}
}
This is more flexible in that you can access any arbitrary line of the file, in any order, but as mentioned the file does have to be small enough to fit into memory.
A nice way to handle these problems is using Tie::File, which allows you to treat a file like an array, without the performance penalty of actually loading the file into memory. It is also a core module since perl v5.7.3.
use Tie::File;
tie my #file, 'Tie::File', "test.txt" or die $!;
for my $linenr (0 .. $#file) { # loop over line numbers
if ($file[$linenr] =~ /foo/) { # this is the current line
if ($file[$linenr + 1] =~ /^>/ && # this is the next line
$linenr <= $#file) { # don't go past end of file
# do stuff
}
}
}
untie #file; # all done
This question already has answers here:
Closed 11 years ago.
Possible Duplicate:
Why does Perl's glob return undef for every other call?
This is a continuation of another problem I was having where I needed to find a file that had a long name but I knew what part of the name was, here is the code I used:
my #RTlayerRabcd = ("WIRA_Rabcd_RT","WIRB_Rabcd_RT","WIRC_Rabcd_RT","WIRD_Rabcd_RT","WIRE_Rabcd_RT","BASE_Rabcd_RT");
#Rabcd calculations
for($i = 0; $i < 6; $i++)
{
print "#RTlayerRabcd[$i]\n";
#Searching for Rabcd Room Temperature readings
my $file = glob($dir . "*#RTlayerRabcd[$i]" . '.txt');
print "$file\n";
my $Rtot = 0;
#Open file, Read line
open (FILE, $file);
while (<FILE>)
{
#read line and and seperate at "tab" into $temp, $Res
chomp;
($Res, $temp) = split("\t");
$j++;
$Rtot=$Res+$Rtot;
}
close (FILE);
$Ravg = $Rtot/$j;
print FILE_OUT "$Ravg \t";
}
After I run the code I get the following print outs:
WIRA_Rabcd_RT
Vesuvious_C6R8_051211/vesu_R6C8_05112011_WIRA_Rabcd_Rt.txt
WIRB_Rabcd_RT
WIRC_Rabcd_RT
Vesuvious_C6R8_051211/vesu_R6C8_05112011_WIRC_Rabcd_Rt.txt
WIRD_Rabcd_RT
WIRE_Rabcd_RT
BASE_Rabcd_RT
Vesuvious_C6R8_051211/vesu_R6C8_05112011_BASE_Rabcd_Rt.txt
The program seems to be skipping files, any idea why?
In scalar context, glob iterates through all files matched by the glob, returning undef after the last one. It's intended to be used as the condition in a while loop, for example:
while (my $file = glob('*.c')) {
say $file;
}
The iterator is tied to that particular call to glob. Once the iteration has started, there's no way to reset the iterator early. glob ignores its argument until after it has returned undef.
You can fix your problem by using glob in list context:
my ($file) = glob($dir . "*#RTlayerRabcd[$i]" . '.txt');
My guess would be that the directory Vesuvious_C6R8_051211 does not contain the files *WIRB_Rabcd_RT.txt, *WIRD_Rabcd_RT.txt and *WIRE_Rabcd_RT.txt. What is the file listing of that directory? Also, I recommend you check the return code for open(), to be sure that it indeed opened a file successfully.