Parsing help in Perl - perl

Question

My (existing) perl files creates a Log file using Log4Perl in the following format
[2011-11-21 08:50:22,406] States_Sync INFO Logger.pm:33 script starts
[2011-11-21 08:50:22,610] States_Sync ERROR Logger.pm:36 Error occurred ....
[2011-11-21 08:50:22,406] States_Sync INFO Logger.pm:33 ...
[2011-11-21 08:50:22,610] States_Sync ERROR Logger.pm:36 Error occurred ....
[2011-11-21 08:50:22,406] States_Sync INFO Logger.pm:33 ...
[2011-11-21 08:50:22,610] States_Sync ERROR Logger.pm:36 Error occurred ....
The above is only an example of my log file. I use the following formatter
$layout = Log::Log4perl::Layout::PatternLayout->new("[%d{ISO8601}] %c %p %F{1}:%L %m%n");
Currently I have to send an email in case of error.
Instead of modifying the existing script, I thought of parsing the generated log files only for Error and send all the messages related to "Error" from the log file as email
Is there any easy way of parsing the log file ?
Regards,
Karthik

Answer 1

use grep(1):
grep ERROR log.file
or use perl:
perl -ne 'print if /ERROR/' log.file

Answer 2

I might try this:
if ( /^[^:]+?\s+ERROR\s/ ) {
# pull fields
# send email
}
Try that one out and see if it gives you too many lines. More elaborate version might be:
if ( /^\[\d{4}-\d\d-\d\d \d\d:\d\d:\d\d,\d{3}\]\s+([\w\s])?\s+ERROR\s+/ ) {
but only if the data was more complicated.

Answer 3

Since you're using log4perl, you could use the built-in methods there to check for an ERROR (or above) and send an e-mail based on that.
if ($layout->is_error()){
# Put e-mail logic here
}
And if you wanted just error information and not warn, info, debug or trace you can do the following:
if($layout->is_error() && ! $layout->is_warn()){
# Put e-mail logic here
}

Answer 4

You did not specify how often do you have to check the log file and send the e-mails. If it once a day then the solution provided by tadmc would be your best bet.
However, if you would like to automate it and send an e-mail after a specific interval of time (see maxinterval) for each new ERROR entry encountered, you may want to check the following.
NOTE #1 Adjust the interval and maxinterval as per your requirements to not flood people with e-mails
NOTE #2 Run it in accordance to your log rotate intervals
#!/usr/bin/perl
use strict;
use warnings;
use File::Tail;
my #logs_to_email;
my $log_file = "file.log";
my $error_pattern = qr(^\[.*?\]\s*States_Sync\s*ERROR);
my $tail = File::Tail->new(
name => $log_file,
maxinterval => 60,
interval => 10,
adjustafter => 10,
);
while (defined (my $line = $tail->read)) {
chomp $line;
next if $line =~ /^\s*$/;
next unless $line =~ $error_pattern;
push #logs_to_email, $line;
##
## put e-mail logic to send
## #logs_to_email here
##
}