Here's my perl code snippet
if($line =~ m/^Warning: (.*)$/){
$subStepValues = {
Warning => $1,
Warning_timeStamp => `date`,
};
push #{$subsSteps->{'subStepValues'}}, $subStepValues;
}
I am parsing the output of tail -f from a file to my perl code and i am really interested to get the actual on the go time stamp, currently some how executing date is not working
any other better suggestion?
How about a nice ISO timestamp?
use POSIX qw(strftime);
if ($line =~ m/^Warning: (.*)$/)
{
$subStepValues = {
Warning => $1,
Warning_timeStamp => strftime("%Y-%m-%dT%H:%M:%S", localtime),
};
push #{$subsSteps->{'subStepValues'}}, $subStepValues;
}
Here is a simple proof of concept from the command line using an empty file and running tail -f on it and then going to another terminal and appending a few lines to it in the manner echo something >> log
schumack#daddyo2 12-18T1:57:23 338> touch log
schumack#daddyo2 12-18T1:57:26 339> tail -f log | perl -lne 'BEGIN{use POSIX qw(strftime);}chomp; printf "%s -- %s\n", strftime("%Y-%m-%dT%H:%M:%S", localtime), $_;'
2015-12-18T01:57:40 -- hello
2015-12-18T01:57:46 -- line 2
2015-12-18T01:57:50 -- line 3
Related
I want to count the lines in a file and print a string which depends on the line number. But my while loop misses the first line. I believe the while (<>) construct is necessary to increment the $n variable; anyway, is not this construct pretty standard in perl?
How do I get the while loop to print the first line? Or should I not be using while?
> printf '%s\n%s\n' dog cat
dog
cat
> printf '%s\n%s\n' dog cat | perl -n -e 'use strict; use warnings; print; '
dog
cat
> printf '%s\n%s\n' dog cat | perl -n -e 'use strict; use warnings; while (<>) { print; } '
cat
>
> printf '%s\n%s\n' dog cat | perl -n -e 'use strict; use warnings; my $n=0; while (<>) { $n++; print "$n:"; print; } '
1:cat
The man perlrun shows:
-n causes Perl to assume the following loop around your program, which makes it iterate over filename
arguments somewhat like sed -n or awk:
LINE:
while (<>) {
... # your program goes here
}
Note that the lines are not printed by default. See "-p" to have lines printed. If a file named by an
argument cannot be opened for some reason, Perl warns you about it and moves on to the next file.
Also note that "<>" passes command line arguments to "open" in perlfunc, which doesn't necessarily
interpret them as file names. See perlop for possible security implications.
...
...
"BEGIN" and "END" blocks may be used to capture control before or after the implicit program loop, just as in awk.
So, in fact you running this script
LINE:
while (<>) {
# your progrem start
use strict;
use warnings;
my $n=0;
while (<>) {
$n++;
print "$n:";
print;
}
# end
}
Solution, just remove the -n.
printf '%s\n%s\n' dog cat | perl -e 'use strict; use warnings; my $n=0; while (<>) { $n++; print "$n:"; print; }'
Will print:
1:dog
2:cat
or
printf '%s\n%s\n' dog cat | perl -ne 'print ++$n, ":$_"'
with the same result
or
printf '%s\n%s\n' dog cat | perl -pe '++$n;s/^/$n:/'
but the ikegami's solution
printf "one\ntwo\n" | perl -ne 'print "$.:$_"'
is the BEST
There's a way to figure out what your one-liner is actually doing. The B::Deparse module has a way to show you how perl interpreted your source code. It's actually from the O (capital letter O, not zero) namespace that you can load with -M (ikegami explains this on Perlmonks):
$ perl -MO=Deparse -ne 'while(<>){print}' foo bar
LINE: while (defined($_ = readline ARGV)) {
while (defined($_ = readline ARGV)) {
print $_;
}
-e syntax OK
Heh, googling for the module link shows I wrote about this for The Effective Perler. Same example. I guess I'm not that original.
If you can't change the command line, perhaps because it's in the middle of a big script or something, you can set options in PERL5OPT. Then those options last for just the session. I hate changing the original scripts because it seems that no matter how careful I am, I mess up something (how many times has my brain told me "hey dummy, you know what a git branch is, so you should have used that first"):
$ export PERL5OPT='-MO=Deparse'
If I run a script with perl -Fsomething, is that something value saved anywhere in the Perl environment where the script can find it? I'd like to write a script that by default reuses the input delimiter (if it's a string and not a regular expression) as the output delimiter.
Looking at the source, I don't think the delimiter is saved anywhere. When you run
perl -F, -an
the lexer actually generates the code
LINE: while (<>) {our #F=split(q\0,\0);
and parses it. At this point, any information about the delimiter is lost.
Your best option is to split by hand:
perl -ne'BEGIN { $F="," } #F=split(/$F/); print join($F, #F)' foo.csv
or to pass the delimiter as an argument to your script:
F=,; perl -F$F -sane'print join($F, #F)' -- -F=$F foo.csv
or to pass the delimiter as an environment variable:
export F=,; perl -F$F -ane'print join($ENV{F}, #F)' foo.csv
As #ThisSuitIsBlackNot says it looks like the delimiter is not saved anywhere.
This is how the perl.c stores the -F parameter
case 'F':
PL_minus_a = TRUE;
PL_minus_F = TRUE;
PL_minus_n = TRUE;
PL_splitstr = ++s;
while (*s && !isSPACE(*s)) ++s;
PL_splitstr = savepvn(PL_splitstr, s - PL_splitstr);
return s;
And then the lexer generates the code
LINE: while (<>) {our #F=split(q\0,\0);
However this is of course compiled, and if you run it with B::Deparse you can see what is stored.
$ perl -MO=Deparse -F/e/ -e ''
LINE: while (defined($_ = <ARGV>)) {
our(#F) = split(/e/, $_, 0);
}
-e syntax OK
Being perl there is always a way, however ugly. (And this is some of the ugliest code I have written in a while):
use B::Deparse;
use Capture::Tiny qw/capture_stdout/;
BEGIN {
my $f_var;
}
unless ($f_var) {
$stdout = capture_stdout {
my $sub = B::Deparse::compile();
&{$sub}; # Have to capture stdout, since I won't bother to setup compile to return the text, instead of printing
};
my (undef, $split_line, undef) = split(/\n/, $stdout, 3);
($f_var) = $split_line =~ /our\(\#F\) = split\((.*)\, \$\_\, 0\);/;
print $f_var,"\n";
}
Output:
$ perl -Fe/\\\(\\[\\\<\\{\"e testy.pl
m#e/\(\[\<\{"e#
You could possible traverse the bytecode instead, since the start probably will be identical every time until you reach the pattern.
I've wrote super simple script that rotate trough my logs files and tail them, endlessly.
My problem that I can't Ctrl+C this program and I need to Ctrl+Z and kill it after.
How can I solve this ?
I've tried with perl, but I have a msg "alarm clock" that I want to avoid.
with perl i could Ctrl+C it fine.
perl -e "alarm 10; exec #ARGV" "tail -15f $line | filter"
And my code:
#!/bin/bash
#
while :
do while read line
do
charcount=$(ls $line | awk '{ print length; }')
printf '%0.s=' $(seq 1 $charcount)
echo -e "\n$line"
printf '%0.s=' $(seq 1 $charcount)
printf '\n'
timeout 10s tail -15f $line | filter
done < <(ls /var/log/net/*.log)
done
Thanks!
The 'Alarm Clock' alert you refer to is because that's the default way of handling a SIGALRM signal, generated by the alarm function in perl.
The usual reason why Ctrl-C doesn't work, is because there's a blocking call occuring, and so the SIGINT signal sent doesn't get caught and handled. I can't see anything obvious in your code that would be causing this though.
To my mind, the most obvious way is - stop mixing perl and bash, because that way lies madness.
How about using the File::Tail module in Perl? This even has an example of how to do what you're wanting:
#!/usr/bin/perl
use strict;
use warnings;
use File::Tail;
foreach (#ARGV) {
push( #files, File::Tail->new( name => "$_", debug => $debug ) );
}
while (1) {
( $nfound, $timeleft, #pending ) =
File::Tail::select( undef, undef, undef, $timeout, #files );
unless ($nfound) {
# timeout - do something else here, if you need to
}
else {
foreach (#pending) {
print $_->{"input"} . " ("
. localtime(time) . ") "
. $_->read;
}
}
}
This will probably allow you to ctrl-c whilst running it anyway, but Perl does allow you to have better control over signals via the %SIG hash - allowing you to define custom handlers for kill signals - such as SIGALRM from alarm and SIGINT from Ctrl-C.
I've got a one-liner like this:
date +%H | perl -ne 'printf "%02d", $_ - ($ARGV[0] - 1);' 1
It says:
Can't open 1: Datei oder Verzeichnis nicht gefunden.
The error message means "File or directory not found".
I want it to take both the output from date and the commandline argument at the same time.
Essentially it should get me the current hour minus the argument minus one. If there are better ways to achieve this, I'll happily accept them. I'd still be grateful about an explanation as to why this doesn't work.
Let's assume it's after 10am now.
Param Output
1 10
2 09
3 08
The result might be yesterday or even further back in the past, it does not make much sense to print just the hour.
perl -mDateTime -e'
my $dt = DateTime->now;
$dt->subtract(hours => 1);
$dt->subtract(hours => shift #ARGV);
print $dt->hour;
' 4
Whereever possible, use a standard datetimestamp, such as RFC3339 which is in wide use.
perl -mDateTime -mDateTime::Format::RFC3339 -e'
my $dt = DateTime->now;
$dt->subtract(hours => 1);
$dt->subtract(hours => shift #ARGV);
print DateTime::Format::RFC3339->new->format_datetime($dt);
' 4
Your Perl one-liner deparses to:
LINE: while (defined($_ = <ARGV>)) {
printf '%02d', $_ - ($ARGV[0] - 1);
}
… because of -n. ARGV:
The special filehandle that iterates over command-line filenames in #ARGV.
But you have no filenames as arguments.
perl -n creates an implicit loop reading files listed as arguments or STDIN if there are no arguments; this conflicts with your use of an argument for something different. You can fix it by clearing #ARGV in a BEGIN block:
date +%H | perl -ne 'BEGIN{ $arg=shift } printf "%02d", $_ - ($arg - 1);' 1
but for this particular task, you're better off doing the date calculation entirely in perl anyway.
I'd like to run a subcommand from Perl (or pipe it into a Perl script) and have the script process the command's output immediately, rather than waiting for a timeout, a newline, or a certain number of blocks. For example, let's say I want to surround each chunk of input with square brackets. When I run the script like this:
$ ( echo -n foo ; sleep 5 ; echo -n bar ; sleep 5; echo baz) | my_script.pl
I'd like the output to be this, with each line appearing five seconds after the previous one:
[foo]
[bar]
[baz]
How do I do that?
This works, but is really ugly:
#! /usr/bin/perl -w
use strict;
use Fcntl;
my $flags = '';
fcntl(STDIN, F_GETFL, $flags);
$flags |= O_NONBLOCK;
fcntl(STDIN, F_SETFL, $flags);
my $rin = '';
vec($rin,fileno(STDIN),1) = 1;
my $rout;
while (1) {
select($rout=$rin, undef, undef, undef);
last if eof();
my $buffer = '';
while (my $c = getc()) {
$buffer .= $c;
}
print "[$buffer]\n";
}
Is there a more elegant way to do it?
From perlfaq5: How can I read a single character from a file? From the keyboard?. You probably also want to read How can I tell whether there's a character waiting on a filehandle?. Poll the filehandle. If there is a character there, read it and reset a timer. If there is not character there, try again. If you've retried and passed a certain time, process the input.
After you read the characters, it's up to you to decide what to do with them. With all the flexibility of reading single characters comes the extra work of handling them.
Term::ReadKey can do this for you. In particular setting the ReadKey() mode to do the polling for you.
use Term::ReadKey;
$| = 1;
while( my $key = ReadKey(10) ) {
print $key;
}
If there's time inbetween each character, you might be able to detect the pauses.
Perl also does line input - if you don't use getc you should be able to add newlines to the end of foo, bar, etc and perl will give you each line.
If you can't add newlines, and you can't depend on a pause, then what exactly do you expect the system to do to tell perl that it's started a new command? As far as perl is concerned, there's a stdin pipe, it's eating data from it, and there's nothing in the stdin pipe to tell you when you are executing a new command.
You might consider the following instead:
$ echo "( echo -n foo ; sleep 5 ; echo -n bar ; sleep 5; echo baz)" | my_script.pl
or
$ my_script.pl$ "echo -n foo ; sleep 5 ; echo -n bar ; sleep 5; echo baz"
And modify your perl program to parse the input "command line" and execute each task, eating the stdout as needed.
-Adam
See How to change Open2 input buffering. (Basically, you have to make the other program think it's talking to a tty.)
You didn't mention how you are reading input in your Perl script, but you might want to look at the getc function:
$|++; # set autoflush on output
while ($c = getc(STDIN)) {
print $c;
}