I'm working on a Perl one liner tutorial and there are one liners like this:
ls -lAF | perl -e 'while (<>) {next if /^[dt]/; print +(split)[8] . " size: " . +(split)[4] . "\n"}'
You see the function name split has been inside parentheses. Documentation about this use of functions is hard to find on Google so I couldn't find any information on it. Could somebody explain it? Thank you.
It probably doesn't help that the use of split is defaulting everything - it's splitting $_ by spaces and returning a list of values.
The (...)[8] is called a list slice, and it filters out all but the 9th value returned by split. The preceding plus is there to prevent Perl from misparsing the brackets as being part of a function call. Which also means you don't need it on the second instance.
So print +(split)[8]; is basically a very succinct way of writing
my #results=split(/ /,$_);
print $results[8];
The example you've included is performing the split twice so it might be more efficient to do the more verbose version as you can get $results[4] from the above without any extra effort.
Or because you can put a list of indexes inside the [], you could do the split once and use printf to format the output like this
printf "%s size: %s\n", (split)[8,4];
In my opinion you should be avoiding this author's advice, both for the reasons laid out in my comments on your question, and because they don't appear to know their topic at all well.
The original "one-liner" was this
ls -lAF | perl -e 'while (<>) {next if /^[dt]/; print +(split)[8] . " size: " . +(split)[4] . "\n"}'
This could be written much more succinctly by using the -n and -a options, giving this
ls -lAF | perl -wane 'print $F[8] size: $F[4]\n" unless /^[dt]/'
Even without the "luxury" of these options you could write
ls -lAF | perl -e '/^[dt]/ or printf "%s size: %s\n", (split)[8,4] while <>'
I recommend that you go and read the Camel Book several times over the next few years. That is the best way to learn the language that I have found.
Most installations of Perl include a full set of documentation, accessible using the perldoc command.
You need to read the Slices section of perldoc perldata which makes very clear this use of slicing.
Related
I have the following Perl 'one-liner' script (found it online, so not mine):
perl -lsne '
/$today.* \[([0-9.]+)\]:.+dovecot_(?:login|plain):([^\s]+).* for (.*)/
and $sender{$2}{r}+=scalar (split / /,$3)
and $sender{$2}{i}{$1}=1;
END {
foreach $sender(keys %sender){
printf"Recip=%05d Hosts=%03d Auth=%s\n",
$sender{$sender}{r},
scalar (keys %{$sender{$sender}{i}}),
$sender;
}
}
' -- -today=$(date +%F) /var/log/exim_mainlog | sort
I've been trying to understand its innards, because I would like to modify it to re-use its functionality.
Some questions I got:
What does the flag -lsne does? (From what I know, it's got to be, at least, 3 different flags in one)
Where does $sender gets its value from?
What about that (?:login|plain) segment, are they 'variables'? (I get that's ReGex, I'm just not familiarized with it)
What I'm trying to achieve:
Get the number of emails sent by each user in a SMTP relay periodically (cron job)
If there's an irregular number of emails (say, 500 in a 1-hour timespan), do something (like shutting of the service, or send a notification)
Why I'm trying to achieve this:
Lately, someone has been using my SMTP server to send spam, so I would like to monitor email activity so they stop abusing the SMTP relay resources. (Security-related suggestions are always welcomed, but out of topic for this question. Trying to focus on the script for now)
What I'm NOT trying to achieve:
To get the script done by third-parties. (Just try and point me in the right direction, maybe an example)
So, any suggestions, guidance,and friendly comments are welcomed. I understand this may be an out-of-topic question, yet I've been struggling with this for almost a week and my background with Perl is null.
Thanks in advance.
What does the flag -lsne does? (From what I know, it's got to be, at least, 3 different flags in one)
-l causes lines of input read in to be auto-chomped, and lines of
output printed out to have "\n" auto-appended
-s enables switch
parsing. This is what creates the variable $today, because a
command-line switch of --today=$(date +%F) was passed.
-n surrounds the entire "one-liner" in a while (<>) { ... } loop.
Effectively reading every line from standard input and running the
body of the one liner on that line
-e is the switch that tells
perl to execute the following code from the command line, rather
than running a file containing Perl code
Where does $sender gets its value from?
I suspect you are confusing $sender with %sender. The code uses $sender{$2}{r} without explicitly mentioning %sender. This is a function of Perl called "auto-vivification". Basically, because we used $sender{$2}{r}, perl automatically created a variable %sender, and added a key whose name is whatever is in $2, and set the value of that key in %sender to be a reference to a new hash. It then set that new hash to have a key 'r' and a value of scalar (split / /,$3)
What about that (?:login|plain) segment, are they 'variables'? (I get that's ReGex, I'm just not familiarized with it)
It's saying that this portion of the regular expression will match either 'login' or 'plain'. The ?: at the beginning tells Perl that these parentheses are used only for clustering, not capturing. In other words, the result of this portion of the pattern match will not be stored in the $1, $2, $3, etc variables.
-MO=Deparse is your friend for understanding one-liners (and one liners that wrap into five lines on your terminal):
$ perl -MO=Deparse -lsne '/$today.* \[([0-9.]+)\]:.+dovecot_( ...
BEGIN { $/ = "\n"; $\ = "\n"; }
LINE:
while ( defined($_ = <ARGV>) ) {
chomp $_;
$sender{$2}{'i'}{$1} = 1 if
/$today.* \[([0-9.]+)\]:.+dovecot_(?:login|plain):([^\s]+).* for (.*)/
and $sender{$2}{'r'} += scalar split(/ /, $3, 0);
sub END {
foreach $sender (keys %sender) {
printf "Recip=%05d Hosts=%03d Auth=%s\n",
$sender{$sender}{'r'},
scalar keys %{$sender{$sender}{'i'};}, $sender;
}
}
}
-e syntax OK
[newlines and indentation added for clarity]
What does the flag -lsne does? (From what I know, it's got to be, at least, 3 different flags in one)
You can access a summary of the available perl command line options by running '~$ perl -h' in the terminal. Below are filtered out the specific command line options you were asking about.
~$ perl -h|perl -ne 'print if /^\s+(-l|-s|-n|-e)/'
-e program one line of program (several -e's allowed, omit programfile)
-l[octal] enable line ending processing, specifies line terminator
-n assume "while (<>) { ... }" loop around program
-s enable rudimentary parsing for switches after programfile
Two examples of the '-s' command line option in use.
~$ perl -se 'print "Todays date is $today\n"' -- -today=`date +%F`
Todays date is 2016-10-17
~$ perl -se 'print "The sky is $color.\n"' -- -color='blue'
The sky is blue.
For detailed explanations of those command line options read the online documentation below.
http://perldoc.perl.org/perlrun.html
Or run the command below from your terminal.
~$ perldoc perlrun
Unrelated to the questions of the OP, I'm aware that this is not a complete answer (added as much as I was able to at the moment), so if this post/answer violates any SO rules, the moderators should remove it. Thx.
I just had a task in where I needed to replace each 3rd value in a tabulator separated file with a fixed value. I guess it can be done in Perl on a Unix shell like so
$perl -a -n -i -F'/\t/' -e '$F[2]="THE FIXED VALUE";print join "\t", #F' bla.txt
I just wanted to know if this is a "correct" way, or if there is a better (for a currently lacking definition of better) to do it?
I think your one-liner is reasonable and readable. There are many more ways to do it. I would stack the perlrun options and save a few keystrokes:
perl -F'\t' -i -ape'$F[2]="THE FIXED VALUE"; $_ = join "\t", #F' bla.txt
A shame that $, does not get populated with the argument of -F, so there's still a piece of repetition.
I have a file, someFile, like this:
$cat someFile
hdisk1 active
hdisk2 active
I use this shell script to check:
$cat a.sh
#!/usr/bin/ksh
for d in 1 2
do
grep -q "hdisk$d" someFile && echo "$d : ok"
done
I am trying to convert it to Perl:
$cat b.sh
#!/usr/bin/ksh
export d
for d in 1 2
do
cat someFile | perl -lane 'BEGIN{$d=$ENV{'d'};} print "$d: OK" if /hdisk$d\s+/'
done
I export the variable d in the shell script and get the value using %ENV in Perl. Is there a better way of passing this value to the Perl one-liner?
You can enable rudimentary command line argument with the "s" switch. A variable gets defined for each argument starting with a dash. The -- tells where your command line arguments start.
for d in 1 2 ; do
cat someFile | perl -slane ' print "$someParameter: OK" if /hdisk$someParameter\s+/' -- -someParameter=$d;
done
See: perlrun
Sometimes breaking the Perl enclosure is a good trick for these one-liners:
for d in 1 2 ; do cat kk2 | perl -lne ' print "'"${d}"': OK" if /hdisk'"${d}"'\s+/';done
Pass it on the command line, and it will be available in #ARGV:
for d in 1 2
do
perl -lne 'BEGIN {$d=shift} print "$d: OK" if /hdisk$d\s+/' $d someFile
done
Note that the shift operator in this context removes the first element of #ARGV, which is $d in this case.
Combining some of the earlier suggestions and adding my own sugar to it, I'd do it this way:
perl -se '/hdisk([$d])/ && print "$1: ok\n" for <>' -- -d='[value]' [file]
[value] can be a number (i.e. 2), a range (i.e. 2-4), a list of different numbers (i.e. 2|3|4) (or almost anything else, that's a valid pattern) or even a bash variable containing one of those, example:
d='2-3'
perl -se '/hdisk([$d])/ && print "$1: ok\n" for <>' -- -d=$d someFile
and [file] is your filename (that is, someFile).
If you are having trouble writing a one-liner, maybe it is a bit hard for one line (just my opinion). I would agree with #FM's suggestion and do the whole thing in Perl. Read the whole file in and then test it:
use strict;
local $/ = '' ; # Read in the whole file
my $file = <> ;
for my $d ( 1 .. 2 )
{
print "$d: OK\n" if $file =~ /hdisk$d\s+/
}
You could do it looping, but that would be longer. Of course it somewhat depends on the size of the file.
Note that all the Perl examples so far will print a message for each match - can you be sure there are no duplicates?
My solution is a little different. I came to your question with a Google search the title of your question, but I'm trying to execute something different. Here it is in case it helps someone:
FYI, I was using tcsh on Solaris.
I had the following one-liner:
perl -e 'use POSIX qw(strftime); print strftime("%Y-%m-%d", localtime(time()-3600*24*2));'
which outputs the value:
2013-05-06
I was trying to place this into a shell script so I could create a file with a date in the filename, of X numbers of days in the past. I tried:
set dateVariable=`perl -e 'use POSIX qw(strftime); print strftime("%Y-%m-%d", localtime(time()-3600*24*$numberOfDaysPrior));'`
But this didn't work due to variable substitution. I had to mess around with the quoting, to get it to interpret it properly. I tried enclosing the whole lot in double quotes, but this made the Perl command not syntactically correct, as it messed with the double quotes around date format. I finished up with:
set dateVariable=`perl -e "use POSIX qw(strftime); print strftime('%Y-%m-%d', localtime(time()-3600*24*$numberOfDaysPrior));"`
Which worked great for me, without having to resort to any fancy variable exporting.
I realise this doesn't exactly answer your specific question, but it answered the title and might help someone else!
That looks good, but I'd use:
for d in $(seq 1 2); do perl -nle 'print "hdisk$ENV{d} OK" if $_ =~ /hdisk$ENV{d}/' someFile; done
It's already written on the top in one long paragraph but I am also writing for lazy developers who don't read those lines.
Double quotes and single quote has big different meaning for the bash.
So please take care
Doesn't WORK perl '$VAR' $FILEPATH
WORKS perl "$VAR" $FILEPATH
Is it possible one way or another to, within a Perl script, effectively execute grep against a Perl variable? An equivalent Perl function would be equally acceptable, I just want to keep the solution as simple as possible.
For example:
#!/usr/bin/perl
#!/bin/grep
$var="foobar";
$newvar="system('grep -o "foo" $var');
sprintf $newvar;
Where I expect sprintf $newvar to output foo.
Would also welcome any feedback on best practice here. I am not extremely familiar with Perl.
you can just use regex matching in Perl. No need to call external "grep" command.
$var =~ /foo/;
please read documentation perlrequick for introduction on how to search for patterns in variables. Also of interest is Perl's own grep.
$var="foobar";
if ( $var =~ /foo/){
print "found foo\n";
}
Is it possible to get the current source line number in Perl?
The equivalent in C++ is __LINE__.
The __LINE__ literal is documented in the Special Literals section of the perldata man page.
print "File: ", __FILE__, " Line: ", __LINE__, "\n";
or
warn("foo");
Note there's a gotcha with
$ perl -e'warn("foo")'
foo at -e line 1.
If it ends with a newline it won't print the line number
$ perl -e'warn("foo\n")'
foo
This is documented in perldoc -f die, but is perhaps easy to miss in the perldoc -f warn section's reference to die.
This prints out the line where you are, and also the "stack" (list of lines from the calling programs (scripts/modules/etc) that lead to the place you are now)
while(my #where=caller($frame++)) { print "$frame:" . join(",",#where) . "\n"; }
"use Carp" and play with the various routines and you also get a stack - not sure if this way is better or worse than the "caller" method suggested by cnd. I have used the LINE and FILE variables (and probably other similar variables) in C and Perl to show where I got in the code and other information when debugging but have seen little value outside a debug environment.