Okay, so what I'm trying to do is print out a percentage complete to my command line, now, I would like this to simply 'update' the number shown on the screen. So somehow go back to the beginning of the line and change it.
For example the windows relog.exe command-line utility (which can convert a .blg file to a .csv file) does this. If you run it, it will display a percentage complete.
Now this is probably written in C++.
I don't know if this is possible in perl as well ?
Use "\r" or "\015" octal (aka "Return caret" aka "Carriage Return" character originating from typewriter days :)
> perl5.8 -e 'print "11111\r222\r3\n";'
32211
> perl5.8 -e 'print "11111\015222\0153\n";'
32211
Just don't forget to print at least as many characters as the longest string already printed to overwrite any old characters (as you can see in example above, the failure to do so will keep old characters).
Another thing to be aware of is, as Michael pointed in the commment, the autoflush needs to be turned on while these printings happen, so that the output doesn't wait for newline character at the very end of the processing.
UPDATE: Please note that 013 octal character recommended in another answer is actually a Vertical Tab:
> perl5.8 -e 'print "11111\013222\0133\n";'
11111
222
3
Depending on what you'd like to do, pv might solve your problem. It can wrap any script that takes a file as input, and add a progress bar.
For example
pv data.gz | gunzip -c | ./complicated-perl-script-that-reads-stdin
pv is packaged for RedHat/CentOS and Ubuntu at least. More information: http://www.ivarch.com/programs/pv.shtml
Otherwise I'd use CPAN, e.g. Term::ProgressBar.
You can also use \b to move back one character:
local $| = 1; #flush immediately
print "Doing it - 10%";
sleep(1);
print "\b\b\b";
print "20%";
print "\n", "Done", "\n";
In C and C++, the trick is to print char #13. Maybe it can work in Perl.
for (int pc = 0 ; pc <= 100 ; ++pc)
printf("Percentage: %02d %% %c", pc, 13);
printf("\n");
Related
I have the following Perl 'one-liner' script (found it online, so not mine):
perl -lsne '
/$today.* \[([0-9.]+)\]:.+dovecot_(?:login|plain):([^\s]+).* for (.*)/
and $sender{$2}{r}+=scalar (split / /,$3)
and $sender{$2}{i}{$1}=1;
END {
foreach $sender(keys %sender){
printf"Recip=%05d Hosts=%03d Auth=%s\n",
$sender{$sender}{r},
scalar (keys %{$sender{$sender}{i}}),
$sender;
}
}
' -- -today=$(date +%F) /var/log/exim_mainlog | sort
I've been trying to understand its innards, because I would like to modify it to re-use its functionality.
Some questions I got:
What does the flag -lsne does? (From what I know, it's got to be, at least, 3 different flags in one)
Where does $sender gets its value from?
What about that (?:login|plain) segment, are they 'variables'? (I get that's ReGex, I'm just not familiarized with it)
What I'm trying to achieve:
Get the number of emails sent by each user in a SMTP relay periodically (cron job)
If there's an irregular number of emails (say, 500 in a 1-hour timespan), do something (like shutting of the service, or send a notification)
Why I'm trying to achieve this:
Lately, someone has been using my SMTP server to send spam, so I would like to monitor email activity so they stop abusing the SMTP relay resources. (Security-related suggestions are always welcomed, but out of topic for this question. Trying to focus on the script for now)
What I'm NOT trying to achieve:
To get the script done by third-parties. (Just try and point me in the right direction, maybe an example)
So, any suggestions, guidance,and friendly comments are welcomed. I understand this may be an out-of-topic question, yet I've been struggling with this for almost a week and my background with Perl is null.
Thanks in advance.
What does the flag -lsne does? (From what I know, it's got to be, at least, 3 different flags in one)
-l causes lines of input read in to be auto-chomped, and lines of
output printed out to have "\n" auto-appended
-s enables switch
parsing. This is what creates the variable $today, because a
command-line switch of --today=$(date +%F) was passed.
-n surrounds the entire "one-liner" in a while (<>) { ... } loop.
Effectively reading every line from standard input and running the
body of the one liner on that line
-e is the switch that tells
perl to execute the following code from the command line, rather
than running a file containing Perl code
Where does $sender gets its value from?
I suspect you are confusing $sender with %sender. The code uses $sender{$2}{r} without explicitly mentioning %sender. This is a function of Perl called "auto-vivification". Basically, because we used $sender{$2}{r}, perl automatically created a variable %sender, and added a key whose name is whatever is in $2, and set the value of that key in %sender to be a reference to a new hash. It then set that new hash to have a key 'r' and a value of scalar (split / /,$3)
What about that (?:login|plain) segment, are they 'variables'? (I get that's ReGex, I'm just not familiarized with it)
It's saying that this portion of the regular expression will match either 'login' or 'plain'. The ?: at the beginning tells Perl that these parentheses are used only for clustering, not capturing. In other words, the result of this portion of the pattern match will not be stored in the $1, $2, $3, etc variables.
-MO=Deparse is your friend for understanding one-liners (and one liners that wrap into five lines on your terminal):
$ perl -MO=Deparse -lsne '/$today.* \[([0-9.]+)\]:.+dovecot_( ...
BEGIN { $/ = "\n"; $\ = "\n"; }
LINE:
while ( defined($_ = <ARGV>) ) {
chomp $_;
$sender{$2}{'i'}{$1} = 1 if
/$today.* \[([0-9.]+)\]:.+dovecot_(?:login|plain):([^\s]+).* for (.*)/
and $sender{$2}{'r'} += scalar split(/ /, $3, 0);
sub END {
foreach $sender (keys %sender) {
printf "Recip=%05d Hosts=%03d Auth=%s\n",
$sender{$sender}{'r'},
scalar keys %{$sender{$sender}{'i'};}, $sender;
}
}
}
-e syntax OK
[newlines and indentation added for clarity]
What does the flag -lsne does? (From what I know, it's got to be, at least, 3 different flags in one)
You can access a summary of the available perl command line options by running '~$ perl -h' in the terminal. Below are filtered out the specific command line options you were asking about.
~$ perl -h|perl -ne 'print if /^\s+(-l|-s|-n|-e)/'
-e program one line of program (several -e's allowed, omit programfile)
-l[octal] enable line ending processing, specifies line terminator
-n assume "while (<>) { ... }" loop around program
-s enable rudimentary parsing for switches after programfile
Two examples of the '-s' command line option in use.
~$ perl -se 'print "Todays date is $today\n"' -- -today=`date +%F`
Todays date is 2016-10-17
~$ perl -se 'print "The sky is $color.\n"' -- -color='blue'
The sky is blue.
For detailed explanations of those command line options read the online documentation below.
http://perldoc.perl.org/perlrun.html
Or run the command below from your terminal.
~$ perldoc perlrun
Unrelated to the questions of the OP, I'm aware that this is not a complete answer (added as much as I was able to at the moment), so if this post/answer violates any SO rules, the moderators should remove it. Thx.
I have this simple command:
printf TEST | perl -nle 'print lc'
Which prints:
test
I want:
test
...without the newline. I tried perl's printf but that removes all newlines, and I'd like to keep existing one's in place. Plus, that wouldn't work for my second example that doesn't even use print in it:
printf "BOB'S BIG BOY" | perl -ple 's/([^\s.,-]+)/\u\L$1/g'
Which prints:
Bob's Big Boy
...with that annoying newline as well. I'm hoping for a magical switch like --no-newline but I'm guessing it's something more involved.
EDIT: I've changed my use of echo in the examples to printf to clarify the problem. A few commenters were correct in stating that my problem wouldn't actually be fixed as it was written.
You simply have to remove the -l switch, see perldoc perlrun
-l[octnum]
enables automatic line-ending processing. It has two separate
effects. First, it automatically chomps $/ (the input record
separator) when used with -n or -p. Second, it assigns $\ (the output
record separator) to have the value of octnum so that any print
statements will have that separator added back on. If octnum is
omitted, sets $\ to the current value of $/.
I have a file, someFile, like this:
$cat someFile
hdisk1 active
hdisk2 active
I use this shell script to check:
$cat a.sh
#!/usr/bin/ksh
for d in 1 2
do
grep -q "hdisk$d" someFile && echo "$d : ok"
done
I am trying to convert it to Perl:
$cat b.sh
#!/usr/bin/ksh
export d
for d in 1 2
do
cat someFile | perl -lane 'BEGIN{$d=$ENV{'d'};} print "$d: OK" if /hdisk$d\s+/'
done
I export the variable d in the shell script and get the value using %ENV in Perl. Is there a better way of passing this value to the Perl one-liner?
You can enable rudimentary command line argument with the "s" switch. A variable gets defined for each argument starting with a dash. The -- tells where your command line arguments start.
for d in 1 2 ; do
cat someFile | perl -slane ' print "$someParameter: OK" if /hdisk$someParameter\s+/' -- -someParameter=$d;
done
See: perlrun
Sometimes breaking the Perl enclosure is a good trick for these one-liners:
for d in 1 2 ; do cat kk2 | perl -lne ' print "'"${d}"': OK" if /hdisk'"${d}"'\s+/';done
Pass it on the command line, and it will be available in #ARGV:
for d in 1 2
do
perl -lne 'BEGIN {$d=shift} print "$d: OK" if /hdisk$d\s+/' $d someFile
done
Note that the shift operator in this context removes the first element of #ARGV, which is $d in this case.
Combining some of the earlier suggestions and adding my own sugar to it, I'd do it this way:
perl -se '/hdisk([$d])/ && print "$1: ok\n" for <>' -- -d='[value]' [file]
[value] can be a number (i.e. 2), a range (i.e. 2-4), a list of different numbers (i.e. 2|3|4) (or almost anything else, that's a valid pattern) or even a bash variable containing one of those, example:
d='2-3'
perl -se '/hdisk([$d])/ && print "$1: ok\n" for <>' -- -d=$d someFile
and [file] is your filename (that is, someFile).
If you are having trouble writing a one-liner, maybe it is a bit hard for one line (just my opinion). I would agree with #FM's suggestion and do the whole thing in Perl. Read the whole file in and then test it:
use strict;
local $/ = '' ; # Read in the whole file
my $file = <> ;
for my $d ( 1 .. 2 )
{
print "$d: OK\n" if $file =~ /hdisk$d\s+/
}
You could do it looping, but that would be longer. Of course it somewhat depends on the size of the file.
Note that all the Perl examples so far will print a message for each match - can you be sure there are no duplicates?
My solution is a little different. I came to your question with a Google search the title of your question, but I'm trying to execute something different. Here it is in case it helps someone:
FYI, I was using tcsh on Solaris.
I had the following one-liner:
perl -e 'use POSIX qw(strftime); print strftime("%Y-%m-%d", localtime(time()-3600*24*2));'
which outputs the value:
2013-05-06
I was trying to place this into a shell script so I could create a file with a date in the filename, of X numbers of days in the past. I tried:
set dateVariable=`perl -e 'use POSIX qw(strftime); print strftime("%Y-%m-%d", localtime(time()-3600*24*$numberOfDaysPrior));'`
But this didn't work due to variable substitution. I had to mess around with the quoting, to get it to interpret it properly. I tried enclosing the whole lot in double quotes, but this made the Perl command not syntactically correct, as it messed with the double quotes around date format. I finished up with:
set dateVariable=`perl -e "use POSIX qw(strftime); print strftime('%Y-%m-%d', localtime(time()-3600*24*$numberOfDaysPrior));"`
Which worked great for me, without having to resort to any fancy variable exporting.
I realise this doesn't exactly answer your specific question, but it answered the title and might help someone else!
That looks good, but I'd use:
for d in $(seq 1 2); do perl -nle 'print "hdisk$ENV{d} OK" if $_ =~ /hdisk$ENV{d}/' someFile; done
It's already written on the top in one long paragraph but I am also writing for lazy developers who don't read those lines.
Double quotes and single quote has big different meaning for the bash.
So please take care
Doesn't WORK perl '$VAR' $FILEPATH
WORKS perl "$VAR" $FILEPATH
Okay, so what I'm trying to do is print out a percentage complete to my command line, now, I would like this to simply 'update' the number shown on the screen. So somehow go back to the beginning of the line and change it.
For example the windows relog.exe command-line utility (which can convert a .blg file to a .csv file) does this. If you run it, it will display a percentage complete.
Now this is probably written in C++.
I don't know if this is possible in perl as well ?
Use "\r" or "\015" octal (aka "Return caret" aka "Carriage Return" character originating from typewriter days :)
> perl5.8 -e 'print "11111\r222\r3\n";'
32211
> perl5.8 -e 'print "11111\015222\0153\n";'
32211
Just don't forget to print at least as many characters as the longest string already printed to overwrite any old characters (as you can see in example above, the failure to do so will keep old characters).
Another thing to be aware of is, as Michael pointed in the commment, the autoflush needs to be turned on while these printings happen, so that the output doesn't wait for newline character at the very end of the processing.
UPDATE: Please note that 013 octal character recommended in another answer is actually a Vertical Tab:
> perl5.8 -e 'print "11111\013222\0133\n";'
11111
222
3
Depending on what you'd like to do, pv might solve your problem. It can wrap any script that takes a file as input, and add a progress bar.
For example
pv data.gz | gunzip -c | ./complicated-perl-script-that-reads-stdin
pv is packaged for RedHat/CentOS and Ubuntu at least. More information: http://www.ivarch.com/programs/pv.shtml
Otherwise I'd use CPAN, e.g. Term::ProgressBar.
You can also use \b to move back one character:
local $| = 1; #flush immediately
print "Doing it - 10%";
sleep(1);
print "\b\b\b";
print "20%";
print "\n", "Done", "\n";
In C and C++, the trick is to print char #13. Maybe it can work in Perl.
for (int pc = 0 ; pc <= 100 ; ++pc)
printf("Percentage: %02d %% %c", pc, 13);
printf("\n");
I've been parsing through some log files and I've found that some of the lines are too long to display on one line so Terminal.app kindly wraps them onto the next line. However, I've been looking for a way to truncate a line after a certain number of characters so that Terminal doesn't wrap, making it much easier to spot patterns.
I wrote a small Perl script to do this:
#!/usr/bin/perl
die("need max length\n") unless $#ARGV == 0;
while (<STDIN>)
{
$_ = substr($_, 0, $ARGV[0]);
chomp($_);
print "$_\n";
}
But I have a feeling that this functionality is probably built into some other tools (sed?) That I just don't know enough about to use for this task.
So my question sort of a reverse question: how do I truncate a line of stdin Without writing a program to do it?
Pipe output to:
cut -b 1-LIMIT
Where LIMIT is the desired line width.
Another tactic I use for viewing log files with very long lines is to pipe the file to "less -S". The -S option for less will print lines without wrapping, and you can view the hidden part of long lines by pressing the right-arrow key.
Not exactly answering the question, but if you want to stick with Perl and use a one-liner, a possibility is:
$ perl -pe's/(?<=.{25}).*//' filename
where 25 is the desired line length.
The usual way to do this would be
perl -wlne'print substr($_,0,80)'
Golfed (for 5.10):
perl -nE'say/(.{0,80})/'
(Don't think of it as programming, think of it as using a command line tool with a huge number of options.) (Yes, the python reference is intentional.)
A Korn shell solution (truncating to 70 chars - easy to parameterize though):
typeset -L70 line
while read line
do
print $line
done
You can use a tied variable that clips its contents to a fixed length:
#! /usr/bin/perl -w
use strict;
use warnings
use String::FixedLen;
tie my $str, 'String::FixedLen', 4;
while (defined($str = <>)) {
chomp;
print "$str\n";
}
This isn't exactly what you're asking for, but GNU Screen (included with OS X, if I recall correctly, and common on other *nix systems) lets you turn line wrapping on/off (C-a r and C-a C-r). That way, you can simply resize your terminal instead of piping stuff through a script.
Screen basically gives you "virtual" terminals within one toplevel terminal application.
use strict;
use warnings
use String::FixedLen;
tie my $str, 'String::FixedLen', 4;
while (defined($str = <>)) {
chomp;
print "$str\n";
}
Unless I'm missing the point, the UNIX "fold" command was designed to do exactly that:
$ cat file
the quick brown fox jumped over the lazy dog's back
$ fold -w20 file
the quick brown fox
jumped over the lazy
dog's back
$ fold -w10 file
the quick
brown fox
jumped ove
r the lazy
dog's bac
k
$ fold -s -w10 file
the quick
brown fox
jumped
over the
lazy
dog's back