How to truncate STDIN line length? - perl

I've been parsing through some log files and I've found that some of the lines are too long to display on one line so Terminal.app kindly wraps them onto the next line. However, I've been looking for a way to truncate a line after a certain number of characters so that Terminal doesn't wrap, making it much easier to spot patterns.
I wrote a small Perl script to do this:
#!/usr/bin/perl
die("need max length\n") unless $#ARGV == 0;
while (<STDIN>)
{
$_ = substr($_, 0, $ARGV[0]);
chomp($_);
print "$_\n";
}
But I have a feeling that this functionality is probably built into some other tools (sed?) That I just don't know enough about to use for this task.
So my question sort of a reverse question: how do I truncate a line of stdin Without writing a program to do it?

Pipe output to:
cut -b 1-LIMIT
Where LIMIT is the desired line width.

Another tactic I use for viewing log files with very long lines is to pipe the file to "less -S". The -S option for less will print lines without wrapping, and you can view the hidden part of long lines by pressing the right-arrow key.

Not exactly answering the question, but if you want to stick with Perl and use a one-liner, a possibility is:
$ perl -pe's/(?<=.{25}).*//' filename
where 25 is the desired line length.

The usual way to do this would be
perl -wlne'print substr($_,0,80)'
Golfed (for 5.10):
perl -nE'say/(.{0,80})/'
(Don't think of it as programming, think of it as using a command line tool with a huge number of options.) (Yes, the python reference is intentional.)

A Korn shell solution (truncating to 70 chars - easy to parameterize though):
typeset -L70 line
while read line
do
print $line
done

You can use a tied variable that clips its contents to a fixed length:
#! /usr/bin/perl -w
use strict;
use warnings
use String::FixedLen;
tie my $str, 'String::FixedLen', 4;
while (defined($str = <>)) {
chomp;
print "$str\n";
}

This isn't exactly what you're asking for, but GNU Screen (included with OS X, if I recall correctly, and common on other *nix systems) lets you turn line wrapping on/off (C-a r and C-a C-r). That way, you can simply resize your terminal instead of piping stuff through a script.
Screen basically gives you "virtual" terminals within one toplevel terminal application.

use strict;
use warnings
use String::FixedLen;
tie my $str, 'String::FixedLen', 4;
while (defined($str = <>)) {
chomp;
print "$str\n";
}

Unless I'm missing the point, the UNIX "fold" command was designed to do exactly that:
$ cat file
the quick brown fox jumped over the lazy dog's back
$ fold -w20 file
the quick brown fox
jumped over the lazy
dog's back
$ fold -w10 file
the quick
brown fox
jumped ove
r the lazy
dog's bac
k
$ fold -s -w10 file
the quick
brown fox
jumped
over the
lazy
dog's back

Related

Printing to same line STDOUT in perl [duplicate]

Okay, so what I'm trying to do is print out a percentage complete to my command line, now, I would like this to simply 'update' the number shown on the screen. So somehow go back to the beginning of the line and change it.
For example the windows relog.exe command-line utility (which can convert a .blg file to a .csv file) does this. If you run it, it will display a percentage complete.
Now this is probably written in C++.
I don't know if this is possible in perl as well ?
Use "\r" or "\015" octal (aka "Return caret" aka "Carriage Return" character originating from typewriter days :)
> perl5.8 -e 'print "11111\r222\r3\n";'
32211
> perl5.8 -e 'print "11111\015222\0153\n";'
32211
Just don't forget to print at least as many characters as the longest string already printed to overwrite any old characters (as you can see in example above, the failure to do so will keep old characters).
Another thing to be aware of is, as Michael pointed in the commment, the autoflush needs to be turned on while these printings happen, so that the output doesn't wait for newline character at the very end of the processing.
UPDATE: Please note that 013 octal character recommended in another answer is actually a Vertical Tab:
> perl5.8 -e 'print "11111\013222\0133\n";'
11111
222
3
Depending on what you'd like to do, pv might solve your problem. It can wrap any script that takes a file as input, and add a progress bar.
For example
pv data.gz | gunzip -c | ./complicated-perl-script-that-reads-stdin
pv is packaged for RedHat/CentOS and Ubuntu at least. More information: http://www.ivarch.com/programs/pv.shtml
Otherwise I'd use CPAN, e.g. Term::ProgressBar.
You can also use \b to move back one character:
local $| = 1; #flush immediately
print "Doing it - 10%";
sleep(1);
print "\b\b\b";
print "20%";
print "\n", "Done", "\n";
In C and C++, the trick is to print char #13. Maybe it can work in Perl.
for (int pc = 0 ; pc <= 100 ; ++pc)
printf("Percentage: %02d %% %c", pc, 13);
printf("\n");

Using an awk command inside a Perl script

This may not be the best way to do the below, so any comments are appreciated.
I'm currently tailing a number of log files and outputting them to screen, so that I get a quick overview of the system.
What I would like to do is to highlight different messages [INFO], [WARN] and [ERROR]
The following syntax works fine on the command line, but fails when being called from Perl
system ("tail -n 5 $ArchiverLog | awk '{if ($4 ~ /DEBUG/)print "\033[1;33m"$0"\033[0m"; else if ($6 ~ /ERROR/) print "\033[1;31m"$0"\033[0m"; else print $0}'");
I believe Perl can do this
Should I read in the file line by line, match on the words and print to screen (I only want the last 10 lines). Is that a better option?
I've also seen reference to a2p, which is an awk to Perl translator. Would that be people's preferred choice?
It seems crazy to use one powerful scripting language to call up another one so it can do something which the first one can do very well, so I would not persist with trying to call up awk from perl.
I have not had much experience with a2p, rather I tend to just translate such snippets by hand.
#!/usr/bin/perl
use strict;
foreach(`tail -n 5 $ArchiverLog`) {
my #f = split;
if ($f[4] =~ /DEBUG/) {
print "\033[1;33m$_\033[0m";
} elsif ($f[6] =~ /ERROR/) {
print "\033[1;31m$_\033[0m";
} else {
print $_;
}
}
(Hard to say if the above is completely correct without some sample input to test it with).
As Borodin says in the comments a more elegant solution would be to use a Tailing module from CPAN rather than calling up tail as a subprocess. But for a quick tool that might be overkill.
NB: if $ArchiverLog comes from anywhere you don't have control of, remember to sanitise it, otherwise you are creating a nice security hole.

In-place deletion of a block of lines already found in a file using perl

I want to be able to locate a block of lines in a file determined by start and end keywords and then delete this block. I am using "if (/START/../END/)" to locate this block, but am not sure how to delete the lines in this block. What statement can I use inside the 'if' above to achieve this?
Note: It does not have to be true deletion. I mean, it can be simply replace the line with empty space.
PS: I am a first-time perl user and pardon me if this seems to be a stupid question. I know there are MANY similar questions out there, but no one seems to be dealing with in-place deletion and suggest options like print entire file to another file excluding the desired block.
Perl makes this pretty easy.
One line, in place deletion of lines between pattern1 and pattern2:
perl -i -ne 'print unless /pattern1/../pattern2/' input_file
See perlrun to understand perl's various command-line switches
You could just invert your logic:
use warnings;
use strict;
while (<DATA>) {
print unless /START/ .. /END/;
}
=for output
foo
bar
=cut
__DATA__
foo
START
goo
END
bar
With sed:
sed '/START/,/END/d' input_file
to modify the original file:
sed -i '/START/,/END/d' input_file

Is there a better way to detab (expand tabs) using Perl?

I wanted to detab my source files. (Please, no flame about WHY I wanted to detab my sources. That's not the point :-) I couldn't find a utility to do that. Eclipse didn't do it for me, so I implemented my own.
I couldn't fit it into a one liner (-e) program.
I came with the following, which did the job just fine.
while( <> )
{
while( /\t/ ) {
s/^(([^\t]{4})*)\t/$1 /;
s/^((([^\t]{4})*)[^\t]{1})\t/$1 /;
s/^((([^\t]{4})*)[^\t]{2})\t/$1 /;
s/^((([^\t]{4})*)[^\t]{3})\t/$1 /;
}
print;
}
However, it makes me wonder if Perl - the champion language of processing text - is the right tool. The code doesn't seem very elegant. If I had to detab source that assume tab=8 spaces, the code would look even worse.
Specifically because I can think of a deterministic state machine with only 4 states to do the job.
I have a feeling that a more elegant solution exists. Am I missing a Perl idiom? In the spirit of TIMTOWTDI I'm curious about the other ways to do it.
u.
What ever happened to the old Unix program "expand"? I used to use that all the time.
I remember a detabify script from one of the O'Reilly books, but I can't seem to find a link now.
I have had to solve this problem as well, and I settled on this concise solution to detabify a line:
1 while $line =~ s/\t/" " x ($tablength - ($-[0] % $tablength))/e ;
In this regular expression $-[0] is the length of the "pre-matched" portion of the line -- the number of characters before the tab character.
As a one-liner:
perl -pe '1 while s/\t/" "x(4-($-[0]%4))/e' input
Whenever I want to expand tabs, I use Text::Tabs.
This can be easily done in vim:
:set expandtab
:retab
http://vim.wikia.com/wiki/Converting_tabs_to_spaces
Can't let vi be all alone here. Emacs:
M-x tabify
M-x untabify
I do this in vim with:
:%s/^V^I/ /g
(That's a literal ^V followed by a literal tab), and then :%gq to fix incorrect spacing. Perl is overkill.
The exact expression is:
1 while $line =~ s/\t/" " x ($tablength+1 - ($-[0] % $tablength))/e ;
And expand is useful for command line not inside a program which may expand or not some lines.

Perl: print back to beginning of line

Okay, so what I'm trying to do is print out a percentage complete to my command line, now, I would like this to simply 'update' the number shown on the screen. So somehow go back to the beginning of the line and change it.
For example the windows relog.exe command-line utility (which can convert a .blg file to a .csv file) does this. If you run it, it will display a percentage complete.
Now this is probably written in C++.
I don't know if this is possible in perl as well ?
Use "\r" or "\015" octal (aka "Return caret" aka "Carriage Return" character originating from typewriter days :)
> perl5.8 -e 'print "11111\r222\r3\n";'
32211
> perl5.8 -e 'print "11111\015222\0153\n";'
32211
Just don't forget to print at least as many characters as the longest string already printed to overwrite any old characters (as you can see in example above, the failure to do so will keep old characters).
Another thing to be aware of is, as Michael pointed in the commment, the autoflush needs to be turned on while these printings happen, so that the output doesn't wait for newline character at the very end of the processing.
UPDATE: Please note that 013 octal character recommended in another answer is actually a Vertical Tab:
> perl5.8 -e 'print "11111\013222\0133\n";'
11111
222
3
Depending on what you'd like to do, pv might solve your problem. It can wrap any script that takes a file as input, and add a progress bar.
For example
pv data.gz | gunzip -c | ./complicated-perl-script-that-reads-stdin
pv is packaged for RedHat/CentOS and Ubuntu at least. More information: http://www.ivarch.com/programs/pv.shtml
Otherwise I'd use CPAN, e.g. Term::ProgressBar.
You can also use \b to move back one character:
local $| = 1; #flush immediately
print "Doing it - 10%";
sleep(1);
print "\b\b\b";
print "20%";
print "\n", "Done", "\n";
In C and C++, the trick is to print char #13. Maybe it can work in Perl.
for (int pc = 0 ; pc <= 100 ; ++pc)
printf("Percentage: %02d %% %c", pc, 13);
printf("\n");