Is there a simple way to mimic the effect of the -l command-line switch within perl scripts? (Of course, I can always chomp each line and then append "\n" to each line I print, but the point is to avoid having to do this.)
No. You can get the automatic appending of "\n" by using $\, but you have to add the chomp yourself.
Here's how -l works.
$ perl -MO=Deparse -ne 'print $_'
LINE: while (defined($_ = <ARGV>)) {
print $_;
}
$ perl -MO=Deparse -lne 'print $_'
BEGIN { $/ = "\n"; $\ = "\n"; } # -l added this line
LINE: while (defined($_ = <ARGV>)) {
chomp $_; # -l added this line
print $_;
}
(The comments are mine.) Notice that -l added a literal chomp $_ at the beginning of the loop generated by -n (and it only does that if you use -n or -p). There's no variable you can set to mimic that behaviour.
It's a little-known fact that -l, -n, and -p work by wrapping boilerplate text around the code you supply before it's compiled.
Yes, try using this at the beginning of your script after the shebang and strictures:
$/ = $\ = "\n"; # setting the output/input record separator like OFS in awk
and use in the loop :
chomp;
print;
Or like this :
use strict; use warnings;
use English qw/-no_match_vars/;
$OUTPUT_RECORD_SEPARATOR = "\n";
while (<>) {
chomp;
print;
}
I do not recommend to use
#!/usr/bin/perl -l
for a better clarity =)
See perldoc perlvar
You can add it to your shebang line:
#!/usr/bin/perl -l
Related
I've started playing around with perl and I'm trying to figure out what is wrong with telling perl to use a loop if I also provide a loop?
It looks like perl is getting confused with the same open file descriptors but what I don't get is why does it eat the first line?
perl -ne 'while (<>) { print $_; }'
Of course in this simple example, I can simply perl -ne '{print $_}' to arrive at the same functional logic.
But what I want to know is what is going wrong with the double loop that the first line disappears if yet another while (<>) { } gets wrapped?
$ perl -ne '{print $_}' hello
hello
hello
world
world
^C
$ perl -ne 'while (<>) { print $_; }'
hello
world
world
^C
Update: According to the answers what seems to be happening is that Perl is waiting on the first loop for STDIN input. Upon receiving input on STDIN, the input is assigned to the internal buffer $_ and the logic proceeds to the second loop where again it waits for new STDIN input. Upon receiving new STDIN input, it clobbers the STDIN buffer $_ with the new STDIN input and begins printing.
You can itself check the code generated by one-liner using O=Deparse.
First:
$ perl -MO=Deparse -ne 'print $_;' file
LINE: while (defined($_ = <ARGV>)) {
print $_;
}
-e syntax OK
Second:
$ perl -MO=Deparse -ne 'while (<>) { print $_; }' file
LINE: while (defined($_ = <ARGV>)) {
while (defined($_ = <ARGV>)) {
print $_;
}
}
-e syntax OK
Now, It is easy to know what is wrong with second case. Outer while eats the first line of file and it is lost.
The -n flag wraps your code inside a while (<>) { ... } construct.
So in your second example, the code that is actually executed is
while (<>) # reads a line from STDIN, places it in $_
{
# you don't do anything with the contents of $_ here
while (<>) # reads a line from STDIN, places it in $_, overwriting the previous value
{
print $_; # prints the contents of $_
}
}
Which means the line that was read by the first <> is just lost.
I'm able to do this on the command line and it works :
~/Tools/perl/edif_extr_cell.pl design.edif nmos1p8v | perl -p -e 's/^/\n/ if /portImplementation|figure\s+device/;' | perl -n -000 -e 'print if /portImplementation/;'
(basically, extracting a section of the EDIF file).
Now, I want to make a utility of this. And my script is below. Question : can this code be more efficient? If feel like it's very inelegant. I could pipe streams easily on the command line but, in a script, I feel lost.
#!/usr/bin/perl -w -p
BEGIN{ $file = '';}
s/^/\n/ if /portImplementation|figure\s+device/;
$file .= $_;
END{
$cmd = q{\rm -f /tmp/dump}.$$.'.txt';
system( $cmd );
open( OUT, ">/tmp/dump$$.txt");
print OUT $file;
close OUT;
$out = `perl -n -000 -e 'print if /portImplementation/;' /tmp/dump$$.txt`;
system( $cmd );
print $out;
}
If I understand correct, you want to be able to do
~/Tools/perl/edif_extr_cell.pl design.edif nmos1p8v | myfilter
Ideally, you'd merge the two Perl scripts into one rather than having one script launch two instances of Perl, but this turns out to be rather hard because of the change to $/ (via -00) and because you insert newlines in the first filter.
The simplest answer:
#!/bin/sh
perl -pe's/^/\n/ if /portImplementation|figure\s+device/' |
perl -00ne'print if /portImplementation/'
It appears that you were trying to write the equivalent of that sh script in Perl. It would look like the following:
#!/usr/bin/perl
use strict;
use warnings;
use IPC::Open qw( open3 );
# open3 has issues with lexical file handles.
pipe(local *PIPE_READER, local *PIPE_WRITER)
or die($!);
my $pid1 = open3('<&STDIN', '>&PIPE_WRITER', '>&STDERR',
'perl', '-pes/^/\n/ if /portImplementation|figure\s+device/');
my $pid2 = open3('<&PIPE_READER', '>&STDOUT', '>&STDERR',
'perl', '-00neprint if /portImplementation/');
waitpid($pid1);
waitpid($pid2);
I'd normally recommend IPC::Run3 or IPC::Run for launching and interfacing with child processes, but low-level open3 does the trick nicely in this particular situation.
I downloaded a random EDIF file from GitHub, running the following script on it gives the same output as your code:
#! /usr/bin/perl
use warnings;
use strict;
my #buffer;
my $found;
my $prepend = q();
while (<>) {
if (/portImplementation|figure\s+device/) {
if ($found && #buffer) {
print $prepend, #buffer;
$prepend = "\n";
}
undef $found;
#buffer = ();
}
$found ||= /portImplementation/;
push #buffer, $_;
}
# Don't forget to output the last paragraph!
print $prepend, #buffer if $found && #buffer;
I would like to know what is the equivalent code that Perl runs when executed with the options perl -pi -e?
On some SO question I can read this:
while (<>) {
... # your script goes here
} continue {
print;
}
But this example does not show the part where the file is saved.
How does Perl determine the EOL? Does it touch the file when no changes occured? For example if I have a old MAC file (\r only). How does it deal with s/^foo/bar/gm?
I tried to use the Perl debugger but it doesn't really help. So I am just trying to guess:
#!/usr/bin/env perl
my $pattern = shift;
map &process, #ARGV;
# perl -pi -e PATTERN <files>...
sub process {
next unless -f;
open my $fh, '<', $_;
my $extract;
read $fh, $extract, 1024;
seek &fh, 0, 0;
if ($extract =~ /\r\n/) {
$/ = "\r\n";
} elsif ($extract =~ /\r[^\n]/) {
$/ = "\r";
} else {
$/ = "\n";
}
my $out = '';
while(<&fh>) {
my $__ = $_;
eval $pattern;
my $changes = 1 if $_ ne $__;
$out .= $_;
}
if($changes)
{
open my $fh, '>', $_;
print $fh $out;
}
close &fh;
}
You can inspect the code actually used by Perl with the core module B::Deparse. This compiler backend module is activated with the option -MO=Deparse.
$ perl -MO=Deparse -p -i -e 's/X/U/' ./*.txt
BEGIN { $^I = ""; }
LINE: while (defined($_ = <ARGV>)) {
s/X/U/;
}
continue {
die "-p destination: $!\n" unless print $_;
}
-e syntax OK
Thus perl is looping over the lines in the given files, executes the code with $_ set to the line and prints the resulting $_.
The magic variabe $^I is set to an empty string. This turns on in place editing. In place editing is explained in perldoc perlrun. There is no check whether the file is unchanged. Thus the modified time of the edited file is always updated. Apparently the modified time of the backup file is the same as the modified time of the original file.
Using the -0 flag you can set the input record separator for using "\r" for your Mac files.
$ perl -e "print qq{aa\raa\raa}" > t.txt
$perl -015 -p -i.ori -e 's/a/b/' t.txt
$cat t.txt
ba
$ perl -MO=Deparse -015 -p -i.ori -e 's/a/b/'.txt
BEGIN { $^I = ".ori"; }
BEGIN { $/ = "\r"; $\ = undef; }
LINE: while (defined($_ = <ARGV>)) {
s/a/b/;
}
continue {
die "-p destination: $!\n" unless print $_;
}
-e syntax OK
From the perlrun documentation:
-p assumes an input loop around your script. Lines are printed.
-i files processed by the < > construct are to be edited in place.
-e may be used to enter a single line of script. Multiple -e commands may be given to build up a multiline script.
I ran into this problem while trying to print single quotes in a Perl one-liner. I eventually figured out you have to escape them with '\''. Here's some code to illustrate my question.
Let's start with printing a text file.
perl -ne 'chomp; print "$_\n"' shortlist.txt
red
orange
yellow
green
blue
Now let's print the name of the file instead for each line.
perl -ne 'chomp; print "$ARGV\n"' shortlist.txt
shortlist.txt
shortlist.txt
shortlist.txt
shortlist.txt
shortlist.txt
Then we can add single quotes around each line.
perl -ne 'chomp; print "'$_'\n"' shortlist.txt
shortlist.txt
shortlist.txt
shortlist.txt
shortlist.txt
shortlist.txt
Wait that didn't work. Let's try again.
perl -ne 'chomp; print "'\''$_'\''\n"' shortlist.txt
'red'
'orange'
'yellow'
'green'
'blue'
So I got it working now. But I'm still confused on why '$_' evaluates to the program name. Maybe this is something easy but can someone explain or link to some documentation?
edit: I'm running Perl 5.8.8 on Red Hat 5
To your shell, 'chomp; print "'$_'\n"' results in a string that's the concatenation of
chomp; print " (the first sequence inside single quotes),
the value of its variable $_, and
\n" (the second sequence inside single quotes).
In bash, $_ "... expands to the last argument to the previous command, after expansion. ...". Since this happens to be shortlist.txt, the following is passed to perl:
chomp; print "shortlist.txt\n"
For example,
$ echo foo
foo
$ echo 'chomp; print "'$_'\n"'
chomp; print "foo\n"
Note that the above mechanism shouldn't be used to pass values to a Perl one-liner. You shouldn't be generating Perl code from the shell. See How can I process options using Perl in -n or -p mode? for how to provide arguments to a one-liner.
You use single quotes in one-liners to protect your Perl code from being evaluated by the shell. In this command:
perl -ne 'chomp; print "'$_'\n"' shortlist.txt
you close the single quotes before $_, so the shell expands $_ to the last argument to the previous command. In your case, this happened to be the name of your input file, but the output would be different if you ran a different command first:
$ echo foo
$ perl -ne 'chomp; print "'$_'\n"' shortlist.txt
foo
foo
foo
foo
foo
I try to avoid quotes in one liners for just this reason. I use generalized quoting when I can:
% perl -ne 'chomp; print qq($_\n)'
Although I can avoid even that with the -l switch to get the newline for free:
% perl -nle 'chomp; print $_'
If I don't understand a one-liner, I use -MO=Deparse to see what Perl thinks it is. The first two are what you expect:
% perl -MO=Deparse -ne 'chomp; print "$_\n"' shortlist.txt
LINE: while (defined($_ = <ARGV>)) {
chomp $_;
print "$_\n";
}
-e syntax OK
% perl -MO=Deparse -ne 'chomp; print "$ARGV\n"' shortlist.txt
LINE: while (defined($_ = <ARGV>)) {
chomp $_;
print "$ARGV\n";
}
-e syntax OK
You see something funny in the one where you saw the problem. The variable has disappeared before perl ever saw it and there's a constant string in its place:
% perl -MO=Deparse -ne 'chomp; print "'$_'\n"' shortlist.txt
LINE: while (defined($_ = <ARGV>)) {
chomp $_;
print "shortlist.txt\n";
}
-e syntax OK
Your fix is curious too because Deparse puts the variable name in braces to separate it from the old package specifier ':
% perl -MO=Deparse -ne 'chomp; print "'\''$_'\''\n"' shortlist.txt
LINE: while (defined($_ = <ARGV>)) {
chomp $_;
print "'${_}'\n";
}
-e syntax OK
How could I convert:
awk '{print $2 >> $1}' file
in a short Perl one-liner?
"file" could look like this:
fruit banana
vegetable beetroot
vegetable carrot
mushroom chanterelle
fruit apple
there may some other ways, but here's what i can think of
perl -ane 'open(FILE,">>",$F[0]); print FILE $F[1];close(FILE);' file
I guess awk has to be better at some things :-)
This is right at the limit of what I'd do on the command line, but it avoids reopening filehandles.
$ perl -lane '$fh{$F[0]} || open $fh{$F[0]}, ">>", $F[0]; print {$fh{$F[0]}} $F[1]' file
Not pure Perl, but you can do:
perl -nae '`echo $F[1] >> $F[0]`' input_file
This is what a2p <<< '{print $2 >> $1}' produces
#!/usr/bin/perl
eval 'exec /usr/bin/perl -S $0 ${1+"$#"}'
if $running_under_some_shell;
# this emulates #! processing on NIH machines.
# (remove #! line above if indigestible)
eval '$'.$1.'$2;' while $ARGV[0] =~ /^([A-Za-z_0-9]+=)(.*)/ && shift;
# process any FOO=bar switches
$, = ' '; # set output field separator
$\ = "\n"; # set output record separator
while (<>) {
($Fld1,$Fld2) = split(' ', $_, -1);
&Pick('>>', $Fld1) &&
(print $fh $Fld2);
}
sub Pick {
local($mode,$name,$pipe) = #_;
$fh = $name;
open($name,$mode.$name.$pipe) unless $opened{$name}++;
}