I ran into this problem while trying to print single quotes in a Perl one-liner. I eventually figured out you have to escape them with '\''. Here's some code to illustrate my question.
Let's start with printing a text file.
perl -ne 'chomp; print "$_\n"' shortlist.txt
red
orange
yellow
green
blue
Now let's print the name of the file instead for each line.
perl -ne 'chomp; print "$ARGV\n"' shortlist.txt
shortlist.txt
shortlist.txt
shortlist.txt
shortlist.txt
shortlist.txt
Then we can add single quotes around each line.
perl -ne 'chomp; print "'$_'\n"' shortlist.txt
shortlist.txt
shortlist.txt
shortlist.txt
shortlist.txt
shortlist.txt
Wait that didn't work. Let's try again.
perl -ne 'chomp; print "'\''$_'\''\n"' shortlist.txt
'red'
'orange'
'yellow'
'green'
'blue'
So I got it working now. But I'm still confused on why '$_' evaluates to the program name. Maybe this is something easy but can someone explain or link to some documentation?
edit: I'm running Perl 5.8.8 on Red Hat 5
To your shell, 'chomp; print "'$_'\n"' results in a string that's the concatenation of
chomp; print " (the first sequence inside single quotes),
the value of its variable $_, and
\n" (the second sequence inside single quotes).
In bash, $_ "... expands to the last argument to the previous command, after expansion. ...". Since this happens to be shortlist.txt, the following is passed to perl:
chomp; print "shortlist.txt\n"
For example,
$ echo foo
foo
$ echo 'chomp; print "'$_'\n"'
chomp; print "foo\n"
Note that the above mechanism shouldn't be used to pass values to a Perl one-liner. You shouldn't be generating Perl code from the shell. See How can I process options using Perl in -n or -p mode? for how to provide arguments to a one-liner.
You use single quotes in one-liners to protect your Perl code from being evaluated by the shell. In this command:
perl -ne 'chomp; print "'$_'\n"' shortlist.txt
you close the single quotes before $_, so the shell expands $_ to the last argument to the previous command. In your case, this happened to be the name of your input file, but the output would be different if you ran a different command first:
$ echo foo
$ perl -ne 'chomp; print "'$_'\n"' shortlist.txt
foo
foo
foo
foo
foo
I try to avoid quotes in one liners for just this reason. I use generalized quoting when I can:
% perl -ne 'chomp; print qq($_\n)'
Although I can avoid even that with the -l switch to get the newline for free:
% perl -nle 'chomp; print $_'
If I don't understand a one-liner, I use -MO=Deparse to see what Perl thinks it is. The first two are what you expect:
% perl -MO=Deparse -ne 'chomp; print "$_\n"' shortlist.txt
LINE: while (defined($_ = <ARGV>)) {
chomp $_;
print "$_\n";
}
-e syntax OK
% perl -MO=Deparse -ne 'chomp; print "$ARGV\n"' shortlist.txt
LINE: while (defined($_ = <ARGV>)) {
chomp $_;
print "$ARGV\n";
}
-e syntax OK
You see something funny in the one where you saw the problem. The variable has disappeared before perl ever saw it and there's a constant string in its place:
% perl -MO=Deparse -ne 'chomp; print "'$_'\n"' shortlist.txt
LINE: while (defined($_ = <ARGV>)) {
chomp $_;
print "shortlist.txt\n";
}
-e syntax OK
Your fix is curious too because Deparse puts the variable name in braces to separate it from the old package specifier ':
% perl -MO=Deparse -ne 'chomp; print "'\''$_'\''\n"' shortlist.txt
LINE: while (defined($_ = <ARGV>)) {
chomp $_;
print "'${_}'\n";
}
-e syntax OK
Related
I've started playing around with perl and I'm trying to figure out what is wrong with telling perl to use a loop if I also provide a loop?
It looks like perl is getting confused with the same open file descriptors but what I don't get is why does it eat the first line?
perl -ne 'while (<>) { print $_; }'
Of course in this simple example, I can simply perl -ne '{print $_}' to arrive at the same functional logic.
But what I want to know is what is going wrong with the double loop that the first line disappears if yet another while (<>) { } gets wrapped?
$ perl -ne '{print $_}' hello
hello
hello
world
world
^C
$ perl -ne 'while (<>) { print $_; }'
hello
world
world
^C
Update: According to the answers what seems to be happening is that Perl is waiting on the first loop for STDIN input. Upon receiving input on STDIN, the input is assigned to the internal buffer $_ and the logic proceeds to the second loop where again it waits for new STDIN input. Upon receiving new STDIN input, it clobbers the STDIN buffer $_ with the new STDIN input and begins printing.
You can itself check the code generated by one-liner using O=Deparse.
First:
$ perl -MO=Deparse -ne 'print $_;' file
LINE: while (defined($_ = <ARGV>)) {
print $_;
}
-e syntax OK
Second:
$ perl -MO=Deparse -ne 'while (<>) { print $_; }' file
LINE: while (defined($_ = <ARGV>)) {
while (defined($_ = <ARGV>)) {
print $_;
}
}
-e syntax OK
Now, It is easy to know what is wrong with second case. Outer while eats the first line of file and it is lost.
The -n flag wraps your code inside a while (<>) { ... } construct.
So in your second example, the code that is actually executed is
while (<>) # reads a line from STDIN, places it in $_
{
# you don't do anything with the contents of $_ here
while (<>) # reads a line from STDIN, places it in $_, overwriting the previous value
{
print $_; # prints the contents of $_
}
}
Which means the line that was read by the first <> is just lost.
I am looking for a Perl oneliner (inserting it into Bash script), and I need the next interface:
perl -0777 -nlE 'commands' file1 file2 .... fileN
I created the next:
perl -0777 -lnE 'BEGIN{$str=quotemeta(do{local(#ARGV, $/)="file1"; <>})} say "working on $ARGV" if $_ =~ /./' "$#"
Prettier:
perl -0777 -lnE '
BEGIN{
$str = quotemeta(
do{
local(#ARGV, $/)="file1"; <> #localize ARGV to "file1.txt" for <>
}
)
}
say "working on $ARGV" if $_ =~ /./ #demo only action
' "$#"
It works, but with this I need edit the source code every time when needing to change file1.
How do I change the script to the following?
Slurp the $ARGV[0] (file1) into $str (in the BEGIN block)
And slurp the other arguments into $_ in the main loop
Pass it as an argument, removing it from #ARGV in the BEGIN block.
$ echo foo >refile
$ echo -ne 'foo\nbar\nfood\nbaz\n' >file1
$ echo -ne 'foo\nbar\nfood\nbaz\n' >file2
$ perl -lnE'
BEGIN {
local #ARGV = shift(#ARGV);
$re = join "|", map quotemeta, <>;
}
say "$ARGV:$.:$_" if /$re/;
close(ARGV) if eof; # Reset $.
' refile file1 file2
file1:1:foo
file1:3:food
file2:1:foo
file2:3:food
Can anyone explain the difference in output of the two perl (using cygwin) commands below:
$ echo abc | perl -n -e 'if ($_ =~ /a/) {print 1;}'
prints :
1
$ echo abc | perl -e 'if ($_ =~ /a/) {print 1;}'
The first prints '1' while second one outputs blank?
Thanks
-n switch adds while loop around your code, so in your case $_ is populated from standard input. In second example there is no while loop thus $_ is leaved undefined.
Using Deparse you can ask perl to show how your code is parsed,
perl -MO=Deparse -n -e 'if ($_ =~ /a/) {print 1;}'
LINE: while (defined($_ = <ARGV>)) {
if ($_ =~ /a/) {
print 1;
}
}
perl -MO=Deparse -e 'if ($_ =~ /a/) {print 1;}'
if ($_ =~ /a/) {
print 1;
}
Is there a simple way to mimic the effect of the -l command-line switch within perl scripts? (Of course, I can always chomp each line and then append "\n" to each line I print, but the point is to avoid having to do this.)
No. You can get the automatic appending of "\n" by using $\, but you have to add the chomp yourself.
Here's how -l works.
$ perl -MO=Deparse -ne 'print $_'
LINE: while (defined($_ = <ARGV>)) {
print $_;
}
$ perl -MO=Deparse -lne 'print $_'
BEGIN { $/ = "\n"; $\ = "\n"; } # -l added this line
LINE: while (defined($_ = <ARGV>)) {
chomp $_; # -l added this line
print $_;
}
(The comments are mine.) Notice that -l added a literal chomp $_ at the beginning of the loop generated by -n (and it only does that if you use -n or -p). There's no variable you can set to mimic that behaviour.
It's a little-known fact that -l, -n, and -p work by wrapping boilerplate text around the code you supply before it's compiled.
Yes, try using this at the beginning of your script after the shebang and strictures:
$/ = $\ = "\n"; # setting the output/input record separator like OFS in awk
and use in the loop :
chomp;
print;
Or like this :
use strict; use warnings;
use English qw/-no_match_vars/;
$OUTPUT_RECORD_SEPARATOR = "\n";
while (<>) {
chomp;
print;
}
I do not recommend to use
#!/usr/bin/perl -l
for a better clarity =)
See perldoc perlvar
You can add it to your shebang line:
#!/usr/bin/perl -l
In exploring an alternative answer to sarathi's current file line number question, I wrote this one-liner with the expectation that it would print the first line of all files provided:
$ perl -ne 'print "$ARGV : $_" if __LINE__ == 1;' *txt
This did not work as expected; all lines were printed.
Running the one-liner through -MO=Deparse shows that the conditional is not present. I assume this is because it has been constant-folded at compile time:
$ perl -MO=Deparse -ne 'print "$ARGV : $_" if __LINE__ == 1;' *txt
LINE: while (defined($_ = <ARGV>)) {
print "$ARGV : $_";
}
-e syntax OK
But why?
Run under Perl 5.8.8.
__LINE__ corresponds to the line number in the Perl source, not in the input file.
__LINE__ is the source line number i.e., the program line number.
$. will give you the input file line number.
if you want to print all the first lines of all the files then you can try this:
perl -lne '$.=0 if eof;print $_ if ($.==1)' *.txt