First off, does anyone have a comprehensive list of the Perl special variables?
Second, are there any tasks that are much easier using them? I always unset $/ to read in files all at once, and $| to automatically flush buffers, but I'm not sure of any others.
And third, should one use the Perl special variables, or be more explicit in their coding. Personally I'm a fan of using the special variables to manipulate the way code behaves, but I've heard others argue that it just confuses things.
They are all documented in perlvar.
Note that the long names are only usable if you use English qw( -no_match_vars ); first.
Always remember to local'ize your changes to the punctuation variables. Some of the punctuation variables are useful, others should not be used. For instance, $[ should never be used (it changes the base index of arrays, so local $[ = 1; will cause 1 to refer to the first item in a list or array). Others like $" are iffy. You have to balance the usefulness of not having to do the join manually. For instance, which of these is easier to understand?
local $" = " :: "; #"
my $s = "#a / #b / #c\n";
versus
my $sep = " :: ";
my $s = join(" / ", join($sep, #a), join($sep, #a), join($sep, #a)) . "\n";
or
my $s = join(" / ", map { join " :: ", #$_ }, \(#a, #b, #c)) . "\n";
1) As far as which ones I use often:
$! is quintessential for IO error handling
$# for eval error handling when calling mis-designed libraries (like database ones) whose coders weren't considerate enough to code in decent error handling other than "die"
$_ for map/grep blocks, although I 100% agree with a poster above that using it for regular code is not a good practice.
$| for flushing buffers
2) As far as using punctuation vs. English names, I'll pick on Marc Bollinger's reply above although the same rebuttal goes for anyone arguing that there's no benefit to using English names.
"if you're using Perl, you're obviously not choosing it for neophyte readability"
Marc, I find that is not always (or rather almost never) true. Then again, 99% of my Perl experience is writing production Perl code for large companies, 90% of it full fledged applications instead of 10-line hack scripts, so my analysis may not apply in other domains. The reasons such thinking as Marc's is wrong are:
Just because I'm a Perl non-neophyte (to put it mildly), some noob analyst hired a year ago - or an outsourced "genius" - is probably not. You may not want to confuse them any more than they already are. "If code was hard to write, it should be hard to read" is not exactly high on the list of good attitudes of professional developers, in any language.
When I'm up at 2am, half-asleep and troubleshooting a production problem, I really do not want to depend on the ability of my already-nearly-blind eyes to distinguish between $! and $|. Especially in a code written by before mentioned "genius" who may not have known which one of them to use and switched them around.
When I'm reading a code left unfinished by a guy who was cough "restructured" cough out of the company a year ago, I'd rather concentrate on intricacies of screwy logic than readability of the punctuation soup.
The three I use the most are $_, #_ and $!.
I like to use $_ when looping through an array, retrieving parameters (as pointed out by Motti, this is actually #_) or performing substitutions:
Example 1.1:
foreach (#items)
{
print $_;
}
Example 1.2:
my $prm1 = shift; # implicit use of #_ or #ARGV depending on context
Example 1.3:
s/" "/""/ig; # implicit use of $_
I use $! in cases like this:
Example 2.1:
open(FILE, ">>myfile") || die "Error: $!";
I do agree though, it makes the code more confusing to someone not familiar with Perl. But confusing other people is one of the joys of knowing the language! :)
Typical ones I use are $_, #_, #ARGV, $!, $/. Other ones I comment heavily.
Brad notes that $# is also a pretty common variable. (Error value from eval()).
I say use them--if you're using Perl, you're obviously not choosing it for neophyte readability. Any more-than-casual developer will likely have a browser/reference window open, and sifting through the perlvar manpage in one window is likely no less arduous than looking up definitions of (and assignments to!) global or external variables. As an example, I just recently encountered the new-in-5.10.x named capture buffers:
/^(?<myName>.*)$/;
# and later
my $capture = %+{'myName'};
And figuring out what was going on wasn't any harder than going into parlvar/perlre and reading a little bit.
I'd much rather find a bunch of wacky special vars in undocumented code than a bunch of wacky algorithms in undocumented code.
Related
See Update Below
I am going through a whole bunch of Perl scripts that someone at my company wrote. He used join to concatenate strings. For example, he does this (taken hot out of a real Perl script):
$fullpath=join "", $Upload_Loc, "/", "$filename";
Instead of this:
$fullpath = "$Upload_Loc" . "/" . "$filename";
Or even just this:
$fullpath = "$Upload_Loc/$filename";
He's no longer here, but the people who are here tell me he concatenated strings in this way because it was somehow better. (They're not too clear why).
So, why would someone use join in this matter over using the . concatenate operator, or just typing the strings together as in the third example? Is there a valid reason for this style of coding?
I'm trying to clean up lot of the mess here, and my first thought would be to end this practice. It makes the code harder to read, and I'm sure doing a join is not a very efficient way to concatenate strings. However, although I've been writing scripts in Perl since version 3.x, I don't consider myself a guru because I've never had a chance to hang around with people who were better than Perl than I am and could teach me Perl's deep inner secrets. I just want to make sure that my instinct is correct here before I make a fool of myself.
I've got better ways of doing that around here.
Update
People are getting confused. He isn't just for concatenating paths. Here's another example:
$hotfix=join "", "$app", "_", "$mod", "_", "$bld", "_", "$hf", ".zip";
Where as I would do something like this:
$hotfix = $app . "_" $mod . "_" . $bld . "_" . "$hf.zip";
Or, more likely
$hotfix = "${app}_${mod}_${bld}_${hf}.zip";
Or maybe in this case, I might actually use join because the underscore causes problems:
$hotfix = join("_", $app, $mod, $bld, $hf) . ".zip";
My question is still: Is he doing something that real Perl hackers know, and a newbie like me who's been doing this for only 15 years don't know about? Do people look at me concatenating strings using . or just putting them in quotes and say "Ha! What a noob! I bet he owns a Macintosh too!"
Or, does the previous guy just has a unique style of programming much like my son's unique style of driving includes running head on into trees?
I've done my fair share of commercial Perl development for "a well known online retailer", and I've never seen join used like that. Your third example would be my preferred alternative, as it's simple, clean and readable.
Like others here, I don't see any genuine value in using join as a performance enhancer. It might well perform marginally better than string interpolation but I can't imagine a real-world situation where the optimisation could be justified yet the code still written in a scripting language.
As this question demonstrates, esoteric programming idioms (in any language) just lead to a lot of misunderstanding. If you're lucky, the misunderstanding is benign. The developers I enjoy working alongside are the ones who code for readability and consistency and leave the Perl Golf for the weekends. :)
In short: yes, I think his unique style is akin to your son's unique style of driving. :)
I would consider
$fullpath = join "/", $Upload_Loc, $filename;
clearer than the alternatives. However, File::Spec has been in the core for a long time, so
use File::Spec::Functions qw( catfile );
# ...
$fullpath = catfile $Upload_Loc, $filename;
is much better. And, better yet, there is Path::Class:
use Path::Class;
my $fullpath = file($Upload_Loc, $filename);
Speed is usually not a factor I consider in concatenating file names and paths.
The example you give in your update:
$hotfix=join "", "$app", "_", "$mod", "_", "$bld", "_", "$hf", ".zip";
demonstrates why the guy is clueless. First, there is no need to interpolate those individual variables. Second, that is better written as
$hotfix = join '_', $app, $mod, $bld, "$hf.zip";
or, alternatively, as
$hotfix = sprintf '%s_%s_%s_%s.zip', $app, $mod, $bld, $hf;
with reducing unnecessary punctuation being my ultimate goal.
In general, unless the lists of items to be joined are huge, you will not see much of a performance difference changing them over to concatenations. The main concern is readability and maintainability, and in those cases, if the string interpolation form is clearer, you can certainly use that.
I would guess that this is just a personal coding preference of the original programmer.
In general, I use join when the length of the list is large/unknown, or if I am joining with something other than the empty string (or a single space for array interpolation). Otherwise, using . or simple string interpolation is usually shorter and easier to read.
Perl compiles double-quoted strings into things with join and . catenation in them:
$ perl -MO=Deparse,-q -e '$fullpath = "$Upload_Loc/$filename"'
$fullpath = $Upload_Loc . '/' . $filename;
-e syntax OK
$ perl -MO=Deparse,-q -le 'print "Got #ARGV"'
BEGIN { $/ = "\n"; $\ = "\n"; }
print 'Got ' . join($", #ARGV);
-e syntax OK
which may inspire you to things like this:
$rx = do { local $" = "|"; qr{^(?:#args)$} };
as in:
$ perl -le 'print $rx = do { local $" = "\t|\n\t"; qr{ ^ (?xis: #ARGV ) $ }mx }' good stuff goes here
(?^mx: ^ (?xis: good |
stuff |
goes |
here ) $ )
Nifty, eh?
Interpolation is a little slower than joining a list. That said I've never known anyone to take it to this extreme.
You could use the Benchmark module to determine how much difference there is.
Also, you could ask this question over on http://perlmonks.org/. There are real gurus there who can probably give you the inner secrets much better than I can.
All of those approaches are fine.
Join can sometimes be more powerful than . concatentate, particularly when some of the things you are joining are arrays:
join "/", "~", #document_path_elements, $myDocument;
While recognizing that in all the examples I see here, there are no significant performance differences, a series of concatenation, whether with . or with double-quotish interpolation, is indeed going to be more memory-inefficient than a join, which precomputes the needed string buffer for the result instead of expanding it several times (potentially even needing to move the partial resutl to a new location each time).
I have a problem with the criticism I see leveled here; there are many right ways to speak perl, and this is certainly one of them.
Inconsistent indentation, on the other hand...
Starting in Perl 5.10, it is now possible to lexically scope the context variable $_, either explicitly as my $_; or in a given / when construct.
Has anyone found good uses of the lexical $_? Does it make any constructs simpler / safer / faster?
What about situations that it makes more complicated? Has the lexical $_ introduced any bugs into your code? (since control structures that write to $_ will use the lexical version if it is in scope, this can change the behavior of the code if it contains any subroutine calls (due to loss of dynamic scope))
In the end, I'd like to construct a list that clarifies when to use $_ as a lexical, as a global, or when it doesn't matter at all.
NB: as of perl5-5.24 these experimental features are no longer part of perl.
IMO, one great thing to come out of lexical $_ is the new _ prototype symbol.
This allows you to specify a subroutine so that it will take one scalar or if none is provided it will grab $_.
So instead of writing:
sub foo {
my $arg = #_ ? shift : $_;
# Do stuff with $_
}
I can write:
sub foo(_) {
my $arg = shift;
# Do stuff with $_ or first arg.
}
Not a big change, but it's just that much simpler when I want that behavior. Boilerplate removal is a good thing.
Of course, this has the knock on effect of changing the prototypes of several builtins (eg chr), which may break some code.
Overall, I welcome lexical $_. It gives me a tool I can use to limit accidental data munging and bizarre interactions between functions. If I decide to use $_ in the body of a function, by lexicalizing it, I can be sure that whatever code I call, $_ won't be modified in calling code.
Dynamic scope is interesting, but for the most part I want lexical scoping. Add to this the complications around $_. I've heard dire warnings about the inadvisability of simply doing local $_;--that it is best to use for ( $foo ) { } instead. Lexicalized $_ gives me what I want 99 times out of 100 when I have localized $_ by whatever means. Lexical $_ makes a great convenience and readability feature more robust.
The bulk of my work has had to work with perl 5.8, so I haven't had the joy of playing with lexical $_ in larger projects. However, it feels like this will go a long way to make the use of $_ safer, which is a good thing.
I once found an issue (bug would be way too strong of a word) that came up when I was playing around with the Inline module. This simple script:
use strict qw(vars subs);
for ('function') {
$_->();
}
sub function {
require Inline;
Inline->bind(C => <<'__CODE__');
void foo()
{
}
__CODE__
}
fails with a Modification of a read-only value attempted at /usr/lib/perl5/site_perl/5.10/Inline/C.pm line 380. error message. Deep in the internals of the Inline module is a subroutine that wanted to modify $_, leading to the error message above.
Using
for my $_ ('function') { ...
or otherwise declaring my $_ is a viable workaround to this issue.
(The Inline module was patched to fix this particular issue).
[ Rationale: A short additional answer with a quick summary for perl newcomers that may be passing by. When searching for "perl lexical topic" one can end up here.]
By now (2015) I suppose it is common knowledge that the introduction of lexical topic (my $_ and some related features) led to some difficult to detect at the outset unintended behaviors and so was marked as experimental and then entered into a deprecation stage.
Partial summary of #RT119315:
One suggestion was for something like use feature 'lextopic'; to make use of a new
lexical topic variable:
$^_.
Another point made was that an "implicit name for the topicalizing operator ... other than $_" would work best when combined with explicitly lexical functions (e.g. lexical map or lmap). Whether these approaches would somehow make it possible to salvage given/when is not clear. In the afterlife of the experimental and depreciation phases perhaps something may end up living on in the river of CPAN.
Haven't had any problems here, although I tend to follow somewhat of a "Don't ask, don't tell" policy when it comes to Perls magic. I.e. the routines are not usually expected to rely on their peers screwing with non lexical data as a side effect, nor letting them.
I've tested code against various 5.8 and 5.10 versions of perl, while using a 5.6 describing Camel for occasional reference. Haven't had any problems. Most of my stuff was originally done for perl 5.8.8.
Lately, I've decided to start using Perl::Critic more often on my code. After programming in Perl for close to 7 years now, I've been settled in with most of the Perl best practices for a long while, but I know that there is always room for improvement. One thing that has been bugging me though is the fact that Perl::Critic doesn't like the way I unpack #_ for subroutines. As an example:
sub my_way_to_unpack {
my $variable1 = shift #_;
my $variable2 = shift #_;
my $result = $variable1 + $variable2;
return $result;
}
This is how I've always done it, and, as its been discussed on both PerlMonks and Stack Overflow, its not necessarily evil either.
Changing the code snippet above to...
sub perl_critics_way_to_unpack {
my ($variable1, $variable2) = #_;
my $result = $variable1 + $variable2;
return $result;
}
...works too, but I find it harder to read. I've also read Damian Conway's book Perl Best Practices and I don't really understand how my preferred approach to unpacking falls under his suggestion to avoid using #_ directly, as Perl::Critic implies. I've always been under the impression that Conway was talking about nastiness such as:
sub not_unpacking {
my $result = $_[0] + $_[1];
return $result;
}
The above example is bad and hard to read, and I would never ever consider writing that in a piece of production code.
So in short, why does Perl::Critic consider my preferred way bad? Am I really committing a heinous crime unpacking by using shift?
Would this be something that people other than myself think should be brought up with the Perl::Critic maintainers?
The simple answer is that Perl::Critic is not following PBP here. The
book explicitly states that the shift idiom is not only acceptable, but
is actually preferred in some cases.
Running perlcritic with --verbose 11 explains the policies. It doesn't look like either of these explanations applies to you, though.
Always unpack #_ first at line 1, near
'sub xxx{ my $aaa= shift; my ($bbb,$ccc) = #_;}'.
Subroutines::RequireArgUnpacking (Severity: 4)
Subroutines that use `#_' directly instead of unpacking the arguments to
local variables first have two major problems. First, they are very hard
to read. If you're going to refer to your variables by number instead of
by name, you may as well be writing assembler code! Second, `#_'
contains aliases to the original variables! If you modify the contents
of a `#_' entry, then you are modifying the variable outside of your
subroutine. For example:
sub print_local_var_plus_one {
my ($var) = #_;
print ++$var;
}
sub print_var_plus_one {
print ++$_[0];
}
my $x = 2;
print_local_var_plus_one($x); # prints "3", $x is still 2
print_var_plus_one($x); # prints "3", $x is now 3 !
print $x; # prints "3"
This is spooky action-at-a-distance and is very hard to debug if it's
not intentional and well-documented (like `chop' or `chomp').
An exception is made for the usual delegation idiom
`$object->SUPER::something( #_ )'. Only `SUPER::' and `NEXT::' are
recognized (though this is configurable) and the argument list for the
delegate must consist only of `( #_ )'.
It's important to remember that a lot of the stuff in Perl Best Practices is just one guy's opinion on what looks the best or is the easiest to work with, and it doesn't matter if you do it another way. Damian says as much in the introductory text to the book. That's not to say it's all like that -- there are many things in there that are absolutely essential: using strict, for instance.
So as you write your code, you need to decide for yourself what your own best practices will be, and using PBP is as good a starting point as any. Then stay consistent with your own standards.
I try to follow most of the stuff in PBP, but Damian can have my subroutine-argument shifts and my unlesses when he pries them from my cold, dead fingertips.
As for Critic, you can choose which policies you want to enforce, and even create your own if they don't exist yet.
In some cases Perl::Critic cannot enforce PBP guidelines precisely, so it may enforce an approximation that attempts to match the spirit of Conway's guidelines. And it is entirely possible that we have misinterpreted or misapplied PBP. If you find something that doesn't smell right, please mail a bug report to bug-perl-critic#rt.cpan.org and we'll look into it right away.
Thanks,
-Jeff
I think you should generally avoid shift, if it is not really necessary!
Just ran into a code like this:
sub way {
my $file = shift;
if (!$file) {
$file = 'newfile';
}
my $target = shift;
my $options = shift;
}
If you start changing something in this code, there is a good chance you might accidantially change the order of the shifts or maybe skip one and everything goes southway. Furthermore it's hard to read - because you cannot be sure you really see all parameters for the sub, because some lines below might be another shift somewhere... And if you use some Regexes in between, they might replace the contents of $_ and weird stuff begins to happen...
A direct benefit of using the unpacking my (...) = #_ is you can just copy the (...) part and paste it where you call the method and have a nice signature :) you can even use the same variable-names beforehand and don't have to change a thing!
I think shift implies list operations where the length of the list is dynamic and you want to handle its elements one at a time or where you explicitly need a list without the first element. But if you just want to assign the whole list to x parameters, your code should say so with my (...) = #_; no one has to wonder.
I have a Perl codebase, and there are a lot of redundant functions and they are spread across many files.
Is there a convenient way to identify those redundant functions in the codebase?
Is there any simple tool that can verify my codebase for this?
You could use the B::Xref module to generate cross-reference reports.
I've run into this problem myself in the past. I've slapped together a quick little program that uses PPI to find subroutines. It normalizes the code a bit (whitespace normalized, comments removed) and reports any duplicates. Works reasonably well. PPI does all the heavy lifting.
You could make the normalization a little smarter by normalizing all variable names in each routine to $a, $b, $c and maybe doing something similar for strings. Depends on how aggressive you want to be.
#!perl
use strict;
use warnings;
use PPI;
my %Seen;
for my $file (#ARGV) {
my $doc = PPI::Document->new($file);
$doc->prune("PPI::Token::Comment"); # strip comments
my $subs = $doc->find('PPI::Statement::Sub');
for my $sub (#$subs) {
my $code = $sub->block;
$code =~ s/\s+/ /; # normalize whitespace
next if $code =~ /^{\s*}$/; # ignore empty routines
if( $Seen{$code} ) {
printf "%s in $file is a duplicate of $Seen{$code}\n", $sub->name;
}
else {
$Seen{$code} = sprintf "%s in $file", $sub->name;
}
}
}
It may not be convenient, but the best tool for this is your brain. Go through all the code and get an understanding of its interrelationships. Try to see the common patterns. Then, refactor!
I've tagged your question with "refactoring". You may find some interesting material on this site filed under that subject.
If you are on Linux you might use grep to help you make list all of the functions in your codebase. You will probably need to do what Ether suggests and really go through the code to understand it if you haven't already.
Here's an over-simplified example:
grep -r "sub " codebase/* > function_list
You can look for duplicates this way too. This idea may be less effective if you are using Perl's OOP capability.
It might also be worth mentioning NaturalDocs, a code documentation tool. This will help you going forward.
What is the historical reason to that last is called that in Perl rather than break as it is called in C?
The design of Perl was influenced by C (in addition to awk, sed and sh - see man page below), so there must have been some reasoning behind not going with the familiar C-style naming of break/last.
A bit of history from the Perl 1.000 (released 18 December, 1987) man page:
[Perl] combines (in the author's opinion, anyway) some of the best features of C, sed, awk, and sh, so people familiar with those languages should have little difficulty with it. (Language historians will also note some vestiges of csh, Pascal, and even BASIC|PLUS.)
The semantics of 'break' or 'last' are
defined by the language (in this case
Perl), not by you.
Why not think of 'last' as "this is
the last statement to run for the
loop".
It's always struck me as odd that the
'continue' statement in 'C' starts the
next pass of a loop. This is
definitely a strange use of the
concept of "continue". But it is the
semantics of 'C', so I accept it.
By trying to map particular
programming concepts into single
English words with existing meaning
there is always going to be some sort
of mismatching oddity
Source
Plus, Larry Wall is kinda weird. Have you seen his picture?
(source: wired.com)
I expect that this is because Perl was created by a linguist, not a computer scientist. In normal English usage, the concept of declaring that you have completed your final pass through a loop is more strongly connected to the word "last" ("this is the last pass") than to the word "break" ("break the loop"? "break out of the loop"? - it's not even clear how "break" is intended to relate to exiting the loop).
The term 'last' makes more sense when you remember that you can use it with more than just the immediate looping control. You can apply it to labeled blocks one or more levels above
the block it is in:
LINE: while( <> ) {
WORD: foreach ( split ) {
last LINE if /^__END__\z/;
...
}
}
It reads more naturally to say "last" in english when you read it as "last line if it matches ...".
Theres an additional reason you might want to consider:
Last does more than just loop control.
sub hello {
my ( $arg ) = #_;
scope: {
foo();
bar();
last if $arg > 4;
baz();
quux();
}
}
Last as such is a general flow control mechanism not limited to loops. While of course, you can generalise the above as a loop that runs at most 1 times, the absence of a loop to me indicates "Break? What are we breaking out of?"
Instead, I think of "last" as "Jump to the position of the last brace", which is for this purpose, more semantically sensible.
I was asking the same question to Damian Conway about say. Perl 6 will introduce say, which is nothing more than print that automatically adds a newline. My question was why not simply use echo, because this is what echo does in Bash (and probably elsewhere).
His answer was: echo is 33% longer than say.
He has a point there. :)
Because it goes to the last of the loop. And because Larry Wall was a weird guy.