What different methods can be used to compile Perl differently, in such a way that would actually improve the performance of Perl scripts run on that machine? Though outdated, http://dan.corlan.net/bench.html, seems to indicate that different performance results can be achieved by compiling things differently. Is that the case, or am I misunderstanding something?
Are there any performance gains from not using a default Perl package (or one that is installed by default in Linux)?
I never measured this but I was led to believe that perl compiled without threads is 10% faster. I am not sure if this is "on average" or on "certain operations" or if it is true at all.
The perl that comes with most (or all?) Linux distributions was compiled with threads.
Based on this, if you build your own perl without threads it should be faster. Incidentally this is what you get when you compile it with the default flags.
Steffen Schwingon has been doing some performance measurements and wrote about them here:
http://blogs.perl.org/users/steffen_schwigon/2012/01/perlformance.html
It would be nice if made some measurements and showed some results.
Related
This question already has an answer here:
Using Perl modules vs. using system() calls
(1 answer)
Closed 9 years ago.
On occasion I see people calling the system grep from Perl (and other scripting languages for that matter) instead of using the built-in language facilities/libraries to parse files. I would like to encourage people to use the built-in facilities and I want to solicit some reasons as to why it is good practice to use the built-in tools. I can think of some such as
Using libraries/language facilities is faster. Performance suffers due to the overhead of executing external commands.
Sticking to language facilities is more portable.
any other reasons?
On the other side of the coin, are there ever reasons to favour using system commands instead of the built-in language facilities? On that note, if a Perl script is basically only calling external commands (e.g. custom utilities without libraries), might it be better just to make a shell script of it?
Actually, when it matters, a specialized tool can be faster.
The real gains of keeping the work in Perl are:
Portability (even between machines with the same OS).
Ease of error detection.
Flexibility in handling of errors.
Greater customizability/flexibility.
Fewer "moving parts". (Are you sure you correctly escaped everything and setup the environment correctly?)
Less expertise needed. (You don't need to know both Perl and the external tools (and their ports) to code and maintain the program.)
On that note, if a Perl script is basically only calling external commands (e.g. custom utilities without libraries), might it be better just to make a shell script of it?
Possibly. You can configure some shells to exit if any program returns an unsuccessful error code. This can make some scripts quite robust. For example, I have a couple of bash scripts featuring the line
trap 'e=$? ; echo "Error." ; exit $e' ERR
"On the other side of the coin, are there ever reasons to favour using system commands instead of the built-in language facilities? On that note, if a Perl script is basically only calling external commands (e.g. custom utilities without libraries), might it be better just to make a shell script of it?"
Risking the wrath of Perl hardliners here. But for me there is an easy reason to use system grep instead of perl grep: I know its syntax.
Same reason to use a Perl script instead of a bash script: I know how to do stuff in Perl and never bothered with bash script syntax.
And as we are talking scripts here, my main concern is getting it done fast and reliable (and readable). At work i do not have to bother with portability as all production is done on the very same system, down to the same software versions of everything for the whole product lifespan.
At home i do not have to care about lifetime or whatever either as the script most likely is single-purpose.
And in neither case i care about performance or software security as i would be using C++ or something else for commercial software or in time or memory limited scenarios.
edit: Not saying these reasons would apply to anyone, or even anyone else. But while in reality i know how to use Perls grep, i really have no idea how to write a bash script and most likely never will. Just putting a few lines in Perl is always faster for me.
Using external tools lead to do more error.
Moreover you have you to parse the results (if any) of the external command, which is an other source of error.
No need to say that it is bad in terms of security.
I'm using the following command to test my perl code:
perl -MB::Lint::StrictOO -MO=Lint,all,oo -M-circular::require -M-indirect -Mwarnings::method -Mwarnings::unused -c $file
On a system with a perl version less than 5.10 I am also using uninit.
I am also using Perl::Critic and Perl::Tidy and have set up the appropriate rc files to my liking.
These modules have done a great job in helping me break some bad habits I learned when first learning perl.
Are there any more modules or pragmas that will kick me back on the straight and narrow when I mess up?
Using tests, and the Test::* family of modules and some good books have been pointed out. This new information has caused me to reconsider some assumptions about the relationship between testing and code skill building. These are all appreciated and already being researched and put to use.
It seems to me that these are two separate parts of a whole. 'perl -c', Perl::Critic and Perl::Tidy all help during the process of writing code and before execution of code. Devel::Cover, Devel::NYTProf and Tests happen during and after execution of code.
Good development dictates an iterative process, so tests will be run, and code developed over and over, but we still have this separation.
It appears to me that the focus in the answers have been on the 'during and after execution' of code. Again, this is very appreciated. Can I assume that I have the 'writing and pre-execution' part down pretty well then? At least, insomuch as the pragmas, modules and utilities are concerned.
I'm a little worried that you're using Perl 5.9. For two reasons.
Firstly it's a little old. 5.9.0 was released in 2003 and 5.9.5 (the last version in the 5.9.x series) was released in 2007. There have been several high quality versions of Perl since then.
Secondly (and most importantly), 5.9 is an unstable development version of Perl. 5.9 is basically the series of experiments that eventually led to Perl 5.10.0. The only reason to use it is to test that 5.10 will be a stable version of Perl. No-one should be using it at all now.
You don't appear to be testing your code, merely checking that it will compile. I suggest that you look at Test::More (which makes writing actual tests nice and easy), Test::Class (which makes dealing with very large test suites easier), and Devel::Cover (to see which bits of your code are covered by your tests and which aren't).
I would like to know techniques (coding, libraries, configurations) for measuring the duration of execution of CGI Perl code at various stages:
starting up the Perl interpreter
beginning running the Perl code
loading in local Perl .pm modules for routines
completed running the code
I'm particularly interested in 3 and 4, I don't believe there is much I can do about 1) or 2) as I wouldn't want to try to optimise the Perl interpreter, the only thing I can do here is upgrade the hardware to a faster machine and/or use mod_perl instead of classic CGI.
With 3) loading the local Perl modules I would like to measure how long it takes but I'm not sure how to code this as I don't know (or am not sure) how to get code to execute before loading these modules. If I did know, then I would record the time before they load, then record the time after they have loaded and calculate the difference.
4) should be the easiest to obtain as I would record the time (in a variable) at start of execution and then at end.
I've done a search at stackoverflow.com and found:
How can I speed up my Perl program? - which is what I expect to be using at some point. BUT I need to prove the reduced time (i.e. speed improvement, so I need to be able to measure in the first place). The tool http://search.cpan.org/dist/Devel-NYTProf looks useful for profiling my source code but I'm not sure it covers 3) loading of modules
Does Perl language aim at producing fast programs at runtime? - more of a verbose discussion rather than succinct answers, but a good read later when time
Google search results included:
http://www.testingreflections.com/node/view/3622 - not enough information here
You can reduce 1) with FastCGI. It will reduce stages 2) and 3) too.
For measuring 3) you can use BEGIN blocks. Example:
use Benchmark ':hireswallclock';
my ($t0,$t1);
BEGIN {$t0 = Benchmark->new;}
use DBIx::Class;
BEGIN {$t1 = Benchmark->new;}
print "the loading took:",timestr(timediff($t1, $t0)),"\n";
Deve::NYTProf will help you with 4). Also there are specific modules like Template::Timer, CGI::Application::Plugin::DevPopup::Timing, DBIx::Class::Storage::Statistics.
In addition to FastCGI, there is also mod_perl and more importantly PSGI. With PSGI you can decouple your app from concrete webserver backend.
Is there a way to save a compiled version of my perl scripts?
Or a way to do a JavaScript style compile where you just remove comments, whitespace, etc?
You're trying to optimize in the wrong place. If you are running scripts in a web/cgi environment, there is no need to take a compile hit every time the script is executed. The scripts should be running persistently, which you can do with Apache mod/perl, FastCGI, or a number of newer technologies and frameworks such as Plack and Catalyst. If you are more specific about your needs, you will discover that there are a number of options available to you.
Do you realize that Javascript is minified to save bandwidth, not startup time or runtime? And that the practice of minifying Javascript started in the times of dialup connections?
Sure, there was a time where interpreted programs were often minified like that, but back then typical CPUs were Z80s and 8086's running at 4-8 MHz, and using loads of cycles to execute a single instruction. To show: my Athlon XP-M 2400 is ~10,000 times faster than my 8MHz 8086 for CPU-bound programs.
Try the perl compiler, to C B::C or to B::Bytecode (similar to python pyc).
http://search.cpan.org/dist/B-C/perlcompile.pod
You could use PPI to strip out comments and POD.
Perl::Squish is the "minifier" you're looking for. Caveat: It's not going to help you at all. You're trying to optimize on the wrong end.
If you're doing this for fun you might want to check out parrot vm
If not.. see my comment ;)
Other than the purely obvious: "It translates Perl to C."; are there any real world uses (a.k.a. hacks) for the Perl compiler's optimized C translation backend, B::CC?
Not really. It means you can convert a (small) Perl script into a (big) C program, which will be much harder for the recipient to reverse engineer. In some paranoid circles, this might be accounted an advantage (for example, if your Perl code is embarrassingly bad and you'd rather conceal that fact from your paying customers). But mostly it is of limited to negative value.
Compiling a Perl program to an optree, which can then be executed, can take a while sometimes. You can safe some of that time by using perlcc with any of its backends. That'll, in one way or another, serialise the compiled optree and make loading it later, when executing your compiled binary, somewhat faster. I can see that being useful in, for example, CGI environments, for which, however, much better alternatives of avoiding startup costs are available.
Contrary to popular believe, perlcc doesn't make it very hard to reverse-engineer the resulting binary, as discussed in How can I reverse-engineer a Perl program that has been compiled with perlcc?