Is there a way to save a compiled version of my perl scripts?
Or a way to do a JavaScript style compile where you just remove comments, whitespace, etc?
You're trying to optimize in the wrong place. If you are running scripts in a web/cgi environment, there is no need to take a compile hit every time the script is executed. The scripts should be running persistently, which you can do with Apache mod/perl, FastCGI, or a number of newer technologies and frameworks such as Plack and Catalyst. If you are more specific about your needs, you will discover that there are a number of options available to you.
Do you realize that Javascript is minified to save bandwidth, not startup time or runtime? And that the practice of minifying Javascript started in the times of dialup connections?
Sure, there was a time where interpreted programs were often minified like that, but back then typical CPUs were Z80s and 8086's running at 4-8 MHz, and using loads of cycles to execute a single instruction. To show: my Athlon XP-M 2400 is ~10,000 times faster than my 8MHz 8086 for CPU-bound programs.
Try the perl compiler, to C B::C or to B::Bytecode (similar to python pyc).
http://search.cpan.org/dist/B-C/perlcompile.pod
You could use PPI to strip out comments and POD.
Perl::Squish is the "minifier" you're looking for. Caveat: It's not going to help you at all. You're trying to optimize on the wrong end.
If you're doing this for fun you might want to check out parrot vm
If not.. see my comment ;)
Related
I'm using the following command to test my perl code:
perl -MB::Lint::StrictOO -MO=Lint,all,oo -M-circular::require -M-indirect -Mwarnings::method -Mwarnings::unused -c $file
On a system with a perl version less than 5.10 I am also using uninit.
I am also using Perl::Critic and Perl::Tidy and have set up the appropriate rc files to my liking.
These modules have done a great job in helping me break some bad habits I learned when first learning perl.
Are there any more modules or pragmas that will kick me back on the straight and narrow when I mess up?
Using tests, and the Test::* family of modules and some good books have been pointed out. This new information has caused me to reconsider some assumptions about the relationship between testing and code skill building. These are all appreciated and already being researched and put to use.
It seems to me that these are two separate parts of a whole. 'perl -c', Perl::Critic and Perl::Tidy all help during the process of writing code and before execution of code. Devel::Cover, Devel::NYTProf and Tests happen during and after execution of code.
Good development dictates an iterative process, so tests will be run, and code developed over and over, but we still have this separation.
It appears to me that the focus in the answers have been on the 'during and after execution' of code. Again, this is very appreciated. Can I assume that I have the 'writing and pre-execution' part down pretty well then? At least, insomuch as the pragmas, modules and utilities are concerned.
I'm a little worried that you're using Perl 5.9. For two reasons.
Firstly it's a little old. 5.9.0 was released in 2003 and 5.9.5 (the last version in the 5.9.x series) was released in 2007. There have been several high quality versions of Perl since then.
Secondly (and most importantly), 5.9 is an unstable development version of Perl. 5.9 is basically the series of experiments that eventually led to Perl 5.10.0. The only reason to use it is to test that 5.10 will be a stable version of Perl. No-one should be using it at all now.
You don't appear to be testing your code, merely checking that it will compile. I suggest that you look at Test::More (which makes writing actual tests nice and easy), Test::Class (which makes dealing with very large test suites easier), and Devel::Cover (to see which bits of your code are covered by your tests and which aren't).
What different methods can be used to compile Perl differently, in such a way that would actually improve the performance of Perl scripts run on that machine? Though outdated, http://dan.corlan.net/bench.html, seems to indicate that different performance results can be achieved by compiling things differently. Is that the case, or am I misunderstanding something?
Are there any performance gains from not using a default Perl package (or one that is installed by default in Linux)?
I never measured this but I was led to believe that perl compiled without threads is 10% faster. I am not sure if this is "on average" or on "certain operations" or if it is true at all.
The perl that comes with most (or all?) Linux distributions was compiled with threads.
Based on this, if you build your own perl without threads it should be faster. Incidentally this is what you get when you compile it with the default flags.
Steffen Schwingon has been doing some performance measurements and wrote about them here:
http://blogs.perl.org/users/steffen_schwigon/2012/01/perlformance.html
It would be nice if made some measurements and showed some results.
I have a bunch of scripts that I wrote at times when I did not realize how use v1.2.3; can be useful. So some of them may be using features from later versions of perl, some of them may be OK with, say, perl 5.8.
Now I would like to get that into some order and add proper uses where there is need for them, just to be able to sleep better. :-)
How should I do that? Is there any tool that could help me make an educated guess?
Perl::MinimumVersion
Find a minimum required version of perl for Perl code
The most reliable way is 1) to write a decent test suite, then 2) to run your tests using each version of Perl.
You've surely already done the first part (!), and the second part is actually pretty easy to do using perlbrew.
I am trying to find some online tool for compressing and optimizing my perl. Is there any benefit to removal of (at a minimum) whitespace and comments from server side cgi?
Perl source undergoes a compilation phase, it is not directly interpreted. Perl code is executed at the server, not delivered to the client. Unlike JavaScript, there is no benefit from minimisation.
If you want to optimise, measure first where the bottle-neck is. I presume that switching from CGI to a persistent technology will give you a big pay-off.
Related:
How can I compile my Perl script so to reduce startup time?
How can I reduce Perl CGI script start-up time?
Edit:
You mention in a comment that you deploy on Apache httpd. To reduce start-up time without changing existing code, install mod_perl2 and run your CGI programs with the perl-script handler. In the long term, switch over your code base from CGI to PSGI and deploy on Plack::Handler::Apache2, or preferably, if you also have an FastCGI adapter for the web server, Plack::Handler::Net::FastCGI.
Perltidy is a free pretty-printer with a lot of options. Some of them might do what you want in some way, remove whitespace for instance. It's not a minifier, but good to know nevertheless, I'd recommend to add it to your toolchest
SOLVED see Edit 2
Hello,
I've been writing a Perl program to handle automatic upgrading of local (proprietary) programs (for the company I work for).
Basically, it runs via cron, and unfortunately has a memory leak (or something similar). The problem is that the leak only happens when I'm not looking (aka when run via cron, not via command line).
My code does not contain any circular (or other) references, so the commonly cited tools will not help me (Devel::Cycle, Devel::Peek).
How would I go about figuring out what is using so much memory that the kernel kills it?
Basically, the code SFTPs into a server (using ```sftp...`` `), calls OpenSSL to verify the file, and then SFTPs more if more files are needed, and installs them (untars them).
I have seen delays (~15 sec) before the first SFTP session, but it has never used so much memory as to be killed (in my presence).
If I can't sort this out, I'll need to re-write in a different language, and that will take precious time.
Edit: The following message is printed out by the kernel which led me to believe it was a memory leak:
[100023.123] Out of memory: kill process 9568 (update.pl) score 325406 or a child
[100023.123] Killed Process 9568 (update.pl)
I don't believe it is an issue with cron because of the stalling (for ~15 sec, sometimes) when running it via the command-line. Also, there are no environmental variables used (at least by what I've written, maybe underlying things do?)
Edit 2: I found the issue myself, with help from the below comment by mobrule (in response to this question). It turns out that the script was called from a crontab of a user (non-root) just once a day and that (non-root privs) caused a special infinite loop situation.
Sorry guys, I feel kinda stupid for not finding this before, but thanks.
mobrule, if you submit your comment as an answer, I will accept it as it lead to me finding the problem.
End Edits
Thanks,
Brian
P.S. I may be able to post small snippets of code, but not the whole thing due to company policy.
You could try using Devel::Size to profile some of your objects. e.g. in the main:: scope (the .pl file itself), do something like this:
use Devel::Size qw(total_size);
foreach my $varname (qw(varname1 varname2 ))
{
print "size used for variable $varname: " . total_size($$varname) . "\n";
}
Compare the actual size used to what you think is a reasonable value for each object. Something suspicious might pop out immediately (e.g. a cache that is massively bloated beyond anything that sounds reasonable).
Other things to try:
Eliminate bits of functionality one at a time to see if suddenly things get a lot better; I'd start with the use of any external libraries
Is the bad behaviour localized to just one particular machine, or one particular operating system? Move the program to other systems to see how its behaviour changes.
(In a separate installation) try upgrading to the latest Perl (5.10.1), and also upgrade all your CPAN modules
How do you know that it's a memory leak? I can think of many other reasons why the OS would kill a program.
The first question I would ask is "Does this program always work correctly from the command line?". If the answer is "No" then I'd fix these issues first.
On the other hand if the answer is "Yes", I would investigate all the differences between having the program executed under cron and from the command line to find out why it is misbehaving.
If it is run by cron, that shouldn't it die after iteration? If that is the case, hard for me to see how a memory leak would be a big deal...
Are you sure it is the script itself, and not the child processes that are using the memory? Perhaps it ends up creating a real lot of ssh sessions , instead of doing a bunch of stuff in one session?