Each time I build a Catalyst application I come to a point where the application gets painfully slow to (re)start, the delay is about 10 seconds. Today I figured the delay is caused by the following lines:
use lib '/home/zoul/opt/lib/perl/5.8';
use lib '/home/zoul/opt/share/perl/5.8';
use lib '/home/zoul/opt/lib/perl/5.8.8';
use lib '/home/zoul/opt/share/perl/5.8.8';
These lines are only needed on the server, since I haven’t got root access there and have my Perl modules installed under ~/opt. (I can’t use Apache’s SetEnv module, since the hoster does not support it. Therefore I have to enter the library paths into App.pm.) On my development machine that exhibits the gory delay the paths do not exist.
My questions: (1) Why do the lines cause so much delay, about 7 seconds? (2) What’s a nice way to solve this? Naive conditional use does not work:
if ($on_the_hosting_machine)
{
use lib '…';
}
I guess I could eval somehow, or is there a better way?
I do not do Catalyst, so I am not sure if this is going solve your problem, but you can try to do what is essentially what lib.pm does:
BEGIN {
if ( $on_the_hosting_machine ) {
unshift #INC, qw'
/home/zoul/opt/lib/perl/5.8
/home/zoul/opt/share/perl/5.8
/home/zoul/opt/lib/perl/5.8.8
/home/zoul/opt/share/perl/5.8.8
';
}
};
1) Every time you have a use or require statement, it searches through all the directories in lib in order. Each use lib does (at least) two stat calls.
use lib is just a wrapper for pushing things onto #LIB... but it also searches for the presence of an arch directory and pushes that on to #LIB if it exists, too.
You can reverse the change using the no lib pragma:
no lib ('/home/zoul/opt/lib/perl/5.8', '/home/zoul/opt/share/perl/5.8', '/home/zoul/opt/lib/perl/5.8.8', '/home/zoul/opt/share/perl/5.8.8');
Better yet, you could modify your dev environment to match production, or even just symlink those directories to the real locations for your dev setup.
Check out "A Timely Start" by Jean-Louis Leroy on Perl.com. He describes the same problem and a clever fix for it.
Related
I'm trying to profile a big application that contains a module creatively named DB. After running it under -d:NYTProf and calling nytprofhtml, both without any additional switches or related environment variables, I get usual nytprof directory with HTML output. However it seems that due to some internal logic any output related to my module DB is severely mangled. Just to make sure, DB is pure Perl.
Top and all subroutines list: while other function links point to relevant -pm-NN-line.html file, links to subs from DB point to entry script instead.
"line" link in "Source Code Files" section does point to DB-pm-NN-line.html and it does exists, but unlike all other files it doesn't have "Statements" table inside and "Line" table have absolutely no lines of code, only summary of calls.
Actually, here's a small example:
# main.pl
use lib '.';
use DB;
for (1..10) {
print DB::do_stuff();
}
# DB.pm
package DB;
sub do_stuff {
my $a = 1;
my $b = 2;
my $c = $a + $b;
return $c;
}
1;
Try running perl -d:NYTProf main.pl, then nytprofhtml and then inspect nytprof/DB-pm-8-line.html.
I don't know if it happens because NYTProf itself have internal module named DB or it handles modules starting with DB in some magical way - I've noticed output for functions from DBI looks somewhat different too.
Is there a way to change/disable this behavior short of renaming my DB module?
That's hardly an option
You don't really have an option. The special DB package and the associated Devel:: namespace are coded into the perl compiler / interpreter. Unless you want to do without any debugging facilities altogether and live in fear of any mysterious unquantifiable side effects then you must refactor your code to rename your proprietary DB library
Anyway, the name is very generic and that's exactly why it is expected to be encountered
On the contrary, Devel::NYTProf is bound to use the existing core package DB. It's precisely because it is a very generic identifier that an engineer should reject it as a choice for production code, which could be required to work with pre-existing third-party code at any point
a module creatively named DB
This belies your own support of the choice. DBhas been a core module since v5.6 in 2000, and anything already in CPAN should have been off the cards from the start. I feel certain that the clash of namespaces must been discovered before now. Please don't be someone else who just sweeps the issue under the carpet
Remember that, as it stands, any code you are now putting in the DB package is sharing space with everything that perl has already put there. It is astonishing that you are not experiencing strange and inexplicable symptoms already
I don't see that it's too big a task to fix this as long as you have a proper test suite for your 10MB of Perl code. If you don't, then hopefully you will at least not make the same mistakes again
I am using this software called Simple Agent Pro, and it primarily uses TCL code. I was wondering anybody familiar with TCL or Sapro would be kind enough to tell me how to import the modules into the .tel file for Sapro.
When I try this:
package require tclOO.h
The program stops working.
Any help would be appreciated.
I don't know Simple Agent Pro at all, but if you're doing a “guerilla install” of TclOO then you need a few things:
Make sure you're using Tcl 8.5 (see what package require Tcl returns).
If you're using 8.4 (note: 8.4 EOLed this month), TclOO will not work at all (and it cannot be backported).
If you're using 8.6, it already provides the TclOO package and you shouldn't need to fuss around with all this.
Do a build of TclOO and install it to a location you prefer.
This will require Tcl's internal source files; TclOO explicitly pokes its nose into places where most code shouldn't.
You probably don't need to have a custom build of 8.5; just the configured sources somewhere will do. (You might need to hack the configure scripts a little bit.)
Add the location that you installed TclOO to to the search path inside your Tcl 8.5 program.
lappend auto_path /the_dir/you_put/it_in
If you're using Windows, it's probably easiest to use forward slashes for this path anyway (this is a directory name that is always highly protected before it hits the OS, so that's OK).
Now you should be able to require/use the package.
package require TclOO
oo::class create Foo {
# etc.
}
Note that the case and exactly how you write it matters. The version you get ought to be at least 1.0 (earlier versions were for development only) which corresponds exactly with the API as supported in Tcl 8.6 (modulo a few things that require 8.6 for other reasons, such as being able to yield inside a method which only works in 8.6 because that's where yield was first defined).
You probably mean
package require TclOO
Case and other stuff is important there.
Next time you should also include the stack trace. If the program stops working, it should display that either as dialog or on stdout.
I find a module, that I want to change.
My problem have some features like this:
I want to add functionality and flexibility to this module.
Now this module solves tasks, but web-service, for what it was written, change API
And also, I want to use code of this module.
It is not my module
Fix some bugs
How i should be in this situation?
Inheriting from this module and add functionality and upload to CPAN?
Ask author about my modifications (and reload module)?
Something else?
There are various ways to modify a module as you use it, and I cover most of them in Mastering Perl.
As Dave Cross mentions, send fixes upstream or become part of that project. It sounds like you have the ambition to be a significant contributor. :)
Create a subclass to replace methods
Override or overload subroutines or methods
Wrap subroutines to modify or adapt either inputs or outputs (e.g. Hook::LexWrap)
Create a locally patched version, and store it separately from the main code so it doesn't disappear in an upgrade
For example, this is something I often do directly in program code while I wait for an upstream fix:
use Some::Module; # load the original first
BEGIN {
package Some::Module;
no warnings 'redefine';
if( $VERSION > 1.23 and $VERSION < 1.45 ) {
*broken = sub { ... fixed version ... };
}
}
This way, I have the fix even if the target module is upgraded.
I think that your a and b options are pretty much the best approach - although I'd probably do them the other way round.
Approach the module author with your suggestions. Read the module
documentation to find out how the author likes to be contacted. Some
like email, some like RT, other have more specific methods.
Authors tend to like suggestions better if they come with tests for
the new/changed code and patches that can be applied freely. Perhaps
the code is on Github. Can you fork it, make your changes and send
the author a pull request?
If the author is unresponsive, then consider either forking or
subclassing their code. In this case, naming is important as you
want people to be able to find your module as well as the original
one. You'll also want to carefully document the differences between
your version and the original one so that people can choose which
one they want.
I have been USEing .pm files willy-nilly in my programs without really getting into using packages unless really needed. In essence, I would just have common routines in a .pm and they would become part of main when use'd (the .pm would have no package or Exporter or the like... just a bunch of routines).
However, I came across a situation the other day which I think I know what happened and why, but I was hoping to get some coaching from the experts here on best practices and how to resolve this. In essence, should packages be used always? Should I "do" files when I just want common routines absorbed into main (or the parent module/package)? Is Exporter really the way to handle all of this?
Here's example code of what I came across (I won't post the original code as it's thousands of lines... this is just the essence of the problem).
Pgm1.pl:
use PM1;
use PM2;
print "main\n";
&pm2sub1;
&pm1sub1;
PM1.pm:
package PM1;
require Exporter;
#ISA=qw(Exporter);
#EXPORT=qw(pm1sub1);
use Data::Dump 'dump';
use PM2;
&pm2sub1;
sub pm1sub1 {
print "pm1sub1 from caller ",dump(caller()),"\n";
&pm2sub1;
}
1;
PM2.pm:
use Data::Dump 'dump';
sub pm2sub1 {
print "pm2sub1 from caller ",dump(caller()),"\n";
}
1;
In essence, I'd been use'ing PM2.pm for some time with its &pm2sub1 subroutine. Then I wrote PM1.pm at some point and it needed PM2.pm's routines as well. However, in doing it like this, PM2.pm's modules got absorbed into PM2.pm's package and then Pgm1.pl couldn't do the same since PM2.pm had already been use'd.
This code will produce
Undefined subroutine &main::pm2sub1 called at E:\Scripts\PackageFun\Pgm1.pl line 4.
pm2sub1 from caller ("PM1", "PM1.pm", 7)
main
However, when I swap the use statements like so in Pgm1.pl
use PM2;
use PM1;
print "main\n";
&pm2sub1;
&pm1sub1;
... perl will allow PM2.pm's modules into main, but then not into PM1.pm's package:
Undefined subroutine &PM1::pm2sub1 called at PM1.pm line 7.
Compilation failed in require at E:\Scripts\PackageFun\Pgm1.pl line 2.
BEGIN failed--compilation aborted at E:\Scripts\PackageFun\Pgm1.pl line 2.
So, I think I can fix this by getting religious about packages and Exporter in all my modules. Trouble is, PM2.pm is already used in a great number of other programs, so it would be a ton of regression testing to make sure I didn't break anything.
Ideas?
See my answer to What is the difference between library files and modules?.
Only use require (and thus use) for modules (files with package, usually .pm). For "libraries" (files without package, usually .pl), use do.
Better yet, only use modules!
use will not load same file more than once. It will, however, call target package's import sub every time. You should format your PM2 as proper package, so use can find its import and export function to requestor's namespace from there.
(Or you could sneak your import function into proper package by fully qualifying its name, but don't do that.)
You're just asking for trouble arranging your code like this. Give each module a package name (namespace), then fully qualify calls to its functions, e.g. PM2::sub1() to call sub1 in package PM2. You are already naming the functions with the package name on them (pm2sub1); it is two extra characters (::) to do it the right way and then you don't need to bother with Exporter either.
I'm getting ready to try to deploy some code to multiple machines. As far as I know, using a Makefile.pm to track dependencies is the best way to ensure they are installed everywhere. The problem I have is I'm not sure our Makefile.pm has been updated as this application has passed through a few different developers.
Is there any way to automatically parse through either my source or a few full runs of my program to determine exactly what versions of what modules my application is depending on? On top of that, is there any way to filter it based on CPAN packages? (So that I only depend on Moose instead of every single module that comes with Moose.)
A third related question is, if you depend on a version of a module that is not the latest, what is the best way to have someone else install it? Should I start including entire localized Perl installations with my application?
Just to be clear - you can not generically get a list of modules that the app depends on by code analysis alone. E.g. if your apps does eval { require $module; $module->import() }, where $module is passed via command line, then this can ONLY be detected by actually running the specific command line version with ALL the module values.
If you do wish to do this, you can figure out every module used by a combination of runs via:
Devel::Cover. Coverage reports would list 100% of modules used. But you don't get version #s.
Print %INC at every single possible exit point in the code as slu's answer said. This should probably be done in END{} block as well as __DIE__ handler to cover all possible exit points, and even then may be not fully 100% covering in generic case if somewhere within the program your __DIE__ handler gets overwritten.
Devel::Modlist (also mentioned by slu's answer) - the downside compared to Devel::Cover is that it does NOT seem to be able to aggregate a database across multiple sample runs like Devel::Cover does. On the plus side, it's purpose-built, so has a lot of very useful options (CPAN paths, versions).
Please note that the other module (Module::ScanDeps) does NOT seem to allow you to do runtime analysis based on arbitrary command line arguments (e.g. it seems at first glance to only allow you to execute the program with no arguments) and if that's true, is inferior to all the above 3 methods for any code that may possibly load modules dynamically.
Module::ScanDeps - Recursively scan Perl code for dependencies
Does both static and runtime scanning. Just modules, I don't know of any exact way of verifying what versions from what distributions. You could get old packages from BackPan, or just package your entire chain of local dependencies up with PAR.
You could look at %INC, see http://www.perlmonks.org/?node_id=681911 which also mentions Devel::Modlist
I would definitely use Devel::TraceUse, which also shows a tree of the modules, so it's easy to guess where they are being loaded.