I have a Perl program, that needs to use packages (that I also write). Some of those packages are only chosen in Runtime (based on some environment variable). I don't want to put in my code a "use" line for all of those packages, of course, but only one "use" line, based on this variable, something like:
use $ENV{a};
Unfortunately, this doesn't work, of course. Any ideas on how to do this?
Thanks in advance,
Oren
eval "require $ENV{a}";
"use" doesn't work well here because it only imports in the context of the eval.
As #Manni said, actually, it's better to use require. Quoting from man perlfunc:
If EXPR is a bareword, the require assumes a ".pm" extension and
replaces "::" with "/" in the filename for you, to make it easy to
load standard modules. This form of loading of modules does not
risk altering your namespace.
In other words, if you try this:
require Foo::Bar; # a splendid bareword
The require function will actually look for the "Foo/Bar.pm" file
in the directories specified in the #INC array.
But if you try this:
$class = 'Foo::Bar';
require $class; # $class is not a bareword
#or
require "Foo::Bar"; # not a bareword because of the ""
The require function will look for the "Foo::Bar" file in the #INC
array and will complain about not finding "Foo::Bar" there. In this
case you can do:
eval "require $class";
"use" Statements are run at compilation time, not at run time. You will need to require your modules instead:
my $module = "Foo::Bar";
eval "require $module";
I would use UNIVERSAL::require. It has both require and use methods to use a package. The use method will also call import for the package.
use UNIVERSAL::require;
$ENV{a}->use or die 'Could not import package: ' . $#;
You probably want to use require instead of use if you don't want it to happen at compile time, and then manually import any symbols you might need. See this link to the Perl Cookbook (from Google Books) for a good discussion of methods you can use to achieve what you want.
I think that a module loaded in runtime can be a Plugin. I have this kind of problem, having specific modules to some cases that are loaded in run time as plugins with Module::Pluggable.
Maybe you need to change the logic of your modules, but it works and scale very well (my app started with four modules and now have twenty and it's growing).
How about using the core module Module::Load
With your example:
use Module::Load;
load $ENV{a};
"Module::Load - runtime require of both modules and files"
"load eliminates the need to know whether you are trying to require either a file or a module."
If it fails it will die with something of the like "Can't locate xxx in #INC (#INC contains: ...".
Many years later, eval "use $mod_name"; seems to work just fine (as of at least 5.8.8).
The advantage over eval "require $mod_name"; is that the loaded module's default exports are automatically imported; in other words:
eval "use $mod_name";
is the shorter equivalent of
eval "require $mod_name"; $mod_name->import();
Here's a test command, which passes the name of the module via env. variable mod_name, loads and imports the module, then uses an imported function (assumes a POSIX-like shell):
$ mod_name='Data::Dumper' perl -e 'eval "use $ENV{mod_name}"; print Dumper("hi!")'
$VAR1 = 'hi!';
I may be missing subtleties; if so, please let me know.
Related
I'm getting an "undefined subroutine" for sub2 in the code below but not for sub1.
This is the perl script (try.pl)...
#!/usr/bin/env perl
use strict;
use IO::CaptureOutput qw(capture_exec_combined);
use FindBin qw($Bin);
use lib "$Bin";
use try_common;
print "Running try.pl\n";
sub1("echo \"in sub1\"");
sub2("echo \"in sub2\"");
exit;
sub sub1 {
(my $cmd) = #_;
print "Executing... \"${cmd}\"\n";
my ($stdouterr, $success, $exit_code) = capture_exec_combined($cmd);
print "${stdouterr}\n";
return;
}
This is try_common.pm...
#! /usr/bin/env perl
use strict;
use IO::CaptureOutput qw(capture_exec_combined);
package try_common;
use Exporter;
our #ISA = qw(Exporter);
our #EXPORT = qw(
sub2
);
sub sub2 {
(my $cmd) = #_;
print "Executing... \"${cmd}\"\n";
my ($stdouterr, $success, $exit_code) = capture_exec_combined($cmd);
print "${stdouterr}\n";
return;
}
1;
When I run try.pl I get...
% ./try.pl
Running try.pl
Executing... "echo "in sub1""
in sub1
Executing... "echo "in sub2""
Undefined subroutine &try_common::capture_exec_combined called at
/home/me/PERL/try_common.pm line 20.
This looks like some kind of scoping issue because if I cut/paste the "use IO::CaptureOutput qw(capture_exec_combined);" as the first line of sub2, it works. This is not necessary in the try.pl (it runs sub1 OK), but a problem in the perl module. Hmmmm....
Thanks in Advance for any help!
You imported capture_exec_combined by the use clause before declaring the package, so it was imported into the main package, not the try_common. Move the package declaration further up.
You should take a look at the perlmod document to understand how modules work. In short:
When you use package A (in Perl 5), you change the namespace of the following code to A, and all global symbol (e.g. subroutine) definitions after that point will go into that package. Subroutines inside a scope need not be exported and may be used preceded by their scope name: A::function. This you seem to have found.
Perl uses package as a way to create modules and split code in different files, but also as the basis for its object orientation features.
Most of the times, modules are handled by a special core module called Exporter. See Exporter. This module uses some variables to know what to do, like #EXPORT, #EXPORT_OK or #ISA. The first defines the names that should be exported by default when you include the module with use Module. The second defines the names that may be exported (but need to be mentioned with use Module qw(name1 name2). The last tells in an object oriented fashion what your module is. If you don't care about object orientation, your module typically "is a" Exporter.
Also, as stated in another answer, when you define a module, the package module declaration should be the first thing to be in the file so anything after it will be under that scope.
I hate when I make this mistake although I don't make it much anymore. There are two habits you can develop:
Most likely, make the entire file the package. The first lines will be the package statement and no other package statements show up in the file.
Or, use the new PACKAGE BLOCK syntax and put everything for that package inside the block. I do this for small classes that I might need only locally:
package Foo {
# everything including use statements go in this block
}
I think I figured it out. If, in the perl module, I prefix the "capture_exec_combined" with "::", it works.
Still, why isn't this needed in the main, try.pl ?
Consider the following script p.pl:
use strict;
use warnings;
use AA;
BB::bfunc();
where the file AA.pm is:
package AA;
use BB;
1;
and the file BB.pm is:
package BB;
sub bfunc {
print "Running bfunc..\n";
}
1;
Running p.pl gives output (with no warnings or errors):
Running bfunc..
Q: Why is it possible to call BB::bfunc() from p.pl even though there is no use BB; in p.pl? Isn't this odd behavior? Or are there situation where this could be useful?
(To me, it seems like this behavior only presents an information leak to another package and violates the data hiding principle.. Leading to programs that are difficult to maintain.. )
You're not polluting a namespace, because the function within BB isn't being 'imported' into your existing namespace.
They are separate, and may be referenced autonomously.
If you're making a module, then usually you'll define via Exporter two lists:
#EXPORT and #EXPORT_OK.
The former is the list of things that should be imported when you use the package. The latter is the things that you can explicity import via:
use MyPackage qw ( some_func );
You can also define package variables in your local namespace via our and reference them via $main.
our $fish = "haddock";
print $main::fish;
When you do this, you're explicitly referencing the main namespace. When you use a module, then you cause perl to go and look for it, and include it in your %INC. I then 'knows about' that namespace - because it must in order for the dependencies to resolve.
But this isn't namespace pollution, because it doesn't include anything in your namespace until your ask.
This might make a bit more sense if you have multiple packages within the same program:
use strict;
use warnings;
package CC;
our $package_var = "Blong";
sub do_something {
print $package_var,"\n";
}
package main;
use Data::Dumper;
our $package_var = "flonk";
print Dumper $package_var;
print Dumper $CC::package_var;
Each package is it's own namespace, but you can 'poke' things in another. perl will also let you do this with object - poking at the innards of instantiated objects or indeed "patch" them.
That's quite powerful, but I'd generally suggest Really Bad Style.
While it's good practice to use or require every dependency that you are planning to access (tried to avoid use here), you don't have to do that.
As long as you use full package names, that is fine. The important part is that Perl knows about the namespaces. If it does not, it will fail.
When you use something, that is equivalent to:
BEGIN {
require Foo::Bar;
Foo::Bar->import();
}
The require will take the Foo::Bar and convert it to a path according to the operating system's conventions. On Linux, it will try to find Foo/Bar.pm somewhere inside #INC. It will then load that file and make a note in %INC that it loaded the file.
Now Perl knows about that namespace. In case of the use it might import something into your own namespace. But it will always be available from everywhere after that as long as you use the full name. Just the same, stuff that you have in your main script.pl would be available inside of packages by saying main::frobnicate(). (Please don't do that!)
It's also not uncommon to bundle several namespaces/packages in one .pm module file. There are quite a few big names on CPAN that do it, like XML::Twig.
If you do that, and don't import anything, the only way to get to the stuff under the different namespaces is by using the full name.
As you can see, this is not polluting at all.
First of all, I think somebody needs to rewrite my question, I know what I am asking, not how I should ask it precisely.
Assume I have some local module LOCAL::Commons and it has one subroutine called globalTrim. I am testing that subroutine below, and while LOCAL::Commons is installed under /usr/local/lib/perl, the module I am testing below is located in directory /home/me/perl/LOCAL/. See how I am using use lib ... to make sure I am not using LOCAL::Commons located in /usr/local/lib/perl (I have this directory on my path).
Now, if LOCAL::Commons is using another local module LOCAL::Cool (that is, not from cpan), and I have also made some changes to that module, how can I make sure my tests are using the correct modules? That is, I want LOCAL::Commons to use /home/me/perl/LOCAL/Cool and not /usr/local/lib/perl/LOCAL/Cool.
#!/usr/bin/perl
# test_local_commons.pl
# directory: /home/me/perl
use strict;
use warnings;
use Test::More 'no_plan';
use File::Temp qw( tempfile tempdir );
use Cwd qw();
use lib Cwd::abs_path();
# Testing
use LOCAL::Commons qw ( globalTrim );
sub newTest($) {
my $name = shift;
print "---------------------------------------------------\n";
print $name, "\n";
print "---------------------------------------------------\n";
}
sub testTraverse {
is(globalTrim(" - stackoverflow - ), "-stackoverflow-", "Passed" );
}
newTest "testTraverse"; testTraverse ();
If you run this like so:
perl -I/home/me/perl test_local_commons.pl
It should ensure your /home/me/perl version is checked first
After you load the files you want, put this:
die 'Included PRODUCTION Module!!!'
if grep { m{/usr/local/lib/perl/LOCAL/Cool} } values %INC
;
Of course you could die for any module you wanted.
For example, we've got this tool at work that works with Activestate's PerlEZ.dll. We don't want to deploy it with Perl installed. But we need to use some libraries, which we hide elsewhere. When I'm testing my code I include a module from our hidden path. It blows up anytime it sees a standard library path in %INC. We want to make sure it almost everything from the #INC hook and the special libraries.
I suggest using Perlbrew to make a completely separate Perl installation for testing. That way you control the testing environment and don't really have to worry about it.
use lib has process scope -- that is, it changes the value #main::INC with global visibility, just as a non-localized assignment to #INC would. That means that any modules that you load after saying use lib will check the include path you've set.
As to what path use lib sets, it explicitly adds to the front of #INC so that later calls to use lib will be considered earlier when searching for modules.
The upshot of which is that it looks to me like your code sample will just work. Note that I'm going to discourage you from checking that you're using the under-development version in your test. That would cause your test to fail based on something unrelated to whether the function under test actually works correctly. (But note also that you should also have written unit tests for LOCAL::Cool.)
You could just add a BEGIN{ #INC = qw(directories you want to allow)} block. While using PERL5LIB, use lib etc. just appends to the include path, this will completely replace it, thus eliminating the danger that the module is picked up later in the search path, e.g. because you forgot to install it /home/me/perl/.
But the real TDD answer is probably to use mock modules for everything that's not the module under test.
Is there a way to load entire modules during runtime in Perl? I had thought I found a good solution with autouse but the following bit of code fails compilation:
package tryAutouse2;
use autouse 'tryAutouse';
my $obj = tryAutouse->new();
I imagine this is because autouse is specifically meant to be used with exported functions, am I correct? Since this fails compilation, is it impossible to have a packaged solution? Am I forced to require before each new module invocation if I want dynamic loading?
The reasoning behind this is that my team loads many modules, but we're afraid this is eating up memory.
You want Class::Autouse or ClassLoader.
Due to too much magic, I use ClassLoader only in my REPL for convenience. For serious code, I always load classes explicitely. Jack Maney points out in a comment that Module::Load and Module::Load::Conditional are suitable for delayed loading.
There's nothing wrong with require IMO. Skip the export of the function and just call the fully qualified name:
require Some::Module;
Some::Module::some_function(#some_arguments);
eval 'use tryAutouse; 1;' or die $#;
Will work. But you might want to hide the ugliness.
When you say:
use Foo::Bar;
You're loading module Foo::Bar in at compile time. Thus, if you want to load your module in at run time, you'd use require:
require Foo::Bar;
They are sort of equivalent, but there are differences. See the Perldoc on use to understand the complete difference. For example, require used in this way won't automatically load in imported functions. That might be important to you.
If you want to test whether a module is there or not, wrap up your require statement in an eval and test whether or not eval is successful.
I use a similar technique to see if a particular Perl module is available:
eval { require Mail::Sendmail; };
if ($#) {
$watch->_Send_Email_Net_SMTP($watcher);
return;
}
In the above, I'll attempt to use Mail::Sendmail which is an optional module if it's available. If not, I'll run another routine that uses Net::SMTP:
sub _Send_Email_Net_SMTP {
my $self = shift;
my $watcher = shift;
require Net::SMTP; #Standard module: It should be available
WORD O'WARNING: You need to use curly braces around your eval statement and not parentheses. Otherwise, if the require doesn't work, your program will exit which is probably not what you want to do.
Instruction 'use' is performed at compile time, so check the path to the module also takes place at compile time. This may cause incorrect behavior, which are difficult to understand until you consider the contents of the #INC array.
One solution is to add block 'BEGIN', but the solution shown below is inelegant.
BEGIN { unshift #INC, '/path/to/module/'; }
use My::Module;
You can replace the whole mess a simple directive:
use lib '/path/to/module';
use My::Module;
This works because it is performed at compile time. So everything is ready to execute 'use' instruction.
Instead of the 'BEGIN' block, you can also decide to different instruction executed at compile time ie declaring a constant.
use constant LIB_DIR => '/path/to/module';
use lib LIB_DIR;
use My::Module;
I have an object-oriented web-app that is installed in multiple locations on my server. Once for "live", once for "beta", etc. Being object-oriented, it consists of many perl modules. In the main module, I must "use lib" the appropriate directory for all of the custom perl modules for that instance of the app.
This is no big deal, I have a BEGIN block that checks the location of the main program and sets the library directory appropriately. However I also have a lot of utility, command line programs that need to do the same thing. I don't want to cut and paste this code everywhere.
What is the best way to share this code snippet amongst the various programs that need it?
I can't "use" it because the libary path isn't set up yet. Maybe "do" or "require" would the be the right answer, but both of those will search #INC, which is inappropriate.
Maybe something like eval `cat GetLib.pl`; would be appropriate but it seems kind of clunky and fragile.
Here is the BEGIN block that I currently use:
BEGIN {
use FindBin qw ($Bin);
require lib;
if ($Bin =~ /^\/home\/w\/myapp_live/) {
lib->import('/home/w/myapp_live/lib');
print STDERR "live site\n";
}
if ($Bin =~ /^\/home\/w\/myapp_beta/) {
lib->import('/home/w/myapp_beta/lib');
print STDERR "beta site\n";
}
if ($Bin =~ /^\/home\/w\/myapp_test/) {
lib->import('/home/w/myapp_test/lib');
print STDERR "testing site\n";
}
}
Thank you!
FindBin::libs is excellent for that. I've used it for a while in a large system with no problems at all.
The default invocation looks like it'll work for you, simply:
use FindBin::libs;
This will search for all the ./lib dirs in all the parent directories of the current file's dir and use lib them. So, for example, if your script lives in /home/w/myapp_live/scripts/defurblise_widgets.pl (and use()es FindBin::libs), it will look for:
/home/w/myapp_live/scripts/lib
/home/w/myapp_live/lib
/home/w/lib
/home/lib
/lib # (presumably!)
Any that it finds with be added to you #INC with use lib.
But, if that's not quite what you need, it's a very flexible module. I'd be surprised if you can't find a way to make it do what you want.
If you're running a program from the command line, don't programmatically set the lib: just pass it in as an argument, e.g.: perl -I/location/of/my/lib myprog.pl.
For your web app, why don't you make your library relative to the location of the script itself? Then just install it on each machine where the libraries live.
use FindBin;
use File::Spec::Functions;
use Cwd qw(abs_path getcwd);
BEGIN {
my $curdir = getcwd;
my $selfdir = $FindBin::Bin;
my $libdir = abs_path(catdir($selfdir, 'lib'));
chdir $libdir or die "can't chdir to $libdir: $#";
use lib $libdir;
}
Of course, the easiest option of all is to not use different lib directories. Why can't you be consistent across all your environments?
Edit. Re your comment "The reason I have to use different lib directories is because the code running in the live site is different than the code running on the beta site... that's the point of having a beta site." -- why don't you handle this at the level of the installer, rather than making the code itself have to know whether it's live vs. beta? e.g. store your code in different directories in your source tree as you do now, but only install the relevant code to each box. After all, that's exactly what a good revision control system would do for you -- you only check out one branch at a time, and you should only be installing one version of code at a time (as brian d foy alluded to).
I use the following in many of my scripts:
use strict;
use warnings;
use 5.008;
use FindBin;
use lib $FindBin::Bin;
That last line could be modified as such:
use FindBin;
use lib "$FindBin::Bin/lib";
The different environments could have different settings for the environment variable $PERL5LIB.
The code you've shown looks reasonable. You could install one copy of that code into a system-wide location, and then the code to invoke it would boil down to
require '/path/to/findlib.pl';
findlib->import;
The form of require that takes a filename doesn't search #INC.
As an alternative if you wanted to change lots of things around you could look into deploying the app in a way that would be more friendly to local::lib usage.