How to dynamically avoid 'use module' to reduce memory footprint - perl

Given the following module:
package My::Object;
use strict;
use warnings;
use My::Module::A;
use My::Module::B;
use My::Module::C;
use My::Module::D;
...
1;
I would like to be able to call My::Object in the next 2 scenarios:
Normal use
use My::Object;
My::Module->new();
Reduced memory use. Call the same object but with a condition or a flag telling the object to skip the use modules to reduce memory usage. Somehow like:
use My::Object -noUse;
My::Module->new();
If tried the Perl if condition without success.
The problem I'm having it's with big objects with a lot of uses, then only loading this object consumes a lot of RAM. I know I can refactor them but it will be wonderful if somehow I can avoid these uses when I'm sure none of them is used on the given scenario.
One solution will be to replace all uses with requires on all places when the modules are needed, but I don't see convenient when some of them are used in a lot of methods.
Any ideas?
Thanks

The native pragma autouse will load modules needed when plain subroutines are called:
use autouse 'My::Module::A' => qw(a_sub);
# ... later ...
a_sub "this will dynamically load My::Module::A";
For proper OO methods, Class::Autouse will load modules (classes) when methods are called:
use Class::Autouse;
Class::Autouse->autouse( 'My::Module::A' );
# ... later ...
print My::Module::A->a_method('this will dynamically load My::Module::A');

What I think you're looking for is perhaps require - require is evaluated later so you can use it successfully in a conditional:
if ( $somecondition ) {
require Some::Module;
}
Of course, you won't be able to do Some::Module->new() if you've not loaded it - there's just no way around that.
Where use is triggered at compile time (and thus will trigger warnings under perl -c if the module is unavailable) require happens are runtime. You should probably test if require was successful as a result.
e.g.:
if ( $somecondition ) {
eval { require Some::Module };
warn "Module Not loaded: ".$# if $#;
}
Otherwise you may be looking for:
Is it possible to pass parameters to a Perl module loading?
#!/usr/bin/perl
package MyObject;
sub import {
my ( $package, $msg ) = #_;
if ( defined $msg and $msg eq "NO_USE" ) {
#don't load module
}
else {
require XML::Twig;
}
}
1;
And then call:
use if $somecondition, MyObject => ( 'NO_USE' );
Or just simpler:
use MyObject qw( NO_USE );
Edit:
After a bit of fiddling with 'use' - there's a couple of gotchas, in that use if doesn't seem to like lexical variables. So you need to do something like:
#!/usr/bin/perl
package MyObject;
use strict;
use warnings;
our $import_stuff = 1;
sub import {
my ( $package, $msg ) = #_;
if ( $msg and $msg eq "NO_USE" ) {
$import_stuff = 0;
}
use if $import_stuff, 'Text::CSV';
}
1;
And call:
#!/usr/bin/perl
use strict;
use warnings;
use MyObject qw( NO_USE );
use Data::Dumper;
print Dumper \%INC;
my $test = Text::CSV -> new();
(Which errors if you set NO_USE and doesn't otherwise).
I think that's an artifact of use being a compile time directive still, so requires a (package scoped) condition.

Related

Perl Import Package in different Namespace

is it possible to import (use) a perl module within a different namespace?
Let's say I have a Module A (XS Module with no methods Exported #EXPORT is empty) and I have no way of changing the module.
This Module has a Method A::open
currently I can use that Module in my main program (package main) by calling A::open I would like to have that module inside my package main so that I can directly call open
I tried to manually push every key of %A:: into %main:: however that did not work as expected.
The only way that I know to achieve what I want is by using package A; inside my main program, effectively changing the package of my program from main to A.
Im not satisfied with this. I would really like to keep my program inside package main.
Is there any way to achieve this and still keep my program in package main?
Offtopic: Yes I know usually you would not want to import everything into your namespace but this module is used by us extensively and we don't want to type A:: (well the actual module name is way longer which isn't making the situation better)in front of hundreds or thousands of calls
This is one of those "impossible" situations, where the clear solution -- to rework that module -- is off limits.
But, you can alias that package's subs names, from its symbol table, to the same names in main. Worse than being rude, this comes with a glitch: it catches all names that that package itself imported in any way. However, since this package is a fixed quantity it stands to reason that you can establish that list (and even hard-code it). It is just this one time, right?
main
use warnings;
use strict;
use feature 'say';
use OffLimits;
GET_SUBS: {
# The list of names to be excluded
my $re_exclude = qr/^(?:BEGIN|import)$/; # ...
my #subs = grep { !/$re_exclude/ } sort keys %OffLimits::;
no strict 'refs';
for my $sub_name (#subs) {
*{ $sub_name } = \&{ 'OffLimits::' . $sub_name };
}
};
my $name = name('name() called from ' . __PACKAGE__);
my $id = id('id() called from ' . __PACKAGE__);
say "name() returned: $name";
say "id() returned: $id";
with OffLimits.pm
package OffLimits;
use warnings;
use strict;
sub name { return "In " . __PACKAGE__ . ": #_" }
sub id { return "In " . __PACKAGE__ . ": #_" }
1;
It prints
name() returned: In OffLimits: name() called from main
id() returned: In OffLimits: id() called from main
You may need that code in a BEGIN block, depending on other details.
Another option is of course to hard-code the subs to be "exported" (in #subs). Given that the module seems to be immutable in practice this option is reasonable and more reliable.
This can also be wrapped in a module, so that you have the normal, selective, importing.
WrapOffLimits.pm
package WrapOffLimits;
use warnings;
use strict;
use OffLimits;
use Exporter qw(import);
our #sub_names;
our #EXPORT_OK = #sub_names;
our %EXPORT_TAGS = (all => \#sub_names);
BEGIN {
# Or supply a hard-coded list of all module's subs in #sub_names
my $re_exclude = qr/^(?:BEGIN|import)$/; # ...
#sub_names = grep { !/$re_exclude/ } sort keys %OffLimits::;
no strict 'refs';
for my $sub_name (#sub_names) {
*{ $sub_name } = \&{ 'OffLimits::' . $sub_name };
}
};
1;
and now in the caller you can import either only some subs
use WrapOffLimits qw(name);
or all
use WrapOffLimits qw(:all);
with otherwise the same main as above for a test.
The module name is hard-coded, which should be OK as this is meant only for that module.
The following is added mostly for completeness.
One can pass the module name to the wrapper by writing one's own import sub, which is what gets used then. The import list can be passed as well, at the expense of an awkward interface of the use statement.
It goes along the lines of
package WrapModule;
use warnings;
use strict;
use OffLimits;
use Exporter qw(); # will need our own import
our ($mod_name, #sub_names);
our #EXPORT_OK = #sub_names;
our %EXPORT_TAGS = (all => \#sub_names);
sub import {
my $mod_name = splice #_, 1, 1; # remove mod name from #_ for goto
my $re_exclude = qr/^(?:BEGIN|import)$/; # etc
no strict 'refs';
#sub_names = grep { !/$re_exclude/ } sort keys %{ $mod_name . '::'};
for my $sub_name (#sub_names) {
*{ $sub_name } = \&{ $mod_name . '::' . $sub_name };
}
push #EXPORT_OK, #sub_names;
goto &Exporter::import;
}
1;
what can be used as
use WrapModule qw(OffLimits name id); # or (OffLimits :all)
or, with the list broken-up so to remind the user of the unusual interface
use WrapModule 'OffLimits', qw(name id);
When used with the main above this prints the same output.
The use statement ends up using the import sub defined in the module, which exports symbols by writing to the caller's symbol table. (If no import sub is written then the Exporter's import method is nicely used, which is how this is normally done.)
This way we are able to unpack the arguments and have the module name supplied at use invocation. With the import list supplied as well now we have to push manually to #EXPORT_OK since this can't be in the BEGIN phase. In the end the sub is replaced by Exporter::import via the (good form of) goto, to complete the job.
You can forcibly "import" a function into main using glob assignment to alias the subroutine (and you want to do it in BEGIN so it happens at compile time, before calls to that subroutine are parsed later in the file):
use strict;
use warnings;
use Other::Module;
BEGIN { *open = \&Other::Module::open }
However, another problem you might have here is that open is a builtin function, which may cause some problems. You can add use subs 'open'; to indicate that you want to override the built-in function in this case, since you aren't using an actual import function to do so.
Here is what I now came up with. Yes this is hacky and yes I also feel like I opened pandoras box with this. However at least a small dummy program ran perfectly fine.
I renamed the module in my code again. In my original post I used the example A::open actually this module does not contain any method/variable reserved by the perl core. This is why I blindly import everything here.
BEGIN {
# using the caller to determine the parent. Usually this is main but maybe we want it somewhere else in some cases
my ($parent_package) = caller;
package A;
foreach (keys(%A::)) {
if (defined $$_) {
eval '*'.$parent_package.'::'.$_.' = \$A::'.$_;
}
elsif (%$_) {
eval '*'.$parent_package.'::'.$_.' = \%A::'.$_;
}
elsif (#$_) {
eval '*'.$parent_package.'::'.$_.' = \#A::'.$_;
}
else {
eval '*'.$parent_package.'::'.$_.' = \&A::'.$_;
}
}
}

perl use vs require and import, constants are only functions

I've got a Perl script that I'm trying to make compatible with two different Perl environments. To work around the two different versions of Socket I have, I'm doing a little hackery with require and import. I've got it working, but I'm not happy with the behavior.
Mod.pm:
package Mod;
use base 'Exporter';
our #EXPORT = qw( MAGIC_CONST );
sub MAGIC_CONST() { 42; }
test.pl:
use Mod;
#require Mod;
#import Mod;
printf "MAGIC_CONST = ". MAGIC_CONST ."\n";
printf "MAGIC_CONST = ". MAGIC_CONST() ."\n";
Outputs:
MAGIC_CONST = 42
MAGIC_CONST = 42
But using the 'require' and 'import' instead, I get this:
Output:
MAGIC_CONST = MAGIC_CONST
MAGIC_CONST = 42
So the question is: Is there a clean way I can get the normal behavior of the constants? I can certainly do sub MAGIC_CONST { Mod::MAGIC_CONST(); } but that's pretty ugly.
What I'm actually doing is something like this:
use Socket;
if ($Socket::VERSION > 1.96) {
import Socket qw(SO_KEEPALIVE); # among others
setsockopt($s, SOL_SOCKET, SO_KEEPALIVE); # among others
}
The reason the require version prints MAGIC_CONST instead of 42 is because use is what tells perl to import the symbols from one module to another. Without the use, there is no function called MAGIC_CONST defined, so perl interprets it as a string instead. You should use strict to disable the automatic conversion of barewords like that into strings.
#!/usr/bin/env perl
no strict;
# forgot to define constant MAGIC_CONST...
print 'Not strict:' . MAGIC_CONST . "\n";
produces
Not strict:MAGIC_CONST
But
#!/usr/bin/env perl
use strict;
# forgot to define constant MAGIC_CONST...
print 'Strict:' . MAGIC_CONST . "\n";
Produces an error:
Bareword "MAGIC_CONST" not allowed while "strict subs" in use at
./test.pl line 4. Execution of ./test.pl aborted due to compilation
errors.
So if you want to use one module's functions in another, you either have to import them with use, or call them with the full package name:
package Foo;
sub MAGIC_CONST { 42 };
package Bar;
print 'Foo from Bar: ' . Foo::MAGIC_CONST . "\n";
Foo from Bar: 42
It's usually best to avoid conditionally importing things. You could resolve your problem as follows:
use Socket;
if ($Socket::VERSION > 1.96) {
setsockopt($s, SOL_SOCKET, Socket::SO_KEEPALIVE);
}
If you truly want to import, you still need to do it at compile-time.
use Socket;
use constant qw( );
BEGIN {
if ($Socket::VERSION > 1.96) {
Socket->import(qw( SO_KEEPALIVE ));
} else {
constant->import({ SO_KEEPALIVE => undef });
}
}
setsockopt($s, SOL_SOCKET, SO_KEEPALIVE) if defined(SO_KEEPALIVE);
Adam's answer gives a good explanation of what is going on and how to get the desired behavior. I am going to recommend not using constant.pm to define symbolic names for constants. A fairly nice looking and convenient alternative with fewer gotchas is to use constants defined using Const::Fast and allow them to be imported.
In addition, by using Importer in the module which wants to import the constants, the module that defines the constants can avoid inheriting Exporter's heavy baggage, or having to use Exporter's import.
The fact that Const::Fast allows you to define real constant arrays and hashes is a bonus.
For example:
package MyConstants;
use strict;
use warnings;
use Const::Fast;
use Socket;
const our #EXPORT => ();
const our #EXPORT_OK => qw(
%SOCKET_OPT
);
const our %SOCKET_OPT => (
keep_alive => ($Socket::VERSION > 1.96) ? Socket::SO_KEEPALIVE : undef,
);
__PACKAGE__;
__END__
Using these constants in a script:
#!/usr/bin/env perl
use strict;
use warnings;
use Socket;
use Importer 'MyConstants' => qw( %SOCKET_OPT );
if ( defined $SOCKET_OPT{keep_alive} ) {
setsockopt($s, SOL_SOCKET, $SOCKET_OPT{keep_alive});
}
As I note in my blog post:
f I want to read the constants from a configuration file, that's trivial. If I want to export them to JSON, YAML, INI, or whatever else, that's also trivial. I can interpolate them willy-nilly.
For those who have taken seriously Exporter's stance against exporting variables, this takes some getting used to. Keep in mind that the admonition is there to make sure you don't write code that willy nilly modifies global variables. However, in this case, the variables we are exporting are not modifiable. Maybe you can convince yourself that the concern really does not apply in this instance. If you try to refer to a non-existent constant, you get an error (albeit during run time) ...
In most cases, the benefits of this approach outweigh the speed penalty of not using constant.pm.

Prototype mismatch: sub main::any: none vs (&#) at Exporter.pm

use Dancer2;
use List::Util qw(any);
sub use_any_subroutine {
foreach my $foo (#$foo_array) {
next unless any { $_ == $foo->id } #hello_world;
}
return;
}
there is a conflict with using List::Util and throwing a warning
Prototype mismatch: sub main::any: none vs (&#) at Exporter.pm(...).
i found a solution that is we can use List::Util::any instead of importing it before use,but i want to import it once,so how to avoid this warning
Thanks very much for comments.
There are a few possible solutions
Note that I'm using the core List::Util module here, because it also contains an any function, and shouldn't need installing on most recent releases of perl
As you have said yourself, you could fully-qualify the subroutine instead of importing it, and use List::Util::any
You could disable the import of any from Dancer2 by using
use Dancer2 '!any'
which will work as long as you never need Dancer2's any to write your code
You could also use first instead of any from List::Util, which returns the first element of the list for which the block returns a true value, and will work fine as long as everything in your list is true
Another alternative is to import any from List::Util with a different name
BEGIN {
require List::Util;
*my_any = \&List::Util::any;
}
and now you have a subroutine my_any that behaves exactly as the original any from List::Util but doesn't clash with the operator of the same name from Dancer2
Both Dancer2 and List::MoreUtils export an any function into your namespace.
For Dancer2, it's part of its DSL and is used as a route definition that matches any HTTP verb on incoming requests.
Defines a route for multiple HTTP methods at once
The List::MoreUtils one is like a grep instead.
Returns a true value if any item in LIST meets the criterion given through BLOCK.
The warning you are seeing is because you have imported the Dancer2 one first, so Perl learned about the prototype of that one. Then you imported the one from List::MoreUtils, which overwrote &main::any in your namespace main::, but the prototype is still there.
You can avoid importing any from Dancer2.
use Dancer2 qw( !any );
use List::MoreUtils qw( any );
get '/' => sub {
my #foo = ( 1, 2, 3 );
return List::MoreUtils::any { $_ == 2 } #foo;
};
dance;
Or you can avoid importing any from List::MoreUtils (using List::MoreUtils::any instead of any).
use Dancer2;
use List::MoreUtils qw( );
get '/' => sub {
my #foo = ( 1, 2, 3 );
return List::MoreUtils::any { $_ == 2 } #foo;
};
dance;
It is impossible to be sure without seeing more of your program, but at a guess, you have declared your own subroutine sub any as well as trying to import it from List::MoreUtils.
You need only the use statement.

Error in Perl Rose::DB : Can't use string ... as a HASH ref while "strict"

I am getting an error when using Rose::DB.
#MyApp/DB.pm
package MyIMDB::DB;
use strict; use warnings;
use base qw(Rose::DB);
__PACKAGE__->use_private_registry;
__PACKAGE__->register_db (
driver => 'SQLite',
....
);
1;
# MyApp/DB/Object.pm
package MyApp::DB::Object;
use strict; use warnings;
use MyApp::DB;
use base qw(Rose::DB::Object);
sub init_db { MyIMDB::DB->new }
1;
#
package MyApp::Users; #controller
use strict; use warnings;
use base 'Mojolicious::Controller';
use Mojo::ByteStream 'b';
use MyApp::Models::User;
use Data::Dumper;
sub my_action {
my $uc = shift;
my $err = MyApp::Models::User::->validate(...); #extra ::
# http://perldoc.perl.org/perlobj.html#Invoking-Class-Methods
}
# MyApp/Models/User.pm # 2 packages in this file
package MyApp::Models::User::Manager;
use base qw(Rose::DB::Object::Manager);
use MyApp::Models::User;
sub object_class { 'MyApp::Models::User'}
__PACKAGE__->make_manager_methods('users');
# class methods get_x, get_x_iterator, get_x_count, delete_x, update_x
1;
MyApp::Models::User
use strict; use warnings;
use base qw(MyApp::DB::Object);
__PACKAGE__->meta->setup(
#setup tables, columns....
);
sub validate {
my $u = shift;
my $n = MyApp::Models::User::Manager::->get_users_count(query => [user_name => $user]);
}
1;
The error I get is:
"Can't use string ("MyApp::Models::User") as a HASH ref while "strict refs"
in use at /usr/local/share/perl/5.18.2/Rose/DB/Object.pm line 91, <DATA> line 2231."
The entry point is my_action() method of MyApp:Users class.
I tried alternative setups of creating class MyApp::Models::User::Manager : separate .pm file, make_manager_class(), but to no avail.
(I found this discussion from 2007 with the same error message, but it does not help me out http://comments.gmane.org/gmane.comp.lang.perl.modules.dbi.rose-db-object/1537).
This may indicate I am trying to call an object method as if it were a class method. I tried the tricks listed here http://perldoc.perl.org/perlobj.html#Invoking-Class-Methods, but no success.
I now I can examine the contents of variables with Data::Dumper, but I have no clue what to dump as there are very little data structures used.
While use strict is a good idea when writing Perl code, you may want to relax the strict-ness by adding
no strict `refs`;
to get past the current error. As #ikegami pointed out another way to fix this is to get rid of the bad reference, but if you don't want to rewrite the module working around it with relaxing strict-ness is your best bet.

Argument to Perl module use

Having a C background, I may be trying to write something the wrong way so excuse the beginner question. Here is what I'm looking for :
I'm willing to have a perl module Verbose (not a class) that define a subroutine called verbose_print(). This subroutine will print its argument (a string) only if module's variable $verbose is true. Of course, what I'm willing to do is to get the -V option from the command line (using Getopt::Long) and then, is the -V switch is on, call the Verbose module with 'true' being the value for $Verbose::verbose.
One workaround is to have a verbose_init function that set the $Verbose::verbose variable to true inside the Verbose module.
Another one was to declare $verbose in my module using our and then $Verbose::verbose = $command_line_verbose_switch in the main script.
I was wondering if there is another way to do this in perl?
Don't be so afraid of classes in Perl, they're just packages and modules treated a wee bit differently. They don't bite. However, you said no classes, so no classes.
I prefer not to touch package variables directly. Instead, I'll use a subroutine to set them.
Here's my Local::Verbose (stored under Local/Verbose.pm)
package Local::Verbose;
use strict;
use warnings;
use Exporter 'import';
# Could have used just '#EXPORT', but that's bad manners
our #EXPORT_OK = qw(verbose verbose_switch);
# Use "our", so $verbose_value is a package variable.
# This makes it survive between calls to this package
our $verbose_value;
# prints out message, but only if $verbose_value is set to non-blank/zero value
sub verbose {
my $message = shift;
if ( $verbose_value ) {
print "VERBOSE: $message\n";
return $message;
}
else {
return;
}
}
sub verbose_switch {
my $switch_value = shift;
$verbose_value = $switch_value;
return $switch_value;
}
1;
Notice the our. That makes $verbose_value a package variable which means it lives on outside of the package between calls.
Notice how I use Exporter and the #EXPORT_OK array. You can use #EXPORT which will export all of the named subroutines automatically, but it's now considered bad manners since you could end up covering over someone's local subroutine with the same name. Better make it explicit. If there's a problem, they can use the Local::Verbose::verbose name of the verbose subroutine.
And how it's used
use strict;
use warnings;
use Local::Verbose qw(verbose verbose_switch);
verbose ("This is a test");
verbose_switch(1);
verbose ("This is a second test");
By the way, imagine calling the verbose subroutine like this:
verbose($message, $my_verbose_level);
Now, your verbose subroutine could look like this:
sub verbose {
my $message = shift;
my $verbose_level = shift;
if (not defined $verbose) {
$verbose_level = 1;
}
if ( $verbose_value =< $verbose_level ) {
print "VERBOSE: $message\n";
return $message;
}
else {
return;
}
}
Now, you can set your verbose level to various values, and have your verbose statements give you different levels of verbosity. (I do the same thing, but call it debug).
One of 'another ways' is create an import function:
package Verbose;
my $verbose_on;
sub import {
$verbose_on = shift;
}
#... Your code.
Now you can set your verbose like this:
if( ... ) { #check getopt
use Verbose (1); #This will require package Verbose and call "import"
}
But, i think more simple and obivious to further use is make a function-setter.