Consider that I have 100 Perl modules in 12 directories. But, looking into the main Perl script, it looks like 100 use p1 ; use p2 ; etc. What is the to best way to solve this issue?
It seems unlikely to me that you're useing all 100 modules directly in your main program. If your program uses a function in module A which then calls a function from module B, but the main program itself doesn't reference anything in module B, then the program should only use A. It should not use B unless it directly calls anything from module B.
If, on the other hand, your main program really does talk directly to all 100 modules, then it's probably just plain too big. Identify different functional groupings within the program and break each of those groups out into its own module. The main reason for doing this is so that it will result in code that is more maintainable, flexible, and reusable, but it will also have the happy side-effect of reducing the number of modules that the main program talks to directly, thus cutting down on the number of use statements required in any one place.
(And, yes, I do realize that 100 was probably an exaggeration, but, if you're getting uncomfortable about the number of modules being used by your code, then that's usually a strong indication that the code in question is trying to do too much in one place and should be broken down into a collection of modules.)
Put all the use statements in one file, say Mods.pm:
package Mods;
use Mod1;
use Mod2;
...
and include the file in your main script:
use Mods;
I support eugene's solution, but you could group the use statements in files by topic, like:
package Math;
use ModMatrix;
use ModFourier;
...
And of course you should name the modules and the mod-collections meaningful.
Putting all of the use statements in a separate file as eugene y suggested is probably the best approach. you can minimize the typing in that module with a bit of meta programming:
package Mods;
require Exporter;
our #ISA = 'Exporter';
my #packages = qw/Mod1 Mod2 Mod3 .... /;
# or map {"Mod$_"} 1 .. 100 if your modules are actually named that way
for (#packages) {
eval "require $_" or die $#; # 'use' means "require pkg; pkg->import()"
$_->import(); # at compile time
}
our #EXPORT = grep {*{$Mods::{$_}}{CODE}} keys %Mods::; # grab imported subs
#or #EXPORT_OK
Related
I'm a perl newbie who is working on a module that is getting quite long and I want to split it into multiple files for easier maintainability and organization. Right now it looks something like this
#ABC.pm
package ABC;
use strict;
use warnings;
my $var1;
my $var2;
sub func1 {
#some operations on a $var
}
sub func2 {
#some operations on a $var
}
return 1;
I'd like it to look something like
#ABC_Part_1.pm
package ABC;
use strict;
use warnings;
my $var1;
my $var2;
sub func1 {
#some operations on a $var
}
return 1;
#ABC_Part_2.pm
package ABC;
use strict;
use warnings;
sub func2 {
#some operations on a $var
}
return 1;
The issue I'm having is getting the variables to be seen across the separate files. I tried to declare them using 'our', but then I have to use the scope resolution operator which I don't want to do. I'd like to treat them as local variables within the module files, but have them hidden to the calling script. I'd also like to only have one include in the calling script, like
#!/usr/bin/env perl
#script.pl
use strict;
use warnings;
use ABC;
func1();
func2();
Thanks
The issue I'm having is getting the variables to be seen across the separate files.
Your best option is to stop wanting that.
The whole point of lexical variables is for them to be accessible in only a small locally visible bit of code. A variable that needs to be accessed from multiple different files is a sign of "code smell".
If you're really sure you want this though…
I tried to declare them using 'our', but then I have to use the scope resolution operator which I don't want to do.
Yes, this should work, but you need to declare the variable at the top of each file you use it in.
# ABC_part_1.pm
package ABC;
our $foo;
# code that accesses $foo goes here
1;
And then:
# ABC_part_2.pm
package ABC;
our $foo;
# code that accesses $foo goes here
1;
You can make your ABC.pm file a collection of require statements.
package ABC;
require ABC1;
require ABC2;
require ABC3;
1;
It's important to work with require instead of use, because use will try to import things automatically. But that will not work since there is no package ABC1 in your ABC1.pm file, so ABC1->import will fail.
Regarding the variables, there's really no way to get lexical variables into different files. You could use do instead of require, which would read and run the files directly in the line with the do. That way, the scope would stay the same, and you could have this.
package ABC;
my $foo;
my $bar;
do 'lib/ABC1.pm';
do 'lib/ABC2.pm';
Please do not do this. It's crazy!
If you feel that your library is getting too big, first add proper documentation to every function, and sort so that things that belong together are together. If that does not help you, split up the file into smaller logical units and make those individual packages that talk to each other through a defined interface, but also are able to stand alone where needed.
If repeating a bunch of use statements feels like too much boiler plate, write your own module collection (like strictures) using Import::Into.
Furthermore, don't use lexical variables in the file scope. If you want to have state for things, create object oriented code and write classes. Then you'll have state and behavior. If you have package/class data, use package variables.
Perl doesn't have the concept of private things for a reason. There are conventions in place to mark things as private, like naming them _stuff with a leading underscore. That's a sign for everyone that this is internal, not a stable API, might change at any moment and shouldn't be messed with. Do that, instead of trying to hide things. It's a strength of Perl to allow you to mess with everything. But that doesn't mean you have to do it. It's an option that you should embrace.
Consider the following script p.pl:
use strict;
use warnings;
use AA;
BB::bfunc();
where the file AA.pm is:
package AA;
use BB;
1;
and the file BB.pm is:
package BB;
sub bfunc {
print "Running bfunc..\n";
}
1;
Running p.pl gives output (with no warnings or errors):
Running bfunc..
Q: Why is it possible to call BB::bfunc() from p.pl even though there is no use BB; in p.pl? Isn't this odd behavior? Or are there situation where this could be useful?
(To me, it seems like this behavior only presents an information leak to another package and violates the data hiding principle.. Leading to programs that are difficult to maintain.. )
You're not polluting a namespace, because the function within BB isn't being 'imported' into your existing namespace.
They are separate, and may be referenced autonomously.
If you're making a module, then usually you'll define via Exporter two lists:
#EXPORT and #EXPORT_OK.
The former is the list of things that should be imported when you use the package. The latter is the things that you can explicity import via:
use MyPackage qw ( some_func );
You can also define package variables in your local namespace via our and reference them via $main.
our $fish = "haddock";
print $main::fish;
When you do this, you're explicitly referencing the main namespace. When you use a module, then you cause perl to go and look for it, and include it in your %INC. I then 'knows about' that namespace - because it must in order for the dependencies to resolve.
But this isn't namespace pollution, because it doesn't include anything in your namespace until your ask.
This might make a bit more sense if you have multiple packages within the same program:
use strict;
use warnings;
package CC;
our $package_var = "Blong";
sub do_something {
print $package_var,"\n";
}
package main;
use Data::Dumper;
our $package_var = "flonk";
print Dumper $package_var;
print Dumper $CC::package_var;
Each package is it's own namespace, but you can 'poke' things in another. perl will also let you do this with object - poking at the innards of instantiated objects or indeed "patch" them.
That's quite powerful, but I'd generally suggest Really Bad Style.
While it's good practice to use or require every dependency that you are planning to access (tried to avoid use here), you don't have to do that.
As long as you use full package names, that is fine. The important part is that Perl knows about the namespaces. If it does not, it will fail.
When you use something, that is equivalent to:
BEGIN {
require Foo::Bar;
Foo::Bar->import();
}
The require will take the Foo::Bar and convert it to a path according to the operating system's conventions. On Linux, it will try to find Foo/Bar.pm somewhere inside #INC. It will then load that file and make a note in %INC that it loaded the file.
Now Perl knows about that namespace. In case of the use it might import something into your own namespace. But it will always be available from everywhere after that as long as you use the full name. Just the same, stuff that you have in your main script.pl would be available inside of packages by saying main::frobnicate(). (Please don't do that!)
It's also not uncommon to bundle several namespaces/packages in one .pm module file. There are quite a few big names on CPAN that do it, like XML::Twig.
If you do that, and don't import anything, the only way to get to the stuff under the different namespaces is by using the full name.
As you can see, this is not polluting at all.
I have two perl scripts that I need to run together.
The first script defines a number of common functions and a main method.
Script 1(one.pl) Example:
#!/usr/bin/perl
sub getInformation
{
my $serverMode = $_[0];
my $port = $_[1];
return "You\nneed\nto\nparse\nme\nright\nnow\n!\n";
}
#main
&parseInformation(&getInformation($ARGV[0], $ARGV[1]));
The second is a script that calls the second script after defining two functions.
Script 2(two.pl) Example:
#!/usr/bin/perl
sub parseInformation
{
my $parsedInformation = $_[0];
#omitted
print "$parsedInformation";
}
my $parameterOne = $ARGV[0];
my $parameterTwo = $ARGV[1];
do "./one.pl $parameterOne $parameterTwo";
Command line usage:
> ./two.pl bayside 20
I have attempted to do this and the script seems to run however, whenever I run the script in perl -d two.pl mode I get no information from the debugger about the other script.
I have done some research and read about system, capture, require and do. If use the system function to run the script, how will I be able to export the functions defined in script two?
Questions:
1. Is there anyway to do this in perl?
2. If so how exactly do I need to achieve that?
I fully understand that perl is perl. Not another programming language. Unfortunately, when transitioning one tends to bring with what they knew with them. My apologies.
References:
How to run a per script from within a perl script
Perl documentation for require function
Generally speaking, that is not the way you should write reusable common functions in Perl. Instead, you should put the bulk of your code into Perl modules, and write just short scripts that act as wrappers for the modules. These short scripts should basically just grab and validate command-line arguments, pass those arguments to the modules for the real work, then format and output the results.
I really wish I could recommend perldoc perlmod to learn about writing modules, but it seems to mostly concentrate on the minutiae rather than a high-level overview of how to write and use a Perl module. Gabor Szabo's tutorial is perhaps a better place to start.
Here's a simple example, creating a script that outputs the Unix timestamp. This is the module:
# This file is called "lib/MyLib/DateTime.pm"
use strict;
use warnings;
package MyLib::DateTime;
use parent "Exporter";
our #EXPORT_OK = qw( get_timestamp );
sub get_timestamp {
my $ts = time;
return $ts;
}
1;
And this is the script that uses it:
#!/usr/bin/env perl
use strict;
use warnings;
use lib "/path/to/lib"; # path to the module we want, but
# excluding the "MyLib/DateTime.pm" part
use MyLib::DateTime qw( get_timestamp ); # import the function we want
# Here we might deal with input; e.g. #ARGV
# but as get_timestamp doesn't need any input, we don't
# have anything to do.
# Here we'll call the function we defined in the module.
my $result = get_timestamp();
# And here we'll do the output
print $result, "\n";
Now, running the script should output the current Unix timestamp. Another script that was doing something more complex with timestamps could also use MyLib::DateTime.
More importantly, another module which needed to do something with timestamps could use MyLib::DateTime. Putting logic into modules, and having those modules use each other is really the essence of CPAN. I've been demonstrating a really basic date and time library, but the king of datetime manipulation is the DateTime module on CPAN. This in turn uses DateTime::TimeZone.
The ease of re-using code, and the availability of a large repository of free, well-tested, and (mostly) well-documented modules on CPAN, is one of the key selling points of Perl.
Exactly.
Running 2 separate scripts at the same time won't give either script access to the others functions at all. They are 2 completely separate processes. You need to use modules. The point of modules is so that you don't repeat yourself, often called "dry" programming. A simple rule of thumb is:
If you are going to use a block of code more than once put it into a subroutine in the current script.
If you are going to use the same block in several programs put it in a module.
Also remember that common problems usually have a module on CPAN
That should be enough to get you going. Then if you're going to do much Perl Programming you should buy the book "Programming Perl" by Larry Wall, if you've programmed in other languages, or "Learning Perl" by Randal Schwartz if you're new to programming. I'm old fashioned so I have both books in print but you can still get them as ebooks. Also check out Perl.org as you're not alone.
I have a main program mytool.pl to be run from the command line. There are several auxillary scripts special1.pl, special2.pl, etc. which each contain a couple subroutines and a hash, all identically named across scripts. Let's suppose these are named MySpecialFunction(), AnotherSpecialFunction() and %SpecialData.
I'd like for mytool to include/use/import the contents of one of the special*.pl files, only one, according to a command line option. For example, the user will do:
bash> perl mytool.pl --specialcase=5
and mytools will use MySpecialFunction() from special5.pl, and ignore all other special*.pl files.
Is this possible and how to do it?
It's important to note that the selection of which special file to use is made at runtime, so adding a "use" at the top of mytool.pl probably isn't the right thing to do.
Note I am a long-time C programmer, not a perl expert; I may be asking something obvious.
This is for a one-off project that will turn to dust in only a month. Neither mytool.pl nor special?.pl (nor perl itself) will be of interest beyond the end of this short project. Therefore, we don't care for solutions that are elaborate or require learning some deep magic. Quick and dirty preferred. I'm guessing that Perl's module mechanism is overkill for this, but have no idea what the alternatives are.
You can use a hash or array to map values of specialcase to .pl files and require or do them as needed.
#!/usr/bin/env perl
use strict; use warnings;
my #handlers = qw(one.pl two.pl);
my ($case) = #ARGV;
$case = 0 unless defined $case;
# check that $case is within range
do $handlers[$case];
print special_function(), "\n";
When you use a module, Perl just require's the module in a BEGIN block (and imports the modules exported items). Since you want to change what script you load at runtime, call require yourself.
if ($special_case_1) {
require 'special1.pl';
# and go about your business
}
Here's a good reference on when to use use vs. require.
Is there a way to load entire modules during runtime in Perl? I had thought I found a good solution with autouse but the following bit of code fails compilation:
package tryAutouse2;
use autouse 'tryAutouse';
my $obj = tryAutouse->new();
I imagine this is because autouse is specifically meant to be used with exported functions, am I correct? Since this fails compilation, is it impossible to have a packaged solution? Am I forced to require before each new module invocation if I want dynamic loading?
The reasoning behind this is that my team loads many modules, but we're afraid this is eating up memory.
You want Class::Autouse or ClassLoader.
Due to too much magic, I use ClassLoader only in my REPL for convenience. For serious code, I always load classes explicitely. Jack Maney points out in a comment that Module::Load and Module::Load::Conditional are suitable for delayed loading.
There's nothing wrong with require IMO. Skip the export of the function and just call the fully qualified name:
require Some::Module;
Some::Module::some_function(#some_arguments);
eval 'use tryAutouse; 1;' or die $#;
Will work. But you might want to hide the ugliness.
When you say:
use Foo::Bar;
You're loading module Foo::Bar in at compile time. Thus, if you want to load your module in at run time, you'd use require:
require Foo::Bar;
They are sort of equivalent, but there are differences. See the Perldoc on use to understand the complete difference. For example, require used in this way won't automatically load in imported functions. That might be important to you.
If you want to test whether a module is there or not, wrap up your require statement in an eval and test whether or not eval is successful.
I use a similar technique to see if a particular Perl module is available:
eval { require Mail::Sendmail; };
if ($#) {
$watch->_Send_Email_Net_SMTP($watcher);
return;
}
In the above, I'll attempt to use Mail::Sendmail which is an optional module if it's available. If not, I'll run another routine that uses Net::SMTP:
sub _Send_Email_Net_SMTP {
my $self = shift;
my $watcher = shift;
require Net::SMTP; #Standard module: It should be available
WORD O'WARNING: You need to use curly braces around your eval statement and not parentheses. Otherwise, if the require doesn't work, your program will exit which is probably not what you want to do.
Instruction 'use' is performed at compile time, so check the path to the module also takes place at compile time. This may cause incorrect behavior, which are difficult to understand until you consider the contents of the #INC array.
One solution is to add block 'BEGIN', but the solution shown below is inelegant.
BEGIN { unshift #INC, '/path/to/module/'; }
use My::Module;
You can replace the whole mess a simple directive:
use lib '/path/to/module';
use My::Module;
This works because it is performed at compile time. So everything is ready to execute 'use' instruction.
Instead of the 'BEGIN' block, you can also decide to different instruction executed at compile time ie declaring a constant.
use constant LIB_DIR => '/path/to/module';
use lib LIB_DIR;
use My::Module;