How should I distribute data files with Perl modules? - perl

I've started a module-starter (with --builder=Module::Build). I want to use SQL::Library to collect my SQL into an .ini file... but in order to find the file during run time, I need to know its exact path. How do I get the path of the "data directory" of a module at run time?
Until now, I've been using FindBin (like "$FindBin::Bin/../../path/to/module/datafiles/foo.ini", but I found this is not very robust (For example, it seems to break when there are two programs with the same name in two different dirs in PATH).

This is what File::ShareDir is for. Since you're using Module::Build, you'll need to set the share_dir parameter (and require Module::Build 0.36) in order to have your data files installed along with your module. Then, in your code, you'll use File::ShareDir to calculate the path of foo.ini (e.g. dist_file('My-Dist', 'foo.ini'))

Related

Where to put non-essential .pl files in a distribution?

I would like to include a few additional .pl files in my CPAN module. These files are not essential to use the module, but are provide useful functionality/glue when the module is used in some common frameworks and applications.
Currently, I just include the .pl files in a "extras" directory of the distribution. This has the drawback that the files are not installed on make install. Is there a way to include them in the installation and where should they go? (They aren't executables and don't belong in "bin".) Would "share" make sense? Or are these kinds of files usually just not installed and it is left to the user to get them out of the .tgz archive and use as needed?
I use Dist::Zilla to manage my distribution.
I would suggest the following:
If they're actual complete programs or are almost complete, polish then up and make them standalone items that could go into /bin with POD of their own.
If they're utility glue, make a ::Utils module for them to live in and document their usage.
If these are useful code snippets but not something you can install somewhere or are sample usages or handy idioms, create a ::Cookbook all-POD module and include them there with the appropriate illuminating explanation for each one.
I don't know exactly how Dist::Zilla works, but the resulting archive has to be compatible with what ExtUtils::MakeMaker creates.
When you create a module with module-starter, it creates a module template using ExtUtils::MakeMaker. It creates several files and directories like the lib directory where your module lives and the t directories where your tests live.
One thing it doesn't create is a bin directory. However, if you create a bin directory, and you put files under this directory (such as Perl scripts), these files will be installed under the bin directory in your Perl's distribution and linked to /usr/local/bin or /usr/bin. Would this be a good place for your scripts?
I liked #Joe's answer, except that in my case the files were WebWork macros -- individual .pl files that make my module callable from WebWork end-user's code. So they don't fit under any of the categories discussed here, and as .pl files can't be made a module.
This is what I ended up doing:
put all .pl macro files into 'extras/WebWork' in the distribution.
add to "dist.ini" file a [ShareDir] stanza with dir = extras property.
now the WebWork admin can install my distribution from CPAN and then use perl -MFile::ShareDir -e 'print File::ShareDir::dist_dir("Statistics-R-IO")' to find the macros and make them available in WebWork.

import/require/use for perl modules

I have 2 files perl files which have dependency on one another.
The 1st file, A is a .pl file which multiple package/modules declared inside.
The 2nd file, B will try to access one of the many packages declared in file A. How can that be done?
You may be familiar with how Java or a similar language finds an import com.example.AppName namespace by looking for the com/example/AppName.java file in the classpath. When you ask Perl for use HTML::Template, it likewise looks for HTML/Template.pm in the directories listed in #INC.
The difference is that a Perl file can have multiple packages. When you say use HTML::Template, you'll be pulling in all packages listed in HTML/Template.pm (use automatically translates the :: into the right directory separator for your system, and adds the .pm). If there happens to be a HTML::Template::Extension package in that file, then you can use that package without having to explicitly use it. It's already loaded the complete file, and that's good enough.
If you know the exact file name relative where you'll be running the script, the easiest way to grab it is:
require 'path/to/file.pl';
You need to declare a module in each pl file that is using a function of the module.

How do I best use a library whose directory may change occasionally?

I am writing Perl scripts and when I have too many functions, I usually move them all into a library (also good for code reuse). So I usually create a package (e.g. my_lib.pm) and add use lib 'path/to/lib'; use my_lib; to my script.
I wonder if it's possible to skip the use lib 'path/to/lib';, which sometimes gives me trouble since I reorganize my directory hierarchy, and make Perl look for packages in the same dir where the script is running from.
Thank you.
First, i suggest you - "Never mess up with Core Perl and its libraries - never put your lib in among there".
If you want your script look into current dir, then use like:
require "mylibrary/functions.pm";
where mylibrary is a dir that exists the same path as your caller script.
I would put my .pm file into one directory so you can use if from Perl scrips irrespective of their location.
Then create an envrironment variable PERL5LIB with the name of that directory.
You need
use lib '.';

How do I start a new Perl module distribution?

I'm trying to set up a large-ish project, written in Perl. The IBM MakeMaker tutorial has been very helpful so far, but I don't understand how to link all the modules into the main program. In my project root, I have MANIFEST, Makefile.PL, README, a bin directory, and a lib directory. In my bin directory, I have my main script (Main.pl). In the lib directory, I have each of my modules, divided up into their own respective directories (i.e. Utils::Util1 and Utils::Utils2 in the utils directory, etc). In each module directory, there is also a t directory, containing tests
My MANIFEST file has the following:
bin/Main.pl
lib/Utils/Util1.pm
lib/Utils/Util2.pm
lib/Utils/t/Utils1.t
lib/Utils/t/Utils2.t
Makefile.PL
MANIFEST
README
Makefile.PL is the following:
use ExtUtils::MakeMaker;
WriteMakefile(
'NAME'=>'Foo',
'VERSION_FROM'=>'bin/Main.pl',
'PREREQ_PM'=>{
"XML::Simple"=> 2.18}, #The libraries that we need and their
#minimum version numbers
'EXE_FILES' =>[("bin/Main.pl")]
);
After I make and run, the program crashes, complaining that it cannot find Utils::Util1, and when I run 'make test, it says no tests defined. Can anyone make any suggestions? I have never done a large scale project like this in perl, and I will need to add many more modules
If you are just starting to create Perl modules (which is also Perl's equivalent of a project), don't use Makemaker. Module::Build is the way to go, and it's now part of the standard library. Makemaker is for us old salts who haven't converted to Module::Build yet. :) I'll strike that now that Module::Build is unmaintained and out of favor; I still use MakeMaker.
You should never start off a Perl project by trying to create the structure yourself. It's too much work and you'll always forget something.
There's h2xs, a program that comes with perl and was supposed to be a tool to convert .h files into Perl's glue language XS. It works fine, but its advantage is that it comes with perl:
% h2xs -AXn Module::Name
Something like Module::Starter is a bit more sophisticated, although you have to get it from CPAN. It's the tool we use in Intermediate Perl because it's simple. It fills in some templates with your information:
% module-starter --author=... --email=... --module=...
If you are doing to do this quite a bit, you might then convert that to Distribution::Cooker so you can customize your files and contents. It's a dinky utility I wrote for myself so I could use my own templates.
% dist_cooker Module::Name
If you're really hard core, you might want Dist::Zilla, but that's more for people who already know what they are doing.
Might I also suggest module-starter? It'll automatically create a skeleton project which "Just Works". I learned what little I know about Perl modules organization by reading the generated skeleton files. It's all well-documented, and quite easy to use as a base for growing a larger project in. You can check out the getting-started docs to see what it gives you.
Running module-starter will give you a Perl distribution, consisting of a number of modules (use the command line option --module, such as:
module-starter --distro=Project --module=Project::Module::A,Project::Module::B [...]
to create multiple modules in a single distribution). It's then up to you whether you'd prefer to organize your project as a single distribution consisting of a number of modules working together, or as a number of distributions which can be released separately but which depend on each other (as configured in your Build or Makefile.PL file) to provide a complete system.
Try this structure:
bin/Main.pl
lib/Utils/Util1.pm
lib/Utils/Util2.pm
Makefile.PL
MANIFEST
README
t/Utils1.t
t/Utils2.t
As ysth said, make does not install your modules, it just builds them in a blib directory. (In your case it just copies them there, but if you had XS code, it would be compiled with a C compiler.) Use make install to install your modules for regular scripts to use.
If you want to run your script between make and make install, you can do:
perl -Mblib bin/Main.pl
The -Mblib instructs perl to temporarily add the appropriate directories to the search path, so you can try out an uninstalled module. (make test does that automatically.)
By default, tests are looked for in a top-level t directory (or a test.pl file, but that has some limitations, so should be avoided).
You say "After I make and run"...make puts things into a blib directory structure ready to be installed, but doesn't do anything special to make running a script access them. (make test is special; it does add appropriate paths from blib to perl's #INC to be able to run the tests.) You will need to do a "make install" to install the modules where your script will find them (or use a tool like PAR to package them together with your script).

How can I force Perl to run the module from the test directory only?

I have perl module file and test script pl which I use to run some function, and after browsing through the use lib and this previous question here...
I still don't know how to do this.
Basically I have a production directory which contains the module file, and I have a test directory file which contains the same module and the test script file. each time I try to run the script file, it will automatically calls the module in the directory. by printing out the #INC, it seems that the production directory is hosted in there. thus I try to remove it by using
no lib qw(prod_dir);
and while printing out the #INC shows that the directory is no longer there, somehow the script is still calling that other module...
I know this probably sounds really dumb, but hope someone can help me see the light here :)
thanks.
After you have required or used the module, check %INC to see where it came from.
For example:
use Data::Dumper;
print $INC{'Data/Dumper.pm'}."\n";
Note that "::" becomes "/" and you append ".pm". That might give you a clue.
Remember that the current directory (".") is usually an entry in #INC. But the first step is finding out what directory the module was loaded from.
Another thing to remember is that the directories in #INC are searched in order. use lib prepends to that list (making it the first-searched directory), so you may just need to add the appropriate directory.
Can you say more about what you are trying to do and how you are trying to do it? Is this stuff in a standard Perl distribution structure? If you aren't using the standard distribution structure, can you show us a directory listing so we know where things are? Can you also include the code you use to try to load the module? Just update your original question when you pull together the details.
Typically, I run tests through the build runner, which automatically sets up the right #INC.
If I want to run one test in my distribution, I use the blib module to find the build library which has the development versions of my modules:
% perl -Mblib t/test.t
Some people do the same thing with prove.
If you aren't using the basic distribution set-up, consider using it. The tools and best techniques rely on it.
If you just have your module and test file in the same directory, have you tried adding the current directory to #INC with PERL5LIB?