I'm currently using the PAR Packer (pp) to package a couple of pl scripts such that they can be copied to a machine and "just work" without my client having to muck with CPAN.
Unfortunately, the PAR Packer doesn't work for deep dependencies. For example, a script imports CHI::Driver::File but does not explicitly import Log::Any::Adapter::Null which CHI::Driver::File requires. PAR does not pick up this dependency and running the generated script will crash with the following error message:
Can't locate Log/Any/Adapter/Null.pm in #INC (#INC contains: CODE(0x874aab8) /tmp/par-apache/cache-7b4508ab92efb43271da1629e8eb654c1572cc55/inc/lib /tmp/par-apache/cache-7b4508ab92efb43271da1629e8eb654c1572cc55/inc CODE(0x87e8f54) CODE(0x87e9194)) at (eval 215) line 3.
Compilation failed in require at CHI/Driver/File.pm line 11.
BEGIN failed--compilation aborted at CHI/Driver/File.pm line 11.
My work-around is to explicitly import "Log::Any::Adapter::Null" but there must be a better way. Perhaps this is a bug with the PAR Packer? I have installed the latest version (0.994).
Well, have you read the documentation? It gives a number of command-line options to include dependencies, and even the option to manually force modules to be included.
If you want a stand-alone script as opposed to a stand-alone binary (which is the default mode for pp), then add the -P option. I'd advise against it, though. It's the least-tested mode of operation.
Note also that PAR::Packer DOES work for deep dependencies. The subject of your question is a bit pretentious. In fact, PAR::Packer doesn't really check any dependencies itself but delegates this to Module::ScanDeps. Now, Module::ScanDeps is a heuristic approach to dependency scanning and it can be broken by ugly fuzzing with dynamic loading of dependencies. (i.e. generating module names at run time and then using them in an eval).
Indeed, if you look at the sources for Log::Any (which is used by CHI::Driver::File), you'll quickly see that it uses dynamic loading of modules. This is why the ::Adapter::Null module isn't being picked up.
Generally, we fix these issues by adding a special case to Module::ScanDeps for such modules whose author thought it'd be a good idea to defeat any sort of static analysis. Until you get a fixed version of Module::ScanDeps, you can use the -c or -x options to pp to have the dependency resolution use compilation or execution of the program instead of only relying on static analysis. The augmented Module::ScanDeps has version 0.95 and should be available from CPAN within the day.
Related
I have a series of perl scripts for which I'm writing a Makefile.PL script, but I'm rather inexperienced with ExtUtils::MakeMaker.
One of the scripts I wrote makes a system call to a command line utility that must be installed in order for the script to run properly. My script can gracefully detect that the utility is missing and issue an error about installing it and putting it in the user's path, but is there some standard way to handle this in the Makefile.PL script? Could it even gasp attempt to install the third-party utility if I enter the download link in the Makefile.PL script?
At the very least, I'd like the script to warn the user if the external dependency was not found. I know I can write a test case that uses it. Is this as simple as copying and pasting the subroutine I wrote in the script itself that checks for the third party utility and prints an error if it's not found or would that be the "wrong way to do it"?
Let's call this external dependency foobar, for sake of argument.
As per #KeepCalmAndCarryOn's comment, firstly consider whether foobar could be replaced by something from CPAN (maybe Foo::Bar), or a few lines of Perl.
Otherwise, the best course of action is:
Create a new CPAN distribution called Alien::Foobar. The job of Alien::Foobar is to download, perhaps compile, and then install foobar, as part of Alien::Foobar's Makefile.PL or Build.PL.
(There exists a module called Alien::Base which aims to make doing this sort of thing easier. It's mostly aimed at installing libraries rather than binaries, though I've had some success using it for the latter.)
Now the Makefile.PL you were originally working on can declare a dependency on Alien::Foobar.
If you have an external dependency on a command-line utility (i.e. there's no perl module that does what the utility does), ExtUtils::MakeMaker is not designed to handle such a dependency. What you need to do is write an install script or edit the make file to handle the dependency. Here are the considerations in doing so:
Check if the dependency exists and if the version is sufficient.
Download the dependent package
Configure, compile, & install the dependent package
Test to make sure it works
Update the user's environment setup if necessary
Run your perl package's installation steps (e.g. perl makefile.PL;make;sudo make install)
Note, you may need to know whether your script is running as root or not, which you can verify using id -u to check if the user ID is root (i.e. '0').
I'm working on a fresh Ubuntu vm, I've imported Mojolicious, Mango and MongoDB all through terminal (apt-gets and curls) but when I try and run any project that implements Mango - using command:
morbo project/script/project
It returns the following error:
Couldn't load application from file "project/script/project": "dumper" is not exported by the Mojo::Util module
Can't continue after import errors at /etc/perl/Mango.pm line 10
BEGIN failed--compilation aborted at /etc/perl/Mango.pm line 10.
Compilation failed in require at /home/user/project/script/../lib/project.pm line 3.
BEGIN failed--compilation aborted at /home/user/project/script/../lib/project.pm line 3.
Compilation failed in require at (eval 94) line 1.
The Mojo::Util and Mango.pm are all unedited dependencies as downloaded.
Never had any of these problems getting it to run on windows so I'm a bit lost where to start looking for a solution now.
Does anyone know what might be causing this?
Much the same as with other languages such as python or ruby, using system perl is a bad idea and will cause you headaches down the track. What will also cause you problems (much like this one) is pulling down packages "via curl" and simply unpacking to a directory. You need to install most things properly, and there are tools to manage both of these problems.
For instance, the underlying MongoDB driver module requires a C compiler to build as part of it's install process.
You may have installed that one via apt-get. But it is not recommended.
I would recommend using an "environment based" installation such as plenv or perlbrew in order to "manage" your installed perl installation, and not "mess around" with the system installed perl.
In either case, both will integrate well with the excellent cpanminus module and utility. This allows you to install packages from CPAN in a correct way, so that your application works and you avoid errors.
All options allow methods so you can install the modules locally for your application in this way. plenv may be better for you if you need "per application" separation on the same machine in this way.
Use the correct methods, and you are far less likely to run into problems.
I am newbie to perl cross compilation.
I'm trying to install perl from source as given in www.perl.org.
Commands followed are ./Configure, Make, Make test.
To find the path for specific installation, the CLI "perl -V" shows error as follows.
# ./perl -V
Can't locate Config.pm in #INC (#INC contains: /usr/local/lib/perl5/5.9 .).
BEGIN failed--compilation aborted.
I think i have done some mistake in path inclusions.
Please let me know the reason and solution for successfull cross compilation.
Thanks in advance.
It's probably possible to cross-compile Perl (building on Linux and targeting for BSD), but there's not much point in doing so. NetBSD is as capable as Linux of building Perl locally, and that's almost certainly what you should do.
You can build it from source, following the installation instructions, or better yet you can install a binary package, since somebody else has already done the work of building Perl from source for you.
Unless you have a very specific requirement for an old version, you should install the most recent production version, which I believe is 5.14.2.
You mentioned in a comment that when you build Perl from source on your target BSD machine, Make test had 2 failures. You had a 99.86% success rate. Surely that's good enough -- and you're not likely to do any better by cross-compiling. You might want to look into those failures and the LD_LIBRARY_PATH issue (I don't know what it should be set to for NetBSD), but those issues aren't so serious that you should give up and use a radically different approach.
I have an app that I pack into "binary" form using PerlApp for distribution. Since my clients want a simple install for their Win32 systems, this works very nicely.
Now a client has decided that they need to run all unit tests, like in a standard install. However, they still will not install a normal Perl.
So, I find myself in need of a way to package my unit tests for operation on my client's systems.
My first thought was that I could pack up prove in one file and pack each of my tests separately. Then ship a zip file with the appropriate structure.
A bit of research showed that Test::Harness::Straps invokes perl from the command line.
Is there an existing tool that helps with this process?
Perhaps I could use PAR::Packer's parl tool to handle invocation of my test scripts.
I'm interested in thoughts on how to apply either PAR or PerlApp, as well as any thoughts on how to approach overriding Test::Harness and friends.
Thanks.
Update: I don't have my heart set on PAR or PerlApp. Those are just the tools I am familiar with. If you have an idea or a solution that requires a different packager (such as Cava Packager), I would love to hear about it.
Update 2: tsee pointed out a great new feature in PAR that gets me close. Are there any TAP experts out there that can supply some ideas or pointers on where to look in the new Test::Harness distribution?
I'm probably not breaking big news if I tell you that PAR (and probably also perlapp) aren't meant to package the whole test suite and plethora of CPAN-module build byproducts. They're intended to package stand-alone applications or binary JAR-like module libraries.
This being said, you can add arbitrary files to a PAR archive (both to .par libraries and stand-alone .exe's) using pp's -a switch. In case of the stand-alone executable, the contents will be extracted to $ENV{PAR_TEMP}."/inc" at run-time.
That leaves you with the problem of reusing the PAR-packaged executable to run the test harness (and letting that run your executable as a "perl"). Now, I have no ready and done solution for that, but I've recently worked on making PAR-packaged executables re-useable as more-or-less general purpose perl interpreters. Two gotchas before I explain how you can use that:
Your application isn't going to magically be called "perl" and add itself to your $PATH.
The "reuse" of the application as a general purpose perl requires special options and does not currently support the normal perl options (those in perlrun). It can simply run an external perl script of your choice.
Unfortunately, the latter problem is what may kill this approach for you. Support for perl command line options is something I've been thinking about, but won't implement any time soon.
Here's the recipe how you get PAR with "reusable exe" support:
Install the newest version of PAR from CPAN.
Install the newest, developer version of PAR::Packer from CPAN (0.992_02 or 03).
Add the "--reusable" option to your pp command line.
Run your executable with the following options to run an external script "foo.pl":
./myapp --par-options --reuse foo.pl FOO-PL-OPTIONS-HERE
Unfortunately, how you're going to teach Test::Harness that "./myapp --par-options --reuse" is a perl interpreter is beyond me.
Cava Packager allows you to package test scripts with your packaged executables. This is primarily to allow you to run tests against the packaged code before distribution. However the option is there to also distribute the tests and test capability to your end users.
Note: As indicated by my name, I am affiliated with Cava Packager.
I want another developer to run a Perl script I have written. The script uses many CPAN modules that have to be installed before the script can be run. Is it possible to make the script (or the perl binary) to dump a list of all the missing modules? Perl prints out the missing modules’ names when I attempt to run the script, but this is verbose and does not list all the missing modules at once. I’d like to do something like:
$ cpan -i `said-script --list-deps`
Or even:
$ list-deps said-script > required-modules # on my machine
$ cpan -i `cat required-modules` # on his machine
Is there a simple way to do it? This is not a show stopper, but I would like to make the other developer’s life easier. (The required modules are sprinkled across several files, so that it’s not easy for me to make the list by hand without missing anything. I know about PAR, but it seems a bit too complicated for what I want.)
Update: Thanks, Manni, that will do. I did not know about %INC, I only knew about #INC. I settled with something like this:
print join("\n", map { s|/|::|g; s|\.pm$||; $_ } keys %INC);
Which prints out:
Moose::Meta::TypeConstraint::Registry
Moose::Meta::Role::Application::ToClass
Class::C3
List::Util
Imager::Color
…
Looks like this will work.
Check out Module::ScanDeps and the "scandeps.pl" utility that comes with it. It can do a static (and recursive) analysis of your code for dependencies as well as the %INC dump either after compiling or running the program.
Please note that the static source scanning always errs on the side of including too many dependencies. (It is the dependency scanner used by PAR and aims at being easiest on the end-user.)
Finally, you could choose to distribute your script as a CPAN distribution. That sounds much more complicated than it really is. You can use something like Module::Starter to set up a basic skeleton of a tentative App::YourScript distribution. Put your script in the bin/ subdirectory and edit the Makefile.PL to reference all of your direct dependencies. Then, for distribution you do:
perl Makefile.PL
make
make dist
The last step generates a nice App-YourScript-VERSION.tar.gz
Now, when the client wants to install all dependencies, he does the following:
Set up the CPAN client correctly. Simply run it and answer the questions. But you're requiring that already anyway.
"tar -xz App-YourScript-VERSION.tar.gz && cd App-YourScript-VERSION"
Run "cpan ."
The CPAN client will now install all direct dependencies and the dependencies of those distributions automatically. Depending on how you set it up, it will either follow the prerequisites recursively automatically or prompt with a y/n each time.
As an example of this, you might check out a few of the App::* distributions on CPAN. I would think App::Ack is a good example. Maybe one of the App::* distributions from my CPAN directory (SMUELLER).
You could dump %INC at the end of your script. It will contain all used and required modules. But of course, this will only be helpful if you don't require modules conditionally (require Foo if $bar).
For quick-and-dirty, infrequent use, the %INC is the best way to go. If you have to do this with continuous integration testing or something more robust, there are some other tools to help.
Steffen already mentioned the Module::ScanDeps.
The code in Test::Prereq does this, but it has an additional layer that ensures that your Makefile.PL or Build.PL lists them as a dependency. If you make your scripts look like a normal Perl distribution, that makes it fairly easy to check for new dependencies; just run the test suite again.
Aside from that, you might use a tool such as Module::Extract::Use, which parses the static code looking for use and require statements (although it won't find them in string evals). That gets you just the modules you told your script to load.
Also, once you know which modules you loaded, you can combine that with David Cantrell's CPANdeps tool that has already created the dependency tree for most CPAN modules.
Note that you also have to think about optional features too. Your code in this case my not have them, but sometimes you don't load a module until you need it:
sub foo
{
require Bar; # don't load until we need to use it
....
}
If you don't exercise that feature in your trial run or test, you won't see that you need Bar for that feature. A similar problem comes up when a module loads a different set of dependency modules in a different environment (say, mod_perl or Windows, and so on).
There's not a good, automated way of testing optional features like that so you can get their dependencies. However, I think that should be on my To Do list since it sounds like an interesting problem.
Another tool in this area, which is used by Dist::Zilla and its AutoPrereqs plugin, is Perl::PrereqScanner. It installs a scan-perl-prereqs program that will use PPI and a few plugins to search for most kinds of prereq declaration, using the minimum versions you define. In general, I suggest this over scanning %INC, which can bring in bogus requirements and ignores versions.
Today I develop my Perl apps as CPAN-like distributions using Dist::Zilla that can take care of the dependencies through the AutoPrereq plugin. Another interesting piece of code in this area is carton.