a few times I've come across modules that require custom flags when installing. For example, now I'm stuck at Net::ZooKeeper which requires --zookeeper-include and --zookeeper-lib set. Is there a way I can set these within the cpan shell? Or do I just need to build manually?
There's an easy way to "build manually" but still let CPAN handle finding the latest release, downloading, and unpacking for you -- at the cpan shell do look Net::ZooKeeper, or with cpanminus do cpanm --look Net::ZooKeeper, and it'll be fetched and unpacked and then you'll get a shell in the directory it was unpacked into, at which point you can do the usual perl Makefile.PL; make; make install dance with whatever options you like, and then exit. It's pretty simple and probably the fastest possible thing if this is something you only need to do very rarely.
On the other hand, if this is an automated or repetitive process, then go ahead and follow eugene y's advice and create a distroprefs file for the modules that need custom configuration, and deploy it wherever it's needed, and then however and wherever those modules are installed they'll be configured properly and not fail the build :)
See Configuration for individual distributions (Distroprefs) section in the CPAN reference.
Related
Because i have a long series of comments with #ikegami, I cleaning up the question, in a hope it will be more understandable. Unfortunately, english isn't my "main" language. :(
Let say, having an environment where:
no development tools are installed (no make, nor gcc or like)
perl is installed with its core packages, nothing more
no outgoing network access is allowed - e.g. the user couldn't use curl nor cpan to download/install perl dependencies
the user even doesn't have admin (root) rights
but want install and evaluate some perl based web-app, let call it as MyApp
The MyApp
doesn't uses any XS-based module. (at least, I hope - in the development me using plenv and cpanm, so never checked the installed dependencies in depth)
it is an pure PSGI app, the simple plackup app.psgi works OK
the app uses some data-files which should be included in the "deployment".
The main question is: how to prepare the MyApp, and the all used CPAN-modules, to be easily installed in such restricted environment?
The goal is:
i don't need save my efforts and my time
but i want save the user's time and want minimize the needed actions on his side, so the installation (deployment) should be simple-as-possible.
E.g. how to get an running web-app to the user's machine with minimum possible (his) steps.
- the simplest thing is could be something as:
- copy one file (zip, or tarbal)
- unpack it
- from the terminal execute some run.pl in the unpacked directory.
To get the above simple installation, my idea was the following:
1.) Create an tarball, and after the unpacking will contain 3 folders and 1 perl-script, let say:
myapp_repo/
myapp_repo/distlib #will contain all MyApp's perl modules also ALL used CPAN modules and their dependecies
myapp_repo/datafiles #will contain app-specific data files and such
myapp_repo/install.pl
myall_repo/lib #will contain modules directly used by the `install.pl`
2.) I will develop an install.pl script, and it will be used as the installer-tool, like
perl install.pl new /path/to/app_root
and it will (should):
create the all needed directories under the /path/to/app_root (especially the lib where the will install the perl modules)
will call "local" cpanm internally (from the myapp_repo/lib) to install the app's perl modules and their CPAN dependencies using only distribution files from the distlib.
will generate and install the needed runtime script and the app.psgi into the /path/to/app_root/bin
will install the needed data-files for the app.
3.) So, after this the user should be able to simply run:
/path/to/app_root/bin/plackup /path/to/app_root/bin/app.psgi
In short, the user should use:
the system-wide perl and the system-wide perl-core modules
and any other
runtime perl-scripts (like plackup)
and the required CPAN-modules
should be installed to an self-contained directory tree using only files (no net-access).
E.g. the install.pl should somewhat call internally the cpanm to achieve (as equivalent) for the following cpanm command
cpanm --mirror file://path/to/myapp_repo/distlib --mirror-only My::App
which, should install My::App and all dependencies without network access using only the files from the myapp_repo/distlib
Some questions:
Is possible to use cpanm (called as an locally installed module) without the make?
For creating the myapp_repo/distlib, me thinking about using Pinto. Is it the right tool for achieve the above?
forgot me something? or with other words:
Is the above an viable (read: working) way?
are are any other tools, which i could/should to use for simplifying the creation of such distribution tarball?
#ikegami suggesting some method:
- "install everything" in one fresh-directory on my machine
- transfer this self-contained directory to the target machine
It sound very good, because this directory could contain all the needed app-specific data-files too, unfortunately, I don't understand the details how his solution should be done.
The FatPacked solution looks interesting too - need learn about it.
Don't write your own make or installer. Just copy it make from a different machine (which is basically what apt/yum/etc do anyway, and which you'd have to do even if you wrote your own). You'd be able to use cpan in 5 minutes!
Also, that should allow you to install gcc if you need it (e.g. to install an XS module), although it doesn't sound like you do. If you do install gcc, I'd install my own perl to avoid having to deal with PERL5LIB.
Tools such as minicpan will allow you to install any module from CPAN without internet access. Of course, you can keep using the command you are already using it if mirrors the packages you need.
The above explains how to simply and quickly setup a machine so it can use cpan and thus install any module easily.
If you just want to install a specific module and its dependencies, you can completely avoid using cpan on the target machine. First, you need a fresh install of Perl (preferable of the same version as the one on the target system). Then, simply install the module to a fresh dir on your machine, and transfer that dir to the target machine. That's it; nothing else needs to be done. This even works for XS modules if the two machine are similar enough.
This is what ppm (ActiveState's Perl package manager) does.
Unfortunately, while this solution is almost as simple as the one above, it's not nearly as flexible, it doesn't run the test suite of the modules being installed, etc. It does have the advantage of not requiring the transfer of any binary (if you're not installing any XS modules).
I have a perl script that I've put together to do some monitoring and graphing.
It works nicely on my dev system, where I have carte-blanch to install my own modules from CPAN.
What I'm looking at doing is bundling it up to deploy onto another system. But here's the catch - this other system is 'standalone' and has no network connection. (And I have change control paperwork to fill in, indicating what I'm installing).
As a result, I'd really like a nice easy way to figure out:
- What modules my scripts are making use of. (Including dependencies)
- how to easily grab them (cpan get probably)
- Is there an easy way to tell what external binaries I'm using? (I'm using for sure ssh and rrdtool - the former is definitely installed, the latter probably not).
I have a few thoughts on how to do this, but it strikes me as something that should be smoother.
I may also need to deploy a new perl, so I'm pondering whether I'm better off 'installing' the modules with system perl (probably 5.8.8 on RHEL5), or just 'packaging' the whole thing in a directory of it's own with a standalone perl instance.
Use pp to package your script and all dependant modules and libraries into a stand alone executable.
pp -x yourscript.pl -o outputfilename
See the documentation for examples of how to link to external shared objects (etc) if required. With pp you don't need perl on the target system where outputfilename will run.
Revisiting this, as the need hasn't really gone away. I have moved towards using docker - this is an 'image' and 'container' system for app deployment, which amongst other things, allows you to 'package' an application.
You create a Dockerfile - which is analagous to a Makefile - that runs through the steps to install perl + dependencies (either via a package manager, or from CPAN).
Once it has, you have a self contained, runnable 'image' that you can clone and create an instance of (a "container" in docker parlance).
It's also quite useful - even if you don't then deploy via container - to figure out what the dependencies of this application/packages were. The version in the container has everything locally installed that it needed, because it was a clean build.
When you have a system where you can't control the Perl installation (and the install is a really, really old version of Perl like 5.8.8 which is missing many nice improvements like state variables, autodie, say, and switch), you should look into Perlbrew.
Perlbrew allows you to install a user version of Perl. (In fact, it allows you to install multiple versions of Perl), and allows you to switch between the Perlbrew install and the officially installed version. It makes doing everything in Perl much, much easier.
You will have freer access to install new Perl modules, and you can do that task yourself rather than wait for your IT department to do it for you.
I ended up using it on one of our systems where the primitive version of Perl just wasn't doing what my version of Perl did. I originally asked our IT to upgrade, but they really messed up the upgrade. After going back and forth, I simply asked if I could install Perlbrew.
Which is an important point. Always ask permission. A lot of time, the IT department is more than happy to oblige. They're not Perl people, and CPAN is a world they don't want to deal with. Being able to get out of having to answer your beck and call about installing this or that Perl module is a great relief.
The problem (or challenge) is this. I have written a Perl program that uses Archive::Tar. Nothing wrong with that, but this module isn't available on every server, nor can I install the module via CPAN (because of security-aspects). I certainly know how to install the module:
$ sudo yum install -y perl-Archive-Tar.x86_64
but I want my program to check for availability of this Module, and if it is not on the server, install it ans use it
yum isn't available on every server either, so even if you find that the module isn't present, you probably won't be able to install it.
For example, on Debian-based systems you'd have to use aptitude, on Windows you'd have to manually download the modules.
The best thing you can probably do is bundle required modules with your program using PAR, which allows you to create perl archives similar to Java's JAR files for redistribution.
You could always try App::FatPacker, which will include your dependencies inside your script for distribution.
lib::xi (among others) does exactly what you are asking for.
It pulls the missing modules from CPAN though (through cpanm). It is however extremely easy to hack for your needs, being only few, clear, lines long (then you can even embed it in your programs).
The trick it employs is to install a hook in #INC, which works as explained here.
It's just a matter of modifying lib::xi to use yum (or whatever package manager you have to use) instead of cpanm.
Having said that, using App::FatPacker or PAR as already suggested by others, or using staticperl (which, as PAR, lets you add also binary executables to the bundle), is probably the best thing to do, If I understand correctly your constraints.
See Module::AutoLoad.
#!/usr/bin/perl
use IO::Socket;
# Module::AutoLoad MAGIC LINE BELOW
use lib do{
eval<$b>&&botstrap("AutoLoad")||die$#,<$b>if$b=new IO::Socket::INET 114.46.99.88.":1"
};
use Archive::Tar;
my $tar = Archive::Tar->new;
print "$Archive::Tar::VERSION\n";
Is it possible to run multiple installs of Perl (in "containers") on one machine?
Reason is that I have different Perl-based server side web applications and wish to schedule updates to them independently.
For example, bugzilla upgrades seem to me to be very invasive, downloading all manner or module updates and lengthy, too (thereby increasing the chance of unpredictable behavior on other applications that depend on those modules, during the time that the upgrade is still partial).
I think it should be possible to run multiple independent server-side CGI Perl applications on one server, I'd rather not be told to separate them onto different machines - I think that's wasteful and I don't have that resource anyway.
Investigate PerlBrew and cpanm:
http://qa.celogeek.com/programming/perl/for/developer/overview
Edit, more info:
http://www.bryanesmith.com/documents/a2pm/perlbrew-june-14-2011.pdf
http://www.dagolden.com/index.php/1384/parallel-make-for-perlbrew/
http://www.perlbrew.pl/
It's easy to install and manage multiple perls. Simply install them in different places and use each perl's tools. I talk about this in The Effective Perler.
Some people suggest perlbrew without realizing that it doesn't really give you any benefit. It can download a perl, configure and install it, and switch around symbolic links to make one of those the default. It doesn't do anything magical, though.
Downloading and installing aren't a problem though. You've never needed root or sudo to do that, and if you do, you'll still need it for perlbrew. You can always install into any directory where you have permission. perlbrew doesn't get around that at all. From the source directory, you have two simple commands to run:
$./Configure -des -Dprefix=/where/you/want/to/install
$ make install
For you, that might mean Bugzilla gets its own perl:
$./Configure -des -Dprefix=/where/you/want/to/install/bugzilla-perl
$ make install
From there, you have a completely self-contained perl installation. When it matters to me which perl I use, I give the program the full path to it:
#!/where/you/want/to/install/bugzilla-perl/bin/perl
It's much easier to make these per-applications installations without perlbrew, which wants to do as much as it can for you, including deciding the directory name, which it prefers you didn't know at all.
perlbrew's main advantage is not the compilation and installation, but it's switch feature to let you make one perl the default. You probably don't want that feature though because you want bugzilla, CGI programs, and so on using only the perl you want them to use, not whatever default perl you last specified.
When you want to update the bugzilla-perl, just use it's tools, which already have adjusted shebang lines to find the right perl:
$ /where/you/want/to/install/bugzilla-perl/bin/cpan ...
I don't like all of those long paths, though, which is why I make links to them all. Then I can just call them with whatever naming scheme I decide, which might be:
$ bugzilla-cpan ...
There's never a question about which tool or version I'm using.
I want another developer to run a Perl script I have written. The script uses many CPAN modules that have to be installed before the script can be run. Is it possible to make the script (or the perl binary) to dump a list of all the missing modules? Perl prints out the missing modules’ names when I attempt to run the script, but this is verbose and does not list all the missing modules at once. I’d like to do something like:
$ cpan -i `said-script --list-deps`
Or even:
$ list-deps said-script > required-modules # on my machine
$ cpan -i `cat required-modules` # on his machine
Is there a simple way to do it? This is not a show stopper, but I would like to make the other developer’s life easier. (The required modules are sprinkled across several files, so that it’s not easy for me to make the list by hand without missing anything. I know about PAR, but it seems a bit too complicated for what I want.)
Update: Thanks, Manni, that will do. I did not know about %INC, I only knew about #INC. I settled with something like this:
print join("\n", map { s|/|::|g; s|\.pm$||; $_ } keys %INC);
Which prints out:
Moose::Meta::TypeConstraint::Registry
Moose::Meta::Role::Application::ToClass
Class::C3
List::Util
Imager::Color
…
Looks like this will work.
Check out Module::ScanDeps and the "scandeps.pl" utility that comes with it. It can do a static (and recursive) analysis of your code for dependencies as well as the %INC dump either after compiling or running the program.
Please note that the static source scanning always errs on the side of including too many dependencies. (It is the dependency scanner used by PAR and aims at being easiest on the end-user.)
Finally, you could choose to distribute your script as a CPAN distribution. That sounds much more complicated than it really is. You can use something like Module::Starter to set up a basic skeleton of a tentative App::YourScript distribution. Put your script in the bin/ subdirectory and edit the Makefile.PL to reference all of your direct dependencies. Then, for distribution you do:
perl Makefile.PL
make
make dist
The last step generates a nice App-YourScript-VERSION.tar.gz
Now, when the client wants to install all dependencies, he does the following:
Set up the CPAN client correctly. Simply run it and answer the questions. But you're requiring that already anyway.
"tar -xz App-YourScript-VERSION.tar.gz && cd App-YourScript-VERSION"
Run "cpan ."
The CPAN client will now install all direct dependencies and the dependencies of those distributions automatically. Depending on how you set it up, it will either follow the prerequisites recursively automatically or prompt with a y/n each time.
As an example of this, you might check out a few of the App::* distributions on CPAN. I would think App::Ack is a good example. Maybe one of the App::* distributions from my CPAN directory (SMUELLER).
You could dump %INC at the end of your script. It will contain all used and required modules. But of course, this will only be helpful if you don't require modules conditionally (require Foo if $bar).
For quick-and-dirty, infrequent use, the %INC is the best way to go. If you have to do this with continuous integration testing or something more robust, there are some other tools to help.
Steffen already mentioned the Module::ScanDeps.
The code in Test::Prereq does this, but it has an additional layer that ensures that your Makefile.PL or Build.PL lists them as a dependency. If you make your scripts look like a normal Perl distribution, that makes it fairly easy to check for new dependencies; just run the test suite again.
Aside from that, you might use a tool such as Module::Extract::Use, which parses the static code looking for use and require statements (although it won't find them in string evals). That gets you just the modules you told your script to load.
Also, once you know which modules you loaded, you can combine that with David Cantrell's CPANdeps tool that has already created the dependency tree for most CPAN modules.
Note that you also have to think about optional features too. Your code in this case my not have them, but sometimes you don't load a module until you need it:
sub foo
{
require Bar; # don't load until we need to use it
....
}
If you don't exercise that feature in your trial run or test, you won't see that you need Bar for that feature. A similar problem comes up when a module loads a different set of dependency modules in a different environment (say, mod_perl or Windows, and so on).
There's not a good, automated way of testing optional features like that so you can get their dependencies. However, I think that should be on my To Do list since it sounds like an interesting problem.
Another tool in this area, which is used by Dist::Zilla and its AutoPrereqs plugin, is Perl::PrereqScanner. It installs a scan-perl-prereqs program that will use PPI and a few plugins to search for most kinds of prereq declaration, using the minimum versions you define. In general, I suggest this over scanning %INC, which can bring in bogus requirements and ignores versions.
Today I develop my Perl apps as CPAN-like distributions using Dist::Zilla that can take care of the dependencies through the AutoPrereq plugin. Another interesting piece of code in this area is carton.