I've written any number of perl modules in the past, and more than a few stand-alone perl programs, but I've never released a multi-file perl program into the wild before.
I have a perl program that is almost at the beta stage and is going to be released open source. It requires a number of data files, as well as some external perl modules -- some I've written myself, and some from CPAN -- that I'll have to bundle with it so as to ensure that someone can just download my program and install it without worrying about hunting for obscure modules.
So, it sounds to me like I need to write an installer to copy all the files to standard locations so that a user can easily install everything. The trouble is, I have no idea what the standard practice would be for this. I have found lots of tutorials on perl module standards, but none on perl program standards.
Does anyone have any pointers to standard paths, installation proceedures, etc, for perl programs? This is going to be complicated by the fact that the program is multi-platform. I've been testing it in Linux, but its designed to work equally well in Windows.
Take a look at PAR and PAR::Packer. You can bundle all of your requirements (even non-Perl requirements) into one file. With PAR::Packer, the user doesn't even need to have Perl installed for it to work.
You might also look at how the various App::* distributions are setup.
The standard installers for modules (ExtUtils::MakeMaker, Module::Build, Module::Install) also work the same way for scripts.
Using such a standard Perl tool will help you to:
distribute your application on the
CPAN (and you'll benefit from
automated tests on various platforms
by CPAN Testers), and so your app
will be installable (with all its dependencies) from the CPAN
shell
help packagers of Linux/BSD distributions to make packages for your product
Related
I've done some research and it seems like ActivePerl had issues with earlier releases of it's product with certain CPAN modules not installing properly. However, I'm running the 5.14.x version and I've not had any problems.
According to some quotes I've seen:
ActivePerl is 100% compatible with the reference distribution of Perl.
Code tested with ActivePerl will run on any Perl installation that has the appropriate extensions installed.
I assume that the first statement refers to the standard modules you get with the Perl installation and for the second, I'm not sure what they are saying?
In any event, is there any way to find out how compatible ActivePerl is with the current CPAN modules or is that something that isn't known? I just don't want to spend time with it, only to have to switch to something like Strawberry Perl next month to avoid CPAN module build failures for the more common modules.
If ActivePerl is compatible with say 80% or higher with the CPAN modules I would feel more comfortable about using it, but I couldn't find any information on this.
I doubt a generic statistic will be that useful. In general I would expect all "pure perl" modules should work more or less out of the box. Keep in mind however that certain perl modules really are interfaces to lower level linux/unix style shared libraries (dlls in Windows terms), where availability is less certain. In my experience (having written a few perl applications being hosted on Windows, against my advice) most things will work, and/or are fairly easy to work around, and both ActiveState and Strawberryperl seem to have decent support for most common modules.
The first statement doesn't refer to modules at all. It says that ActivePerl is not based on Perl, it is Perl. As such, anything that will run on Perl will also run on ActivePerl.
This also means that all modules on CPAN are compatible with ActivePerl since ActivePerl is Perl.
Whether a module is compatible with Windows is an entirely different question, and it can only be answered on a module-by-module basis.
The second statement points out that if you had a script or module that runs on a pristine ActivePerl, it might not necessarily run on a pristine Perl because ActiveState includes modules in its distribution that aren't core modules (e.g. LWP). But all you'd need to do to make the script or module run on the other distribution is to install those modules.
You can check on the availability of PPM modules at http://code.activestate.com/ppm/. For example, one module that doesn't work well through PPM is PAR::Packer.
Being very new to Perl but not to dynamic languages, I'm a bit surprised at how not straight forward the manage of modules is.
Sure, cpan X does theoretically work, but I'm working on the same project from three different machines and OSs (at work, at home, testing in an external environment).
At work (Windows 7) I have problem using cpan because of our firewall that makes ftp unusable
At home (Mac OS X) it does work
In the external environment (Linux CentOs) it worked after hours because I don't have root access and I had to configure cpan to operate as a non-root user
I've tried on another server where I have an access. If the previous external environment is a VPS and so I have a shell access, this other one is a cheap shared hosting where I have no way to install new modules other than the ones pre-installed
At the moment I still can't install Template under Windows. I've seen that as an alternative I could compile it and I've also tried ActiveState's PPM but the module is not existent there.
Now, my perplexity is about Perl being a dynamic language. I've had all these kind of problems while working, for example, with C where I had to compile all the libraries for all the platform, but I thought that with Perl the approach would have been very similar to Python's or PHP's where in 90% of the cases copying the module in a directory and importing it simply works.
So, my question: if Perl's modules are written in Perl, why the copy/paste approach will not work? If some (or some part) of the modules have to be compiled, how to see in CPAN if a module is Perl-only or it relies upon compiled libraries? Isn't there a way to download the module (tar, zip...) and use cpan to deploy it? This would solve my problem under Windows.
Now, Perl is a dynamic language, but that doesn't imply that everything that people write is portable across platforms. That's not the fault of the language. It's not even the fault of the programmer. Some things, like Win32::OLE shouldn't work on Unix. :)
Other dynamic languages will have some of the same problems. If you have to compile C code, you won't be able to merely copy files to another machine. Some distributions configure the code slightly differently depending on your operating system, etc.
Even if you could copy files, you have to ensure that you copy all of the files that you need. Do you know everything that you need for a particular module? Remember, many of them have dependencies.
Most of the problems you're having aren't anything to do with the language. You're having trouble with the tools. If you want a zero conf CPAN tool that makes all the decisions for you, try cpanminus. It's mostly the same thing that you'd get out of cpan (although different code), but it makes all of the decisions for you. It doesn't run any of the distribution tests, and it installs into your user directory. When you need something that gives you control, come back to cpan.
In the external environment (Linux CentOs) it worked after hours because I don't have root access and I had to configure cpan to operate as a non-root user
This is one of those times when it helps to know The Trick. In this case local::lib, which lets you configure a non-root install area and all the ENV variables in about three minutes.
if perl's modules are written in perl, why the copy/past approach will not work?
Some are written in Pure Perl, but many are written partially in C (using Perl's XS API) for efficiency.
Sometimes you end up with situations like JSON::XS, JSON::PP and JSON::Any to autoselect the best one that is installed.
Isn't there a way to download the module (tar, zip...) and use cpan to deploy it?
The cpan program is all about getting things from the Internet. You can download the package (there will be a link along the lines of "Download: CGI.pm-3.49.tar.gz" on the right hand side of the CPAN page), untar it, then
perl Makefile.PL
make
make install
You would probably be better off configuring your cpan installation to use only HTTP sources (in the urllist config option). Possibly going to far as to create a mini CPAN mirror inside your network.
I have an app that I pack into "binary" form using PerlApp for distribution. Since my clients want a simple install for their Win32 systems, this works very nicely.
Now a client has decided that they need to run all unit tests, like in a standard install. However, they still will not install a normal Perl.
So, I find myself in need of a way to package my unit tests for operation on my client's systems.
My first thought was that I could pack up prove in one file and pack each of my tests separately. Then ship a zip file with the appropriate structure.
A bit of research showed that Test::Harness::Straps invokes perl from the command line.
Is there an existing tool that helps with this process?
Perhaps I could use PAR::Packer's parl tool to handle invocation of my test scripts.
I'm interested in thoughts on how to apply either PAR or PerlApp, as well as any thoughts on how to approach overriding Test::Harness and friends.
Thanks.
Update: I don't have my heart set on PAR or PerlApp. Those are just the tools I am familiar with. If you have an idea or a solution that requires a different packager (such as Cava Packager), I would love to hear about it.
Update 2: tsee pointed out a great new feature in PAR that gets me close. Are there any TAP experts out there that can supply some ideas or pointers on where to look in the new Test::Harness distribution?
I'm probably not breaking big news if I tell you that PAR (and probably also perlapp) aren't meant to package the whole test suite and plethora of CPAN-module build byproducts. They're intended to package stand-alone applications or binary JAR-like module libraries.
This being said, you can add arbitrary files to a PAR archive (both to .par libraries and stand-alone .exe's) using pp's -a switch. In case of the stand-alone executable, the contents will be extracted to $ENV{PAR_TEMP}."/inc" at run-time.
That leaves you with the problem of reusing the PAR-packaged executable to run the test harness (and letting that run your executable as a "perl"). Now, I have no ready and done solution for that, but I've recently worked on making PAR-packaged executables re-useable as more-or-less general purpose perl interpreters. Two gotchas before I explain how you can use that:
Your application isn't going to magically be called "perl" and add itself to your $PATH.
The "reuse" of the application as a general purpose perl requires special options and does not currently support the normal perl options (those in perlrun). It can simply run an external perl script of your choice.
Unfortunately, the latter problem is what may kill this approach for you. Support for perl command line options is something I've been thinking about, but won't implement any time soon.
Here's the recipe how you get PAR with "reusable exe" support:
Install the newest version of PAR from CPAN.
Install the newest, developer version of PAR::Packer from CPAN (0.992_02 or 03).
Add the "--reusable" option to your pp command line.
Run your executable with the following options to run an external script "foo.pl":
./myapp --par-options --reuse foo.pl FOO-PL-OPTIONS-HERE
Unfortunately, how you're going to teach Test::Harness that "./myapp --par-options --reuse" is a perl interpreter is beyond me.
Cava Packager allows you to package test scripts with your packaged executables. This is primarily to allow you to run tests against the packaged code before distribution. However the option is there to also distribute the tests and test capability to your end users.
Note: As indicated by my name, I am affiliated with Cava Packager.
This is a follow-up to my previous question about developing Perl applications. Let’s say I develop an application as a CPAN module using Module::Install. Now I upload the code to the production server, say using a git push, and I would like to install the application dependencies listed in Makefile.PL. If I simply run cpan ., the thing tries to install the application like a regular Perl module, ie. starts to copy the modules and documentation to standard Perl directories all over the system.
Is this the way it’s supposed to be? Do you install the application into the standard Perl directories? I am used to having my Perl applications in one directory with separate lib. Otherwise it seems I’d have to manage a lot of other things, like installing the resources somewhere on path etc. If I just want to install the deps declared in Makefile.PL and run the application tests to make sure everything works, what should I do?
(Is this documented somewhere? I mean, is there something like best practice for deploying and updating non-trivial Perl applications? Or is everybody doing this his own way?)
I might be misunderstanding, but I think what you're looking for is
perl Makefile.PL
make installdeps
If you are using Module::Install, you're really using ExtUtils::MakeMaker behind the scenes. You can use all of the MakeMaker features and the targets it provides. Although the documentation doesn't show every feature, there are some valuable things to be found in the generated Makefile.
However, MakeMaker is old news and most everyone has asked Santa Claus for it to disappear. If you want better control, including creating your own targets and process, Module::Build is orders of magnitude easier to work with as well as cross-platform (even if that just means not using a different make, gmake, or whatever on the same OS on different boxes). If you deviate from the normal, consumer-grade installation process, you're life will be easier without MakeMaker.
Some people appreciate the brevity of the Module::Install build file, but once constructed, you don't spend a lot of time messing with your build file so it's not that much of a real benefit. When the little benefit you get locks you into MakeMaker, it's not a win at all.
A 2014 update: Module::Build has now fallen out of favor and needs a maintainer. It never quite got to the point where people could use it to build and distribute XS modules. It was deprecated in Perl v5.19 although you can still get it from CPAN.
You could look at Module::ScanDeps to generate a list of dependent modules for installing. Or Par::Packer for packaging up the whole thing as an "app".
I have several scripts that I supply to users as tools for their engineering projects. These scripts contain a lot of duplicated code. I want to extract this code and maintain it in a set of modules. However, in order for this to work, the users would have to install these modules. I don't want to have to tell my users to "make install", etc., as I'm sure none of them would have the patience for that.
I understand that one option is to package everything together with PAR, but ideally the user would be able to open up && edit these scripts if they need to, just like they can now. They also need to be able to move them to whatever directory they want, and I don't want them to have to move a bunch of library files as well.
Is it possible to make a double-click file that installs some bundled Perl modules?
I distribute my script as modules, and then use the normal CPAN toolchain to install them. See my articles Scripts as Modules and Automating Script Distributions with scriptdist. Once you have them in a conventional module distribution, you can install them from their current directory with cpan:
% cpan . # install from distribution in the current directory
Depending on how complex your situation is, you might want to create a DPAN, which is a private version of CPAN that your CPAN tools can draw from. You put all of your private modules there, point your CPAN tools at it, and everything happens as it does with a real CPAN mirror. You never have to share your work though.
yeah package with either PAR or Shipwright (not sure about binaries). Also use scandeps.pl along the way.
If the users are using systems with a packaging system (dpkg, cygwin, etc.), consider using that.
If you don't mind spending some green, one of the better bet is Perl Dev Kit from Activestate.
From their own description of the product,
Develop and deploy your Perl programs
to anyone on any platform with
PerlApp's new cross-platform
wrapping.
Deliver code as
executables or as Windows Services,
ActiveX components, .NET assemblies
or in the System Tray.
Easily create
MSI files using Perl code.
You may also consider using sparrow - scripts distribution system. Sparrow plays nice with Perl as it writen on it. It supports CPAN modules dependencies via carton tool.
PS Disclaimer - I am the tool author