I have several scripts that I supply to users as tools for their engineering projects. These scripts contain a lot of duplicated code. I want to extract this code and maintain it in a set of modules. However, in order for this to work, the users would have to install these modules. I don't want to have to tell my users to "make install", etc., as I'm sure none of them would have the patience for that.
I understand that one option is to package everything together with PAR, but ideally the user would be able to open up && edit these scripts if they need to, just like they can now. They also need to be able to move them to whatever directory they want, and I don't want them to have to move a bunch of library files as well.
Is it possible to make a double-click file that installs some bundled Perl modules?
I distribute my script as modules, and then use the normal CPAN toolchain to install them. See my articles Scripts as Modules and Automating Script Distributions with scriptdist. Once you have them in a conventional module distribution, you can install them from their current directory with cpan:
% cpan . # install from distribution in the current directory
Depending on how complex your situation is, you might want to create a DPAN, which is a private version of CPAN that your CPAN tools can draw from. You put all of your private modules there, point your CPAN tools at it, and everything happens as it does with a real CPAN mirror. You never have to share your work though.
yeah package with either PAR or Shipwright (not sure about binaries). Also use scandeps.pl along the way.
If the users are using systems with a packaging system (dpkg, cygwin, etc.), consider using that.
If you don't mind spending some green, one of the better bet is Perl Dev Kit from Activestate.
From their own description of the product,
Develop and deploy your Perl programs
to anyone on any platform with
PerlApp's new cross-platform
wrapping.
Deliver code as
executables or as Windows Services,
ActiveX components, .NET assemblies
or in the System Tray.
Easily create
MSI files using Perl code.
You may also consider using sparrow - scripts distribution system. Sparrow plays nice with Perl as it writen on it. It supports CPAN modules dependencies via carton tool.
PS Disclaimer - I am the tool author
Related
I have a perl script that I've put together to do some monitoring and graphing.
It works nicely on my dev system, where I have carte-blanch to install my own modules from CPAN.
What I'm looking at doing is bundling it up to deploy onto another system. But here's the catch - this other system is 'standalone' and has no network connection. (And I have change control paperwork to fill in, indicating what I'm installing).
As a result, I'd really like a nice easy way to figure out:
- What modules my scripts are making use of. (Including dependencies)
- how to easily grab them (cpan get probably)
- Is there an easy way to tell what external binaries I'm using? (I'm using for sure ssh and rrdtool - the former is definitely installed, the latter probably not).
I have a few thoughts on how to do this, but it strikes me as something that should be smoother.
I may also need to deploy a new perl, so I'm pondering whether I'm better off 'installing' the modules with system perl (probably 5.8.8 on RHEL5), or just 'packaging' the whole thing in a directory of it's own with a standalone perl instance.
Use pp to package your script and all dependant modules and libraries into a stand alone executable.
pp -x yourscript.pl -o outputfilename
See the documentation for examples of how to link to external shared objects (etc) if required. With pp you don't need perl on the target system where outputfilename will run.
Revisiting this, as the need hasn't really gone away. I have moved towards using docker - this is an 'image' and 'container' system for app deployment, which amongst other things, allows you to 'package' an application.
You create a Dockerfile - which is analagous to a Makefile - that runs through the steps to install perl + dependencies (either via a package manager, or from CPAN).
Once it has, you have a self contained, runnable 'image' that you can clone and create an instance of (a "container" in docker parlance).
It's also quite useful - even if you don't then deploy via container - to figure out what the dependencies of this application/packages were. The version in the container has everything locally installed that it needed, because it was a clean build.
When you have a system where you can't control the Perl installation (and the install is a really, really old version of Perl like 5.8.8 which is missing many nice improvements like state variables, autodie, say, and switch), you should look into Perlbrew.
Perlbrew allows you to install a user version of Perl. (In fact, it allows you to install multiple versions of Perl), and allows you to switch between the Perlbrew install and the officially installed version. It makes doing everything in Perl much, much easier.
You will have freer access to install new Perl modules, and you can do that task yourself rather than wait for your IT department to do it for you.
I ended up using it on one of our systems where the primitive version of Perl just wasn't doing what my version of Perl did. I originally asked our IT to upgrade, but they really messed up the upgrade. After going back and forth, I simply asked if I could install Perlbrew.
Which is an important point. Always ask permission. A lot of time, the IT department is more than happy to oblige. They're not Perl people, and CPAN is a world they don't want to deal with. Being able to get out of having to answer your beck and call about installing this or that Perl module is a great relief.
I wrote a script for my company and I am using some libraries I obtained from CPAN. My manager wanted me to consolidate and remove the extra libraries - which is a little funny because I include them for the script to work.
A few notes:
I do not have root access on this server nor can I request access
To use CPAN modules w/o root I have them installed to my user directory
To allow other users to run my scripts I usually include a folder called 'libs' and inside of my script's directory and in the script I have: use 'libs'; at the top before I use my CPAN modules.
The only solution I have right now is to literally put the contents of the perl modules inside of my perl script. However I want to give credit where it is due and also not get in trouble for including opensource code w/o proper credit to its authors and organizations.
Therefore, how should I go about this? I am not trying to get away with anything.. I honestly want to go about doing this the right way.
All three modules say "licensed under the same terms as Perl itself" but I feel like it shouldn't be this easy.
I would also like to explore any other ideas too!
The modules are:
Text::Table
Text::Aligner
Term::ANSIColor
Is using PAR Packager an option for you? That would generate a standalone executable.
If the modules are pure Perl modules, you may be able to simply append the code (including those package statements) into your program. I'd also include the POD which would include the copyright statements and the names of the authors too. That should satisfy the Artistic License Requirement (but may not satisfy GNU licensing requirements).
Another possibility is to use Perlbrew which will allow you to install a user version of Perl on the system. This way, you can install CPAN modules without needing Administrative permission, and you can tell other users to use Perlbrew too.
I use it because I can install and switch between various versions of Perl which allows me to test my Perl scripts in various versions of Perl. I've also used it on our servers where I need a newer version of Perl or modules that weren't included in the standard release.
You need to get your IT approval before installing Perlbrew, but a lot of times they're relieved that they no longer have to be bothered with maintaining and installing CPAN modules for your use.
Interesting question & perspective. I don't understand what is against using libraries or modules, but I'll let your manager do the thinking ;-)
Regarding copyright, you're best to consult a lawyer if you want to be sure, but as far as I understand it, you can combine the work of others provided you retain the copyright notices. The combined work may not be covered by copyleft, so you may be able to use it commercially (i.e., distribute it without disclosing the source). But do check with a lawyer.
But, since you said you wanted to explore other ideas, App::Staticperl may be a solution? I do not have experience with it, but I tried it with a simple example and got a working executable.
App::Staticperl builds a stand-alone executable from the Perl interpreter with embedded CPAN modules. The steps I followed were roughly (you'll need to adapt, because obviously I couldn't test with your script):
latest version of App::Staticperl is 1.43: https://cpan.metacpan.org/authors/id/M/ML/MLEHMANN/App-Staticperl-1.43.tar.gz
either install the module via CPAN, or simply extract bin/staticperl from the tar - it's a standalone script
edit staticperl to change EMAIL and CPAN (optional, but you may want to change the CPAN mirror)
./staticperl install downloads and builds Perl; it ended with an error message on my box, but did produce a working Perl
./staticperl cpan enters an interactive CPAN prompt; install Text::Table, install Term::ANSIColor, and whatever else you need
./staticperl mkapp my_app --boot path/to/your/script -MText::Table -MText::Aligner -MTerm::ANSIColor
try the app: ./my_app - it will most likely fail with an error message about missing modules; repeat the previous step and include the missing modules in the -M flags
Good luck!
Can you reduce the unnecessary code (to satisfy your manager's concerns). Leave in tact the needed code in the file it came in - and give the author's credit within that module/package.
Eg: This was inspired (stolen) from Joe E Perl.
I develop and maintain a bioinformatics application suite of 50+ scripts and its deployment process is a mess:
Entire suite is in one big git repository. It has lots of CPAN dependencies, and dozens of internal modules as well.
Development platform is Linux.
Deployment platforms are Windows (20+ users), Mac (10+), Linux (2-3). Most are not 'power users'.
For windows, I have one installer (made with NSIS) for strawberry perl + required modules (ie, I installed strawberry on a windows box, installed all modules and zipped c:\strawberry), and another installer for the suite-- I did this b/c the suite is updated a lot more than the list of required modules.
For Mac, I bundle perl 5.14, all required cpan modules, and the application suite into a double-clickable installer. I don't use the system perl b/c it tends to be out of date. I bundle everything together unlike on windows b/c I suck at mac.
For Linux, I handle their installations manually since there are only a few of them, and they use different distros.
This is obviously a mess that grew organically over several generation of developers. Ideally I would like to create cpan-installable distributions out of the internal libraries and various groups of related scripts, and use module dependencies to let cpan install them for me.
But I'm not sure what the best approach for this is, b/c I'd still need distribute perl itself, would have to write some sort of non-command-line interface to CPAN, control the exact versions of 3rd party CPAN modules, point it by default to my "DarkPan" where I would store our modules, how I would push updates, etc. etc.
I don't think I can use PerlApp or Par since afaik those are for bundling single scripts, not an entire suite of them.
Any advice highly appreciated.
Besides the 3 platforms mentioned (more, if you count the Linux variants), you really have a couple different problems:
Deployment of a standard known-good Perl executable and libraries (CPAN modules).
Deployment of your Perl scripts and modules.
Once upon a time, I supported a large Solaris Perl installation. I tried for a while to stand up a Linux Perl installation "side-by-side", re-using the same CPAN modules. Didn't work. The big problem for me is that a fair number of Perl modules require compilation, which means they target a specific platform. I ended up just with 2 installs, and always remembered to install a new CPAN module in both areas.
We're now 100% Windows, so I don't have the same issue. However, we do run Perl off a shared network drive. All the users map this drive, and run a Registry script that associates .PL files with the network install of Perl. (See my answer to this other Perl question.)
So, besides the mapped drive and the Registry script, users don't need to install anything. Even the CPAN modules are picked up from the network. This solves item #1 (for Windows only users).
Same thing holds true for item #2: the scripts are stored on a network drive (same one) and the users run another Registry script to include the scripts folder in their search PATH. We edit our scripts in one area, and have a "Check-In 'n Release" ("CINR") that we use to, well, check-in and release scripts to the area the users point to. The users can double-click the scripts in Explorer, run them in DOS, or even better yet get them included in the contextual-menu in Explorer, etc. (Actually, we use a .NET application to map the drive and make all these settings for the user, but it can be done much simpler.)
So, how does this help with the other platforms, Linux and Mac? As I ran into with my Solaris/Linux experiment, I think you're stuck with different Perl installation for all 3 platforms, although you should be able to reach the same network drive for your Perl scripts and modules.
The Perl installation might even be OK on a network drive for the Linux users. It's probably easier for them than the Windows users. Mac users are tough. I administer a home Mac network, and I think network drives are very difficult to do in Mac OS X compared to other OSes. It should be as easy as in Linux since so much is the same, but there are very strange problems (for me) mapping NFS and SMB drives. AFP drives are a little easier for the user to map manually, but not so easy to map programmatically.
My Mac recommendation is to try using Platypus. It's definitely great at bundling scripts into a double-clickable application, although your interface options are limited to output only (no user input allowed during execution that I can tell). Not sure if you could put an entire Perl installation into the Platypus app or not, but if you could the paths figured out, you might be able to.
Good luck!
You may wish to check out CAVA packager. It can deal with multiple scripts in a single package.
I need to install my Perl-based software on networked machines which aren't connected to the internet. Therefore, I would like to download specific versions and/or latest versions of the Perl modules and I would also like to know if there is an install procedure required for these modules.
Background:
The machines aren't connected to the internet for security reasons and its deemed unnecessary also.
I would place the downloaded modules on a machine that I call the 'install server' and it contains my Perl based software and would also contain the local copies of the Perl modules.
I call a machine that I want to install my Perl-based software on, the 'target machine', also not connected to the internet. There can be several target machines, each can run this software that I want to install. I log onto the target machine and run an install script which would connect to the install machine via the local network to obtain the Perl-based software and dependent Perl modules and installs them.
So I need to know:
How/Where to get specific versions of Perl modules, e.g. CGI.pm etc
How to install these Perl modules. Is it a case of just placing them in a directory somewhere, e.g. a library path and making sure that this directory path is in the #INC library path environmental variable, if it is not already?
I would prefer not to have to do anything like make install etc. as part of installing the modules. I would like to modules to be pre-compiled or prepared as necessary so it is as simple as possible to install them. I want to avoid additional dependencies like make and its configuration, and having to parse its output to check whether it was successful.
Please help me by asking the above specific questions as I am not able to change the concept of 'install machine' and 'target machine' which aren't connected to the internet - I have to provide a solution that works within this arrangement.
The usual way to solve "I want to install stuff from CPAN but without network" problems is to use a minicpan as David Dorward wrote in his answer. But since you're going one step further, saying that you'd rather not do any real installation on the client (target) machines at all, and that you want to use precompiled modules if possible, I urge you to check out PAR and specifically PAR::Repository (server) and PAR::Repository::Client.
Since this approach needs some research before you're up to speed, I wouldn't suggest it for "I just need Foo.pm" like problems. Once you're talking about a handful of dependencies and at least a handful of clients, then it becomes a more appropriate solution.
For an outline of how it works, check out the slides of my talk at YAPC::EU 2008. It also hints at solutions to the bootstrapping problem of making the PAR::Repository::Client module available on the clients (hint: PAR can generate self-contained executables).
You can create a MiniCPAN that has just the latest versions of everything from CPAN. You can insert additional, non-public modules into it with CPAN::Mini::Inject. If you need to greater control over versions (i.e. not choosing the latest versions), you might want to create a DPAN.
With any of these solutions, you can configure your CPAN client to pull from your local source. That could be a directory you know ahead of time or something that you figure out dynamically, like a CD or a thumb-drive. It's just a matter of setting up the configuration correctly.
You might be able to get away with creating operating-system packages for most of your work, but that still means you have to compile them at least the first time.
1) How/Where to get specific versions of Perl modules, e.g. CGI.pm etc
http://search.cpan.org/
If you don't want the latest version, you can get an earlier version by following the link in the breadcrumbs.
http://img.skitch.com/20091209-bu7kt3bj65374k7iijfnhrue2y.png
2) How to install these Perl modules. Is it a case of just placing them
in a directory somewhere, e.g. a library path and making sure that this
directory path is in the #INC library path environmental variable, if
it is not already?
That sometimes work, but you really should go through the perl Makefile.PL && make && make test && make install process.
Doing this would require that you manually chase all the dependencies though. You would probably be better off with something like minicpan.
It seems that most of the installers for Perl are centered around installing Perl modules, not applications. Things like ExtUtils::MakeMaker and Module::Build are very well suited for modules, but require some additional work for Web Apps.
Ideally it would be nice to be able to do the following after checking out the source from the repository:
Have missing dependencies detected
Download and install dependencies from CPAN
Run a command to "Build" the source into a final state (perform any source parsing or configuration necessary for the local environment).
Run a command to install the built files into the appropriate locations. Not only the perl modules, but also things like template (.tt) files, and CGI scripts, JS and image files that should be web-accessible.
Make sure proper permissions are set on installed files (and SELinux context if necessary).
Right now we have a system based on Module::Build that does most of this. The work was done by done by my co-worker who was learning to use Module::Build at the time, and we'd like some advice on generalizing our solution, since it's fairly app-specific right now. In particular, our system requires us to install dependencies by hand (although it does detect them).
Is there any particular system you've used that's been particularly successful? Do you have to write an installer based on Module::Build or ExtUtils::MakeMaker that's particular to your application, or is something more general available?
EDIT: To answer brian's questions below:
We can log into the machines
We do not have root access to the machines
The machines are all (ostensibly) identical builds of RHEL5 with SELinux enabled
Currently, the people installing the machines are only programmers from our group, and our source is not available to the general public. However, it's conceivable our source could eventually be installed on someone else's machines in our organization, to be installed by their programmers or systems people.
We install by checking out from the repository, though we'd like to have the option of using a distributed archive (see above).
The answer suggesting RPM is definitely a good one. Using your system's package manager can definitely make your life easier. However, it might mean you also need to package up a bunch of other Perl modules.
You might also take a look at Shipwright. This is a Perl-based tool for packaging up an app and all its Perl module dependencies. It's early days yet, but it looks promising.
As far as installing dependencies, it wouldn't be hard to simply package up a bunch of tarballs and then have you Module::Build-based solution install them. You should take a look at pip, which makes installing a module from a tarball quite trivial. You could package this with your code base and simply call it from your own installer to handle the deps.
I question whether relying on CPAN is a good idea. The CPAN shell always fetches the latest version of a distro, rather than a specific version. If you're interested in ensuring repeatable installs, it's not the right tool.
What are your limitations for installing web apps? Can you log into the machine? Are all of the machines running the same thing? Are the people installing the web apps co-workers or random people from the general public? Are the people installing this sysadmins, programmers, web managers, or something else? Do you install by distributed an archive or checking out from source control?
For most of my stuff, which involves sysadmins familiar with Perl installing in control environments, I just use MakeMaker. It's easy to get it to do all the things you listed if you know a little about MakeMaker. If you want to know more about that, ask a another question. ;) Module::Build is just as easy, though, and the way to go if you don't already like using MakeMaker.
Module::Build would be a good way to go to handle lots of different situations if the people are moderately clueful about the command line and installing software. You'll have a lot of flexibility with Module::Build, but also a bit more work. And, the cpan tool (which comes with Perl), can install from the current directory and handle dependencies for you. Just tell it to install the current directory:
$ cpan .
If you only have to install on a single platorm, you'll probably have an easier time making a package in the native format. You could even have Module::Build make that package for you so the developers have the flexibility of Module::Build, but the installers have the ease of the native process. Sticking with Module::Build also means that you could create different packages for different platforms from a single build tool.
If the people installing the web application really have no idea about command lines, CPAN, and other things, you'll probably want to use a packager and installer that doesn't scare them or make them think about what is going on, and can accurately report problems to you automatically.
As Dave points out, using a real CPAN mirror always gets you the latest version of a module, but you can also make your own "fake" CPAN mirror with exactly the distributions you want and have the normal CPAN tools install from that. For our customers, we make "CPAN on a CD" (although thumb drives are good now too). With a simple "run me" script everything gets installed in exactly the versions they need. See, for instance, my Making my own CPAN talk if you're interested in that. Again, consider the audience when you think about that. It's not something you'd hand to the general public.
Good luck, :)
I'd recommend seriously considering a package system such as RPM to do this. Even if you're running on Windows I'd consider RPM and cygwin to do the installation. You could even set up a yum or apt repository to deliver the packages to remote systems.
If you're looking for a general installer for customers running any number of OSes and distros, then the problem becomes much harder.
Take a look at PAR.
Jonathan Rockway as a small section on using this with Catalyst in his book.