I wrote a script for my company and I am using some libraries I obtained from CPAN. My manager wanted me to consolidate and remove the extra libraries - which is a little funny because I include them for the script to work.
A few notes:
I do not have root access on this server nor can I request access
To use CPAN modules w/o root I have them installed to my user directory
To allow other users to run my scripts I usually include a folder called 'libs' and inside of my script's directory and in the script I have: use 'libs'; at the top before I use my CPAN modules.
The only solution I have right now is to literally put the contents of the perl modules inside of my perl script. However I want to give credit where it is due and also not get in trouble for including opensource code w/o proper credit to its authors and organizations.
Therefore, how should I go about this? I am not trying to get away with anything.. I honestly want to go about doing this the right way.
All three modules say "licensed under the same terms as Perl itself" but I feel like it shouldn't be this easy.
I would also like to explore any other ideas too!
The modules are:
Text::Table
Text::Aligner
Term::ANSIColor
Is using PAR Packager an option for you? That would generate a standalone executable.
If the modules are pure Perl modules, you may be able to simply append the code (including those package statements) into your program. I'd also include the POD which would include the copyright statements and the names of the authors too. That should satisfy the Artistic License Requirement (but may not satisfy GNU licensing requirements).
Another possibility is to use Perlbrew which will allow you to install a user version of Perl on the system. This way, you can install CPAN modules without needing Administrative permission, and you can tell other users to use Perlbrew too.
I use it because I can install and switch between various versions of Perl which allows me to test my Perl scripts in various versions of Perl. I've also used it on our servers where I need a newer version of Perl or modules that weren't included in the standard release.
You need to get your IT approval before installing Perlbrew, but a lot of times they're relieved that they no longer have to be bothered with maintaining and installing CPAN modules for your use.
Interesting question & perspective. I don't understand what is against using libraries or modules, but I'll let your manager do the thinking ;-)
Regarding copyright, you're best to consult a lawyer if you want to be sure, but as far as I understand it, you can combine the work of others provided you retain the copyright notices. The combined work may not be covered by copyleft, so you may be able to use it commercially (i.e., distribute it without disclosing the source). But do check with a lawyer.
But, since you said you wanted to explore other ideas, App::Staticperl may be a solution? I do not have experience with it, but I tried it with a simple example and got a working executable.
App::Staticperl builds a stand-alone executable from the Perl interpreter with embedded CPAN modules. The steps I followed were roughly (you'll need to adapt, because obviously I couldn't test with your script):
latest version of App::Staticperl is 1.43: https://cpan.metacpan.org/authors/id/M/ML/MLEHMANN/App-Staticperl-1.43.tar.gz
either install the module via CPAN, or simply extract bin/staticperl from the tar - it's a standalone script
edit staticperl to change EMAIL and CPAN (optional, but you may want to change the CPAN mirror)
./staticperl install downloads and builds Perl; it ended with an error message on my box, but did produce a working Perl
./staticperl cpan enters an interactive CPAN prompt; install Text::Table, install Term::ANSIColor, and whatever else you need
./staticperl mkapp my_app --boot path/to/your/script -MText::Table -MText::Aligner -MTerm::ANSIColor
try the app: ./my_app - it will most likely fail with an error message about missing modules; repeat the previous step and include the missing modules in the -M flags
Good luck!
Can you reduce the unnecessary code (to satisfy your manager's concerns). Leave in tact the needed code in the file it came in - and give the author's credit within that module/package.
Eg: This was inspired (stolen) from Joe E Perl.
Related
I have a perl script that I've put together to do some monitoring and graphing.
It works nicely on my dev system, where I have carte-blanch to install my own modules from CPAN.
What I'm looking at doing is bundling it up to deploy onto another system. But here's the catch - this other system is 'standalone' and has no network connection. (And I have change control paperwork to fill in, indicating what I'm installing).
As a result, I'd really like a nice easy way to figure out:
- What modules my scripts are making use of. (Including dependencies)
- how to easily grab them (cpan get probably)
- Is there an easy way to tell what external binaries I'm using? (I'm using for sure ssh and rrdtool - the former is definitely installed, the latter probably not).
I have a few thoughts on how to do this, but it strikes me as something that should be smoother.
I may also need to deploy a new perl, so I'm pondering whether I'm better off 'installing' the modules with system perl (probably 5.8.8 on RHEL5), or just 'packaging' the whole thing in a directory of it's own with a standalone perl instance.
Use pp to package your script and all dependant modules and libraries into a stand alone executable.
pp -x yourscript.pl -o outputfilename
See the documentation for examples of how to link to external shared objects (etc) if required. With pp you don't need perl on the target system where outputfilename will run.
Revisiting this, as the need hasn't really gone away. I have moved towards using docker - this is an 'image' and 'container' system for app deployment, which amongst other things, allows you to 'package' an application.
You create a Dockerfile - which is analagous to a Makefile - that runs through the steps to install perl + dependencies (either via a package manager, or from CPAN).
Once it has, you have a self contained, runnable 'image' that you can clone and create an instance of (a "container" in docker parlance).
It's also quite useful - even if you don't then deploy via container - to figure out what the dependencies of this application/packages were. The version in the container has everything locally installed that it needed, because it was a clean build.
When you have a system where you can't control the Perl installation (and the install is a really, really old version of Perl like 5.8.8 which is missing many nice improvements like state variables, autodie, say, and switch), you should look into Perlbrew.
Perlbrew allows you to install a user version of Perl. (In fact, it allows you to install multiple versions of Perl), and allows you to switch between the Perlbrew install and the officially installed version. It makes doing everything in Perl much, much easier.
You will have freer access to install new Perl modules, and you can do that task yourself rather than wait for your IT department to do it for you.
I ended up using it on one of our systems where the primitive version of Perl just wasn't doing what my version of Perl did. I originally asked our IT to upgrade, but they really messed up the upgrade. After going back and forth, I simply asked if I could install Perlbrew.
Which is an important point. Always ask permission. A lot of time, the IT department is more than happy to oblige. They're not Perl people, and CPAN is a world they don't want to deal with. Being able to get out of having to answer your beck and call about installing this or that Perl module is a great relief.
I am relatively new to Perl and I need some help with redistributing the non-core modules. Here's the whole story.
There are two non-core modules that were used in the Perl script: XML::Simple and SOAP::Lite. The version that I'm using (currently on Windows) is Strawberry Perl, so these two modules are already included. However, we don't know if the end users (Unix/Linux system) have these two modules as they might only have the standard version and so have only the core modules. My goal is to make the end users do as little configurations/installs as possible.
First, I tried to see if there's any core modules that's similar to XML::Simple and SOAP::Lite. Unfortunately, I didn't find any (Please correct me if I'm wrong).
So I guess now the only option is to redistribute the two modules. I checked and that these two modules allow redistribution. My problem right now is how to do it. I tried googling with keywords "perl redistribute" but didn't find anything useful. My guess is that we use the exporter tool to achieve this. but these two modules are rather complicated modules and they have several nested folders/pm files (and a whole bunch of other files like MAKE, pod, ini files) so I'm not sure what I should do. The examples I found using exporter are rather simple: They only have 1 pm file and 1 pl file and they are placed into one folder.
Also, I'm open to any other better ways to deal with the problem. The goal is just to make sure all end users can use my script with the least configuration/install efforts as we don't want them to run into a whole bunch of compatibility issues.
Any help would be appreciated. Thanks! =D
I want to elaborate a little bit on #ikegami said.
SOAP::Lite has a large number of CPAN dependencies, so the people installing your module are going to need CPAN access to get it to build, whether you provide it for them, or list it as a dependency. Otherwise, you'll need to provide your entire dependency tree, at which point, you end up using perlbrew, maybe carton, possibly local::lib, and then you might decide you need the next higher level and produce RPMs and DEBs.
Probably better to just provide your script, packaged as a CPAN module, list your dependencies within, and let the chips fall where they may.
Just state a dependency on the modules in your Makefile.PL or Build.PL, then give them the following installation instruction:
cpanm script.tar.gz
One of the best things about Perl is The CPAN, the Comprehensive Perl Archive Network. It's a mirroring service that since about the time Perl 5 originally came out has allowed people to share useful add-on modules, like XML::Simple or SOAP::Lite through a standard, common tool, the cpan client that comes with Perl. Almost all Perl distributions (such as Strawberry Perl and most Perl distributions that come with linux) have a CPAN client configured and included with them. This client lets people download and install modules from CPAN simply by knowing the name of the module.
Almost all module distributions on CPAN follow the exact same layout. They usually have a Makefile.PL file (if it uses ExtUtils::MakeMaker to generate the install script), Build.PL file (if it uses Module::Build to generate the install script), or both. These Perl scripts, once run, create a 'Makefile' or a 'Build' file that can let you install the module and verify that all prerequisites are met.
If you've never made a Perl distribution before, you can download any distribution you want from CPAN and take a look at how things are laid out. The folders and file locations are pretty intuitive once you've seen one. They are usually laid out with the install script and supporting files (like a readme) in the root directory, with the custom modules (modules you make) in the lib directory, and with unit tests in the t directory.
I'd recommend looking at the Build.PL based ones if you're a novice; these are pure Perl based install scripts. If you decide to make a Module.PL based distribution, it's really easy to specify that your module distribution needs XML::Simple and SOAP::Lite. First, create a basic Module::Build based install script. This looks something like:
use Module::Build;
my $build = Module::Build->new(
module_name => 'Foo::Bar',
license => 'perl',
requires => {
'perl' => '5.6.1',
'Some::Module' => '1.23',
'Other::Module' => '>= 1.2, != 1.5, < 2.0',
},
);
$build->create_build_script;
(This is taken right from the Module::Build::Authoring docs).
Then, specify the libraries you need and minimum versions of them. Zero (0) is an acceptable version if you don't care, but that means "anything" is good. I'd recommend specifying at least the version of the libraries installed on the machines you're testing with.
(Neat short cut: you can find out the version of any library that has a $VERSION package variable defined by doing:
perl -MSome::Lib -E "say Some::Lib->VERSION()"
.)
To install the module, the steps look something like this:
cd folder\where\my\lib\is
perl Build.PL
Build
Build test
Build install
This will create the install tool, prepare the folder for testing (usually just copying stuff to a build library area for simple modules), run all .t scripts in the t folder (the "tests", which usually use Test::More for unit testing of the module prior to install), and then finally, install your module to your PC's Perl site libraries.
The Build script, as part of the 'setting things up' phase, will look at your prerequisites, and warn you if you don't have them yet.
Then, as pointed out in ikegami's answer, if you use the cpanm client to install your library, the cpan client will automatically go out, download, test, and install your dependencies for you! Alternatively, Build.PL based installers also have the 'installdeps' option, which will do the same thing. Then any and all dependencies (and potentially recursive dependencies) are automatically downloaded, tested, and installed, even if they change in the future.
I need to install my Perl-based software on networked machines which aren't connected to the internet. Therefore, I would like to download specific versions and/or latest versions of the Perl modules and I would also like to know if there is an install procedure required for these modules.
Background:
The machines aren't connected to the internet for security reasons and its deemed unnecessary also.
I would place the downloaded modules on a machine that I call the 'install server' and it contains my Perl based software and would also contain the local copies of the Perl modules.
I call a machine that I want to install my Perl-based software on, the 'target machine', also not connected to the internet. There can be several target machines, each can run this software that I want to install. I log onto the target machine and run an install script which would connect to the install machine via the local network to obtain the Perl-based software and dependent Perl modules and installs them.
So I need to know:
How/Where to get specific versions of Perl modules, e.g. CGI.pm etc
How to install these Perl modules. Is it a case of just placing them in a directory somewhere, e.g. a library path and making sure that this directory path is in the #INC library path environmental variable, if it is not already?
I would prefer not to have to do anything like make install etc. as part of installing the modules. I would like to modules to be pre-compiled or prepared as necessary so it is as simple as possible to install them. I want to avoid additional dependencies like make and its configuration, and having to parse its output to check whether it was successful.
Please help me by asking the above specific questions as I am not able to change the concept of 'install machine' and 'target machine' which aren't connected to the internet - I have to provide a solution that works within this arrangement.
The usual way to solve "I want to install stuff from CPAN but without network" problems is to use a minicpan as David Dorward wrote in his answer. But since you're going one step further, saying that you'd rather not do any real installation on the client (target) machines at all, and that you want to use precompiled modules if possible, I urge you to check out PAR and specifically PAR::Repository (server) and PAR::Repository::Client.
Since this approach needs some research before you're up to speed, I wouldn't suggest it for "I just need Foo.pm" like problems. Once you're talking about a handful of dependencies and at least a handful of clients, then it becomes a more appropriate solution.
For an outline of how it works, check out the slides of my talk at YAPC::EU 2008. It also hints at solutions to the bootstrapping problem of making the PAR::Repository::Client module available on the clients (hint: PAR can generate self-contained executables).
You can create a MiniCPAN that has just the latest versions of everything from CPAN. You can insert additional, non-public modules into it with CPAN::Mini::Inject. If you need to greater control over versions (i.e. not choosing the latest versions), you might want to create a DPAN.
With any of these solutions, you can configure your CPAN client to pull from your local source. That could be a directory you know ahead of time or something that you figure out dynamically, like a CD or a thumb-drive. It's just a matter of setting up the configuration correctly.
You might be able to get away with creating operating-system packages for most of your work, but that still means you have to compile them at least the first time.
1) How/Where to get specific versions of Perl modules, e.g. CGI.pm etc
http://search.cpan.org/
If you don't want the latest version, you can get an earlier version by following the link in the breadcrumbs.
http://img.skitch.com/20091209-bu7kt3bj65374k7iijfnhrue2y.png
2) How to install these Perl modules. Is it a case of just placing them
in a directory somewhere, e.g. a library path and making sure that this
directory path is in the #INC library path environmental variable, if
it is not already?
That sometimes work, but you really should go through the perl Makefile.PL && make && make test && make install process.
Doing this would require that you manually chase all the dependencies though. You would probably be better off with something like minicpan.
I have several scripts that I supply to users as tools for their engineering projects. These scripts contain a lot of duplicated code. I want to extract this code and maintain it in a set of modules. However, in order for this to work, the users would have to install these modules. I don't want to have to tell my users to "make install", etc., as I'm sure none of them would have the patience for that.
I understand that one option is to package everything together with PAR, but ideally the user would be able to open up && edit these scripts if they need to, just like they can now. They also need to be able to move them to whatever directory they want, and I don't want them to have to move a bunch of library files as well.
Is it possible to make a double-click file that installs some bundled Perl modules?
I distribute my script as modules, and then use the normal CPAN toolchain to install them. See my articles Scripts as Modules and Automating Script Distributions with scriptdist. Once you have them in a conventional module distribution, you can install them from their current directory with cpan:
% cpan . # install from distribution in the current directory
Depending on how complex your situation is, you might want to create a DPAN, which is a private version of CPAN that your CPAN tools can draw from. You put all of your private modules there, point your CPAN tools at it, and everything happens as it does with a real CPAN mirror. You never have to share your work though.
yeah package with either PAR or Shipwright (not sure about binaries). Also use scandeps.pl along the way.
If the users are using systems with a packaging system (dpkg, cygwin, etc.), consider using that.
If you don't mind spending some green, one of the better bet is Perl Dev Kit from Activestate.
From their own description of the product,
Develop and deploy your Perl programs
to anyone on any platform with
PerlApp's new cross-platform
wrapping.
Deliver code as
executables or as Windows Services,
ActiveX components, .NET assemblies
or in the System Tray.
Easily create
MSI files using Perl code.
You may also consider using sparrow - scripts distribution system. Sparrow plays nice with Perl as it writen on it. It supports CPAN modules dependencies via carton tool.
PS Disclaimer - I am the tool author
I want another developer to run a Perl script I have written. The script uses many CPAN modules that have to be installed before the script can be run. Is it possible to make the script (or the perl binary) to dump a list of all the missing modules? Perl prints out the missing modules’ names when I attempt to run the script, but this is verbose and does not list all the missing modules at once. I’d like to do something like:
$ cpan -i `said-script --list-deps`
Or even:
$ list-deps said-script > required-modules # on my machine
$ cpan -i `cat required-modules` # on his machine
Is there a simple way to do it? This is not a show stopper, but I would like to make the other developer’s life easier. (The required modules are sprinkled across several files, so that it’s not easy for me to make the list by hand without missing anything. I know about PAR, but it seems a bit too complicated for what I want.)
Update: Thanks, Manni, that will do. I did not know about %INC, I only knew about #INC. I settled with something like this:
print join("\n", map { s|/|::|g; s|\.pm$||; $_ } keys %INC);
Which prints out:
Moose::Meta::TypeConstraint::Registry
Moose::Meta::Role::Application::ToClass
Class::C3
List::Util
Imager::Color
…
Looks like this will work.
Check out Module::ScanDeps and the "scandeps.pl" utility that comes with it. It can do a static (and recursive) analysis of your code for dependencies as well as the %INC dump either after compiling or running the program.
Please note that the static source scanning always errs on the side of including too many dependencies. (It is the dependency scanner used by PAR and aims at being easiest on the end-user.)
Finally, you could choose to distribute your script as a CPAN distribution. That sounds much more complicated than it really is. You can use something like Module::Starter to set up a basic skeleton of a tentative App::YourScript distribution. Put your script in the bin/ subdirectory and edit the Makefile.PL to reference all of your direct dependencies. Then, for distribution you do:
perl Makefile.PL
make
make dist
The last step generates a nice App-YourScript-VERSION.tar.gz
Now, when the client wants to install all dependencies, he does the following:
Set up the CPAN client correctly. Simply run it and answer the questions. But you're requiring that already anyway.
"tar -xz App-YourScript-VERSION.tar.gz && cd App-YourScript-VERSION"
Run "cpan ."
The CPAN client will now install all direct dependencies and the dependencies of those distributions automatically. Depending on how you set it up, it will either follow the prerequisites recursively automatically or prompt with a y/n each time.
As an example of this, you might check out a few of the App::* distributions on CPAN. I would think App::Ack is a good example. Maybe one of the App::* distributions from my CPAN directory (SMUELLER).
You could dump %INC at the end of your script. It will contain all used and required modules. But of course, this will only be helpful if you don't require modules conditionally (require Foo if $bar).
For quick-and-dirty, infrequent use, the %INC is the best way to go. If you have to do this with continuous integration testing or something more robust, there are some other tools to help.
Steffen already mentioned the Module::ScanDeps.
The code in Test::Prereq does this, but it has an additional layer that ensures that your Makefile.PL or Build.PL lists them as a dependency. If you make your scripts look like a normal Perl distribution, that makes it fairly easy to check for new dependencies; just run the test suite again.
Aside from that, you might use a tool such as Module::Extract::Use, which parses the static code looking for use and require statements (although it won't find them in string evals). That gets you just the modules you told your script to load.
Also, once you know which modules you loaded, you can combine that with David Cantrell's CPANdeps tool that has already created the dependency tree for most CPAN modules.
Note that you also have to think about optional features too. Your code in this case my not have them, but sometimes you don't load a module until you need it:
sub foo
{
require Bar; # don't load until we need to use it
....
}
If you don't exercise that feature in your trial run or test, you won't see that you need Bar for that feature. A similar problem comes up when a module loads a different set of dependency modules in a different environment (say, mod_perl or Windows, and so on).
There's not a good, automated way of testing optional features like that so you can get their dependencies. However, I think that should be on my To Do list since it sounds like an interesting problem.
Another tool in this area, which is used by Dist::Zilla and its AutoPrereqs plugin, is Perl::PrereqScanner. It installs a scan-perl-prereqs program that will use PPI and a few plugins to search for most kinds of prereq declaration, using the minimum versions you define. In general, I suggest this over scanning %INC, which can bring in bogus requirements and ignores versions.
Today I develop my Perl apps as CPAN-like distributions using Dist::Zilla that can take care of the dependencies through the AutoPrereq plugin. Another interesting piece of code in this area is carton.