How to use CPAN modules without access to CPAN? - perl

I am greatly disappointed every time I am forced to retrieve a module from CPAN. In most environments I work in, internet access is severely restricted or completely denied. All compilers have been removed during the OS hardening process. And all digital storage media is scanned by a security team before entering or exiting the site. Mind you, I understand security, and all of this is OK with me, but...
What is the recommended or best practice for accessing code only CPAN modules provide.
If I only need a snippet, a function, or a single string of functionality a module gives me, how can I extract "just what I need/want" without installing an entire module? Keeping in mind that I may be literally printing out, writing down, and typing in, to transfer the data from an off-site location with internet access.

When you can't access CPAN - how you want use it? So, you can either:
when have internet access - you can install modules
if havent, you can bring an minicpan on USB stick or soo
if this is not possible because security policy, you can only use e.g. you mobile phone to show source on CPAN and retype the needed parts manually
if neither is possible - you can at home print the source code on your tshirt and retype it in the work :)
or, simply must programm all things yourself - by learning at home
or, find the better job :)
EDIT: More serious approach
First, it is strange to have an company who developing with perl, but doesn't allow use CPAN. Of course, I understand than direct access to a tons of unscanned sw is not acceptable for many companies, but in this case here should be exists some "company policies" - how to allow access.
Here are several questions:
the hardening is at technical level only (firewalls etc) - so you can bring e.g. USB stick, CD or any other medium inside, or
policy level - the policy does not allow using any external source
If it is at policy level - IMHO, you're out of luck. Simply when it is NOT ALLOWED using any external source - you can use really only the "print & retype" method.
Here is some possibilities:
establish an company-wide local CPAN repository
Create an local CPAN (minicpan) server with "trusted" modules. This repository can function as repository for locally developed modules too. In this case must exists some "auditing authority" (policies & procedures) how to get modules into the local repo. IMHO, this can be the most useful way - when the company using perl on regular basis.
Of course, mixing system-wide (default perl modules) with CPAN modules not the best idea. Therefore is possible to setup:
local::lib or build-prefix
local::lib - create and use a local lib/ for perl modules with PERL5LIB. Google for perl "local::lib" or something similar. Also read some other SQ questions:
How can I install a CPAN module into a local directory?
How can I use a new Perl module without install permissions?
How can I install CPAN modules locally without root access (DynaLoader.pm line 229 error)?
and other
Using local::lib is nice solution, because doesn't break system-wide perl modules. Of course, again - you will need some "auditing process" how to get modules inside.
perlbrew
Using your own built perl - perlbrew - is more general solution if the system hasn't installed perl. You don't need root access for building your own perl. Of course, here is still some problems (besides the auditing), e.g. the "missing compiler problem".
virtual machine
You can try setup an virtual machine for development (or isolated physical machine) with full CPAN access and develop here. When you finish the development, you can forward your work with all required modules to "auditing process".
other
If you need only extract an function or a modules from the CPAN modules, do it on external machine. Extracting a function or some part is not a technical problem (when you know perl) it is more an license problem - using a part of modules in your work - you need cite the author.
For this you need fetch all needed functions - can find interesting this discussion. Google for "perl functions dependencies" or something similar, or:
How can I determine CPAN dependencies before I deploy a Perl project?
http://metacpan.org/pod/Module::Extract::Use
http://metacpan.org/pod/Module::ScanDeps
Perl: CPAN - Module modifying and adding functionality
Maybe, you will find this discussion interesting too...

Related

Manually installing a Perl module using cygwin GNU make

I am trying to install a few Perl modules, one of them being Time::Format. My corporate group policy does not allow me to use CPAN (since it uses FTP) to install modules. Tried using makefile.pl, but nmake.exe (I am using windows cygwin) seems to be missing too. The policy does not allow to download the nmake.exe executable too. PPM isn't available too.
How do I go about installing module manually? Is there a way where I could manually copy files from this module on to individual folders in my Perl directory? If yes, which files, go where?
You might be able to get around the FTP restriction by using a minicpan. Basically this lets you build your own local CPAN mirror, for example on a USB key drive. It's intended to carry a snapshot of CPAN with you, e.g. when you are in transatlantic flights and just really need that module you didn't install.
However, those restrictions are probably there for a reason. Downloading and installing/copying stuff from the internet might be against the corporate policy just the same as it's an external source. You should check that. On the other hand, those policies are often intended for the average office user that is not a developer, so you might be able to talk to them and explain why you and your team should be excepted from this policy, and how reusing code from CPAN would save your company a lot of time, which equals money.

Giving Credit for Perl Module

I wrote a script for my company and I am using some libraries I obtained from CPAN. My manager wanted me to consolidate and remove the extra libraries - which is a little funny because I include them for the script to work.
A few notes:
I do not have root access on this server nor can I request access
To use CPAN modules w/o root I have them installed to my user directory
To allow other users to run my scripts I usually include a folder called 'libs' and inside of my script's directory and in the script I have: use 'libs'; at the top before I use my CPAN modules.
The only solution I have right now is to literally put the contents of the perl modules inside of my perl script. However I want to give credit where it is due and also not get in trouble for including opensource code w/o proper credit to its authors and organizations.
Therefore, how should I go about this? I am not trying to get away with anything.. I honestly want to go about doing this the right way.
All three modules say "licensed under the same terms as Perl itself" but I feel like it shouldn't be this easy.
I would also like to explore any other ideas too!
The modules are:
Text::Table
Text::Aligner
Term::ANSIColor
Is using PAR Packager an option for you? That would generate a standalone executable.
If the modules are pure Perl modules, you may be able to simply append the code (including those package statements) into your program. I'd also include the POD which would include the copyright statements and the names of the authors too. That should satisfy the Artistic License Requirement (but may not satisfy GNU licensing requirements).
Another possibility is to use Perlbrew which will allow you to install a user version of Perl on the system. This way, you can install CPAN modules without needing Administrative permission, and you can tell other users to use Perlbrew too.
I use it because I can install and switch between various versions of Perl which allows me to test my Perl scripts in various versions of Perl. I've also used it on our servers where I need a newer version of Perl or modules that weren't included in the standard release.
You need to get your IT approval before installing Perlbrew, but a lot of times they're relieved that they no longer have to be bothered with maintaining and installing CPAN modules for your use.
Interesting question & perspective. I don't understand what is against using libraries or modules, but I'll let your manager do the thinking ;-)
Regarding copyright, you're best to consult a lawyer if you want to be sure, but as far as I understand it, you can combine the work of others provided you retain the copyright notices. The combined work may not be covered by copyleft, so you may be able to use it commercially (i.e., distribute it without disclosing the source). But do check with a lawyer.
But, since you said you wanted to explore other ideas, App::Staticperl may be a solution? I do not have experience with it, but I tried it with a simple example and got a working executable.
App::Staticperl builds a stand-alone executable from the Perl interpreter with embedded CPAN modules. The steps I followed were roughly (you'll need to adapt, because obviously I couldn't test with your script):
latest version of App::Staticperl is 1.43: https://cpan.metacpan.org/authors/id/M/ML/MLEHMANN/App-Staticperl-1.43.tar.gz
either install the module via CPAN, or simply extract bin/staticperl from the tar - it's a standalone script
edit staticperl to change EMAIL and CPAN (optional, but you may want to change the CPAN mirror)
./staticperl install downloads and builds Perl; it ended with an error message on my box, but did produce a working Perl
./staticperl cpan enters an interactive CPAN prompt; install Text::Table, install Term::ANSIColor, and whatever else you need
./staticperl mkapp my_app --boot path/to/your/script -MText::Table -MText::Aligner -MTerm::ANSIColor
try the app: ./my_app - it will most likely fail with an error message about missing modules; repeat the previous step and include the missing modules in the -M flags
Good luck!
Can you reduce the unnecessary code (to satisfy your manager's concerns). Leave in tact the needed code in the file it came in - and give the author's credit within that module/package.
Eg: This was inspired (stolen) from Joe E Perl.

Setting up a common perl/cpan environment

so I'm having a lot of fun with Perl at home for some time now.
How much more difficult do things get when you develop Perl modules (In my case it's mostly catalyst) in a team? How do we make sure we all got the same development environment (Perl/Module versions)? Simply by keeping up to date with CPAN? Do some teams setup their 'private' CPANs?
Using the following things should make your life easier.
check out local::lib you could easily then create a server that each member could sync these modules too.
You probably don't really want to mirror all of cpan. just the most recent modules which is why you'd use minicpan.
If you're using recommended modules in Task::Kensho then using the latest releases shouldn't be a problem as they should be surprisingly changing API on you. Basically by doing this you make sure you don't end up with your team reinventing the wheel or hopefully using 3 different modules that do the same thing.
And you want to make sure that your team uses good Perl coding practices and not the bad ones. There are a lot of bad ones. Read Perl Best Practices, remember it's just a guideline you should tune it too your team and your style.
local::lib
minicpan
Task::Kensho
Perl Best Practices
It is not exactly clear what is meant by "in a team".
If the team is at some company, the best solution is of course a shared directory where only the CPAN modules you need are installed.
If the team is a bunch of guys working collaboratively from their home computers, there are a couple of solutions.
One that comes to mind is as follows:
Have a shared "latest version of module to install" list in a file, accessible publicly from the web (on someone's home page, your favorite source control system, Google docs, whatever).
Write a little Perl script which retrieves that file from the web or checks it out of repository, loops over each CPAN module listed in the file, and verifies that locally installed version is the correct one. If upgrade is needed, have the script install update from CPAN.
Have that script run as a scheduled job (cron on Unix, or at/scheduler on Windows) as admin/root account, or at least account which has enough perms to install CPAN modules.
I won't provide details of script implementation, because I don't even know if this is for Windows or Unix, and doing all those tasks are fairly routine Perl coding - if you get stuck, you are always welcome to ask follow up questions on SO! :)

Why can't I simply copy installed Perl modules to other machines?

Being very new to Perl but not to dynamic languages, I'm a bit surprised at how not straight forward the manage of modules is.
Sure, cpan X does theoretically work, but I'm working on the same project from three different machines and OSs (at work, at home, testing in an external environment).
At work (Windows 7) I have problem using cpan because of our firewall that makes ftp unusable
At home (Mac OS X) it does work
In the external environment (Linux CentOs) it worked after hours because I don't have root access and I had to configure cpan to operate as a non-root user
I've tried on another server where I have an access. If the previous external environment is a VPS and so I have a shell access, this other one is a cheap shared hosting where I have no way to install new modules other than the ones pre-installed
At the moment I still can't install Template under Windows. I've seen that as an alternative I could compile it and I've also tried ActiveState's PPM but the module is not existent there.
Now, my perplexity is about Perl being a dynamic language. I've had all these kind of problems while working, for example, with C where I had to compile all the libraries for all the platform, but I thought that with Perl the approach would have been very similar to Python's or PHP's where in 90% of the cases copying the module in a directory and importing it simply works.
So, my question: if Perl's modules are written in Perl, why the copy/paste approach will not work? If some (or some part) of the modules have to be compiled, how to see in CPAN if a module is Perl-only or it relies upon compiled libraries? Isn't there a way to download the module (tar, zip...) and use cpan to deploy it? This would solve my problem under Windows.
Now, Perl is a dynamic language, but that doesn't imply that everything that people write is portable across platforms. That's not the fault of the language. It's not even the fault of the programmer. Some things, like Win32::OLE shouldn't work on Unix. :)
Other dynamic languages will have some of the same problems. If you have to compile C code, you won't be able to merely copy files to another machine. Some distributions configure the code slightly differently depending on your operating system, etc.
Even if you could copy files, you have to ensure that you copy all of the files that you need. Do you know everything that you need for a particular module? Remember, many of them have dependencies.
Most of the problems you're having aren't anything to do with the language. You're having trouble with the tools. If you want a zero conf CPAN tool that makes all the decisions for you, try cpanminus. It's mostly the same thing that you'd get out of cpan (although different code), but it makes all of the decisions for you. It doesn't run any of the distribution tests, and it installs into your user directory. When you need something that gives you control, come back to cpan.
In the external environment (Linux CentOs) it worked after hours because I don't have root access and I had to configure cpan to operate as a non-root user
This is one of those times when it helps to know The Trick. In this case local::lib, which lets you configure a non-root install area and all the ENV variables in about three minutes.
if perl's modules are written in perl, why the copy/past approach will not work?
Some are written in Pure Perl, but many are written partially in C (using Perl's XS API) for efficiency.
Sometimes you end up with situations like JSON::XS, JSON::PP and JSON::Any to autoselect the best one that is installed.
Isn't there a way to download the module (tar, zip...) and use cpan to deploy it?
The cpan program is all about getting things from the Internet. You can download the package (there will be a link along the lines of "Download: CGI.pm-3.49.tar.gz" on the right hand side of the CPAN page), untar it, then
perl Makefile.PL
make
make install
You would probably be better off configuring your cpan installation to use only HTTP sources (in the urllist config option). Possibly going to far as to create a mini CPAN mirror inside your network.

How do I install Perl modules on machines without an Internet connection?

I need to install my Perl-based software on networked machines which aren't connected to the internet. Therefore, I would like to download specific versions and/or latest versions of the Perl modules and I would also like to know if there is an install procedure required for these modules.
Background:
The machines aren't connected to the internet for security reasons and its deemed unnecessary also.
I would place the downloaded modules on a machine that I call the 'install server' and it contains my Perl based software and would also contain the local copies of the Perl modules.
I call a machine that I want to install my Perl-based software on, the 'target machine', also not connected to the internet. There can be several target machines, each can run this software that I want to install. I log onto the target machine and run an install script which would connect to the install machine via the local network to obtain the Perl-based software and dependent Perl modules and installs them.
So I need to know:
How/Where to get specific versions of Perl modules, e.g. CGI.pm etc
How to install these Perl modules. Is it a case of just placing them in a directory somewhere, e.g. a library path and making sure that this directory path is in the #INC library path environmental variable, if it is not already?
I would prefer not to have to do anything like make install etc. as part of installing the modules. I would like to modules to be pre-compiled or prepared as necessary so it is as simple as possible to install them. I want to avoid additional dependencies like make and its configuration, and having to parse its output to check whether it was successful.
Please help me by asking the above specific questions as I am not able to change the concept of 'install machine' and 'target machine' which aren't connected to the internet - I have to provide a solution that works within this arrangement.
The usual way to solve "I want to install stuff from CPAN but without network" problems is to use a minicpan as David Dorward wrote in his answer. But since you're going one step further, saying that you'd rather not do any real installation on the client (target) machines at all, and that you want to use precompiled modules if possible, I urge you to check out PAR and specifically PAR::Repository (server) and PAR::Repository::Client.
Since this approach needs some research before you're up to speed, I wouldn't suggest it for "I just need Foo.pm" like problems. Once you're talking about a handful of dependencies and at least a handful of clients, then it becomes a more appropriate solution.
For an outline of how it works, check out the slides of my talk at YAPC::EU 2008. It also hints at solutions to the bootstrapping problem of making the PAR::Repository::Client module available on the clients (hint: PAR can generate self-contained executables).
You can create a MiniCPAN that has just the latest versions of everything from CPAN. You can insert additional, non-public modules into it with CPAN::Mini::Inject. If you need to greater control over versions (i.e. not choosing the latest versions), you might want to create a DPAN.
With any of these solutions, you can configure your CPAN client to pull from your local source. That could be a directory you know ahead of time or something that you figure out dynamically, like a CD or a thumb-drive. It's just a matter of setting up the configuration correctly.
You might be able to get away with creating operating-system packages for most of your work, but that still means you have to compile them at least the first time.
1) How/Where to get specific versions of Perl modules, e.g. CGI.pm etc
http://search.cpan.org/
If you don't want the latest version, you can get an earlier version by following the link in the breadcrumbs.
http://img.skitch.com/20091209-bu7kt3bj65374k7iijfnhrue2y.png
2) How to install these Perl modules. Is it a case of just placing them
in a directory somewhere, e.g. a library path and making sure that this
directory path is in the #INC library path environmental variable, if
it is not already?
That sometimes work, but you really should go through the perl Makefile.PL && make && make test && make install process.
Doing this would require that you manually chase all the dependencies though. You would probably be better off with something like minicpan.