so I'm having a lot of fun with Perl at home for some time now.
How much more difficult do things get when you develop Perl modules (In my case it's mostly catalyst) in a team? How do we make sure we all got the same development environment (Perl/Module versions)? Simply by keeping up to date with CPAN? Do some teams setup their 'private' CPANs?
Using the following things should make your life easier.
check out local::lib you could easily then create a server that each member could sync these modules too.
You probably don't really want to mirror all of cpan. just the most recent modules which is why you'd use minicpan.
If you're using recommended modules in Task::Kensho then using the latest releases shouldn't be a problem as they should be surprisingly changing API on you. Basically by doing this you make sure you don't end up with your team reinventing the wheel or hopefully using 3 different modules that do the same thing.
And you want to make sure that your team uses good Perl coding practices and not the bad ones. There are a lot of bad ones. Read Perl Best Practices, remember it's just a guideline you should tune it too your team and your style.
local::lib
minicpan
Task::Kensho
Perl Best Practices
It is not exactly clear what is meant by "in a team".
If the team is at some company, the best solution is of course a shared directory where only the CPAN modules you need are installed.
If the team is a bunch of guys working collaboratively from their home computers, there are a couple of solutions.
One that comes to mind is as follows:
Have a shared "latest version of module to install" list in a file, accessible publicly from the web (on someone's home page, your favorite source control system, Google docs, whatever).
Write a little Perl script which retrieves that file from the web or checks it out of repository, loops over each CPAN module listed in the file, and verifies that locally installed version is the correct one. If upgrade is needed, have the script install update from CPAN.
Have that script run as a scheduled job (cron on Unix, or at/scheduler on Windows) as admin/root account, or at least account which has enough perms to install CPAN modules.
I won't provide details of script implementation, because I don't even know if this is for Windows or Unix, and doing all those tasks are fairly routine Perl coding - if you get stuck, you are always welcome to ask follow up questions on SO! :)
Related
I wrote a script for my company and I am using some libraries I obtained from CPAN. My manager wanted me to consolidate and remove the extra libraries - which is a little funny because I include them for the script to work.
A few notes:
I do not have root access on this server nor can I request access
To use CPAN modules w/o root I have them installed to my user directory
To allow other users to run my scripts I usually include a folder called 'libs' and inside of my script's directory and in the script I have: use 'libs'; at the top before I use my CPAN modules.
The only solution I have right now is to literally put the contents of the perl modules inside of my perl script. However I want to give credit where it is due and also not get in trouble for including opensource code w/o proper credit to its authors and organizations.
Therefore, how should I go about this? I am not trying to get away with anything.. I honestly want to go about doing this the right way.
All three modules say "licensed under the same terms as Perl itself" but I feel like it shouldn't be this easy.
I would also like to explore any other ideas too!
The modules are:
Text::Table
Text::Aligner
Term::ANSIColor
Is using PAR Packager an option for you? That would generate a standalone executable.
If the modules are pure Perl modules, you may be able to simply append the code (including those package statements) into your program. I'd also include the POD which would include the copyright statements and the names of the authors too. That should satisfy the Artistic License Requirement (but may not satisfy GNU licensing requirements).
Another possibility is to use Perlbrew which will allow you to install a user version of Perl on the system. This way, you can install CPAN modules without needing Administrative permission, and you can tell other users to use Perlbrew too.
I use it because I can install and switch between various versions of Perl which allows me to test my Perl scripts in various versions of Perl. I've also used it on our servers where I need a newer version of Perl or modules that weren't included in the standard release.
You need to get your IT approval before installing Perlbrew, but a lot of times they're relieved that they no longer have to be bothered with maintaining and installing CPAN modules for your use.
Interesting question & perspective. I don't understand what is against using libraries or modules, but I'll let your manager do the thinking ;-)
Regarding copyright, you're best to consult a lawyer if you want to be sure, but as far as I understand it, you can combine the work of others provided you retain the copyright notices. The combined work may not be covered by copyleft, so you may be able to use it commercially (i.e., distribute it without disclosing the source). But do check with a lawyer.
But, since you said you wanted to explore other ideas, App::Staticperl may be a solution? I do not have experience with it, but I tried it with a simple example and got a working executable.
App::Staticperl builds a stand-alone executable from the Perl interpreter with embedded CPAN modules. The steps I followed were roughly (you'll need to adapt, because obviously I couldn't test with your script):
latest version of App::Staticperl is 1.43: https://cpan.metacpan.org/authors/id/M/ML/MLEHMANN/App-Staticperl-1.43.tar.gz
either install the module via CPAN, or simply extract bin/staticperl from the tar - it's a standalone script
edit staticperl to change EMAIL and CPAN (optional, but you may want to change the CPAN mirror)
./staticperl install downloads and builds Perl; it ended with an error message on my box, but did produce a working Perl
./staticperl cpan enters an interactive CPAN prompt; install Text::Table, install Term::ANSIColor, and whatever else you need
./staticperl mkapp my_app --boot path/to/your/script -MText::Table -MText::Aligner -MTerm::ANSIColor
try the app: ./my_app - it will most likely fail with an error message about missing modules; repeat the previous step and include the missing modules in the -M flags
Good luck!
Can you reduce the unnecessary code (to satisfy your manager's concerns). Leave in tact the needed code in the file it came in - and give the author's credit within that module/package.
Eg: This was inspired (stolen) from Joe E Perl.
I am greatly disappointed every time I am forced to retrieve a module from CPAN. In most environments I work in, internet access is severely restricted or completely denied. All compilers have been removed during the OS hardening process. And all digital storage media is scanned by a security team before entering or exiting the site. Mind you, I understand security, and all of this is OK with me, but...
What is the recommended or best practice for accessing code only CPAN modules provide.
If I only need a snippet, a function, or a single string of functionality a module gives me, how can I extract "just what I need/want" without installing an entire module? Keeping in mind that I may be literally printing out, writing down, and typing in, to transfer the data from an off-site location with internet access.
When you can't access CPAN - how you want use it? So, you can either:
when have internet access - you can install modules
if havent, you can bring an minicpan on USB stick or soo
if this is not possible because security policy, you can only use e.g. you mobile phone to show source on CPAN and retype the needed parts manually
if neither is possible - you can at home print the source code on your tshirt and retype it in the work :)
or, simply must programm all things yourself - by learning at home
or, find the better job :)
EDIT: More serious approach
First, it is strange to have an company who developing with perl, but doesn't allow use CPAN. Of course, I understand than direct access to a tons of unscanned sw is not acceptable for many companies, but in this case here should be exists some "company policies" - how to allow access.
Here are several questions:
the hardening is at technical level only (firewalls etc) - so you can bring e.g. USB stick, CD or any other medium inside, or
policy level - the policy does not allow using any external source
If it is at policy level - IMHO, you're out of luck. Simply when it is NOT ALLOWED using any external source - you can use really only the "print & retype" method.
Here is some possibilities:
establish an company-wide local CPAN repository
Create an local CPAN (minicpan) server with "trusted" modules. This repository can function as repository for locally developed modules too. In this case must exists some "auditing authority" (policies & procedures) how to get modules into the local repo. IMHO, this can be the most useful way - when the company using perl on regular basis.
Of course, mixing system-wide (default perl modules) with CPAN modules not the best idea. Therefore is possible to setup:
local::lib or build-prefix
local::lib - create and use a local lib/ for perl modules with PERL5LIB. Google for perl "local::lib" or something similar. Also read some other SQ questions:
How can I install a CPAN module into a local directory?
How can I use a new Perl module without install permissions?
How can I install CPAN modules locally without root access (DynaLoader.pm line 229 error)?
and other
Using local::lib is nice solution, because doesn't break system-wide perl modules. Of course, again - you will need some "auditing process" how to get modules inside.
perlbrew
Using your own built perl - perlbrew - is more general solution if the system hasn't installed perl. You don't need root access for building your own perl. Of course, here is still some problems (besides the auditing), e.g. the "missing compiler problem".
virtual machine
You can try setup an virtual machine for development (or isolated physical machine) with full CPAN access and develop here. When you finish the development, you can forward your work with all required modules to "auditing process".
other
If you need only extract an function or a modules from the CPAN modules, do it on external machine. Extracting a function or some part is not a technical problem (when you know perl) it is more an license problem - using a part of modules in your work - you need cite the author.
For this you need fetch all needed functions - can find interesting this discussion. Google for "perl functions dependencies" or something similar, or:
How can I determine CPAN dependencies before I deploy a Perl project?
http://metacpan.org/pod/Module::Extract::Use
http://metacpan.org/pod/Module::ScanDeps
Perl: CPAN - Module modifying and adding functionality
Maybe, you will find this discussion interesting too...
I would like to ask my system administrator to install various Perl modules such as Moose and Data::Alias. The system is Red Hat Enterprise Linux 5, running Perl 5.8.8. The only problem that I can think of is that some already-installed modules might need to be upgraded and thus run the risk of breaking something. What else should I be concerned about?
I do not know your company's policy, but it would probably be a better idea not to mess with your system perl and install the additional libraries and their prerequisites (and even a dedicated perl) in a different location and use lib.
See also How do I keep my own module/library directory? in perlfaq.
Our systems administrators are the ones who are concerned with making sure requested Perl modules don't break anything else on their systems.
So we ask, then they can say "No, that will break X, please install that in user-space." or "We'll install that at non-standard location Y, for compatibility reasons." I don't know all of the things they take into consideration when making that decision. The only thing I am supposed to know is whether or not my applications will break when modules are installed/updated.
If your system administrators are passing this responsibility (i.e. knowing the impact on the system as a whole) to you as a developer, it's probably much safer to go along with Sinan's suggestion and install the modules you need in a non-standard location for yourself. Having to be a system administrator and a developer at the same time while being paid to be only one is no fun.
The trick is to not get rid of your old setup until you know your new setup works. The rub is that the CPAN toolchain doesn't care and will happily install new files over old ones without giving you a way to uninstall your damage.
If I'm working on a big project where I'm going to upgrade modules, I like to put the new modules in their own, new directory. I can test things by settingPERL5LIB to the new directories, and if it doesn't work out, I'm back to the old setup with a simple change of the environment.
Some people do something similar with source control. They put their modules directory in a versioning system. When they install new modules, they check in the source. When something breaks, they just roll back the changes. They can also tag versions so they can roll back to any working point.
This is an often asked question especially if you are using shared hosting or have a hosting provider that is leery of installing perl modules.
There is a fix, a way to work around this problem of mixing system modules and your own modules, and that fix is called local::lib. local::lib provides a way for you to install modules locally, in a library specified by you, which you have control over. You can use /home/you/perl5/ for example, or any path you think will work.
PS - Moose modules will most likely not interfere with other perl 5 modules since Moose has clean and separate namespace. It does require lots of modules from CPAN however, so warn your sysadmin in advance. :)
Also, ask your sysadmin to upgrade perl to 5.10, that brings lots of good new stuff into perl's core and allows you to use some shiny new perl technology.
This is a follow-up to my previous question about developing Perl applications. Let’s say I develop an application as a CPAN module using Module::Install. Now I upload the code to the production server, say using a git push, and I would like to install the application dependencies listed in Makefile.PL. If I simply run cpan ., the thing tries to install the application like a regular Perl module, ie. starts to copy the modules and documentation to standard Perl directories all over the system.
Is this the way it’s supposed to be? Do you install the application into the standard Perl directories? I am used to having my Perl applications in one directory with separate lib. Otherwise it seems I’d have to manage a lot of other things, like installing the resources somewhere on path etc. If I just want to install the deps declared in Makefile.PL and run the application tests to make sure everything works, what should I do?
(Is this documented somewhere? I mean, is there something like best practice for deploying and updating non-trivial Perl applications? Or is everybody doing this his own way?)
I might be misunderstanding, but I think what you're looking for is
perl Makefile.PL
make installdeps
If you are using Module::Install, you're really using ExtUtils::MakeMaker behind the scenes. You can use all of the MakeMaker features and the targets it provides. Although the documentation doesn't show every feature, there are some valuable things to be found in the generated Makefile.
However, MakeMaker is old news and most everyone has asked Santa Claus for it to disappear. If you want better control, including creating your own targets and process, Module::Build is orders of magnitude easier to work with as well as cross-platform (even if that just means not using a different make, gmake, or whatever on the same OS on different boxes). If you deviate from the normal, consumer-grade installation process, you're life will be easier without MakeMaker.
Some people appreciate the brevity of the Module::Install build file, but once constructed, you don't spend a lot of time messing with your build file so it's not that much of a real benefit. When the little benefit you get locks you into MakeMaker, it's not a win at all.
A 2014 update: Module::Build has now fallen out of favor and needs a maintainer. It never quite got to the point where people could use it to build and distribute XS modules. It was deprecated in Perl v5.19 although you can still get it from CPAN.
You could look at Module::ScanDeps to generate a list of dependent modules for installing. Or Par::Packer for packaging up the whole thing as an "app".
It seems that most of the installers for Perl are centered around installing Perl modules, not applications. Things like ExtUtils::MakeMaker and Module::Build are very well suited for modules, but require some additional work for Web Apps.
Ideally it would be nice to be able to do the following after checking out the source from the repository:
Have missing dependencies detected
Download and install dependencies from CPAN
Run a command to "Build" the source into a final state (perform any source parsing or configuration necessary for the local environment).
Run a command to install the built files into the appropriate locations. Not only the perl modules, but also things like template (.tt) files, and CGI scripts, JS and image files that should be web-accessible.
Make sure proper permissions are set on installed files (and SELinux context if necessary).
Right now we have a system based on Module::Build that does most of this. The work was done by done by my co-worker who was learning to use Module::Build at the time, and we'd like some advice on generalizing our solution, since it's fairly app-specific right now. In particular, our system requires us to install dependencies by hand (although it does detect them).
Is there any particular system you've used that's been particularly successful? Do you have to write an installer based on Module::Build or ExtUtils::MakeMaker that's particular to your application, or is something more general available?
EDIT: To answer brian's questions below:
We can log into the machines
We do not have root access to the machines
The machines are all (ostensibly) identical builds of RHEL5 with SELinux enabled
Currently, the people installing the machines are only programmers from our group, and our source is not available to the general public. However, it's conceivable our source could eventually be installed on someone else's machines in our organization, to be installed by their programmers or systems people.
We install by checking out from the repository, though we'd like to have the option of using a distributed archive (see above).
The answer suggesting RPM is definitely a good one. Using your system's package manager can definitely make your life easier. However, it might mean you also need to package up a bunch of other Perl modules.
You might also take a look at Shipwright. This is a Perl-based tool for packaging up an app and all its Perl module dependencies. It's early days yet, but it looks promising.
As far as installing dependencies, it wouldn't be hard to simply package up a bunch of tarballs and then have you Module::Build-based solution install them. You should take a look at pip, which makes installing a module from a tarball quite trivial. You could package this with your code base and simply call it from your own installer to handle the deps.
I question whether relying on CPAN is a good idea. The CPAN shell always fetches the latest version of a distro, rather than a specific version. If you're interested in ensuring repeatable installs, it's not the right tool.
What are your limitations for installing web apps? Can you log into the machine? Are all of the machines running the same thing? Are the people installing the web apps co-workers or random people from the general public? Are the people installing this sysadmins, programmers, web managers, or something else? Do you install by distributed an archive or checking out from source control?
For most of my stuff, which involves sysadmins familiar with Perl installing in control environments, I just use MakeMaker. It's easy to get it to do all the things you listed if you know a little about MakeMaker. If you want to know more about that, ask a another question. ;) Module::Build is just as easy, though, and the way to go if you don't already like using MakeMaker.
Module::Build would be a good way to go to handle lots of different situations if the people are moderately clueful about the command line and installing software. You'll have a lot of flexibility with Module::Build, but also a bit more work. And, the cpan tool (which comes with Perl), can install from the current directory and handle dependencies for you. Just tell it to install the current directory:
$ cpan .
If you only have to install on a single platorm, you'll probably have an easier time making a package in the native format. You could even have Module::Build make that package for you so the developers have the flexibility of Module::Build, but the installers have the ease of the native process. Sticking with Module::Build also means that you could create different packages for different platforms from a single build tool.
If the people installing the web application really have no idea about command lines, CPAN, and other things, you'll probably want to use a packager and installer that doesn't scare them or make them think about what is going on, and can accurately report problems to you automatically.
As Dave points out, using a real CPAN mirror always gets you the latest version of a module, but you can also make your own "fake" CPAN mirror with exactly the distributions you want and have the normal CPAN tools install from that. For our customers, we make "CPAN on a CD" (although thumb drives are good now too). With a simple "run me" script everything gets installed in exactly the versions they need. See, for instance, my Making my own CPAN talk if you're interested in that. Again, consider the audience when you think about that. It's not something you'd hand to the general public.
Good luck, :)
I'd recommend seriously considering a package system such as RPM to do this. Even if you're running on Windows I'd consider RPM and cygwin to do the installation. You could even set up a yum or apt repository to deliver the packages to remote systems.
If you're looking for a general installer for customers running any number of OSes and distros, then the problem becomes much harder.
Take a look at PAR.
Jonathan Rockway as a small section on using this with Catalyst in his book.