perlbrew and local::lib at the same time? - perl

So far I have been using the system perl (on Ubuntu 10.10) and I was using local::lib to install CPAN modules in my private directory ~/perl5
As I am trying to use perlbrew it seems that they don't know about each other. I installed perl-5.12.3 using perlbrew but when I switch to it using perlbrew use perl-5.12.3 I still see the PERL5LIB and PERL_MM_OPT set by local::lib.
That's not good:
$ cpan XML::Simple
/home/gabor/perl5/perlbrew/perls/perl-5.12.3/bin/perl: symbol lookup error: /home/gabor/perl5/lib/perl5/x86_64-linux-gnu-thread-multi/auto/Cwd/Cwd.so: undefined symbol: Perl_Gthr_key_ptr
while
$ which cpan
/home/gabor/perl5/perlbrew/perls/perl-5.12.3/bin/cpan
so it is using the right version of the cpan client but dues to the PERL5LIB environment variable it picks up the modules from the wrong place.
Does perlbrew have some compability mode or do I need to turn off PERL5LIB and PERL_MM_OPT manually?

Since I started using perlbrew I stopped using local::lib for the shell use, because now that I have my own perl that i have write permissions to everything, just installing to site_perl is much more straightforward, and that allows me to have different versions of modules for each perl.
I still use local::lib (or more specifically, cpanm's -l or -L options that automatically sets up local::lib directory) to keep application specific dependencies inside an application directory.

local::lib was not designed to work with multiple versions of Perl installed at the same time. Pure-Perl modules aren't usually a problem, but XS modules aren't compatible across major releases.
You can continue to use local::lib for pure-Perl modules (so you don't have to install them for every version of Perl you have brewed up, but XS modules should be installed into the perlbrew-created directories. You don't need to clear PERL5LIB (and you shouldn't, as XS modules might have pure-Perl dependencies that are installed there), but you will need to clear PERL_MB_OPT and PERL_MM_OPT when installing XS modules to keep them from installing into the local::lib directory.
If you need to continue using local::lib for XS modules for the system Perl, then I suggest creating a second local::lib environment for that (perhaps in ~/perl5sys). It might be easier to use perlbrew to install a copy of the same version of Perl as the system Perl, and then use that instead of the system Perl.
You can clean out the XS modules in your existing local::lib by removing the /home/gabor/perl5/lib/perl5/x86_64-linux-gnu-thread-multi directory.

It is possible, but not comfortable. If this is a single-user setup, you might be better off not using local::lib and just letting perlbrew manage the modules for you. Also if it's a multi-user setup on a homogenous network, where everyone has the same machine and OS, then you can just set PERLBREW_ROOT to e.g. /net/share/perlbrew and then your installed perls (and their modules) can be shared. As noted in the other answers, this will be a problem if you try to mix machines (and possibly also problematic if you have different operating systems).
On a very diverse network, we prefer to keep everything separate. You can simply setup your local::lib to be a function of your current perl and your current platform, e.g.
distro=lsb_release -d|cut -f2|tr ' ' '-'
arch=`uname -m`
platform="$distro-$arch"
export PERLBREW_ROOT=/net/share/perlbrew/$platform
# You will have to first do 'perlbrew init' (just once for all users)
# In this case you don't need (and shouldn't have) a ~/.perlbrew
source $PERLBREW_ROOT/etc/bashrc
perl5base=/net/share/perl
# When $PERLBREW_PERL is not defined, local::lib puts modules in $perl5base/$platform
perl5=$perl5base/$platform/$PERLBREW_PERL
# We also found that we needed to clean PERL5LIB in between
export PERL5LIB=`echo -n $PERL5LIB|sed "s|${perl5base}[^:]*||g"`
export PATH=`echo -n $PATH|sed "s|${perl5base}[^:]*||g"`
# Setup local lib, relative to the perl being used
lib=$perl5/lib/perl5
mkdir -p $lib
eval $(perl -I"$lib" -Mlocal::lib="$perl5")
This is not our exact script, in particular you would need to check that these directories all exist first. You need to run perlbrew init once per platform and you need to bootstrap local::lib each time as well.
I don't recommend this approach, but provide as an example of one way to make this work, which it does for our mixed network (even on Mac OS). Leaving local::lib out and just using perlbrew (ignoring the system perl), would be a cleaner approach.

perlbrew is quite happy to use local::lib and has been for some time -- there are even special options for it -- after running perlbrew install ..., you can create a new local-lib directory with perlbrew lib create ...
for example:
perlbrew install -j 9 --as 34.0 5.34.0
chmod -R a-w $HOME/perl5/perlbrew/perls/34.0
perlbrew lib create 34.0#std
perlbrew switch 34.0#std
This installs a new 5.34.0 build, locks down its modules so you can't change them, then creates a local-lib directory. This install can be used with perlbrew use 34.0#std -- you can create a new set of module installations by perlbrew use 34.0; perlbrew lib create 34.0#other_install, to use side-by-side with the existing one without having to build a new perl again.

As miyagawa said, it might not be necessary to use local::lib if you use the Perls installed by Perlbrew exclusively.
But if you still want to be able to switch back and forth between your brewed Perls and the system Perl, there is a script called Perlswitcher for this. It's not pretty but it works. All you need to do is download the script, you could save it as ~/perl5/userperls/bashrc and source it.
It provides two commands. perlswitch allows you to switch to a Perl that was installed by Perlbrew or to the system Perl. perlinfo tells you which Perl is currently being used. You can then use cpanm, which will install packages to your local lib when using the system Perl or directly into the site Perl when using a custom Perl.
After switching to a custom Perl using perlswitch, perlbrew list will also know which Perl is being used:
$ perlswitch perl-5.18.4
Setting new perl /var/www/perl5/perlbrew/perls/perl-5.18.4/bin/perl...
Using user perl (site_perl) instead of local::lib
$ perlbrew list
perl-5.16.3
* perl-5.18.4
perl-5.20.2

Related

Why do official perl docker images have two version of perl?

I'm working on a legacy product which uses the Docker perl:5.10-threaded image and ran into an issue trying to debug things when I discovered there are two version of perl - one in /usr/local/bin/perl and one in /usr/bin/perl. In this particular image, they are actually different versions
/usr/local/bin/perl -> 5.10.1
/usr/bin/perl -> 5.20.2
The issue it was causing is that each has a different #INC path.
$ /usr/local/bin/perl -V
[snip]
/usr/local/lib/perl5/5.10.1/x86_64-linux-thread-multi
/usr/local/lib/perl5/5.10.1
/usr/local/lib/perl5/site_perl/5.10.1/x86_64-linux-thread-multi
/usr/local/lib/perl5/site_perl/5.10.1
$ /usr/bin/perl -V
[snip]
/etc/perl
/usr/local/lib/x86_64-linux-gnu/perl/5.20.2
/usr/local/share/perl/5.20.2
/usr/lib/x86_64-linux-gnu/perl5/5.20
/usr/share/perl5
/usr/lib/x86_64-linux-gnu/perl/5.20
/usr/share/perl/5.20
/usr/local/lib/site_perl
.
The latest versions, like perl:5.30-threaded also have two versions and different #INC paths as well.
/usr/local/bin/perl -> 5.30.0
/usr/bin/perl -> 5.28.1
It turns out my legacy app uses #!/usr/bin/perl, which was very confusing to me when perl foo.pl didn't work the same as foo.pl (it complained about missing libraries).
To add a bit more color, this legacy app also installs a bunch of perl libraries via apt-get. E.g.,
libcpanel-json-xs-perl libxml-libxml-perl libcgi-pm-perl
libtie-ixhash-perl
libswitch-perl libmime-lite-perl liblist-moreutils-perl
libdate-calc-perl libnet-sftp-foreign-perl
libxml-libxslt-perl liburi-escape-xs-perl
libdatetime-perl
These seem to install things accessible by /usr/bin/perl, which is perhaps why the app uses it in the shebang lines.
The other thing this app does is install some cpanm items and then copy them over, so they are accessible to /usr/bin/perl
RUN cpanm XML::XML2JSON
RUN cpanm JSON
RUN cp /usr/local/lib/perl5/site_perl/5.10.1/XML/XML2JSON.pm /usr/lib/x86_64-linux-gnu/perl5/5.20/XML
RUN cp /usr/local/lib/perl5/site_perl/5.10.1/JSON.pm /usr/lib/x86_64-linux-gnu/perl5/5.20
This seems like a hack to me, but I'm new to perl development, so who knows?
All this leads up to my questions:
why are there two version of perl in the Docker images?
are apt-get and cpanm incompatible with each other?
is there a better way to install these perl libs?
Perl is an essential part of many Linux distributions, and has to come pre-installed. The system perl that is used by the operating system is usually installed as /usr/bin/perl. Modules for it are managed through the package manager (e.g. apt) and not via cpan/cpanm. If you were to install modules for the system perl yourself, this might conflict with modules expected by the operating system. Worse, installing the wrong module version could break parts of the OS. Similarly, replacing the system perl is a bad idea. That's why those Docker images install the different perl alongside.
For your apps, you should avoid the system perl. If you want to install extra modules for use with the system perl, consider using local::lib. In some cases you might install dependencies such as C libraries or external tools via apt, but you wouldn't use apt-provided Perl modules.
Unless you are targeting a specific operating system, do not hardcode the #!/usr/bin/perl shebang. Instead, prefer #!/usr/bin/env perl so that the script will use the perl that is first in the PATH. Alternatively, use wrapper scripts to explicitly invoke the correct perl installation. For example:
#!/bin/sh
exec /usr/local/bin/perl -I/path/to/extra/modules /path/to/my/script "$#"
Note that you cannot share modules between different versions of Perl. During installation of XS modules they are compiled for a specific version, and will fail to load with a different Perl version. For your local perl, just install dependencies via cpanm and ignore modules that were installed for the system perl.

Using local Perl module instead of one installed by CPAN

I have found it necessary to expand upon a CPAN module. (Unicode::CharName goes up to Unicode 4.1; I need some characters from Unicode 5.0 & 5.1).
I've made the changes needed and have my own CharName.pm module.
I now would like to use it with my various Perls. I currently use:
Strawberry Perl for Windows
git for Windows MINGW64; My .bashrc sets
$PATH to Strawberry perl and $PERL5LIB=/c/Strawberry/perl/vendor/lib:/c/Strawberry/perl/site/lib
WSL Ubuntu
Where should I put my version of Unicode::CharName, so that it over-rides the ones installed by CPAN?
I don't want to have to change any scripts that currently
use Unicode::CharName;
Using cpanm you could download the module, patch it, and install it as normal:
$ cpanm --look Unicode::CharName
# new shell opens
$ patch lib/Unicode/CharName.pm custom.patch # or whatever process
$ perl Makefile.PL
$ make install
$ exit
You can also install it to a local::lib to avoid overwriting it globally, by adding the -l local/ option to the cpanm command. Then you can add the absolute path of this local::lib to your PERL5LIB or via -I or use lib. If you specified /path/to/local for the -l option, it would be /path/to/local/lib/perl5.
Manually copying files rather than going through the normal installation process is likely to lead to problems. Many distributions depend on the installation process to build the modules correctly. Also, you will need to install the module separately for each Perl you want to use it for; installed Perl modules are not generally cross-compatible between Perl versions or architectures. (A strictly simple pure-Perl module can be an exception to these rules, but the only module I feel comfortable abusing this way is App::cpanminus, because it was designed to do this.)

perl installation directory specification

I have perl 5.10.1 on my server but I need a higher version of perl than that. I do not have privileges to modify this version so I installed the latest one on my home directory. How do I specify unix to preferentially use the newer version?
The simplest answer is to use perlbrew. This is a tool to install and manage multiple Perl installations in your home directory. You can switch between which one is active.
Follow the instructions on their page to install perlbrew, and then install the latest stable Perl with...
perlbrew install stable
Then tell perlbrew that's the Perl you want to use with...
perlbrew switch stable
If you want to install more modules, use cpanm to install modules to the currently active perl.
cpanm Acme::Pony
If you don't want to use perlbrew, you need to mess with your PATH environment variable. This is a list of directories your shell will search through when finding a program to run. You'd need to put /path/to/where/you/installed/perl/bin at the front. More on this in this article.
I highly recommend you use perlbrew. It takes care of all this for you.

Installing Perl modules and dependencies with non-root and without CPAN

I have been writing Perl scripts for my work and the machine that I have been given to work on makes installing Perl modules difficult:
We cannot have gcc on my machine for security reasons, so I cannot use CPAN to install modules, for most modules.
I do not have access to the root account.
Usually, when I want to install a module, I put in a request and I have to wait a day or two before it gets installed. I know that nobody would have a problem with me installing them myself, so to save everyone's time and my sanity I would like to install them myself. It's just an issue of how to best do that. I have talked to various people and they said to use an RPM to install them (to get around not having gcc). However, when trying to install modules from RPMs, it does not handle the dependencies so I would manually need to handle the dependencies, which could take a while.
How can I best install Perl modules with these limitations?
On a similar machine with a similarly built Perl, install the module(s) using
mkdir ~/foo
cpan
o conf makepl_arg 'PREFIX=~/foo LIB=~/foo/lib/perl5'
o conf mbuildpl_arg '--prefix ~/foo --lib ~/foo/lib/perl5'
install Some::Module
As long as you don't do o conf commit, the configuration change will be temporary, so don't do that.
Copy ~/foo over, and set env var PERL5LIB to include the LIB directory. You can merge a newer ~/foo into an older one to add new modules.
This won't install any non-Perl libraries on which the modules depend.
See also How do I keep my own module/library directory? in section 8 of the Perl FAQ.
When you build modules, tell Perl where to install the modules.
For Makefile.PL-based distributions, use the INSTALL_BASE option when generating Makefiles:
perl Makefile.PL INSTALL_BASE=/mydir/perl
For Build.PL-based distributions, use the --install_base option:
perl Build.PL --install_base /mydir/perl
INSTALL_BASE tells these tools to put your modules into /mydir/perl/lib/perl5. See How do I add a directory to my include path (#INC) at runtime? for details on how to run your newly installed modules.
There is one caveat with INSTALL_BASE, though, since it acts differently from the PREFIX and LIB settings that older versions of ExtUtils::MakeMaker advocated. INSTALL_BASE does not support installing modules for multiple versions of Perl or different architectures under the same directory. You should consider whether you really want that and, if you do, use the older PREFIX and LIB settings. See the ExtUtils::Makemaker documentation for more details.

How can I use a new Perl module without install permissions?

Here is my situation: I know almost nothing about Perl but it is the only language available on a porting machine. I only have permissions to write in my local work area and not the Perl install location. I need to use the Parallel::ForkManager Perl module from CPAN
How do I use this Parallel::ForkManager without doing a central install? Is there an environment variable that I can set so it is located?
Thanks
JD
From perlfaq8: How do I keep my own module/library directory?:
When you build modules, tell Perl where to install the modules.
For C-based distributions, use the INSTALL_BASE option
when generating Makefiles:
perl Makefile.PL INSTALL_BASE=/mydir/perl
You can set this in your CPAN.pm configuration so modules automatically install
in your private library directory when you use the CPAN.pm shell:
% cpan
cpan> o conf makepl_arg INSTALL_BASE=/mydir/perl
cpan> o conf commit
For C-based distributions, use the --install_base option:
perl Build.PL --install_base /mydir/perl
You can configure CPAN.pm to automatically use this option too:
% cpan
cpan> o conf mbuild_arg --install_base /mydir/perl
cpan> o conf commit
INSTALL_BASE tells these tools to put your modules into
F. See L for details on how to run your newly
installed moudles.
There is one caveat with INSTALL_BASE, though, since it acts
differently than the PREFIX and LIB settings that older versions of
ExtUtils::MakeMaker advocated. INSTALL_BASE does not support
installing modules for multiple versions of Perl or different
architectures under the same directory. You should consider if you
really want that , and if you do, use the older PREFIX and LIB
settings. See the ExtUtils::Makemaker documentation for more details.
Download package form CPAN to a folder:
wget http://search.cpan.org/CPAN/authors/id/S/SZ/SZABGAB/Parallel-ForkManager-1.06.tar.gz
gunzip Parallel-ForkManager-1.06.tar.gz
tar -xvf Parallel-ForkManager-1.06.tar
before this create a folder in home to store your local modules, now go into downloaded folder and run follwing cmmands:
perl Makefile.PL PREFIX=/home/username/myModules
make
make test
make install
get the path to ForkManager from the installed folder,/home/username/myModules
and locate Parallel folder and get the full path to this.
Now in your perl file put these at the beggining
use lib '/home/username/myModules/bin.../Parallel';
use parallel::ForkManager;
--That should do it.
Check out this post from Mark Dominus
Excerpt:
Set PREFIX=X when building the Makefile
Set INSTALLDIRS=vendor and VENDORPREFIX=X when building the Makefile
Or maybe instead of VENDORPREFIX you need to set INSTALLVENDORLIB or something
Or maybe instead of setting them while building the Makefile you need to set them while running the make install target
Set LIB=X/lib when building the Makefile
Use PAR
Use local::lib
Mark also gives another solution in his blog which takes a bit more space to desribe but boils down to running make and make test but not make install and then using the stuff in blib/.
There's the PERL5LIB environment variable, and -I on the command line when it comes to using the module. There are mechanisms for telling CPAN and CPANPLUS.
There is information in question 5 of the CPAN manual (perldoc CPAN, or look at CPAN itself).
use lib 'directory';
use Parallel::ForkManager;
You can use the -I (capital i) command-line switch followed by the directory where you'll place the module; or try the "use lib" directive followed by the directory.
Yes Even You Can Use CPAN
perl Makefile.PL LIB=/my/perl_modules/lib/
make
make install
PERL5LIB=$PERL5LIB:/my/perl_modules/lib/
perl myperlcode.pl
use cpanm -l $DIR_NAME option.
perlbrew lets you use a local perl and installs it's packages to a local directory.
\curl -L https://install.perlbrew.pl | bash
perlbrew init # put this in .bash_profile etc
perlbrew install 5.27.11
perlbrew switch 5.27.11
See also https://opensource.com/article/18/7/perlbrew.
Consider using cpanminus, a suggested on this other thread