During installation with cpan (e.g. cpan XML:Parser) source code is compiled and sometimes requires additional variables to be set (in the example EXPATLIBPATH and EXPATINCPATH). cpan prints a helpful message what needs to be specified, but says that the specification needs to be ''Makefile'' argument/variables - when running cpan that doesn' add up.
I tried to specify environement variables with env (only INC is recognized for C includes, but the value is trimmed after the first space, so adding a -L directive doesn't work) and appending the variables which makes cpan search for modules with the variable name and argument. The manpage of cpan leaves this (quite crucial) matter completely uncovered. Setting LD_LIBRARY_PATH to the prefix I installed expat to doesn't work (header files isn't found).
If there's not a generic way, then I'd appreacite a solution for the cpan XML::Parser installation. I don't have root permissions on the system, so I need to specify the variables.
I use cpan 1.61 with perl 5.20.2 on Ubuntu 15.04 with Linux 4.0.1.
You can always do
install cpanm if you don't have it, with curl -L https://cpanmin.us | perl - App::cpanminus
cpanm --look XML::Parser
That'll download the needed tarballs, unpack, and open a shell inside the distribution. So you're free to do:
perl Makefile.PL
make with any options you need (like setting EXPATLIBPATH)
make test
(sudo) make install
Related
I have found it necessary to expand upon a CPAN module. (Unicode::CharName goes up to Unicode 4.1; I need some characters from Unicode 5.0 & 5.1).
I've made the changes needed and have my own CharName.pm module.
I now would like to use it with my various Perls. I currently use:
Strawberry Perl for Windows
git for Windows MINGW64; My .bashrc sets
$PATH to Strawberry perl and $PERL5LIB=/c/Strawberry/perl/vendor/lib:/c/Strawberry/perl/site/lib
WSL Ubuntu
Where should I put my version of Unicode::CharName, so that it over-rides the ones installed by CPAN?
I don't want to have to change any scripts that currently
use Unicode::CharName;
Using cpanm you could download the module, patch it, and install it as normal:
$ cpanm --look Unicode::CharName
# new shell opens
$ patch lib/Unicode/CharName.pm custom.patch # or whatever process
$ perl Makefile.PL
$ make install
$ exit
You can also install it to a local::lib to avoid overwriting it globally, by adding the -l local/ option to the cpanm command. Then you can add the absolute path of this local::lib to your PERL5LIB or via -I or use lib. If you specified /path/to/local for the -l option, it would be /path/to/local/lib/perl5.
Manually copying files rather than going through the normal installation process is likely to lead to problems. Many distributions depend on the installation process to build the modules correctly. Also, you will need to install the module separately for each Perl you want to use it for; installed Perl modules are not generally cross-compatible between Perl versions or architectures. (A strictly simple pure-Perl module can be an exception to these rules, but the only module I feel comfortable abusing this way is App::cpanminus, because it was designed to do this.)
I recently ran into some trouble installing some modules and discovered to my surprise that many of the modules that had been installed, have duplicated installations and versions. Trying to track where stuff goes in a standard (if there is such a thing) installation using cpanm, I found the following results very confusing.
The reports show these locations:
Using cpan -V:
# cpan -V
/usr/bin/cpan script version 1.672, CPAN.pm version 2.22
--------------------------------------------------
Checking install dirs...
Checking core
+ /usr/share/perl5/5.26
+ /usr/lib/perl5/5.26/x86_64-cygwin-threads
Checking vendor
+ /usr/share/perl5/vendor_perl/5.26
+ /usr/lib/perl5/vendor_perl/5.26/x86_64-cygwin-threads
Checking site
+ /usr/local/share/perl5/site_perl/5.26
+ /usr/local/lib/perl5/site_perl/5.26/x86_64-cygwin-threads
Checking PERL5LIB
no directories for PERL5LIB
Checking PERLLIB
no directories for PERLLIB
Using perl -V:.*site.*:
# perl -V:.*site.* |column -t -s "=" |sort -d -i -k 1.22
d_sitearch 'define';
usesitecustomize 'undef';
siteprefix '/usr/local';
siteprefixexp '/usr/local';
installsitebin '/usr/local/bin';
installsitescript '/usr/local/bin';
sitebin '/usr/local/bin';
sitebinexp '/usr/local/bin';
sitescript '/usr/local/bin';
sitescriptexp '/usr/local/bin';
installsitearch '/usr/local/lib/perl5/site_perl/5.26/x86_64-cygwin-threads';
sitearch '/usr/local/lib/perl5/site_perl/5.26/x86_64-cygwin-threads';
sitearchexp '/usr/local/lib/perl5/site_perl/5.26/x86_64-cygwin-threads';
installsitehtml1dir '/usr/local/share/doc/perl/html/html1';
sitehtml1dir '/usr/local/share/doc/perl/html/html1';
sitehtml1direxp '/usr/local/share/doc/perl/html/html1';
installsitehtml3dir '/usr/local/share/doc/perl/html/html3';
sitehtml3dir '/usr/local/share/doc/perl/html/html3';
sitehtml3direxp '/usr/local/share/doc/perl/html/html3';
installsiteman1dir '/usr/local/share/man/man1';
siteman1dir '/usr/local/share/man/man1';
siteman1direxp '/usr/local/share/man/man1';
installsiteman3dir '/usr/local/share/man/man3';
siteman3dir '/usr/local/share/man/man3';
siteman3direxp '/usr/local/share/man/man3';
installsitelib '/usr/local/share/perl5/site_perl/5.26';
sitelib '/usr/local/share/perl5/site_perl/5.26';
sitelib_stem '/usr/local/share/perl5/site_perl/5.26';
sitelibexp '/usr/local/share/perl5/site_perl/5.26';
Using #INC:
# perl -e 'print join("\n",#INC,"")'
/usr/local/lib/perl5/site_perl/5.26/x86_64-cygwin-threads
/usr/local/share/perl5/site_perl/5.26
/usr/lib/perl5/vendor_perl/5.26/x86_64-cygwin-threads
/usr/share/perl5/vendor_perl/5.26
/usr/lib/perl5/5.26/x86_64-cygwin-threads
/usr/share/perl5/5.26
The result is that cpan-outdated -p --verbose show a completely different (and shorter) list of outdated modules than what cpan -lO does. Needless to say, modules are installed all over the place and I don't see how to understand if there is a default install location and where it goes, or should go.
QUESTION:
So what is the difference between the core, vendor and site type of paths?
Why are there 2 paths in each type?
The best reference for these install locations is probably the ExtUtils::MakeMaker documentation for where it installs things. In essence:
core (also known as privlib) - is where core modules that are installed with Perl goes. On Perls older than 5.12, updates to dual-life modules also need to be installed here over the core versions instead of into site or vendor lib, because privlib came first in #INC before 5.12. This is especially dangerous in a system Perl where the files in privlib are usually managed by a package manager.
vendor - is where a distribution vendor may install modules to. This is usually where system package managers install non-core modules to.
site - is where CPAN clients install modules to when invoked directly, barring unusual configuration like for dual-life modules as mentioned above.
(Dual-life modules are core modules which are also available separately on CPAN, which means that you can install an updated version.)
Each of these lib locations have an arch variant, which is where distributions with build-specific output files are installed to. Pure-perl distributions with no dynamic configuration are installed into the standard architecture-agnostic directory, and can usually run unmodified in other installations of Perl and architectures provided their requirements are still satisfied (though it's not a good idea unless you really know what you are doing). Distributions with any compiled XS modules or that generate modules dynamically in the build process are installed into the arch directory and are not safe to use from another Perl.
All of these locations are configured when the Perl is built and can be discovered using the perl -V option as you showed. They also each have accompanying script and bin directories (which are usually the same) and directories for manpages.
As to the discrepancy of cpan-outdated - this tool (like many tools using ExtUtils::Installed) is restricted to finding modules that have packlists, which are included when installing a module using a CPAN client, but not with core modules and they are generally stripped from vendor packages. So most likely cpan-outdated will only discover modules in sitelib, but this is usually all you need to find. I'm not sure what mechanism the cpan command uses.
UPDATE:
Thanks to ikegami's SO link, I have since done some source browsing, where we can find a lot of info about this. To get the more specific locations:
# perl -V:'install(privlib|archlib|vendorlib|vendorarch|sitelib|sitearch)' |column -t -s "="
installarchlib '/usr/lib/perl5/5.26/x86_64-cygwin-threads';
installprivlib '/usr/share/perl5/5.26';
installsitearch '/usr/local/lib/perl5/site_perl/5.26/x86_64-cygwin-threads';
installsitelib '/usr/local/share/perl5/site_perl/5.26';
installvendorarch '/usr/lib/perl5/vendor_perl/5.26/x86_64-cygwin-threads';
installvendorlib '/usr/share/perl5/vendor_perl/5.26';
Then for the meaning of those, we can look (in the perl sources) in the file: ./Porting/Glossary:
installarchlib '- is the same for modules with arch- or build-dependent components.'
installprivlib '- contains the "pure Perl" modules that came with Perl.'
installsitearch '- is the same for modules with arch- or build-dependent components.'
installsitelib '- contains the "pure Perl" modules installed by you.'
installvendorarch '- is the same for modules with arch- or build-dependent components.'
installvendorlib '- contains the "pure Perl" modules installed by your distro.'
with the additional interesting note about the installstyle option:
installstyle (installstyle.U):
This variable describes the style of the perl installation. This
is intended to be useful for tools that need to manipulate entire perl
distributions. Perl itself doesn't use this to find its libraries --
the library directories are stored directly in Config.pm. Currently,
there are only two styles: lib and lib/perl5. The default
library locations (e.g. privlib, sitelib) are either $prefix/lib or
$prefix/lib/perl5. The former is useful if $prefix is a
directory dedicated to perl (e.g. /opt/perl), while the latter is
useful if $prefix is shared by many packages, e.g. if
$prefix=/usr/local.
Unfortunately, while this "style" variable is used to set defaults for
all three directory hierarchies (core, vendor, and site), there is no
guarantee that the same style is actually appropriate for all those
directories. For example, $prefix might be /opt/perl, but
$siteprefix might be /usr/local. (Perhaps, in retrospect, the "lib"
style should never have been supported, but it did seem like a nice
idea at the time.)
The situation is even less clear for tools such as MakeMaker that can
be used to install additional modules into non-standard places. For
example, if a user intends to install a module into a private
directory (perhaps by setting PREFIX on the Makefile.PL command line),
then there is no reason to assume that the Configure-time
$installstyle setting will be relevant for that PREFIX.
This may later be extended to include other information, so be careful
with pattern-matching on the results.
For compatibility with perl5.005 and earlier, the default setting is
based on whether or not $prefix contains the string "perl".
All the gritty details can then be found in the INSTALLATION file, under the Installation Directories heading.
Directories for the perl distribution:
By default, Configure will use the following directories (5.28.1):
Configure variable Default value
$prefixexp /usr/local
$binexp $prefixexp/bin
$scriptdirexp $prefixexp/bin
$privlibexp $prefixexp/lib/perl5/$version
$archlibexp $prefixexp/lib/perl5/$version/$archname
Directories for site-specific add-on files:
Configure Default
variable value
$siteprefixexp $prefixexp
$sitebinexp $siteprefixexp/bin
$sitescriptexp $siteprefixexp/bin
$sitelibexp $siteprefixexp/lib/perl5/site_perl/$version
$sitearchexp $siteprefixexp/lib/perl5/site_perl/$version/$archname
Directories for vendor-supplied add-on files:
If are building a binary distribution of perl for distribution, Configure
can optionally set up the following directories for you to use to
distribute add-on modules.
Configure Default
variable value
$vendorprefixexp (none)
(The next ones are set only if vendorprefix is set.)
$vendorbinexp $vendorprefixexp/bin
$vendorscriptexp $vendorprefixexp/bin
$vendorlibexp $vendorprefixexp/lib/perl5/vendor_perl/$version
$vendorarchexp $vendorprefixexp/lib/perl5/vendor_perl/$version/$archname
otherlibdirs:
As a final catch-all, Configure also offers an $otherlibdirs variable.
This variable contains a colon-separated list of additional directories
to add to #INC. By default, it will be empty.
APPLLIB_EXP:
There is one other way of adding paths to #INC at perl build
time, and that is by setting the APPLLIB_EXP C pre-processor
token. The directories defined by APPLLIB_EXP get added to
#INC first, ahead of any others.
sh Configure -Accflags='-DAPPLLIB_EXP=\"/usr/libperl\"'
So far I have been using the system perl (on Ubuntu 10.10) and I was using local::lib to install CPAN modules in my private directory ~/perl5
As I am trying to use perlbrew it seems that they don't know about each other. I installed perl-5.12.3 using perlbrew but when I switch to it using perlbrew use perl-5.12.3 I still see the PERL5LIB and PERL_MM_OPT set by local::lib.
That's not good:
$ cpan XML::Simple
/home/gabor/perl5/perlbrew/perls/perl-5.12.3/bin/perl: symbol lookup error: /home/gabor/perl5/lib/perl5/x86_64-linux-gnu-thread-multi/auto/Cwd/Cwd.so: undefined symbol: Perl_Gthr_key_ptr
while
$ which cpan
/home/gabor/perl5/perlbrew/perls/perl-5.12.3/bin/cpan
so it is using the right version of the cpan client but dues to the PERL5LIB environment variable it picks up the modules from the wrong place.
Does perlbrew have some compability mode or do I need to turn off PERL5LIB and PERL_MM_OPT manually?
Since I started using perlbrew I stopped using local::lib for the shell use, because now that I have my own perl that i have write permissions to everything, just installing to site_perl is much more straightforward, and that allows me to have different versions of modules for each perl.
I still use local::lib (or more specifically, cpanm's -l or -L options that automatically sets up local::lib directory) to keep application specific dependencies inside an application directory.
local::lib was not designed to work with multiple versions of Perl installed at the same time. Pure-Perl modules aren't usually a problem, but XS modules aren't compatible across major releases.
You can continue to use local::lib for pure-Perl modules (so you don't have to install them for every version of Perl you have brewed up, but XS modules should be installed into the perlbrew-created directories. You don't need to clear PERL5LIB (and you shouldn't, as XS modules might have pure-Perl dependencies that are installed there), but you will need to clear PERL_MB_OPT and PERL_MM_OPT when installing XS modules to keep them from installing into the local::lib directory.
If you need to continue using local::lib for XS modules for the system Perl, then I suggest creating a second local::lib environment for that (perhaps in ~/perl5sys). It might be easier to use perlbrew to install a copy of the same version of Perl as the system Perl, and then use that instead of the system Perl.
You can clean out the XS modules in your existing local::lib by removing the /home/gabor/perl5/lib/perl5/x86_64-linux-gnu-thread-multi directory.
It is possible, but not comfortable. If this is a single-user setup, you might be better off not using local::lib and just letting perlbrew manage the modules for you. Also if it's a multi-user setup on a homogenous network, where everyone has the same machine and OS, then you can just set PERLBREW_ROOT to e.g. /net/share/perlbrew and then your installed perls (and their modules) can be shared. As noted in the other answers, this will be a problem if you try to mix machines (and possibly also problematic if you have different operating systems).
On a very diverse network, we prefer to keep everything separate. You can simply setup your local::lib to be a function of your current perl and your current platform, e.g.
distro=lsb_release -d|cut -f2|tr ' ' '-'
arch=`uname -m`
platform="$distro-$arch"
export PERLBREW_ROOT=/net/share/perlbrew/$platform
# You will have to first do 'perlbrew init' (just once for all users)
# In this case you don't need (and shouldn't have) a ~/.perlbrew
source $PERLBREW_ROOT/etc/bashrc
perl5base=/net/share/perl
# When $PERLBREW_PERL is not defined, local::lib puts modules in $perl5base/$platform
perl5=$perl5base/$platform/$PERLBREW_PERL
# We also found that we needed to clean PERL5LIB in between
export PERL5LIB=`echo -n $PERL5LIB|sed "s|${perl5base}[^:]*||g"`
export PATH=`echo -n $PATH|sed "s|${perl5base}[^:]*||g"`
# Setup local lib, relative to the perl being used
lib=$perl5/lib/perl5
mkdir -p $lib
eval $(perl -I"$lib" -Mlocal::lib="$perl5")
This is not our exact script, in particular you would need to check that these directories all exist first. You need to run perlbrew init once per platform and you need to bootstrap local::lib each time as well.
I don't recommend this approach, but provide as an example of one way to make this work, which it does for our mixed network (even on Mac OS). Leaving local::lib out and just using perlbrew (ignoring the system perl), would be a cleaner approach.
perlbrew is quite happy to use local::lib and has been for some time -- there are even special options for it -- after running perlbrew install ..., you can create a new local-lib directory with perlbrew lib create ...
for example:
perlbrew install -j 9 --as 34.0 5.34.0
chmod -R a-w $HOME/perl5/perlbrew/perls/34.0
perlbrew lib create 34.0#std
perlbrew switch 34.0#std
This installs a new 5.34.0 build, locks down its modules so you can't change them, then creates a local-lib directory. This install can be used with perlbrew use 34.0#std -- you can create a new set of module installations by perlbrew use 34.0; perlbrew lib create 34.0#other_install, to use side-by-side with the existing one without having to build a new perl again.
As miyagawa said, it might not be necessary to use local::lib if you use the Perls installed by Perlbrew exclusively.
But if you still want to be able to switch back and forth between your brewed Perls and the system Perl, there is a script called Perlswitcher for this. It's not pretty but it works. All you need to do is download the script, you could save it as ~/perl5/userperls/bashrc and source it.
It provides two commands. perlswitch allows you to switch to a Perl that was installed by Perlbrew or to the system Perl. perlinfo tells you which Perl is currently being used. You can then use cpanm, which will install packages to your local lib when using the system Perl or directly into the site Perl when using a custom Perl.
After switching to a custom Perl using perlswitch, perlbrew list will also know which Perl is being used:
$ perlswitch perl-5.18.4
Setting new perl /var/www/perl5/perlbrew/perls/perl-5.18.4/bin/perl...
Using user perl (site_perl) instead of local::lib
$ perlbrew list
perl-5.16.3
* perl-5.18.4
perl-5.20.2
I've recently set up a new system and wanted to install Padre to check it out. The Padre install instructions specifically said to install local::lib, so I did so (although I've never had need of it before). I then went on my way installing several other modules, running CPAN from my normal user account with sudo to handle the root-required portions of the installation.
Then time came to test out one of the web apps these modules were needed to support and, lo and behold, apache couldn't find them. They loaded fine from the command line and a quick look in ~/perl5 confirmed my suspicion that local::lib had hijacked my CPAN sessions and installed these modules there instead of in a site-wide location, despite my CPAN config including
makepl_arg [INSTALLDIRS=site]
mbuildpl_arg [--installdirs site]
What do I need to do to my CPAN config so that modules will be installed site-wide even though local::lib is installed? Or will Padre work without it and I can just remove local::lib entirely?
(I do not want any modules installed under ~/perl5 unless Padre insists on them being there. My code under development has its own project-specific directory locations and everything else should be site-wide. I have no need for a private catch-all location.)
Got it. Per the instructions on local::lib's CPAN page, I had set export PERL_MM_OPT='INSTALL_BASE=~me/perl', which was overriding the setting in my CPAN config. A quick export PERL_MM_OPT= got me back to a proper install location.
On the one hand, that's what I get for following the instructions blindly. On the other, I would have expected o conf to show the actual config settings that are being used rather than those which are in the saved CPAN config (even if an environment variable is overriding them).
A quick export PERL_MM_OPT= got me back to a proper install location.
This didn't help me when I tried to install Starman globally. I had to edit /root/.bashrc to comment (or to delete) next lines:
export PERL_LOCAL_LIB_ROOT="$PERL_LOCAL_LIB_ROOT:/root/perl5";
export PERL_MB_OPT="--install_base /root/perl5";
export PERL_MM_OPT="INSTALL_BASE=/root/perl5";
export PERL5LIB="/root/perl5/lib/perl5:$PERL5LIB";
export PATH="/root/perl5/bin:$PATH";
Based on this comment, it seems that local::lib installs its own version of CPAN.pm. If this is the case, you may need to find the original CPAN.pm and make sure that that one is loaded when you run your CPAN shell. Something like:
perl -I /usr/lib/perl5 -MCPAN -e shell
might do it. You might also find perl -V useful to see what include path the Perl compiler is using for its modules.
Here is my situation: I know almost nothing about Perl but it is the only language available on a porting machine. I only have permissions to write in my local work area and not the Perl install location. I need to use the Parallel::ForkManager Perl module from CPAN
How do I use this Parallel::ForkManager without doing a central install? Is there an environment variable that I can set so it is located?
Thanks
JD
From perlfaq8: How do I keep my own module/library directory?:
When you build modules, tell Perl where to install the modules.
For C-based distributions, use the INSTALL_BASE option
when generating Makefiles:
perl Makefile.PL INSTALL_BASE=/mydir/perl
You can set this in your CPAN.pm configuration so modules automatically install
in your private library directory when you use the CPAN.pm shell:
% cpan
cpan> o conf makepl_arg INSTALL_BASE=/mydir/perl
cpan> o conf commit
For C-based distributions, use the --install_base option:
perl Build.PL --install_base /mydir/perl
You can configure CPAN.pm to automatically use this option too:
% cpan
cpan> o conf mbuild_arg --install_base /mydir/perl
cpan> o conf commit
INSTALL_BASE tells these tools to put your modules into
F. See L for details on how to run your newly
installed moudles.
There is one caveat with INSTALL_BASE, though, since it acts
differently than the PREFIX and LIB settings that older versions of
ExtUtils::MakeMaker advocated. INSTALL_BASE does not support
installing modules for multiple versions of Perl or different
architectures under the same directory. You should consider if you
really want that , and if you do, use the older PREFIX and LIB
settings. See the ExtUtils::Makemaker documentation for more details.
Download package form CPAN to a folder:
wget http://search.cpan.org/CPAN/authors/id/S/SZ/SZABGAB/Parallel-ForkManager-1.06.tar.gz
gunzip Parallel-ForkManager-1.06.tar.gz
tar -xvf Parallel-ForkManager-1.06.tar
before this create a folder in home to store your local modules, now go into downloaded folder and run follwing cmmands:
perl Makefile.PL PREFIX=/home/username/myModules
make
make test
make install
get the path to ForkManager from the installed folder,/home/username/myModules
and locate Parallel folder and get the full path to this.
Now in your perl file put these at the beggining
use lib '/home/username/myModules/bin.../Parallel';
use parallel::ForkManager;
--That should do it.
Check out this post from Mark Dominus
Excerpt:
Set PREFIX=X when building the Makefile
Set INSTALLDIRS=vendor and VENDORPREFIX=X when building the Makefile
Or maybe instead of VENDORPREFIX you need to set INSTALLVENDORLIB or something
Or maybe instead of setting them while building the Makefile you need to set them while running the make install target
Set LIB=X/lib when building the Makefile
Use PAR
Use local::lib
Mark also gives another solution in his blog which takes a bit more space to desribe but boils down to running make and make test but not make install and then using the stuff in blib/.
There's the PERL5LIB environment variable, and -I on the command line when it comes to using the module. There are mechanisms for telling CPAN and CPANPLUS.
There is information in question 5 of the CPAN manual (perldoc CPAN, or look at CPAN itself).
use lib 'directory';
use Parallel::ForkManager;
You can use the -I (capital i) command-line switch followed by the directory where you'll place the module; or try the "use lib" directive followed by the directory.
Yes Even You Can Use CPAN
perl Makefile.PL LIB=/my/perl_modules/lib/
make
make install
PERL5LIB=$PERL5LIB:/my/perl_modules/lib/
perl myperlcode.pl
use cpanm -l $DIR_NAME option.
perlbrew lets you use a local perl and installs it's packages to a local directory.
\curl -L https://install.perlbrew.pl | bash
perlbrew init # put this in .bash_profile etc
perlbrew install 5.27.11
perlbrew switch 5.27.11
See also https://opensource.com/article/18/7/perlbrew.
Consider using cpanminus, a suggested on this other thread