I have found it necessary to expand upon a CPAN module. (Unicode::CharName goes up to Unicode 4.1; I need some characters from Unicode 5.0 & 5.1).
I've made the changes needed and have my own CharName.pm module.
I now would like to use it with my various Perls. I currently use:
Strawberry Perl for Windows
git for Windows MINGW64; My .bashrc sets
$PATH to Strawberry perl and $PERL5LIB=/c/Strawberry/perl/vendor/lib:/c/Strawberry/perl/site/lib
WSL Ubuntu
Where should I put my version of Unicode::CharName, so that it over-rides the ones installed by CPAN?
I don't want to have to change any scripts that currently
use Unicode::CharName;
Using cpanm you could download the module, patch it, and install it as normal:
$ cpanm --look Unicode::CharName
# new shell opens
$ patch lib/Unicode/CharName.pm custom.patch # or whatever process
$ perl Makefile.PL
$ make install
$ exit
You can also install it to a local::lib to avoid overwriting it globally, by adding the -l local/ option to the cpanm command. Then you can add the absolute path of this local::lib to your PERL5LIB or via -I or use lib. If you specified /path/to/local for the -l option, it would be /path/to/local/lib/perl5.
Manually copying files rather than going through the normal installation process is likely to lead to problems. Many distributions depend on the installation process to build the modules correctly. Also, you will need to install the module separately for each Perl you want to use it for; installed Perl modules are not generally cross-compatible between Perl versions or architectures. (A strictly simple pure-Perl module can be an exception to these rules, but the only module I feel comfortable abusing this way is App::cpanminus, because it was designed to do this.)
Related
I'm working on a legacy product which uses the Docker perl:5.10-threaded image and ran into an issue trying to debug things when I discovered there are two version of perl - one in /usr/local/bin/perl and one in /usr/bin/perl. In this particular image, they are actually different versions
/usr/local/bin/perl -> 5.10.1
/usr/bin/perl -> 5.20.2
The issue it was causing is that each has a different #INC path.
$ /usr/local/bin/perl -V
[snip]
/usr/local/lib/perl5/5.10.1/x86_64-linux-thread-multi
/usr/local/lib/perl5/5.10.1
/usr/local/lib/perl5/site_perl/5.10.1/x86_64-linux-thread-multi
/usr/local/lib/perl5/site_perl/5.10.1
$ /usr/bin/perl -V
[snip]
/etc/perl
/usr/local/lib/x86_64-linux-gnu/perl/5.20.2
/usr/local/share/perl/5.20.2
/usr/lib/x86_64-linux-gnu/perl5/5.20
/usr/share/perl5
/usr/lib/x86_64-linux-gnu/perl/5.20
/usr/share/perl/5.20
/usr/local/lib/site_perl
.
The latest versions, like perl:5.30-threaded also have two versions and different #INC paths as well.
/usr/local/bin/perl -> 5.30.0
/usr/bin/perl -> 5.28.1
It turns out my legacy app uses #!/usr/bin/perl, which was very confusing to me when perl foo.pl didn't work the same as foo.pl (it complained about missing libraries).
To add a bit more color, this legacy app also installs a bunch of perl libraries via apt-get. E.g.,
libcpanel-json-xs-perl libxml-libxml-perl libcgi-pm-perl
libtie-ixhash-perl
libswitch-perl libmime-lite-perl liblist-moreutils-perl
libdate-calc-perl libnet-sftp-foreign-perl
libxml-libxslt-perl liburi-escape-xs-perl
libdatetime-perl
These seem to install things accessible by /usr/bin/perl, which is perhaps why the app uses it in the shebang lines.
The other thing this app does is install some cpanm items and then copy them over, so they are accessible to /usr/bin/perl
RUN cpanm XML::XML2JSON
RUN cpanm JSON
RUN cp /usr/local/lib/perl5/site_perl/5.10.1/XML/XML2JSON.pm /usr/lib/x86_64-linux-gnu/perl5/5.20/XML
RUN cp /usr/local/lib/perl5/site_perl/5.10.1/JSON.pm /usr/lib/x86_64-linux-gnu/perl5/5.20
This seems like a hack to me, but I'm new to perl development, so who knows?
All this leads up to my questions:
why are there two version of perl in the Docker images?
are apt-get and cpanm incompatible with each other?
is there a better way to install these perl libs?
Perl is an essential part of many Linux distributions, and has to come pre-installed. The system perl that is used by the operating system is usually installed as /usr/bin/perl. Modules for it are managed through the package manager (e.g. apt) and not via cpan/cpanm. If you were to install modules for the system perl yourself, this might conflict with modules expected by the operating system. Worse, installing the wrong module version could break parts of the OS. Similarly, replacing the system perl is a bad idea. That's why those Docker images install the different perl alongside.
For your apps, you should avoid the system perl. If you want to install extra modules for use with the system perl, consider using local::lib. In some cases you might install dependencies such as C libraries or external tools via apt, but you wouldn't use apt-provided Perl modules.
Unless you are targeting a specific operating system, do not hardcode the #!/usr/bin/perl shebang. Instead, prefer #!/usr/bin/env perl so that the script will use the perl that is first in the PATH. Alternatively, use wrapper scripts to explicitly invoke the correct perl installation. For example:
#!/bin/sh
exec /usr/local/bin/perl -I/path/to/extra/modules /path/to/my/script "$#"
Note that you cannot share modules between different versions of Perl. During installation of XS modules they are compiled for a specific version, and will fail to load with a different Perl version. For your local perl, just install dependencies via cpanm and ignore modules that were installed for the system perl.
During installation with cpan (e.g. cpan XML:Parser) source code is compiled and sometimes requires additional variables to be set (in the example EXPATLIBPATH and EXPATINCPATH). cpan prints a helpful message what needs to be specified, but says that the specification needs to be ''Makefile'' argument/variables - when running cpan that doesn' add up.
I tried to specify environement variables with env (only INC is recognized for C includes, but the value is trimmed after the first space, so adding a -L directive doesn't work) and appending the variables which makes cpan search for modules with the variable name and argument. The manpage of cpan leaves this (quite crucial) matter completely uncovered. Setting LD_LIBRARY_PATH to the prefix I installed expat to doesn't work (header files isn't found).
If there's not a generic way, then I'd appreacite a solution for the cpan XML::Parser installation. I don't have root permissions on the system, so I need to specify the variables.
I use cpan 1.61 with perl 5.20.2 on Ubuntu 15.04 with Linux 4.0.1.
You can always do
install cpanm if you don't have it, with curl -L https://cpanmin.us | perl - App::cpanminus
cpanm --look XML::Parser
That'll download the needed tarballs, unpack, and open a shell inside the distribution. So you're free to do:
perl Makefile.PL
make with any options you need (like setting EXPATLIBPATH)
make test
(sudo) make install
I'm a C/C++/Java/Unix geek by trade, but now I have to write a Perl program in Windows. So I've got Cygwin, Eclipse with EPIC installed, and simple test programs run. I do not have a Unix machine available to me that I can run Eclipse on.
But I need to use Net::TCP::Server and I can't figure out how to install it. It doesn't appear to be part of the Cygwin Perl install, and I don't want to spend 5 days learning the world of Perl and CPAN and Perl configuration. I just want to write my program.
Any simple ways of installing a Perl module in Cygwin? I'm thinking something like apt-get install would be nice, but I expect that's too much to hope for.
Thanks
$ perl -MCPAN -e shell
cpan shell -- CPAN exploration and modules installation (v1.9402)
Enter 'h' for help.
cpan[1]> install Net::TCP::Server
And it's instructive to list the configuration with the o conf command.
Seeing that some of the info here is a bit outdated and too complicated, I'd rather suggest the following. There are a few different Perl package managers in use. They are all installed with cpan (which is already part of the Cygwin Perl installation), like this:
# Install ppm (outdated)
cpan PPM
# Install cpanp (still used)
cpan CPANPLUS
# Install cpanm (most recent)
cpan App::cpanminus
Then you can install any Perl package you like, as for example in the OP, using cpanm:
cpanm Net::TCP::Server
Sometimes (as noted above) Cygwin may fail certain tests. For example, when using IPv6 on a machine only configured with IPv4, or when your windows firewall is blocking some tests, etc. To attempt to install anyway, try to use the force flag; -f.
cpanm -f Net::TCP::Server
I'm a C/C++/java unix geek by trade, but now I have to write a perl program in windows. So I've got cygwin, eclipse with EPIC installed, and simple test programs run. I do not have a unix machine available to me that I can run eclipse on.
You should be able to run Eclipse with EPIC right under Windows without Cygwin. I like Cygwin for many things, but it isn't exactly a very stable platform. Eclipse runs as a Java program, so all you have to do is make sure Java is installed on your PC. There is even a pre-built Eclipse package.
You can also get a decent Perl that runs right under Windows. Either ActivePerl or Strawberry Perl. Either one will work although many Unix heads prefer Strawberry Perl because it comes with the same MIGW environment that Cygwin has and many feel it's more compatible with the Unix Perl brethren. However, ActiveState Perl can use the Perl Package Manager (PPM) that contains pre-built modules. Unfortunately, the module you want isn't available via PPM.
But I need to use Net::TCP::Server and I can't figure out how to install it. It doesn't appear to be part of the cygwin perl install, and I don't want to spend 5 days learning the world of perl and cpan and perl configuration. I just want to write my program.
Installing from CPAN is really quite simple. The first time you run CPAN, it should pretty much configure itself. Then, to do an install, you run cpan install Net::TCP::Server and that should be it. However, I've tried on several Mac, Unix, and Linux machines, and I can't get it to work. So, this isn't a Windows problem as much as a problem with this module. It is fairly old, and might not work well in newer versions of Perl. You might have to force the install over test's objections.
Maybe you can use one of the already installed IO modules that come with Perl instead. That'll save you a boatload of trouble because the required modules are part of Perl's existing package.
Despite Cygwin's "problems," I use it regularly whenever I have to use Windows. I would recommend first installing a separate installation of Perl using perlbrew so that you won't interfere with Cygwin's copy of Perl in case something bad happens since Cygwin does not enforce root-user policy. In cygwin shell, type
\curl -L http://install.perlbrew.pl | bash
This should walk through the installation for perlbrew and set it up in one of your executable path. Next type
perlbrew init
perlbrew install --force stable
perlbrew switch stable
Wait a bit while a mint Perl is compiled. For unknown reason, Perl can only pass 99.23% of the core module tests on Cygwin (at least on my machine),hence the --force flag). My experience is that it mostly have something to do with handling of device files, like ports and pipes. I am unaware of people trying to resolve the issue as it seems like a Cygwin problem. Although it has not presented much problem for me with general system and web programming tasks. The module testing routines will fail if any problem exists so I am not fretting over it.
Next step is to install cpanm (cpanminus), type
perlbrew install-cpanm
From here on out, to install any library from CPAN, just type
cpanm [library::name1] [library::name2]
cpanm makes it trivial to install any Perl modules. You can even install from your local directory instead of CPAN.
Your mileage may vary, but I had a lot of trouble until I realized that Strawberry perl had a lot of bin folders in my PATH, and when I changed my .bashrc to export only a very simple PATH=/bin:/usr/bin:/usr/local/bin, Cygwin's perl installation's cpan started working beautifully. I used local:lib as Cygwin doesn't support sudo. Before it got into a bad loop saying "Press SPACE and ENTER to exit Patch" over and over.
Yesterday I wanted to test some software and in the documentation it said, to install I just needed to type
cpan -i Software
I never used cpan, I just know that it is the perl package manager. (Is it..?) However, it turned out that I needed loads of dependencies, and stupid as I am, I just installed all of them. (First, I had to set up cpan which asked me lots of questions) Long story short, I just want to remove all of it again. I googled a bit, and it seems like cpan does not have an uninstall routine, especially for all the packages at once.
Can I just remove some directory or will I run into troubles?
the cpan command isn't really a package manager like apt-get is. It is more a tool that downloads and installs from CPAN (the site, or one of its mirrors). After it has finished this task it doesn't remember much about what was done before, at least not enough to remove previously installed modules, at least not reliably, cleanly or dependency-safely. (Update: After looking at App::pmuninstall, it can be used to handle dependencies, but it does so by connecting to outside (read: web) sources, which compute these separately, which is fine, but I stand by the previous statement that CPAN.pm doesn't do this.)
I used to worry about removing modules, but now I realize that most Perl modules take up so little room that I just don't worry about having a few extra modules installed that you will never use. So unless you are on a computer with a REALLY small disc, I would just let it be.
On Windows or if you are using a non-system Perl on Linux/Mac you could just remove Perl and reinstall it. I would not recommend this if you are using the system installed Perl on Linux/Mac however as you could break your OS doing this (you might be ok if you were careful, but not worth it to save a few Mb!).
In the future, you can easily install a local version of Perl using perlbrew, there are tutorials all over the web if the docs aren't sufficient (they should be). This also has the bonus of letting you play with the newest and greatest Perl versions, which your system likely doesn't come with yet. Then if you install a mountain of junk, or even break it doing something crazy, remove that version and reinstall/install a different version.
Another nice tool is cpanminus (or cpanm for short) which is a newer, more user friendly cpan tool. All the cool kids are using it.
You can uninstall individual modules with cpanplus (ships with Perl) like this:
cpanp uninstall SQL::Abstract
You can view all modules installed with the cpan script like this:
perldoc perllocal
Putting the two together:
for module in $(perldoc -u perllocal | grep -F 'C<Module> L<' | sed 's/^.*L<\(.*\)|.*>$/\1/') ; do
cpanp uninstall "$module"
done
I'm not sure about removing "all of it". But to remove a single module you can use App::pmuninstall with it's sole script pm-uninstall to uninstall modules. You might then be able to write some kind of script to recursively remove the deps.
If you can't use cpan any more because there are incompatible modules in you path, you can remove all installed modules by hand. For example, I upgraded from Fedora 22 to Fedora 23 and the Perl version changed. All modules installed previously via cpanm into /usr/local/lib64/perl5 did not work any more and prevented me from using cpanm.
$ cpanm --uninstall Apache::DBI
Attempt to reload Scalar/Util.pm aborted.
Compilation failed in require at /usr/share/perl5/vendor_perl/File/Temp.pm line 18.
...
I could solve this by moving that directory:
$ mv /usr/local/lib64/perl5 /root/usr-local-lib64-perl5
The name of that directory may vary on your system.
Carefull: If a module installed files outside of that directory, for example system library files, these files will remain there.
I will change Flimm's answer to use cpanm and optionally uninstall cpanm itself in the end of the script:
#!/usr/bin/env bash
for module in $(perldoc -u perllocal | grep -F 'C<Module> L<' | sed 's/^.*L<\(.*\)|.*>$/\1/' | sort | uniq) ; do
if [[ "$module" =~ "App::cpanminus" ]]; then
continue
fi
echo "Uninstalling $module..."
yes | cpanm --uninstall "$module"
done
cpanm --uninstall App::cpanminus
I think the best option is uninstall Perl and install it again.
So far I have been using the system perl (on Ubuntu 10.10) and I was using local::lib to install CPAN modules in my private directory ~/perl5
As I am trying to use perlbrew it seems that they don't know about each other. I installed perl-5.12.3 using perlbrew but when I switch to it using perlbrew use perl-5.12.3 I still see the PERL5LIB and PERL_MM_OPT set by local::lib.
That's not good:
$ cpan XML::Simple
/home/gabor/perl5/perlbrew/perls/perl-5.12.3/bin/perl: symbol lookup error: /home/gabor/perl5/lib/perl5/x86_64-linux-gnu-thread-multi/auto/Cwd/Cwd.so: undefined symbol: Perl_Gthr_key_ptr
while
$ which cpan
/home/gabor/perl5/perlbrew/perls/perl-5.12.3/bin/cpan
so it is using the right version of the cpan client but dues to the PERL5LIB environment variable it picks up the modules from the wrong place.
Does perlbrew have some compability mode or do I need to turn off PERL5LIB and PERL_MM_OPT manually?
Since I started using perlbrew I stopped using local::lib for the shell use, because now that I have my own perl that i have write permissions to everything, just installing to site_perl is much more straightforward, and that allows me to have different versions of modules for each perl.
I still use local::lib (or more specifically, cpanm's -l or -L options that automatically sets up local::lib directory) to keep application specific dependencies inside an application directory.
local::lib was not designed to work with multiple versions of Perl installed at the same time. Pure-Perl modules aren't usually a problem, but XS modules aren't compatible across major releases.
You can continue to use local::lib for pure-Perl modules (so you don't have to install them for every version of Perl you have brewed up, but XS modules should be installed into the perlbrew-created directories. You don't need to clear PERL5LIB (and you shouldn't, as XS modules might have pure-Perl dependencies that are installed there), but you will need to clear PERL_MB_OPT and PERL_MM_OPT when installing XS modules to keep them from installing into the local::lib directory.
If you need to continue using local::lib for XS modules for the system Perl, then I suggest creating a second local::lib environment for that (perhaps in ~/perl5sys). It might be easier to use perlbrew to install a copy of the same version of Perl as the system Perl, and then use that instead of the system Perl.
You can clean out the XS modules in your existing local::lib by removing the /home/gabor/perl5/lib/perl5/x86_64-linux-gnu-thread-multi directory.
It is possible, but not comfortable. If this is a single-user setup, you might be better off not using local::lib and just letting perlbrew manage the modules for you. Also if it's a multi-user setup on a homogenous network, where everyone has the same machine and OS, then you can just set PERLBREW_ROOT to e.g. /net/share/perlbrew and then your installed perls (and their modules) can be shared. As noted in the other answers, this will be a problem if you try to mix machines (and possibly also problematic if you have different operating systems).
On a very diverse network, we prefer to keep everything separate. You can simply setup your local::lib to be a function of your current perl and your current platform, e.g.
distro=lsb_release -d|cut -f2|tr ' ' '-'
arch=`uname -m`
platform="$distro-$arch"
export PERLBREW_ROOT=/net/share/perlbrew/$platform
# You will have to first do 'perlbrew init' (just once for all users)
# In this case you don't need (and shouldn't have) a ~/.perlbrew
source $PERLBREW_ROOT/etc/bashrc
perl5base=/net/share/perl
# When $PERLBREW_PERL is not defined, local::lib puts modules in $perl5base/$platform
perl5=$perl5base/$platform/$PERLBREW_PERL
# We also found that we needed to clean PERL5LIB in between
export PERL5LIB=`echo -n $PERL5LIB|sed "s|${perl5base}[^:]*||g"`
export PATH=`echo -n $PATH|sed "s|${perl5base}[^:]*||g"`
# Setup local lib, relative to the perl being used
lib=$perl5/lib/perl5
mkdir -p $lib
eval $(perl -I"$lib" -Mlocal::lib="$perl5")
This is not our exact script, in particular you would need to check that these directories all exist first. You need to run perlbrew init once per platform and you need to bootstrap local::lib each time as well.
I don't recommend this approach, but provide as an example of one way to make this work, which it does for our mixed network (even on Mac OS). Leaving local::lib out and just using perlbrew (ignoring the system perl), would be a cleaner approach.
perlbrew is quite happy to use local::lib and has been for some time -- there are even special options for it -- after running perlbrew install ..., you can create a new local-lib directory with perlbrew lib create ...
for example:
perlbrew install -j 9 --as 34.0 5.34.0
chmod -R a-w $HOME/perl5/perlbrew/perls/34.0
perlbrew lib create 34.0#std
perlbrew switch 34.0#std
This installs a new 5.34.0 build, locks down its modules so you can't change them, then creates a local-lib directory. This install can be used with perlbrew use 34.0#std -- you can create a new set of module installations by perlbrew use 34.0; perlbrew lib create 34.0#other_install, to use side-by-side with the existing one without having to build a new perl again.
As miyagawa said, it might not be necessary to use local::lib if you use the Perls installed by Perlbrew exclusively.
But if you still want to be able to switch back and forth between your brewed Perls and the system Perl, there is a script called Perlswitcher for this. It's not pretty but it works. All you need to do is download the script, you could save it as ~/perl5/userperls/bashrc and source it.
It provides two commands. perlswitch allows you to switch to a Perl that was installed by Perlbrew or to the system Perl. perlinfo tells you which Perl is currently being used. You can then use cpanm, which will install packages to your local lib when using the system Perl or directly into the site Perl when using a custom Perl.
After switching to a custom Perl using perlswitch, perlbrew list will also know which Perl is being used:
$ perlswitch perl-5.18.4
Setting new perl /var/www/perl5/perlbrew/perls/perl-5.18.4/bin/perl...
Using user perl (site_perl) instead of local::lib
$ perlbrew list
perl-5.16.3
* perl-5.18.4
perl-5.20.2