I'm investigating making a Perl application that uses many modules into either a Debian and/or Redhat package. Currently, I believe the 'cleanest' way to do this is to cite, where possible, the modules that are packaged already for the given distribution.
The alternative would be to use CPAN and probably have some duplications, problems with #INC etc.
However, I can find or interrogate a list of Debian packages here: http://pkg-perl.alioth.debian.org/cpan2deb/ but I can't currently find an equivalent for Redhat/Fedora. Also I don't really know whether cpan2deb is authoritative and up to date.
If there's another clean way to do this, I'd welcome any other ideas too.
The Debian Perl Group is your best bet on the Debian world. Not only do they intercept all spread modules packages for Debian but they also try to keep them up to date.
See this page:
http://qa.debian.org/developer.php?login=pkg-perl-maintainers#lists.alioth.debian.org
There is cpanspec but it's not been touched in a few years. I seem to remember cpan2rpm but I don't have much experience with that one.
The alternative would be to use CPAN and probably have some duplications, problems with #INC etc.
I've got a perl program packaged for debian with a large number of dependencies. For expediency, I've chosen a grubby hybrid approach with some packaged modules as dependencies, plus a cheat backdoor CPAN install, which runs from my post-installation script. I hive a copy of my application's META.yml, then recheck my dependencies.
1.debian/rules file makes a copy of META.yml:
override_dh_auto_install:
dh_auto_install; \
cp META.yml etc/;\
2.debian/libmyapp-perl.install then installs META.yml:
etc/META.yml /usr/share/myapp/etc/
3.debian/libmyapp-perl.postinstall then cross-checks dependencies:
echo "Cross checking with cpan";
for m in `perl -Mstrict -MYAML::Syck -e'my $r = YAML::Syck::LoadFile("/usr/share/myapp/etc/META.yml")->{requires}; for (grep {$_ ne "perl"} (sort keys %$r)) {eval "use $_ $r->{$_}"; print "$_\n" if $#}'`; do
# would prefer App::cpanminus, but that's not packaged for debian either?
PERL_MM_USE_DEFAULT=1 perl -MCPAN -e "install $m";
done
Not exactly clean, but a quick approach to installing a mix of dependent debian packages plus a few CPAN modules.
Related
I have a clean install of perl 5.26.3, and I am trying to install Dancer2.
Dancer2 depends on App::Cmd::Setup, which depends on Getopt::Long::Descriptive, which depends on Params::Validate, which depends on Devel::Peek.
Looking at Devel::Peek on MetaCPAN, it is part of the perl 5.30.2 distribution, and I can not find the Devel::Peek package by itself. The "Jump to version" list on the Devel::Peek page lists previous version of perl, not of Devel::Peek.
Installing Dancer2 on perl 5.26.3 apparently has a hard, transitive dependency of installing perl 5.30.2, but I do not have the authority -- nor any desire -- to update the perl installation.
Is there a version of Devel::Peek available by itself?
Thank you.
Edit
It looks like I have a broken perl installation. I have no Data/Peek.pm anywhere on the filesystem. When I try to use Devel::Peek, I am told Can't locate Devel/Peek.pm in #INC.
I'll install from scratch and start over.
You should be able to fix it with the installation of
yum perl-Devel-Peek
For alpine, be sure to do apk add perl-dev to add additional modules needed to build some packages.
No. It's only available as part of Perl. It's has been part of Perl since 5.6 in 2000.
As such, if you don't have Devel::Peek, you have a broken or partial Perl installation. But this is actually quite common, since many package managers break down Perl into different packages. For example, RHEL split off Devel::Peek into the perl-Devel-Peek package. So if we're talking about a Perl managed by a package manager, check the package manager for the missing bits.
I am using perlbrew. I have installed lots of cpan modules under perlbrew perl-5.20.2.
Is there a way (or what is the best way) to do a tarball installation of my perl-5.20.2 plus all the CPAN modules that I have installed under perlbrew so that I can just clone it unto another machine?
I am aware of perlbrew download perl-5.20.2 but that only seems to tarball perl-5.20.2 and not all the CPAN modules that I have installed.
With perlbrew you can use the lib command to create a local::lib to go with your perlbrew perl.
perlbrew lib create perl-5.20.2#app_reqs
Then, if all goes well, when you install modules you will find them in:
$HOME/.perlbrew/libs/perl-5.20.2#app_reqs
If you don't use the perbrew lib create approach to managing your modules, then they are installed to $HOME/perl5/perlbrew/perls/perl-5.20.1/lib/site_perl/5.20.2. Cloning either of those directories might work, but you are likely better off reinstalling all the modules using the techniques from the perlbrew.pl website since XS modules should be rebuilt etc.
If you want to reuse and track local sources the most reliable approach is to create a local CPAN mirror to work from using App::lcpan or minicpan. If you have already downloaded the source using cpanm a quick hackish approach is to find the source files (under $HOME/.cpanm/) and do something like this in your shell:
% mkdir ~/cpansourcefiles
% for source in ~/.cpanm/work/*/* ; do cp $source ~/cpansourcefiles ;done
Then you can get cpanm to install using those sources by passing the filename as the argument instead of the module name:
% cpanm ~/cpansourcefiles/List-MoreUtils-0.406.tar.gz
or even:
% cpanm ~/cpansourcefiles/*
NB: YMMV since this is prone to breakage and might skip some of the version and dependency checking you normally get with cpanm but it is simpler than setting up a mirror when it works.
A few other high powered tools that make deploying Perl applications robust and reliable:
Carton
Pinto
EDIT:
Tools like perlbrew, pinto, carton, and cpanm are a vast improvement over a motley personal collection of scripts for doing similar things. Thanks to all the developers and contributors to these tools!
I'm not aware of any features in cpanm or perlbrew that allow you to reliably produce a list of installed files and their versions. Something like:
cpanm --list_installed
perlbrew list_installed_versions
or:
cpanm --export-cpanfile
perlbrew list_installed_as_cpanfile
might be a welcome feature.
As I noted in a comment to the OP above, you can garner useful information about your module installation from the install.json files that are installed by cpanm. Modules like CPAN::Meta, Module::Metadata and Distribution::Metadata can be helpful too.
The suggestion to use find and jq (which was from #Ilmari Karonen see Upgrade all modules installed by local::lib in this answer) led to the quick unfinished hack below. One challenge/issue is there's sometimes install.json files left lying around in multiple locations:
lib/perl5/$Config{archname}/.meta/Whatever-Mod-1.0050000/install.json
lib/perl5/$Config{archname}/.meta/Whatever-Mod-1.0090000/install.json
... etc.
This is likely because these files are not always cleanly removed when upgrading, reinstalling or other mucking about PEBKAC errors. To work properly, the code below should be changed so that it notices when there are multiple name-version combinations of a module's install.json and then does a more thorough check that the module is installed and gets its version. The script should have options: $dir could come from #ARGV. TIMTOWTDI, "well volunteered", etc.
#!perl
# list_installed_mods.pl
# DOES NOT THOROUGHLY VERIFY CURRENT VERSION
use File::Find;
use JSON;
use v5.16;
my $dir = "$ENV{HOME}/perl5/lib/perl5";
for my $installed ( find_installed($dir) ) {
say parse_install_json( $installed );
}
sub find_installed {
my $libdir = shift;
my #files;
File::Find::find ({ wanted =>
sub { push #files, $File::Find::name if /install\.json/i} },
$libdir );
return #files;
}
sub parse_install_json {
my $filename = shift;
my $json_text = do {
open(my $json_fh, "<:encoding(UTF-8)", $filename)
or die("Can't open \$filename\": $!\n");
local $/;
<$json_fh>
};
my $install = decode_json($json_text) ;
return ( $install->{name} ,"\#", $install->{version} ) ;
}
Possibly not the best way, but here's what I did recently.
I committed my Perlbrewed Perl version to a git repo, so I could use git archive to create a tar for me. Likewise with my Local::Lib modules. Then I wrote a little bit of scripting so that I could tag master, build archives from a tag, copy the archive to the remote server, unpack and chmod/chown the files.
I did this because at the time it was a quick and dirty solution to not having the time to set up Pinto or Carton.
It's only part of the solution but want mentioned so far: Perl's configure script has a -D relocateableinc parameter since some versions.
When building/brewing a Perl with that option the lib paths won't be hardcoded absolute path but relative to the perl binary which allows you to move the entire directory around or just rename it.
I'm building all my Perl's with that option since years and it hasn't created any problems so far.
The clone-modules command might be what you're seeking.
Example output for the below command follows:
perlbrew clone-modules perl-5.36.0 perl-5.37.6
Installing 398 modules from perl-5.36.0 to perl-5.37.6 ...
! Couldn't find module or a distribution AWS-Signature4
Successfully installed Algorithm-Diff-1.201
Successfully installed Algorithm-Loops-1.032
Successfully installed Module-Build-0.4231
Successfully installed ExtUtils-PkgConfig-1.16
Successfully installed Alien-VideoLAN-LibVLC-0.04
Successfully installed File-Slurp-9999.32
You can run this command after building a new perl with perlbrew to migrate (the latest versions of) CPAN modules from one version to another by reinstalling them for the new version that was just built.
I've been using local::lib to handle the installation of Perl modules on a server so I can get the right versions for some development work without polluting the system installation.
However, the system administrator has recently upgraded Perl from 5.16 to 5.18 and I'm now getting errors relating to binary modules, e.g.
perl -e 'use Scalar::Util'
Perl API version v5.16.0 of List::Util does not match v5.18.0 at /usr/lib64/perl5/5.18.2/XSLoader.pm line 92.
Compilation failed in require at /home/paul/perl5/lib/perl5/x86_64-linux/Scalar/Util.pm line 11.
Compilation failed in require at -e line 1.
BEGIN failed--compilation aborted at -e line 1.
My understanding is that I can fix the problem by forcing local::lib to rebuild all of its modules, but I can't find anything in the documentation which tells me how to do this, or even how to get a list of all the modules that have been installed using local::lib (if I had that in a 'one module per line' text file, I could easily write a Bash script to process it).
Is this possible, or do I have to remove the ~/perl5 directory and reinstall all the modules from scratch (possibly missing some as I can't remember them all)?
Update: For some time now, INSTALL_BASE has been producing a better directory structure that avoids this problem for new installs.
And that's why the perl Makefile.PL INSTALL_BASE=... convention (and the corresponding one for Build.PL) used by install::lib sucks.
Removing (or renaming the directory so you have backup) is the easiest solution. You can find out what you had installed by looking for .pm files.
cd ~
mv perl5{,16}
cd perl516/lib/perl5
find -name '*.pm' | xargs perl -MConfig -E'
for (#ARGV) {
s!^\./!!;
s!^5\.\d+\.\d+/!!;
s!^x86_64-linux/!!;
s!^auto/!!;
s!\.pm\z!!;
s!/!::!g;
say;
}
' | xargs cpan
(Do a dry run — one without the trailing | xargs cpan — first.)
Note that if you don't want to be at the mercy of your admin's upgrades, you can use perlbrew to easily install a whole build of Perl in your home dir.
If you are using cpanm you can force it to rebuild modules in your local::lib location by using the -L and --reinstall switches:
list_modules | cpanm -L ~/perl5 --reinstall
where list_modules is a script that feeds the names of modules to cpanm (there's also an -f option to cpanm). This script could be like #ikegami's above or something like cpan-outdated (which only lists outdated modules however). Here is a
hackish attempt that mostly worked for me recently (note: ikegami's is probably better) - when it fails or the script gives cpanm a module name it doesn't recognize, cpanm keeps going and doesn't seem to break anything (but make backups):
cd $PERL_LOCAL_LIB_ROOT
perl -MFile::Find -MConfig -E'
find {
wanted => sub {
$mod = $_ if /\.pm\z/;
$mod =~ s/lib\/perl5\/auto\/.*//g;
$mod =~ s/lib\/perl5\/\Q$Config{archname}\E\/.*//g;
$mod =~ s/lib\/perl5\///g;
push #mods, $mod unless $mod =~ /^$/;
},no_chdir=>1
},"lib/perl5"; #modhash{#mods}=(); say for sort keys %modhash '
By changing where and what File::Find finds with wanted() you can feed a different list of modules to cpanm. It would be nice if cpan or cpanm had an internal _method or -switch that allowed you to force the rebuilding of local::lib installed modules that use XS. Is there such a thing?
It was cpanm, carton (and seeing the node.js tool npm in action) that inspired me to do a lot more local::lib based installs. Now the CORE bundled "CPAN client" that ships with perl (cpan) seems to be getting more automagical and easier to use as well. I really like local::lib since it allows you to use the system perl but manage your own module stack without system level privileges. However, it can be easier overall to manage changes and upgrades if you use perlbrew to run a "non system" ("local") perl. Of course you can do this and still have local::lib, carton etc. manage a directory or application specific stack of modules.
1). Another way to do an upgrade is to use perllocal to generate a list of your currently installed modules (NB I'm not entirely sure if perllocal.pod is kept in a reliable/useful state). [Edit: In fact perllocal keeps a history of your installed modules rather than a list of those currently installed. You'll want to filter this or you will end up reinstalling the entire series of module versions one after the other!). This perlmonks node shows how to clean up your perllocal.pod: http://www.perlmonks.org/?node_id=483020. I prefer to keep the history.]
To parse perllocal.pod for input to cpanm, search through the file saving the matches in an array, then split them by twos to create a hash from the array elements (key,value,key,value). More recent installations and versions numbers are lower in the file, so you can create a hash with module names as keys and have the values updated by later entries:
perl -ne 'push #arr, grep {defined}
(/\A=head2.*:\s+C<Module>\s+L<(.*)\||.*C<VERSION:\s(.*)>\Z/msx); }{
%h = map{ split/,/,$_,2 } #arr; print "$_\#$h{$_}\n" for keys %h' perllocal.pod
(NB: this doesn't error check - entries in perllocal.pod occasionally lack VERSION data and other oddities so beware.)
2). But this method pales in comparison to the shell script mentioned by ilmari in #perl-help on IRC. It uses
jq - a commandline utility you really need to have (you'll want it more after this). If you have used cpanm to install your modules it will have created install.json files. You can leverage that to make list to feed to cpanm for reinstalling your current set of modules:
find ~/perl5/ -name install.json -exec jq '.name + "#" + .version' {} +
Very fast, very simple and you can combine it with the cpanm method of using curl to self-install in order to rebuild your modules.
3). If you use perlbrew to manage your perl installations you can easily copy and reinstall all your modules from one perl version to another.
HTH!
Yesterday I wanted to test some software and in the documentation it said, to install I just needed to type
cpan -i Software
I never used cpan, I just know that it is the perl package manager. (Is it..?) However, it turned out that I needed loads of dependencies, and stupid as I am, I just installed all of them. (First, I had to set up cpan which asked me lots of questions) Long story short, I just want to remove all of it again. I googled a bit, and it seems like cpan does not have an uninstall routine, especially for all the packages at once.
Can I just remove some directory or will I run into troubles?
the cpan command isn't really a package manager like apt-get is. It is more a tool that downloads and installs from CPAN (the site, or one of its mirrors). After it has finished this task it doesn't remember much about what was done before, at least not enough to remove previously installed modules, at least not reliably, cleanly or dependency-safely. (Update: After looking at App::pmuninstall, it can be used to handle dependencies, but it does so by connecting to outside (read: web) sources, which compute these separately, which is fine, but I stand by the previous statement that CPAN.pm doesn't do this.)
I used to worry about removing modules, but now I realize that most Perl modules take up so little room that I just don't worry about having a few extra modules installed that you will never use. So unless you are on a computer with a REALLY small disc, I would just let it be.
On Windows or if you are using a non-system Perl on Linux/Mac you could just remove Perl and reinstall it. I would not recommend this if you are using the system installed Perl on Linux/Mac however as you could break your OS doing this (you might be ok if you were careful, but not worth it to save a few Mb!).
In the future, you can easily install a local version of Perl using perlbrew, there are tutorials all over the web if the docs aren't sufficient (they should be). This also has the bonus of letting you play with the newest and greatest Perl versions, which your system likely doesn't come with yet. Then if you install a mountain of junk, or even break it doing something crazy, remove that version and reinstall/install a different version.
Another nice tool is cpanminus (or cpanm for short) which is a newer, more user friendly cpan tool. All the cool kids are using it.
You can uninstall individual modules with cpanplus (ships with Perl) like this:
cpanp uninstall SQL::Abstract
You can view all modules installed with the cpan script like this:
perldoc perllocal
Putting the two together:
for module in $(perldoc -u perllocal | grep -F 'C<Module> L<' | sed 's/^.*L<\(.*\)|.*>$/\1/') ; do
cpanp uninstall "$module"
done
I'm not sure about removing "all of it". But to remove a single module you can use App::pmuninstall with it's sole script pm-uninstall to uninstall modules. You might then be able to write some kind of script to recursively remove the deps.
If you can't use cpan any more because there are incompatible modules in you path, you can remove all installed modules by hand. For example, I upgraded from Fedora 22 to Fedora 23 and the Perl version changed. All modules installed previously via cpanm into /usr/local/lib64/perl5 did not work any more and prevented me from using cpanm.
$ cpanm --uninstall Apache::DBI
Attempt to reload Scalar/Util.pm aborted.
Compilation failed in require at /usr/share/perl5/vendor_perl/File/Temp.pm line 18.
...
I could solve this by moving that directory:
$ mv /usr/local/lib64/perl5 /root/usr-local-lib64-perl5
The name of that directory may vary on your system.
Carefull: If a module installed files outside of that directory, for example system library files, these files will remain there.
I will change Flimm's answer to use cpanm and optionally uninstall cpanm itself in the end of the script:
#!/usr/bin/env bash
for module in $(perldoc -u perllocal | grep -F 'C<Module> L<' | sed 's/^.*L<\(.*\)|.*>$/\1/' | sort | uniq) ; do
if [[ "$module" =~ "App::cpanminus" ]]; then
continue
fi
echo "Uninstalling $module..."
yes | cpanm --uninstall "$module"
done
cpanm --uninstall App::cpanminus
I think the best option is uninstall Perl and install it again.
I installed some Perl modules in my Linux machine. If I enter perldoc perllocal it shows a list of Perl modules installed in my machine, but now I don't need those Perl modules, so I want to remove them.
Does anyone know how to uninstall or remove those modules installed in Linux (debian 2.6.26)?
The Perl installers are just that... installers. They don't verify that they're not overwriting existing files, and they don't record precisely what they install. Therefore, they cannot uninstall. If you want a proper packaging system, you can't use the Perl installers.
If you use CPANPLUS to install a module, you can (at least in theory) also use it to uninstall it:
$ cpanp
...
> u Unwanted::Module
...
> q
$
The older CPAN module does not support an uninstall option. And, as Randal Schwartz notes, uninstalling modules is not always reliable.
Use cpanp (its uninstall is not limited to cpanplus-installed modules), or see ExtUtils::Packlist's modrm example.
Uninstall tools have historically been not readily provided because the install process is not robustly reversible, as Randal cautions.
As I've mentioned somewhere else on SO, my answer is to just leave them. There are VERY few Perl modules large enough to take up any actual space on you system. I'm not saying don't try if you really need the space, but if you don't ... its not worth it.
You can try App::pmuninstall
DESCRIPTION
App::pmuninstall is a fast module uninstaller. delete files from
.packlist.
App::cpanminus and, App::cpanoutdated with a high affinity.
I tried cpanp uninstall and it didn't work for me. I did find success using App::pmuninstall.
pm-uninstall [options] Module ...
pm-uninstall - Uninstall modules - metacpan.org