I am trying to install check_redis plugin and for that i need redis perl library but i don't have access to CPAN and internet.So Is there any way to install by downloading zip file and all the dependencies.I am using nagios core.Here is the plugin i am trying to install
Because I had a similar problem at work I happened to write my personal Q&A on this topic. Here it is:
Q: How to install CPAN modules without Internet access?
A: 1. Install CPAN::Mini on the internet machine:
$ cpan CPAN::Mini
2. Find a CPAN mirror here: http://mirrors.cpan.org/#de
e.g. http://ftp.hosteurope.de/pub/CPAN/
3. on the internet machine:
$ minicpan -l /path/to/local/cpan/mirror -r http://ftp.hosteurope.de/pub/CPAN/
4. Copy /path/to/local/cpan/mirror to the non-internet machine (or mount it)
5. On the non-internet machine add the directory to CPAN's search path:
$ cpan
cpan shell -- CPAN exploration and modules installation (v1.xxxx)
cpan> o conf urllist unshift file:///path/to/local/cpan/mirror
cpan> o conf commit
commit: wrote /home/user/.cpan/CPAN/MyConfig.pm
6. Install as usual on the non-internet machine.
To update your CPAN mirror go to your internet machine and repeat steps 3 and 4.
References:
http://www.perlmonks.org/?node_id=590386
http://www.cpan.org/misc/how-to-mirror.html
http://search.cpan.org/~rjbs/CPAN-Mini-1.111016/lib/CPAN/Mini.pm
http://search.cpan.org/~rjbs/CPAN-Mini-1.111016/bin/minicpan
Some notes
Although CPAN says the initial mirror will only occupy around 1 GB it will need around 3.5 GB. I think their documentation is outdated in this regard.
The mirror will initially contain the latest versions of all modules on CPAN. Every time you update your mirror the then-latest versions will be downloaded (leaving the old versions on your disk), so the mirror will grow with each update as new versions arrive.
I updated my mirror three or four times and it now has 3.9 GB.
You can install thing manually, of course, but it will be painful.
You can download the Redis distribution tarball from CPAN. But it might well require other modules that you don't have installed. You can see its list of requirements in its META.yml file. So you can download the distribution tarballs for each of those modules too. But, of course, each of those modules is likely to require other modules that you don't have. So you can look at all of their META.yml files. And so it goes on...
Honestly, it might be easier to work out how hard it would be to get the machine connected to the internet temporarily :-)
You could fatpack all the dependencies you cannot install into the check_nagios.pl using App::FatPacker. I'll outline how to do that.
First, let's make sure that the Nagios plugin has a licence that allows us to do this.
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation; either version 2 of the License, or
(at your option) any later version.
Ok, that's great. We're allowed to alter the source. Now let's clone it so we have the whole thing.
$ git clone git#github.com:willixix/WL-NagiosPlugins.git
$ cd WL-NagiosPlugins
Now we need to install App::FatPacker to our local Perl. We need to do this on a machine were installing stuff from the internet is not a problem. You can do it on the machine you used to write your question.
$ cpanm App::FatPacker
Afterwards, we need to also install all the dependencies. We know this needs Redis, so
$ cpanm Redis
Now typically all you need to do is run the fatpack pack command, but that would pack everything. And we don't really need that. We do have core modules on the target system after all. So we will do this step by step.
First, we need to tell fatpack to find all the dependencies.
$ fatpack trace check_redis.pl
This will create a fatpacker.trace file.
$ cat fatpacker.trace
IO/Handle.pm
Data/Dumper.pm
IO.pm
bytes.pm
Socket.pm
overload.pm
vars.pm
Redis/Sentinel.pm
lib.pm
Symbol.pm
Redis.pm
IO/Select.pm
PerlIO/via/Timeout.pm
base.pm
Scalar/Util.pm
fields.pm
IO/Socket/UNIX.pm
DynaLoader.pm
constant.pm
overloading.pm
Errno.pm
Getopt/Long.pm
warnings/register.pm
Carp.pm
IO/Socket.pm
SelectSaver.pm
Exporter/Heavy.pm
IO/Socket/Timeout.pm
List/Util.pm
Sub/Util.pm
Fcntl.pm
Time/HiRes.pm
IO/Socket/INET.pm
Try/Tiny.pm
Config.pm
Text/ParseWords.pm
I would suggest removing everything that looks like a core module. You turn the file names back into module names by replacing / with :: and removing the .pm, and then pass that into the corelist utility that ships with Perl on the target machine. All the pragmata (the ones that start with small letters) can be ignored, they are part of the core.
Here is the list after I cut it down to what I think makes sense to include. This might not be complete, or incorrect, but I can't try on your machine. The file now looks like this.
$ cat fatpacker.trace
Redis/Sentinel.pm
Redis.pm
Try/Tiny.pm
Now we follow the manual in RECIPES in the fatpack documentation.
$ fatpack packlists-for `cat fatpacker.trace` >packlists
The packlists file contains a couple of paths for where to find the packlists for those modules. This is specific to your Perl installation.
$ fatpack tree `cat packlists`
This will create a directory fatlib that contains all the stuff we want to bundle.
$ tree fatlib/
fatlib/
├── Redis
│ ├── Hash.pm
│ ├── List.pm
│ └── Sentinel.pm
├── Redis.pm
└── Try
└── Tiny.pm
2 directories, 5 files
Last, we need to run the actual packing command.
$ fatpack file check_redis.pl >check_redis.packed.pl
And now we have a file called check_redis.packed.pl that. If you open that and look at it, you'll see a BEGIN block that loads a bunch of stuff. That stuff is the dependencies that you could not install. They are now bundled into the script.
Move that script to the machine with your Nagios, and use it instead of the check_nagios.pl. It should now just work.
Please note that I have not actually tried if it works because I neither have a Redis nor a Nagios to try it. But it should work.
This worked for me in centOs 7.
step 1: Download dependencies in a centOs7 that is connected to internet by these commands:
yumdownloader --destdir=/etc/LinuxRepos/Redis epel-release
yumdownloader --destdir=/etc/LinuxRepos/Redis jemalloc
yumdownloader --destdir=/etc/LinuxRepos/Redis epel-release
step 2: copy downloaded files from /etc/LinuxRepos/Redis and copy them to destination centOS 7.
step 3: Install packages by
yum localinstal epel-release-7-11.noarch.rpm
yum localinstall jemalloc-3.6.0-1.el7.x86_64.rpm
yum localinstall epel-release-7-11.noarch.rpm
NOTICE: The name of files in step 3 may be changed due to current last
version.
Related
I trying to get set up with Variant Effect Predictor (VEP) on the command line. I'm following the setup tutorial but I'm encountering some errors around dependencies. I'm also quite new to the command line so if anyone is able to break down the solution too then I'd be very grateful. Thanks!
Tutorial: https://www.ensembl.org/info/docs/tools/vep/script/vep_tutorial.html
VEP requirements: http://www.ensembl.org/info/docs/tools/vep/script/vep_download.html#requirements
I also have Perl v5.32.1.
What I've done:
I installed dependencies (listed in the requirements page) with the following commands:
- sudo -s cpanm DBI
- sudo -s cpanm Archive::Zip
- sudo -s cpanm DBD::mysql
For DBD:mysql, I got the follwoing message:
--> Working on DBD::mysql
Fetching http://www.cpan.org/authors/id/D/DV/DVEEDEN/DBD-mysql-4.050.tar.gz ... OK
Configuring DBD-mysql-4.050 ... N/A
! Configure failed for DBD-mysql-4.050. See /root/.cpanm/work/1626111140.5937/build.log for details.
Trying it out anyway, I ran perl INSTALL.pl (from the tutorial page) and got the message below. I would like VEP to run in online mode too if possible.
`WARNING: DBD::mysql module not found. VEP can only run in offline (--offline) mode without DBD::mysql installed
http://www.ensembl.org/info/docs/tools/vep/script/vep_download.html#requirements
Hello! This installer is configured to install v104 of the Ensembl API for use by the VEP.
It will not affect any existing installations of the Ensembl API that you may have.
It will also download and install cache files from Ensembl's FTP server.
Checking for installed versions of the Ensembl API...done
Setting up directories
Destination directory ./Bio already exists.
Do you want to overwrite it (if updating VEP this is probably OK) (y/n)? y
- fetching BioPerl
- unpacking ./Bio/tmp/release-1-6-924.zip
ERROR: Unable to unpack file ./Bio/tmp/release-1-6-924.zip without Archive::Extract or tar/unzip/gzip`
You show us this error:
Configure failed for DBD-mysql-4.050. See /root/.cpanm/work/1626111140.5937/build.log for details.
So looking in there will give you more clues about what the problems are. Without that, we can only guess.
But we can make educated guesses. The DBD::mysql distribution comes with a file called DBD::mysql::INSTALL which will talk you through some of the problems you'll find while installing this module.
It's important to note that DBD::mysql is a wrapper around MySQL's client libraries. They are written in C, so you'll need a C compiler installed in order to build DBD::mysql. You'll also need the client libraries and the development versions of the client libraries (for the C header files that you'll need to compile the module). On Ubuntu, those packages are called "libmysqlclient" and "libmysqlclient-dev". If you don't have a C compiler, then you'll want to install "gcc" too.
But this is all getting a bit complicated. There's another, simpler, approach. If you're using the system version of Perl (the version that was installed as part of the operating system and probably lives in /usr/bin/perl) then I'd recommend using the pre-build Ubuntu version of the package, which you can install by running:
$ sudo apt-get install libdbd-mysql-perl
Installing that version uses the OS's own package manager, and the package manager knows which other packages are needed in order for it to work - so it will install those as well.
People will probably complain that you're better off installing the modules from CPAN as it gives you more flexibility and allows you to use more up-to-date packages than the versions from your OS repos. And they're right. But, honestly, if you're a non-Perl programmer who just wants to get an application up and running, this is by far the simplest approach.
(But, as I said above, this is all guesswork as you haven't shared the most important errors with us.)
I've already read related threads like these, but they do not fully capture our situation.
This is on a firewalled machine. No net access. We can ftp files to folders and install modules from there.
We have CHMOD 777 for our users on some folders. We can install Perl modules if we locally build them by downloading the relevant .pm files. But when these files cannot install, we do not have any cpan or cpanm.
I'd like to install, for example, HTML::Restrict. If I do the download + install thing, the Restrict.pm gives me this error:
/lib/HTML/Restrict.PM:328: Unknown command paragraph "=encoding UTF-8"
Reading a bit online suggests that this could be an old Perl problem. We use 5.8.x. Our own dev machines have the luxury of 5.16.x and internet access so installing module is a cinch. Anyway, one of my older machines also has 5.8.x, and installing the module via cpanminus worked there (with internet).
So, question: is it possible to install "cpanminus" (cpanm) through FTP, then upload specific module files to the server through FTP too, and then go into shell and install modules via cpanm by pointing it to respective .pm files?
Thank you for any pointers.
You should take a look at perldoc perlmodinstall which goes into detail about how to install a module from its distribution. It follows what should be a familiar incantation
Decompress
Unpack
Build
Test
Install
Assuming you're on a Linux system, this commonly takes take the form of
gzip -d My-Module-Distribution.tar.gz
tar -xof My-Module-Distribution.tar
perl Makefile.PL
make
make test
make install
But after the Unpack stage you will often find a README file or other text file that will describe any unusual steps to be taken
Clearly some of these steps can be combined. For instance, most people will probably want to use
tar -xvfz My-Module-Distribution.tar.gz
to avoid having to invoke gzip separately. Likewise, the make system will force a build phase as a prerequisite if you use just
make test
without the preceding make
The linked document has a lot to say about how to install on other platforms, should you not be running a Linux variant
I still don't really understand your thinking, but you can get a stand-alone version of cpanm using curl. For instance
curl -sS --location https://cpanmin.us/ --output cpanm
then you should be able to just copy it to your target machine, put it on your PATH, and do
cpanm HTML-Restrict-2.2.2.tar.gz
but I doubt if you will find any change to the specific errors you are getting
I'm attempting to create a local yum repo on my system containing various packages from, chiefly, the CentOS base repos. The server which is hosting the yum repo will not necessarily have the same base packages installed by default as the servers which will be using the yum repo. For this reason, I need to ensure that my repos contain the packages that I want and every single one of their dependencies.
I'm creating my repos using the yumdownloader tool provided in the yum-utils package to try to download an RPM file for a package using yum from the standard CentOS mirrors. Helpfully it provides a command line option, --resolve, which also downloads dependencies. However, because it's built on yum itself, yumdownloader will only download dependencies for the package that are not already present on the system.
For example, I wish to download package A, which depends on Packages B, C and D. If package D is already installed on the system, yumdownloader --resolve A will only download A, B and C, but not D.
Is there a way to download the RPMs for all dependencies on a package from a yum repo?
There's this bash script, which the maintainer of rpm has kindly shared with me, and I put on github. Hope you find it useful!
You can also read the original SO question, where the issue was discussed.
The script works on Fedora 23+ as it uses dnf's download plugin. It's probably very easy to make it work on Fedora 22-, as yum surely has got a similar plugin.
Additionaly, it's valuable since repotrack does not work on fedora 23 (at least it doesn't work for me).
After a lot of frustration looking around for a solution I have written a simple script that uses repotrace and wget. I've found that yumdownloader (even with the resolve flag) does not resolve all dependencies.
if you have a long list of packages you are bound to run into duplicates, downloading just the urls first with the "repotrack -u flag" and then getting unique records resolves having to download the same rpm multiple times.
#!/bin/bash
while read i; do
repotrack -u $i >> dep_rpm_urls_02.txt
done < list_of_packages_01.txt
awk '!seen[$0]++' dep_rpm_urls_02.txt > dep_rpm_urls_clean_03.txt
while read j; do
wget $j
echo dowloaded $j
done < dep_rpm_urls_clean_03.txt
happy rpming
I have been writing Perl scripts for my work and the machine that I have been given to work on makes installing Perl modules difficult:
We cannot have gcc on my machine for security reasons, so I cannot use CPAN to install modules, for most modules.
I do not have access to the root account.
Usually, when I want to install a module, I put in a request and I have to wait a day or two before it gets installed. I know that nobody would have a problem with me installing them myself, so to save everyone's time and my sanity I would like to install them myself. It's just an issue of how to best do that. I have talked to various people and they said to use an RPM to install them (to get around not having gcc). However, when trying to install modules from RPMs, it does not handle the dependencies so I would manually need to handle the dependencies, which could take a while.
How can I best install Perl modules with these limitations?
On a similar machine with a similarly built Perl, install the module(s) using
mkdir ~/foo
cpan
o conf makepl_arg 'PREFIX=~/foo LIB=~/foo/lib/perl5'
o conf mbuildpl_arg '--prefix ~/foo --lib ~/foo/lib/perl5'
install Some::Module
As long as you don't do o conf commit, the configuration change will be temporary, so don't do that.
Copy ~/foo over, and set env var PERL5LIB to include the LIB directory. You can merge a newer ~/foo into an older one to add new modules.
This won't install any non-Perl libraries on which the modules depend.
See also How do I keep my own module/library directory? in section 8 of the Perl FAQ.
When you build modules, tell Perl where to install the modules.
For Makefile.PL-based distributions, use the INSTALL_BASE option when generating Makefiles:
perl Makefile.PL INSTALL_BASE=/mydir/perl
For Build.PL-based distributions, use the --install_base option:
perl Build.PL --install_base /mydir/perl
INSTALL_BASE tells these tools to put your modules into /mydir/perl/lib/perl5. See How do I add a directory to my include path (#INC) at runtime? for details on how to run your newly installed modules.
There is one caveat with INSTALL_BASE, though, since it acts differently from the PREFIX and LIB settings that older versions of ExtUtils::MakeMaker advocated. INSTALL_BASE does not support installing modules for multiple versions of Perl or different architectures under the same directory. You should consider whether you really want that and, if you do, use the older PREFIX and LIB settings. See the ExtUtils::Makemaker documentation for more details.
I have a Linux server that has no access to the internet (access is prevented by a firewall). I would like to install a new Perl. What are my options and what is the best way to do this? The system Perl (included in OS installation) must remain unchanged.
I have been using perlbrew and I think it is the best way to do an online installation. But all the steps involved in perlbrew seem to require internet access: you download it from the net, it downloads new Perl versions from the net etc. and I haven't found a glue how to make it work offline.
If perlbrew is out of question I could build Perl from source into a custom location on the server. I assume that this could end up being complicated, time-consuming and error-prone. And every time I update Perl I have make a new build manually.
There can also be other ways to install that I'm not currently aware of. And of course I could stick with the system Perl but it is an outdated version and I'm already using the new syntax features. Or I could start negotiations to change the firewall policy to allow internet access for perlbrew.
But all the steps involved in perlbrew seem to require internet access
Not if properly configured.
To install perlbrew itself off-line, install the App-perlbrew dist. Following its dependencies manually is a chore, so instead prepare a MiniCPAN mirror (with -p to include Perl dists), take it over to the target machine and configure CPAN to use the local mirror. Run cpan App::perlbrew to install.
After perlbrew is installed, run its mirror command to configure a CPAN mirror into $PERLBREWROOT/Config.pm. Edit this file to change it to the local MiniCPAN mirror. Drop Perl dist tarballs into $PERLBREWROOT/dists/.
Be aware that compiling Perl requires a working C compiler toolchain, and optionally the development files for libdb (BerkeleyDB) and gdbm. (Read the INSTALL file once over, even though perlbrew's autoconfiguration and Perl's configure.SH defaults hide these details from you.)
The compiler toolchain is probably much more difficult to procure off-line, unless the OS installation has already been used before for compiling other C stuff.
There's nothing that special about perlbrew. If you aren't going to use it to download the Perl sources, it's not saving you that much. Once you have the Perl sources, you just need to configure and install it:
% ./Configure -des -Dprefix=/path/to/installation
% make install
Once done, everything for that Perl is under that installation path.
I dislike perlbrew mostly because it hides from people how amazingly simple this task is so they feel like they can't do it on their own.
Have you considered attacking it from a different direction? Keeping this up-to-date is going to be a pain if you have to request internet access each time. Likewise, if you've missed out/misconfigured any packages in your CPAN mirror it's difficult to correct once you're actually trying to use them.
Perhaps just build a small VM with a cut-down linux + perl + modules. Keep that up-to-date at your end and just take the whole lot in on a USB stick. You'd have a known-working easy-to-setup installation.
What I personally do is using git checkout when I'm offline (and not on vacation). Once you have the whole git work directory, it's trivial to build any released version by checking out the tags:
git checkout v5.17.4
git clean -f # cleanup previously compiled .o files etc
sh ./Configure ...
Depending on how you can transfer files to your host, this can be handy, since you you can also setup a private git repo there so other computer can git push new commits to there.