So I have my Ruby-on-rails app that i successfully released to heroku. However to populate the database I have several perl scripts that basically sync up tables on local database with a remote database.
On my development machine (on ec2) it works fine but I also have a lot of stuff i have installed on it over time.
When I try to run my scripts I get the following error (latest perl is installed by the way)
Can't locate DBI.pm in #INC (#INC contains: /etc/perl /usr/local/lib/perl/5.10.1 /usr/local/share/perl/5.10.1 /usr/lib/perl5 /usr/share/perl5 /usr/lib/perl/5.10 /usr/share/perl/5.10 /usr/local/lib/site_perl .) at copyStaff.pl line 4.
BEGIN failed--compilation aborted at copyStaff.pl line 4.
How do I install DBI module for Perl on heroku?
I tried through bash
apt-get install libdbi-perl
This gave me an output of
W: Not using locking for read only lock file /var/lib/dpkg/lock
E: Unable to write to /var/cache/apt/
E: The package lists or status file could not be parsed or opened.
[This might be irrelevant]
I tried to run this command
dpkg --configure -a
But it told me it requires super user privileges
su -l root -c "dpkg --configure -a"
asked for root password and apparently I don't know the password (it's not the heroku login), no clue what it might be.
I also downloaded the file directly, but I need super user access. I also tried
$perl -MCPAN -e shell
cpan> install DBI
(this had same problem of not beng able to save the file)
I have a feeling i might be going at it completely the wrong way, but I can't figure out what I need to do to run those scripts.
You're going to have to investigate Perl Buildpacks for Heroku, of which I know of the existence of Perloku, and two others calling themselves something along the lines of perl-heroku.
Buildpacks attempt to install environments that Heroku does not support natively, like Perl. I found some install Mojolicious, another installs an apache2/mod_perl env.
You'll have to talk to the buildpack maintainers to find out if they have DBI/ DBD::mysql installed, but it is likely they do, just based on the popularity of those modules.
If you get that far, then all you have to do is sync the databases :-)
Well I got the answer from customer support, just pasting it here
Unfortunately we don't support installing additional libraries or binaries to our stacks. The best workaround is to vendor these into your project. You'll need to use 64-bit Linux versions to make them work on Heroku; compiling statically can also help ensure that any dependencies needed are included. Similarly, for gems that depend on external libraries, we recommend compiling the gem statically and vendoring it into your project.
If you do wish to try to vendor your binary, library, or gem, you can use Heroku as your build environment. One of our engineers created a build server that allows you to upload source code, run the compilation step, and then download the resulting binary. You can find this project here:
https://github.com/heroku/vulcan
Related
I trying to get set up with Variant Effect Predictor (VEP) on the command line. I'm following the setup tutorial but I'm encountering some errors around dependencies. I'm also quite new to the command line so if anyone is able to break down the solution too then I'd be very grateful. Thanks!
Tutorial: https://www.ensembl.org/info/docs/tools/vep/script/vep_tutorial.html
VEP requirements: http://www.ensembl.org/info/docs/tools/vep/script/vep_download.html#requirements
I also have Perl v5.32.1.
What I've done:
I installed dependencies (listed in the requirements page) with the following commands:
- sudo -s cpanm DBI
- sudo -s cpanm Archive::Zip
- sudo -s cpanm DBD::mysql
For DBD:mysql, I got the follwoing message:
--> Working on DBD::mysql
Fetching http://www.cpan.org/authors/id/D/DV/DVEEDEN/DBD-mysql-4.050.tar.gz ... OK
Configuring DBD-mysql-4.050 ... N/A
! Configure failed for DBD-mysql-4.050. See /root/.cpanm/work/1626111140.5937/build.log for details.
Trying it out anyway, I ran perl INSTALL.pl (from the tutorial page) and got the message below. I would like VEP to run in online mode too if possible.
`WARNING: DBD::mysql module not found. VEP can only run in offline (--offline) mode without DBD::mysql installed
http://www.ensembl.org/info/docs/tools/vep/script/vep_download.html#requirements
Hello! This installer is configured to install v104 of the Ensembl API for use by the VEP.
It will not affect any existing installations of the Ensembl API that you may have.
It will also download and install cache files from Ensembl's FTP server.
Checking for installed versions of the Ensembl API...done
Setting up directories
Destination directory ./Bio already exists.
Do you want to overwrite it (if updating VEP this is probably OK) (y/n)? y
- fetching BioPerl
- unpacking ./Bio/tmp/release-1-6-924.zip
ERROR: Unable to unpack file ./Bio/tmp/release-1-6-924.zip without Archive::Extract or tar/unzip/gzip`
You show us this error:
Configure failed for DBD-mysql-4.050. See /root/.cpanm/work/1626111140.5937/build.log for details.
So looking in there will give you more clues about what the problems are. Without that, we can only guess.
But we can make educated guesses. The DBD::mysql distribution comes with a file called DBD::mysql::INSTALL which will talk you through some of the problems you'll find while installing this module.
It's important to note that DBD::mysql is a wrapper around MySQL's client libraries. They are written in C, so you'll need a C compiler installed in order to build DBD::mysql. You'll also need the client libraries and the development versions of the client libraries (for the C header files that you'll need to compile the module). On Ubuntu, those packages are called "libmysqlclient" and "libmysqlclient-dev". If you don't have a C compiler, then you'll want to install "gcc" too.
But this is all getting a bit complicated. There's another, simpler, approach. If you're using the system version of Perl (the version that was installed as part of the operating system and probably lives in /usr/bin/perl) then I'd recommend using the pre-build Ubuntu version of the package, which you can install by running:
$ sudo apt-get install libdbd-mysql-perl
Installing that version uses the OS's own package manager, and the package manager knows which other packages are needed in order for it to work - so it will install those as well.
People will probably complain that you're better off installing the modules from CPAN as it gives you more flexibility and allows you to use more up-to-date packages than the versions from your OS repos. And they're right. But, honestly, if you're a non-Perl programmer who just wants to get an application up and running, this is by far the simplest approach.
(But, as I said above, this is all guesswork as you haven't shared the most important errors with us.)
I am trying to install check_redis plugin and for that i need redis perl library but i don't have access to CPAN and internet.So Is there any way to install by downloading zip file and all the dependencies.I am using nagios core.Here is the plugin i am trying to install
Because I had a similar problem at work I happened to write my personal Q&A on this topic. Here it is:
Q: How to install CPAN modules without Internet access?
A: 1. Install CPAN::Mini on the internet machine:
$ cpan CPAN::Mini
2. Find a CPAN mirror here: http://mirrors.cpan.org/#de
e.g. http://ftp.hosteurope.de/pub/CPAN/
3. on the internet machine:
$ minicpan -l /path/to/local/cpan/mirror -r http://ftp.hosteurope.de/pub/CPAN/
4. Copy /path/to/local/cpan/mirror to the non-internet machine (or mount it)
5. On the non-internet machine add the directory to CPAN's search path:
$ cpan
cpan shell -- CPAN exploration and modules installation (v1.xxxx)
cpan> o conf urllist unshift file:///path/to/local/cpan/mirror
cpan> o conf commit
commit: wrote /home/user/.cpan/CPAN/MyConfig.pm
6. Install as usual on the non-internet machine.
To update your CPAN mirror go to your internet machine and repeat steps 3 and 4.
References:
http://www.perlmonks.org/?node_id=590386
http://www.cpan.org/misc/how-to-mirror.html
http://search.cpan.org/~rjbs/CPAN-Mini-1.111016/lib/CPAN/Mini.pm
http://search.cpan.org/~rjbs/CPAN-Mini-1.111016/bin/minicpan
Some notes
Although CPAN says the initial mirror will only occupy around 1 GB it will need around 3.5 GB. I think their documentation is outdated in this regard.
The mirror will initially contain the latest versions of all modules on CPAN. Every time you update your mirror the then-latest versions will be downloaded (leaving the old versions on your disk), so the mirror will grow with each update as new versions arrive.
I updated my mirror three or four times and it now has 3.9 GB.
You can install thing manually, of course, but it will be painful.
You can download the Redis distribution tarball from CPAN. But it might well require other modules that you don't have installed. You can see its list of requirements in its META.yml file. So you can download the distribution tarballs for each of those modules too. But, of course, each of those modules is likely to require other modules that you don't have. So you can look at all of their META.yml files. And so it goes on...
Honestly, it might be easier to work out how hard it would be to get the machine connected to the internet temporarily :-)
You could fatpack all the dependencies you cannot install into the check_nagios.pl using App::FatPacker. I'll outline how to do that.
First, let's make sure that the Nagios plugin has a licence that allows us to do this.
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation; either version 2 of the License, or
(at your option) any later version.
Ok, that's great. We're allowed to alter the source. Now let's clone it so we have the whole thing.
$ git clone git#github.com:willixix/WL-NagiosPlugins.git
$ cd WL-NagiosPlugins
Now we need to install App::FatPacker to our local Perl. We need to do this on a machine were installing stuff from the internet is not a problem. You can do it on the machine you used to write your question.
$ cpanm App::FatPacker
Afterwards, we need to also install all the dependencies. We know this needs Redis, so
$ cpanm Redis
Now typically all you need to do is run the fatpack pack command, but that would pack everything. And we don't really need that. We do have core modules on the target system after all. So we will do this step by step.
First, we need to tell fatpack to find all the dependencies.
$ fatpack trace check_redis.pl
This will create a fatpacker.trace file.
$ cat fatpacker.trace
IO/Handle.pm
Data/Dumper.pm
IO.pm
bytes.pm
Socket.pm
overload.pm
vars.pm
Redis/Sentinel.pm
lib.pm
Symbol.pm
Redis.pm
IO/Select.pm
PerlIO/via/Timeout.pm
base.pm
Scalar/Util.pm
fields.pm
IO/Socket/UNIX.pm
DynaLoader.pm
constant.pm
overloading.pm
Errno.pm
Getopt/Long.pm
warnings/register.pm
Carp.pm
IO/Socket.pm
SelectSaver.pm
Exporter/Heavy.pm
IO/Socket/Timeout.pm
List/Util.pm
Sub/Util.pm
Fcntl.pm
Time/HiRes.pm
IO/Socket/INET.pm
Try/Tiny.pm
Config.pm
Text/ParseWords.pm
I would suggest removing everything that looks like a core module. You turn the file names back into module names by replacing / with :: and removing the .pm, and then pass that into the corelist utility that ships with Perl on the target machine. All the pragmata (the ones that start with small letters) can be ignored, they are part of the core.
Here is the list after I cut it down to what I think makes sense to include. This might not be complete, or incorrect, but I can't try on your machine. The file now looks like this.
$ cat fatpacker.trace
Redis/Sentinel.pm
Redis.pm
Try/Tiny.pm
Now we follow the manual in RECIPES in the fatpack documentation.
$ fatpack packlists-for `cat fatpacker.trace` >packlists
The packlists file contains a couple of paths for where to find the packlists for those modules. This is specific to your Perl installation.
$ fatpack tree `cat packlists`
This will create a directory fatlib that contains all the stuff we want to bundle.
$ tree fatlib/
fatlib/
├── Redis
│ ├── Hash.pm
│ ├── List.pm
│ └── Sentinel.pm
├── Redis.pm
└── Try
└── Tiny.pm
2 directories, 5 files
Last, we need to run the actual packing command.
$ fatpack file check_redis.pl >check_redis.packed.pl
And now we have a file called check_redis.packed.pl that. If you open that and look at it, you'll see a BEGIN block that loads a bunch of stuff. That stuff is the dependencies that you could not install. They are now bundled into the script.
Move that script to the machine with your Nagios, and use it instead of the check_nagios.pl. It should now just work.
Please note that I have not actually tried if it works because I neither have a Redis nor a Nagios to try it. But it should work.
This worked for me in centOs 7.
step 1: Download dependencies in a centOs7 that is connected to internet by these commands:
yumdownloader --destdir=/etc/LinuxRepos/Redis epel-release
yumdownloader --destdir=/etc/LinuxRepos/Redis jemalloc
yumdownloader --destdir=/etc/LinuxRepos/Redis epel-release
step 2: copy downloaded files from /etc/LinuxRepos/Redis and copy them to destination centOS 7.
step 3: Install packages by
yum localinstal epel-release-7-11.noarch.rpm
yum localinstall jemalloc-3.6.0-1.el7.x86_64.rpm
yum localinstall epel-release-7-11.noarch.rpm
NOTICE: The name of files in step 3 may be changed due to current last
version.
I'm trying to remove postgres from my Ubuntu machine, but when I do, I'm getting the error message
Can't locate PgCommon.pm (#INC is [... a list of paths ...])
Any idea why PostgreSQL would lose track of a file it installed itself?
This can happen if you changed Perl interpreters since installing postgres. It can't find the package anymore because it was installed into a different modules directory. If this command
$ which perl
doesn't give this response
/usr/bin/perl
It's likely that you're just using the wrong Perl binary. In my case, I had symlinked /usr/local/bin/perl to my custom Perl binary (which my app needs in order to function) so I just renamed the symlink temporarily, ran apt-get remove postgres again, then put the symlink back where it was!
I've already read related threads like these, but they do not fully capture our situation.
This is on a firewalled machine. No net access. We can ftp files to folders and install modules from there.
We have CHMOD 777 for our users on some folders. We can install Perl modules if we locally build them by downloading the relevant .pm files. But when these files cannot install, we do not have any cpan or cpanm.
I'd like to install, for example, HTML::Restrict. If I do the download + install thing, the Restrict.pm gives me this error:
/lib/HTML/Restrict.PM:328: Unknown command paragraph "=encoding UTF-8"
Reading a bit online suggests that this could be an old Perl problem. We use 5.8.x. Our own dev machines have the luxury of 5.16.x and internet access so installing module is a cinch. Anyway, one of my older machines also has 5.8.x, and installing the module via cpanminus worked there (with internet).
So, question: is it possible to install "cpanminus" (cpanm) through FTP, then upload specific module files to the server through FTP too, and then go into shell and install modules via cpanm by pointing it to respective .pm files?
Thank you for any pointers.
You should take a look at perldoc perlmodinstall which goes into detail about how to install a module from its distribution. It follows what should be a familiar incantation
Decompress
Unpack
Build
Test
Install
Assuming you're on a Linux system, this commonly takes take the form of
gzip -d My-Module-Distribution.tar.gz
tar -xof My-Module-Distribution.tar
perl Makefile.PL
make
make test
make install
But after the Unpack stage you will often find a README file or other text file that will describe any unusual steps to be taken
Clearly some of these steps can be combined. For instance, most people will probably want to use
tar -xvfz My-Module-Distribution.tar.gz
to avoid having to invoke gzip separately. Likewise, the make system will force a build phase as a prerequisite if you use just
make test
without the preceding make
The linked document has a lot to say about how to install on other platforms, should you not be running a Linux variant
I still don't really understand your thinking, but you can get a stand-alone version of cpanm using curl. For instance
curl -sS --location https://cpanmin.us/ --output cpanm
then you should be able to just copy it to your target machine, put it on your PATH, and do
cpanm HTML-Restrict-2.2.2.tar.gz
but I doubt if you will find any change to the specific errors you are getting
I'm trying to setup a Perl development environment on my Mac laptop and have been having a really hard time getting it working. I thought I had everything configured correctly but when I try to run a sample script it is reporting errors with the DBI module and can't access the DB.
Here is what is reported in the Apache error logs:
[Fri Apr 30 23:11:33 2010] [error] [client 127.0.0.1] Can't locate DBI.pm in #INC (#INC contains: /Library/Perl/Updates/5.10.0/darwin-thread-multi-2level /Library/Perl/Updates/5.10.0 /System/Library/Perl/5.10.0/darwin-thread-multi-2level /System/Library/Perl/5.10.0 /Library/Perl/5.10.0/darwin-thread-multi-2level /Library/Perl/5.10.0 /Network/Library/Perl/5.10.0/darwin-thread-multi-2level /Network/Library/Perl/5.10.0 /Network/Library/Perl /System/Library/Perl/Extras/5.10.0/darwin-thread-multi-2level /System/Library/Perl/Extras/5.10.0 .) at main.pm line 5.
I downloaded and installed both modules manually to work with MAMP using the following commands as specified in this forum post:
For DBI
1. cd /Library/Perl/DBI-1.611
2. sudo Perl Makefile.PL
3. sudo make
4. sudo make install
For DBD
1. cd /Library/Perl/DBD-mysql-4.014
2. sudo Perl Makefile.PL --mysql_config=/Applications/MAMP/Library/bin/mysql_config
3. sudo make
4. sudo make install
What I noticed while running the above commands is that the files seems to be getting installed in the '/opt/local/lib/perl5/site_perl/5.8.9/darwin-2level/' directory which doesn't seem to be one of the search directories that Apache mentions in the error at the beginning of this post. Here is what I'm seeing during the install:
$ sudo make install
Files found in blib/arch: installing files in blib/lib into architecture dependent library tree
Installing /opt/local/lib/perl5/site_perl/5.8.9/darwin-2level/auto/DBI/DBI.bundle
Installing /opt/local/lib/perl5/site_perl/5.8.9/darwin-2level/auto/DBI/dbipport.h
Installing /opt/local/lib/perl5/site_perl/5.8.9/darwin-2level/auto/DBI/DBIXS.h
Installing /opt/local/lib/perl5/site_perl/5.8.9/darwin-2level/auto/DBI/dbixs_rev.h
Installing /opt/local/lib/perl5/site_perl/5.8.9/darwin-2level/auto/DBI/Driver.xst
Installing /opt/local/lib/perl5/site_perl/5.8.9/darwin-2level/auto/DBI/Driver_xst.h
Installing /opt/local/lib/perl5/site_perl/5.8.9/darwin-2level/DBI.pm
Installing /opt/local/lib/perl5/site_perl/5.8.9/darwin-2level/TASKS.pod
Installing /opt/local/lib/perl5/site_perl/5.8.9/darwin-2level/DBD/DBM.pm
Installing /opt/local/lib/perl5/site_perl/5.8.9/darwin-2level/DBD/File.pm
Installing /opt/local/lib/perl5/site_perl/5.8.9/darwin-2level/DBD/Gofer.pm
Installing /opt/local/lib/perl5/site_perl/5.8.9/darwin-2level/DBI/Changes.pm
Installing /opt/local/lib/perl5/site_perl/5.8.9/darwin-2level/DBI/DBD.pm
Installing /opt/local/lib/perl5/site_perl/5.8.9/darwin-2level/DBI/Profile.pm
Installing /opt/local/lib/perl5/site_perl/5.8.9/darwin-2level/DBI/ProxyServer.pm
Installing /opt/local/lib/perl5/site_perl/5.8.9/darwin-2level/DBI/PurePerl.pm
Installing /opt/local/share/man/man3/DBD::DBM.3pm
Installing /opt/local/share/man/man3/DBD::File.3pm
Installing /opt/local/share/man/man3/DBD::Gofer.3pm
Installing /opt/local/share/man/man3/DBI.3pm
Installing /opt/local/share/man/man3/DBI::DBD.3pm
Installing /opt/local/share/man/man3/DBI::Profile.3pm
Installing /opt/local/share/man/man3/DBI::ProxyServer.3pm
Installing /opt/local/share/man/man3/DBI::PurePerl.3pm
Installing /opt/local/share/man/man3/TASKS.3pm
Installing /opt/local/bin/dbiprof
Installing /opt/local/bin/dbiproxy
Writing /opt/local/lib/perl5/site_perl/5.8.9/darwin-2level/auto/DBI/.packlist
Appending installation info to /opt/local/lib/perl5/5.8.9/darwin-2level/perllocal.pod
My question is, what am I doing wrong and how can I either 1) Get Apache to look in the right directory where the DBD & DBI modules are installed or 2) Update the way I'm installing the module to install them into one of the search directories. I honestly don't know what option makes more sense and could use guidance on that as well.
As you can probably tell I'm pretty lost at the moment. Please help!!! Thanks in advance.
It looks like you've already installed another Perl via macports (/opt/local is where all macports installations go), and /opt/local/bin is earlier in your $PATH than the system Perl in /usr/bin. That's fine, if you are happy running Perl 5.8.9 rather than Perl 5.10.0 (hint: if you aren't sure of the differences, then the differences don't matter).
It's usually advised to not make extra installations to the system Perl. Apple may upgrade components through regular system updates, which could interfere with any modifications you have made, and if you make a mistake with an installation, it's difficult to remedy it without doing a full system reinstallation or having some serious understanding of the operating system guts. So, since you've already got another Perl installation ready, I would strongly encourage you to stick with that one.
However, you probably shouldn't be manually installing libraries if there is already a distribution available on macports. I used port search dbi and port search dbd to find them: the distributions are named p5-dbi and p5-dbd-mysql. You can install those like any other macports module: with sudo port install <distroname>. (You may need to install mod_perl itself, too.)
After that, you simply need to tell Apache/mod_perl to use that Perl installation rather than the system perl. I've never done that, so I can't advise on the best way to do it. However, quick searches on http://superuser.com suggest that the macports version of apache will run by default (via the same $PATH ordering), so I'd just Try It And See :).
Great answer, Ether. Having done this far too many times to count, I can give you a few pieces of advice:
Note: I am apparently limited to a single link in the post, so I had to remove all of my annotations. Thankfully, there is Delicious where I've stored them all with a stackoverflowmacports tag. Any place below where I removed a link to fit under Stack Overflow's ridiculous anti-spam measure, I've marked it with (*).
If having a reliably-working development environment at all times is important to you, rely on as LITTLE Mac OS X bundled software as possible. I love Apple but they have absolutely no qualms about breaking custom setups of their software as often as possible.
If #1 sounds like what you need to do, Macports is an EXCELLENT choice. I used to use Fink but they got left in the dust ages ago in terms of ease of use and spectrum of available software. The easiest route to installing macports is via the binary package install method (*)
As Ether mentions, when you have everything set up correctly, the Macports-provided MySQL, PHP and Apache all work together well without the system-installed analogs interfering. Most of that has to do with your PATH setting but all of those details are handled by the package installers post-flight script (*)
Once you're on the Macports train, it should become the very first place you look for any software. port search and port info are constant companions. They've got 6863 ports (*) currently which covers MOST of your bases.
When you do need to go outside of the Macports realm to find something, install it in /usr/local. That part of the file system hierarchy is yours to play with. Don't be lulled into a false sense of security and think that because Macports doesn't have what you're installing, it's okay to put it in /opt/local because invariably that software will install some dependency that will ALSO be a dependency for some piece of Macports software down the line and Macports will not allow a port to be installed if any one of its files would overwrite an existing file not managed by Macports (unless you force it which is always bad manners)
If you do any work with Perl and you use Macports' version, you will absolutely find yourself in a situation Macports doesn't have the one CPAN module you're looking for. (And, really, given that there are two and a half billion CPAN modules, who can blame them?). This will happen often enough that you will most likely tire of the manual installation method (*) (perl Makefile.PL; make; make test; sudo make install; cha; cha; cha) and long for the ease of use you've grown accustomed to with Macports.
If so, you can absolutely use the cpan (*) utility, CPANPLUS (*) or cpanminus (*) for all of your installing needs. Just make sure to make the necessary adjustments in the configuration of your tool of choice to instruct it to install your modules into /usr/local/lib/perl5, ignoring /opt/local/bin/perl's insistence that modules go into /opt/local/lib/perl5. You can set the PERL5LIB environment variable in your shell's init scripts to additionally look in /usr/local/lib/perl5 for modules. Just grab the #INC output from perl -V and tack it on the end...
And finally... Leveraging the system's daily init scripts or third-party software like Anacron (*) or MacPorts Notifier (*) (both available through MacPorts), make sure to update your software frequently. You don't have Mother Apple protecting you with Software Updates for the Macports installed software which have just as many bugs and security exploits as the very same software Apple bundles.
By updating frequently, you'll stay ahead of the baddies and by automating it, the upgrades will actually happen and you won't end up as I have in the past with a full weekend blown because you had a mountain of outdated ports to upgrade. Note: Macports stages its updates and if it fails at any point, your current version continues to work. Apple could learn a thing or two from them, I tell ya...
So, that's all I can think of a the moment. Hopefully the lessons above will get you going quickly and save all of the time, effort and stress I've experienced in past years in learning it. I would argue it's still far better than the alternative: Hating Apple because they break all of your nice things... :-)
I had a similar problem with Apache using the wrong Perl. I fixed it by appending the following lines to my httpd.conf file:
SetEnv PATH [colon-separated list of directories]
SetEnv PERL5LIB [colon-sep'd list of directories]
In my case, it looked like this:
SetEnv PATH /opt/local/bin:/opt/local/sbin:/usr/local/bin:/usr/local/mysql/bin:$PATH
SetEnv PERL5LIB /opt/local/lib:/usr/local/lib