So I'm a convert with pip and virtualenv. One of the things I've found is that when I use a requirements.txt with contents which look like this the subsequent pip -E -r requirements.txt works as intended..
PIL==1.1.7
chardet==1.0.1
distribute==0.6.14
django-extensions==0.6
django-profiles==0.2
But when you start getting fancy with your requirements.txt by pulling source trees, the following behavior is seen. 1) It pull the source into "source" tree. (That's good) 2) It fails to create a symlink to this in the site-packages tree but rather create *.link (.ie django-debug-toolbar.egg-link )
-e git+https://github.com/django-debug-toolbar/django-debug-toolbar.git#9067568b0bbf0c39322f5b761b05463685355b4f#egg=django_
debug_toolbar-dev
-e hg+https://bitbucket.org/ubernostrum/django-registration#fad7080fe76955e7d0d3c97bc468ea8360260a6b#egg=django_registration
-dev
-e git://github.com/yourcelf/django-registration-defaults.git#df80e13fd5f4ee3c96d4508fa12278c47c660c37#egg=django_registrati
on_defaults-dev
The problem is that now I have to create the symlink to the source tree from the site-packages tree.
To fix this I shift over to the site-packages tree and start creating symlinks back to the sources tree.
I'm sure there is a better way to do this - can someone please enlighten me??
You don't have to create a symlink at all, the so called *.egg-link files in the site-packages directory of the virtualenv will act as a symlink on Python (well, actually setuptools) level.
EDIT: I added a bit of background info here: http://github.com/pypa/pip/issues/435
Related
I tried doing the following. First, I install a relocatable perl, e.g.:
% perlbrew -Duserelocatableinc -j4 -n --as perl-5.28.1-fresh_src 5.28.1
Then, whenever I need a new throw-away fresh installation of perl to test things out, I do:
% cp -la ~/perl5/perlbrew/perls/perl-5.28.1-fresh{_src,1}
% perlbrew switch 5.28.1-fresh1
(then would later create fresh2, fresh3 and so on as needed; and when I want to throw these away I just rm -rf ~/perl5/perlbrew/perls/*fresh1 and so on.)
However, something (cpanm? EUMM?) is still confused. Scripts installed from CPAN still has the original perl path (in this case, /home/USER/perl5/perlbrew/perls/perl-5.28.1-fresh_src/bin) in their shebang line.
Any hint to the problem above, or an alternative way to quickly create a fresh throw-away perl installation is appreciated. Also, would the -l (--link) option of cp in the above command cause issue (other than perllocal.pod being appended and will contain installation records of perl installations, which is fine for me)?
It's not quite the same, but I find docker works if I'm needing a clean install for certain testing scenarios - build up a base image and a clean perl install, fire up the container and run 'whatever'.
You'll need root access and be able to install packages (e.g. docker) so it's not going to be suitable for every use case.
https://buildlogs.centos.org/centos/7/docker/ has a base image you can use to build a 'clean' CentOS (or use whatever OS you prefer frankly - most have container versions)
FROM scratch
ADD CentOS-7-20140625-x86_64-docker_01.img.tar.xz
LABEL name="CentOS Base Image"
CMD ["/bin/bash"]
In the working directory:
docker build -t mycentos .
This will give you a very basic image:
Then your perl dockerfile:
FROM mycentos
COPY local.repo /etc/yum.repos.d/local.repo
RUN yum clean all
RUN yum install --nogpgcheck -y make gcc tar
ADD perl-5.28.1.tar.gz /build/
RUN cd /build/perl-5.28.1 && ./Configure -de
RUN cd /build/perl-5.28.1 && make && make test && make install
COPY MyConfig.pm /root/.cpan/CPAN/MyConfig.pm
RUN cpan install Bundle::CPAN
CMD ["/bin/bash"]
You'll be able to spin this up, with e.g. docker run and invoke a script using /usr/local/bin/perl.
It's not quite what you asked for, but your base image can be started and discarded as you wish - you can maintain image for a few different OS and perl combinations too.
I am trying to install check_redis plugin and for that i need redis perl library but i don't have access to CPAN and internet.So Is there any way to install by downloading zip file and all the dependencies.I am using nagios core.Here is the plugin i am trying to install
Because I had a similar problem at work I happened to write my personal Q&A on this topic. Here it is:
Q: How to install CPAN modules without Internet access?
A: 1. Install CPAN::Mini on the internet machine:
$ cpan CPAN::Mini
2. Find a CPAN mirror here: http://mirrors.cpan.org/#de
e.g. http://ftp.hosteurope.de/pub/CPAN/
3. on the internet machine:
$ minicpan -l /path/to/local/cpan/mirror -r http://ftp.hosteurope.de/pub/CPAN/
4. Copy /path/to/local/cpan/mirror to the non-internet machine (or mount it)
5. On the non-internet machine add the directory to CPAN's search path:
$ cpan
cpan shell -- CPAN exploration and modules installation (v1.xxxx)
cpan> o conf urllist unshift file:///path/to/local/cpan/mirror
cpan> o conf commit
commit: wrote /home/user/.cpan/CPAN/MyConfig.pm
6. Install as usual on the non-internet machine.
To update your CPAN mirror go to your internet machine and repeat steps 3 and 4.
References:
http://www.perlmonks.org/?node_id=590386
http://www.cpan.org/misc/how-to-mirror.html
http://search.cpan.org/~rjbs/CPAN-Mini-1.111016/lib/CPAN/Mini.pm
http://search.cpan.org/~rjbs/CPAN-Mini-1.111016/bin/minicpan
Some notes
Although CPAN says the initial mirror will only occupy around 1 GB it will need around 3.5 GB. I think their documentation is outdated in this regard.
The mirror will initially contain the latest versions of all modules on CPAN. Every time you update your mirror the then-latest versions will be downloaded (leaving the old versions on your disk), so the mirror will grow with each update as new versions arrive.
I updated my mirror three or four times and it now has 3.9 GB.
You can install thing manually, of course, but it will be painful.
You can download the Redis distribution tarball from CPAN. But it might well require other modules that you don't have installed. You can see its list of requirements in its META.yml file. So you can download the distribution tarballs for each of those modules too. But, of course, each of those modules is likely to require other modules that you don't have. So you can look at all of their META.yml files. And so it goes on...
Honestly, it might be easier to work out how hard it would be to get the machine connected to the internet temporarily :-)
You could fatpack all the dependencies you cannot install into the check_nagios.pl using App::FatPacker. I'll outline how to do that.
First, let's make sure that the Nagios plugin has a licence that allows us to do this.
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation; either version 2 of the License, or
(at your option) any later version.
Ok, that's great. We're allowed to alter the source. Now let's clone it so we have the whole thing.
$ git clone git#github.com:willixix/WL-NagiosPlugins.git
$ cd WL-NagiosPlugins
Now we need to install App::FatPacker to our local Perl. We need to do this on a machine were installing stuff from the internet is not a problem. You can do it on the machine you used to write your question.
$ cpanm App::FatPacker
Afterwards, we need to also install all the dependencies. We know this needs Redis, so
$ cpanm Redis
Now typically all you need to do is run the fatpack pack command, but that would pack everything. And we don't really need that. We do have core modules on the target system after all. So we will do this step by step.
First, we need to tell fatpack to find all the dependencies.
$ fatpack trace check_redis.pl
This will create a fatpacker.trace file.
$ cat fatpacker.trace
IO/Handle.pm
Data/Dumper.pm
IO.pm
bytes.pm
Socket.pm
overload.pm
vars.pm
Redis/Sentinel.pm
lib.pm
Symbol.pm
Redis.pm
IO/Select.pm
PerlIO/via/Timeout.pm
base.pm
Scalar/Util.pm
fields.pm
IO/Socket/UNIX.pm
DynaLoader.pm
constant.pm
overloading.pm
Errno.pm
Getopt/Long.pm
warnings/register.pm
Carp.pm
IO/Socket.pm
SelectSaver.pm
Exporter/Heavy.pm
IO/Socket/Timeout.pm
List/Util.pm
Sub/Util.pm
Fcntl.pm
Time/HiRes.pm
IO/Socket/INET.pm
Try/Tiny.pm
Config.pm
Text/ParseWords.pm
I would suggest removing everything that looks like a core module. You turn the file names back into module names by replacing / with :: and removing the .pm, and then pass that into the corelist utility that ships with Perl on the target machine. All the pragmata (the ones that start with small letters) can be ignored, they are part of the core.
Here is the list after I cut it down to what I think makes sense to include. This might not be complete, or incorrect, but I can't try on your machine. The file now looks like this.
$ cat fatpacker.trace
Redis/Sentinel.pm
Redis.pm
Try/Tiny.pm
Now we follow the manual in RECIPES in the fatpack documentation.
$ fatpack packlists-for `cat fatpacker.trace` >packlists
The packlists file contains a couple of paths for where to find the packlists for those modules. This is specific to your Perl installation.
$ fatpack tree `cat packlists`
This will create a directory fatlib that contains all the stuff we want to bundle.
$ tree fatlib/
fatlib/
├── Redis
│ ├── Hash.pm
│ ├── List.pm
│ └── Sentinel.pm
├── Redis.pm
└── Try
└── Tiny.pm
2 directories, 5 files
Last, we need to run the actual packing command.
$ fatpack file check_redis.pl >check_redis.packed.pl
And now we have a file called check_redis.packed.pl that. If you open that and look at it, you'll see a BEGIN block that loads a bunch of stuff. That stuff is the dependencies that you could not install. They are now bundled into the script.
Move that script to the machine with your Nagios, and use it instead of the check_nagios.pl. It should now just work.
Please note that I have not actually tried if it works because I neither have a Redis nor a Nagios to try it. But it should work.
This worked for me in centOs 7.
step 1: Download dependencies in a centOs7 that is connected to internet by these commands:
yumdownloader --destdir=/etc/LinuxRepos/Redis epel-release
yumdownloader --destdir=/etc/LinuxRepos/Redis jemalloc
yumdownloader --destdir=/etc/LinuxRepos/Redis epel-release
step 2: copy downloaded files from /etc/LinuxRepos/Redis and copy them to destination centOS 7.
step 3: Install packages by
yum localinstal epel-release-7-11.noarch.rpm
yum localinstall jemalloc-3.6.0-1.el7.x86_64.rpm
yum localinstall epel-release-7-11.noarch.rpm
NOTICE: The name of files in step 3 may be changed due to current last
version.
I am trying to build an application release by using Snapcraft.io, and I have almost all working.Snapcraft already compiles the source code, generates the .snap file, includes all the dependencies, and so on.However, I am stuck at how I can initialize some configuration files in the SNAP_USER_DATA folder after the first app install.I do not want to place the files in the default read-only path SNAP, as the default parameters should be modified by the user, also I need to generate some additional files, like server certificates.So I need to copy some files, and also run a script after the first install. Is this possible?
Thanks.
Because snaps are installed as root, it's impossible to do exactly as you ask at install time, as $SNAP_USER_DATA is user-specific, so it'll always be root's. However, you can do this at install-time using a system-wide directory, such as $SNAP_DATA, using the install hook:
$ snapcraft init
Created snap/snapcraft.yaml.
Edit the file to your liking or run `snapcraft` to get started
Create the hook. In our case we'll just create a new file in $SNAP_DATA, but you could do whatever you wanted.
$ mkdir -p snap/hooks
$ echo "touch \$SNAP_DATA/foo" >> snap/hooks/install
$ chmod a+x snap/hooks/install
Build the snap.
$ snapcraft
Preparing to pull my-part
Pulling my-part
Preparing to build my-part
Building my-part
Staging my-part
Priming my-part
Snapping 'my-snap-name' |
Snapped my-snap-name_0.1_amd64.snap
Install the snap. This will run the install hook.
$ sudo snap install my-snap-name_0.1_amd64.snap --devmode --dangerous
my-snap-name 0.1 installed
Notice a file was created in $SNAP_DATA.
$ ls /var/snap/my-snap-name/current
foo
The only way to get similar functionality for $SNAP_USER_DATA would be to wrap your real command in a script that creates the config. This command is then run by the user, which means you get the $SNAP_USER_DATA you intend. Of course, this isn't at install-time.
I have a relatively big project that has many dependencies, and I would like to distribute this project around, but installing these dependencies where a bit of a pain, and takes a very long time (pip install takes quite some time). So I was wondering if it was possible to migrate a whole virtualenv to another machine and have it running.
I tried copying the whole virtualenv, but whenever I try running something, this virtualenv still uses the path of my old machine. For instance when I run
source activate
pserve development.ini
I get
bash: ../bin/pserve: /home/sshum/backend/bin/python: bad interpreter: No such file or directory
This is my old directory. So is there a way to have virtualenv reconfigure this path with a new path?
I tried sed -i 's/sshum/dev1/g' * in the bin directory and it solved that issue. However, I'm getting a different issue now, my guess is that this sed changed something.
I've confirmed that I have libssl-dev installed but when I run python I get:
E: Unable to locate package libssl.so.1.0.0
E: Couldn't find any package by regex 'libssl.so.1.0.0'
But when I run aptitude search libssl and I see:
i A libssl-dev - SSL development libraries, header files and documentation
I also tried virtualenv --relocatable backend but no go.
Export virtualenvironment
from within the virtual environment:
pip freeze > requirements.txt
as example, here is for myproject virtual environment:
once in the new machine & environment, copy the requirements.txt into the new project folder in the new machine and run the terminal command:
sudo pip install -r requirements.txt
then you should have all the packages previously available in the old virtual environment.
When you create a new virtualenv it is configured for the computer it is running on. I even think that it is configured for that specific directory it is created in. So I think you should always create a fresh virtualenv when you move you code. What might work is copying the lib/Pythonx.x/site-packages in your virtualenv directory, but I don't think that is a particularly good solution.
What may be a better solution is using the pip download cache. This will at least speed up the download part of pip install. Have a look at this thread: How do I install from a local cache with pip?
The clean way seems to be with virtualenv --relocatable.
Alternatively, you can do it manually by editing the VIRTUAL_ENV path in bin/activate to reflect the changes. If you choose to do so, you must also edit the first line (#) of bin/pserve which indicates the interpreter path.
I need to un-install a version of Perl which was built from source. The directory from which it was built exists. However I didn't find a make target called 'uninstall'. The Perl version I have is 5.12.2 and is installed on a Fedora distributed Linux.
Because perl has no 'make uninstall' target, you need to remove the files manually. The best way to do this is to get a complete list of files installed. To do that you need to:
Create a temporary directory, e.g. /usr/local/src/temp/perl
Edit the Makefile in your original perl source directory (hopefully you didn't delete it) and add the path from step 1 above to the beginning of all install lines (e.g. bin = ..., scriptdir = ..., INSTALLPREFIXEXP = ...)
Run make install
Navigate to your temp directory and run: find . -type f > filelist.txt
Edit this file and make sure you actually want to delete eveything in there (you will screw up your system real bad if you mess this up)
Run cat filelist.txt | xargs rm
Manually delete the perl5 library directory (usually at something like /usr/local/lib64/perl5 - you can find it in the filelist.txt file)
That's it, all gone.
Next time isolate it in a separate directory and just symlink it :-)
If the Perl is installed in its own directory - say /opt/perl/v5.12.2 - and was built from source, then the 'ultimate sanction' works well:
rm -fr /opt/perl/v5.12.2
I almost always build my own Perl; I always build my Perl so it installs in its own, unique directory; when I finally get around to removing it, this is how I do it.
A method that
Assumes installation in a subtree of /usr/local,
doesn't require the build directory
is slightly less accurate
Run the following command,
sudo find /usr/local -name '*perl*' -or -name 'pod2*' -or -name '*cpan*' -exec rm -rf {} \;
If you still have the source, you can remove it by using:
make uninstall
while you are in the source directory.
Btw, I sugggest to use checkinstall next time while installing from source.
See this
If you (as you said...) dont have a target uninstall, then you will probably will have to remove it by hand.