python 3.6 pip install fails for distutils installed projects on ubuntu 16.04 - ubuntu-16.04

I am running ubuntu 16.04 and have some python code that uses features from python 3.6. I have been able to install some libraries (scipy) with python3.6 and pip3.6. However there have been a few libraries (pycurl) for which using pip install is not working :
$sudo python3.6 -m pip install pycurl
Requirement already satisfied: pycurl in /usr/lib/python3/dist-packages (7.43.0)
This does not allow me to import pycurl in my python3.6 environment.
And if I instead use upgrade I get an error about pycurl being a distutils installed package.
How can I get access to pycurl for my python3.6 install?
$ sudo python3.6 -m pip install --upgrade pycurl
The directory '/home/riaps/.cache/pip/http' or its parent directory is not owned by the current user and the cache has been disabled. Please check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
The directory '/home/riaps/.cache/pip' or its parent directory is not owned by the current user and caching wheels has been disabled. check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
Collecting pycurl
Downloading https://files.pythonhosted.org/packages/77/d9/d272b38e6e25d2686e22f6058820298dadead69340b1c57ff84c87ef81f0/pycurl-7.43.0.1.tar.gz (195kB)
100% |████████████████████████████████| 204kB 2.8MB/s
Installing collected packages: pycurl
Found existing installation: pycurl 7.43.0
Cannot uninstall 'pycurl'. It is a distutils installed project and thus we cannot accurately determine which files belong to it which would lead to only a partial uninstall.

So the problem seems to have been that it was installed in /usr/lib/python3/dist-packages. By running sudo pip3 install --upgrade pycurl It was moved to : /usr/local/lib/python3.5/dist-packages and then sudo python3.6 -m pip install pycurl could install.
I don't know why this was a problem, but it resolved the issue.

Related

I got an error while creating virtualenv. It thows cmd not defined error

└─$ python3 -m venv venv
Error: name 'cmd' is not defined
┌──()-[~/Documents/Software Development/DjangoAuth/simplejwt]
└─$ source venv/bin/activate
source: no such file or directory: venv/bin/activate
It used to work fine on my other laptop while I just freshly installed new OS here, it throwing an odd error.
This bug report (found by search) suggests you don't have python-venv package installed. Install it:
sudo apt-get install -y python3-venv
The error message is due to a conflict in the versions of Python installed on my system. The package python3.10-venv depends on version 3.10.8-3 of Python, but version 3.10.9-1 is to be installed.
To resolve this issue, I tried the following:
Check the version of Python I have installed by running the command python3 --version.
If I have a version of Python other than 3.10.8-3, I tried uninstalling it and installing version 3.10.8-3 instead.
Once I have installed the correct version of Python, I install the python3-venv package again using the sudo apt-get install -y python3-venv command and it worked.
Thanks all who helped!

How to install egg file with easy_install?

On a HPC node, I have a file named pysdf-0.1-py3.8-linux-x86_64.egg.
I found a Nvidia Manual to install this file using python -m easy_install pysdf-0.1-py3.8-linux-x86_64.egg.
However, when I run this command I get the following error.
(/scratch/s.1915438/modulus) [s.1915438#sl1 eggs]$ python -m easy_install pysdf-0.1-py3.8-linux-x86_64.egg
/scratch/s.1915438/modulus/bin/python: No module named easy_install
Similarly, if I use pip install as follows
(/scratch/s.1915438/modulus) [s.1915438#sl1 eggs]$ python -m pip install pysdf-0.1-py3.8-linux-x86_64.egg
ERROR: Could not find a version that satisfies the requirement pysdf-0.1-py3.8-linux-x86_64.egg (from versions: none)
ERROR: No matching distribution found for pysdf-0.1-py3.8-linux-x86_64.egg
I do not have admin access on HPC server to use sudo apt. My python is installed at /scratch/s.1915438/modulus/bin/python on the HPC machine.
Does anyone knows why it says Could not find a version that satisfies the requirement.
After 3 months, I figured out how to do this. So, egg files can be installed using something called easy_install which was depreciated back in 2019. The last version of setuptools that supported easy_install was setuptools 42.0.0..
So, you need to downgrade to that version as follows.
pip3 install setuptools==42.0.0
And then install the egg files using the following command.
python3 -m easy_install example.egg
If you wish to upgrade the setuptools to the latest version, then type
pip3 install setuptools --upgrade

Unable to create virtual environment on CentOS 7.8 and python3.6

I have two servers that are running CentOS 7.8 with all the latest patches as of the date of this post. On one server I can create a venv, and on the other I get an error. Here is the command used to create the venv:
python3.6 -m venv venv
The error is:
Error: Command '['/root/venv/bin/python3.6', '-Im', 'ensurepip', '--upgrade', '--default-pip']' returned non-zero exit status 1.
So it seems there is a problem after the venv is created. In fact, it makes the links to the python interpreters in venv/bin, but the lib64/python3.6 and lib/python3.6 directories that are created have only empty site-packages in them. I get the problem whether running the command with root privileges (as above) or as a non-privileged user. On the other system, which is not quite the same hardware but very similar, I get no errors.
I have tried a bunch of things including completely uninstalling all the python3 rpms and reinstalling them on both systems. The python3 modules installed are:
python3, python3-devel, python3-libs, python3-other-rpm-macros, python3-pip, python3-rpm-generators, python3-setuptools, python3-tkinter, python3-wheel. All these are installed using the centos repos.
I have searched the internet and one thing that works is to use --no-pip. However, not surprisingly, if I do that I get a venv with no pip, which does not work very well. The version of pip that is installed with the rpm is version 9.0.3. However, I have tried upgrading pip:
python3.6 -m pip install --upgrade pip
This installs pip 20.1. Interestingly, it is installed in /usr/local/lib/python/site-packages. In any event, it doesn't change anything and I get essentially the same error message.
I have even gone as far as rsyncing the /usr/lib/python3.6 and /usr/lib64/python3.6 (using -avu --delete) on the two servers so that they are identical. All to no avail. I am at a loss to know what to do.
I can of course hack things to make them work, but since I am using this to automatically build test environments I really need to get it working automatically. Suggestions would be greatly appreciated.
I've had the same issue on CentOS Linux release 7.9.2009 (Core) and solved it by installing the following system packages:
yum install python36-devel
yum install python36-setuptools
yum install python36-virtualenv
python3.6 -m pip install --upgrade pip
Used virtualenv to install the venv into the project-root:
python3 -m virtualenv venv-dev
After sourcing the venv everything looks good and I could install the needed requirements:
. ./venv-dev/bin/activate
python3 --version
Python 3.6.8
pip3 --version
pip 21.1.1 from /project/root/dir/venv-dev/lib/python3.6/site-packages/pip (python 3.6)
pip3 install -r requirements/dev.txt
Successfully installed ...

Error "the PGXS Makefile cannot be found" when installing PostGis on Debian

I am in the process of installing PostGis over psql on a Debian machine (actually crunchbang).
I have completed the following steps:
$ wget http://download.osgeo.org/postgis/source/postgis-2.0.3.tar.gz
$ tar xzf postgis-2.0.3.tar.gz
$ cd postgis-2.0.3
$ ./configure
On the final step I get the following error:
configure: error: the PGXS Makefile /usr/lib/postgresql/9.1/lib/pgxs/src/makefiles/pgxs.mk cannot be found. Please install the PostgreSQL server development packages and re-run configure.
The issue is that I do already have Postgres installed:
$ psql --version
psql (9.1.9)
I have checked this on two machines with the same configuration and get the same error. What am I missing here?
PostgreSQL is broken down into several packages, and having psql installed doesn't imply that the development packages are also installed.
According to the error message:
Please install the PostgreSQL server development packages and re-run
configure
you need:
# apt-get install postgresql-server-dev-9.1
Also note there's a APT pgdg repository providing recent pre-compiled versions of postgres-related packages (including postgis) that you may use instead of self-compiling.
If your system is set up to use this repository, just do:
# apt-get install postgresql-9.1-postgis-2.0
Daniel's answer works great except that it needs the following update:
$ sudo apt-get install postgresql-9.1-postgis-2.1
These packages can be updated some time in future again. So, it is recommended to search for new packages using aptitude and install the appropriate one:
$ aptitude search postgis

Installing psycopg2 into virtualenv when PostgreSQL is not installed on development system

Is it possible to install psycopg2 into a virtualenv when PostgreSQL isn't installed on my development system—MacBook Pro with OS X 10.6?
When I run pip install psycopg2 from within my virtualenv, I received the error shown below.
I'm trying to connect to a legacy database on a server using Django, and I'd prefer not to install PostgreSQL on my development system if possible.
Why not install PostgreSQL?
I received an error when installing PostgreSQL using homebrew. I have Xcode4—and only Xcode4—installed on my MacBook Pro and am thinking it's related to missing gcc 4.0. However, this is a problem for another StackOverflow question.
Update 8:37 AM on April 12, 2011: I'd still like to know if this is possible without installing PostgreSQL on my MacBook Pro. However, I ran brew update and forced a reinstallation of ossp-uuid with brew install --force ossp-uuid and now brew install postgresql works. With PostgreSQL successfully installed, I was able to pip install psycopg2 from within my virtualenv.
Error from pip install psycopg2
$ pip install psycopg2
Downloading/unpacking psycopg2
Running setup.py egg_info for package psycopg2
Error: pg_config executable not found.
Please add the directory containing pg_config to the PATH
or specify the full executable path with the option:
python setup.py build_ext --pg-config /path/to/pg_config build ...
or with the pg_config option in 'setup.cfg'.
Complete output from command python setup.py egg_info:
running egg_info
writing pip-egg-info/psycopg2.egg-info/PKG-INFO
writing top-level names to pip-egg-info/psycopg2.egg-info/top_level.txt
writing dependency_links to pip-egg-info/psycopg2.egg-info/dependency_links.txt
warning: manifest_maker: standard file '-c' not found
Error: pg_config executable not found.
Please add the directory containing pg_config to the PATH
or specify the full executable path with the option:
python setup.py build_ext --pg-config /path/to/pg_config build ...
or with the pg_config option in 'setup.cfg'.
----------------------------------------
Command python setup.py egg_info failed with error code 1
Storing complete log in /Users/matthew/.pip/pip.log
Preliminary Research
Below are the articles I read as preliminary research:
Installing psycopg2 to use Django with PostgreSQL on OS X
Installing psycopg2 on OS X
Using psycopg2 with virtualenv on Ubuntu JauntyLucid
Postgres, psycopg2, virtualenv install hints
apt-get install libpq-dev
helped me on debian squeeze
From comments:
apt-get install python-dev-is-python3
psycopg depends on pg_config command, and if you don't have it, you can't install psycopg.
If system installation is a problem to you, why don't you try compiling PostgreSQL and including generated bin files in your $PATH? Like:
export PATH=/path/to/compiled/postgresql/bin:"$PATH"
pip install psycopg2
brew install postgresql
export LDFLAGS="-L/usr/local/opt/openssl/lib"
export CPPFLAGS="-I/usr/local/opt/openssl/include"
pip3 install psycopg2
apt-get install libpq-dev
helped me in debian squeeze too . After that do pip install psycopg2. I faced problem of pg_config not found problem when i was setting up my environment on heroku , now its working fine .
You need to install the python-dev package in order to make use of python extensions such as psycopg2. I don't know how to install packages in mac but I run the following commands to install a python package on my Ubuntu machine.
sudo apt-get install python-dev
Or
sudo apt-get install python3-dev
if you are using Python3.x.
Once the installation is finished run the following command within your virtual environment.
pip install psycopg2
You don't need the complete PostgreSQL installed: you only need the client side libraries.
I solved it in MAC OSX using :
$ wget https://ftp.postgresql.org/pub/source/v9.5.3/postgresql-9.5.3.tar.bz2
$ tar xfv postgresql-9.5.3.tar.bz2
$ cd postgresql-9.5.3
$ ./configure
$ make
$ cd src/bin/pg_config
$ export PATH=`pwd`:"$PATH"
$ pip install psycopg2
Use pip install psycopg2-binary, it worked for me when pip install psycopg2 wasn't working.
Using the method you described in your April 12th update, I was able to install PostgreSQL(+1). Note that I originally was running Python 2.7.1 (32bit) and homebrew threw several errors and warnings regarding using a 32bit version of Python. I have since switched the 64/32 bit version of 2.7.1 and it works like a champ.
Regarding the pyscopg2, I was able to install it into my virtual environment from source by editing setup.cfg. I set pg_config to correct path inside homebrew's Cellar (pg_config=/usr/local/Cellar/postgresql/9.0.4/bin/pg_config). After saving the changes, I ran python setup.py install with zero issues. It's worth noting that I did not set the Mac's default python setting to 32 bit. I used the new 64 bit from start to finish.
After looking over some of the documentation, I think if I added homebrew's postgresql path to the system path I could have used pip to install it.
Reference:
http://favosdream.blogspot.com/2009/09/make-psycopg2-and-readline-work-in-snow.html
Update 6-8-2011:
While porting a project written on OS X to Windows 7, I found out that I had to install PostgreSQL on Windows as well. This ended up creating another user on my start up screen and other things that I just didn't like. While doing some digging I found Windows drivers for PostgreSQL here. I have since uninstalled the full PostgreSQL and installed the ODBC drivers which, thus far, work great.
To address the original question, after doing a bit more digging I think I found the equivalent ODBC for OS X here. I have not had a chance to try them out, but the concept works very well on Windows 7. I will update this when I get a chance to try them out. Until then, I hope this helps.
All from the above doesn't work for me (OS Catalina 10.15.1)
There was a conflict with classical postgres and homebrew version.
Please delete homebrew version by command
$ brew uninstall postgresql
and then install it from the official website:
$ sudo mkdir -p /etc/paths.d && echo /Applications/Postgres.app/Contents/Versions/latest/bin | sudo tee /etc/paths.d/postgresapp
It is actual for a simple psycopg2 install or django-heroku package.
if you have recently updated python or changed default python (let's say from 3.6 to 3.8). The following code
sudo apt-get install python-dev OR sudo apt-get install python3-dev
will be installing/working for the previous python version.
so if you want this command to work for the recently updated/changed python version try mentioning that specific version like python3.8 in command like
sudo apt-get install python3.8-dev
try above with following
pip install wheel
export PATH=/path/to/compiled/postgresql/bin:"$PATH"
sudo apt-get install libpq-dev
sudo apt-get install python3.x-dev **Change x with your version, eg python3.8**
pip install psycopg2-binary
pip install psycopg2
As I never needed to install postgresql database on this server I installed the following libraries on Ubuntu 14_04 version before running pip install psycopg2 on the same server
apt-get install libpq-dev python-dev
and then executed pip install psycopg2 within virtual env.
Output
Collecting psycopg2
Using cached psycopg2-2.6.1.tar.gz
Building wheels for collected packages: psycopg2
Running setup.py bdist_wheel for psycopg2
Stored in directory: /root/.cache/pip/wheels/e2/9a/5e/7b620848bbc7cfb9084aafea077be11618c2b5067bd532f329
Successfully built psycopg2
Installing collected packages: psycopg2
Successfully installed psycopg2-2.6.1
I know you are asking for development environment but if you are deploying on server say, Heroku.
Just add below line in the requirements.txt of your project.
django-heroku==0.3.1
As this package itself will install the required packages like psycopg2 on server deployment.
Try to specify the version and it'll works, do it inside the venv
pip install psycopg2-binary==2.8.6
based on my experience, apt install libpq-dev then try to install psycopg2 inside venv will fix many issues, however for more information you can refer to this link:
https://www.psycopg.org/docs/install.html