I'm trying to install SciPy using a bootstrap script to use it to calculate a new field in a dataframe, running Spark on AWS EMR.
Although the SciPy website states that only NumPy is needed, when I tried to install SciPy using pip, pip kept failing, complaining about missing software, until I ended up with this bootstrap script:
sudo yum install -y python3-devel
sudo pip3 install -U Cython
sudo pip3 install -U pybind11
sudo pip3 install -U pythran
sudo pip3 install -U numpy
sudo pip3 install -U scipy
At this point, the bootstrap seems to be successful, but then at this line:
from scipy.stats import norm
I get the following error:
ValueError: numpy.ndarray size changed, may indicate binary incompatibility. Expected 88 from C header, got 80 from PyObject
I tried --no-binary :all:, but that made no difference.
Any advice? Thanks!
Best, Oliver
When trying to import SkewT into my python3 code on a Mac (Mojave 10.14.6):
from metpy.plots import SkewT
I get the error:
ModuleNotFoundError: No module named 'cartopy'
pip3 install cartopy gives the output
Collecting cartopy
Downloading https://files.pythonhosted.org/packages/e5/92/fe8838fa8158931906dfc4f16c5c1436b3dd2daf83592645b179581403ad/Cartopy-0.17.0.tar.gz (8.9MB)
|████████████████████████████████| 8.9MB 616kB/s
Installing build dependencies ... done
Getting requirements to build wheel ... error
ERROR: Complete output from command /usr/local/opt/python/bin/python3.7 /usr/local/lib/python3.7/site-packages/pip/_vendor/pep517/_in_process.py get_requires_for_build_wheel /tmp/tmpj50b1vfe:
ERROR: setup.py:171: UserWarning: Unable to determine GEOS version. Ensure you have 3.3.3 or later installed, or installation may fail.
'.'.join(str(v) for v in GEOS_MIN_VERSION), ))
Proj 4.9.0 must be installed.
----------------------------------------
ERROR: Command "/usr/local/opt/python/bin/python3.7 /usr/local/lib/python3.7/site-packages/pip/_vendor/pep517/_in_process.py get_requires_for_build_wheel /tmp/tmpj50b1vfe" failed with error code 1 in /private/tmp/pip-install-b5cu8485/cartopy
To start, I tried to install Proj and geos, but pip3 only lists version 0.1.0 for proj and 0.2.2 for geos. Before I get too far down this rabbit hole, I thought I'd see if anyone else has encountered this problem. Thanks!
So it looks like MetPy 0.10 accidentally picked up a hard dependency on CartoPy, which we did not really plan. You can track our resolution of that here.
CartoPy depends on a lot of compiled libraries that are not pip-installable unfortunately. Your best bet is to look at CartoPy's install instructions. If you're using Anaconda or Canopy, those distributions have pre-built CartoPy packages available.
One option to work around this is to install MetPy 0.9:
pip install metpy==0.9
Do you use Conda? The easiest way to remedy this problem is to install CartoPy (or MetPy for that manner) via conda, so that all of the right dependencies are also downloaded: conda install -c conda-forge cartopy or conda install -c conda-forge metpy. Pip doesn't bring all of them together, so that leads to this problem being raised.
Thanks. Without conda, I was also able to complete this (more painful) installation:
- brew install geos
- brew install proj
- pip3 install cython
- pip3 install git+https://github.com/SciTools/cartopy.git#master
(see http://louistiao.me/posts/installing-cartopy-on-mac-osx-1011/)
It keeps freezing or gives me the
'arm-linux-gnueabihf-gcc': error.
One option is to just use the packaged versions of the lxml module.
If you're on Python 3:
apt-get install python3-lxml
Or on Python 2:
apt-get install python-lxml
If you create your virtualenv with --system-site-packages, you will have access to this install of lxml from the virtualenv.
I'm running Raspbian Stretch on my Pi, and it looks like pip install grabs a binary build (so no gcc required):
(venv) pi#lbb:~ $ pip install lxml
Looking in indexes: https://pypi.org/simple, https://www.piwheels.org/simple
Collecting lxml
ww Downloading https://www.piwheels.org/simple/lxml/lxml-4.2.1-cp35-cp35m-linux_armv7l.whl (4.9MB)
100% |████████████████████████████████| 4.9MB 1.0MB/s
Installing collected packages: lxml
Successfully installed lxml-4.2.1
Are you running the same distribution on your Pi? If not, please update your question to include that information.
i am Nobe about python and its package,
I want to install networkx package for some graph calculation. I found it https://pypi.python.org/pypi/networkx/1.8.1 . But the file is .egg.
Please help me to install that *.egg file for my python3.3 .
Thank you, in advance
The install docs show you several ways to install networkx. pip is one of the easiest ways to get python packages installed (managing dependencies and updates etc).
On debian systems you can install pip by running:
sudo apt-get install python3-pip
(note that there are typically different packages for pip3 and for pip2)
and then you should be able to install networkx as follows:
sudo pip3 install networkx
If you use windows maybe read How do I install pip on Windows? to get started with pip.
Is it possible to install psycopg2 into a virtualenv when PostgreSQL isn't installed on my development system—MacBook Pro with OS X 10.6?
When I run pip install psycopg2 from within my virtualenv, I received the error shown below.
I'm trying to connect to a legacy database on a server using Django, and I'd prefer not to install PostgreSQL on my development system if possible.
Why not install PostgreSQL?
I received an error when installing PostgreSQL using homebrew. I have Xcode4—and only Xcode4—installed on my MacBook Pro and am thinking it's related to missing gcc 4.0. However, this is a problem for another StackOverflow question.
Update 8:37 AM on April 12, 2011: I'd still like to know if this is possible without installing PostgreSQL on my MacBook Pro. However, I ran brew update and forced a reinstallation of ossp-uuid with brew install --force ossp-uuid and now brew install postgresql works. With PostgreSQL successfully installed, I was able to pip install psycopg2 from within my virtualenv.
Error from pip install psycopg2
$ pip install psycopg2
Downloading/unpacking psycopg2
Running setup.py egg_info for package psycopg2
Error: pg_config executable not found.
Please add the directory containing pg_config to the PATH
or specify the full executable path with the option:
python setup.py build_ext --pg-config /path/to/pg_config build ...
or with the pg_config option in 'setup.cfg'.
Complete output from command python setup.py egg_info:
running egg_info
writing pip-egg-info/psycopg2.egg-info/PKG-INFO
writing top-level names to pip-egg-info/psycopg2.egg-info/top_level.txt
writing dependency_links to pip-egg-info/psycopg2.egg-info/dependency_links.txt
warning: manifest_maker: standard file '-c' not found
Error: pg_config executable not found.
Please add the directory containing pg_config to the PATH
or specify the full executable path with the option:
python setup.py build_ext --pg-config /path/to/pg_config build ...
or with the pg_config option in 'setup.cfg'.
----------------------------------------
Command python setup.py egg_info failed with error code 1
Storing complete log in /Users/matthew/.pip/pip.log
Preliminary Research
Below are the articles I read as preliminary research:
Installing psycopg2 to use Django with PostgreSQL on OS X
Installing psycopg2 on OS X
Using psycopg2 with virtualenv on Ubuntu JauntyLucid
Postgres, psycopg2, virtualenv install hints
apt-get install libpq-dev
helped me on debian squeeze
From comments:
apt-get install python-dev-is-python3
psycopg depends on pg_config command, and if you don't have it, you can't install psycopg.
If system installation is a problem to you, why don't you try compiling PostgreSQL and including generated bin files in your $PATH? Like:
export PATH=/path/to/compiled/postgresql/bin:"$PATH"
pip install psycopg2
brew install postgresql
export LDFLAGS="-L/usr/local/opt/openssl/lib"
export CPPFLAGS="-I/usr/local/opt/openssl/include"
pip3 install psycopg2
apt-get install libpq-dev
helped me in debian squeeze too . After that do pip install psycopg2. I faced problem of pg_config not found problem when i was setting up my environment on heroku , now its working fine .
You need to install the python-dev package in order to make use of python extensions such as psycopg2. I don't know how to install packages in mac but I run the following commands to install a python package on my Ubuntu machine.
sudo apt-get install python-dev
Or
sudo apt-get install python3-dev
if you are using Python3.x.
Once the installation is finished run the following command within your virtual environment.
pip install psycopg2
You don't need the complete PostgreSQL installed: you only need the client side libraries.
I solved it in MAC OSX using :
$ wget https://ftp.postgresql.org/pub/source/v9.5.3/postgresql-9.5.3.tar.bz2
$ tar xfv postgresql-9.5.3.tar.bz2
$ cd postgresql-9.5.3
$ ./configure
$ make
$ cd src/bin/pg_config
$ export PATH=`pwd`:"$PATH"
$ pip install psycopg2
Use pip install psycopg2-binary, it worked for me when pip install psycopg2 wasn't working.
Using the method you described in your April 12th update, I was able to install PostgreSQL(+1). Note that I originally was running Python 2.7.1 (32bit) and homebrew threw several errors and warnings regarding using a 32bit version of Python. I have since switched the 64/32 bit version of 2.7.1 and it works like a champ.
Regarding the pyscopg2, I was able to install it into my virtual environment from source by editing setup.cfg. I set pg_config to correct path inside homebrew's Cellar (pg_config=/usr/local/Cellar/postgresql/9.0.4/bin/pg_config). After saving the changes, I ran python setup.py install with zero issues. It's worth noting that I did not set the Mac's default python setting to 32 bit. I used the new 64 bit from start to finish.
After looking over some of the documentation, I think if I added homebrew's postgresql path to the system path I could have used pip to install it.
Reference:
http://favosdream.blogspot.com/2009/09/make-psycopg2-and-readline-work-in-snow.html
Update 6-8-2011:
While porting a project written on OS X to Windows 7, I found out that I had to install PostgreSQL on Windows as well. This ended up creating another user on my start up screen and other things that I just didn't like. While doing some digging I found Windows drivers for PostgreSQL here. I have since uninstalled the full PostgreSQL and installed the ODBC drivers which, thus far, work great.
To address the original question, after doing a bit more digging I think I found the equivalent ODBC for OS X here. I have not had a chance to try them out, but the concept works very well on Windows 7. I will update this when I get a chance to try them out. Until then, I hope this helps.
All from the above doesn't work for me (OS Catalina 10.15.1)
There was a conflict with classical postgres and homebrew version.
Please delete homebrew version by command
$ brew uninstall postgresql
and then install it from the official website:
$ sudo mkdir -p /etc/paths.d && echo /Applications/Postgres.app/Contents/Versions/latest/bin | sudo tee /etc/paths.d/postgresapp
It is actual for a simple psycopg2 install or django-heroku package.
if you have recently updated python or changed default python (let's say from 3.6 to 3.8). The following code
sudo apt-get install python-dev OR sudo apt-get install python3-dev
will be installing/working for the previous python version.
so if you want this command to work for the recently updated/changed python version try mentioning that specific version like python3.8 in command like
sudo apt-get install python3.8-dev
try above with following
pip install wheel
export PATH=/path/to/compiled/postgresql/bin:"$PATH"
sudo apt-get install libpq-dev
sudo apt-get install python3.x-dev **Change x with your version, eg python3.8**
pip install psycopg2-binary
pip install psycopg2
As I never needed to install postgresql database on this server I installed the following libraries on Ubuntu 14_04 version before running pip install psycopg2 on the same server
apt-get install libpq-dev python-dev
and then executed pip install psycopg2 within virtual env.
Output
Collecting psycopg2
Using cached psycopg2-2.6.1.tar.gz
Building wheels for collected packages: psycopg2
Running setup.py bdist_wheel for psycopg2
Stored in directory: /root/.cache/pip/wheels/e2/9a/5e/7b620848bbc7cfb9084aafea077be11618c2b5067bd532f329
Successfully built psycopg2
Installing collected packages: psycopg2
Successfully installed psycopg2-2.6.1
I know you are asking for development environment but if you are deploying on server say, Heroku.
Just add below line in the requirements.txt of your project.
django-heroku==0.3.1
As this package itself will install the required packages like psycopg2 on server deployment.
Try to specify the version and it'll works, do it inside the venv
pip install psycopg2-binary==2.8.6
based on my experience, apt install libpq-dev then try to install psycopg2 inside venv will fix many issues, however for more information you can refer to this link:
https://www.psycopg.org/docs/install.html