Working on POC with netCDF(.nc) files. Would like do some visualisation and while trying to install Basemap having some issues.
As per the pre-requisites, got numpy and matplotlib installed.
geos is already installed
When installing basemap from git %sh pip install pip install --user git+https://github.com/matplotlib/basemap.git getting below error.
Collecting git+https://github.com/matplotlib/basemap.git
Cloning https://github.com/matplotlib/basemap.git to /tmp/pip-req-build-w20pcpms
Running command git clone -q https://github.com/matplotlib/basemap.git /tmp/pip-req-build-w20pcpms
ERROR: Command errored out with exit status 1:
command: /databricks/python3/bin/python3.7 -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-req-build-w20pcpms/setup.py'"'"'; __file__='"'"'/tmp/pip-req-build-w20pcpms/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base /tmp/pip-req-build-w20pcpms/pip-egg-info
cwd: /tmp/pip-req-build-w20pcpms/
Complete output (18 lines):
checking for GEOS lib in /root ....
checking for GEOS lib in /root/local ....
checking for GEOS lib in /usr ....
checking for GEOS lib in /usr/local ....
checking for GEOS lib in /sw ....
checking for GEOS lib in /opt ....
checking for GEOS lib in /opt/local ....
Can't find geos library in standard locations ('/root', '/root/local', '/usr', '/usr/local', '/sw', '/opt', '/opt/local').
Please install the corresponding packages using your
systems software management system (e.g. for Debian Linux do:
'apt-get install libgeos-3.3.3 libgeos-c1 libgeos-dev' and/or
set the environment variable GEOS_DIR to point to the location
where geos is installed (for example, if geos_c.h
is in /usr/local/include, and libgeos_c is in /usr/local/lib,
set GEOS_DIR to /usr/local), or edit the setup.py script
manually and set the variable GEOS_dir (right after the line
that says "set GEOS_dir manually here".
----------------------------------------
ERROR: Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.
Depending on your runtime version you may be pointing at the wrong version of python (I'm assuming python 3) with your pip commands. Also, if you're installing python packages through pip inside of a notebook environment you're going to have a bad time. The best way to install through pip on a cluster is to use an init script.
Related
I installed Scipy using the command sudo apt-get install python-numpy python-scipy and looks like the latest version of both are installed. But when I try to install a python package named wfdb using pip install wfdb, it get an error while Building wheel for scipy (PEP 517).
ERROR: Failed building wheel for scipy
Running setup.py clean for scipy
ERROR: Command errored out with exit status 1:
command: /usr/bin/python -u -c 'import sys, setuptools, tokenize; sys.argv[0] =
'"'"'/tmp/pip-install-
fhfmfpqi/scipy/setup.py'"'"'; __file__='"'"'/tmp/pip-install-
fhfmfpqi/scipy/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)
(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code,
__file__,
'"'"'exec'"'"'))' clean --all
cwd: /tmp/pip-install-fhfmfpqi/scipy
Complete output (9 lines):
`setup.py clean` is not supported, use one of the following instead:
- `git clean -xdf` (cleans all files)
- `git clean -Xdf` (cleans all versioned files, doesn't touch
files that aren't checked into the git repo)
Add `--force` to your command to use it anyway if you must (unsupported).
----------------------------------------
ERROR: Failed cleaning build dir for scipy
Failed to build scipy
ERROR: Could not build wheels for scipy which use PEP 517 and cannot be installed directly`
I upgraded pip using pip install --upgrade pip.
python version used: 3.7
Can I get some help on this?
Clean out the install (I don't know which system you're using) i.e. get rid of the $PATH variables and just do a fresh install.
If you're using Anaconda:
conda install -c anaconda scipy
should do it
also the goops old
pip install scipy
gets it done as well. May I ask why you approached your installation in this way (again, I don't know which system you're on)
Never do sudo pip. Instead do pip install --user. Or (better) work in a virtualenv.
EDIT: typo: there is no space between -- and user.
There are errors when I try to install psycopg2
pip install psycopg2
I don't know whether I have pg_config or not. I tried these to find out where is pg_config, psql. but I don't understand how pg_config is not in the system when I have installed PostgreSQL(11.4). And if it is there in the system how to set the PATH so that psycopg2 installation is possible.
(myvenv) [jpsofficedev#localhost djangogirls]$ postgres --version
postgres (PostgreSQL) 11.4
(myvenv) [jpsofficedev#localhost djangogirls]$ echo $PATH
/home/jpsofficedev/Documents/coding_for_girls/djangogirls/myvenv/bin:/home/jpsofficedev/.local/bin:/home/jpsofficedev/bin:/usr/share/Modules/bin:/usr/local/bin:/usr/local/sbin:/usr/bin:/usr/sbin:/var/lib/snapd/snap/bin:/Library/PostgreSQL/11.4/bin
(myvenv) [jpsofficedev#localhost djangogirls]$ which psql
/usr/bin/psql
(myvenv) [jpsofficedev#localhost djangogirls]$ which pg_config
/usr/bin/which: no pg_config in (/home/jpsofficedev/Documents/coding_for_girls/djangogirls/myvenv/bin:/home/jpsofficedev/.local/bin:/home/jpsofficedev/bin:/usr/share/Modules/bin:/usr/local/bin:/usr/local/sbin:/usr/bin:/usr/sbin:/var/lib/snapd/snap/bin:/Library/PostgreSQL/11.4/bin)
Collecting psycopg2
Using cached https://files.pythonhosted.org/packages/5c/1c/6997288da181277a0c29bc39a5f9143ff20b8c99f2a7d059cfb55163e165/psycopg2-2.8.3.tar.gz
ERROR: Command errored out with exit status 1:
command: /home/jpsofficedev/Documents/coding_for_girls/djangogirls/myvenv/bin/python -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-w3r253fu/psycopg2/setup.py'"'"'; __file__='"'"'/tmp/pip-install-w3r253fu/psycopg2/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base pip-egg-info
cwd: /tmp/pip-install-w3r253fu/psycopg2/
Complete output (23 lines):
running egg_info
creating pip-egg-info/psycopg2.egg-info
writing pip-egg-info/psycopg2.egg-info/PKG-INFO
writing dependency_links to pip-egg-info/psycopg2.egg-info/dependency_links.txt
writing top-level names to pip-egg-info/psycopg2.egg-info/top_level.txt
writing manifest file 'pip-egg-info/psycopg2.egg-info/SOURCES.txt'
Error: pg_config executable not found.
pg_config is required to build psycopg2 from source. Please add the directory
containing pg_config to the $PATH or specify the full executable path with the
option:
python setup.py build_ext --pg-config /path/to/pg_config build ...
or with the pg_config option in 'setup.cfg'.
If you prefer to avoid building psycopg2 from source, please install the PyPI
'psycopg2-binary' package instead.
For further information please check the 'doc/src/install.rst' file (also at
<http://initd.org/psycopg/docs/install.html>).
----------------------------------------
ERROR: Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.
WARNING: You are using pip version 19.2.2, however version 19.2.3 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
This worked for me
sudo yum install python-devel postgresql-devel rpm-build
pip install psycopg2
As per suggested by one of the users, I used these commands and it worked for me.
sudo dnf install python-devel postgresql-devel rpm-build
pip install psycopg2
OP put this as a comment. It also worked for me. It would be best if you added this as an answer.
sudo yum install libpq-devel python-devel
ruamel.yaml seems to require PIP to install, which is not the default buildroot solution to build and install a Python package.
Is is possible to -at least- install a pure Python version of ruamel.yaml into a buildroot image - and how to circumvent the pip limitation?
Is is possible to cross-build ruamel.yaml?
Forcing RUAMEL_NO_PIP_INSTALL_CHECK env. var. does not help:
test compiling test_ruamel_yaml
running install
Checking .pth file support in ...
Failed to import the site module
ModuleNotFoundError: No module named '_sysconfigdata_m_linux_arm-linux-gnueabihf'
error: command '.../output/host/bin/python' failed with exit status 1
package/pkg-generic.mk:310: recipe for target '.../output/build/python-ruamel-yaml-0.15.45/.stamp_target_installed' failed
ruamel.yaml indeed requires pip to install from PyPI (using the .tar.gz or a .whl appropriate for your platform), this is documented.
The reason for this is that the fixes necessary to enable nested package installs where only implemented for pip (and not for easy_install or python setup.py installs).
That however does not preclude you from using ruamel.yaml, especially if you don't need the C extension (which is checked for at load time).
You can either check out a tagged version from bitbucket or untar a .tar.gz from PyPI and move the result to your site-packages directory:
$ virtualenv /tmp/ruamel_yaml_no_pip
Using base prefix '/opt/python/3.7'
New python executable in /tmp/ruamel_yaml_no_pip/bin/python
Installing setuptools, pip, wheel...done.
$ cd /tmp/ruamel_yaml_no_pip/
$ source bin/activate
(ruamel_yaml_no_pip) $ mkdir lib/python3.7/site-packages/ruamel/
(ruamel_yaml_no_pip) $ touch lib/python3.7/site-packages/ruamel/__init__.py
(ruamel_yaml_no_pip) $ wget -q https://files.pythonhosted.org/packages/63/a5/dba37230d6cf51f4cc19a486faf0f06871d9e87d25df0171b3225d20fc68/ruamel.yaml-0.15.45.tar.gz
(ruamel_yaml_no_pip) $ python -m ruamel.yaml
/tmp/ruamel_yaml_no_pip/bin/python: Error while finding module specification for 'ruamel.yaml' (ModuleNotFoundError: No module named 'ruamel')
(ruamel_yaml_no_pip) $ tar xf ruamel.yaml-0.15.45.tar.gz
(ruamel_yaml_no_pip) $ mv ruamel.yaml-0.15.45 lib/python3.7/site-packages/ruamel/yaml
(ruamel_yaml_no_pip) $ python -c 'from ruamel.yaml import YAML; print(YAML().load("{hello: world}")["hello"])'
world
(ruamel_yaml_no_pip)
(ruamel_yaml_no_pip) $ python -c 'from ruamel.yaml import __with_libyaml__ as X; print(X)'
False
(The URL is copied from the 0.15.45 project download page)
For development I normally just make a soft link from a virtualenv's site-packages to my ruamel directory.
I don't know how and if that translates to a buildroot environment (if so please publish your result).
I overlooked the buildroot documentation.
There is a critical parameter to define: SETUP_TYPE = setuptools rather than SETUP_TYPE = distutils.
With the following snippet:
PYTHON_RUAMEL_YAML_VERSION = 0.15.45
PYTHON_RUAMEL_YAML_SOURCE = ruamel.yaml-$(PYTHON_RUAMEL_YAML_VERSION).tar.gz
PYTHON_RUAMEL_YAML_SITE = https://pypi.python.org/packages/63/a5/dba37230d6cf51f4cc19a486faf0f06871d9e87d25df0171b3225d20fc68
PYTHON_RUAMEL_YAML_SETUP_TYPE = setuptools
PYTHON_RUAMEL_YAML_LICENSE = MIT
PYTHON_RUAMEL_YAML_LICENSE_FILES = LICENSE
PYTHON_RUAMEL_YAML_ENV += RUAMEL_NO_PIP_INSTALL_CHECK=1
$(eval $(python-package))
ruamel.yaml installs perfectly on the target image.
Here is a list of packages I had to install for CKAN 2.7.0
python-setuptools-36.2.4-4.mga7.noarch.rpm
pgdg-redhat96-9.6-3.noarch.rpm
xml-commons git subversion mercurial postgresql96-server postgresql96-devel postgresql96 python-devel libxslt libxslt-devel libxml2 libxml2-devel python-virtualenv gcc gcc-c++ make java-1.6.0-openjdk redis tomcat tomcat-webapps tomcat-admin-webapps xalan-j2unzip policycoreutils-python mod_wsgi httpd tcl-devel redis
after installing these tools i was able to download the git successfully using
pip install --ignore-installed -e git+https://github.com/okfn/ckan.git#ckan-2.7.0#egg=ckan
I then added pgsql-9.6 to the path in order for ckan to recognize pg_config
PATH=$PATH:/usr/pgsql-9.6/bin
Now when I attempt to run the requirements.txt installation it fails with this error
ImportError: No module named pytz
----------------------------------------
Cleaning up...
Command /usr/lib/ckan/default/bin/python -c "import setuptools;__file__='/usr/lib/ckan/default/build/pbr/setup.py';exec(compile(open(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /tmp/pip-Aun2rR-record/install-record.txt --single-version-externally-managed --install-headers /usr/lib/ckan/default/include/site/python2.7 failed with error code 1 in /usr/lib/ckan/default/build/pbr
Storing complete log in /usr/lib/ckan/.pip/pip.log
If anybody can explain what the error means and what I need to install/modify I would greatly appreciate it. Seems there is a package called pytz missing but I cannot find information about it.
Thanks
pytz must must installed inside the virtualenv and the pip-requirements-docs.txt must be run prior to running requirements.txt.
First of all, I am sorry for asking a question that was asked million times, however, I couldn't resolve my issue.
TL;DR:
psycopg2 builds in Python3.4 virtualenv, but not in Python3.2; suspected dev packages missing, where can I get dev packages for old python releases?
Long story:
I should be writing code for Python3.2 using django with PostgreSQL database engine.
Ubuntu 15.04 by default ships with Python3.4 so I have build Python3.2 from source:
$ python3.2 --version
Python 3.2.6
virtualenv is created so:
myproject $ virtualenv -p python3.2 venv
New python executable in venv/bin/python3.2
Also creating executable in venv/bin/python
Installing setuptools, pip...done.
Installing requirements:
myproject $ source venv/bin/activate
(venv)myproject $ pip install psycopg2
The output log can be found in pastebin.
What I have read there:
GCC finishes whatever its doing without error messages; last non-error reporting line is
running install_lib
then it fails with
Command /home/julka/LP/myproject/venv/bin/python3.2 -c "import setuptools, tokenize;__file__='/tmp/pip-build-cbu7a3/psycopg2/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /tmp/pip-cyu65o-record/install-record.txt --single-version-externally-managed --compile --install-headers /home/julka/LP/myproject/venv/include/site/python3.2 failed with error code 1 in /tmp/pip-build-cbu7a3/psycopg2
where it was compiling some lib.
Building with Python3.4
julka#Pyragas-vo2:~/L1P/myproject$ virtualenv -p python3.4 venv
Running virtualenv with interpreter /usr/bin/python3.4
Using base prefix '/usr'
New python executable in venv/bin/python3.4
Also creating executable in venv/bin/python
Installing setuptools, pip...done.
julka#Pyragas-vo2:~/LP/myproject$ source venv/bin/activate
(venv)julka#Pyragas-vo2:~/LP/myproject$ pip install psycopg2 --vv > psycopg2.log
Successfully installed psycopg2
Cleaning up...
(venv)julka#Pyragas-vo2:~/LP/myproject$
Successful installation. I have put the log in pastebin.
Maybe I'm missing some kind of development files for python3.2, what do i need to check this out?