In one of my setuptools-based Python projects I would like to add a dependency to numpy-stl. That package is on PyPI and depends on numpy. Due to a bug, setuptools is unable to install packages with dashes in their names in some circumstances.
This is a minimal setup.py which reproduces the problem:
import setuptools
setuptools.setup(
install_requires=['numpy-stl'])
Running python setup.py develop produces the following output:
running install
running bdist_egg
running egg_info
writing UNKNOWN.egg-info/PKG-INFO
writing top-level names to UNKNOWN.egg-info/top_level.txt
writing dependency_links to UNKNOWN.egg-info/dependency_links.txt
writing requirements to UNKNOWN.egg-info/requires.txt
reading manifest file 'UNKNOWN.egg-info/SOURCES.txt'
writing manifest file 'UNKNOWN.egg-info/SOURCES.txt'
installing library code to build/bdist.macosx-10.9-x86_64/egg
running install_lib
warning: install_lib: 'build/lib' does not exist -- no Python modules to install
creating build/bdist.macosx-10.9-x86_64/egg
creating build/bdist.macosx-10.9-x86_64/egg/EGG-INFO
copying UNKNOWN.egg-info/PKG-INFO -> build/bdist.macosx-10.9-x86_64/egg/EGG-INFO
copying UNKNOWN.egg-info/SOURCES.txt -> build/bdist.macosx-10.9-x86_64/egg/EGG-INFO
copying UNKNOWN.egg-info/dependency_links.txt -> build/bdist.macosx-10.9-x86_64/egg/EGG-INFO
copying UNKNOWN.egg-info/requires.txt -> build/bdist.macosx-10.9-x86_64/egg/EGG-INFO
copying UNKNOWN.egg-info/top_level.txt -> build/bdist.macosx-10.9-x86_64/egg/EGG-INFO
zip_safe flag not set; analyzing archive contents...
creating 'dist/UNKNOWN-0.0.0-py3.5.egg' and adding 'build/bdist.macosx-10.9-x86_64/egg' to it
removing 'build/bdist.macosx-10.9-x86_64/egg' (and everything under it)
Processing UNKNOWN-0.0.0-py3.5.egg
Copying UNKNOWN-0.0.0-py3.5.egg to /Users/michi/Desktop/STL_Plot/stl_plot/venv/lib/python3.5/site-packages
Adding UNKNOWN 0.0.0 to easy-install.pth file
Installed /Users/michi/Desktop/STL_Plot/stl_plot/venv/lib/python3.5/site-packages/UNKNOWN-0.0.0-py3.5.egg
Processing dependencies for UNKNOWN==0.0.0
Searching for numpy-stl
Reading https://pypi.python.org/simple/numpy-stl/
[... installing numpy-stl and some of its dependencies ...]
Installed /Users/michi/Desktop/STL_Plot/stl_plot/venv/lib/python3.5/site-packages/nine-0.3.4-py3.5.egg
Searching for numpy
Best match: numpy stl-1.7.1
Downloading https://pypi.python.org/packages/source/n/numpy-stl/numpy-stl-1.7.1.tar.gz#md5=fa3a9324418b72a6827f266671198fe1
Processing numpy-stl-1.7.1.tar.gz
Writing /var/folders/r3/lpx2z49s5hs8mmh3vzrgxd9c0000gs/T/easy_install-yvi_spnk/numpy-stl-1.7.1/setup.cfg
Running numpy-stl-1.7.1/setup.py -q bdist_egg --dist-dir /var/folders/r3/lpx2z49s5hs8mmh3vzrgxd9c0000gs/T/easy_install-yvi_spnk/numpy-stl-1.7.1/egg-dist-tmp-72gff1lq
File "build/bdist.macosx-10.9-x86_64/egg/stl2/stl.py", line 41
except RuntimeError, (recoverable, e):
^
SyntaxError: invalid syntax
zip_safe flag not set; analyzing archive contents...
tests.__pycache__.test_convert.cpython-35: module references __file__
removing '/Users/michi/Desktop/STL_Plot/stl_plot/venv/lib/python3.5/site-packages/numpy_stl-1.7.1-py3.5.egg' (and everything under it)
creating /Users/michi/Desktop/STL_Plot/stl_plot/venv/lib/python3.5/site-packages/numpy_stl-1.7.1-py3.5.egg
Extracting numpy_stl-1.7.1-py3.5.egg to /Users/michi/Desktop/STL_Plot/stl_plot/venv/lib/python3.5/site-packages
File "/Users/michi/Desktop/STL_Plot/stl_plot/venv/lib/python3.5/site-packages/numpy_stl-1.7.1-py3.5.egg/stl2/stl.py", line 41
except RuntimeError, (recoverable, e):
^
SyntaxError: invalid syntax
numpy-stl 1.7.1 is already the active version in easy-install.pth
Installing stl2ascii script to /Users/michi/Desktop/STL_Plot/stl_plot/venv/bin
Installing stl2bin script to /Users/michi/Desktop/STL_Plot/stl_plot/venv/bin
Installing stl script to /Users/michi/Desktop/STL_Plot/stl_plot/venv/bin
Installed /Users/michi/Desktop/STL_Plot/stl_plot/venv/lib/python3.5/site-packages/numpy_stl-1.7.1-py3.5.egg
error: The 'numpy' distribution was not found and is required by numpy-stl
(venv)michi ~/.../STL_Plot/stl_plot$
After installing nine, setuptools tries to find a numpy distribution and finds numpy-stl, thinking its the package numpy in version stl and installs numpy-stl again. In the end it detects that numpy is still not installed and gives up.
Is this just the way it is, that setuptools projects cannot depend on packages with dashes in their names, which is about 1/4 of the packages on PyPI? Or is there a workaround which allows setuptools to install such dependencies?
Related
I'm working on a pybind11 extension written in C++ but I'm having a hard time understanding how should it be distributed.
The project links to a number of third party libraries (e.g. libpng, glew etc.).
The project builds fine with CMAKE and it generates a .so file. Now I am not sure what is the right way of installing this extension. The extension seems to work, as if I try copy the file into the python lib directories it is picked up (I can import it, and it works correctly). However, this is clearly not the way to go I think.
I also tried the setuptools route (from https://pybind11.readthedocs.io/en/stable/compiling.html) by creating a setup.py files like this:
import sys
# Available at setup time due to pyproject.toml
from pybind11 import get_cmake_dir
from pybind11.setup_helpers import Pybind11Extension, build_ext
from setuptools import setup
from glob import glob
files = sorted(glob("*.cpp"))
__version__ = "0.0.1"
ext_modules = [
Pybind11Extension("mylib",
files,
# Example: passing in the version to the compiled code
define_macros = [('VERSION_INFO', __version__)],
),
]
setup(
name="mylib",
version=__version__,
author="fab",
author_email="fab#fab",
url="https://github.com/pybind/python_example",
description="mylib",
long_description="",
ext_modules=ext_modules,
extras_require={"test": "pytest"},
cmdclass={"build_ext": build_ext},
zip_safe=False,
python_requires=">=3.7",
)
and now I can build the extension by simply calling
pip3 install
however it looks like all the links are broken because whenever I try importing the extension in Python I get linkage errors, as if setuptools does not link correctly the extension with the 3rd party libs. For instance errors in linking with libpng as in:
>>> import mylib
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ImportError: /home/fabrizio/.local/lib/python3.8/site-packages/mylib.cpython-38-x86_64-linux-gnu.so: undefined symbol: png_sig_cmp
However I have no clue how to add this link info to setuptools, and don't even know if that's possible (it should be the setuptools equivalent of CMAKE's target_link_libraries).
I am really at a loss after weeks of reading documentation, forum threads and failed attempts. If anyone is able to point me in the right way or to clear some of the fog it would be really appreciated!
Thanks!
Fab
/home/fabrizio/.local/lib/python3.8/site-packages/mylib.cpython-38-x86_64-linux-gnu.so: undefined symbol: png_sig_cmp
This line pretty much says it clearly. Your local shared object file .so can't find the libpng.so against which it is linked.
You can confirm this by running:
ldd /home/fabrizio/.local/lib/python3.8/site-packages/mylib.cpython-38-x86_64-linux-gnu.so
There is no equivalent of target_link_libraries() in setuptools. Because that wouldn't make any sense. The library is already built and you've already linked it. This is your system more or less telling you that it can't find the libraries it needs. And those most likely need to be installed.
This is also one of the reasons why Linux distributions provide their own package managers and why you should use the developer packages provided by said distributions.
So how do you fix this? Well your .so file needs to find the other .so files against which you linked to understand how this works I will refer you to this link.
My main guess is based on the fact that when you manually copy the files it works - That during the build process you probably specify the rpath to a local directory. Hence what you most likely need to do is specify to your setuptools that it needs to copy those files when installing.
How do I install Helm (https://hackage.haskell.org/package/helm) on Windows 7 (64-bit)?
(Update: I had posted a lot of error messages here, but I've moved them to my answer to not clutter up the question.)
Installation for Windows 64-bit:
I'm including error messages, for if you follow all the steps up to that point and then just try to install directly. This is a conglomeration of a bunch of ad-hoc steps from following many different posts. Any simplification would be appreciated!
Note: Do all work in directories without spaces. I'm doing all work in C:/PF; modify this to your directory.
Download MSYS2-x86_64 from https://msys2.github.io/ and install it. Cabal install cairo (or helm) will give something like:
Configuring cairo-0.13.1.0...
setup.exe: Missing dependencies on foreign libraries:
Missing C libraries: z, cairo, z, gobject-2.0, ffi, pixman-1, fontconfig,
expat, freetype, iconv, expat, freetype, z, bz2, harfbuzz, glib-2.0, intl,
ws2_32, ole32, winmm, shlwapi, intl, png16, z
Download C libraries. In MINGW64 (NOT MSYS2 - I had trouble with MSYS2 at random stages in the process), use the package manager:
pacman -Ss cairo
to search for the Cairo package. You'll find "mingw64/mingw-w64-x86_64-cairo", so install that:
pacman -S mingw64/mingw-w64-x86_64-cairo
*.pc files should have been added to C:\PF\msys64\mingw64\lib\pkgconfig and C:\PF\msys64\usr\lib\pkgconfig. (pkg-config needs to be able to find these files. It looks in PKG_CONFIG_PATH, which by default should have the lib/pkgconfig folder above. Moving the file here is easiest. See Can't install sdl2 via cabal) If you get
The pkg-config package ... version ... cannot be found
errors then check your *.pc files.
Repeat with other required libraries, like atk
pacman -S mingw64/mingw-w64-x86_64-atk
(I don't know the complete list, but error messages later on will let you know what to get.)
Get the development files for these libraries (as suggested by How to install cairo on Windows). Most of them are bundled up at http://ftp.gnome.org/pub/gnome/binaries/win64/gtk+/2.22/. Unzip.
Copy files (.a, .dll.a) in lib to C:\PF\msys64\mingw64\lib. Copy the pkgconfig folder, which contains the .pc files.
Copy files in include to C:\PF\msys64\mingw64\include.
Add C:\PF\gtk+-2.22.1\bin to the path.
(2) and (3) might be redundant. I don't know - I did them both.
At this point you can probably do "cabal install cairo". (Warning: if your end goal is something else, you may not want to "cabal install" intermediate packages, see https://wiki.haskell.org/Cabal/Survival#Issue_.232_--_Not_installing_all_the_packages_in_one_go.)
See (4) for the syntax in specifying extra-include-dirs and extra-lib-dirs (but if you copied the files above this shouldn't be necessary),
Any time you get
Missing (or bad) header file
check to see you copied the *.h files to mingw64\include and/or add the include folder to the PATH. Use cabal install -v3 to get verbose error messages if the problem persists.
If you get something like
cairo-0.13.1.0: include-dirs: /mingw64/include/freetype2 is a relative path
which makes no sense (as there is nothing for it to be relative to). You can
make paths relative to the package database itself by using ${pkgroot}. (use
--force to override)
try --ghc-pkg-options="--force" (as mentioned at https://github.com/gtk2hs/gtk2hs/issues/139).
Get SDL. Otherwise you'll get
configure: error: *** SDL not found! Get SDL from www.libsdl.org.
If you already installed it, check it's in the path. If problem remains,
please send a mail to the address that appears in ./configure --version
indicating your platform, the version of configure script and the problem.
Failed to install SDL-0.6.5.1
Follow the instructions in (2) to get sdl/sdl2 libraries. (See instructions here Installing SDL on Windows for Haskell (GHC).)
The new version helm-0.7.1 requires sdl2, but there are other dependency issues with helm-0.7.1 as of writing. Download SDL from http://sourceforge.net/projects/msys2/files/REPOS/MINGW/x86_64/ (direct download link to newest version as of writing http://sourceforge.net/projects/msys2/files/REPOS/MINGW/x86_64/mingw-w64-x86_64-SDL-1.2.15-7-any.pkg.tar.xz.sig/download), unzip. "cabal install sdl" gives
* Missing (or bad) header file: SDL/SDL.h
* Missing C library: SDL
This problem can usually be solved by installing the system package that
provides this library (you may need the "-dev" version). If the library is
already installed but in a non-standard location then you can use the flags
--extra-include-dirs= and --extra-lib-dirs= to specify where it is.
so we specify where the dirs are (change the name depending on where you extracted sdl to)
cabal install sdl --extra-include-dirs=C:/PF/sdl\include --extra-lib-dirs=C:/sdl/lib
If you got SDL2 (http://libsdl.org/download-2.0.php) (for a newer version of Helm): there is a fatal bug that hasn't been fixed in the release version. (If you don't fix it, cabal install -v3 things which depends on it will give error
winapifamily.h: No such file or directory
("winapifamily.h: No such file or directory" when compiling SDL in Code::Blocks) Download https://hg.libsdl.org/SDL/raw-file/e217ed463f25/include/SDL_platform.h, replace the file in the include folder and in C:/PF/msys64/mingw64/include/SDL2.
Download gtk2hs from http://code.haskell.org/gtk2hs and run
the following
cd gtk2hs/tools
cabal install
cd ../glib
cabal install
cd ../gio
cabal install
cd ../pango
cabal install --ghc-pkg-options="--force"
(Maybe you have already installed glib and gio from before? I did this step because normal install of Pango caused an error for me (https://github.com/gtk2hs/gtk2hs/issues/110)
pango-0.13.1.0: include-dirs: /mingw64/include/freetype2 is a relative path
which makes no sense (as there is nothing for it to be relative to). You can
make paths relative to the package database itself by using ${pkgroot}. (use
--force to override)
Once the Helm developers get things updated you should be able to do "cabal install helm" but right now there seem to be dependency issues. For me, cabal automatically tries to install helm-0.4 (probably because 0.4 didn't give upper bounds on dependencies, while newer versions do. You could try "cabal unpack"ing and deleting the upper bounds...). Then
cabal unpack helm-0.4
Installing gives an error because "pure" got moved to Prelude. Open helm-0.4\src\FRP\Helm\Automaton.hs and change line 17:
import Prelude hiding (id, (.), pure)
Now
cabal install
Try to compile and run a program using Helm
(This is 0.4 - look on the website for a newer sample if you tried a newer Helm)
import FRP.Helm
import qualified FRP.Helm.Window as Window
render :: (Int, Int) -> Element
render (w, h) = collage w h [filled red $ rect (fromIntegral w) (fromIntegral h)]
main :: IO ()
main = run $ fmap (fmap render) Window.dimensions
If you get an error about a missing .dll (sdl.dll), find it in a bin/ folder and add the folder to your PATH (or copy it to somewhere on your path).
I was following a coq HelloWorld tutorial (code below), and couldn't get the program to compile. I followed the installation steps and installed opam install coq:io:system. My opam installation is at the default location ~/.opam. But still, I got an error about
Toplevel input, characters 53-67:
Error: The reference System.effects was not found in the current environment.
This is with either emacs/proofgeneral or coqide (8.4pl6, ubuntu 14.04). Does any one know how to fix the issue?
Here's the code which I copied into a file called hello_world.v and loaded into emacs/coqide:
Require Import Coq.Lists.List.
Require Import Io.All.
Require Import Io.System.All.
Require Import ListString.All.
Import ListNotations.
Import C.Notations.
(** The classic Hello World program. *)
Definition hello_world (argv : list LString.t) : C.t System.effects unit :=
System.log (LString.s "Hello world!").
-- Update ---
#gtzinos, I followed the readme in https://github.com/clarus/coq-hello-world. This time there was no complaint about System.effects, but there was a new error about Extraction.launch not found. I tried:
git clone https://github.com/clarus/coq-hello-world.git
cd coq-hello-world
./configure.sh && make
and got:
"coqc" -q -R src HelloWorld src/Main
File "/.../coq-hello-world/src/Main.v", line 32, characters 19-36:
Error: The reference Extraction.launch was not found in the current
environment.
I tried also to make in the extraction folder, but without success. Any pointers?
New versions of the coq:io and coq:io:system libraries were just released. Run:
opam update
opam upgrade
to make sure you have coq:io:system in version at least 2.3.0. Now Extraction.launch should be available. System.effects has been replaced by System.effect.
I have a mixed Python/C++ library with test files mixed in amongst source files in the same directories. The layout looks like
/home/irving/geode
geode
__init__.py
vector
__init__.py
test_vector.py
...
...
Unfortunately, the library is unusable in-place since it lacks .so extension modules. Question: Can I make py.test always use an installed version, even when run from /home/irving/geode or a subdirectory?
The test files have from __future__ import absolute_import, and run fine if executed directly as scripts. For example, if I do
cd geode/vector
./test_vector.py
which does import geode, it finds the installed version. However, if I run py.test in geode/vector, it finds the local copy of geode, and then dies.
I think you have two options:
run py.test --pyargs geode.vector.test_vector to make pytest interpretet the argument as an import path, deriving the file system path from it. This should run the test against the installed version.
move the tests out into a tests directory without an __init__.py file. This way you need to pip install -e . to work in-place or can do python setup.py install and the py.test tests to run tests against the installed version.
Please see Part 2 which list latest errors while installing module continued post.
Normally when I try to install XML::Xerces CPAN module using standard cpan> install XML::Xercers than I get following error message after some processing:
XML-Xerces-2.7.0-0/samples/SEnumVal.pl
...
XML-Xerces-2.7.0-0/postSource.pl
XML-Xerces-2.7.0-0/xerces-headers.txt
Removing previously used /home/adoshi/.cpan/build/XML-Xerces-2.7.0-0
CPAN.pm: Going to build J/JA/JASONS/XML-Xerces-2.7.0-0.tar.gz
WARNING
You have not defined any of the following environment variables:
XERCESCROOT
XERCES_LIB
XERCES_INCLUDE
These instruct me how to locate the Xerces header files, and the
Xerces dynamic library. If they are installed in a standard system
directory, I will located them without those variables.
However, if they have been installed in a non-standard location
(e.g. '/usr/include/xerces'), then I will need help. See the README
for more info.
Proceeding ...
WARNING
You have not defined any of the following environment variables:
XERCESCROOT
XERCES_CONFIG
Without these I cannot find the config.status file that was used to
build your Xerces-C library. Without that file, I may not be able to properly
build the C++ glue files that come with Xerces.pm.
Proceeding anyway ...
Couldn't find XercesVersion.hpp in your include directory at Makefile.PL line 1 88.
Running make test
Make had some problems, maybe interrupted? Won't test
Running make install
Make had some problems, maybe interrupted? Won't install
After Setting Enviornment Variables to /home/username/XML-Xerces-2.7.0-0/XML-Xerces-2.7.0-0/Xerces.pm, note here am not sure whether I should point my environment variable to Xerces.pm or Xerces.cpp or Xerces-extra.pm or Xerces.i, but for now am pointing environment variables to /home/username/XML-Xerces-2.7.0-0/XML-Xerces-2.7.0-0/Xerces.pm
After setting environment variables as mentioned and entering cpan>install XML::Xerces I get following message:
CPAN: Storable loaded ok
Going to read /home/username/.cpan/Metadata
Database was generated on Fri, 16 Oct 2009 18:27:06 GMT
Running install for module XML::Xerces
Running make for J/JA/JASONS/XML-Xerces-2.7.0-0.tar.gz
CPAN: Digest::MD5 loaded ok
CPAN: Compress::Zlib loaded ok
Checksum for /home/adoshi/.cpan/sources/authors/id/J/JA/JASONS/XML-Xerces-2.7.0-0.tar.gz ok
Scanning cache /home/adoshi/.cpan/build for sizes
XML-Xerces-2.7.0-0/
...
XML-Xerces-2.7.0-0/postSource.pl
XML-Xerces-2.7.0-0/xerces-headers.txt
Removing previously used /home/adoshi/.cpan/build/XML-Xerces-2.7.0-0
CPAN.pm: Going to build J/JA/JASONS/XML-Xerces-2.7.0-0.tar.gz
Using XERCES_LIB = /home/adoshi/XML-Xerces-2.7.0-0/XML-Xerces-2.7.0-0/Xerces.pm
using XERCES_CONFIG: /home/adoshi/XML-Xerces-2.7.0-0/XML-Xerces-2.7.0-0/Xerces.pm
- Found CXX =
- Found CXXFLAGS =
- Found LDFLAGS =
Couldn't find XercesVersion.hpp in your include directory at Makefile.PL line 188, <CONF> line 6823.
Running make test
Make had some problems, maybe interrupted? Won't test
Running make install
Make had some problems, maybe interrupted? Won't install
Note: I have tried downloading XML::Xercesand trying to again install it, both manually as well as using CPAN but am getting above mentioned error message.
What can be the possible reason and what can be suggested turn around to take care of this issue ?
Update: Even after building Xerces-C, XML::Xerces module is not building and am getting following error message.
[adoshi#upc01.dev XML-Xerces-2.7.0-0]$ perl Makefile.PL
Using XERCES_LIB = /adoshi/lib
Using XERCES_INCLUDE = /adoshi/include/xerces
WARNING
You have defined the XERCESCROOT variable, but the file:
XERCESCROOT/src/xercesc/config.status
does not seem to point to the config.status file that was used to
build your Xerces-C library. Without that file, I may not be able to
properly build the C++ glue files that come with Xerces.pm.
Proceeding anyway ...
Couldn't find XercesVersion.hpp in your include directory /adoshi/include/xerces at Makefile.PL line 188.
Update2Here is the error which am getting, it says there is somekind of version mismatch.
Using XERCES_LIB = /home/adoshi/XML-Parser/Parser2/xerces-c_2_8_0-hppa-hpux-acc_3(1)/xerces-c_2_8_0-hppa-hpux-acc_3/lib
Using XERCES_INCLUDE = /home/adoshi/XML-Parser/Parser2/xerces-c_2_8_0-hppa-hpux-acc_3(1)/xerces-c_2_8_0-hppa-hpux-acc_3/include
WARNING
You have defined the XERCESCROOT variable, but the file:
XERCESCROOT/src/xercesc/config.status
does not seem to point to the config.status file that was used to
build your Xerces-C library. Without that file, I may not be able to
properly build the C++ glue files that come with Xerces.pm.
Proceeding anyway ...
Using Xerces-C version info from /home/adoshi/XML-Parser/Parser2/xerces-c_2_8_0-hppa-hpux-acc_3(1)/xerces-c_2_8_0-hppa-hpux-acc_3/include/xercesc/util/XercesVersion.hpp
*** Version Mismatch ***
You are attempt to build XML::Xerces-2.7.0-0 using Xerces-C-2.8.0,
this will most likely fail, so I am aborting.
You must use Xerces-C-2.7.0
Here's a general rule: any environment variable that is named something like "ROOT" is asking for a directory, not a file.
However, it does not appear that you have installed the Xerces library, which is necessary before you install the perl module. I'll quote some portions of the output you provided, as the hint you missed as to what to do next:
"...These instruct me how to locate the Xerces header files, and the Xerces dynamic library..."
"Without these I cannot find the config.status file that was used to
build your Xerces-C library"
So, did you install Xerces-C? You'll have much better results installing the Perl module after that.
Did you try using the PPM to install XML::Xerces?