Boost.Python __init__() should return None, not 'NoneType' - boost-python

I have a whole bunch of working C++ code that I want to write Python bindings for. I'm trying to use Boost.Python since it seems to be the easiest way to get this working, but it isn't cooperating. Here's part of the code for the extension module I'm trying to build:
BOOST_PYTHON_MODULE(libpcap_ext) {
using namespace boost::python;
class_<PacketEngine>("PacketEngine")
.def("getAvailableDevices", &PacketEngine_getAvailableDevices);
}
Bjam seems to be a pain and refuses to recognize my Pythonpath or allow me to link with libpcap, so I'm using CMake. Here's my CMakeLists file, which can import and build everything just fine (outputs libpcap.so as expected):
CMAKE_MINIMUM_REQUIRED(VERSION 2.8)
IF(NOT CMAKE_BUILD_TYPE)
SET(CMAKE_BUILD_TYPE "DEBUG")
#SET(CMAKE_BUILD_TYPE "RELEASE")
#SET(CMAKE_BUILD_TYPE "RELWITHDEBINFO")
#SET(CMAKE_BUILD_TYPE "MINSIZEREL")
ENDIF()
FIND_PACKAGE(Boost 1.55.0)
find_package(PythonLibs REQUIRED)
IF(Boost_FOUND)
INCLUDE_DIRECTORIES("${Boost_INCLUDE_DIRS}" "${PYTHON_INCLUDE_DIRS}")
SET(Boost_USE_STATIC_LIBS OFF)
SET(Boost_USE_MULTITHREADED ON)
SET(Boost_USE_STATIC_RUNTIME OFF)
FIND_PACKAGE(Boost 1.55.0 COMPONENTS python)
ADD_LIBRARY(pcap_ext MODULE PacketWarrior/pcap_ext.cc PacketWarrior/PacketEngine.h PacketWarrior/PacketEngine.cc PacketWarrior/Packet.h PacketWarrior/Packet.cc)
TARGET_LINK_LIBRARIES(pcap_ext pcap)
TARGET_LINK_LIBRARIES(pcap_ext ${Boost_LIBRARIES} ${PYTHON_LIBRARIES})
ELSEIF(NOT Boost_FOUND)
MESSAGE(FATAL_ERROR "Unable to find correct Boost version. Did you set BOOST_ROOT?")
ENDIF()
ADD_DEFINITIONS("-Wall")
And my pcap.py file that attempts to utilize the module:
import libpcap_ext
engine = libpcap_ext.PacketEngine()
print engine.getAvailableDevices()
But whenever I try to run the module, I get the following error:
Traceback (most recent call last):
File "../pcap.py", line 2, in <module>
engine = libpcap_ext.PacketEngine()
TypeError: __init__() should return None, not 'NoneType
I'm assuming it's because Boost.Python is trying to use Python 3 and my system default is Python 2.7.3. I've tried changing my user-config.jam file (in my boost_1_55_0 directory) to point to Python 2.7 and tried building:
# Configure specific Python version.
# using python : 2.7 : /usr/bin/python2.7 : /usr/include/python2.7 : /usr/lib ;
Boost.Python's installation instructions [0] seem to fail for me when I try to build quickstart with bjam (lots of warnings), so I tried following the Boost Getting Started instructions [1] to build a Python header binary, which is I think what is causing this problem. Any recommendations as to how to fix this would be amazing, I've spent hours on this.

This error is probably due to linking against the wrong Python library. Make sure your extension as well as the Boost Python library are linked against the Python installation you are using to import the module.
On Linux you can check against which libraries you've linked with ldd. On OS X otool -L does the same thing. So, for example
otool -L libpcap_ext.so
otool -L /path/to/libboost_python-mt.dylib
should list the Python library they are linked against.
With CMake you can use the variable PYTHON_LIBRARY to change which Python library is used. As an example, on the command line you can set it with
cmake -DPYTHON_LIBRARY="/path/to/libpython2.7.dylib" source_dir
Lastly, on OS X a quick and dirty way (i.e. without recompiling) to change the dynamically linked libraries is install_name_tool -change.

Related

Distributing pybind11 extension linked to third party libraries

I'm working on a pybind11 extension written in C++ but I'm having a hard time understanding how should it be distributed.
The project links to a number of third party libraries (e.g. libpng, glew etc.).
The project builds fine with CMAKE and it generates a .so file. Now I am not sure what is the right way of installing this extension. The extension seems to work, as if I try copy the file into the python lib directories it is picked up (I can import it, and it works correctly). However, this is clearly not the way to go I think.
I also tried the setuptools route (from https://pybind11.readthedocs.io/en/stable/compiling.html) by creating a setup.py files like this:
import sys
# Available at setup time due to pyproject.toml
from pybind11 import get_cmake_dir
from pybind11.setup_helpers import Pybind11Extension, build_ext
from setuptools import setup
from glob import glob
files = sorted(glob("*.cpp"))
__version__ = "0.0.1"
ext_modules = [
Pybind11Extension("mylib",
files,
# Example: passing in the version to the compiled code
define_macros = [('VERSION_INFO', __version__)],
),
]
setup(
name="mylib",
version=__version__,
author="fab",
author_email="fab#fab",
url="https://github.com/pybind/python_example",
description="mylib",
long_description="",
ext_modules=ext_modules,
extras_require={"test": "pytest"},
cmdclass={"build_ext": build_ext},
zip_safe=False,
python_requires=">=3.7",
)
and now I can build the extension by simply calling
pip3 install
however it looks like all the links are broken because whenever I try importing the extension in Python I get linkage errors, as if setuptools does not link correctly the extension with the 3rd party libs. For instance errors in linking with libpng as in:
>>> import mylib
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ImportError: /home/fabrizio/.local/lib/python3.8/site-packages/mylib.cpython-38-x86_64-linux-gnu.so: undefined symbol: png_sig_cmp
However I have no clue how to add this link info to setuptools, and don't even know if that's possible (it should be the setuptools equivalent of CMAKE's target_link_libraries).
I am really at a loss after weeks of reading documentation, forum threads and failed attempts. If anyone is able to point me in the right way or to clear some of the fog it would be really appreciated!
Thanks!
Fab
/home/fabrizio/.local/lib/python3.8/site-packages/mylib.cpython-38-x86_64-linux-gnu.so: undefined symbol: png_sig_cmp
This line pretty much says it clearly. Your local shared object file .so can't find the libpng.so against which it is linked.
You can confirm this by running:
ldd /home/fabrizio/.local/lib/python3.8/site-packages/mylib.cpython-38-x86_64-linux-gnu.so
There is no equivalent of target_link_libraries() in setuptools. Because that wouldn't make any sense. The library is already built and you've already linked it. This is your system more or less telling you that it can't find the libraries it needs. And those most likely need to be installed.
This is also one of the reasons why Linux distributions provide their own package managers and why you should use the developer packages provided by said distributions.
So how do you fix this? Well your .so file needs to find the other .so files against which you linked to understand how this works I will refer you to this link.
My main guess is based on the fact that when you manually copy the files it works - That during the build process you probably specify the rpath to a local directory. Hence what you most likely need to do is specify to your setuptools that it needs to copy those files when installing.

or-tools: build examples on vs2022

I've downloaded the binaries: or-tools_VisualStudio2022-64bit_v9.3.10497
I'm using vs2022 on win10. My shell has cygwin in the path if it's related.
I ran
%comspec% /k "C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Auxiliary\Build\vcvars64.bat"
cl.exe is in the path, and which.exe finds it.
I ran make test_cc, but it complained
the cl command was not found in your PATH
exit 127
make: *** [Makefile:271: test_cc] Error 127
The var CXX_BIN was empty even though which cl returned the correct path. I set it manually to cl.
Then, there was a complaint about echo and a newline, which I commented out. Then, it couldn't find md, so I created manually md objs.
A few of the examples were built, but then it stopped with another error. For now, I just got what I want:
make run SOURCE=examples/cpp/solve.cc
but probably there was an easier way to get it?
I tried to build it from the source using cmake. Doesn't work off-the-shelf as well:
Build abseil-cpp: OFF
...
CMake Error at C:/prj-external-libs/vcpkg/scripts/buildsystems/vcpkg.cmake:824 (_find_package):
By not providing "Findabsl.cmake" in CMAKE_MODULE_PATH this project has
asked CMake to find a package configuration file provided by "absl", but
CMake did not find one.
Could not find a package configuration file provided by "absl" with any of
the following names:
abslConfig.cmake
absl-config.cmake
Add the installation prefix of "absl" to CMAKE_PREFIX_PATH or set
"absl_DIR" to a directory containing one of the above files. If "absl"
provides a separate development package or SDK, be sure it has been
installed.
Call Stack (most recent call first):
cmake/deps.cmake:33 (find_package)
CMakeLists.txt:304 (include)
If finds gurobi95.dll, but it can't find the function GRBtunemodeladv.
On failure, solve.exe crashes with (unknown) names in the stack trace. Need to add debug symbols and graceful error handling.
cmake looks more promising, and I was missing dependencies. Should give it a flag -DBUILD_DEPS:BOOL=ON.
OR-Tools depends on few external dependencies so CMake build will try to find them using the idiomatic find_package() => your distro/env(vcpkg ?) must provide them, just regular CMake stuff here.
ref: https://cmake.org/cmake/help/latest/command/find_package.html
note: we provide few findFoo.cmake here https://github.com/google/or-tools/tree/main/cmake
We also provide a meta option to build statically all our dependencies, simply pass -DBUILD_DEPS=ON cmake option at configure time.
You can also build only some of them, please take a look at
https://github.com/google/or-tools/tree/main/cmake#dependencies
Concerning Gurobi and GRBtunemodeladv symbol, this one has been removed by last version of Gurobi so we fix it in v9.4/main/stable branch...
see: https://github.com/google/or-tools/commit/d6e0feb8ae96368523deb99fe4318d32e80e8145

Why can't I add a package (module) I created in Julia?

I am having trouble installing a module I created in Julia. I am running the Julia plugin under Visual Studio Code. If I run the file below with Ctrl+F5 I get a message
ERROR: LoadError: ArgumentError: Package Utils not found in current path:
- Run `import Pkg; Pkg.add("Utils")` to install the Utils package.
This is the file:
module demo
using Utils
greet() = print("Hello World!")
end # module
If I follow the advice on the error message I get another error message:
ERROR: LoadError: The following package names could not be resolved:
* Utils (not found in project, manifest or registry)
I also tried inserting this line:
import Pkg; Pkg.add(path="C:/Dropbox/Code/Julia/demo/src/Utils.jl")
and got this message (although the path definitely exists):
ERROR: LoadError: Path `C:/Dropbox/Code/Julia/demo/src/Utils.jl` does not exist.
The files demo.jl and Utils.jl are in C:\Dropbox\Code\Julia\demo\src\ and the demo project has been activated as can be seen in the REPL. The OS is Windows 10 Pro.
Any help will be greatly appreciated. Lots of time wasted trying to make this work.
Module and packages are not the same things. In short, packages are modules plus a set of metadata that make it easy for the package to be found and interact well with other packages. See here for a tutorial to write Julia packages:
https://syl1.gitbook.io/julia-language-a-concise-tutorial/language-core/11-developing-julia-packages
In your case, if you want to load a local module, just type include("fileWhereThereIsTheModule.jl") followed by a using Main.MyModule or using .MyModule. Note the dot... without it, Julia would indeed look for a package and to let it find your Demo or Util module you would have to either change an environmental variable or place your module file in certain predefined folders. Using include followed by the "absolute or relative position" of the module you don't have to do either.

How can I get Compass to work in Visual Studio via NuGet?

My developer friend who has the luxury of developing in a non-Windows environment has been raving about Compass. I finally decided I wanted to give it a try. I'm tired of trying to keep up with all of the intricacies of cross-browser CSS.
So, I found it on NuGet, and installed it.
I installs to my solutions root directory in the packages directory:
$(SolutionDir)packages\Ruby.Compass.0.12.2.3\
It comes with a Readme that states the following message:
Ruby Compass v. 0.12.2
Compass is installed in its own NuGet package dir, and available by
'compass' command in "packages\Ruby.Compass.0.12.2.3" folder.
To compile Compass files during build, add the next line to the
project pre-build events:
"$(SolutionDir)packages\Ruby.Compass.0.12.2.3\compass" compile
"$(ProjectDir)."
So, I placed the line in my pre-build events, saved, and tried to build my project. However, I get an error as follows:
The command
""$(SolutionDir)packages\Ruby.Compass.0.12.2.3\compass" compile "$(ProjectDir)."" exited with code 1.
Notice: It actually shows the full path to the ProjectDir and SolutionDir as it's supposed too in the error message. I replaced them with the tokens to keep the project name unanimous.
Let me mention that I tried variations of the suggestion pre-build line:
"$(SolutionDir)packages\Ruby.Compass.0.12.2.3\compass" compile "$(ProjectDir)"
"$(SolutionDir)packages\Ruby.Compass.0.12.2.3\compass" compile "$(ProjectDir)css"
"$(SolutionDir)packages\Ruby.Compass.0.12.2.3\compass" compile "$(ProjectDir)css\test.scss"
The first one just removed that trailing .. The second one pointed it to the directory where all my css files are stored. The third one pointed it to the exact file I was trying to compile was located.
I opened up compass.cmd which is the file it is calling, and it looks like the following:
#echo off
"%~dp0ruby\bin\compass" %*
I'm assuming this calls the compass file in the ruby/bin folder, which looks like this:
#!C:/downloads/ruby-2.0.0-p247-x64-mingw32/ruby-2.0.0-p247-x64-mingw32/bin/ruby.exe
#
# This file was generated by RubyGems.
#
# The application 'compass' is installed as part of a gem, and
# this file is here to facilitate running it.
#
require 'rubygems'
version = ">= 0"
if ARGV.first
str = ARGV.first
str = str.dup.force_encoding("BINARY") if str.respond_to? :force_encoding
if str =~ /\A_(.*)_\z/
version = $1
ARGV.shift
end
end
gem 'compass', version
load Gem.bin_path('compass', 'compass', version)
From there, I'm not sure what is going on. I'm not a Ruby person.
Is there an issue that I'm overlooking here?
Has anyone else been able to install Ruby.Compass via NuGet?
How can I get this working in Visual Studio without having to fight with Ruby?
From: http://codewith.us/automating-css-generation-in-visual-studio-using-sasscompass/
"Note that, if there are issues with your SCSS files, you will receive some variation of the error below.
Error 36 The command "del "C:Projectspubliccss*.css" /S
compass compile "C:Projectspublic" --force" exited with code 1.
Open your Output window (click View -> Output or press Ctrl+W, O), and select “Build” in the “Show output from:” menu. Scroll up until you find your command in the log and you should get a little more insight into what portion of the command failed."

How do I install XML::Xerces?

Please see Part 2 which list latest errors while installing module continued post.
Normally when I try to install XML::Xerces CPAN module using standard cpan> install XML::Xercers than I get following error message after some processing:
XML-Xerces-2.7.0-0/samples/SEnumVal.pl
...
XML-Xerces-2.7.0-0/postSource.pl
XML-Xerces-2.7.0-0/xerces-headers.txt
Removing previously used /home/adoshi/.cpan/build/XML-Xerces-2.7.0-0
CPAN.pm: Going to build J/JA/JASONS/XML-Xerces-2.7.0-0.tar.gz
WARNING
You have not defined any of the following environment variables:
XERCESCROOT
XERCES_LIB
XERCES_INCLUDE
These instruct me how to locate the Xerces header files, and the
Xerces dynamic library. If they are installed in a standard system
directory, I will located them without those variables.
However, if they have been installed in a non-standard location
(e.g. '/usr/include/xerces'), then I will need help. See the README
for more info.
Proceeding ...
WARNING
You have not defined any of the following environment variables:
XERCESCROOT
XERCES_CONFIG
Without these I cannot find the config.status file that was used to
build your Xerces-C library. Without that file, I may not be able to properly
build the C++ glue files that come with Xerces.pm.
Proceeding anyway ...
Couldn't find XercesVersion.hpp in your include directory at Makefile.PL line 1 88.
Running make test
Make had some problems, maybe interrupted? Won't test
Running make install
Make had some problems, maybe interrupted? Won't install
After Setting Enviornment Variables to /home/username/XML-Xerces-2.7.0-0/XML-Xerces-2.7.0-0/Xerces.pm, note here am not sure whether I should point my environment variable to Xerces.pm or Xerces.cpp or Xerces-extra.pm or Xerces.i, but for now am pointing environment variables to /home/username/XML-Xerces-2.7.0-0/XML-Xerces-2.7.0-0/Xerces.pm
After setting environment variables as mentioned and entering cpan>install XML::Xerces I get following message:
CPAN: Storable loaded ok
Going to read /home/username/.cpan/Metadata
Database was generated on Fri, 16 Oct 2009 18:27:06 GMT
Running install for module XML::Xerces
Running make for J/JA/JASONS/XML-Xerces-2.7.0-0.tar.gz
CPAN: Digest::MD5 loaded ok
CPAN: Compress::Zlib loaded ok
Checksum for /home/adoshi/.cpan/sources/authors/id/J/JA/JASONS/XML-Xerces-2.7.0-0.tar.gz ok
Scanning cache /home/adoshi/.cpan/build for sizes
XML-Xerces-2.7.0-0/
...
XML-Xerces-2.7.0-0/postSource.pl
XML-Xerces-2.7.0-0/xerces-headers.txt
Removing previously used /home/adoshi/.cpan/build/XML-Xerces-2.7.0-0
CPAN.pm: Going to build J/JA/JASONS/XML-Xerces-2.7.0-0.tar.gz
Using XERCES_LIB = /home/adoshi/XML-Xerces-2.7.0-0/XML-Xerces-2.7.0-0/Xerces.pm
using XERCES_CONFIG: /home/adoshi/XML-Xerces-2.7.0-0/XML-Xerces-2.7.0-0/Xerces.pm
- Found CXX =
- Found CXXFLAGS =
- Found LDFLAGS =
Couldn't find XercesVersion.hpp in your include directory at Makefile.PL line 188, <CONF> line 6823.
Running make test
Make had some problems, maybe interrupted? Won't test
Running make install
Make had some problems, maybe interrupted? Won't install
Note: I have tried downloading XML::Xercesand trying to again install it, both manually as well as using CPAN but am getting above mentioned error message.
What can be the possible reason and what can be suggested turn around to take care of this issue ?
Update: Even after building Xerces-C, XML::Xerces module is not building and am getting following error message.
[adoshi#upc01.dev XML-Xerces-2.7.0-0]$ perl Makefile.PL
Using XERCES_LIB = /adoshi/lib
Using XERCES_INCLUDE = /adoshi/include/xerces
WARNING
You have defined the XERCESCROOT variable, but the file:
XERCESCROOT/src/xercesc/config.status
does not seem to point to the config.status file that was used to
build your Xerces-C library. Without that file, I may not be able to
properly build the C++ glue files that come with Xerces.pm.
Proceeding anyway ...
Couldn't find XercesVersion.hpp in your include directory /adoshi/include/xerces at Makefile.PL line 188.
Update2Here is the error which am getting, it says there is somekind of version mismatch.
Using XERCES_LIB = /home/adoshi/XML-Parser/Parser2/xerces-c_2_8_0-hppa-hpux-acc_3(1)/xerces-c_2_8_0-hppa-hpux-acc_3/lib
Using XERCES_INCLUDE = /home/adoshi/XML-Parser/Parser2/xerces-c_2_8_0-hppa-hpux-acc_3(1)/xerces-c_2_8_0-hppa-hpux-acc_3/include
WARNING
You have defined the XERCESCROOT variable, but the file:
XERCESCROOT/src/xercesc/config.status
does not seem to point to the config.status file that was used to
build your Xerces-C library. Without that file, I may not be able to
properly build the C++ glue files that come with Xerces.pm.
Proceeding anyway ...
Using Xerces-C version info from /home/adoshi/XML-Parser/Parser2/xerces-c_2_8_0-hppa-hpux-acc_3(1)/xerces-c_2_8_0-hppa-hpux-acc_3/include/xercesc/util/XercesVersion.hpp
*** Version Mismatch ***
You are attempt to build XML::Xerces-2.7.0-0 using Xerces-C-2.8.0,
this will most likely fail, so I am aborting.
You must use Xerces-C-2.7.0
Here's a general rule: any environment variable that is named something like "ROOT" is asking for a directory, not a file.
However, it does not appear that you have installed the Xerces library, which is necessary before you install the perl module. I'll quote some portions of the output you provided, as the hint you missed as to what to do next:
"...These instruct me how to locate the Xerces header files, and the Xerces dynamic library..."
"Without these I cannot find the config.status file that was used to
build your Xerces-C library"
So, did you install Xerces-C? You'll have much better results installing the Perl module after that.
Did you try using the PPM to install XML::Xerces?