Is there a way to publish multiplatform wheels on PyPI with same version number? - raspberry-pi

I'm trying to deploy a Cython wrapped C++ module to PyPI as wheel. The goal is to make I2C hardware modules work with Python on any Raspberry Pi. So far I compiled the code and I know it works if I just copy compiled module from Pi 3B running Buster to Pi Zero W running Stretch, but when I deploy the wheel to test.pypi.org from Buster and try to install it on Stretch I get:
Could not find a version that satisfies the requirement pyiArduinoI2Crelay (from versions: )
No matching distribution found for pyiArduinoI2Crelay
If I merely rename the wheel from *_armv7l.whl to *_armv6l.whl
module downloads and works on Pi Zero. But Pi 3B downloads previous version from PyPI (and i used --no-cache-dir and rm -r .cache/pip/)
If I do sdist and upload it complains about not having Cython installed upon installation of the module, although I know for a fact that it is installed, because it's the same Pi I compiled wheels before. (setup_requires and install_requires dont's seem to work)
So far none of these helped:
https://www.python.org/dev/peps/pep-0425/#platform-tag
https://packaging.python.org/guides/distributing-packages-using-setuptools/#platform-wheels
https://wheel.readthedocs.io/en/stable/user_guide.html#building-wheels
Prepare C-based Cython package to publish on pypi
Here's the link to the project:
https://github.com/tremaru/pyiArduinoI2Crelay
Here's the setup.py:
from setuptools import setup
from distutils.extension import Extension
from Cython.Build import cythonize
from Cython.Distutils import build_ext
def readme():
with open('README.md') as readme:
return readme.read()
setup(name='pyiArduinoI2Crelay',
version='1.6.4.dev8',
description='iarduino.ru module for Raspberry Pi',
long_description=readme(),
classifiers=[
'Programming Language :: Python :: 3',
],
url='http://github.com/tremaru/pyiArduinoI2Crelay',
author='iarduino.ru',
author_email='shop#iarduino.ru',
license='MIT',
package=['pyiArduinoI2Crelay'],
ext_modules = [Extension(
name="pyiArduinoI2Crelay",
sources=["pyiArduinoI2Crelay/pyiArduinoI2Crelay.cpp"])],
include_package_data=True,
python_requires='>=3',
setup_requires=['Cython'],
install_requires=['Cython'],
cmdclass = {
"build_ext": build_ext
}
)
I want to be able to publish one version of the module for all Raspberries. So, is there a way to pack multiple .so's to one wheel? Or maybe some kind of manylinux1 tag for arm architecture?

Turns out I'm a knucklehead. I didn't need any Cython stuff in my setup after generating cpp file. All I needed to do is remove Cython stuff and remove cmdclass = { "build_ext": build_ext }. And then in publishes and builds as source distribution no problem.

Related

python 3.7.6 pip setup.py test with ContextualVersionConflict

my setup.py is like below:
from setuptools import find_packages, setup
tests_require = ['pytest-env', 'pytest-mock', 'pytest-cov', 'pytest-xdist', 'pytest', 'mock', 'moto<=1.3.10']
setup(
name='repo_name',
version='0.1.0',
description='repo_name',
keywords=['?'],
packages=find_packages('src', exclude=['tests', 'venv']),
package_dir={'': 'src'},
package_data={'': ['schema/*.yaml']},
install_requires=[
'boto3<=1.10.19',
'requests<=2.22.0',
'jsonschema<=3.0.1',
'objectpath<=0.6.1',
'pyyaml<=5.1.1',
'sqlalchemy<=1.3.5',
'psycopg2-binary<=2.8.3',
'auth-client<=1.0.23', # internal package
'policy_client<=1.0.9', # internal package
'audit-client<=1.1.20', # internal package
'flask<=1.1.1',
'click<=7.0',
'Werkzeug<=0.15.5',
'itsdangerous<=1.1.0',
'Jinja2<=2.10.1',
'MarkupSafe<=1.1.1',
'structlog<=19.2.0',
'python-rapidjson<=0.9.1'
],
setup_requires=['pytest-runner'],
tests_require=tests_require,
extras_require={'test': tests_require},
include_package_data=True,
zip_safe=False
)
When I run python setup.py test, it will always reinstall the most of the packages that I already installed into .eggs folder, which I understand. While it's installing the package with different versions that I provided in the install_requires section, which results in below error:
pkg_resources.ContextualVersionConflict: (urllib3 1.25.10 (/Users/***/Desktop/repo/.eggs/urllib3-1.25.10-py3.7.egg), Requirement.parse('urllib3<1.25,>=1.21.1'), {'requests'})
I have to pin the requests version to 2.22.0 which requires urllib3 version no larger than 1.25, which caused my issue. So, is there anyway I can pin the urllib3 version in somewhere so when it runs, it will only install the provided version? I have tried to add the urllib3 version in install_requires, tests_require and in requirements.txt, but no luck so far.
Probably not what you want to hear but... setup_requires and tests_require are deprecated. Although they (probably) still work (mostly) fine, you might want to look for alternative solutions. As far as I know the most common alternative solutions are to use Tox instead of tests_require and PEP 517 to replace setup_requires.
In your case, it seems that your usage of setup_requires is only a consequence of your usage of tests_require. So by switching to something like Tox you get rid of both tests_require and setup_requires.
I got it resolve to simply replace python setup.py test to be pip instll -e . [test], and then pytest directly.
It will still install all the tests_require packages and then run pytest directly. Instead of going through all the list packages and find the best match versions for all packages.
UPDATES:
The real problem is I did not remove the old .eggs/ and venv/ folder when I made the packages version change. So the solution is updating the requests version to be 2.21.0 in the setup.py file, then remove the .eggs/ and venv/ folder and rerun everything.

Error: No module named 'PyQt5.sip' - When installing eric6-19.03

I have noticed a number of similar questions on stackoverflow, but none that really helped me with my current challenge. I have been attempting to install eric6 on my Raspberry Pi setup. And then I encountered the following messages:
Checking dependencies
Python Version: 3.6.8
PyQt5 could not be detected.
Error: No module named 'PyQt5.sip'
Shall 'PyQt5' be installed using pip? (Y/n) Y
Looking in indexes: https://pypi.org/simple, https://www.piwheels.org/simple
Requirement already satisfied: PyQt5 in /usr/local/lib/python3.6/site-packages (5.12)
Sorry, please install PyQt5.
Error: No module named 'PyQt5.sip'
And when I attempt to use the pip list to look at what has been installed (as I previously already managed to make and install the PyQt5 from source code):
Package Version
---------- -------
pip 19.0.3
PyQt5 5.12
setuptools 40.6.2
sip 4.19.14
So, how do I overcome the issue that I am currently facing? By the way, I am using the following command when attempting to install eric6:
python3 install.py
Thought you might need to know that if it helps.
Python 3.6 isn't available in any Raspbian distribution, so I assume you installed this yourself. You can't install an out-of-distribution Python version and expect distribution Python packages to work. You have PyQt5 installed, yes, but it's the version of PyQt built for Python 3.5 (I'm assuming you're on Raspbian Stretch).
Sometimes you can install Python libraries with pip, and in theory that could work to install the correct version of PyQt for your version of Python, but PyQt only provides wheels for x86 on PyPI, not for Arm, so PyQt is not pip installable on the Pi. The only way to get PyQt on Raspberry Pi is via apt, where your choices are Jessie (old stable - with Python 3.4), Stretch (current stable - with Python 3.5), and Buster (unstable/testing - with Python 3.7). If there's a need for Python >3.5, your best bet is to try upgrading to Buster, but bear in mind it's not ready for release yet.

How to run swift and libDispatch on ARM (Raspberry Pi)

The aim is to run swift and libDispatch on a Raspberry Pi 3. I’m using Ubuntu Core 16.04
Where I’m at: I’ve tried to get SR-397 (https://bugs.swift.org/browse/SR-397) to compile - it is a minimal Swift libDispatch example.
The projects for this SR are here: https://github.com/sheffler/gcd4 and here: https://github.com/sheffler/CDispatch
When I try to compile the gcd4 project, I get this error: “error: Empty manifest file is not supported anymore. Use swift package init to autogenerate.”
I assumed the error message was due to the empty Package.swift file, so then I copied the Sheffler CDispatch project to https://github.com/tree700/CDispatch and added a Package.swift file. I still get the same error and haven't been able to get past it.
By way of background, I’m using a precompiled download of Swift 3.1.1 from https://github.com/uraimo/buildSwiftOnARM, and have installed Ubuntu libdispatch0 and libdispatch-dev packages and dependencies on my system. The swift compiler and Swift Package Manager are working fine,
I’d love to get this going, I use Dispatch on OS X, and it makes concurrency a breeze. Thanks
The answer was simple... the extra information required is in the associated blog. To obtain and install Swift 3.1.1 go to https://www.uraimo.com/2017/09/06/A-small-update-on-Swift-for-raspberry-pi-zero-1-2-3/
Which says to do this...
sudo apt-get install git cmake ninja-build clang-3.8 python uuid-dev libicu-dev icu-devtools libbsd-dev libedit-dev libxml2-dev libsqlite3-dev swig libpython-dev libncurses5-dev pkg-config libblocksruntime-dev libcurl4-openssl-dev autoconf libtool systemtap-sdt-dev
sudo update-alternatives --install /usr/bin/clang++ clang++ /usr/bin/clang++-3.8 100
sudo update-alternatives --install /usr/bin/clang clang /usr/bin/clang-3.8 100
After that, there is the the usual process of decompressing the compressed project file and adding the usr/bin directory into one's path (or run swift, swift etc from inside usr/bin)
Note: I also had to install the package libpthread-workqueue-dev. If running on Raspbian, requires an upgrade to Stretch .
———
By the way, I posted the SO question when using a Raspberry 3. In fact, I was on an early Pi model B when I got this working. The Pi 3 will soon have Swift also. Here is a simple Swift program to test libdispatch:-
import Foundation
import Dispatch
let queue = DispatchQueue(label: "queueName")
let workItem = DispatchWorkItem(qos: .userInitiated, flags: .assignCurrentContext) {
print("Hello")
}
queue.async(execute: workItem)
DispatchQueue.main.async {
print("Hello2")
}
dispatchMain()

How to deploy a package in another operating system

I have a project written in C++ 14 (compiled in my machine with GCC 6.2) and my machine run on Fedora 26. Now I need to deploy into customer's server that run RedHat 6, and GCC 4.8. I can not change or install new things in this server (except my package). But this server doesn't contain 2 package that required to my project is numactl and hwloc.
If I build these library as static library and import into my project (in Fedora 26), then package these library with my project into a RPM package, will it run on the server.
Is it the best way? I never do this work before.
I'm sorry for my English.
You need to build your package for RedHat 6. Create source rpm (src.rpm) and run:
mock -r epel-6-x86_64 your-package.src.rpm
This way you will get package which can be installed on RHEL6.

How to have PyPI package install header files for C extension with distutils/setuptools?

We have a package (rebound) up on PyPI that includes a C extension . The relevant part of the setup.py file looks like this (simplified):
libreboundmodule = Extension('librebound',
sources = [ 'src/rebound.c'],
include_dirs = ['src'],)
Additional libraries need access to rebound.h, but when one runs
pip install rebound
it doesn't install rebound.h anywhere. How can we get distutils/setuptools to install rebound.h somewhere along with all the python modules? We're hoping that we can have pip install rebound do all the work so the user doesn't have to run any additional commands.