Building Python3 Twisted module in yocto project - yocto

When trying to build the python twisted module using the recipe.
I have the runtime dependencies python recipes.
I don't have an idea about the error to add a native recipe since I don't have such an incremental-native recipe file.
inherit pypi setuptools3
LICENSE = "MIT"
LIC_FILES_CHKSUM = "file://LICENSE;md5=27ac6d9597237e7e76282edd7a40cd68"
SRC_URI[sha256sum] = "d72c55b5d56e176563b91d11952d13b01af8725c623e498db5507b6614fc1e10"
PYPI_PACKAGE = "Twisted"
PYPI_PACKAGE_EXT = "tar.bz2"
RDEPENDS_${PN} = " ${PYTHON_PN}-zope-interface ${PYTHON_PN}-incremental ${PYTHON_PN}-constantly \
${PYTHON_PN}-hyperlink ${PYTHON_PN}-pyhamcrest ${PYTHON_PN}-attrs "
Getting error, add native recipes to DEPENDS.
ERROR: python3-twisted-20.3.0-r0 do_compile: Execution of '/home/pravin/yocto-project/poky/build/tmp/work/armv7vet2hf-neon-poky-linux-gnueabi/python3-twisted/20.3.0-r0/temp/run.do_compile.179353' failed with exit code 1:
WARNING: The pip package is not available, falling back to EasyInstall for handling setup_requires/test_requires; this is deprecated and will be removed in a future version.
ERROR: Do not try to fetch `incremental>=16.10.1' for building. Please add its native recipe to DEPENDS.
Traceback (most recent call last):
File "/home/pravin/yocto-project/poky/build/tmp/work/armv7vet2hf-neon-poky-linux-gnueabi/python3-twisted/20.3.0-r0/Twisted-20.3.0/setup.py", line 20, in <module>
setuptools.setup(**_setup["getSetupArgs"]())
File "/home/pravin/yocto-project/poky/build/tmp/work/armv7vet2hf-neon-poky-linux-gnueabi/python3-twisted/20.3.0-r0/recipe-sysroot-native/usr/lib/python3.8/site-packages/setuptools/__init__.py", line 162, in setup
_install_setup_requires(attrs)
File "/home/pravin/yocto-project/poky/build/tmp/work/armv7vet2hf-neon-poky-linux-gnueabi/python3-twisted/20.3.0-r0/recipe-sysroot-native/usr/lib/python3.8/site-packages/setuptools/__init__.py", line 157, in _install_setup_requires
dist.fetch_build_eggs(dist.setup_requires)
File "/home/pravin/yocto-project/poky/build/tmp/work/armv7vet2hf-neon-poky-linux-gnueabi/python3-twisted/20.3.0-r0/recipe-sysroot-native/usr/lib/python3.8/site-packages/setuptools/dist.py", line 699, in fetch_build_eggs
resolved_dists = pkg_resources.working_set.resolve(
File "/home/pravin/yocto-project/poky/build/tmp/work/armv7vet2hf-neon-poky-linux-gnueabi/python3-twisted/20.3.0-r0/recipe-sysroot-native/usr/lib/python3.8/site-packages/pkg_resources/__init__.py", line 785, in resolve
raise DistributionNotFound(req, requirers)
pkg_resources.DistributionNotFound: The 'incremental>=16.10.1' distribution was not found and is required by the application
WARNING: exit code 1 from a shell command.```

Related

How Can I Solve "Failed To Source Bitbake" With Xilinx Petalinux SDK

I am running Ubuntu 16.04 with Xilinx Petalinux 2018.03 SDK. After a number of successful compilations I am now facing this error
$ petalinux-build
[INFO] building project
[INFO] sourcing bitbake
ERROR: Failed to source bitbake
ERROR: Failed to build project
How can I solved this issue?
Another reason to get the errors "ERROR: Failed to source bitbake" as well as "ERROR: Failed to build project" is a possible upgrade of Python on the build machine. The Petalinux SDK requires python v2 (>= 2.7.3) for the 2018.3 edition.
You can check under [YOUR_PROJECT]/build/build.log and you might see a log similar to this one below:
[INFO] building project
[INFO] sourcing bitbake
SDK environment now set up; additionally you may now run devtool to perform development tasks.
Run devtool --help for further details.
OpenEmbedded requires 'python' to be python v2 (>= 2.7.3), not python v3.
Please set up python v2 as your default 'python' interpreter.
ERROR: Failed to source bitbake
ERROR: Failed to build project
To remedy this issue remove the symbolic link under /usr/bin and make sure to create a new one that is pointing to Python 2.7:
sudo rm /usr/bin/python
sudo ln -s /usr/bin/python2.7 /usr/bin/python
First you need to investigate the error a little further, do this:
source /opt/pkg/petalinux/2018.3/settings.sh
It will return something similar to this below:
PetaLinux environment set to '/opt/pkg/petalinux/2018.3'
INFO: Checking free disk space
INFO: Checking installed tools
INFO: Checking installed development libraries
INFO: Checking network and other services
Source the environment setup:
source /opt/pkg/petalinux/2018.3/components/yocto/source/aarch64/environment-setup-aarch64-xilinx-linux
followed by:
devtool --help
In my case I can see more about the actual error:
NOTE: Starting bitbake server...
ERROR: Unable to start bitbake server
ERROR: Last 10 lines of server log for this session (/opt/pkg/petalinux/2018.3/components/yocto/source/aarch64/bitbake-cookerdaemon.log):
Traceback (most recent call last):
File "/opt/pkg/petalinux/2018.3/components/yocto/source/aarch64/layers/core/bitbake/lib/bb/daemonize.py", line 77, in createDaemon
function()
File "/opt/pkg/petalinux/2018.3/components/yocto/source/aarch64/layers/core/bitbake/lib/bb/server/process.py", line 433, in _startServer
self.cooker = bb.cooker.BBCooker(self.configuration, self.featureset)
File "/opt/pkg/petalinux/2018.3/components/yocto/source/aarch64/layers/core/bitbake/lib/bb/cooker.py", line 178, in __init__
self.configwatcher = pyinotify.WatchManager()
File "/opt/pkg/petalinux/2018.3/components/yocto/source/aarch64/layers/core/bitbake/lib/pyinotify.py", line 1764, in __init__
raise OSError(err % self._inotify_wrapper.str_errno())
OSError: Cannot initialize new instance of inotify, Errno=Too many open files (EMFILE)
ERROR: Unable to start bitbake server
ERROR: Last 10 lines of server log for this session (/opt/pkg/petalinux/2018.3/components/yocto/source/aarch64/bitbake-cookerdaemon.log):
Traceback (most recent call last):
File "/opt/pkg/petalinux/2018.3/components/yocto/source/aarch64/layers/core/bitbake/lib/bb/daemonize.py", line 77, in createDaemon
function()
File "/opt/pkg/petalinux/2018.3/components/yocto/source/aarch64/layers/core/bitbake/lib/bb/server/process.py", line 433, in _startServer
self.cooker = bb.cooker.BBCooker(self.configuration, self.featureset)
File "/opt/pkg/petalinux/2018.3/components/yocto/source/aarch64/layers/core/bitbake/lib/bb/cooker.py", line 178, in __init__
self.configwatcher = pyinotify.WatchManager()
File "/opt/pkg/petalinux/2018.3/components/yocto/source/aarch64/layers/core/bitbake/lib/pyinotify.py", line 1764, in __init__
raise OSError(err % self._inotify_wrapper.str_errno())
OSError: Cannot initialize new instance of inotify, **Errno=Too many open files (EMFILE)**
This is pointing to the /proc/sys/fs/inotify/max_user_instances that need to be increased. In my case I went from 128 to 256 by doing this:
sudo su
echo 256 > /proc/sys/fs/inotify/max_user_instances
You need to become root with "su" and change the mac_user_instances.

VEINS building error: too many subdirs for --deep

I have downloaded a framework F2MD from git repository https://github.com/josephkamel/F2MD.git which integrates VEINS framework. I am using Sumo 1.5.0 and Omnet++ 5.6.1 on Ubuntu 18.04. This framework had some submodules. I am getting an error while building the one of the submodules veins-f2md by using the following command:
./configure && make -j$(nproc) MODE=release all**
This command gives an error message:
Creating Makefile in /home/rukhsar/F2MD-master/F2MD/veins-f2md/src...
/home/rukhsar/Downloads/omnetpp-5.6.1/bin/opp_makemake: error: too many subdirs for --deep
Traceback (most recent call last):
File "./configure", line 78, in <module>
subprocess.check_call(['env', 'opp_makemake'] + makemake_flags, cwd='src')
File "/usr/lib/python3.6/subprocess.py", line 311, in check_call
raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['env', 'opp_makemake', '-f', '--deep', '--no-deep-includes', '--make-so', '-I', '.', '-o', 'veins', '-O', 'out', '-p', 'VEINS']' returned non-zero exit status 1.
The following is a part of code written in opp_makemake file from where this error occurs:
# collect source files
if ($isDeep) {
my #allExcludedDirs = ();
push(#allExcludedDirs, $outDir);
push(#allExcludedDirs, #exceptSubdirs);
push(#allExcludedDirs, #submakeDirs);
#sourceDirs = collectDirs(".", \#allExcludedDirs);
error("too many subdirs for --deep") if (#sourceDirs > 1000);
} else {
#sourceDirs = ();
if (isGoodDir(".", \#exceptSubdirs)) {
#sourceDirs = (".");
}
}
I am unable to understand that which directories or files are for --deep. I have not used these frameworks and simulators before. Please give a solution so that I can fixed this issue and build the framework successfully.
At the time of building f2md, there should be a boost library in the "F2MD/veins-f2md/src" directory containing required header files. The error was occurring because of too many subdirectories and header files (more than 1000) in the downloaded boost library. So I deleted the unwanted header files and subdirectories from that library. After that, the framework was built correctly.

Unable to use jsonpath_rw_ext with pyspark

These are the steps I am following:
mkdir spark_lib; cd spark_lib
pip install jsonpath_rw_ext==1.1.3 -t .
zip -r9 ../spark_lib.zip *
Initialize spark context variable sc
sc.addPyFile('spark_lib.zip')
def f(x):
import jsonpath_rw_ext
return jsonpath_rw_ext.match('$.a.b', x)
sc.parallelize([{"a":{"b":10}}]).map(f).collect()
This gives me a pbr versioning error
File "/usr/local/spark/python/lib/pyspark.zip/pyspark/serializers.py", line 268, in dump_stream
vs = list(itertools.islice(iterator, batch))
File "<ipython-input-11-2619fc011e24>", line 6, in f
File "./spark_lib.zip/jsonpath_rw_ext/__init__.py", line 18, in <module>
File "./spark_lib.zip/pbr/version.py", line 467, in version_string
return self.semantic_version().brief_string()
File "./spark_lib.zip/pbr/version.py", line 462, in semantic_version
self._semantic = self._get_version_from_pkg_resources()
File "./spark_lib.zip/pbr/version.py", line 449, in _get_version_from_pkg_resources
result_string = packaging.get_version(self.package)
File "./spark_lib.zip/pbr/packaging.py", line 824, in get_version
name=package_name))
Exception: Versioning for this project requires either an sdist tarball,
or access to an upstream git repository. It's also possible that
there is a mismatch between the package name in setup.cfg
and the argument given to pbr.version.VersionInfo.
Project name jsonpath_rw_ext was given, but was not able to be found.
After reading through another similar bug https://bugs.launchpad.net/python-swiftclient/+bug/1379579, found out that pbr needs updated setuptools.
I included setuptools in my pip installation command and everything worked fine.
pip install jsonpath_rw_ext setuptools -t .

#ckan PasteScript Error during CKan Migration from version 2.4 to 2.6

I'm upgrating from version 2.4 to version 2.6 using the documentation link
I'm getting the following paster error when executing the paster command
paster db upgrade -c /etc/ckan/default/production.ini
Traceback (most recent call last):
File "/usr/lib/ckan/default/bin/paster", line 7, in
from paste.script.command import run
File "/usr/lib/ckan/default/local/lib/python2.7/site-packages/paste/script/command.py", line 54, in
dist = pkg_resources.get_distribution('PasteScript')
File "/usr/lib/ckan/default/local/lib/python2.7/site-packages/distribute-0.6.24-py2.7.egg/pkg_resources.py", line 330, in get_distribution
if isinstance(dist,Requirement): dist = get_provider(dist)
File "/usr/lib/ckan/default/local/lib/python2.7/site-packages/distribute-0.6.24-py2.7.egg/pkg_resources.py", line 209, in get_provider
return working_set.find(moduleOrReq) or require(str(moduleOrReq))[0]
File "/usr/lib/ckan/default/local/lib/python2.7/site-packages/distribute-0.6.24-py2.7.egg/pkg_resources.py", line 686, in require
needed = self.resolve(parse_requirements(requirements))
File "/usr/lib/ckan/default/local/lib/python2.7/site-packages/distribute-0.6.24-py2.7.egg/pkg_resources.py", line 584, in resolve
raise DistributionNotFound(req)
pkg_resources.DistributionNotFound: PasteScript
Any idea?
It looks like you didn't run the 'pip install' step (step 3)with an activated virtual environment. All these commands should be run with an activated virtual environment (step 1) so don't forget to do that again if you start a new terminal/shell.

google-cloud-storage library from nosetests using testbed

I have google-cloud-storage pip installed into a lib directory and vendored in. It's running just fine locally during development of my python appengine app. However, when trying to run unit tests via nose and testbed I'm getting "The 'google-cloud-core' distribution was not found and is required by the application". Here is the stack:
Traceback (most recent call last):
File "/Users/jason/dev/gain-data/venv/lib/python2.7/site-packages/nose/loader.py", line 418, in loadTestsFromName
addr.filename, addr.module)
File "/Users/jason/dev/gain-data/venv/lib/python2.7/site-packages/nose/importer.py", line 47, in importFromPath
return self.importFromDir(dir_path, fqname)
File "/Users/jason/dev/gain-data/venv/lib/python2.7/site-packages/nose/importer.py", line 94, in importFromDir
mod = load_module(part_fqname, fh, filename, desc)
File "/Users/jason/dev/gain-data/data/storage/__init__.py", line 4, in <module>
from google.cloud.storage import Blob, Client
File "/Users/jason/dev/gain-data/lib/google/cloud/storage/__init__.py", line 42, in <module>
from google.cloud.storage.batch import Batch
File "/Users/jason/dev/gain-data/lib/google/cloud/storage/batch.py", line 30, in <module>
from google.cloud.storage.connection import Connection
File "/Users/jason/dev/gain-data/lib/google/cloud/storage/connection.py", line 17, in <module>
from google.cloud import connection as base_connection
File "/Users/jason/dev/gain-data/lib/google/cloud/connection.py", line 31, in <module>
get_distribution('google-cloud-core').version)
File "/Users/jason/dev/gain-data/venv/lib/python2.7/site-packages/pkg_resources/__init__.py", line 557, in get_distribution
dist = get_provider(dist)
File "/Users/jason/dev/gain-data/venv/lib/python2.7/site-packages/pkg_resources/__init__.py", line 431, in get_provider
return working_set.find(moduleOrReq) or require(str(moduleOrReq))[0]
File "/Users/jason/dev/gain-data/venv/lib/python2.7/site-packages/pkg_resources/__init__.py", line 968, in require
needed = self.resolve(parse_requirements(requirements))
File "/Users/jason/dev/gain-data/venv/lib/python2.7/site-packages/pkg_resources/__init__.py", line 854, in resolve
raise DistributionNotFound(req, requirers)
DistributionNotFound: The 'google-cloud-core' distribution was not found and is required by the application
Any thoughts?
I had the same issue with google-cloud-translate, I was forced to also install the package "globally", i.e. pip install google-cloud-translate.
After struggling a lot with this same issue I found out that the error was because the vendor pip lib wasn't in the PYTHONPATH before calling the nosetests.
Try adding the vendor lib to the PYTHONPATH and then run the tests.
export PYTHONPATH="$(HOME)/Projects/myproject/pip_lib:$$PYTHONPATH"; \
nosetests .