VEINS building error: too many subdirs for --deep - perl

I have downloaded a framework F2MD from git repository https://github.com/josephkamel/F2MD.git which integrates VEINS framework. I am using Sumo 1.5.0 and Omnet++ 5.6.1 on Ubuntu 18.04. This framework had some submodules. I am getting an error while building the one of the submodules veins-f2md by using the following command:
./configure && make -j$(nproc) MODE=release all**
This command gives an error message:
Creating Makefile in /home/rukhsar/F2MD-master/F2MD/veins-f2md/src...
/home/rukhsar/Downloads/omnetpp-5.6.1/bin/opp_makemake: error: too many subdirs for --deep
Traceback (most recent call last):
File "./configure", line 78, in <module>
subprocess.check_call(['env', 'opp_makemake'] + makemake_flags, cwd='src')
File "/usr/lib/python3.6/subprocess.py", line 311, in check_call
raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['env', 'opp_makemake', '-f', '--deep', '--no-deep-includes', '--make-so', '-I', '.', '-o', 'veins', '-O', 'out', '-p', 'VEINS']' returned non-zero exit status 1.
The following is a part of code written in opp_makemake file from where this error occurs:
# collect source files
if ($isDeep) {
my #allExcludedDirs = ();
push(#allExcludedDirs, $outDir);
push(#allExcludedDirs, #exceptSubdirs);
push(#allExcludedDirs, #submakeDirs);
#sourceDirs = collectDirs(".", \#allExcludedDirs);
error("too many subdirs for --deep") if (#sourceDirs > 1000);
} else {
#sourceDirs = ();
if (isGoodDir(".", \#exceptSubdirs)) {
#sourceDirs = (".");
}
}
I am unable to understand that which directories or files are for --deep. I have not used these frameworks and simulators before. Please give a solution so that I can fixed this issue and build the framework successfully.

At the time of building f2md, there should be a boost library in the "F2MD/veins-f2md/src" directory containing required header files. The error was occurring because of too many subdirectories and header files (more than 1000) in the downloaded boost library. So I deleted the unwanted header files and subdirectories from that library. After that, the framework was built correctly.

Related

How Can I Solve "Failed To Source Bitbake" With Xilinx Petalinux SDK

I am running Ubuntu 16.04 with Xilinx Petalinux 2018.03 SDK. After a number of successful compilations I am now facing this error
$ petalinux-build
[INFO] building project
[INFO] sourcing bitbake
ERROR: Failed to source bitbake
ERROR: Failed to build project
How can I solved this issue?
Another reason to get the errors "ERROR: Failed to source bitbake" as well as "ERROR: Failed to build project" is a possible upgrade of Python on the build machine. The Petalinux SDK requires python v2 (>= 2.7.3) for the 2018.3 edition.
You can check under [YOUR_PROJECT]/build/build.log and you might see a log similar to this one below:
[INFO] building project
[INFO] sourcing bitbake
SDK environment now set up; additionally you may now run devtool to perform development tasks.
Run devtool --help for further details.
OpenEmbedded requires 'python' to be python v2 (>= 2.7.3), not python v3.
Please set up python v2 as your default 'python' interpreter.
ERROR: Failed to source bitbake
ERROR: Failed to build project
To remedy this issue remove the symbolic link under /usr/bin and make sure to create a new one that is pointing to Python 2.7:
sudo rm /usr/bin/python
sudo ln -s /usr/bin/python2.7 /usr/bin/python
First you need to investigate the error a little further, do this:
source /opt/pkg/petalinux/2018.3/settings.sh
It will return something similar to this below:
PetaLinux environment set to '/opt/pkg/petalinux/2018.3'
INFO: Checking free disk space
INFO: Checking installed tools
INFO: Checking installed development libraries
INFO: Checking network and other services
Source the environment setup:
source /opt/pkg/petalinux/2018.3/components/yocto/source/aarch64/environment-setup-aarch64-xilinx-linux
followed by:
devtool --help
In my case I can see more about the actual error:
NOTE: Starting bitbake server...
ERROR: Unable to start bitbake server
ERROR: Last 10 lines of server log for this session (/opt/pkg/petalinux/2018.3/components/yocto/source/aarch64/bitbake-cookerdaemon.log):
Traceback (most recent call last):
File "/opt/pkg/petalinux/2018.3/components/yocto/source/aarch64/layers/core/bitbake/lib/bb/daemonize.py", line 77, in createDaemon
function()
File "/opt/pkg/petalinux/2018.3/components/yocto/source/aarch64/layers/core/bitbake/lib/bb/server/process.py", line 433, in _startServer
self.cooker = bb.cooker.BBCooker(self.configuration, self.featureset)
File "/opt/pkg/petalinux/2018.3/components/yocto/source/aarch64/layers/core/bitbake/lib/bb/cooker.py", line 178, in __init__
self.configwatcher = pyinotify.WatchManager()
File "/opt/pkg/petalinux/2018.3/components/yocto/source/aarch64/layers/core/bitbake/lib/pyinotify.py", line 1764, in __init__
raise OSError(err % self._inotify_wrapper.str_errno())
OSError: Cannot initialize new instance of inotify, Errno=Too many open files (EMFILE)
ERROR: Unable to start bitbake server
ERROR: Last 10 lines of server log for this session (/opt/pkg/petalinux/2018.3/components/yocto/source/aarch64/bitbake-cookerdaemon.log):
Traceback (most recent call last):
File "/opt/pkg/petalinux/2018.3/components/yocto/source/aarch64/layers/core/bitbake/lib/bb/daemonize.py", line 77, in createDaemon
function()
File "/opt/pkg/petalinux/2018.3/components/yocto/source/aarch64/layers/core/bitbake/lib/bb/server/process.py", line 433, in _startServer
self.cooker = bb.cooker.BBCooker(self.configuration, self.featureset)
File "/opt/pkg/petalinux/2018.3/components/yocto/source/aarch64/layers/core/bitbake/lib/bb/cooker.py", line 178, in __init__
self.configwatcher = pyinotify.WatchManager()
File "/opt/pkg/petalinux/2018.3/components/yocto/source/aarch64/layers/core/bitbake/lib/pyinotify.py", line 1764, in __init__
raise OSError(err % self._inotify_wrapper.str_errno())
OSError: Cannot initialize new instance of inotify, **Errno=Too many open files (EMFILE)**
This is pointing to the /proc/sys/fs/inotify/max_user_instances that need to be increased. In my case I went from 128 to 256 by doing this:
sudo su
echo 256 > /proc/sys/fs/inotify/max_user_instances
You need to become root with "su" and change the mac_user_instances.

Building Python3 Twisted module in yocto project

When trying to build the python twisted module using the recipe.
I have the runtime dependencies python recipes.
I don't have an idea about the error to add a native recipe since I don't have such an incremental-native recipe file.
inherit pypi setuptools3
LICENSE = "MIT"
LIC_FILES_CHKSUM = "file://LICENSE;md5=27ac6d9597237e7e76282edd7a40cd68"
SRC_URI[sha256sum] = "d72c55b5d56e176563b91d11952d13b01af8725c623e498db5507b6614fc1e10"
PYPI_PACKAGE = "Twisted"
PYPI_PACKAGE_EXT = "tar.bz2"
RDEPENDS_${PN} = " ${PYTHON_PN}-zope-interface ${PYTHON_PN}-incremental ${PYTHON_PN}-constantly \
${PYTHON_PN}-hyperlink ${PYTHON_PN}-pyhamcrest ${PYTHON_PN}-attrs "
Getting error, add native recipes to DEPENDS.
ERROR: python3-twisted-20.3.0-r0 do_compile: Execution of '/home/pravin/yocto-project/poky/build/tmp/work/armv7vet2hf-neon-poky-linux-gnueabi/python3-twisted/20.3.0-r0/temp/run.do_compile.179353' failed with exit code 1:
WARNING: The pip package is not available, falling back to EasyInstall for handling setup_requires/test_requires; this is deprecated and will be removed in a future version.
ERROR: Do not try to fetch `incremental>=16.10.1' for building. Please add its native recipe to DEPENDS.
Traceback (most recent call last):
File "/home/pravin/yocto-project/poky/build/tmp/work/armv7vet2hf-neon-poky-linux-gnueabi/python3-twisted/20.3.0-r0/Twisted-20.3.0/setup.py", line 20, in <module>
setuptools.setup(**_setup["getSetupArgs"]())
File "/home/pravin/yocto-project/poky/build/tmp/work/armv7vet2hf-neon-poky-linux-gnueabi/python3-twisted/20.3.0-r0/recipe-sysroot-native/usr/lib/python3.8/site-packages/setuptools/__init__.py", line 162, in setup
_install_setup_requires(attrs)
File "/home/pravin/yocto-project/poky/build/tmp/work/armv7vet2hf-neon-poky-linux-gnueabi/python3-twisted/20.3.0-r0/recipe-sysroot-native/usr/lib/python3.8/site-packages/setuptools/__init__.py", line 157, in _install_setup_requires
dist.fetch_build_eggs(dist.setup_requires)
File "/home/pravin/yocto-project/poky/build/tmp/work/armv7vet2hf-neon-poky-linux-gnueabi/python3-twisted/20.3.0-r0/recipe-sysroot-native/usr/lib/python3.8/site-packages/setuptools/dist.py", line 699, in fetch_build_eggs
resolved_dists = pkg_resources.working_set.resolve(
File "/home/pravin/yocto-project/poky/build/tmp/work/armv7vet2hf-neon-poky-linux-gnueabi/python3-twisted/20.3.0-r0/recipe-sysroot-native/usr/lib/python3.8/site-packages/pkg_resources/__init__.py", line 785, in resolve
raise DistributionNotFound(req, requirers)
pkg_resources.DistributionNotFound: The 'incremental>=16.10.1' distribution was not found and is required by the application
WARNING: exit code 1 from a shell command.```

Unable to use jsonpath_rw_ext with pyspark

These are the steps I am following:
mkdir spark_lib; cd spark_lib
pip install jsonpath_rw_ext==1.1.3 -t .
zip -r9 ../spark_lib.zip *
Initialize spark context variable sc
sc.addPyFile('spark_lib.zip')
def f(x):
import jsonpath_rw_ext
return jsonpath_rw_ext.match('$.a.b', x)
sc.parallelize([{"a":{"b":10}}]).map(f).collect()
This gives me a pbr versioning error
File "/usr/local/spark/python/lib/pyspark.zip/pyspark/serializers.py", line 268, in dump_stream
vs = list(itertools.islice(iterator, batch))
File "<ipython-input-11-2619fc011e24>", line 6, in f
File "./spark_lib.zip/jsonpath_rw_ext/__init__.py", line 18, in <module>
File "./spark_lib.zip/pbr/version.py", line 467, in version_string
return self.semantic_version().brief_string()
File "./spark_lib.zip/pbr/version.py", line 462, in semantic_version
self._semantic = self._get_version_from_pkg_resources()
File "./spark_lib.zip/pbr/version.py", line 449, in _get_version_from_pkg_resources
result_string = packaging.get_version(self.package)
File "./spark_lib.zip/pbr/packaging.py", line 824, in get_version
name=package_name))
Exception: Versioning for this project requires either an sdist tarball,
or access to an upstream git repository. It's also possible that
there is a mismatch between the package name in setup.cfg
and the argument given to pbr.version.VersionInfo.
Project name jsonpath_rw_ext was given, but was not able to be found.
After reading through another similar bug https://bugs.launchpad.net/python-swiftclient/+bug/1379579, found out that pbr needs updated setuptools.
I included setuptools in my pip installation command and everything worked fine.
pip install jsonpath_rw_ext setuptools -t .

breaking on unhandled exceptions in pydev/gae

i am using pydev to develop a google app engine application. i followed the steps mentioned here to configure pydev debugger to break on unhandled exception. i could get it to work on a sample pydev project, but when i try the same steps in my pydev gae project, it doesn't work and gives following error:
pydev debugger: warning: psyco not
available for speedups (the debugger
will still work correctly, but a bit
slower) pydev debugger: starting
...
Traceback (most recent call last):
File "c:\program
files\google\google_appengine\google\appengine\tools\dev_appserver.py",
line 3858, in _HandleRequest
self._Dispatch(dispatcher, self.rfile, outfile, env_dict) File
"c:\program
files\google\google_appengine\google\appengine\tools\dev_appserver.py",
line 3792, in _Dispatch
base_env_dict=env_dict) File "c:\program
files\google\google_appengine\google\appengine\tools\dev_appserver.py",
line 580, in Dispatch
base_env_dict=base_env_dict) File "c:\program
files\google\google_appengine\google\appengine\tools\dev_appserver.py",
line 2918, in Dispatch
self._module_dict) File "c:\program
files\google\google_appengine\google\appengine\tools\dev_appserver.py",
line 2822, in ExecuteCGI
reset_modules = exec_script(handler_path, cgi_path,
hook) File "c:\program
files\google\google_appengine\google\appengine\tools\dev_appserver.py",
line 2702, in ExecuteOrImportScript
exec module_code in script_module.dict File
"C:\Users\siddjain\workspace\rfad\src\main.py",
line 1, in
import pydevd ImportError: No module named pydevd
my debug configuration for gae project is like this:
the sample pydev project where it works is like this and am following same pattern in my gae project:
import pydevd
def f(x,y):
z = y/x;
return z;
def main():
pydevd.set_pm_excepthook()
print f(0,0)
if __name__ == '__main__':
main()
the run config for test project is like this:
the pydevd.py module is under C:\eclipse\plugins\org.python.pydev.debug_2.0.0.2011040403\pysrc. Although this path is not included in the pythonpath for test project, the breaking works in test. i also tried including this path in pythonpath of gae project to see if that fixes my problem, but it didn't
Its still not fixed although following steps got rid of No module named pydevd error:
1. Create a symlink to C:\eclipse\plugins\org.python.pydev.debug_2.0.0.2011040403\pysrc:
src>mklink /d debugger C:\eclipse\plugins\org.python.pydev.debug_2.0.0.2011040403\pysrc
2. Put an empty __init__.py file in C:\eclipse\plugins\org.python.pydev.debug_2.0.0.2011040403\pysrc. i learnt this from some google link that i can't find now.
3. In source code:
import debugger.pydevd as pydevd
after these steps its able to import pydevd, but still doesn't break on uncaught exception.
The development server runs your application on your local computer for testing your application. The server simulates the App Engine datastore, services and sandbox restrictions (hence step 1). although i don't understand why we need to import pydevd since its pydevd who's running our application in the first place!
In the Run->Debug Configurations->Interpreter tab if I click on "see resulting command line..."
C:\Python25\python.exe -u
C:\eclipse\plugins\org.python.pydev.debug_2.0.0.2011040403\pysrc\pydevd.py
--vm_type python --client 127.0.0.1 --port 0 --file "c:\program
files\google\google_appengine\dev_appserver.py"
The PYTHONPATH that will be used is:
C:\eclipse\plugins\org.python.pydev_2.0.0.2011040403\PySrc\pydev_si...

Eclipse using multiple Python interpreters with execnet

I'm using the execnet package to allow communication between Python scripts interpreted by different Python interpreters.
The following code (test_execnet.py):
import execnet
for python_version in ('python', 'python3'):
try:
gw = execnet.makegateway("popen//python="+python_version)
ch = gw.remote_exec('channel.send(1/3)')
res = ch.receive()
print(python_version, ': ', res, sep ="")
except:
print('problems with ', python_version)
Runs perfectly in the command-line Terminal, showing the following output:
$ python3 test_execnet.py
python: 0
python3: 0.333333333333
However, if I try to run the same code from within the Eclipse IDE, I get the following error:
'import site' failed; use -v for traceback
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "<string>", line 4, in <module>
File "<string>", line 2, in <module>
File "/Library/Frameworks/Python.framework/Versions/3.1/lib/python3.1/site-packages/execnet/gateway_base.py", line 8, in <module>
import sys, os, weakref
File "/Library/Frameworks/Python.framework/Versions/3.1/lib/python3.1/os.py", line 380, in <module>
from _abcoll import MutableMapping # Can't use collections (bootstrap)
File "/Library/Frameworks/Python.framework/Versions/3.1/lib/python3.1/_abcoll.py", line 54
class Hashable(metaclass=ABCMeta):
^
SyntaxError: invalid syntax
problems with python
problems with python3
NOTE:
Eclipse Version: 3.6.0
PyDev Interpreter configured for the project: python3
"Preferences/Interpreter - Python"'s Python Interpreters:
python (/usr/bin/python)
python3 (/Library/Frameworks/Python.Framework/Versions/3.1/Resources/Python.app/Contents/MacOS/Python
EDIT:
I write a code to show the os.environ like this:
for python_version in ('python', 'python3'):
try:
import os
for item in os.environ:
print(item, '= ', os.environ[item])
except:
print('problems with ', python_version)
I got the following outputs:
eclipse_output.txt
terminal_output.txt
A FileMerge comparison of the files can be found at eclipse_output.txt vs. terminal_output.pdf.
Any hints?
Thanks
seems like pydev does site-customizations and particularly modifies things for interactive/console usage (judging from a very quick skim of http://github.com/aptana/Pydev/blob/master/plugins/org.python.pydev/pysrc/pydev_sitecustomize/sitecustomize.py ). This is not useful or fitting for execnet-mediated processes.
You could try to "del os.environ['PYTHONPATH']" before you invoke execnet.makegateway, or, to be more careful, just delete the sitecustomize part of it.
hth,
holger
'import site' failed; use -v for traceback
I have seen that when python was unable to find its landmark. Which that indicates there is a PYTHONHOME problem.
Check out http://docs.python.org/using/cmdline.html#envvar-PYTHONHOME maybe eclipse is screwing your environment up.
Edit:
Looked at your env dumps, looks like eclipse is definitely messing with PYTHONPATH, which will cause your child python processes to not work correctly.
Basically what you have going on here is eclipse starts a python v2 instance with a PYTHONPATH pointing to the python v2 directories. Then you spawn a python v3 process which tries to load its landmark from the python v2 directories...
You need to find a way to have eclipse not mess with the PYTHONPATH. I am not sure what eclipse is trying to do by doing that, but it is certainly no friend when you want to spawn new python processes.