'no module named setuptools' but it is contained in the DEPENDS variable - yocto

This Problem regards Openembedded/Yocto.
I have source code which needs to be compiled by a custom python3 script.
That means, that some python3 script should run during the do_compile() process.
The script imports setuptools, therefore, I added DEPENDS += "python3-setuptools-native" to the recipe. As far as I understand the documentation, this should make the setuptools module available for the building process (native).
But when bitbake executes the do_compile() process, I get this error: no module named 'setuptools'.
Let me break it down to a minimal (non-)working example:
FILE: test.bb
LICENSE = "BSD"
LIC_FILES_CHKSUM = "file://test/LICENSE;md5=d41d8cd98f00b204e9800998ecf8427e"
DEPENDS += "python3-setuptools-native"
SRC_URI = "file://test.py \
file://LICENSE"
do_compile() {
python3 ${S}/../test.py
}
FILE: test.py
import setuptools
print("HELLO")
bitbaking:
$ bitbake test
ERROR: test-1.0-r0 do_compile: Function failed: do_compile (log file is located at /path/to/test/1.0-r0/temp/log.do_compile.8532)
ERROR: Logfile of failure stored in: /path/to/test/1.0-r0/temp/log.do_compile.8532
Log data follows:
| DEBUG: Executing shell function do_compile
| Traceback (most recent call last):
| File "/path/to/test-1.0/../test.py", line 1, in <module>
| import setuptools
| ImportError: No module named 'setuptools'
| WARNING: exit code 1 from a shell command.
| ERROR: Function failed: do_compile (log file is located at /path/to/test/1.0-r0/temp/log.do_compile.8532)
ERROR: Task (/path/to/test.bb:do_compile) failed with exit code '1'
NOTE: Tasks Summary: Attempted 400 tasks of which 398 didn't need to be rerun and 1 failed.
NOTE: Writing buildhistory
Summary: 1 task failed:
/path/to/test.bb:do_compile
Summary: There was 1 ERROR message shown, returning a non-zero exit code.
Is my exepectation wrong, that DEPENDS += "python3-setuptools-native" makes the python3 module 'setuptools' available to the python3 script in do_compile()? How may I accomplish this?

Under the hood quite a bit more is needed to get working setuptools support. Luckily there's a class to handle that:
inherit setuptools3
This should be all that's need to package a setuptools based project with OE-Core. As long as your project has a standard setup.py you don't need to write any do_compile() or do_install() functions.
If you do need to look at the details, meta/classes/setuptools3.bbclass and meta/classes/distutils3.bbclass should contain what you need (including the rather unobvious way to call native python from a recipe).

Related

ERROR: 'python3 setup.py build ' execution failed. in yocto dunfell

I am trying to build core-image-minimal(rootfs) using yocto. yocto version :3.1.17"
Am using Poky dunfell branch. trying to build Local image with python3
but getting bellow issue:
Log data follows:
| DEBUG: Executing shell function do_compile
| Traceback (most recent call last):
| File "/home/ubuntu/project/build/tmp/work/x86_64-linux/meson-
native/0.53.2-r0/meson-0.53.2/setup.py", line 24, in <module>
| from setuptools import setup
| **ModuleNotFoundError: No module named 'setuptools'**
| ERROR: 'python3 setup.py build ' execution failed.
| WARNING: /home/ubuntu/project/build/tmp/work/x86_64-linux/meson-native/0.53.2-r0/temp/run.do_compile.33462:1 exit 1 from 'exit 1'
| ERROR: Execution of '/home/ubuntu/project/build/tmp/work/x86_64-linux/meson-native/0.53.2-r0/temp/run.do_compile.33462' failed with exit code 1
what may be root cause ?
I can see setup.py trying to import setuptools, but not finding it. also we have included python3-setuptools as dependency.

How Can I Solve "Failed To Source Bitbake" With Xilinx Petalinux SDK

I am running Ubuntu 16.04 with Xilinx Petalinux 2018.03 SDK. After a number of successful compilations I am now facing this error
$ petalinux-build
[INFO] building project
[INFO] sourcing bitbake
ERROR: Failed to source bitbake
ERROR: Failed to build project
How can I solved this issue?
Another reason to get the errors "ERROR: Failed to source bitbake" as well as "ERROR: Failed to build project" is a possible upgrade of Python on the build machine. The Petalinux SDK requires python v2 (>= 2.7.3) for the 2018.3 edition.
You can check under [YOUR_PROJECT]/build/build.log and you might see a log similar to this one below:
[INFO] building project
[INFO] sourcing bitbake
SDK environment now set up; additionally you may now run devtool to perform development tasks.
Run devtool --help for further details.
OpenEmbedded requires 'python' to be python v2 (>= 2.7.3), not python v3.
Please set up python v2 as your default 'python' interpreter.
ERROR: Failed to source bitbake
ERROR: Failed to build project
To remedy this issue remove the symbolic link under /usr/bin and make sure to create a new one that is pointing to Python 2.7:
sudo rm /usr/bin/python
sudo ln -s /usr/bin/python2.7 /usr/bin/python
First you need to investigate the error a little further, do this:
source /opt/pkg/petalinux/2018.3/settings.sh
It will return something similar to this below:
PetaLinux environment set to '/opt/pkg/petalinux/2018.3'
INFO: Checking free disk space
INFO: Checking installed tools
INFO: Checking installed development libraries
INFO: Checking network and other services
Source the environment setup:
source /opt/pkg/petalinux/2018.3/components/yocto/source/aarch64/environment-setup-aarch64-xilinx-linux
followed by:
devtool --help
In my case I can see more about the actual error:
NOTE: Starting bitbake server...
ERROR: Unable to start bitbake server
ERROR: Last 10 lines of server log for this session (/opt/pkg/petalinux/2018.3/components/yocto/source/aarch64/bitbake-cookerdaemon.log):
Traceback (most recent call last):
File "/opt/pkg/petalinux/2018.3/components/yocto/source/aarch64/layers/core/bitbake/lib/bb/daemonize.py", line 77, in createDaemon
function()
File "/opt/pkg/petalinux/2018.3/components/yocto/source/aarch64/layers/core/bitbake/lib/bb/server/process.py", line 433, in _startServer
self.cooker = bb.cooker.BBCooker(self.configuration, self.featureset)
File "/opt/pkg/petalinux/2018.3/components/yocto/source/aarch64/layers/core/bitbake/lib/bb/cooker.py", line 178, in __init__
self.configwatcher = pyinotify.WatchManager()
File "/opt/pkg/petalinux/2018.3/components/yocto/source/aarch64/layers/core/bitbake/lib/pyinotify.py", line 1764, in __init__
raise OSError(err % self._inotify_wrapper.str_errno())
OSError: Cannot initialize new instance of inotify, Errno=Too many open files (EMFILE)
ERROR: Unable to start bitbake server
ERROR: Last 10 lines of server log for this session (/opt/pkg/petalinux/2018.3/components/yocto/source/aarch64/bitbake-cookerdaemon.log):
Traceback (most recent call last):
File "/opt/pkg/petalinux/2018.3/components/yocto/source/aarch64/layers/core/bitbake/lib/bb/daemonize.py", line 77, in createDaemon
function()
File "/opt/pkg/petalinux/2018.3/components/yocto/source/aarch64/layers/core/bitbake/lib/bb/server/process.py", line 433, in _startServer
self.cooker = bb.cooker.BBCooker(self.configuration, self.featureset)
File "/opt/pkg/petalinux/2018.3/components/yocto/source/aarch64/layers/core/bitbake/lib/bb/cooker.py", line 178, in __init__
self.configwatcher = pyinotify.WatchManager()
File "/opt/pkg/petalinux/2018.3/components/yocto/source/aarch64/layers/core/bitbake/lib/pyinotify.py", line 1764, in __init__
raise OSError(err % self._inotify_wrapper.str_errno())
OSError: Cannot initialize new instance of inotify, **Errno=Too many open files (EMFILE)**
This is pointing to the /proc/sys/fs/inotify/max_user_instances that need to be increased. In my case I went from 128 to 256 by doing this:
sudo su
echo 256 > /proc/sys/fs/inotify/max_user_instances
You need to become root with "su" and change the mac_user_instances.

How to solve error on docker:layers_calculator to compute the Merkle tree on private tangle?

I want to setup a private tangle on my own virtual machine with Ubuntu 18.04, 4GB RAM and 20GB memory.
I have follow this instructions: https://docs.iota.org/docs/compass/0.1/how-to-guides/set-up-a-private-tangle. Every command works fine until reach this one: bazel run //docker:layers_calculator.
It shows an error as follows:
Starting local Bazel server and connecting to it...
ERROR: /home/istabraq/compass/third-party/maven_deps.bzl:3:5: Traceback (most recent call last):
File "/home/istabraq/compass/WORKSPACE", line 42
maven_jars()
File "/home/istabraq/compass/third-party/maven_deps.bzl", line 3, in maven_jars
native.maven_jar(<4 more arguments>)
type 'struct' has no method maven_jar()
ERROR: error loading package '': Encountered error while reading extension file 'protobuf_deps.bzl': no such package '#com_google_protobuf_deps//': error loading package 'external': Could not load //external package
ERROR: error loading package '': Encountered error while reading extension file 'protobuf_deps.bzl': no such package '#com_google_protobuf_deps//': error loading package 'external': Could not load //external package
INFO: Elapsed time: 4.743s
INFO: 0 processes.
FAILED: Build did NOT complete successfully (0 packages loaded)
FAILED: Build did NOT complete successfully (0 packages loaded)
How can I solve this problem? what I have missed?
read carefully the message given after running bazel installer:
Make sure you have "/home/yourusername/bin" in your path. You can also activate bash completion by adding the following line to your :
source /home/yourusername/.bazel/bin/bazel-complete.bash
You can check with: "bazel info" or "bazel version"
Unfortunately, there are further errors:
https://github.com/iotaledger/compass/issues/142
I have solve this issue by using this commands:
Step 3: Set up your environment
If you ran the Bazel installer with the --user flag as above, the Bazel executable is installed in your $HOME/bin directory. It’s a good idea to add this directory to your default paths, as follows:
export PATH="$PATH:$HOME/bin"
You can also add this command to your ~/.bashrc or ~/.zshrc file to make it permanent.
reference:
https://docs.bazel.build/versions/master/install-ubuntu.html

Custom recipe for mgetty in yocto project

I have tried to create a custom recipe, in Yocto project, for mgetty. But when I build my custom image, there are some issues that I cannot figure out.
This is my custom mgetty_1.2.0.bb
DESCRIPTION = "mgetty 1.2.0 recipe"
SECTION = "mgetty"
LICENSE = "GPLv2"
LIC_FILES_CHKSUM = "file://COPYING;md5=fd3b2e6132b4ff4cb475558807ded25f"
SRC_URI = "ftp://mgetty.greenie.net/pub/mgetty/source/1.2/mgetty-${PV}.tar.gz"
SRC_URI[md5sum] = "6700f55a61f3a71ce7b882a0d0b82d19"
inherit autotools
And this is the error output:
ERROR: mgetty-1.2.0-r0 do_compile: oe_runmake failed
ERROR: mgetty-1.2.0-r0 do_compile: Function failed: do_compile (log file is
located at [...path...])
ERROR: Logfile of failure stored in: [...path...]
Log data follows:
| DEBUG: SITE files ['endian-little', 'bit-32', 'arm-common', 'arm-32',
'common-linux', 'common-glibc', 'arm-linux', 'arm-linux-gnueabi', 'common']
| DEBUG: Executing shell function do_compile
| NOTE: make -j 8
| make: *** No targets specified and no makefile found. Stop.
| ERROR: oe_runmake failed
| WARNING: [...path...] exit 1 from 'exit 1'
| ERROR: Function failed: do_compile (log file is located at [...path...])
ERROR: Task ([...path...]/recipes-mgetty/mgetty/mgetty_1.2.0.bb:do_compile)
failed with exit code '1'
NOTE: Tasks Summary: Attempted 2741 tasks of which 2734 didn't need to be
rerun and 1 failed.
NOTE: Writing buildhistory
Did I missed something in my recipe?
I'd suggest using devtool if you are new to Yocto. So, in this case:
devtool add mgetty ftp://mgetty.greenie.net/pub/mgetty/source/1.2/mgetty-1.2.0.tar.gz
This will create a local workspace for you with the recipe and sources. Now, devtool should be able to detect some of the things automatically based on presence of certain files in the package.
But, for projects like this, you might have to make changes to the Makefile as well to make sure that it doesn't override the compiler and flags variables. So, in this case, make changes to ensure CC, CFLAGS, LDFLAGS is not over-ridden etc. (replacing = with ?= would be one way of doing it). Once that is done, use 'devtool build' to build your recipe and resolve any further issues.
Once you're done with the changes, please create commits for them in the devtool workspace and use 'devtool finish' to create patches for these commits and have them included them in your recipe. See:
https://www.yoctoproject.org/docs/current/mega-manual/mega-manual.html#sdk-devtool-use-devtool-modify-to-modify-the-source-of-an-existing-component
For more information, refer to the manual.

BitBake recipe for custom setup.py

In my recipe I have to download git repository and run CMake. After CMake finish its work additional directory OUT is created which contain setup.py file that I like to run in do_install?
I have tried:
DEPENDS = "setuptools python"
do_install () {
python OUT/setup.py install
}
But it raise no setup.py found error.
Can anyone deal with such issue?
That's happen because bitbake doesn't know where setup.py is stored - You need to use ${S} variable generated by bitbake to provide full path to this script.
Please read about how do_install() task work - link
At the moment I have reorganized my recipe which looks like below:
LICENSE = "CLOSED"
BB_STRICT_CHECKSUM = "0"
inherit cmake setuptools pythonnative
DEPENDS = "boost udev python swig-native python-native python-setuptools-native cmake-native"
SRC_URI = " \
git://github.com/my_repo.git;name=my_name \
file://0001-system-install.patch \
"
SRCREV_my_name = "404ff3eeff0d79c15cbfdbc126c4bff2996baea6"
S = "${WORKDIR}/git"
PARALLEL_MAKEINST = ""
Project downloaded from git base on CMake which has install like that:
install(CODE "execute_process(COMMAND python \"${PROJECT_SOURCE_DIR}/python/setup.py\" \"install\")")
But when I call recipe to build (bitbake my_recipe) or build image which contains that recipe (bitbake my_image) i received such error:
ERROR: pc-ble-driver-git-r0 do_compile: python setup.py build execution failed.
ERROR: pc-ble-driver-git-r0 do_compile: Function failed: do_compile (log file is located at /build/yocto-fsl/build/tmp/work/cortexa7hf-neon-poky-linux-gnueabi/pc-ble-driver/git-r0/temp/log.do_compile.16502)
ERROR: Logfile of failure stored in: /build/yocto-fsl/build/tmp/work/cortexa7hf-neon-poky-linux-gnueabi/pc-ble-driver/git-r0/temp/log.do_compile.16502
Log data follows:
| DEBUG: Executing shell function do_compile
| ERROR: python setup.py build execution failed.
| /build/yocto-fsl/build/tmp/work/cortexa7hf-neon-poky-linux-gnueabi/pc-ble-driver/git-r0/recipe-sysroot-native/usr/bin/python-native/python: can't open file 'setup.py': [Errno 2] No such file or directory
| WARNING: exit code 1 from a shell command.
| ERROR: Function failed: do_compile (log file is located at /build/yocto-fsl/build/tmp/work/cortexa7hf-neon-poky-linux-gnueabi/pc-ble-driver/git-r0/temp/log.do_compile.16502)
ERROR: Task (/build/yocto-fsl/sources/meta-slabs/recipes-external/pc-ble-driver/pc-ble-driver_git.bb:do_compile) failed with exit code '1'
NOTE: Tasks Summary: Attempted 2195 tasks of which 2194 didn't need to be rerun and 1 failed.
P.S. On my PC when I build CMake project and call make install everything go as I assumed.
Any other suggestion how to deal with that?
Try add below command in your recipe file
distutils_do_compile() {
:
}
distutils_stage_headers() {
:
}
distutils_stage_all() {
:
}
distutils_do_install() {
:
}
and see more detail information below...
./poky/meta/classes/distutils-tools.bbclass