meson find_program() not finding python program in yocto environment - yocto

I have an external application that converted to an exe using pyinstaller. This needs to be added in yocto framework. But the application has a dependency on pylint & pyinstaller. pylint is already part of yocto framework. I used meson.build file in my application and have a recipe file in yocto that will source the entire application from git repo and builds using instructions in meson.build. In my application meson build file uses find_program() to find pylint program installation. It works currently on my laptop, but when I build my application recipe in yocto environment, it is unable to find pylint. Even though I added "python3-pylint" in IMAGE_INSTALL_append.
| Program python3 found: YES (.../usr/bin/python3-native/python3)
| Program python3 (pylint) found: NO
|
| ../git/src/meson.build:10:0: ERROR: python3 is missing modules: pylint
pyinst = find_program('pylint') // as per meson this is probably searching in /usr/local/bin. But this is not the case in yocto. How can I modify find_program() call to find this in the right location in yocto framework. Or is there any method to handle this?
meson.build under source file folder:
# get & run pylint to check for errors
prog = import('python').find_installation('python3', modules: ['pylint'])
if not prog.found()
message('pylint not found')
else
message(prog.path())
cmd = find_program('pylint', prog.path())
message('Running pylint on src files')
foreach each : src_files
run_command(cmd, '--confidence=HIGH', each)
endforeach
endif
project's meson.build
project('proj_name',
['cpp'],
version : '0.1',
license : 'MIT',
default_options: [
'cpp_std=c++11']
)
project_pretty_name = 'proj_name'
project_url = '<git repo of project>'
python = import('python')
python3 = python.find_installation('python3', required: false)
subdir('src')
subdir('tools') # cpp tools
subdir('build')
yocto recipe file:
inherit meson pkgconfig python3native
DEPENDS += "${PYTHON_PN}-distro-native"
DEPENDS += "zlib"
RDEPENDS:${PN} += "${PYTHON_PN}-requests \
${PYTHON_PN}-pylint"
SRC_URI += "<git branch>"
SRCREV = "<src-rev>"
S = "${WORKDIR}/git"
FILES:${PN}:append = " ${bindir}/mesonexe"

Your Meson build requires pylint to be available during compilation, therefore you need to add:
DEPENDS += "${PYTHON_PN}-pylint-native"
and then you will need to ensure the pylint recipe is available in one of your configured Yocto layers. It's in meta-openembedded/meta-python.
Finally, the python3-pylint recipe and one of its dependencies (python3-astroid) do not have native support enabled, so you'll need to create the followin .bbappends as well:
# python3-pylint_%.bbappend
BBCLASSEXTEND += "native"
# python3-astroid_%.bbappend
BBCLASSEXTEND += "native"
You should then see the following line in the Meson output:
Program pylint found: YES (.../recipe-sysroot-native/usr/bin/pylint)
As a suggestion - if this recipe is building your own source code, I'd suggest linting the code as it is pushed into the repository (via a CI job), rather than at compile-time. This will reduce the dependencies required to build your project.
Of course, there would still be an optional Meson target for developers to lint their code before they push new code. Perhaps something that is enabled for non-release builds by default.

Related

Yocto custom application through custom layer does not copy source files to destination rootfs

I am trying to install a custom application on my Yocto build.
I currently use a Raspberry Pi 4 64 bit setup, for which I want PyQt5 to display an application directly on the frame buffer (so no windowing manager or desktop envoirement).
My current build with Yocto completes and boots on the Raspberry Pi. All the Qt5 libraries are also present in the root fs after the bitbake build.
Although, I'm having problems getting a custom layer, that adds a custom recipe with a custom application to also copy over to the destination root fs.
My custom layer is called 'meta-rpikms' with recipe 'recipes-kms-qt-app' which contains the application bb files. This files is called 'basicquick_0.1.bb' and has the following contents (this test application tries to add a EGLFS friendly Qt5 applicaiton, i'll try PyQt5 later):
SUMMARY = "Simple Qt5 Quick application"
#SECTION = "examples"
LICENSE = "MIT"
#PACKAGE_ARCH = "all"
LIC_FILES_CHKSUM = "file://${COMMON_LICENSE_DIR}/MIT;md5=0835ade698e0bcf8506ecda2f7b4f302"
DEPENDS += "qtbase qtdeclarative qtquickcontrols2"
SRCREV = "${AUTOREV}"
SRC_URI = "git://github.com/shigmas/BasicQuick.git"
S = "${WORKDIR}/git"
require recipes-qt/qt5/qt5.inc
do_install() {
install -d ${D}${bindir_native}
install -m 0755 BasicQuick ${D}${bindir_native}
}
FILES_${PN} += "${bindir_native}"
When I bitbake the custom layer of my meta-rpikms (called 'qt5-kms-rpi-image'), it does compute. When I take a look at the 'image_initial_manifest' file, my custom application does show up, suggesting that it does compile and try to install the application:
# This file was generated automatically and contains the packages
# passed on to the package manager in order to create the rootfs.
# Format:
# <package_type>,<package_name>
# where:
# <package_type> can be:
# 'mip' = must install package
# 'aop' = attempt only package
# 'mlp' = multilib package
# 'lgp' = language package
mip,basicquick
mip,bluez5
mip,bridge-utils
mip,hostapd
mip, bla bla bla etc etc etc.
And if I take a look in: '~/builds/pyqt5_try1/poky/rpi64-build/tmp/work/cortexa72-poky-linux/basicquick/0.1-r0', the expected files do show up. Also suggesting that it does atleast build the application. This makes the think that the 'install' arguments in my .bb file are pointing to the wrong folder.
In my basicquick_*.bb file this is to make the directory and install (copy) the built files:
install -d ${D}${bindir_native}
install -m 0755 BasicQuick ${D}${bindir_native}
I used the 'bitbake -e' command to trace the variable D and bindir_native:
D="/home/mats/builds/pyqt5_try1/poky/rpi64-build/tmp/work/raspberrypi4_64-poky-linux/qt5-kms-rpi-image/1.0-r0/image"
bindir_native="/usr/bin"
This seems okay at first glance, but when I manually follow the destination of the variable 'D', there is no 'images' folder created. I also wonder why everybody installs their custom applications on ${D}/usr/bin? Should this not be written to the build/tmp/deploy directory? Or am I missing a step here.
So, in this 'qt5-kms-rpi-image/1.0-r0' folder, there is a folder called 'deploy-qt5-kms-rpi-image-image-complete', which contains a rootfs. But there also is a rootfs folder in the 'qt5-kms-rpi-image/1.0-r0' folder. Both of these rootfs's do not contain any mention of my basicquick application, or a BasicQuick folder being created in /usr/bin.
Also, the rootfs found in the "build/tmp/deploy/images/raspberrypi4-64/qt5-kms-rpi-image-raspberrypi4-64.tar.bz2" does not contain any mention of the basicquick application being present in the filesystem.
Does anybody have any clues on what I am missing? Am I just not copying my files to the correct location? Or does the final deploy image end up somewhere else from where I am expecting it?
Thanks in advance.
With kind regards,
Mats de Waard

How do I get a complex non-Yocto makefile-based project to cross-compile in a Yocto layer?

This follows on from
How do I strip and objcopy a built .so file in the Yocto bitbake compile step?
This leads back to considerable background information.
As mentioned in the previous question, I am seeking to build OCA, which I have
as a non-Yocto makefile-based project, in Yocto. The project, which builds
fine outside of Yocto, and even in Yocto, is quite complex. The issue is that it
is not cross-compiling for my target, which is aarch64 armv8-a. It is building
successfully, but for my host machine, which is x86-64. Then Yocto sensibly
refuses to package it, saying "Unable to recognise the format of the input
file".
I changed the compile flags in my makefile to -march=armv8-a, but got the error
"cc1plus: error: bad value ('armv8-a') for '-march=' switch" which seems to
mean that I can't use the host's installed gcc for cross-compiling, but rather a
cross-compiler is needed. I previously custom-added two additional layers, a
sample helloworld and mDNS (see previous questions for lots of background), and
they all cross-compiled fine, so I know that Yocto is basically set up to do it.
What is the method to get the cross-compilation to happen? Do I need to do a lot
of pervasive changes to my project's makefile system? It may not ever have been
designed with Yocto in mind.
I am looking into this: "cross-compile library recipe in yocto":
cross-compile library recipe in yocto
which seems to have some relevant information. Edit: it was only standard stuff that I had seen before; nothing about how cross-compilation gets invoked instead of the host machine's gcc.
Edit: My updated recipe, with the FILES_${PN} added and make changed to
oe_runmake.
DESCRIPTION = "OCA"
PRIORITY = "optional"
SECTION = "protocols"
LICENSE = "MIT"
LIC_FILES_CHKSUM = "file://${COMMON_LICENSE_DIR}/MIT;md5=0835ade698e0bcf8506ecda2f7b4f302"
SRC_URI = "file://oca-1.2.7"
S = "${WORKDIR}/oca-1.2.7/Src"
# Need to override S because BitBake expects the source to be in a dir called
# oca-1.2.7 in the work dir, but it's actually additionally under Src/.
# Need a do_compile, since OCA has a makefile with a non-standard name,
# makefileOCA. Also needs non-standard flags, -f and linuxRelease.
do_compile() {
export CAP_HOME="${WORKDIR}/oca-1.2.7"
oe_runmake -f makefileOCA linuxRelease
}
do_install() {
install -d ${D}${libdir}
cp ../Obj/linuxApp/Release/OcaProtoController.so ${D}${libdir}/OcaProtoController.so
chmod 0755 ${D}${libdir}/OcaProtoController.so
}
FILES_${PN} += "${libdir}/OcaProtoController.so"
Edit: I found some info:
https://www.yoctoproject.org/docs/1.8.1/adt-manual/adt-manual.html
https://www.yoctoproject.org/docs/1.8.1/adt-manual/adt-manual.html#setting-up-the-cross-development-environment
I commented-out the setting of CC and LD in makeOCA.inc.
I did the cleaning actions of deleting the tmp folder from my build-wayland
build dir, and did "bitbake -c cleansstate oca". Then I did "time bitbake oca".
Now it looks like it is using the cross compiler.
The first error I got now is:
~/Yocto/imx-yocto-bsp/build-wayland/tmp/work/aarch64-poky-linux/oca/1.2.7-r0/oca-1.2.7/Obj/linuxApp/Release/OcaAgentProxies.a
aarch64-poky-linux-ld: cannot find crus: No such file or directory
So this is the next question: Can you tell me what "crus" means in "LDFLAGS = crus $#"?

How to package CMake Module in NativeSDK?

I am trying to install a set of CMake utilty functions under /usr/share/cmake-3.4/Modules/MYMODULES/useful.cmake within a Yocto build.
Here is my current (sanitized) recipe (call it my-useful-modules.bb)
SECTION = "devel"
LICENSE = "CLOSED"
inherit cmake
EXTERNALSRC := "path/to/source/code"
do_compile() {
:
}
FILES_${PN} += "${datadir}/cmake-3.4/Modules/MYMODULES/*"
BBCLASSEXTEND = "nativesdk"
The configure and install tasks work fine, and if I look under image in tmp/work/..., I see the full tree (including all my host directories as expected).
But I keep getting the following error
ERROR: nativesdk-my-useful-modules-1.0-r0 do_package: QA Issue:
nativesdk-my-useful-modules: Files/directories were installed but not
shipped in any package:
Followed by a long list of files which basically includes everything under image.
These modules need to be available both in the native sysroot during the build, and in the standard SDK built with populate_sdk.
Which package should I be specifying with FILES_${PN} to get them packaged?
I'd also appreciate knowing how to either avoid specifying the cmake version in the FILES statement, or get it from the build system.
Updating FILES_${PN} will resolve your error and also there is no need to provide cmake version in this case.
FILES_${PN} += "${datadir}/*"

Conan.io and netbeans - howto setup a project

I'm trying NetBeans as my new IDE for c++. I would love to use conan.io as package manager.
My conanfile.py looks like this (from the conan site):
class MyConanTestProj(ConanFile):
settings = "os", "compiler", "build_type", "arch"
requires = "Protobuf/3.1.0#inexorgame/stable", "Boost/1.64.0#conan/stable" # comma separated list of requirements
generators = "cmake", "txt"
default_options = "Poco:shared=True", "OpenSSL:shared=True", "Boost:shared=True"
def imports(self):
self.copy("*.dll", dst="bin", src="bin") # From bin to bin
self.copy("*.dylib*", dst="bin", src="lib") # From lib to bin
# self.copy('*.so*', dst='bin', src='lib')
def build(self):
cmake = CMake(self)
cmake.configure()
cmake.build()
My CMakeLists.txt:
project(MyTestProj)
cmake_minimum_required(VERSION 2.8.12)
add_definitions("-std=c++14")
include(${CMAKE_BINARY_DIR}/conanbuildinfo.cmake)
conan_basic_setup()
add_executable(testProj testProj.cpp)
target_link_libraries(testProj ${CONAN_LIBS})
testProj.cpp: (just to see it compile and link...)
#include <boost/filesystem.hpp>
int main(void) { return 0; }
When I create a build dir and run conan install and so on it works:
mkdir build -p && cd build && conan install .. && cmake .. -G "Unix Makefiles" -DCMAKE_BUILD_TYPE=Release && cmake --build
But in NetBeans it shows me an unresolved include (netbeans has no idea there conan has stored the boost files).
How to configurate netbeans to use the include paths generated by conan?
Conan created conanbildinfo.cmake and conanbuildinfo.txt with the full paths included but i don't know how to use them in netbeans.
Hope someone could tell me how to setup this correctly (or give me a short example project for netbeans) - Thanks!
Netbeans might be using a particular build folder layout, as previous versions of CLion did in the past, just putting the "temporary" build folders somewhere in the system.
I guess that the CMake execution is complaining about not finding the conanbuildinfo.cmake file, not the Boost headers.
So, the first thing is to know which folder is being used by Netbeans as CMake binary dir. You could add to your CMakeLists.txt:
if(NOT EXISTS ${CMAKE_BINARY_DIR}/conanbuildinfo.cmake)
message(FATAL_ERROR "Missing conanbuildinfo. Move to ${CMAKE_BINARY_DIR} folder, and run conan install there")
endif()
include(${CMAKE_BINARY_DIR}/conanbuildinfo.cmake)
conan_basic_setup()
Then, just cd to that folder, conan install with the respective configuration (note that this process may need to be run for different configurations, like Debug/Release).

Adding new recipe to Yocto fails during generate root fs

I have been using Yocto to create Linux builds for an ARM board.
I had been cross compiling add on applications manually. Now we are in a place where we would like a nice integrated build so I started adding custom recipes to yocto.
I have been struggling with the ARM build (a x86 build with the same code seems fine).
Even a basic 'hello world' pretty much cut and paste from the development manual does not work (http://www.yoctoproject.org/docs/current/dev-manual/dev-manual.html#new-recipe-writing-a-new-recipe)
Here is the recipe:
SUMMARY = "Simple helloworld application"
SECTION = "examples"
LICENSE = "MIT"
LIC_FILES_CHKSUM = "file://${COMMON_LICENSE_DIR}/MIT;md5=0835ade698e0bcf8506ecda2f7b4f302"
RPROVIDES_${PN} = "helloworld"
FILES_${PN} += "${bindir}"
SRC_URI = "file://helloworld.c"
S = "${WORKDIR}"
do_compile() {
${CC} helloworld.c -o helloworld ${LDFLAGS}
}
do_install() {
install -d ${D}${bindir}
install -m 0755 helloworld ${D}${bindir}
}
Here is the error:
ERROR: helloworld not found in the base feeds (smarc_samx6i cortexa9t2hf-vfp-neon-mx6qdl cortexa9hf-vfp-neon-mx6qdl cortexa9t2hf-vfp-neon cortexa9t2hf-vfp cortexa9hf-vfp-neon cortexa9hf-vfp armv7at2hf-vfp-neon armv7ahf-vfp-neon armv7at2hf-vfp armv7ahf-vfp armv6thf-vfp armv6hf-vfp armv5tehf-vfp armv5ehf-vfp armv5thf-vfp armv5hf-vfp noarch any all).
ERROR: Function failed: do_rootfs
Any suggestions as to what would be causing this error?
The package does build properly; the problem seems to be isolated to finding it for the rootfs.
Thanks!
EDIT:
I have a solution that seems to work, although it is not ideal long term.
Changing the package name under IMAGE_INSTALL from helloworld to helloworld-0.0.1 resolves the issue. Obviously I would rather not hard code the version of each package in the top level recipe and other packages do not require this, so hopefully there is another solution.
EDIT 2:
Renaming the recipe and removing the version string also resolves the issue. Once again, this does not seem ideal long term.
OK, after some further testing I discovered this was a naming issue with the recipe.
It was named helloworld-0.0.1.bb (the same format with the other recipes I had put together driving me to try this simple test).
If anyone else encounters this simply replacing the '-' with a '_' resolves this.
1.Rename your recipe name e.g hello-0.1.bb to hello_0.1.bb
2.Add below line at last only:
FILES_${PN} = "${bindir}/*"
Abvoe line helps you copy your binary to rootfs.