Yocto: Overriding function in existing recipe - yocto

I am new to Yocto. I am trying to modify an existing function in a recipe. I am a little confused on how to override the function using a bbappend file.
I have the following code in an existing recipe:
fn_functionA () {
pushd ${WORKDIR}/folder1
git submodule update --init --recursive submodule1
popd
}
do_unpack[postfuncs] += "fn_functionA"
I need to edit this function to the following:
fn_functionA () {
pushd ${WORKDIR}/folder1
git submodule update --init --recursive submodule1
popd
pushd ${WORKDIR}/folder1/submodule1
git fetch "ssh link to project" refs/changes/99/68099/1 && git checkout FETCH_HEAD
popd
}
do_unpack[postfuncs] += "fn_functionA"
Do I need to copy/paste the whole body of the edited function in the bbappend file, or do an addtask? A detailed answer would be appreciated.
Also, n00b question: Why can't we edit the bb file and modify the function directly instead of overriding the function in a bbappend file?

You can append to functions the same way as you append to recipe's tasks.
Here is an example:
meta-example/recipes-example/example/example_0.1.bb
LICENSE = "CLOSED"
func_example() {
bbwarn "From func_example"
}
do_example() {
bbwarn "From do_example"
}
do_example[postfuncs] += "func_example"
addtask do_example
Now when you run: bitbake example -c example you should see:
...
WARNING: example-0.1-r0 do_example: From do_example
WARNING: example-0.1-r0 do_example: From func_example
...
Now, let's create a bbappend file:
meta-example-2/recipes-example/example/example_%.bbappend
func_example_append() {
bbwarn "From func_example_append"
}
Now, when you run bitbake example -c example again you should see:
...
WARNING: example-0.1-r0 do_example: From do_example
WARNING: example-0.1-r0 do_example: From func_example
WARNING: example-0.1-r0 do_example: From func_example_append
...
EDIT: If you want to override the function, just redefine it without append or prepend, in my example it would be:
func_example() {
bbwarn "From func_example_append"
}
And the output would be:
...
WARNING: example-0.1-r0 do_example: From do_example
WARNING: example-0.1-r0 do_example: From func_example_append
...
Why using bbappend ?
It is not recommended to change/edit something directly in a third party Yocto layer (Example: meta-raspberrypi, meta-qt5, ...), and here is why:
By time, you will forget what you changed.
No one can replicate your exact image unless you send it your modified layer
If you want to upgrade the layer from one release to another, you need to manually redo the modifications unless you did not forget them
It is not a way to work in a Yocto development team
Creating a bbappend in your custom layer gives clear idea of what you have changed, also it will be easy to update the changes or to send the layer to someone knows that he should clone the third party by him self and then use your layer.
The most import point for me, is a respect for those who developed the layer and make it public for everyone, that is why keeping it clean as cloned is one of the best practices techniques in Yocto development.

Related

Yocto: how to configure an out-of-tree kernel module recipe that uses "inherit module"?

I have written a simple inherit module recipe to build a third-party out-of-tree kernel module called u-dma-buf:
SUMMARY = "User space mappable DMA Buffer"
DESCRIPTION = "${SUMMARY}"
LICENSE = "BSD-2-Clause"
LIC_FILES_CHKSUM = "file://LICENSE;md5=bebf0492502927bef0741aa04d1f35f5"
inherit module
SRC_URI = "git://github.com/ikwzm/udmabuf.git"
SRCREV = "9b943d49abc9c92a464e4c71e83d1c479ebbf80e"
S = "${WORKDIR}/git"
RPROVIDES_${PN} += "kernel-module-u-dma-buf"
This works correctly and generates the module file /lib/modules/[version]/extra/u-dma-buf.ko in the image.
However, looking at the docs there is an option that can be enabled: CONFIG_U_DMA_BUF_MGR that is disabled by default. If I can somehow enable this, then I can expect to find /lib/modules/[version]/extra/u-dma-buf-mgr.ko in the image also.
The project has a Kconfig file. Does bitbake have support for integrating Kbuild configuration outside of the kernel tree? If so, how do I hook into this and enable CONFIG_U_DMA_BUF_MGR? And if not, what's my best option, other than patching the Kconfig file to change the default to "y"? (EDIT: this probably won't work, as kernel sources need to be modified to incorporate Kbuild anyway - so probably a dead end unless it's a specific feature of bitbake I haven't encountered yet).
I see that the upstream Makefile also has the option CONFIG_U_DMA_BUF_MGR=m that could be used to enable this feature outside of Kbuild. I'm not sure how to pass that to the make command line though - would I need to write a custom do_compile task? Looking at the module.bbclass code, I can't see any provision for passing such an option to oe_runmake. Should I just copy/paste module_do_compile() from module.bbclass and add CONFIG_U_DMA_BUF_MGR=m? Is that the best way to do this?
So my question is, given I'm using inherit module, what is the proper way to enable this configuration option, given the recipe I have?
According to the Yocto kernel development docs:
If your module Makefile uses a different variable, you might want to override the do_compile step
That suggests to me that the correct and intended course of action in this case is to copy/paste module_do_compile() as a do_compile() override, and modify accordingly (add CONFIG_U_DMA_BUF_MGR=m):
do_compile() {
unset CFLAGS CPPFLAGS CXXFLAGS LDFLAGS
oe_runmake KERNEL_PATH=${STAGING_KERNEL_DIR} \
KERNEL_VERSION=${KERNEL_VERSION} \
CC="${KERNEL_CC}" LD="${KERNEL_LD}" \
AR="${KERNEL_AR}" \
O=${STAGING_KERNEL_BUILDDIR} \
KBUILD_EXTRA_SYMBOLS="${KBUILD_EXTRA_SYMBOLS}" \
CONFIG_U_DMA_BUF_MGR=m \
${MAKE_TARGETS}
}
Additionally, add the new module to RPROVIDES_${PN}:
RPROVIDES_${PN} += "kernel-module-u-dma-buf kernel-module-u-dma-buf-mgr"

how to regenerate meson for newly added yaml files

I have added yaml files to add new dbus objects and I added PHOSPHOR_MAPPER_SERVICE_append = " com/newCoName"
(newCoName is the name of my company)
But when I run bitbake, do_configure for phosphor_mapper bails when it passes the option -Ddata_com_newCoName to meson. The following readme says I need to run ./regenerate_meson from the gen directory when I add new YAML files. But how do I do that from a recipe file?
https://github.com/openbmc/phosphor-dbus-interfaces
One option is to generate these files outside ot yocto environment (i.e. not involving bitbake). Thus
clone that git repo
place your yaml file where you cloned repo
do what readme tells, i.e. go to gen directory and execute meson-regenerate script
collect changes that are done by script and create patch
add patch to your layer and reference it in .bbappend file (meta-/recipes-phosphor/dbus/phosphor-dbus-interfaces_git.bbappend)
Another option would be to add to .bbappend file additional task that runs before do_configure - and call that script from there:
do_configure_prepend() {
cd ${S}/gen && ./meson-regenerate
}
Along this .bbappend you should add your yaml so that it lands inside gen folder in patch or directly in your layer (check FILESEXTRAPATHS).
In both cases you'll need to patch meson_options.txt: add option
option('data_com_newCoName', type: 'boolean', value: true)

How to dynamical create a file with git info and include it in the the image and save it on build system

We have several developers working on a project. The areas we are concerned about (and we regularly modify) are kernel, our custom code, and the yocto space itself.
We'd like to create a file at some point in the process (do_fetch, or do_install?) that contains info about what's being built. Such as the git branch name and hash for each of the repos above. We would then install that file (or files if need be) onto the image as well as archive it away on a centralized server.
I know that some of this info is available in the buildhistory, but I'm not sure if it is there when we'd like to install and package.
Getting the branch and hash should be easy to get via shell commands in the recipe functions.
Before I go off and hack something out, I thought I'd ask if there is a standard way to do something similar to this.
Thanks!
In case you need to include custom information. A nice way consists in creating a custom layer bbclass, defined as follow :
DEPENDS += "git-native"
do_rootfs_save_versions() {
#Do custom tasks here like getting layer names and linked SHA numbers
#Store these information in a file and deploy it in ${DEPLOY_DIR_IMAGE}
}
ROOTFS_POSTPROCESS_COMMAND += "do_rootfs_save_versions;"
Then, include the bbclass in your image file
IMAGE_CLASSES += "<bbclass_name>"
It is very useful when you want to determine the layer version/image name/.. running on target.
OK, Here is what I did.
Added appends to the do_install functions I wanted to keep track of and put them in the top of the build dir:
do_install_append () {
echo ${SRCPV} > ${TOPDIR}/kernel_manifest.txt
git rev-parse --abbrev-ref HEAD >> ${TOPDIR}/kernel_manifest.txt
}
Added a new bbclass in our meta- dir:
DEPENDS += "git-native"
do_rootfs_save_manifests[nostamp] = "1"
do_rootfs_save_manifests() {
date > ${TOPDIR}/buildinfo.txt
hostname >> ${TOPDIR}/buildinfo.txt
git config user.name >> ${TOPDIR}/buildinfo.txt
cp ${TOPDIR}/buildinfo.txt ${IMAGE_ROOTFS}/usr/custom_space/
if [ ! -f ${TOPDIR}/kernel_manifest.txt ]; then
echo "kernel_manifest empty: Rebuild or run cleanall on it's recipe" > ${TOPDIR}/error_kernel_manifest.txt
cp ${TOPDIR}/error_kernel_manifest.txt ${IMAGE_ROOTFS}/usr/custom_space/
else
cp ${TOPDIR}/kernel_manifest.txt ${IMAGE_ROOTFS}/usr/custom_space/
if [ -f ${TOPDIR}/error_kernel_manifest.txt ]; then
rm ${TOPDIR}/error_kernel_manifest.txt
fi
fi
if [ ! -f ${TOPDIR}/buildhistory/metadata-revs ]; then
echo " metadata_revs empty: Make sure INHERIT += \"buildhistory\" and" > ${TOPDIR}/error_yocto_manifest.txt
echo " BUILDHISTORY_COMMIT = "1" are in your local.conf " >> ${TOPDIR}/error_yocto_manifest.txt
cp ${TOPDIR}/error_yocto_manifest.txt ${IMAGE_ROOTFS}/usr/custom_space/
else
if [ -f ${TOPDIR}/error_yocto_manifest.txt ]; then
rm ${TOPDIR}/error_yocto_manifest.txt
fi
cp ${TOPDIR}/buildhistory/metadata-revs ${TOPDIR}/yocto_manifest.txt
cp ${TOPDIR}/buildhistory/metadata-revs ${IMAGE_ROOTFS}/usr/custom_space/yocto_manifest.txt
fi
}
ROOTFS_POSTPROCESS_COMMAND += "do_rootfs_save_manifests;"
Added the following lines to the image recipes that we wanted to use the process:
IMAGE_CLASSES += "manifest"
inherit ${IMAGE_CLASSES}
Thanks for the help!

How can I get "HelloWorld - BitBake Style" working on a newer version of Yocto?

In the book "Embedded Linux Systems with the Yocto Project", Chapter 4 contains a sample called "HelloWorld - BitBake style". I encountered a bunch of problems trying to get the old example working against the "Sumo" release 2.5.
If you're like me, the first error you encountered following the book's instructions was that you copied across bitbake.conf and got:
ERROR: ParseError at /tmp/bbhello/conf/bitbake.conf:749: Could not include required file conf/abi_version.conf
And after copying over abi_version.conf as well, you kept finding more and more cross-connected files that needed to be moved, and then some relative-path errors after that... Is there a better way?
Here's a series of steps which can allow you to bitbake nano based on the book's instructions.
Unless otherwise specified, these samples and instructions are all based on the online copy of the book's code-samples. While convenient for copy-pasting, the online resource is not totally consistent with the printed copy, and contains at least one extra bug.
Initial workspace setup
This guide assumes that you're working with Yocto release 2.5 ("sumo"), installed into /tmp/poky, and that the build environment will go into /tmp/bbhello. If you don't the Poky tools+libraries already, the easiest way is to clone it with:
$ git clone -b sumo git://git.yoctoproject.org/poky.git /tmp/poky
Then you can initialize the workspace with:
$ source /tmp/poky/oe-init-build-env /tmp/bbhello/
If you start a new terminal window, you'll need to repeat the previous command which will get get your shell environment set up again, but it should not replace any of the files created inside the workspace from the first time.
Wiring up the defaults
The oe-init-build-env script should have just created these files for you:
bbhello/conf/local.conf
bbhello/conf/templateconf.cfg
bbhello/conf/bblayers.conf
Keep these, they supersede some of the book-instructions, meaning that you should not create or have the files:
bbhello/classes/base.bbclass
bbhello/conf/bitbake.conf
Similarly, do not overwrite bbhello/conf/bblayers.conf with the book's sample. Instead, edit it to add a single line pointing to your own meta-hello folder, ex:
BBLAYERS ?= " \
${TOPDIR}/meta-hello \
/tmp/poky/meta \
/tmp/poky/meta-poky \
/tmp/poky/meta-yocto-bsp \
"
Creating the layer and recipe
Go ahead and create the following files from the book-samples:
meta-hello/conf/layer.conf
meta-hello/recipes-editor/nano/nano.bb
We'll edit these files gradually as we hit errors.
Can't find recipe error
The error:
ERROR: BBFILE_PATTERN_hello not defined
It is caused by the book-website's bbhello/meta-hello/conf/layer.conf being internally inconsistent. It uses the collection-name "hello" but on the next two lines uses _test suffixes. Just change them to _hello to match:
# Set layer search pattern and priority
BBFILE_COLLECTIONS += "hello"
BBFILE_PATTERN_hello := "^${LAYERDIR}/"
BBFILE_PRIORITY_hello = "5"
Interestingly, this error is not present in the printed copy of the book.
No license error
The error:
ERROR: /tmp/bbhello/meta-hello/recipes-editor/nano/nano.bb: This recipe does not have the LICENSE field set (nano)
ERROR: Failed to parse recipe: /tmp/bbhello/meta-hello/recipes-editor/nano/nano.bb
Can be fixed by adding a license setting with one of the values that bitbake recognizes. In this case, add a line onto nano.bb of:
LICENSE="GPLv3"
Recipe parse error
ERROR: ExpansionError during parsing /tmp/bbhello/meta-hello/recipes-editor/nano/nano.bb
[...]
bb.data_smart.ExpansionError: Failure expanding variable PV_MAJOR, expression was ${#bb.data.getVar('PV',d,1).split('.')[0]} which triggered exception AttributeError: module 'bb.data' has no attribute 'getVar'
This is fixed by updating the special python commands being used in the recipe, because #bb.data was deprecated and is now removed. Instead, replace it with #d, ex:
PV_MAJOR = "${#d.getVar('PV',d,1).split('.')[0]}"
PV_MINOR = "${#d.getVar('PV',d,1).split('.')[1]}"
License checksum failure
ERROR: nano-2.2.6-r0 do_populate_lic: QA Issue: nano: Recipe file fetches files and does not have license file information (LIC_FILES_CHKSUM) [license-checksum]
This can be fixed by adding a directive to the recipe telling it what license-info-containing file to grab, and what checksum we expect it to have.
We can follow the way the recipe generates the SRC_URI, and modify it slightly to point at the COPYING file in the same web-directory. Add this line to nano.bb:
LIC_FILES_CHKSUM = "${SITE}/v${PV_MAJOR}.${PV_MINOR}/COPYING;md5=f27defe1e96c2e1ecd4e0c9be8967949"
The MD5 checksum in this case came from manually downloading and inspecting the matching file.
Done!
Now bitbake nano ought to work, and when it is complete you should see it built nano:
/tmp/bbhello $ find ./tmp/deploy/ -name "*nano*.rpm*"
./tmp/deploy/rpm/i586/nano-dbg-2.2.6-r0.i586.rpm
./tmp/deploy/rpm/i586/nano-dev-2.2.6-r0.i586.rpm
I have recently worked on that hands-on hello world project. As far as I am concerned, I think that the source code in the book contains some bugs. Below there is a list of suggested fixes:
Inheriting native class
In fact, when you build with bitbake that you got from poky, it builds only for the target, unless you mention in your recipe that you are building for the host machine (native). You can do the latter by adding this line at the end of your recipe:
inherit native
Adding license information
It is worth mentioning that the variable LICENSE is important to be set in any recipe, otherwise bitbake rises an error. In our case, we try to build the version 2.2.6 of the nano editor, its current license is GPLv3, hence it should be mentioned as follow:
LICENSE = "GPLv3"
Using os.system calls
As the book states, you cannot dereference metadata directly from a python function. Which means it is mandatory to access metadata through the d dictionary. Bellow, there is a suggestion for the do_unpack python function, you can use its concept to code the next tasks (do_configure, do_compile):
python do_unpack() {
workdir = d.getVar("WORKDIR", True)
dl_dir = d.getVar("DL_DIR", True)
p = d.getVar("P", True)
tarball_name = os.path.join(dl_dir, p+".tar.gz")
bb.plain("Unpacking tarball")
os.system("tar -x -C " + workdir + " -f " + tarball_name)
bb.plain("tarball unpacked successfully")
}
Launching the nano editor
After successfully building your nano editor package, you can find your nano executable in the following directory in case you are using Ubuntu (arch x86_64):
./tmp/work/x86_64-linux/nano/2.2.6-r0/src/nano
Should you have any comments or questions, Don't hesitate !

Yocto: Install different config files based on MACHINE type or target image

I've got a couple of HW platforms (same cpu, etc.) that require different asound.conf files.
The way that I'm controlling the target platform is via the MACHINE variable and target image (i.e., MACHINE=machine_1 nice bitbake machine-1-bringup-image)
Normally, if just replacing the conf file I'd just create an alsa-state.bbappend and create a do_install_append function to replace it.
However since the different HW platforms require differ conf files I'm unsure how to handle it.
I've tried putting some logic into the append file do_install_append function but it's not working out. It's not always picking up the correct file (like it thinks that nothing has changed so that it uses the previous cached conf?)
Here's an example of one of the append files that I've tried:
FILESEXTRAPATHS_prepend := "${THISDIR}/files:"
SRC_URI += " \ file://asound_MACHINE1.conf \
file://asound_MACHINE2.conf \ "
do_install_append() {
echo " alsa-state.bbappend MACHINE: ${MACHINE}"
if [ "${MACHINE}" = "machine_1" ]; then
echo " machine_1"
echo " installing ${WORKDIR}/asound_MACHINE1.conf to ${D}${sysconfdir}/asound.conf"
install -m 644 ${WORKDIR}/asound_MACHINE1.conf {D}${sysconfdir}/asound.conf
else
echo " installing ${WORKDIR}/asound_MACHINE2.conf to ${D}${sysconfdir}/asound.conf"
install -m 644 ${WORKDIR}/asound_MACHINE2.conf ${D}${sysconfdir}/asound.conf
fi
}
I can see the correct echoes in the logs per the logic.
At any rate I don't think that the path I'm going down is the best way to deal with this.
Is there a 'standard' way to have different files installed based on either the target image or MACHINE variable?
do_install_append () {
// install common things here
}
do_install_append_machine-1 () {
// install machine-1 specific things here
}
do_install_append_machine-2 () {
// install machine-2 specific things here
}
The value of MACHINE is automatically added to OVERRIDES, which can be used at the end of a function append to have a MACHINE-specific addition to a function.
Maybe useful: https://www.yoctoproject.org/docs/2.4/mega-manual/mega-manual.html#var-OVERRIDES
You can have configuration files in machine specific directories in your particular case (just a specific configuration file for each machine). OpenEmbedded will fetch the most specific one. The directory structure in your recipe directory will look like:
files/<machine1>/asound.conf
files/<machine2>/asound.conf
And your alsa-state.bbappend will contain just one line (you don't need to change do_install because alsa-state.bb already installs asound.conf):
FILESEXTRAPATHS_prepend := "${THISDIR}/files:"
BTW: We are using that setup to have specific asound.state file per machine in our project.
Moreover, OpenEmbedded will detect that SRC_URI contains machine specific file and change the PACKAGE_ARCH accordingly, see: https://www.yoctoproject.org/docs/2.5/mega-manual/mega-manual.html#var-SRC_URI_OVERRIDES_PACKAGE_ARCH
Few more words on machine, distro or arch specific files: OE is trying to fetch the most specific file in file:// fetcher. It searches also in the directories named by distro (e.g files/<distro>/asound.conf) and architecture (e.g. armv7a, arm). It might be useful if you want to have file specific for some set of devices. More information: https://www.yoctoproject.org/docs/2.5/mega-manual/mega-manual.html#var-FILESOVERRIDES and also https://www.yoctoproject.org/docs/2.5/mega-manual/mega-manual.html#best-practices-to-follow-when-creating-layers (section "Place Machine-Specific Files in Machine-Specific Locations")
The above answer by clsulliv worked better than advertised. For future reference below is the append file I used:
FILESEXTRAPATHS_prepend:= "${THISDIR}/${PN}:"
SRC_URI += " \
file://machine1_asound.conf \
file://machine2_asound.conf \
"
do_install_append_machine1() {
echo " machine1"
echo " installing ${WORKDIR}/machine1_asound.conf to ${D}${sysconfdir}/asound.conf"
install -m 644 ${WORKDIR}/machine1_asound.conf ${D}${sysconfdir}/asound.conf
}
do_install_append_machine2() {
echo " machine2"
echo " installing ${WORKDIR}/machine2_asound.conf to ${D}${sysconfdir}/asound.conf"
install -m 644 ${WORKDIR}/machine2_asound.conf ${D}${sysconfdir}/asound.conf
}
Thanks for the help!