Is there a way to list the order of tasks that shall be executed when a recipe is built with BitBake? I know I can build the recipe and then inspect log.task_order, but that's not what I'm after - I want to know where is the order of tasks defined for given recipe, without actually building it. I also know that there's bitbake <recipe_name> -c listtasks, but AFAIK, that lists all the available tasks, regardless if they are actually executed during the build.
Update:
The recipe I'm interested in is the kernel recipe, this is how its log.task_order looks like after a full clean build:
do_fetch
do_unpack
do_prepare_recipe_sysroot
do_kernel_checkout
do_symlink_kernsrc
do_validate_branches
do_kernel_metadata
do_patch
do_kernel_version_sanity_check
do_populate_lic
do_kernel_configme
do_configure
do_kernel_configcheck
do_compile
do_shared_workdir
do_kernel_link_images
do_compile_kernelmodules
do_strip
do_sizecheck
do_install
do_populate_sysroot
do_package
do_packagedata
do_package_qa
do_package_write_ipk
do_bundle_initramfs
do_deploy
I would expect that this sequence is defined somewhere in the recipe metadata, but I haven't found it.
Related
I have a recipe that looks basically like this :
SUMMARY = "SomeLibrary"
LICENSE = "Apache-2.0"
LIC_FILES_CHKSUM = "file://LICENSE;md5=3b83ef96387f14655fc854ddc3c6bd57"
SRC_URI += "git://gitlab.com/some_library/some-library.git;protocol=https;nobranch=1"
SRCREV = "${PV}"
S = "${WORKDIR}/git"
inherit autotools pkgconfig
It builds successfully with bitbake some-library, and I can see there is a git2/gitlab.com.some_library.some-library.git/ directory and a git2/gitlab.com.some_library.some-library.git.done file in my downloads folder (the one DL_DIR point to).
My understanding is that if I then immediately run bitbake -c cleansstate some-library && bitbake some-library, given that there is no change in the recipe, bitbake shouldn't need to download anything (it already has everything it needs). In practice, if I turn off my network connection or add BB_NO_NETWORK="1" to my local.conf, I get the following error :
Initialising tasks: 100% |################################################################| Time: 0:00:01
Sstate summary: Wanted 12 Found 4 Missed 8 Current 251 (33% match, 96% complete)
NOTE: Executing Tasks
ERROR: some-library-v2.3.0-r0 do_fetch: Bitbake Fetcher Error: NetworkAccess('https://gitlab.com/some_library/some-library.git', 'git -c core.fsyncobjectfiles=0 ls-remote "https://gitlab.com/some_library/some-library.git" ')
ERROR: Logfile of failure stored in: /home/myusername/work/builddir/tmp/work/aarch64-poky-linux/some-library/v2.3.0-r0/temp/log.do_fetch.116252
ERROR: Task (/home/myusername/work/builddir/../../layers/meta-mymeta/recipes-core/some-library/some-library_v2.3.0.bb:do_fetch) failed with exit code '1'
NOTE: Tasks Summary: Attempted 806 tasks of which 804 didn't need to be rerun and 1 failed.
Summary: 1 task failed:
/home/myusername/work/builddir/../../layers/meta-mymeta/recipes-core/some-library/some-library_v2.3.0.bb:do_fetch
Summary: There was 1 ERROR message shown, returning a non-zero exit code.
Why is that ? How do other recipes avoid this pitfall ? (when I build my image, this recipe seems to be the only one trying to fetch things from the network, which suggests to me that I'm doing something wrong here)
EDIT :
What really puzzles me is that bitbakes seems to behave differently with recipes other than my own. For example, the recipe for can-utils located at meta-openembedded/meta-oe/recipes-extended/socketcan/can-utils_git.bb looks like this:
SUMMARY = "Linux CAN network development utilities"
LICENSE = "GPLv2 & BSD-3-Clause"
LIC_FILES_CHKSUM = "file://include/linux/can.h;endline=44;md5=a9e1169c6c9a114a61329e99f86fdd31"
DEPENDS = "libsocketcan"
SRC_URI = "git://github.com/linux-can/${BPN}.git;protocol=https;branch=master"
SRCREV = "da65fdfe0d1986625ee00af0b56ae17ec132e700"
PV = "2020.02.04"
S = "${WORKDIR}/git"
inherit autotools pkgconfig
which is very similar, but when I set BB_NO_NETWORK="1" in my local.conf and run bitbake -c cleansstate can-utils && bitbake can-utils I get Tasks Summary: Attempted 842 tasks of which 822 didn't need to be rerun and all succeeded.
This works for me:
After configuring the project, add the following lines to build/conf/site.conf file:
# Build offline
SOURCE_MIRROR_URL ?= "file:///path/to/oe-downloads"
INHERIT += " own-mirrors"
BB_GENERATE_MIRROR_TARBALLS = "1"
BB_NO_NETWORK = "1"
After that, it might be necessary to build project once when online.
After every re-configuration (different build options) the site.conf is overwritten, so I created a script to add these lines after re-configuration.
I believe I found the issue.
If I replace ${PV} (which was equal to v2.3.0 here) by the hash associated to that tag, then the issue stops happening.
If I interpret this correctly, it means that bitbake is able to tell if SRCREV is a hash or a tag, and that if it is a tag then do_fetch will always run git ls-remote to make sure that the tag has not been moved.
I have a requirement to inherit an image class( *.bbclass )and run a script from image recipe but my do_compile is not getting called.
For simplicity, I am putting up minimal sample code from poky source here which does similar thing which I want.
I have created a test recipe called inherit-test_0.1.bb under my own created layer meta-raxy.
Here is the inherit-test_0.1.bb recipe file,
SUMMARY = "Inherit Test Application"
LICENSE = "CLOSED"
inherit image
do_compile () {
echo MyRecipe
}
When I compile this recipe by bitbake inherit-test after setting up oe-init-build-env, I do not see my do_compile gets compiled since there is no log file present in working directory poky/build/tmp/work/qemux86-poky-linux/inherit-test/0.1-r0/temp/log.do_compile with the line MyRecipe
And if I remove inherit image, I see do_compile gets compiled as shown below in log file,
DEBUG: Executing shell function do_compile
MyRecipe
DEBUG: Shell function do_compile finished
Any help will really be appreciated.
Inheriting "image" states do_compile[noexec] = "1", among others, (as you can see in the image.bblcass file) which means that the do_compile task won't be executed. That is one of the several ways Yocto offers for delete a task.
I am a yocto noob, trying to decipher how the device tree is built from a Xilinx hardware definition (.hdf) file. But my question is more general.
Is there a yocto way to find the source of task?
Given a task name is it possible to find where the tasks source code lives? (presumably in a recipe or class)
As an example, where is the source for the Python task do_create_yaml which is called by recipes in the meta-xilinx-bsp layer that compile the device tree blob?
bitbake -e device-tree
Will dump the python source for do_create_yaml (amongst the rest of it prodigious output) but how can I find where that is coming from?
Device tree is part of Linux Kernel. In Yocto, this is compiled from KERNEL_DEVICETREE variable value either defined as part of Linux Kernel recipe or machine configuration.
For example, for cubieboard7 as defined here,
KERNEL_DEVICETREE = "s700_cb7_linux.dtb"
instructs the compilation to use this dts file for compilation. This is done by yocto by using various classes.
In our example, we inherit kernel.bbclass which in turn inherits kernel-devicetree.bbclass, in this class (copied from kernel-devicetree.bbclass),
do_compile_append() {
for dtbf in ${KERNEL_DEVICETREE}; do
dtb=`normalize_dtb "$dtbf"`
oe_runmake $dtb
done
}
do_install_append() {
for dtbf in ${KERNEL_DEVICETREE}; do
dtb=`normalize_dtb "$dtbf"`
dtb_ext=${dtb##*.}
dtb_base_name=`basename $dtb .$dtb_ext`
dtb_path=`get_real_dtb_path_in_kernel "$dtb"`
install -m 0644 $dtb_path ${D}/${KERNEL_IMAGEDEST}/$dtb_base_name.$dtb_ext
done
}
do_deploy_append() {
for dtbf in ${KERNEL_DEVICETREE}; do
dtb=`normalize_dtb "$dtbf"`
this appends tasks to compile, install and deploy tasks. So defining KERNEL_DEVICETREE enables the automatic build of dtb.
I found that the datastore contains the filename for tasks as a VarFlag,
from a devpyshell
pydevshell> d.getVarFlags("do_create_yaml")
gives
{'filename': '.....yocto/sources/core/../meta-xilinx-tools/classes/xsctyaml.bbclass', 'lineno': '61', 'func': 1, 'task': 1, 'python': '1', 'deps': ['do_prepare_recipe_sysroot']}
So for the example in my question the active definition for the do_create_yaml task is in xsctyaml.bbclass.
In the book "Embedded Linux Systems with the Yocto Project", Chapter 4 contains a sample called "HelloWorld - BitBake style". I encountered a bunch of problems trying to get the old example working against the "Sumo" release 2.5.
If you're like me, the first error you encountered following the book's instructions was that you copied across bitbake.conf and got:
ERROR: ParseError at /tmp/bbhello/conf/bitbake.conf:749: Could not include required file conf/abi_version.conf
And after copying over abi_version.conf as well, you kept finding more and more cross-connected files that needed to be moved, and then some relative-path errors after that... Is there a better way?
Here's a series of steps which can allow you to bitbake nano based on the book's instructions.
Unless otherwise specified, these samples and instructions are all based on the online copy of the book's code-samples. While convenient for copy-pasting, the online resource is not totally consistent with the printed copy, and contains at least one extra bug.
Initial workspace setup
This guide assumes that you're working with Yocto release 2.5 ("sumo"), installed into /tmp/poky, and that the build environment will go into /tmp/bbhello. If you don't the Poky tools+libraries already, the easiest way is to clone it with:
$ git clone -b sumo git://git.yoctoproject.org/poky.git /tmp/poky
Then you can initialize the workspace with:
$ source /tmp/poky/oe-init-build-env /tmp/bbhello/
If you start a new terminal window, you'll need to repeat the previous command which will get get your shell environment set up again, but it should not replace any of the files created inside the workspace from the first time.
Wiring up the defaults
The oe-init-build-env script should have just created these files for you:
bbhello/conf/local.conf
bbhello/conf/templateconf.cfg
bbhello/conf/bblayers.conf
Keep these, they supersede some of the book-instructions, meaning that you should not create or have the files:
bbhello/classes/base.bbclass
bbhello/conf/bitbake.conf
Similarly, do not overwrite bbhello/conf/bblayers.conf with the book's sample. Instead, edit it to add a single line pointing to your own meta-hello folder, ex:
BBLAYERS ?= " \
${TOPDIR}/meta-hello \
/tmp/poky/meta \
/tmp/poky/meta-poky \
/tmp/poky/meta-yocto-bsp \
"
Creating the layer and recipe
Go ahead and create the following files from the book-samples:
meta-hello/conf/layer.conf
meta-hello/recipes-editor/nano/nano.bb
We'll edit these files gradually as we hit errors.
Can't find recipe error
The error:
ERROR: BBFILE_PATTERN_hello not defined
It is caused by the book-website's bbhello/meta-hello/conf/layer.conf being internally inconsistent. It uses the collection-name "hello" but on the next two lines uses _test suffixes. Just change them to _hello to match:
# Set layer search pattern and priority
BBFILE_COLLECTIONS += "hello"
BBFILE_PATTERN_hello := "^${LAYERDIR}/"
BBFILE_PRIORITY_hello = "5"
Interestingly, this error is not present in the printed copy of the book.
No license error
The error:
ERROR: /tmp/bbhello/meta-hello/recipes-editor/nano/nano.bb: This recipe does not have the LICENSE field set (nano)
ERROR: Failed to parse recipe: /tmp/bbhello/meta-hello/recipes-editor/nano/nano.bb
Can be fixed by adding a license setting with one of the values that bitbake recognizes. In this case, add a line onto nano.bb of:
LICENSE="GPLv3"
Recipe parse error
ERROR: ExpansionError during parsing /tmp/bbhello/meta-hello/recipes-editor/nano/nano.bb
[...]
bb.data_smart.ExpansionError: Failure expanding variable PV_MAJOR, expression was ${#bb.data.getVar('PV',d,1).split('.')[0]} which triggered exception AttributeError: module 'bb.data' has no attribute 'getVar'
This is fixed by updating the special python commands being used in the recipe, because #bb.data was deprecated and is now removed. Instead, replace it with #d, ex:
PV_MAJOR = "${#d.getVar('PV',d,1).split('.')[0]}"
PV_MINOR = "${#d.getVar('PV',d,1).split('.')[1]}"
License checksum failure
ERROR: nano-2.2.6-r0 do_populate_lic: QA Issue: nano: Recipe file fetches files and does not have license file information (LIC_FILES_CHKSUM) [license-checksum]
This can be fixed by adding a directive to the recipe telling it what license-info-containing file to grab, and what checksum we expect it to have.
We can follow the way the recipe generates the SRC_URI, and modify it slightly to point at the COPYING file in the same web-directory. Add this line to nano.bb:
LIC_FILES_CHKSUM = "${SITE}/v${PV_MAJOR}.${PV_MINOR}/COPYING;md5=f27defe1e96c2e1ecd4e0c9be8967949"
The MD5 checksum in this case came from manually downloading and inspecting the matching file.
Done!
Now bitbake nano ought to work, and when it is complete you should see it built nano:
/tmp/bbhello $ find ./tmp/deploy/ -name "*nano*.rpm*"
./tmp/deploy/rpm/i586/nano-dbg-2.2.6-r0.i586.rpm
./tmp/deploy/rpm/i586/nano-dev-2.2.6-r0.i586.rpm
I have recently worked on that hands-on hello world project. As far as I am concerned, I think that the source code in the book contains some bugs. Below there is a list of suggested fixes:
Inheriting native class
In fact, when you build with bitbake that you got from poky, it builds only for the target, unless you mention in your recipe that you are building for the host machine (native). You can do the latter by adding this line at the end of your recipe:
inherit native
Adding license information
It is worth mentioning that the variable LICENSE is important to be set in any recipe, otherwise bitbake rises an error. In our case, we try to build the version 2.2.6 of the nano editor, its current license is GPLv3, hence it should be mentioned as follow:
LICENSE = "GPLv3"
Using os.system calls
As the book states, you cannot dereference metadata directly from a python function. Which means it is mandatory to access metadata through the d dictionary. Bellow, there is a suggestion for the do_unpack python function, you can use its concept to code the next tasks (do_configure, do_compile):
python do_unpack() {
workdir = d.getVar("WORKDIR", True)
dl_dir = d.getVar("DL_DIR", True)
p = d.getVar("P", True)
tarball_name = os.path.join(dl_dir, p+".tar.gz")
bb.plain("Unpacking tarball")
os.system("tar -x -C " + workdir + " -f " + tarball_name)
bb.plain("tarball unpacked successfully")
}
Launching the nano editor
After successfully building your nano editor package, you can find your nano executable in the following directory in case you are using Ubuntu (arch x86_64):
./tmp/work/x86_64-linux/nano/2.2.6-r0/src/nano
Should you have any comments or questions, Don't hesitate !
I have a task which has a dependency set as below:
bb.build.addtask('do_function2', 'do_build', 'do_rootfs', d)
There is another task do_function1 that modifies some files in the rootfs:
ROOTFS_POSTPROCESS_COMMAND_append = " do_function1;"
I require do_function2 to be triggered only after rootfs for the image has been populated.
Query is whether the tasks that come under ROOTFS_POSTPROCESS_COMMANDS_append satisfy the do_rootfs dependency that I have set up in the addtask() call?
I want do_function2 to run only after do_function1. Is that the case even if do_function1 is specified under ROOTFS_POSTPROCESS_COMMAND_append?