I want to build two images with every Yocto bitbake command. One for development purposes, one for production releases. I currently control this by commenting the following lines from my application bb file.
# Extra image features required for debugging
IMAGE_INSTALL_append = " systemd-analyze"
IMAGE_INSTALL_append = " packagegroup-core-eclipse-debug"
EXTRA_IMAGE_FEATURES ?= "debug-tweaks"
If my image file is image.bb and I create an development image image-dev.bb how can I build both?
In my bblayers.conf I have included image.bb do I need to include image-dev.bb?
In image-dev.bb I do not want to copy and paste everything from image.bb, can I simply make this file contain:
SUMMARY = "image application dev"
LICENSE = "MIT"
IMAGE_LINGUAS = " "
require image.bb
# Extra image features required for debugging
IMAGE_INSTALL_append = " systemd-analyze"
IMAGE_INSTALL_append = " packagegroup-core-eclipse-debug"
EXTRA_IMAGE_FEATURES ?= "debug-tweaks"
My end goal is to place this into a Jenkins pipeline and archive the two SD card images.
Edited to add my tree output
.
├── busybox
│ ├── busybox
│ │ └── evcc_kernel_features.cfg
│ └── busybox_1.33.1.bbappend
├── helloworld
│ ├── files
│ │ ├── services
│ │ │ └── helloworld.service
│ │ └── src
│ │ └── app
│ │ ├── helloworld.c
│ │ └── helloworld.h
│ └── helloworld_0.1.bb
└── images
├── image.bb
├── image-dev.bb
└── image.inc
Edit 2
Reading the BitBake User Manual I need to move all common functionality to an image.inc file. Then include this in my image.bb and image-dev.bb files. I have done this. Now my image.inc file has my entire setup.
My image.bb is as follows:
SUMMARY = "application"
require evccapplicaion.inc
My image-dev.bb is as follows:
SUMMARY = "image application dev"
require evccapplicaion.inc
# Extra image features required for debugging
IMAGE_INSTALL_append = " systemd-analyze"
IMAGE_INSTALL_append = " packagegroup-core-eclipse-debug"
EXTRA_IMAGE_FEATURES ?= "debug-tweaks"
This gives me the error
image-dev.bb:3: Could not include required file image.inc
This confuses me as the bitbake user manual states that the .inc file should be in the same folder as your .bb files. Looking at my tree output above you can see that I have ensured this to be true.
Edit 3:
My issue was a typo! My official code base name is not image.bb and image-dev.bb but a slightly longer more complicated name. A name which I typed incorrectly. Pro Tip. Spell check!
Related
I need to add debug mode (enable/disable a few kernel config) to a kernel recipe. The recipe resides under recipes-kernel.
We have this inside recipe linux-yocto-onl_5.10.bb:
...
SRC_URI += "\
git://git.yoctoproject.org/yocto-kernel-cache;type=kmeta;name=meta;branch=yocto-5.10;destsuffix=kernel-meta \
file://my-kmeta;type=kmeta;name=my-kmeta;destsuffix=my-kmeta
"
We have this directory structure inside recipe folder:
.
└── my-kmeta
├── bsp
│ ├── armel-iproc
│ │ ├── armel-iproc.cfg
│ │ ├── armel-iproc.scc
│ │ ├── ... (some patches)
│ │ ├── extraconfigs.cfg
│ │ └── extraconfigs_debug.cfg
└── ...
This is content of armel-iproc.scc:
define KMACHINE armel-iproc
define KTYPE standard
define KARCH arm
patch ...
kconf hardware armel-iproc.cfg
kconf non-hardware extraconfigs.cfg
I tried two solution without success:
First I've added another .scc file and appended it to SRC_URI of the recipe. Inside that scc file just added kconf extraconfigs_debug.cfg. after baking the recipe, the config file seems copied to the build directory, but inside the .config file in build directory the changes aren't applied.
The second approach is defining two recipes, one for release and another for debug, for this I need to have two .scc files: one armel-iproc.scc and the other is armel-iproc_debug.scc, the second included the first plus kconfing extraconfigs_debug.cfg. After baking the debug recipe (by including armel-iproc_debug.scc to SRC_URI), I got errors indicating that bsp/armel-iproc is not found.
During experiments with Yocto's Advanced Kernel Metadata I got into a situation where I would like to have the .scc, .patch and .cfg files organized into a directory structure as mentioned in the syntax section, and to keep that structure in all the layers (consider 2 layers here - meta-main_layer and meta-bsp_layer):
meta-main_layer
└── recipes-kernel
└── linux
└── linux-yocto.bb
└── linux-yocto
├── cfg
│ ├── main_frag_1.cfg
│ └── main_frag_1.scc
├── features
│ ├── main_feature_1.cfg
│ ├── main_feature_1.patch
│ └── main_feature_1.scc
└── patches
├── main_patch_1.patch
└── main_patch_1.scc
meta-bsp_layer
└── recipes-kernel
└── linux
└── linux-yocto.bbappend
└── linux-yocto
├── bsp
│ └── bsp_definition.scc
├── cfg
│ ├── bsp_frag_1.cfg
│ └── bsp_frag_1.scc
├── features
│ ├── bsp_feature_1.cfg
│ ├── bsp_feature_1.patch
│ └── bsp_feature_1.scc
└── patches
├── bsp_patch_1.patch
└── bsp_patch_1.scc
meta-main_layer/recipes-kernel/linux/linux-yocto.bb contains:
FILESEXTRAPATHS_prepend := "${THISDIR}:"
...
SRC_URI = "file://linux-yocto;type=kmeta \
"
Edit:
And meta-bsp_layer/recipes-kernel/linux/linux-yocto.bbappend:
FILESEXTRAPATHS_prepend := "${THISDIR}:"
...
SRC_URI += " file://linux-yocto;type=kmeta \
"
... end of edit
This means that after parsing the recipes, the SRC_URI will contain file://linux-yocto;type=kmeta twice, and FILESEXTRAPATHS=meta-bsp_layer/recipes-kernel/linux:meta-main_layer/recipes-kernel/linux:. The way I understand it, the do_unpack() task goes through the files in SRC_URI and for each of them, it searches FILESEXTRAPATHS for that file, and it takes the first one it finds. That means that in my case, only the files from meta-bsp_layer are taken, as its path is the first in FILESEXTRAPATHS, and the files from the meta-main_layer are ignored.
What I would like to achieve is that instead of taking the files from only the first path found in FILESEXTRAPATHS, do_unpack() would go through all of the paths in FILESEXTRAPATHS and merge the directories of the same name (cfg, features and patches) from both layers. Without that, I don't see any big benefits of using the Advanced Kernel Metadata mechanism. Could anyone advise how to get this done?
PS: I'm using Yocto Zeus.
I'm following this tutorial:
https://docs.yoctoproject.org/bitbake/bitbake-user-manual/bitbake-user-manual-hello.html
My bitbake version is 1.50.0 and BBPATH is following:
export BBPATH="/home/testusr/projects/bitbake_example/bitbake_sample_project/hello"
and I'm at the last step of the tutorial with following file structure:
/home/testusr/projects/bitbake_example/bitbake_sample_project $ tree
.
├── hello
│ ├── classes
│ │ └── base.bbclass
│ └── conf
│ ├── bblayers.conf
│ └── bitbake.conf
└── mylayer
├── conf
│ └── layer.conf
└── printhello.bb
The problem is when I try to build that:
/home/testusr/projects/bitbake_example/bitbake_sample_project/hello $ bitbake printhello
WARNING: Layer mylayer should set LAYERSERIES_COMPAT_mylayer in its conf/layer.conf file to list the core layer names it is compatible with.
Loading cache: 100% | | ETA: --:--:--
Loaded 0 entries from dependency cache.
WARNING: No bb files in default matched BBFILE_PATTERN_mylayer '^\/home\/testusr\/projects\/bitbake_example\/bitbake_sample_project\/mylayer/'
ERROR: Nothing PROVIDES 'printhello'
The warning is saying that there are no bb files under:
/home/testusr/projects/bitbake_example/bitbake_sample_project/mylayer/
but this file is clearly there as can be seen in tree output, so I'm totally confused.
The problem was with the tutorial in layer.conf file. It had:
BBFILES += "${LAYERDIR}/\*.bb"
Instead of:
BBFILES += "${LAYERDIR}/*.bb"
and that's why it couldn't be found.
I want to create a small application image. That image installed at a separate partition shall be mounted to /usr/local. I created a recipe like this:
inherit image
IMAGE_BASENAME = "appfs"
IMAGE_NAME_SUFFIX = ".appfs"
IMAGE_INSTALL_append = " app_lib "
IMAGE_INSTALL_append = " app_prog1 "
IMAGE_INSTALL_append = " app_prog2 "
IMAGE_INSTALL_append = " app_prog3 "
The result looks not bad, I can control the installation of the files by the prefix-Variable in the recipes of the application programms. So the will be installed in /bin, which means /usr/local/bin in the device.
But the image contains the whole directory structure of a root filesystem. Is there any comfortable way to generate the reduced directory structure of the /usr/local directory? Or do I need to clean it up by myself in a IMAGE_POSTPROCESS_COMMAND?
Kind regards
It sounds like what you want is a package or package group and not an image.
An image is a full distribution installation, usually using the poky distro.
This is built-in to yocto and unavoidable, as all dependencies of your app will need to be in the image. (runtimes, supporting libs, basic linux env, etc.)
In your tmp/deploy/rpm or tmp/deploy/deb directory you should have packages related to your apps.
Something that may help organize is to create a package group to aggregate your applications.
If you look at your tmp/deploy/images/<machine>/<imagename>.manifest file you will notice that in addition to the packages you specified, some additional ones such as libc6 and base-files are included.
You can remove them by naming the packages in ROOTFS_RO_UNNEEDED. This variable is used to remove recipes such as shadow and run-postinsts which are processed when creating a read-only image. You can add the other undesirable recipes to this list until your manifest matches exactly the packages you want included.
For example:
# tested with Yocto 2.6
IMAGE_FSTYPES = "ext4"
IMAGE_INSTALL = "my-go-app"
inherit image
# Don't run ldconfig on the rootfs
LDCONFIGDEPEND = ""
# If not building an RO filesystem, you can force use of ROOTFS_RO_UNNEEDED
# but be aware of the other packages included in the variable by image.bbclass
FORCE_RO_REMOVE = "1"
# Use _append because image.bbclass includes other packages
ROOTFS_RO_UNNEEDED_append = "base-files base-files-lic \
bash bash-lic \
libc6 libc6-lic \
ncurses-libtinfo libtinfo5 \
opkg-utils-lic \
ncurses-terminfo-base ncurses-lic \
run-postinsts-lic \
"
The resulting filesystem I get looks like this:
tmp
├── etc
│ ├── shells.rpmsave
│ ├── timestamp
│ └── version
├── lost+found [error opening dir]
├── usr
│ ├── bin
│ │ └── my-go-app
│ ├── lib
│ │ └── go
│ │ └── pkg
│ │ └── linux_arm64_dynlink
│ │ └── libstd.so
│ └── share
│ ├── common-licenses
│ │ └── license.manifest
│ └── licenses
│ ├── go-runtime
│ │ ├── generic_BSD-3-Clause
│ │ └── LICENSE
│ └── my-go-app
└── var
├── cache
└── lib
There may still be a few files left to clean up using a postprocess function:
postprocess_partial_image() {
rm -f ${IMAGE_ROOTFS}/etc/shells.rpmsave
rm -rf ${IMAGE_ROOTFS}/var
}
ROOTFS_POSTPROCESS_COMMAND += "postprocess_partial_image; "
I want to run a test for a function which accepts a Path to a file as input via an argument: function(some_path_to_file) via tox. The file I want to pass to the function cannot be created temporarily during test setup (what I usually do via pytests builtin tmpdir fixtures) but resides in the package <package>/data directory besides the test directory <package>/tests (the location <package>/tests/data would probably be better). Because tox runs the tests in a virtualenv it's not clear to me how to make the test data file available to the test. I know that I can define the base temporary directory of pytest with the --basedir option but I did not get it working with tox yet.
tl;dr
The problem was a conversion of some_path_to_file from Path to str (to pass it to sqlite3.connect(database inside the function) using Path.resolve(. No need to configure pytests --basedir option and tox in any way.
This tripped me up as well. The way I was able to solve it was to specify the full path of the text file I wanted the testing function to read relative to the base directory.
So for example, my directory tree looks like this:
.
├── __init__.py
├── my_package
│ ├── __init__.py
│ └── calculate_stats.py
├── my_package.egg-info
│ ├── PKG-INFO
│ ├── SOURCES.txt
│ ├── dependency_links.txt
│ ├── requires.txt
│ └── top_level.txt
├── bin
│ └── calculate_stats
├── requirements
│ ├── default.txt
│ └── development.txt
├── setup.py
├── test
│ ├── __init__.py
│ ├── test_calculate_stats.csv
│ ├── test_calculate_stats.txt
│ └── test_calculate_stats.py
└── tox.ini
In the file test_calculate_stats.py I have the following line:
assert (calculate_stats.calculate_stats_to_csv("test/test_calculate_stats.txt", "test/test_calculate_stats.csv") == 60)
The calculate_stats_to_csv function reads in the test/test_calculate_stats.txt file, calculates some stats, and outputs them to test/test_calculate_stats.csv
Initially I had just specified the input file to be test_calculate_stats.txt because it's in the same directory as the file containing the testing function -- that's when I ran into the error.
tox predfines a number of substitutions. The directory of the virtualenv is {envdir}, site-packages is at {envsitepackagesdir}. Pass a value from the command line to your test script like this:
[testenv]
commands = pytest --basedir={envsitepackagesdir}/mypackage