Make yocto skip a recipe instead of stopping - yocto

I have a recipe that does a check during parsing. What I would like to do is instead of issuing a warning or stopping with an error, I would like to make yocto completely ignore the recipe as if it was never there. It could still error out if some other recipe RDEPENDS on it, but otherwise parsing would be successful.
Is this possible to do?

EDIT: I don't see a way to do it.
But you can "hide" specific recipe(s) using the BBMASK variable. The value is regexp for masking specific files or paths. You can also mask a whole directory.
We are using that mechanism and the variable is set in configuration file (distro configuration in our case, but it may be in a different configuration file).
You can find more information in the documentation for that variable: https://www.yoctoproject.org/docs/latest/mega-manual/mega-manual.html#var-BBMASK
Some examples copied from the linked documentation:
BBMASK += "/meta-ti/recipes-misc/ meta-ti/recipes-ti/packagegroup/"
BBMASK += "/meta-oe/recipes-support/"
BBMASK += "/meta-foo/.*/openldap"
BBMASK += "opencv.*\.bbappend"
BBMASK += "lzma"

When bitbake launches, it first parses everything it can in order to figure out what does it have and are there any obvious errors. Only after this stage it analyzes what have you asked it to do. So if you have syntax errors there is no other way to avoid it except don't add the layer that contains invalid recipe to bblayers.conf.

Yes, you can, by raising bb.parse.SkipRecipe exception
python() {
...
if ... :
raise bb.parse.SkipRecipe("Message")
...
}
I don't find it well documented, but google does return some assuring results for this.

Related

Task is not re-triggered even if variables in vardeps change

I have an issue related to vardeps where I am making a task dependent on some variables.
I have created some new variables e.g., NEW_VARIABLE, added them to BB_ENV_EXTRAWHITE. In some recipes, I wrote my own implementation of some tasks that are dependent on these new variables, and for this dependency to work I added e.g., do_install[vardeps] = "NEW_VARIABLE", so I am now expecting that every time I change this NEW_VARIABLE and perform e.g., bitbake recipename, the do_install task should run. I checked the task signature and I see the NEW_VARIABLE there.
Let's assum I have two possible values for this variable. When I set the variable for the first time "value1", i.e., the first build, everything works and there is no problem. When I change its value to the other value "value2" not used before and build the recipe again, the do_install will also run and no problem occurs. The problem is however, if I set the variable again to the old value "value1", and I execite bitbake recipename again. The do_install will not be re-triggered, and this leads to some wrong/old data located in work directory, and also produced in the image.
I tried setting BB_DONT_CACHE, as I understood in an old question that the problem might be that the recipe needs to be parsed again, however this did not work at all.
I do not want to always run the tasks when I perform a new build, i.e., do_install[[nostamp] = "1" so this solution can not be regarded. I just want it to run again every time I change this NEW_VARIABLE.
Is what I am expecting a normal behavior? Or Yocto does not work this way?
I faced same issue all day, also tried vardeps BB_DONT_CACHE etc but then I changed into using SRC_URI but still got same behavior. Like you I build a separate recipe "bitbake name" and look in the work image/ for the recipe output. For me it turns out that I just think that this will be updated just as if I make other new changes. But once I change input to something old the state cache kicks in and the work dir is left alone - fooling me to think it does not work properly. But when I build the final image-base it is properly populated. I guess one need to force build or disregard compiler cache to get the output this way.

Yocto - exclude files from -dev package

I have a great problem moving some header files from FILES_${PN}-dev to a custom dev-internal package.
In OpenEmbedded documentation there is explicitly stated that there is "no actual support for explicitly excluding files from packaging".
I tried this:
FILES_${PN}-dev = ""
PACKAGES += "${PN}-dev-internal"
FILES_${PN}-dev-internal = "${includedir}/<my-pattern>.h"
FILES_${PN}-dev = "<original-content>"
but it seems that the first defined package captures first anyway.
Is there any known workaround for this? Except of naming everything explicitly in both packages content which is highly, really undesirable.
but it seems that the first defined package captures first anyway.
Yes. So put PN-dev-internal before PN-dev in PACKAGES. This might work:
PACKAGES =+ "${PN}-dev-internal"
If not, PACKAGE_BEFORE_PN is useful.
You can probably change the component installation config to better separate the headers (e.g. place them in different directories).

Bitbake: "The metadata is not deterministic and this needs to be fixed"

I'm building a Bitbake recipe and getting the following error message:
ERROR: When reparsing virtual:native:/path/to/poky/meta/recipes-devtools/cve-check-tool/cve-check-tool_5.6.4.bb.do_populate_cve_db, the basehash value changed from 0b637979bcb5db4263f9ed97497a6330 to bcd28a5efe646ed4d327fefa349f889c. The metadata is not deterministic and this needs to be fixed.
This reproduces in a clean build (after bitbake -c cleanall -c cleansstate <recipe>).
What is the reason for this error? The recipe has not been modified from the upstream version.
the fix I used is go to that recipe and add an empty line in the end, that will help bitbake re-recognize the recipe
The following is the yocto patch that adds this diagnostic message
https://patchwork.openembedded.org/patch/133517/
Here is the commit message, explaining its reasons and a possible way to get further details for the problem:
Bitbake can parse metadata in the cooker and in the worker during
builds. If the metadata isn't deterministic, it can change between
these two parses and this confuses things a lot. It turns out to be
hard to debug these issues currently.
This patch ensures the basehashes from the original parsing are passed
into the workers and that these are checked when reparsing for
consistency. The user is shown an error message if inconsistencies are
found.
There is debug code in siggen.py (see the "Slow but can be useful for
debugging mismatched basehashes" commented code), we don't enable this
by default due to performance issues. If you run into this message,
enable this code and you will find "sigbasedata" files in tmp/stamps
which should correspond to the hashes shown in this error message.
bitbake-diffsigs on the files should show which variables are
changing.
Signed-off-by: Richard Purdie
I stumbled onto a similar error. For me it happened in the do_install routine. My aim was to store a copy of a file with a ${DATETIME} attached to its name referring to the build time in a specific place.
Apparently the recipes get parsed multiple times during a build and since the time has changed by the second time it got parsed a different ${DATETIME} value was inserted and thus the metadata was recognized as changed.
Simplest solution: touch <recipename>. Then run cleansstate on your recipe. Then go on as usual from here.
It helps for me to remove a shared cache for a recipe:
bitbake recipename -c cleansstate
And then, everything works like a charm.
This happens because tasks are evaluated twice, the first time by the cooker, and the second time by bitbake worker. The task hash is calculated twice and if it will not match, meta is considered unstable. Base hash is calculated from variables that are used in the task script. So if you for example use time-related variables like DATETIME, you will get this hash mismatch error. To avoid this error you need to exclude variables from the hash calculation, with VarFlags.
do_something[vardepsexclude]="DATETIME"
here you can find more elaborate explanation.
https://www.kc8apf.net/2017/01/tired-of-taskhash-mismatch/
yocto bitbake hashes
For what it's worth, doing this cleared it up for me:
devtool modify <recipe>
devtool reset <recipe>
this also happens if you modify the recipe while you're building...
this means that the error you stated goes off if you edit a recipe file while a build is in progress. If this is your case just finish your edit and re-launch a build, it should work. Happened to me and solved this way.
This happens when a recipe contains changing data and you try to rebuild it, one way to avoid this is to delete the tmp/ directory [not a good solution] but it is the least possible solution.

Yocto: how to remove/blacklist some dependency from RDEPENDS of a package?

I have a custom machine layer based on https://github.com/jumpnow/meta-wandboard.
I've upgraded the kernel to 4.8.6 and want to add X11 to the image.
I'm modifying to image recipe (console-image.bb).
Since wandboard is based on i.MX6, I want to include the xf86-video-imxfb-vivante package from meta-fsl-arm.
However, it fails complaining about inability to build kernel-module-imx-gpu-viv. I believe that happens because xf86-video-imxfb-vivante DEPENDS on imx-gpu-viv which in turn RDEPENDS on kernel-module-imx-gpu-viv.
I realize that those dependencies have been created with meta-fsl-arm BSP and vanilla Poky distribution. But those things are way outdated for wandboard, hence I am using the custom machine layer with modern kernel.
The kernel is configured to include the Vivante DRM module and I really don't want the kernel-module-imx-gpu-viv package to be built.
Is there a way to exclude it from RDEPENDS? Can I somehow swear my health to the build system that I will take care of this specific run-time dependency myself?
I have tried blacklisting 'kernel-module-imx-gpu-viv' setting PNBLACKLIST[kernel-module-imx-gpu-viv] in my local.conf, but that's just a part of a solution. It helps avoid build failures, but the packaging process still fails.
IIUC you problem comes from these lines in img-gpu-viv recipe:
FILES_libgal-mx6 = "${libdir}/libGAL${SOLIBS} ${libdir}/libGAL_egl${SOLIBS}"
FILES_libgal-mx6-dev = "${libdir}/libGAL${SOLIBSDEV} ${includedir}/HAL"
RDEPENDS_libgal-mx6 += "kernel-module-imx-gpu-viv"
INSANE_SKIP_libgal-mx6 += "build-deps"
I would actually qualify this RDEPENDS as a bug, usually kernel module dependencies are specified as RRECOMMENDS because most modules can be compiled into the kernel thus making no separate package at all while still providing the functionality. But that's another issue.
There are several ways to fix this problem, the first general route is to tweak RDEPENDS for the package. It's just a bitbake variable, so you can either assign it some other value or remove some portion of it. In the first case it's going to look somewhat like this:
RDEPENDS_libgal-mx6 = ""
In the second one:
RDEPENDS_libgal-mx6_remove = "kernel-module-imx-gpu-viv"
Obviously, these two options have different implications for your present and future work. In general I would opt for the softer one which is the second, because it has less potential for breakage when you're to update meta-fsl-arm layer, which can change imx-gpu-viv recipe in any kind of way. But when you're overriding some more complex recipe with big lists in variables and you're modifying it heavily (not just removing a thing or two) it might be easier to maintain it with full hard assignment of variables.
Now there is also a question of where to do this variable mangling. The main option is .bbappend in your layer, that's what appends are made for, but you can also do this from your distro configuration (if you're maintaining your own distro it might be easier to have all these tweaks in one place, rather than sprayed into numerous appends) or from your local.conf (which is a nice place to quickly try it out, but probably not something to use in longer term). I usually use .bbappend.
But there is also a completely different approach to this problem, rather than fixing package dependencies you can also fix what some other package provides. If for example you have a kernel configured to have imx-gpu-viv module built right into the main zimage you can do
RPROVIDES_kernel-image += "kernel-module-imx-gpu-viv"
(also in .bbappend, distro configuration or local.conf) and that's it.
In any case your approach to fixing this problem should reflect the difference between your setup and recipe assumptions. If you do have the module, but in a different package, then go for RPROVIDES, if you have some other module providing the same functionality to libgal-mx6 package then fix libgal-mx6 dependencies (and it's better to fix them, meaning not only drop something you don't need, but also add things that are relevant for your setup).

Find missing but required files for running a script

While deploying packages of Matlab code using Matlab 2015 I encountered the problem of gathering all required files from my repository to run a certain file or set of files. Matlab has a method for simplifying this process, matlab.codetools.requiredFilesAndProducts.
However, sometimes some files are missing in the repository (either because I got a package from someone else who was not that careful or because it was not checked in in the repository).
When running the code one would get of course an error:
Undefined function or variable 'XXX'.
However, this may take long to fix (running takes long, you would have to repeat for every missing file). Therefore I thought to use the function above. Unfortunately, it only lists existing files in the output (I tested this). Functions that are called from your code, but that are not present in the current path are omitted by matlab.codetools.requiredFilesAndProducts.
My problem: I would like to get a list of all files that are required by running a certain file but are not present in the current path so that I can find them and add them to my collection.
I know that this must be an iterative process because the missing files could themselves call other missing files and I know that there would be false positives, some of these items could be unknown variables instead, and I know that the missing files would only have a name, no path (of course).
My question: What is the easiest way to find a list of potentially missing files of my code in one go?
Please note that function depfun has been removed in recent versions of Matlab.