Task is not re-triggered even if variables in vardeps change - yocto

I have an issue related to vardeps where I am making a task dependent on some variables.
I have created some new variables e.g., NEW_VARIABLE, added them to BB_ENV_EXTRAWHITE. In some recipes, I wrote my own implementation of some tasks that are dependent on these new variables, and for this dependency to work I added e.g., do_install[vardeps] = "NEW_VARIABLE", so I am now expecting that every time I change this NEW_VARIABLE and perform e.g., bitbake recipename, the do_install task should run. I checked the task signature and I see the NEW_VARIABLE there.
Let's assum I have two possible values for this variable. When I set the variable for the first time "value1", i.e., the first build, everything works and there is no problem. When I change its value to the other value "value2" not used before and build the recipe again, the do_install will also run and no problem occurs. The problem is however, if I set the variable again to the old value "value1", and I execite bitbake recipename again. The do_install will not be re-triggered, and this leads to some wrong/old data located in work directory, and also produced in the image.
I tried setting BB_DONT_CACHE, as I understood in an old question that the problem might be that the recipe needs to be parsed again, however this did not work at all.
I do not want to always run the tasks when I perform a new build, i.e., do_install[[nostamp] = "1" so this solution can not be regarded. I just want it to run again every time I change this NEW_VARIABLE.
Is what I am expecting a normal behavior? Or Yocto does not work this way?

I faced same issue all day, also tried vardeps BB_DONT_CACHE etc but then I changed into using SRC_URI but still got same behavior. Like you I build a separate recipe "bitbake name" and look in the work image/ for the recipe output. For me it turns out that I just think that this will be updated just as if I make other new changes. But once I change input to something old the state cache kicks in and the work dir is left alone - fooling me to think it does not work properly. But when I build the final image-base it is properly populated. I guess one need to force build or disregard compiler cache to get the output this way.

Related

Assign build clean option at queue time

i have a bitbucket classic build pipeline and i need to have the ability to set the build clean option at queue time. it seems like this used to be possible via the Build.Clean variable but that has since been deprecated.
When editing a build pipeline the Clean option uses an editable drop down but anytime you try and type something, it erases what you just wrote. i would like to set this option to a variable like $(CleanBuild)
Assign build clean option at queue time
Indeed, the variable Build.Clean is already deprecated. But the document Use predefined variables provided another variable Build.Repository.Clean, which will help us to clean the Sources:
Besides, if you want to clean other options fields, like All build directories:
I do not believe there is a way to assign the clean options at queue-time. Even if we use deprecated Build.Clean variable, we still can clear Sources only.
You could check the similar thread for some more details.
Hope this helps.

how to refresh application.conf at runtime

I update a key value in application.conf by setting the environment variable
play.http.secret.key=${?MY_SECRET_KEY}
But it still loads the previous value.
After a system reboot the change takes effect.
Is there a way to refresh the config file without rebooting?
Try the following:
Given in a file called sample1.conf:
a {
b {
c = 30
d = ["Red", "Orange", "Green", "Blue"]
}
}
If you wish to change a property be sure to change it as system property first and call invalidate caches then load again. This is also what allows you to override on the command line.
System.setProperty("a.b.c", "100")
ConfigFactory.invalidateCaches()
val config = ConfigFactory.load("sample1")
config.getDouble("a.b.c") should be (100.0)
Don't know if this would work in all scenarios. So it may or may not work with your application of choice.
AFAIK, there is no Out of the box mechanism to do such thing.
As it turns out, when you start Play through the "sbt run" command, it starts in Dev mode, creating two ClassLoaders, one for your code, and the other one for the immutable code, like libraries, dependencies, and the framework code itself. This is done this way as to provide the hot deploy feature, killing and reloading the first CL.
Because the application.conf is loaded by Play on start up, I would think that it is loaded within the fixed ClassLoader, hence no reload is possible.

Bitbake: "The metadata is not deterministic and this needs to be fixed"

I'm building a Bitbake recipe and getting the following error message:
ERROR: When reparsing virtual:native:/path/to/poky/meta/recipes-devtools/cve-check-tool/cve-check-tool_5.6.4.bb.do_populate_cve_db, the basehash value changed from 0b637979bcb5db4263f9ed97497a6330 to bcd28a5efe646ed4d327fefa349f889c. The metadata is not deterministic and this needs to be fixed.
This reproduces in a clean build (after bitbake -c cleanall -c cleansstate <recipe>).
What is the reason for this error? The recipe has not been modified from the upstream version.
the fix I used is go to that recipe and add an empty line in the end, that will help bitbake re-recognize the recipe
The following is the yocto patch that adds this diagnostic message
https://patchwork.openembedded.org/patch/133517/
Here is the commit message, explaining its reasons and a possible way to get further details for the problem:
Bitbake can parse metadata in the cooker and in the worker during
builds. If the metadata isn't deterministic, it can change between
these two parses and this confuses things a lot. It turns out to be
hard to debug these issues currently.
This patch ensures the basehashes from the original parsing are passed
into the workers and that these are checked when reparsing for
consistency. The user is shown an error message if inconsistencies are
found.
There is debug code in siggen.py (see the "Slow but can be useful for
debugging mismatched basehashes" commented code), we don't enable this
by default due to performance issues. If you run into this message,
enable this code and you will find "sigbasedata" files in tmp/stamps
which should correspond to the hashes shown in this error message.
bitbake-diffsigs on the files should show which variables are
changing.
Signed-off-by: Richard Purdie
I stumbled onto a similar error. For me it happened in the do_install routine. My aim was to store a copy of a file with a ${DATETIME} attached to its name referring to the build time in a specific place.
Apparently the recipes get parsed multiple times during a build and since the time has changed by the second time it got parsed a different ${DATETIME} value was inserted and thus the metadata was recognized as changed.
Simplest solution: touch <recipename>. Then run cleansstate on your recipe. Then go on as usual from here.
It helps for me to remove a shared cache for a recipe:
bitbake recipename -c cleansstate
And then, everything works like a charm.
This happens because tasks are evaluated twice, the first time by the cooker, and the second time by bitbake worker. The task hash is calculated twice and if it will not match, meta is considered unstable. Base hash is calculated from variables that are used in the task script. So if you for example use time-related variables like DATETIME, you will get this hash mismatch error. To avoid this error you need to exclude variables from the hash calculation, with VarFlags.
do_something[vardepsexclude]="DATETIME"
here you can find more elaborate explanation.
https://www.kc8apf.net/2017/01/tired-of-taskhash-mismatch/
yocto bitbake hashes
For what it's worth, doing this cleared it up for me:
devtool modify <recipe>
devtool reset <recipe>
this also happens if you modify the recipe while you're building...
this means that the error you stated goes off if you edit a recipe file while a build is in progress. If this is your case just finish your edit and re-launch a build, it should work. Happened to me and solved this way.
This happens when a recipe contains changing data and you try to rebuild it, one way to avoid this is to delete the tmp/ directory [not a good solution] but it is the least possible solution.

Find missing but required files for running a script

While deploying packages of Matlab code using Matlab 2015 I encountered the problem of gathering all required files from my repository to run a certain file or set of files. Matlab has a method for simplifying this process, matlab.codetools.requiredFilesAndProducts.
However, sometimes some files are missing in the repository (either because I got a package from someone else who was not that careful or because it was not checked in in the repository).
When running the code one would get of course an error:
Undefined function or variable 'XXX'.
However, this may take long to fix (running takes long, you would have to repeat for every missing file). Therefore I thought to use the function above. Unfortunately, it only lists existing files in the output (I tested this). Functions that are called from your code, but that are not present in the current path are omitted by matlab.codetools.requiredFilesAndProducts.
My problem: I would like to get a list of all files that are required by running a certain file but are not present in the current path so that I can find them and add them to my collection.
I know that this must be an iterative process because the missing files could themselves call other missing files and I know that there would be false positives, some of these items could be unknown variables instead, and I know that the missing files would only have a name, no path (of course).
My question: What is the easiest way to find a list of potentially missing files of my code in one go?
Please note that function depfun has been removed in recent versions of Matlab.

Global launch configuration in Eclipse?

This seems like a simple thing, but I can't find an answer in the existing questions:
How do you add a global argument to all your present and existing run or debug configurations? In my case, I need a VM argument, but I see that this could be useful for runline arguments as well.
Basically, every time I create a unit test I need to create a configuration (or run, which creates one), and then manually edit each one with the same VM argument. This seems silly for such a good tool.
This is not true. You can add the VM arguments to the JRE definition. This is exactly what it is for. I use it myself so that assertions are enabled and heap is 1024mb on every run, even future ones.
Ouch: 7-years bug, asking for running configuration template, precisely for that kind or reason.
This thread proposes an interesting workaround, based on duplicating a fake configuration based on string substitution:
You can define variables in Window->Preferences->Run/Debug->String Substitution. For example you can define a projectName_log4j variable with the
correct -Dlog4j.configuration=... value.
In a run configuration you can use ${projectName_log4j} and you don't have to remember the real value.
You can define a project-specific "empty" run configuration.
Set the project and the arguments fields in this configuration but not the main class. If you have to create a new run configuration for this project select this one and use 'Duplicate' from its popup-menu to copy this configuration.
You have to simply set the main class and the program arguments.
Also you can combine both solutions: use a variable and define an "empty"
run configuration which use this variable. The great advantage in this case
is when you begin to use a different log4j config file you have to change
only the variable declaration.
Not ideal, but it may alleviate your process.