removing files in overlay does not remove them in rootfs when running make again - buildroot

I'm using buildroot 2018.05 and I'm noticing that if I remove some files in my overlay that the files are still being copied into the rootfs. I thought that any changes made in the overlay would automatically propagate when running make again without requiring me to do make clean and then make again?
I want to avoid having to re-download all packages and at least avoid re-compiling the toolchain each time I make changes to the overlay.

Buildroot cannot "know" when files have been removed from the overlay; it just copies the files from the overlay into the target rootfs.
Although Buildroot does do some tracking of where each file comes from, this information is not definitive so it can't be used to remove files again. In particular, files created or overwritten by the rootfs overlay or by a post-build script are (currently) not tracked.
It is true that the information in the (current) manual is misleading: "when changes to the root filesystem overlay, a post-build script or a post-image script are made, there is no need for a full rebuild". This is a bug in the manual and should be fixed. Feel free to submit a patch.

Related

How to specify buildroot build process variable to be called on make <package>-dirclean

Is there anyway I can just call into a define such as LIBFOO_DIRCLEAN, and just do what was implemented in the define?
Inside HOST_LIBFOO_INSTALL_CMDS, I copy files to the target directory, and would like the 'make package-dirclean' to delete what was copied into the target directory. 'make clean', would obviously do this(any many more), but that is much more than I want to do.
I see the following buildroot variables. LIBFOO_EXTRACT_CMDS, LIBFOO_CONFIGURE_CMDS, LIBFOO_BUILD_CMDS, HOST_LIBFOO_INSTALL_CMDS, LIBFOO_INSTALL_TARGET_CMDS, etc.
make foo-dirclean is a simple tool that just deletes the package build directory. In most cases, when the list of files installed by a package does not change over time (only files content changes) you can simply rebuild the package and the target directory will be rebuilt correctly.
If you want you can implement your own foo-myclean step that implements your own logic. However you must understand deleting files in the target directory is not supported by Buildroot and thus you are on your own.

Yocto/bitbake/OpenEmbedded: Best place for build/conf/local.conf's content?

I'm trying out yocto (2.0, jethro) and I want to build an image starting from core-image-minimal. This works fine.
Every website out there mention modifying the file build/config/local.conf with (some of) my customization. For example, the target machine (through MACHINE) or some global settings (through EXTRA_IMAGE_FEATURES).
I also need to modify some specific packages and the way to do it is to create a custom layer. So far so good.
What I don't understand is how to "save" all my configuration to version control. I want everything I change to be locate in files that I can commit so that anybody else can reproduce the exact same build (or even contribute to that project). Putting almost everything in build/config/local.conf goes against that goal; the file is under a "build" directory and so I can't just clone a git repo and start the building...
Is it really the way the yocto project works? Or am I missing a different configuration file where I need to put these settings? I though I could place all these in a custom layer but it does not seem to work...
Any idea or suggestion?
Thanks!
Thanks Ross, that clarified it!
Here's some notes about my file organization which I couldn't format into a comment to your answer.
Thanks. So all my custom configurations went into meta-mylayer/conf/distro/mylayer.conf
Almost all my customization went into a layer meta-mylayer, except:
DISTRO which is set in build/conf/local.conf. This is how you tell yocto what you want to build.
MACHINE which is also set in build/conf/local.conf. The reason is that the same image/distro combination could be built for different machines and thus this can't be hard-coded for every images.
Layers are manually added to build/conf/layers.conf. That's the last bit I wish I could moved to my DISTRO or something. For now the folders are git submodules and they are added using bitbake-layers add-layer.
In general everything in your local.conf that is "your project" should be moved to your own distro configuration (MACHINE, image features, package lists). Stuff like where DL_DIR is can be moved to a common site.conf if you wish. Eventually you should end up with a local.conf which just sets DISTRO and some other personal variables.

Creating a clean root filesystem (with overlay) without rebuilding all the packages

I currently have a buildroot configuration that outputs a U-Boot-wrapped root filesystem. Part of that process overlays files from a outside directory, which contains things like network interfaces, profile.d scripts, and some custom executables.
My question is this: what is the best way to recreate the root filesystem image, without rebuilding all the extra packages? Just deleting the output/images and output/target directories, followed by a "make all" doesn't seem to work. For the most common usage, I don't need to rebuild the toolchain, or any other packages. I just need the root filesystem recreated, with the overlay performed. However, there doesn't seem to be a good make target to do a clean on the output only, or a build of the target filesystem/images.
Thanks,
-D
Every time you invoke "make", the root filesystem image is completely re-created from the contents of output/target/, the post-build scripts are executed and the rootfs overlays are copied.
So if you make a change to a rootfs overlay, a change to a post-build script, or you add/remove stuff from output/target/, simply running "make" is sufficient.
However, if your aim is to remove output/target/ completely with the hope that it will reinstall all packages, then indeed this doesn't work, and we have good reasons for not supporting this, because there are many situations where this can give an incorrect result.

Is it common for a developer to keep their NAnt.exe.config file in version control?

Is it common for a developer to keep their NAnt global configuration file (NAnt.exe.config) in version control?
And should or shouldn't the the rest of the files in the NAnt installation be added to the ignore file of the version control system?
One use of version control is as a backup. If the only copy of NAnt.exe.config is on a hard disk that dies, it will take some effort to reconstruct it (along with everything else that disappeared and wasn't backed up).
From the corporate perspective, having all of the work in progress backed up is a method for preserving assets. The corporate owner of the source code asset is assured that the asset will not be destroyed.
When there is another backup strategy, then sometimes the rule of thumb is not to put anything into version control that should not be shared with other developers. Such as customized data relevant only to one user and/or machine, or confidential information.
I keep a copy of the NAnt code for the version I'm using. This includes the .config file. This is so my build system is safe from "it disappeared from the internet" events (unlikely, but still).
Beyond that I see no reason to keep it around on your code repository, unless for some reason you've modified it somehow. Most everything in NAnt can be overridden in build files, like the target framework and so on.

In MonoDevelop, should the "[project]/bin" directory be put under version control?

I have my VCS set up to ignore "[project]/bin". Is this directory essential to restore a project, or can it safely be ignored?
The "bin" directory can be safely ignored. MonoDevelop will create it when building the project.
If MonoDevelop recreates the bin directory when you delete it, then this is simply a matter of taste, otherwise you have to leave it in. (Rails happens to give up if you delete its logs).
Putting empty directories under version control can make the build process more readily apparent to those who glance at the repository, but it can also clutter and distract if you'd rather see only the essentials.