we are about to set up a production build server to maintain a embedded linux product. I know that I can specify a package version with PREFERRED_VERSION_<package> inside the recipe, but this may take a lot of time to lock every single tool in the image (i.e. grep, strace, ecc). This will be useful in case we need to rebuild the server (we could backup the whole server, but this wont be traced. Even commit everything to a git repo seems to be not so clever, correct me if I'm wrong).
Is there a way to get something like package-lock.json or an image footprint?
EDIT
What I'd like to achieve is a list of
PREFERRED_VERSION_<package0> = "xxx"
...
PREFERRED_VERSION_<packageN> = "xxx"
that I can use to replicate the image on a clear system (without any cached file). It seems that there is no such command to do that directly, instead it's possible to get a list of image packages and version by
bitbake <image> -s
and with a simple script generates what I'm looking for.
We have distro configuration, where you can specify all the version preferences. For example, the below samples are copied from poky.conf and bleeding.conf
PREFERRED_VERSION_glib-2.0 ?= "2.17.4"
PREFERRED_VERSION_glib-2.0-native ?= "2.17.4"
PREFERRED_VERSION_atk ?= "1.22.0"
PREFERRED_VERSION_pango ?= "1.21.2"
PREFERRED_VERSION_gtk+ ?= "2.13.3"
require conf/distro/include/poky-floating-revisions.inc
require conf/distro/poky.conf
DISTRO = "poky-bleeding"
DISTROOVERRIDES = "poky"
So you can define all the SRCREV and PREFERRED_VERSION in distro configuration or write a separate distro.inc and add it using require. And to use this for compilation, you can define,
DISTRO ?= "poky-bleeding"
in your conf/local.conf. This way you can control all the version and SRCREV for git based recipes at place.
Related
I just added a new package to my Yocto build (this one: How do you properly build gpiod applications from Yocto?), and it works fine as long as I am connected to the internet. The problem is that I am now trying to make the tarball needed to support an offline build.
A little bit about the setup: I am running Yocto Zeus. I work in a VM (Ubuntu 18.04) connected to the internet, but our build agents are not on the network, so we host a mirror. I have this mirror mounted on /mnt/download-mirror. I am trying to generate the tarball needed for the mirror on my VM.
This is the Yocto config that I am using:
BB_NO_NETWORK = "0"
INHERIT += "own-mirrors"
SOURCE_MIRROR_URL = "file:///mnt/download-mirror"
UNINATIVE_URL = "${SOURCE_MIRROR_URL}/uninative/2.7/"
BB_GENERATE_MIRROR_TARBALLS = "1"
SSTATE_MIRRORS = "file://.* file:///mnt/sstate-mirror/PATH"
But when I run Bitbake (I using command bitbake --runall fetch) it completes, but I don't get a tarball for the new package I added. I have looked for this tarball in the poky-downloads folder (the folder I would normally rsync to the mirror server), but it doesn't appear to be there.
Am I missing a configuration or something? I have all the configurations noted in https://wiki.yoctoproject.org/wiki/How_do_I#Q:_How_do_I_create_my_own_source_download_mirror_.3f
EDIT - I also tried setting DL_DIR ?= "/home/gen-ccm-root/Downloads" in my conf file, but my command still said there was nothing to do, so I think the issue might be my bitbake command. I have also tried bitbake -c mi-dev --runall="fetch" where mi-dev is my target (per 7.23.2 in https://www.yoctoproject.org/docs/current/mega-manual/mega-manual.html#setting-up-effective-mirrors) to no avail.
Hopefully someone can edit this and put in more information as to why, but I did manage to get it to generate a tarball.
Cleaned my VM which entailed deleting the sstate-cache and poky-downloads.
Modify the local.conf to have the following at the end (I think the DL_DIR is optional):
BB_NO_NETWORK = "0"
DL_DIR ="/home/gen-ccm-root/Downloads"
BB_GENERATE_MIRROR_TARBALLS = "1"
Run command: source oe-init-build-env build-dev
Run command: bitbake mi-dev --runonly=fetch
Mount the network share containing the download mirror.
Copy the files over:
sudo rsync -av --ignore-existing --exclude=*.done --exclude=git2 --exclude=svn /home/gen-ccm-root/Downloads/ /mnt/download-mirror/
I will update this as I learn more (I am still kind of a Yocto noob), but the issue appears to be related to a combination of Yocto not reparsing after changing the config, and the fact that the sstate-cache was still accessible and said everything was there.
Wait a bit, with BB_NO_NETWORK it should not go out and fetch stuff from the internet.
If I understand it correctly you want a source mirror for some other build machines locally.
One way would be to share your DL_DIR of the one machine, which sees the internet.
Another way would be to use a source mirror for which you need the BB_GENERATE_MIRROR_TARBALLS = "1" from above.
I use a web server to export my DL_DIR (which also contains the tarballs) and on the machines which should use it I use e.g. in local.conf or site.conf:
INHERIT += "own-mirrors"
SOURCE_MIRROR_URL = "http://mirror/source_mirror_zeus"
Just as a hint besides the sources you could also export SSTATE via SSTATE_MIRRORS, which will decrease build time a lot.
In my yocto source deploy/images/beaglebone dir is nearly 100GB so i want to free that memory
Please help me how can I delete that deploy dir either manually or via command line
I want to clean all images(*.tar.gz,*.sdcard, *.ubifs) of previous compilation in yocto deploy/images/beaglebone/
Maybe when you are with 100GB in the deploy directory, things have gone too far already.
Check your IMAGE_FSTYPES variable. My experience says it is safe to delete all images of these files that are not symlinks, or symlinks targets. Avoid the last one generated to avoid breaking the last build link, and any related with bootloaders and configuration files, as they could be rarely regenerated.
If you are keeping more than one build with the same set of layers, then you can use a common download folder for builds.
DL_DIR ?= "common_dir_across_all_builds/downloads/"
And afterwards:
To keep your /deploy clean:
RM_OLD_IMAGE: Reclaims disk space by removing previously built versions of the same image from the images directory pointed to by the DEPLOY_DIR variable.Set this variable to "1" in your local.conf file to remove these images:
RM_OLD_IMAGE = "1"
IMAGE_FSTYPES Remove the image types that you do not plan to use, you can always enable a particular one when you need it:
IMAGE_FSTYPES_remove = "tar.bz2"
IMAGE_FSTYPES_remove = "rpi-sdimg"
IMAGE_FSTYPES_remove = "ext3"
For /tmp/work, do not need all the workfiles of all recipes. You can specify which ones you are interested in your development.
RM_WORK_EXCLUDE:
With rm_work enabled, this variable specifies a list of recipes whose work directories should not be removed. See the "rm_work.bbclass" section for more details.
INHERIT += "rm_work"
RM_WORK_EXCLUDE += "home-assistant widde"
try this from your build root rm -fr deploy/images. Here is a good discussion on the topic
I just removed the files manually like below
1. goto build/deploy/images/beaglebone
2. $ ll : you will find the softlinks of rootfs with time like
......*20170811091521.rootfs.tar.gz
......*-20170811091521.rootfs.sdcard etc
3. Dont delete recently compiled files. except these this you can remove all *.tar.gz, *.sdcard,*.ext4 manually like below,
4. rm beaglebone-20170811091521.rootfs.tar.gz
rm beaglebone-20170811091521.rootfs.sdcard
rm beaglebone-20170811091521.rootfs.ext4 etc.
I'm new to yocto. I'm trying to learn how the packages are added, how to create new layers and so on... just poking around. Started by cloning poky and playing around.
To my understanding, the bblayers.conf file is critical to the project configuration and what you end up building (what layers and packages go into your final image).
This might be the wrong assumption, but I also have a feeling that the build/ folder is where things you build (bitbake) stay. Images, lots of things needed to build them, a big cache of stuff... You can delete it and rebuild it if you somehow broke it. Or you can just copy everything without the build/ folder and continue working on a different computer.
Apparently it's not quite the case. The build/conf/ folder has the important .conf files like the bblayers.conf.
Can someone explain why is this the case? Is there an elegant way to separate the project config and the build folder?
There are a couple layers to the Yocto Project, mainly:
-BSPDIR: TOPDIR (build),sources,setup-environment
-BSPDIR/setup-environment: initial all the variable to for bitbake;
-BSPDIR/sources: meta-data/
-TOPDIR: conf/ sstate-cache/ cache/ tmp/ downloads/
-TOPDIR/downloads: recipe fetched packages;
-TOPDIR/conf/ : stored all the configuration. Mainly bblayers.conf, local.conf, sanity_info;
-TOPDIR/conf/bblayers.conf: stored all the path to meta-data that will be loaded;
-TOPDIR/conf/local.conf: configuration to build
-TOPDIR/conf/sanity_info: path double check to make sure that all the path used in the last compile match the current compile;
-TOPDIR/tmp/: Where all the compiling and building work happen
In BSPDIR/sources/poky/meta/conf/bitbake.conf
sources/poky/meta/conf/bitbake.conf:TMPDIR ?= "${TOPDIR}/tmp"
sources/poky/meta/conf/bitbake.conf:PERSISTENT_DIR = "${TOPDIR}/cache"
sources/poky/meta/conf/bitbake.conf:DL_DIR ?= "${TOPDIR}/downloads"
sources/poky/meta/conf/bitbake.conf:SSTATE_DIR ?= "${TOPDIR}/sstate-cache"
TOPDIR is where you initialize when run setup-environment or oe-init-build-env; All the other bitbake configuration environment variable can be changed based on your need in conf/local.conf;
e.g. modify conf/local.conf to change the downloads directory from TOPDIR/downloads;
DL_DIR ?= "/home/downloads/"
To create new layer, please watch this video: https://www.youtube.com/watch?v=3HsaoVqX7dg
You might have followed the Yocto Project Quick Start Guide.
The earliest step in yocto after installing (cloning git repositories and installing packages) is to create your OE (OpenEmbedded) environment, which is done via:
source oe-init-build-env
This automatically creates and leads you to the build folder.
Matter that you can give any directory of your system as parameter for this call (Reference Manual - Build Overview):
source oe-init-build-env [build_dir]
⤑ This is also the step, where your 'project config' is separated from the actual build folder.
⤑ As you assumed, in practice you would at most copy the layers and not the build folder. Even better is to leave sources from others in their git repositories and only copy and maintain your own layers.
it is true an issue in the modern Yocto build system.
file bblayers.conf has to be synthesized based on MACHINE and DISTRO information using all provided (usually with the help of repo manifest file) layers by: collecting data from each available layer file layer.conf as well as conf/machine, conf/distro, images.
Instead bblayers.conf is usually copied over from the base layer conf/bblayers.conf location with the help of setup-environment script.
this approach provides no "one click" buildable environment but require maintainer/developer to look into readme to identify what layers are missing to be added to the build/conf/bblayers.conf.
I am having an issue where a recipe i'm using no longer has the variable libdir defined. It appears to only have libdir_native.
This recipe i'm using is poco-1.7.5 for Morty from openembedded so I assume the recipe should work properly.
As a result of the missing libdir variable none of the installed files are being packaged which is screwing up my build.
In the short term i've been able to fix the problem by creating an append file which makes libdir = "${libdir_native} but this doesn't seem like it should be necessary.
The only thing I can think of is that the Bitbake.conf file is not being source properly by Bitbake (or the wrong .conf is being used).
Any suggestions would be appreciated.
Based on the comments this seems to be the problem: Poco upstream installs libraries into /usr/lib/ but the yocto packaging expects them to be in ${libdir} which may be different from /usr/lib/.
The most common reason (for cmake recipes) for this is that the upstream project does not support CMAKE_INSTALL_LIBDIR. Check if the upstream build system has some alternative means for specifying libdir -- this is surprisingly common with cmake projects. If not, you could add support for CMAKE_INSTALL_LIBDIR in the upstream build system (and add a patch to your recipe).
An alternative hack would be to add a do_install_append() that checks if ${libdir} is not /usr/lib/, and moves everything from ${D}/usr/lib/ to ${D}${libdir} in that case.
I am trying to remove some open source packages that have GPLv3 licensing attached to them that we cannot distribute as part of the final production image . While these packages can be removed using "INCOMPATIBLE_LICENSE=GPLv3" in local.conf but it removes every package that's tagged GPLv3.
I have used
PREFERRED_VERSION_recipename=(GPLv2 recipe_version)
and downgraded all the GPLv3 packages to GPLv2 that we want to push into the final image. But there are several GPLv3 packages that we want to keep as part of the debug image for example M4, make, gdb, bison. There is no need for these packages to be in the final production image but these cannot be removed from the build completely as they are needed during the build/debug.
SO, How do i remove these packages/recipe from the final rootfs image while letting them being as part of the build process and letting them build. In other words I want to let the bitbake build them from the GPLv3 source but keep them out the of the final rootfs image.
I did look at the poky/lib/oe/manifest.py and rootfs.py create function ==> where the final rootfs manifest is being created but couldn't figure out what exactly needs to be set from the bitbake. Does the do_rootfs need to be overwritten. if so what variables need to be set for overriding the final manifest.
There's no single way of doing what you want. One way is to have your production image consists of what you need to deliver, and then let your dev-image require production-image.bb. In that way you can easily extend the dev image with your extra packages.
I've got a similar issue, as I want gdbserver in my dev-image. (Sofar, nothing else that's GPLv3 is necessary / wanted in my dev image). What I've done, is to set
INCOMPATIBLE_LICENSE = "GPL-3.0 LGPL-3.0 AGPL-3.0"
INCOMPATIBLE_LICENSE_pn-gdb = ""
That basically allows gdb to be built, but nothing else that's GPLv3 licensed. Then, my images inherit a company-image.bbclass instead of directly inheriting image.bbclass. In company-image.bbclass, I've got a function like:
python () {
for p in d.getVar('IMAGE_INSTALL', True).split():
if p in ['gdb', 'gdbserver'] and not d.getVar('IMAGE_BASENAME', True) == 'company-dev-image':
bb.fatal("gdb/gdbserver is not allowed in this image!")
}
That will ensure that gdb and gdbserver can only be installed in company-dev-image; thus, there's no risk of having them distributed in the production image.
I would like to provide an example that is , if we would like to remove the "openssh" package from build, needs to be updated "local.conf" file as mentioned below.
Please the add the following line into BUILDFOLDER/conf/local.conf file
PACKAGECONFIG_remove = "openssh"