Yocto - Create and populate a separate /home partition - yocto

I'm creating quite a simple Yocto image based on x86.
I want the / file system to be readonly, so I set the
IMAGE_FEATURES_append = " read-only-rootfs "
in a custom copy of the original core-image-minimal.bb. I do want to have the /home writable and on a separate partition, though.
So, I'm adding a line
part /home --ondisk sda --fstype=ext4 --label home --align 1024 --size 600
in genericx86.wks. This creates the actual /home partition in the final wic image, but it naturally does not hold any data, as there's no corresponding rootfs for it. This leads to the following quite expected message after boot: No directory, logging in with HOME=/.
There's surprisingly little info about this on the internet. There's this explanation:
It's much more simpler to create or modify build
recipes to prepare one rootfs directory per partition.
I just wish there was any reference in the documentation or example on how to achieve that.
I can see that the partitions are being populated by python scripts (plugins) like rootfs.py, and that the image parameters like IMAGE_ROOTFS_SIZE are specified in mentioned image recipe files like the genericx86.wks, but this is just not enough for me to connect these pieces together.
I've read the creating-partitioned-images-using-wic and the linked openembedded kickstart manuals, there are no clues there.
Appreciate someone's kind help.

With WIC you can do something like this:
custom.wks.in:
...
part / --source rootfs --ondisk sda --fstype=ext4 --label system --exclude-path=home/
part /home --source rootfs --rootfs-dir=${IMAGE_ROOTFS}/home --ondisk sda --fstype=ext4 --label home
...
Note it is important if you want to use ${IMAGE_ROOTFS} in WKS file to name it with .in suffix.

Related

Where on disk is the BIOS file used by Simics?

(I saw one of my previous posts didn't actually answer the "where's the BIOS file used by simics?" question, so I renamed the previous one and am pulling that question out and making it standalone here.)
I can see the BIOS code for a default "targets\qsp-x86\firststeps.simics" invocation by just stepping through the debugger from the start. But if I want to see the full binary, is there a specific file somewhere I can look at?
you can check "bios" attribute on motherboard image:
simics> board.mb->bios
"%simics%/targets/qsp-x86/images/SIMICSX58IA32X64_1_0_0_bp_r.fd"
You can specify what BIOS image to use by bios_image script parameter to qsp-clear-linux.simics scripts.
Help info for the script:
$ ./simics -h targets/qsp-x86/qsp-clear-linux.simics
System:
bios_image - existing file or NIL
BIOS file.
Default value:
"%simics%/targets/qsp-x86/images/SIMICSX58IA32X64_1_0_0_bp_r.fd"
you can run with your own BIOS like this:
$ ./simics -e '$bios_image=my-bios.bin' targets/qsp-x86/qsp-clear-linux.simics
Now the BIOS is not quite handled consistently with some other things. Typically in Simics, disks and similar things are images. You can list them using list-persistent-images and resolve locations using lookup-file:
simics> list-persistent-images
┌─────────────────────┬────────────┬───────────────────────────────────────────────────────┐
│Image │Unsaved data│File(s) (read-only/read-write) │
├─────────────────────┼────────────┼───────────────────────────────────────────────────────┤
│board.disk0.hd_image │ no│%simics%/targets/qsp-x86/images/cl-b28910-v2.craff (ro)│
│board.disk1.hd_image │ no│ │
│board.mb.sb.spi_image│ yes│%simics%/targets/qsp-x86/images/spi-flash.bin (ro) │
└─────────────────────┴────────────┴───────────────────────────────────────────────────────┘
simics> lookup-file "%simics%/targets/qsp-x86/images/spi-flash.bin"
"/disk1/simics-6/simics-qsp-x86-6.0.47/targets/qsp-x86/images/spi-flash.bin"
The BIOS in the QSP is just loaded straight into target memory for execution. Which is a bit of a cheat for convenience.
Upon searching around, I found the following folder:
C:\Users\yourusername\AppData\Local\Programs\Simics\simics-qsp-x86-6.0.44\targets\qsp-x86\images
Inside that folder are the following 3 files:
SIMICSX58IA32X64_1_0_0_bp_r.fd
SIMICSX58IA32X64-ahci.fd
spi-flash. bin
Both SIMICSX58IA32X64_1_0_0_bp_r. fd and SIMICSX58IA32X64-ahci.fd have UEFI filevolume headers at the start, and a seeming BIOS entry point at the end. The spi-flash. bin seems to have a placeholder of the flash descriptor which would go at the start of the flash, but is mostly empty. So I believe Intel basically either stitches these together in memory, or possibly just uses the spi-flash. bin to allow for "soft strap" configuration or somesuch (since it's a virtual MCH/ICH anyway.)

How bitbake searches for recipe in build process?

I am trying to find out that how bitbake search for recipe in build process ?
For example,
I have a recipe something like below:
DESCRIPTION = "PetaLinux GSTREAMER supported packages"
inherit packagegroup
GSTREAMER_PACKAGES = " \
gstreamer1.0 \
gstreamer1.0-python \
gstreamer1.0-meta-base \
gstreamer1.0-plugins-base \
gstreamer1.0-plugins-good \
gstreamer1.0-plugins-bad \
gstreamer1.0-rtsp-server \
gst-shark \
gstd \
gst-perf \
gst-interpipes \
"
GSTREAMER_PACKAGES_append_zynqmp = " gstreamer1.0-omx"
RDEPENDS_${PN} = "${GSTREAMER_PACKAGES}"
When I searched gstreamer1.0 related recipe in yocto layers, I found two recipe, one of them is gstreamer1.0_1.16.1.bb in meta layer, and the other is gstreamer1.0_%.bbappend in meta-petalinux layer.
Both of these layers was added to the BBLAYERS in bblayers.conf file and the priorities that spesified with BBFILE_PRIORITY_* in related layer's layer.conf file is same.
So,
Which recipe will be used in build process in that case ?
What is the recipe lookup rules in yocto ?
I changed somethings to understand the behaviour:
For example,
I entered the invalid github URL that spesified in gstreamer1.0_%.bbappend recipe. When I tried to build the linux system, I encountered with an error. Thats fine.
Then I corrected the github URL in this recipe and entered invalid source code address that spesified in gstreamer1.0_1.16.1.bb recipe. When I tried to build linux system, process finished successfully.
Then I increased the priority of meta layer. I supposed to encounter with an error in this case but again build process finished successfully.
Could you please help me to understand this behaviour ?
Thanks.
You have two different files: a .bb and a .bbappend.
A .bb is the base recipe of one (or multiple) packages. It generally describe how to fetch, configure, compile, install files in a package for your target.
A .bbappend file is an 'append' file. It allows a meta (here meta-petalinux) to modify an existing recipe in another meta without copying it. A .bbappend can modify any steps of the bb file: source fetch, configure, compile, install...
You can for example create your own bbappend of Gstreamer, to enable pango (disbaled by default on my Yocto). The bbappend filename is gstreamer1.0-plugins-base_%.bbappend and only contains PACKAGECONFIG_append = "pango"
The Yocto Manual can give you more information on bbappend files here.

Getting HASH of individual files within folder uploaded to IPFS

When I upload a folder of .jpg files to IPFS, I get the HASH of that folder - which is cool.
But is each individual file in that folder also getting hashed?
And if so, how do I get the hash of each file?
I basically want to be able to upload a whole bunch of files - like 500 images - and do it all at once, or programmatically, and have the hash of each file be returned to me.
Any way to do this?
Yes! From the command line you get back the CIDs (the Content IDentifier, aka, IPFS hash) for each file added when you run ipfs add -r <path to directory>
$ ipfs add -r gifs
added QmfBAEYhJp9ZjGvv8utB3Yv8uuuxsDKjv9rurkHRsYU3ih gifs/martian-iron-man.gif
added QmRBHTH3p4W2xAzgLxvdh8VJvAmWBgchwCr9G98EprwetE gifs/needs-more-dogs.gif
added QmZbffnCcV598QxsUy7WphXCAMZJULZAzy94tuFZzbFcdK gifs/satisfied-with-your-care.gif
added QmTxnmk85ESr97j2xLNFeVZW2Kk9FquhdswofchF8iDGFg gifs/stone-of-triumph.gif
added QmcN71Qh56oSg2YXsEXuf8o6u5CrBXbyYYzgMyAkdkcxxK gifs/thanks-dog.gif
added QmTnuLaivKc1Aj8LBf2iWBHDXsmedip3zSPbQcGi6BFwTC gifs
the root CID for the directory is always the last item in the list.
You can limit the output of that command to just include the CIDs using the --quiet flag
⨎ ipfs add -r gifs --quiet
QmfBAEYhJp9ZjGvv8utB3Yv8uuuxsDKjv9rurkHRsYU3ih
QmRBHTH3p4W2xAzgLxvdh8VJvAmWBgchwCr9G98EprwetE
QmZbffnCcV598QxsUy7WphXCAMZJULZAzy94tuFZzbFcdK
QmTxnmk85ESr97j2xLNFeVZW2Kk9FquhdswofchF8iDGFg
QmcN71Qh56oSg2YXsEXuf8o6u5CrBXbyYYzgMyAkdkcxxK
QmTnuLaivKc1Aj8LBf2iWBHDXsmedip3zSPbQcGi6BFwTC
Or, if you know the CID for a directory, you can list out the files it contains and their individual CIDs with ipfs ls. Here I list out the contents of the gifs dir from the previous example
$ ipfs ls QmTnuLaivKc1Aj8LBf2iWBHDXsmedip3zSPbQcGi6BFwTC
QmfBAEYhJp9ZjGvv8utB3Yv8uuuxsDKjv9rurkHRsYU3ih 2252675 martian-iron-man.gif
QmRBHTH3p4W2xAzgLxvdh8VJvAmWBgchwCr9G98EprwetE 1233669 needs-more-dogs.gif
QmZbffnCcV598QxsUy7WphXCAMZJULZAzy94tuFZzbFcdK 1395067 satisfied-with-your-care.gif
QmTxnmk85ESr97j2xLNFeVZW2Kk9FquhdswofchF8iDGFg 1154617 stone-of-triumph.gif
QmcN71Qh56oSg2YXsEXuf8o6u5CrBXbyYYzgMyAkdkcxxK 2322454 thanks-dog.gif
You can it programatically with the core api in js-ipfs or go-ipfs. Here is an example of adding a files from the local file system in node.js using js-ipfs from the docs for ipfs.addAll(files) - https://github.com/ipfs/js-ipfs/blob/master/docs/core-api/FILES.md#importing-files-from-the-file-system
There is a super helpful video on how adding files to IPFS works over at https://www.youtube.com/watch?v=Z5zNPwMDYGg
And a walk through of js-ipfs here https://github.com/ipfs/js-ipfs/tree/master/examples/ipfs-101

Yocto - creating a dependency for WIC to cpio.gz image

I'm creating a small Yocto distro that should work in RAM on tmpfs. I use the WIC configuration in the following way:
part /boot --source bootimg-efi --sourceparams="loader=grub-efi,initrd=${PN}-${MACHINE}.cpio.gz,file=${PN}-${MACHINE}.cpio.gz" --ondisk sda --label msdos --active --align 1024
bootloader --ptable gpt --timeout=0 --append="rootfstype=tmpfs rootflags=size=2G console=ttyS0,115200 console=tty0"
I also add IMAGE_FSTYPES_append = " cpio.gz " to my local.conf, so it builds the cpio.gz archive from my rootfs.
My problem is very straightforward - when WIC runs, it tries to create the wic file before it is done with creating the rootfs cpio.gz, and therefore the build fails. What I need is to create a dependency, something that will hold WIC scripts until the cpio.gz is ready. Does anyone know how to achieve it? Can, for instance, WKS_FILE_DEPENDS be used?
Here is the failure:
| ERROR: _exec_cmd: cp .../poky/build/tmp/deploy/images/genericx86-64/core-image-minimal-genericx86-64.cpio.gz .../poky/build/tmp/work/genericx86_64-poky-linux/core-image-minimal/1.0-r0/deploy-core-image-minimal-image-complete/core-image-minimal-genericx86-64-20191121151711/tmp.wic.k00ckxmk/hdd/boot returned '1' instead of 0
| output: cp: cannot stat '.../poky/build/tmp/deploy/images/genericx86-64/core-image-minimal-genericx86-64.cpio.gz': No such file or directory
Currently I bypass the problem by running the wic tool manually after the build. I had to use IMAGE_FSTYPES_remove = " wic wic.bmap hddimg " in my local.conf for that. The command for running wic then is:
wic create ../meta-mylayer/wic/myimage.wks -e core-image-minimal
Thanks!
EDIT:
Maybe the problem is not in creating the required dependency, but in the way I create the image? I just want a UEFI boot, a kernel, and a cpio.gz file with a complete rootfs which will gets mounted on boot. This is not an initramfs, but a complete rootfs that I need there. Except the problematic dependency the resulting image does exactly what I need.
You can specify the dependency with WIC in 2 ways.
Using do_image_wic: The final task to create the WIC is do_image_wic. So you can add dependency for creating your initrd/initramfs image to this task as below,
do_image_wic[depends] += "image-base-initramfs:do_image_complete"
You need to specify this in your WIC image creation recipe. For this example,
DESCRIPTION = "My image"
inherit core-image
export IMAGE_BASENAME = "image-base"
IMAGE_FSTYPES = "wic.xz"
DEPENDS += "image-base-initramfs"
do_image_wic[depends] += "image-base-initramfs:do_image_complete"
WKS_FILES = "my.wks"
Here image-base is used for creating the WIC using my.wks. It waits for the initramfs to complete the building. In image-base-initramfs you will create the initramfs image.
To add, you can also do this with INITRAMFS_IMAGE when using kernel fitImage.
Using WKS_FILE_DEPENDS: You can add any bitbake recipe to dependency before creating the WIC image. Adding image-base-initramfs to this variable will wait for it to complete the initramfs image. We also have WKS_FILE_DEPENDS_BOOTLOADERS when depending on bootloader to complete in WIC creation.

Auditd in Yocto

I'm trying to add auditd to Yocto linux.
I added the selinux layer and it's dependent layers: openembedded-core and meta-virtualization.
I added the layers to bblayers.conf.
I added DISTRO_FEATURES_append = " acl xattr pam selinux"
and PREFERRED_PROVIDER_virtual/refpolicy ?= "refpolicy-mls" to the local.conf file.
After building (by using bitbake core-image-base) and running the qemu, the kauditd process is running, but all user-space tools are not.
The /etc/audit folder is not exist ,non of the audit's config files exists (audit.rules) and no user-space audit process is running.
In the layer's info it is declared - "User space tools for kernel auditing".
What I am missing?
Thanks.
I think I found something that will answer your question: If you know what an example binary or library you expect to be in the target image, you can find what recipe the executable is in, and then add that package to the image.
Start with the name of a binary or library you expect to be in the image and run the following. For me, I am using a CAN bus executable called candump. I wonder what recipe it's in? To find out, I issue:
devtool search candump
Which returns:
can-utils
If nothing is returned, I'd double check your conf/bblayers.conf so that the layer you think it may be in is actually being seen by your build system. If you are unsure, take a look at the link below which points to OpenEmbedded which has a handy search utility for packages.
After you find the recipe, you can then include that recipe into your build.
Here is a good reference in doing what I think you're asking on the OpenEmbedded website:
https://wiki.yoctoproject.org/wiki/Cookbook:Example:Adding_packages_to_your_OS_image
I just added auditd to my system. This is what I did.
First I got the repository checked out.
cd /path/to/yocto
git clone git://git.yoctoproject.org/meta-selinux
cd meta-selinux
# checkout the branch matching the Yocto release you are on
git checkout thud
Then I added auditd to my build.
cd /path/to/build
bitbake-layers add-layer /path/to/yocto/meta-selinux
cat >> conf/local.conf <<'END'
IMAGE_INSTALL_append = " auditd"
END
bitbake my_normal_image_target
Even though the Yocto recipe is called audit, the package name is auditd.
Of course, auditd without selinux is useless but it did attempt to run (journalctl -u auditd) and /etc/audit exists.
FWIW: To get auditd to a point where it reports say, login success/failure, I had to do a few more things. I'm not just adding it to a standard Yocto image, but to a custom image and custom machine. I'm already using systemd so I didn't have to change that (the layer seems to indicate it's required?). My local.conf looked like this.
# enable selinux
DISTRO_FEATURES_append = " acl xattr pam selinux"
# set the policy
PREFERRED_PROVIDER_virtual/refpolicy ?= "refpolicy-mls"
# install selinux packages and auditd
IMAGE_INSTALL_append = " packagegroup-core-selinux auditd"
# tell the kernel to enable selinux (non-enforcing) and audting
APPEND_append = " selinux=1 enforcing=0 audit=1"
I also had to change linux-yocto_selinux.inc to load selinux.cfg later. Probably layer/recipe ordering could have solved this too?
-SRC_URI += "${#bb.utils.contains('DISTRO_FEATURES', 'selinux', 'file://selinux.cfg', '', d)}"
+SRC_URI_append = "${#bb.utils.contains('DISTRO_FEATURES', 'selinux', 'file://selinux.cfg', '', d)}"
With all that in place, I see audit logs in my journal.