How to execute a script in swupdate image recipe in Yocto? - yocto

I am unable to execute a shell script(create_signed_swu.sh) in my recipe(panther2-swu.bb) which inherit swupdate and if I remove inherit swupdate, I see do_compile gets compiled and script executes successfully.
Here is panther2-swu.bb recipe:
DESCRIPTION = "Building swupdate image (.swu ) for panther2 board"
LICENSE = "MIT"
LIC_FILES_CHKSUM = "file://${COMMON_LICENSE_DIR}/MIT;md5=0835ade698e0bcf8506ecda2f7b4f302"
inherit swupdate
FILESEXTRAPATHS_append := "${THISDIR}/../../../../build/tmp/deploy/images/panther2/"
SRC_URI = "file://sw-description \
file://postinstall_swu.sh \
file://bzImage \
file://create_signed_swu.sh \
file://priv.pem \
file://passphrase \
"
do_compile() {
${WORKDIR}/create_signed_swu.sh
}
DEPENDS = "openssl-native"
Here is the create_signed_swu.sh:
#!/bin/bash
IMAGES="bzImage panther2-usb-panther2.ext4"
FILES="sw-description sw-description.sig postinstall_swu.sh $IMAGES"
echo "Executing swu signing script..."
cp ../sw-description .
cp ../postinstall_swu.sh .
cp ../../../../../deploy/images/panther2/bzImage .
cp ../../../../../deploy/images/panther2/panther2-usb-panther2.ext4 .
read -d ' ' SHA_ROOTFS < <(sha256sum panther2-usb-panther2.ext4)
read -d ' ' SHA_BZIMAGE < <(sha256sum bzImage)
read -d ' ' SHA_POSTINSTALL < <(sha256sum postinstall_swu.sh)
sed -i ':a;N;$!ba; s/sha256 = "[0-9A-Za-z]*"/sha256 = '"\"${SHA_ROOTFS}"\"'/1' sw-description
sed -i ':a;N;$!ba; s/sha256 = "[0-9A-Za-z]*"/sha256 = '"\"${SHA_BZIMAGE}"\"'/2' sw-description
sed -i ':a;N;$!ba; s/sha256 = "[0-9A-Za-z]*"/sha256 = '"\"${SHA_POSTINSTALL}"\"'/3' sw-description
openssl dgst -sha256 -sign ../priv.pem -passin file:../passphrase sw-description > sw-description.sig
for i in $FILES;do
echo $i;done | cpio -ov -H crc > panther2-swu-$USER-devbuild.swu
cp panther2-swu-$USER-devbuild.swu ../../../../../deploy/images/panther2
rm -f sw-description
rm -f postinstall_swu.sh
rm -f sw-description.sig
rm -f bzImage
rm -f panther2-usb-panther2.ext4
rm -f panther2-swu-$USER-devbuild.swu
I have tried ROOTFS_POSTPROCESS_COMMAND as well but it doen't execute my srcipt either.
Any help would really be appreciated, Thanks in advance...!!!

If removing the inherit swupdate makes your do_compile execute and launch your script it means that the class swupdate.bbclass do not execute the do_compile task.
The class surely has this line :
do_compile[noexec] = "1"
Which means that the do_compile is not excuted at all.
You could choose between two solutions:
Using prepend/append of some task:
Prepend/append your recipe's task that is meant to launch the script to an existing task like following :
Assuming the do_configure is launched :
do_configure_append()
{
${WORKDIR}/create_signed_swu.sh
}
This will launch your script at the end of do_configure.
Creating your own task:
Assuming you want to launch your script before do_build task
do_launch_script () {
${WORKDIR}/create_signed_swu.sh
}
addtask launch_script after do_configure before do_build

Generating a SWU does not compile anything and not only do_compile, but even do_configure is turned off. The class "swupdate" has an own task : do_swuimage. Changes to default behaviour can be done with a do_swuimage_append().
Anyway, the script above makes no sense: the class does exactly what the script is trying to do, and provide variable substitutions to automatically set sha256 and other values. The "sed" entries make no sense, as well as signing in this script. Instead of that, the class should be inherited and variables must be (all of them starting with SWUPDATE_, please check in documentation) to let the class to do its job. The script is a variant of the script inserted in the documentation to build outside Yocto - instead of following this approach, the functionalities in meta-swupdate should be used.

Related

yocto recipe do_install file name with spaces

I am trying to install a file name with spaces in rootfs and having problems to overcome it.
I tried escape characters before the space, single quote start + end of the file name without success.
See below the bbappend recipe. I am trying to set default address with NetworkManager.
SRC_URI += " \
file://Wired connection 1.nmconnection \
file://Wired connection 2.nmconnection \
"
FILES:${PN} += " \
${sysconfdir}/NetworkManager/system-connections/Wired connection 1.nmconnection \
${sysconfdir}/NetworkManager/system-connections/Wired connection 2.nmconnection \
"
do_install:append() {
install -d ${D}${sysconfdir}/NetworkManager/system-connections
install -m 0644 ${WORKDIR}/Wired connection 1.nmconnection ${D}${sysconfdir}/NetworkManager/system-connections
install -m 0644 ${WORKDIR}/Wired connection 2.nmconnection ${D}${sysconfdir}/NetworkManager/system-connections
The manual specifies:
The OpenEmbedded build system does not support file or directory names that contain spaces. Be sure that the Source Directory you use does not contain these types of names.
In the FAQ, they add this.
But with some additional work, you can use a trick by replacing the spaces by a special character (e.g '_'). For example, rename "Wired connection 1.nmconnection" into "Wired_connection_1.nmconnection". Then, you write a shell function (e.g. named my_install()) which restores the original names at installation time with the tr command. Call this function from the install task in your recipe:
SRC_URI += " \
file://Wired_connection_1.nmconnection \
file://Wired_connection_2.nmconnection \
"
my_install () {
install -d ${D}${sysconfdir}/NetworkManager/system-connections
file=Wired_connection_1.nmconnection
install -m 0644 "${WORKDIR}/$file" ${D}${sysconfdir}/NetworkManager/system-connections
mv "${D}${sysconfdir}/NetworkManager/system-connections/$file" "${D}${sysconfdir}/NetworkManager/system-connections/`echo $file | tr _ ' '`
file=Wired_connection_2.nmconnection
install -m 0644 "${WORKDIR}/$file" ${D}${sysconfdir}/NetworkManager/system-connections
mv "${D}${sysconfdir}/NetworkManager/system-connections/$file" "${D}${sysconfdir}/NetworkManager/system-connections/`echo $file | tr _ ' '`
}
do_install:append() {
my_install
}
Thanks Rachid,
your answers helped me.
This is my final version:
do_install:append() {
install -d ${D}${sysconfdir}/NetworkManager/system-connections
file=Wired-connection-1.nmconnection
install -m 0644 "${WORKDIR}/$file" ${D}${sysconfdir}/NetworkManager/system-connections
mv "${D}${sysconfdir}/NetworkManager/system-connections/$file" "${D}${sysconfdir}/NetworkManager/system-connections/$(echo $file | tr - \ )"
file=Wired-connection-2.nmconnection
install -m 0644 "${WORKDIR}/$file" ${D}${sysconfdir}/NetworkManager/system-connections
mv "${D}${sysconfdir}/NetworkManager/system-connections/$file" "${D}${sysconfdir}/NetworkManager/system-connections/$(echo $file | tr - \ )"
}

Bitbake service file organization

I have a service created and working. However my current bitbake file makes my folder structure very ugly. To have it work I have to have all of my configuration files in my src folder. I would like to have two folders, src and configfiles.
If my tree is as follows
How can I edit my bb file to pull in my src directory and my configfiles?
LICENSE = "MIT"
LIC_FILES_CHKSUM = "file://${COMMON_LICENSE_DIR}/MIT;md5=0835ade698e0bcf8506ecda2f7b4f302"
SRC_URI = "file://src"
S = "${WORKDIR}/src"
CF = "${WORKDIR}/configfiles"
inherit systemd autotools
SYSTEMD_SERVICE_${PN} = "helloworld.service"
do_install_append () {
echo "look at this PN ${PN}"
echo "look at this D ${D}"
echo "look at this S ${S}"
echo "look at this CF ${CF}"
install -d ${D}${systemd_system_unitdir}
install -m 0644 ${CF}/helloworld.service ${D}${systemd_system_unitdir}
sed -i -e 's,#BINDIR#,${bindir},g' ${D}${systemd_system_unitdir}/helloworld.service
}
Update 1:
I added a CF variable. Using bitbake -e helloworld > log.txt I can echo the values of variables and noticed we are not pointing to the config files.
LICENSE = "MIT"
LIC_FILES_CHKSUM = "file://${COMMON_LICENSE_DIR}/MIT;md5=0835ade698e0bcf8506ecda2f7b4f302"
SRC_URI = "file://src"
S = "${WORKDIR}/src"
CF = "${WORKDIR}/configfiles"
inherit systemd autotools
SYSTEMD_SERVICE_${PN} = "helloworld.service"
do_install_append () {
echo "look at this PN ${PN}"
echo "look at this D ${D}"
echo "look at this S ${S}"
echo "look at this CF ${CF}"
install -d ${D}${systemd_system_unitdir}
install -m 0644 ${CF}/helloworld.service ${D}${systemd_system_unitdir}
sed -i -e 's,#BINDIR#,${bindir},g' ${D}${systemd_system_unitdir}/helloworld.service
}
Bitbake still can not find my makefile.
Update 2:
The solution given works but I have one more question.
My makefile is very basic:
ACLOCAL_AMFLAGS = -I m4
AUTOMAKE_OPTIONS = foreign
SUBDIRS = app
CFLAGS = -Wall -pedantic
include_HEADERS = helloworld.h
bin_PROGRAMS = helloworld
helloworld_SOURCES = helloworld.c
I want to separate my files by folder. I want my main application in a folder /src/app. I edited my makefile to point to app/helloworld.h and app/helloworld.c However this does not find the files. My updated bb file follows
LICENSE = "MIT"
LIC_FILES_CHKSUM = "file://${COMMON_LICENSE_DIR}/MIT;md5=0835ade698e0bcf8506ecda2f7b4f302"
FILESEXTRAPATHS:prepend := "${THISDIR}/files/src:${THISDIR}/files/src/app:${THISDIR}/files/services:"
SRC_URI = "file://src \
file://src/app \
"
SRC_URI += "file://helloworld.service \
file://autogen.sh \
file://configure.ac \
file://Makefile.am"
S = "${WORKDIR}/src"
inherit systemd autotools
SYSTEMD_SERVICE_${PN} = "helloworld.service"
do_install_append () {
install -d ${D}${systemd_system_unitdir}
install -m 0644 ${WORKDIR}/helloworld.service ${D}${systemd_system_unitdir}
sed -i -e 's,#BINDIR#,${bindir},g' ${D}${systemd_system_unitdir}/helloworld.service
}
I understand the makefile should be packaged with the code, This just does not work when creating larger applications.
You need to let bitbake know where to look for files, it sounds like you want to keep the Makefile in a separate directory but in most cases the Makefile should be with the source files, feel free to play around with this, the concept is still the same as explained below.
Either way, your directory structure should look something like this:
recipes-core/
helloworld/
helloworld_0.1.bb
files/
configfiles/
helloworld.service
src/
autogen.sh
configure.ac
helloworld.c
helloworld.h
Makefile.am
And to let bitbake know where to pull files you use the FILESESXTRAPATHS variable [1]:
FILESEXTRAPATHS:prepend := "${THISDIR}/files/src:${THISDIR}/files/configfiles:"
At this point bitbake knows WHERE to pull the files from, but it doesn't know which files to pull, for this you use the SRC_URI variable (specifically the file:// fetcher) [2]:
SRC_URI += "file://helloworld.service \
file://autogen.sh \
file://configure.ac \
file://helloworld.c \
file://helloworld.h \
file://Makefile.am"
This assumes your Makefile will try to find files in the same directory.
The rest of your recipe is fine, the CF variable is no longer necessary, and you just need to change where you're installing the service file from, since SRC_URI[1] will pull it to ${WORKDIR}:
install -m 0644 ${WORKDIR}/helloworld.service ${D}${systemd_system_unitdir}
[1] https://docs.yoctoproject.org/singleindex.html#term-FILESEXTRAPATHS
[2] https://docs.yoctoproject.org/bitbake/singleindex.html#term-SRC_URI

Is there a command to got to WORDIR of a recipe?

So while working on a couple of recipes, sometimes I need to go to the WORKDIR of a recipe to make sure if it is doing what is supposed to. I know there is :
bitbake myrecipe -c devshell
but this blocks bitbake and while the new shell is running , i can't test or experiment on other recipes.
is there a command to simply cd to the WORKDIR of a recipe ?
cd bitbake myrecipe -e | grep ^WORKDIR= | cut -d '"' -f 2
bitbake myrecipe -e shows you the variables set in that recipe context, the rest of the shell extracts the directory.

How can I do "export $(dbus-launch) in booting

I'm using GDbus and make a dbus communication.
It using sesstion bus.
Problem is dbus-launch.
I was running dbus in Yocto with c++11.
And, I have to export $(dbus-launch).
But, I want to export $(dbus-launch) or same thing in booting time.
Because dbus start by systemd.
One solution is to have a recipe that adds environment variable:
SRC_URI += "file://dbus-env.sh"
do_install_append() {
install -d -m 0755 ${D}${sysconfdir}/profile.d
install -m 0755 ${WORKDIR}/dbus-env.sh ${D}${sysconfdir}/profile.d/
}
FILES_${PN} += "${sysconfdir}/profile.d/dbus-env.sh"
With dbus-env.sh
#!/bin/sh
export $(dbus-launch)
Use this command in /etc/profile or $HOME/.profile or $HOME/.bashrc :
eval \`dbus-launch --auto-syntax`
this will export "DBUS_SESSION_BUS_ADDRESS" and "DBUS_SESSION_BUS_PID" with proper values
you can also use this script:
[[ -n $SSH_CLIENT ]] && export $(cat /proc/$(command pgrep -u "$USER" -f -- "dbus-daemon --session" )/environ| tr '\0' '\n' | command grep "DBUS_SESSION_BUS_ADDRESS=")

During SDK build: environment-setup.d/ conflicts between attempted installs

I am trying to build an image for beaglebone that contains Qt5, as well as generate the SDK for this image.
Problem
my problem is, that the build fails the do_populate_sdk task to create the SDK with the following error:
Error: Transaction check error:
file /opt/poky/2.3.1/sysroots/x86_64-pokysdk-linux/environment-setup.d conflicts between attempted installs of nativesdk-cmake-3.7.2-r0.x86_64_nativesdk and nativesdk-qtbase-tools-5.8.0+git0+49dc9aa409-r0.x86_64_nativesdk
A little further up the stream I encountered the following error message:
ERROR: Could not invoke dnf. Command '/home/ubuntu/workspace/bbb/build-toaster-2/tmp/work/my_machine-poky-linux-gnueabi/my-image-dev/1.0-r0/recipe-sysroot-native/usr/bin/dnf [...] ' returned 1:
Added oe-repo repo from file:///home/ubuntu/workspace/bbb/build-toaster-2/tmp/work/my-machine-poky-linux-gnueabi/my-image-dev/1.0-r0/oe-rootfs-repo.
Last metadata expiration check: 0:00:00 ago on Wed Aug 16 11:47:27 2017 UTC.
Dependencies resolved.
What I have
To configure my image I followed the advice here as well as similar posts stating the same elsewhere on the web. This is my (shortened and slightly redacted) image bb-file:
SUMMARY = "..."
LICENSE = "MIT"
IMAGE_LINGUAS = "en-us"
inherit core-image
# for populate_sdk to create a valid toolchain
inherit populate_sdk populate_sdk_qt5
CORE_OS = "..."
KERNEL_EXTRA_INSTALL = "..."
WIFI_SUPPORT = "..."
DEV_SDK_INSTALL = " \
binutils \
binutils-symlinks \
coreutils \
cpp \
cpp-symlinks \
diffutils \
file \
g++ \
g++-symlinks \
gdb \
gdbserver \
gcc \
gcc-symlinks \
gettext \
git \
ldd \
libstdc++ \
libstdc++-dev \
libtool \
make \
perl-modules \
pkgconfig \
python-modules \
python3-modules \
"
DEV_EXTRAS = "..."
EXTRA_TOOLS_INSTALL = " \
acpid \
bc \
bzip2 \
cursor-blink \
devmem2 \
dosfstools \
emmc-installer \
ethtool \
findutils \
i2c-tools \
iperf3 \
htop \
less \
memtester \
nano \
netcat \
procps \
rsync \
sysfsutils \
tcpdump \
unzip \
util-linux \
util-linux-blkid \
wget \
zip \
"
MQTT = "..."
ROOTFS_POSTPROCESS_COMMAND += "..."
QT_TOOLS = " \
qtbase \
qtbase-dev \
qtbase-mkspecs \
qtbase-plugins \
qtbase-tools \
qtserialport-dev \
qtserialport-mkspecs \
qt5-env \
"
QT5_PKGS = " \
qt3d \
qt3d-dev \
...
qtxmlpatterns \
qtxmlpatterns-dev \
qtxmlpatterns-mkspecs \
"
FONTS = "..."
TSLIB = "... "
ADDITIONAL_PKGS = "..."
QT_TEST_APPS = "..."
IMAGE_INSTALL += " \
${CORE_OS} \
${DEV_SDK_INSTALL} \
${DEV_EXTRAS} \
${EXTRA_TOOLS_INSTALL} \
${KERNEL_EXTRA_INSTALL} \
${FONTS} \
${QT_TOOLS} \
${QT5_PKGS} \
${QT_TEST_APPS} \
${MQTT} \
${WIFI_SUPPORT} \
${TSLIB} \
${ADDITIONAL_PKGS} \
"
IMAGE_FEATURES_append = " dev-pkgs"
export IMAGE_BASENAME = "my-image-dev"
I also set DISTRO_FEATURES_remove = "busybox x11 wayland" as well as DISTRO_FEATURES_append = " systemd opengl aufs" in my local.conf.
The build runs fine if I remove the inherit populate_sdk_qt5 line, but of course I don't get make for my SDK in this case.
What I found out
I found different people having the same problem (example here) but no-one ever got an answer.
I checked the nativesdk-cmake as well as the nativesdk-qtbase_git recipes (both unchanged standards) to see where the files get created, and neither looks problematic to me:
natives-qtbase_git.bb:
fakeroot do_generate_qt_environment_file() {
mkdir -p ${D}${SDKPATHNATIVE}/environment-setup.d/
script=${D}${SDKPATHNATIVE}/environment-setup.d/qt5.sh
echo 'export PATH=${OE_QMAKE_PATH_HOST_BINS}:$PATH' > $script
echo 'export OE_QMAKE_CFLAGS="$CFLAGS"' >> $script
echo 'export OE_QMAKE_CXXFLAGS="$CXXFLAGS"' >> $script
echo 'export OE_QMAKE_LDFLAGS="$LDFLAGS"' >> $script
echo 'export OE_QMAKE_CC=$CC' >> $script
echo 'export OE_QMAKE_CXX=$CXX' >> $script
echo 'export OE_QMAKE_LINK=$CXX' >> $script
echo 'export OE_QMAKE_AR=$AR' >> $script
echo 'export QT_CONF_PATH=${OE_QMAKE_PATH_HOST_BINS}/qt.conf' >> $script
echo 'export OE_QMAKE_LIBDIR_QT=`qmake -query QT_INSTALL_LIBS`' >> $script
echo 'export OE_QMAKE_INCDIR_QT=`qmake -query QT_INSTALL_HEADERS`' >> $script
echo 'export OE_QMAKE_MOC=${OE_QMAKE_PATH_HOST_BINS}/moc' >> $script
echo 'export OE_QMAKE_UIC=${OE_QMAKE_PATH_HOST_BINS}/uic' >> $script
echo 'export OE_QMAKE_RCC=${OE_QMAKE_PATH_HOST_BINS}/rcc' >> $script
echo 'export OE_QMAKE_QDBUSCPP2XML=${OE_QMAKE_PATH_HOST_BINS}/qdbuscpp2xml' >> $script
echo 'export OE_QMAKE_QDBUSXML2CPP=${OE_QMAKE_PATH_HOST_BINS}/qdbusxml2cpp' >> $script
echo 'export OE_QMAKE_QT_CONFIG=`qmake -query QT_INSTALL_LIBS`${QT_DIR_NAME}/mkspecs/qconfig.pri' >> $script
echo 'export OE_QMAKE_PATH_HOST_BINS=${OE_QMAKE_PATH_HOST_BINS}' >> $script
echo 'export QMAKESPEC=`qmake -query QT_INSTALL_LIBS`${QT_DIR_NAME}/mkspecs/linux-oe-g++' >> $script
# Use relocable sysroot
sed -i -e 's:${SDKPATHNATIVE}:$OECORE_NATIVE_SYSROOT:g' $script
}
cmake-3.7.2.bb:
do_install_append_class-nativesdk() {
mkdir -p ${D}${datadir}/cmake
install -m 644 ${WORKDIR}/OEToolchainConfig.cmake ${D}${datadir}/cmake/
mkdir -p ${D}${SDKPATHNATIVE}/environment-setup.d
install -m 644 ${WORKDIR}/environment.d-cmake.sh ${D}${SDKPATHNATIVE}/environment-setup.d/cmake.sh
}
environment.d-cmake.sh:
alias cmake="cmake -DCMAKE_TOOLCHAIN_FILE=$OECORE_NATIVE_SYSROOT/usr/share/cmake/OEToolchainConfig.cmake"
For the sake of trying I went ahead and executed the
/home/ubuntu/workspace/bbb/build-toaster-2/tmp/work/my_machine-poky-linux-gnueabi/my-image-dev/1.0-r0/recipe-sysroot-native/usr/bin/dnf
script from
/home/ubuntu/workspace/bbb/build-toaster-2/tmp/work/my_machine-poky-linux-gnueabi/my-image-dev/1.0-r0/recipe-sysroot-native
which got me the following error message:
Traceback (most recent call last):
File "/home/ubuntu/workspace/bbb/build-toaster-2/tmp/work/my-machine-poky-linux-gnueabi/my-image-dev/1.0-r0/recipe-sysroot-native/usr/bin/dnf.real", line 57, in <module>
from dnf.cli import main
ImportError: No module named 'dnf'
The dnf module seems to exist though:
<path as above>/recipe-sysroot-native$ find -name dnf
./usr/lib/python3.5/site-packages/dnf
./usr/bin/dnf
./etc/dnf
./etc/bash_completion.d/dnf
./etc/logrotate.d/dnf
Can you see anything that I am doing wrong? I am absolutely clueless...
I'm building an SDK with cmake and Qt5 without any problem...
Your issue seems to stem from dnf, and as I'm building with ipk without any issue, there might very well be a bug in the rpm handling in OpenEmbedded.
Could you try to rebuild with:
PACKAGE_CLASSES = "package_ipk"
in your local.conf and see it that helps?
Edit:
Anders' answer provides a more elegant solution by switching the packaging class. If you can, check out his approach before trying this workaround.
I found a workaround that worked for me but is no ideal solution. I am posting it anyway, in case it helps someone:
I figured out, that the nativesdk-cmake package somehow collided with the Qt one. Therefor I created a nativesdk-packagegroup-sdk-host.bbappend file in my custom layer, with the following content:
RDEPENDS_${PN}_remove = "\
nativesdk-cmake \
"
This removes the cmake dependency from the SDK build, which works for my purposes. But this merely solves the symptoms not the problem. So I am glad for any other solution.
Just try and add DIRFILES = "" in nativesdk-qtbase.bb (you should rather set up a clean and tidy nativesdk-qtbase.bbappend in your custom layer with DIRFILES = "").
This works around clashes due to RPM directory ownership for this RPM, which is a default policy in standard RPM packaging. See package_rpm.bbclass for details # python write_specfile () { .... walk_files method.
Note: DIRFILES must be defined but left empty for this to work on the current package.
Voila.
Cheers.
As so61pi mentioned, RPM has strict checking for files/directories. In my case, after installing nativesdk-qtbase the environment.d folder had permissions 775 whereas nativesk-cmake created that same folder with 755.
I don't know if this was caused due to the fact that the function generate_qt_environment_file is being executed in a fakeroot environment, but I fixed it by adding its function body to the do_install function and removing generate_qt_environment_file.
Not sure if this is the correct fix though. I noticed some other recipes use the fakeroot keyword and others don't. I wonder why...
RPM has strict checking for files/directories. environment-setup.d directory in your question may have different mode or user between the 2 packages.
You could check function rpmfilesCompare for the exact checks that RPM performs.