During SDK build: environment-setup.d/ conflicts between attempted installs - yocto

Question

I am trying to build an image for beaglebone that contains Qt5, as well as generate the SDK for this image.
Problem
my problem is, that the build fails the do_populate_sdk task to create the SDK with the following error:
Error: Transaction check error:
file /opt/poky/2.3.1/sysroots/x86_64-pokysdk-linux/environment-setup.d conflicts between attempted installs of nativesdk-cmake-3.7.2-r0.x86_64_nativesdk and nativesdk-qtbase-tools-5.8.0+git0+49dc9aa409-r0.x86_64_nativesdk
A little further up the stream I encountered the following error message:
ERROR: Could not invoke dnf. Command '/home/ubuntu/workspace/bbb/build-toaster-2/tmp/work/my_machine-poky-linux-gnueabi/my-image-dev/1.0-r0/recipe-sysroot-native/usr/bin/dnf [...] ' returned 1:
Added oe-repo repo from file:///home/ubuntu/workspace/bbb/build-toaster-2/tmp/work/my-machine-poky-linux-gnueabi/my-image-dev/1.0-r0/oe-rootfs-repo.
Last metadata expiration check: 0:00:00 ago on Wed Aug 16 11:47:27 2017 UTC.
Dependencies resolved.
What I have
To configure my image I followed the advice here as well as similar posts stating the same elsewhere on the web. This is my (shortened and slightly redacted) image bb-file:
SUMMARY = "..."
LICENSE = "MIT"
IMAGE_LINGUAS = "en-us"
inherit core-image
# for populate_sdk to create a valid toolchain
inherit populate_sdk populate_sdk_qt5
CORE_OS = "..."
KERNEL_EXTRA_INSTALL = "..."
WIFI_SUPPORT = "..."
DEV_SDK_INSTALL = " \
binutils \
binutils-symlinks \
coreutils \
cpp \
cpp-symlinks \
diffutils \
file \
g++ \
g++-symlinks \
gdb \
gdbserver \
gcc \
gcc-symlinks \
gettext \
git \
ldd \
libstdc++ \
libstdc++-dev \
libtool \
make \
perl-modules \
pkgconfig \
python-modules \
python3-modules \
"
DEV_EXTRAS = "..."
EXTRA_TOOLS_INSTALL = " \
acpid \
bc \
bzip2 \
cursor-blink \
devmem2 \
dosfstools \
emmc-installer \
ethtool \
findutils \
i2c-tools \
iperf3 \
htop \
less \
memtester \
nano \
netcat \
procps \
rsync \
sysfsutils \
tcpdump \
unzip \
util-linux \
util-linux-blkid \
wget \
zip \
"
MQTT = "..."
ROOTFS_POSTPROCESS_COMMAND += "..."
QT_TOOLS = " \
qtbase \
qtbase-dev \
qtbase-mkspecs \
qtbase-plugins \
qtbase-tools \
qtserialport-dev \
qtserialport-mkspecs \
qt5-env \
"
QT5_PKGS = " \
qt3d \
qt3d-dev \
...
qtxmlpatterns \
qtxmlpatterns-dev \
qtxmlpatterns-mkspecs \
"
FONTS = "..."
TSLIB = "... "
ADDITIONAL_PKGS = "..."
QT_TEST_APPS = "..."
IMAGE_INSTALL += " \
${CORE_OS} \
${DEV_SDK_INSTALL} \
${DEV_EXTRAS} \
${EXTRA_TOOLS_INSTALL} \
${KERNEL_EXTRA_INSTALL} \
${FONTS} \
${QT_TOOLS} \
${QT5_PKGS} \
${QT_TEST_APPS} \
${MQTT} \
${WIFI_SUPPORT} \
${TSLIB} \
${ADDITIONAL_PKGS} \
"
IMAGE_FEATURES_append = " dev-pkgs"
export IMAGE_BASENAME = "my-image-dev"
I also set DISTRO_FEATURES_remove = "busybox x11 wayland" as well as DISTRO_FEATURES_append = " systemd opengl aufs" in my local.conf.
The build runs fine if I remove the inherit populate_sdk_qt5 line, but of course I don't get make for my SDK in this case.
What I found out
I found different people having the same problem (example here) but no-one ever got an answer.
I checked the nativesdk-cmake as well as the nativesdk-qtbase_git recipes (both unchanged standards) to see where the files get created, and neither looks problematic to me:
natives-qtbase_git.bb:
fakeroot do_generate_qt_environment_file() {
mkdir -p ${D}${SDKPATHNATIVE}/environment-setup.d/
script=${D}${SDKPATHNATIVE}/environment-setup.d/qt5.sh
echo 'export PATH=${OE_QMAKE_PATH_HOST_BINS}:$PATH' > $script
echo 'export OE_QMAKE_CFLAGS="$CFLAGS"' >> $script
echo 'export OE_QMAKE_CXXFLAGS="$CXXFLAGS"' >> $script
echo 'export OE_QMAKE_LDFLAGS="$LDFLAGS"' >> $script
echo 'export OE_QMAKE_CC=$CC' >> $script
echo 'export OE_QMAKE_CXX=$CXX' >> $script
echo 'export OE_QMAKE_LINK=$CXX' >> $script
echo 'export OE_QMAKE_AR=$AR' >> $script
echo 'export QT_CONF_PATH=${OE_QMAKE_PATH_HOST_BINS}/qt.conf' >> $script
echo 'export OE_QMAKE_LIBDIR_QT=`qmake -query QT_INSTALL_LIBS`' >> $script
echo 'export OE_QMAKE_INCDIR_QT=`qmake -query QT_INSTALL_HEADERS`' >> $script
echo 'export OE_QMAKE_MOC=${OE_QMAKE_PATH_HOST_BINS}/moc' >> $script
echo 'export OE_QMAKE_UIC=${OE_QMAKE_PATH_HOST_BINS}/uic' >> $script
echo 'export OE_QMAKE_RCC=${OE_QMAKE_PATH_HOST_BINS}/rcc' >> $script
echo 'export OE_QMAKE_QDBUSCPP2XML=${OE_QMAKE_PATH_HOST_BINS}/qdbuscpp2xml' >> $script
echo 'export OE_QMAKE_QDBUSXML2CPP=${OE_QMAKE_PATH_HOST_BINS}/qdbusxml2cpp' >> $script
echo 'export OE_QMAKE_QT_CONFIG=`qmake -query QT_INSTALL_LIBS`${QT_DIR_NAME}/mkspecs/qconfig.pri' >> $script
echo 'export OE_QMAKE_PATH_HOST_BINS=${OE_QMAKE_PATH_HOST_BINS}' >> $script
echo 'export QMAKESPEC=`qmake -query QT_INSTALL_LIBS`${QT_DIR_NAME}/mkspecs/linux-oe-g++' >> $script
# Use relocable sysroot
sed -i -e 's:${SDKPATHNATIVE}:$OECORE_NATIVE_SYSROOT:g' $script
}
cmake-3.7.2.bb:
do_install_append_class-nativesdk() {
mkdir -p ${D}${datadir}/cmake
install -m 644 ${WORKDIR}/OEToolchainConfig.cmake ${D}${datadir}/cmake/
mkdir -p ${D}${SDKPATHNATIVE}/environment-setup.d
install -m 644 ${WORKDIR}/environment.d-cmake.sh ${D}${SDKPATHNATIVE}/environment-setup.d/cmake.sh
}
environment.d-cmake.sh:
alias cmake="cmake -DCMAKE_TOOLCHAIN_FILE=$OECORE_NATIVE_SYSROOT/usr/share/cmake/OEToolchainConfig.cmake"
For the sake of trying I went ahead and executed the
/home/ubuntu/workspace/bbb/build-toaster-2/tmp/work/my_machine-poky-linux-gnueabi/my-image-dev/1.0-r0/recipe-sysroot-native/usr/bin/dnf
script from
/home/ubuntu/workspace/bbb/build-toaster-2/tmp/work/my_machine-poky-linux-gnueabi/my-image-dev/1.0-r0/recipe-sysroot-native
which got me the following error message:
Traceback (most recent call last):
File "/home/ubuntu/workspace/bbb/build-toaster-2/tmp/work/my-machine-poky-linux-gnueabi/my-image-dev/1.0-r0/recipe-sysroot-native/usr/bin/dnf.real", line 57, in <module>
from dnf.cli import main
ImportError: No module named 'dnf'
The dnf module seems to exist though:
<path as above>/recipe-sysroot-native$ find -name dnf
./usr/lib/python3.5/site-packages/dnf
./usr/bin/dnf
./etc/dnf
./etc/bash_completion.d/dnf
./etc/logrotate.d/dnf
Can you see anything that I am doing wrong? I am absolutely clueless...

Answer 1

I'm building an SDK with cmake and Qt5 without any problem...
Your issue seems to stem from dnf, and as I'm building with ipk without any issue, there might very well be a bug in the rpm handling in OpenEmbedded.
Could you try to rebuild with:
PACKAGE_CLASSES = "package_ipk"
in your local.conf and see it that helps?

Answer 2

Edit:
Anders' answer provides a more elegant solution by switching the packaging class. If you can, check out his approach before trying this workaround.
I found a workaround that worked for me but is no ideal solution. I am posting it anyway, in case it helps someone:
I figured out, that the nativesdk-cmake package somehow collided with the Qt one. Therefor I created a nativesdk-packagegroup-sdk-host.bbappend file in my custom layer, with the following content:
RDEPENDS_${PN}_remove = "\
nativesdk-cmake \
"
This removes the cmake dependency from the SDK build, which works for my purposes. But this merely solves the symptoms not the problem. So I am glad for any other solution.

Answer 3

Just try and add DIRFILES = "" in nativesdk-qtbase.bb (you should rather set up a clean and tidy nativesdk-qtbase.bbappend in your custom layer with DIRFILES = "").
This works around clashes due to RPM directory ownership for this RPM, which is a default policy in standard RPM packaging. See package_rpm.bbclass for details # python write_specfile () { .... walk_files method.
Note: DIRFILES must be defined but left empty for this to work on the current package.
Voila.
Cheers.

Answer 4

As so61pi mentioned, RPM has strict checking for files/directories. In my case, after installing nativesdk-qtbase the environment.d folder had permissions 775 whereas nativesk-cmake created that same folder with 755.
I don't know if this was caused due to the fact that the function generate_qt_environment_file is being executed in a fakeroot environment, but I fixed it by adding its function body to the do_install function and removing generate_qt_environment_file.
Not sure if this is the correct fix though. I noticed some other recipes use the fakeroot keyword and others don't. I wonder why...

Answer 5

RPM has strict checking for files/directories. environment-setup.d directory in your question may have different mode or user between the 2 packages.
You could check function rpmfilesCompare for the exact checks that RPM performs.