Travis CI for my Github repo keeps failing and I didn't know why, until I read through the Job log and figured out that many dependencies couldn't be downloaded on their machines.
For example, the xcolor package failed to install:
[20/23, 03:54/04:02] install: xcolor [17k]
Downloading
ftp://tug.org/historic/systems/texlive/2015/tlnet-final/archive/xcolor.tar.xz
did not succeed, please retry.
TLPDB::_install_package: couldn't unpack ftp://tug.org/historic/systems/texlive/2015/tlnet-final/archive/xcolor.tar.xz to /home/travis/texmf
which results in the following error:
Latexmk: Missing input file: 'xcolor.sty' from line
'! LaTeX Error: File `xcolor.sty' not found.'
Latexmk: Log file says no output from latex
Latexmk: For rule 'pdflatex', no output was made
The problem is I'm working on a LaTeX project, and LaTeX environments tend to be huge with lots of supporting packages. Here is the relevant part of my setup.sh, which is already the minimal requirement:
sudo tlmgr install \
xkeyval ifthen amsmath bm \
longtable ctex tabu array \
colortbl berasans graphicx longtable \
etoolbox lastpage amssymb mathrsfs \
multirow xeCJK environ after \
booktabs hyperref epstopdf tabu \
fancyhdr listings amsfonts latexsym \
hhline CJK longtable pifont \
geometry ifpdf bmpsize hologo \
fancybox appendix amsbsy paralist \
tabularx xCJK2uni hologo calc \
fontenc ifxetex xcolor palatino
Can I make sure that all required packages are successfully downloaded & installed before the building phase, say, by letting the server to retry several times? If so, how?
From the Travis docs:
If you are getting network timeouts when trying to download dependencies, either use the built in retry feature of your dependency manager or wrap your install commands in the travis_retry function.
For example, in your .travis.yml:
install: travis_retry pip install myawesomepackage
travis_retry will attempt up three times if the return code is non-zero.
Related
I tried to build the Android SDK Addon System Image of halogenOS 13 (AOSP 13, nothing was changed in the AOSP source code yet).
The steps to build are as usual:
source build/envsetup.sh
lunch aosp_sdk_phone_x86_64-eng
m sdk_addon
At 100%, the build fails with following error:
[100% 12483/12483] Packaging SDK Addon System-Image: out/host/linux-x86/sdk_addon/custom-eng.simao--
FAILED: out/host/linux-x86/sdk_addon/custom-eng.simao--img.zip
/bin/bash -c "(cp -R out/target/product/emulator_x86_64/data out/host/linux-x86/obj/SDK_ADDON/custom_intermediates/custom-eng.simao--img/images/x86_64/data ) && (out/host/linux-x86/bin/soong_zip -o out/host/linux-x86/sdk_addon/custom-eng.simao--img.zip -C out/host/linux-x86/obj/SDK_ADDON/custom_intermediates/custom-eng.simao--img/images/ -D out/host/linux-x86/obj/SDK_ADDON/custom_intermediates/custom-eng.simao--img/images/x86_64 )"
cp: bad 'out/target/product/emulator_x86_64/data': No such file or directory
13:17:13 ninja failed with: exit status 1
If I just mkdir out/target/product/emulator_x86_64/data, of course, that just solves the build error but the SDK addon does not actually boot in the emulator due to encryption issues with the userdata partition so I think this is related. This makes me guess that in the data directory there should be some files in there but for some reason the are not created.
EDIT:
What's really odd here is that the file device/generic/goldfish/vendor.mk explicitly adds some data files to PRODUCT_COPY_FILES, notably:
PRODUCT_COPY_FILES += \
device/generic/goldfish/data/etc/dtb.img:dtb.img \
device/generic/goldfish/emulator-info.txt:data/misc/emulator/version.txt \
device/generic/goldfish/data/etc/apns-conf.xml:data/misc/apns/apns-conf.xml \
device/generic/goldfish/radio/RadioConfig/radioconfig.xml:data/misc/emulator/config/radioconfig.xml \
device/generic/goldfish/data/etc/iccprofile_for_sim0.xml:data/misc/modem_simulator/iccprofile_for_sim0.xml \
If I manually build them using, for example, m out/target/product/emulator_x86_64/data/misc/emulator/version.txt, the file is created at the correct location, as expected. Which leads me to wonder when entries PRODUCT_COPY_FILES are considered targets to be built and when they aren't.
EDIT2:
I got the emulator to boot but the data directory is still not being created. (Creating manually or building one target in the data dir is a workaorund).
While enabling libvirt in yocto, I am seeing below data file clash issue while building yocto image,
Below are the packages I am trying to append install to my yocto image
IMAGE_INSTALL_append = " \
packagegroup-core-boot \
qemu \
libvirt \
libvirt-libvirtd \
libvirt-virsh \
kernel-module-kvm \
kernel-module-kvm-intel \
"
But I see below issue, when I build image enabling above packages,
`Collected errors:
check_data_file_clashes: Package iptables wants to install file /-/-/-/rootfs/etc/ethertypes
But that file is already provided by package * ebtables`
FYI: I see that libvirt has both iptables and ebtables dependency.
can someone help on understanding this and how to resolve?
I tried to remove ebtables with PACKAGECONFIG_remove = "ebtables" and image is built but I when starting libvirtd service it is always in dead mode and I see some issue related to socket.
Actually, this is a problem that has no solution except:
Remove one of the packages (ebtables or iptables)
Remove the file ethertypes from one of the recipes
libvirt depends on iptables on compile time only, so I did not know why iptables is present in the image ?
Anyways, it has a config on ebtables and from your comment when you removed it from PACKAGECONFIG it failed to work. So:
I suggest check if iptables is required by other package at run time, if not remove it.
If both are required in your case, then go for the second solution which is removing the file from one of the recipes, using a bbappend file for one of them:
The block that you may need to add is:
do_install_append() {
rm ${D}/etc/ethertypes
}
either to:
meta-custom/recipes-filter/ebtables/ebtables_%.bbappend
or to:
meta-custom/recipes-extended/iptables/iptables_%.bbappend
NOTE
If you go for the second solution, you need to make sure that the file is not present in the FILES variable of the recipe that you will remove the file from, FILES_ebtables or FILES_iptables.
I wrote Yocto recipe for library and try to install to image.
When I run command "bitbake myproject" I get follow error:
ERROR: sxe-0.1-r0 do_package: QA Issue: sxe: Files/directories were installed but not shipped in any package:
/sxe
/usr/share
/usr/share/sxm
/usr/share/sxm/baselines
/usr/share/sxm/baselines/resources.psv
/usr/share/sxm/baselines/resources_bcatl.psv
/usr/share/sxm/baselines/version.txt
/usr/share/sxm/baselines/resources_bcatp.psv
/usr/share/sxm/baselines/cgs
/usr/share/sxm/baselines/alerts
/usr/share/sxm/baselines/IntegratedSportsLogos
/usr/share/sxm/baselines/states
/usr/share/sxm/baselines/IntegratedLogos
/usr/share/sxm/baselines/tabweather
/usr/share/sxm/baselines/trafficplus
/usr/share/sxm/baselines/stocks
/usr/share/sxm/baselines/ev
/usr/share/sxm/baselines/contentAssociatedImage/Album020.zip
/usr/share/sxm/baselines/gmd/GEMBase_1.15.zip
/usr/share/sxm/baselines/parking/ParkingLocation2.zip
/usr/share/sxm/baselines/safeview/sxm_safeviewcameralocations_v120.zip
/usr/share/sxm/baselines/safeview/sxm_safeviewcameratypes_v120.zip
/usr/share/sxm/baselines/IntegratedBackgroundLogos/IntPortBack160816.zip
/usr/share/sxm/baselines/IntegratedBackgroundLogos/IntLandBack160816.zip
/usr/share/sxm/baselines/fuel/sxm_fuelbaseline23.pvn1.zip
/usr/share/sxm/baselines/fuel/sxm_canfuelbaseline3.pvn1.zip
/usr/share/sxm/baselines/phonetics/T005V054.psv.zip
/usr/share/sxm/baselines/phonetics/T000V054.psv.zip
/usr/share/sxm/baselines/phonetics/T001V011.psv.zip
/usr/share/sxm/baselines/phonetics/T006V011.psv.zip
/usr/share/sxm/baselines/phonetics/T003V003.psv.zip
/usr/share/sxm/baselines/phonetics/T004V000.psv.zip
/usr/share/sxm/baselines/phonetics/T002V000.psv.zip
/usr/share/sxm/baselines/movies/sxm_theatersbaseline23.pvn1.zip
/usr/share/sxm/baselines/movies/sxm_movieratingsv0.pvn1.zip
Please set FILES such that these items are packaged. Alternatively if they are unneeded, avoid installing them or delete them within do_install.
sxe: 55 installed and not shipped files. [installed-vs-shipped]
ERROR: sxe-0.1-r0 do_package: Fatal QA errors found, failing task.
ERROR: sxe-0.1-r0 do_package: Function failed: do_package
ERROR: Logfile of failure stored in: /home/local/SWTECNN/rkhairulin/imx-yocto-bspIII/build-wayland-imx8qmmekII/tmp/work/aarch64-poky-linux/sxe/0.1-r0/temp/log.do_package.32253
ERROR: Task (/home/local/SWTECNN/rkhairulin/imx-yocto-bspIII/sources/meta-sxe/recipes-example/example/sxe_0.1.bb:do_package) failed with exit code '1'
How can I fix the issue? What should I add to the ricpe?
By default, package with recipe name contains following files (see here):
FILES_${PN} = "${bindir}/* ${sbindir}/* ${libexecdir}/* ${libdir}/lib*${SOLIBS} \
${sysconfdir} ${sharedstatedir} ${localstatedir} \
${base_bindir}/* ${base_sbindir}/* \
${base_libdir}/*${SOLIBS} \
${base_prefix}/lib/udev/rules.d ${prefix}/lib/udev/rules.d \
${datadir}/${BPN} ${libdir}/${BPN}/* \
${datadir}/pixmaps ${datadir}/applications \
${datadir}/idl ${datadir}/omf ${datadir}/sounds \
${libdir}/bonobo/servers"
So files in /usr/share/sxm and /sxe are created in ${D} deploy folder, but they are not added to any package.
You have to add this line to your recipe (see here):
FILES_${PN} += "/sxe ${datadir}/sxm"
Note /usr/share is defined as ${datadir}
If you don't need those files you can remove them from ${D}:
do_install_append(){
rm -r ${D}/sxe
rm -r ${D}/${datadir}/sxm
}
You have to tell bitbake into which package the files shall go. Probably something like
FILES_sxe += " \
{root_prefix}/sxe \
{datadir}/sxe \
"
Having said that, placing something directly under / is strongly discouraged by the FHS. For more information on the predefined prefixes and directories, look at poky/meta/conf/bitbake.conf
I'm running into an issue including python pyparted as a native dependency in one of my image creation bbclasses.
There is a python scrip that runs to create a partitioned image file, normally I run sudo apt install python-pyparted to have pyparted in the environment in ubuntu. But I'm not sure what I did (update??), the ubuntu environment is completely ignored now. I tried figuring out how to make sure the dependencies are correct in my sdimage bbclass.
do_image_sdimage[depends] = "parted-native:do_populate_sysroot \
dosfstools-native:do_populate_sysroot \
mtools-native:do_populate_sysroot \
virtual/kernel:do_deploy \
splash-images:do_deploy \
python3-native:do_populate_sysroot \
python3-pyparted-native:do_populate_sysroot \
${#d.getVar('IMAGE_BOOTLOADER', True) and d.getVar('IMAGE_BOOTLOADER', True) + ':do_deploy' or ''}"
I get an error showing
ERROR: Nothing PROVIDES 'python3-re-native' (but virtual:native:/home/dev/app/OS/sources/meta-openembedded/meta-python/recipes-extended/python-pyparted/python3-pyparted_3.10.7.bb DEPENDS on or otherwise requires it). Close matches:
python3-rpm-native
python3-native
python3-nose-native
python3-native RPROVIDES python3-re-native
ERROR: Required build target 'my-image-default' has no buildable providers.
Missing or unbuildable dependency chain was: ['my-image-default', 'python3-pyparted-native', 'python3-re-native']
based on this it looks like I should be able to do this, but the depency chain ignores python3-native's RPROVIDES?
I have an autotools-based BitBake recipe which I would like to have binaries installed in /usr/local/bin and libraries installed in /usr/local/lib (instead of /usr/bin and /usr/lib, which are the default target directories).
Here's a part of the autotools.bbclass file which I found important.
CONFIGUREOPTS = " --build=${BUILD_SYS} \
--host=${HOST_SYS} \
--target=${TARGET_SYS} \
--prefix=${prefix} \
--exec_prefix=${exec_prefix} \
--bindir=${bindir} \
--sbindir=${sbindir} \
--libexecdir=${libexecdir} \
--datadir=${datadir} \
--sysconfdir=${sysconfdir} \
--sharedstatedir=${sharedstatedir} \
--localstatedir=${localstatedir} \
--libdir=${libdir} \
...
I thought that the easiest way to accomplish what I wanted to do would be to simply change ${bindir} and ${libdir}, or perhaps change ${prefix} to /usr/local, but I haven't had any success in this area. Is there a way to change these installation variables, or am I thinking about this in the wrong way?
Update:
Strategy 1
As per Ross Burton's suggestion, I've tried adding the following to my recipe:
prefix="/usr/local"
exec_prefix="/usr/local"
but this causes the build to fail during that recipe's do_configure() task, and returns the following:
| checking for GLIB... no
| configure: error: Package requirements (glib-2.0 >= 2.12.3) were not met:
|
| No package 'glib-2.0' found
This package can be found during a normal build without these modified variables. I thought that adding the following line might allow the system to find the package metadata for glib:
PKG_CONFIG_PATH = " ${STAGING_DIR_HOST}/usr/lib/pkgconfig "
but this seems to have made no difference.
Strategy 2
I've also tried Ross Burton's other suggestion to add these variable assignments into my distribution's configuration file, but this causes it to fail during meta/recipes-extended/tzdata's do_install() task. It returns that DEFAULT_TIMEZONE is set to an invalid value. Here's the source of the error from tzdata_2015g.bb
# Install default timezone
if [ -e ${D}${datadir}/zoneinfo/${DEFAULT_TIMEZONE} ]; then
install -d ${D}${sysconfdir}
echo ${DEFAULT_TIMEZONE} > ${D}${sysconfdir}/timezone
ln -s ${datadir}/zoneinfo/${DEFAULT_TIMEZONE} ${D}${sysconfdir}/localtime
else
bberror "DEFAULT_TIMEZONE is set to an invalid value."
exit 1
fi
I'm assuming that I've got a problem with ${datadir}, which references ${prefix}.
Do you want to change paths for everything or just one recipe? Not sure why you'd want to change just one recipe to /usr/local, but whatever.
If you want to change all of them, then the simple way is to set prefix in your local.conf or distro configuration (prefix = "/usr/local").
If you want to do it in a particular recipe, then just assigning prefix="/usr/local" and exec_prefix="/usr/local" in the recipe will work.
These variables are defined in meta/conf/bitbake.conf, where you can see that bindir is $exec_prefix/bin, which is probably why assigning prefix didn't work for you.
Your first strategy was on the right track, but you were clobbering more than you wanted by changing only "prefix". If you look in sources/poky/meta/conf/bitbake.conf you'll find everything you are clobbering when you set the variable "prefix" to something other than "/usr" (like it was in my case). In order to modify only the install path with what would manually be the "--prefix" option to configure, I needed to set all the variables listed here in that recipe:
prefix="/your/install/path/here"
datadir="/usr/share"
sharedstatedir="/usr/com"
exec_prefix="/usr"