ruamel.yaml w/out C compiler on ARM - ruamel.yaml

I am trying to install ruamel.yaml on a raspberry Pi system without a C compiler and encounter a build error installing ruamel.yaml.clib (pasted below).
I see this was previously addressed for ruamel.yaml>=0.15.41, <0.16.0 (How to install ruamel.yaml w/o native extension).
Note in the output below that the path from buildroot is not valid on the device running pip install but is from the device that built the image.
$ pip install ruamel.yaml~=0.16
Defaulting to user installation because normal site-packages is not writeable
Looking in indexes: https://pypi.org/simple, https://www.piwheels.org/simple
Collecting ruamel.yaml~=0.16
Using cached ruamel.yaml-0.17.10-py3-none-any.whl (108 kB)
Collecting ruamel.yaml.clib>=0.1.2; platform_python_implementation == "CPython" and python_version < "3.10"
Using cached ruamel.yaml.clib-0.2.6.tar.gz (180 kB)
ERROR: Command errored out with exit status 1:
command: /usr/bin/python -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-2t0ptfu4/ruamel-yaml-clib/setup.py'"'"'; __file__='"'"'/tmp/pip-install-2t0ptfu4/ruamel-yaml-clib/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base /tmp/pip-pip-egg-info-zdtfb19x
cwd: /tmp/pip-install-2t0ptfu4/ruamel-yaml-clib/
Complete output (3 lines):
sys.argv ['/tmp/pip-install-2t0ptfu4/ruamel-yaml-clib/setup.py', 'egg_info', '--egg-base', '/tmp/pip-pip-egg-info-zdtfb19x']
test compiling /tmp/tmp_ruamel_5les1064/test_ruamel_yaml.c -> test_ruamel_yaml compile error: /tmp/tmp_ruamel_5les1064/test_ruamel_yaml.c
Exception: command '{path from buildroot}/aarch64-buildroot-linux-gnu-gcc' failed with exit status 1
----------------------------------------
ERROR: Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.
A preferred solution would be to have a ruamel.yaml.clib wheel for ARM architectures or to make clib dependency optional (pip install ruamel.yaml[clib])

I'm not sure why this broke after 0.16, but I'll try to have a look as to why this fails again (it might be that setuptools now throws a different Exception that is not caught).
Wheels for ruamel.yaml.clib in aarch64 architecture are availalbe on piwheels.
You should be able to install those after adding:
[global]
extra-index-url=https://www.piwheels.org/simple
to /etc/pip.conf.
Disclaimer: I have no control over how wheels on piwheels are generated.

Related

Is there a way to fix this error? I'm trying to understand it but don't really know

ERROR: Command errored out with exit status 1:
command: 'C:\Users\panda\anaconda3\envs\dmodel\python.exe' -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'C:\Users\panda\AppData\Local\Temp\pip-req-build-0lhigfqj\bindings/torch\setup.py'"'"'; file='"'"'C:\Users\panda\AppData\Local\Temp\pip-req-build-0lhigfqj\bindings/torch\setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(file) if os.path.exists(file) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, file, '"'"'exec'"'"'))' egg_info --egg-base 'C:\Users\panda\AppData\Local\Temp\pip-pip-egg-info-h3yznucj'
cwd: C:\Users\panda\AppData\Local\Temp\pip-req-build-0lhigfqj\bindings/torch
Complete output (6 lines):
Traceback (most recent call last):
File "", line 1, in
File "C:\Users\panda\AppData\Local\Temp\pip-req-build-0lhigfqj\bindings/torch\setup.py", line 41, in
raise RuntimeError("Could not locate a supported Microsoft Visual C++ installation")
RuntimeError: Could not locate a supported Microsoft Visual C++ installation
Building PyTorch extension for tiny-cuda-nn version 1.6
----------------------------------------
WARNING: Discarding git+https://github.com/NVlabs/tiny-cuda-nn/#subdirectory=bindings/torch. Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.
ERROR: Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.

How to fix "pytest: error: unrecognized arguments: -n"

Hi I am trying to run my tests parallely(pytest-xdist) on the azure pipelines.
Till now the tests were running perfectly fine.
Suddenly the pytest is throwing a weird error saying "unrecognized argument".
The file name : integration_test.py
Command used : pytest -n 5 --tb=short integration_test.py -v -s --> to run 5 tests parallely
Total number of tests : 57
Versions :
pytest==6.2.5
pytest-xdist==2.3.0
Even tried with the latest versions of these 2 modules.
Error :
ERROR: usage: pytest [options] [file_or_dir] [file_or_dir] [...]
pytest: error: unrecognized arguments: -n integration_test.py
How can I overcome this error?
This error is what you encountered:
As hoefling mentioned, the solution is to install the pytest-xdist:
pip install pytest-xdist
On MacOS just running pytest might be ran by a different Python's version than you thought.
$ pytest
============================================================================== test session starts ===============================================================================
platform darwin -- Python 3.9.12, pytest-6.2.5, py-1.10.0, pluggy-0.13.1
rootdir: [REDACTED]
plugins: anyio-3.5.0, cov-3.0.0
While
$ python3 -m pytest
============================================================================== test session starts ===============================================================================
platform darwin -- Python 3.10.6, pytest-7.1.2, pluggy-1.0.0
rootdir: [REDACTED]
plugins: xdist-2.5.0, forked-1.4.0, pylama-8.4.1
Be careful, and launch it as a module :)

Error on building Openembedded for Intel Quark

Trying to build Linux for Advantech UBC-221 that has Intel Quark processor.
The system is Debian 9, running on VMware. Was able to build poky succefully before.
Let me summarize what I was doing:
mkdir quark
cd quark
git clone git://git.yoctoproject.org/meta-intel-quark
git clone git://git.openembedded.org/openembedded-core
git clone git://git.yoctoproject.org/poky
cp -r poky/bitbake .
cd openembedded-core
source oe-init-build-env
bitbake-layers add-layer ~/quark/meta-intel-quark/
bitbake core-image-base
Error code I get:
WARNING: Layer quark-bsp should set LAYERSERIES_COMPAT_quark-bsp in its conf/layer.conf file to list the core layer names it is compatible with.
WARNING: Layer quark-bsp should set LAYERSERIES_COMPAT_quark-bsp in its conf/layer.conf file to list the core layer names it is compatible with.
WARNING: /home/bgabor/quark/meta-intel-quark/recipes-kernel/linux/linux-yocto-quark_3.8.bb: Variable key RDEPENDS_${KERNEL_PACKAGE_NAME}-base (${KERNEL_PACKAGE_NAME}-image) replaces original key RDEPENDS_kernel-base ().
WARNING: /home/bgabor/quark/meta-intel-quark/recipes-bsp/grub/grub_0.97.bb: Exception during build_dependencies for CFLAGS
WARNING: /home/bgabor/quark/meta-intel-quark/recipes-bsp/grub/grub_0.97.bb: Error during finalise of /home/bgabor/quark/meta-intel-quark/recipes-bsp/grub/grub_0.97.bb
ERROR: ExpansionError during parsing /home/bgabor/quark/meta-intel-quark/recipes-bsp/grub/grub_0.97.bb
Traceback (most recent call last):
bb.data_smart.ExpansionError: Failure expanding variable CFLAGS, expression was -O2 -pipe -g -feliminate-unused-debug-types -fdebug-prefix-map=/home/bgabor/quark/openembedded-core/build/tmp-glibc/work/i586-oe-linux/grub/0.97+gitAUTOINC+5775f32a62-r0=/usr/src/debug/grub/0.97+gitAUTOINC+5775f32a62-r0 -fdebug-prefix-map=/home/bgabor/quark/openembedded-core/build/tmp-glibc/work/i586-oe-linux/grub/0.97+gitAUTOINC+5775f32a62-r0/recipe-sysroot= -fdebug-prefix-map=/home/bgabor/quark/openembedded-core/build/tmp-glibc/work/i586-oe-linux/grub/0.97+gitAUTOINC+5775f32a62-r0/recipe-sysroot-native= -Os -fno-strict-aliasing -Wall -Werror -Wno-shadow -Wno-unused -Wno-pointer-sign -DINTEL_QUARK_TEST=${#base_contains('PACKAGECONFIG', 'grub_test', '1', '0', d)} which triggered exception NameError: name 'base_contains' is not defined
Summary: There were 5 WARNING messages shown.
Summary: There was 1 ERROR message shown, returning a non-zero exit code.
I'm pretty much stuck here. Some help would be greatly appreciated!
Edit:
Tried to build on Debian 7.11 32 bit system. It went better however another error came:
bgabor#debian:~/quark/dizzy/build$ bitbake core-image-base
WARNING: Host distribution "Debian-7.11" has not been validated with this version of the build system; you may possibly experience unexpected failures. It is recommended that you use a tested distribution.
Loading cache: 100% |####################################################################################################################################################| ETA: 00:00:00
Loaded 1292 entries from dependency cache.
NOTE: Resolving any missing task queue dependencies
Build Configuration:
BB_VERSION = "1.24.0"
BUILD_SYS = "i686-linux"
NATIVELSBSTRING = "Debian-7.11"
TARGET_SYS = "i586-poky-linux"
MACHINE = "quark"
DISTRO = "poky"
DISTRO_VERSION = "1.7.3"
TUNE_FEATURES = "m32 i586"
TARGET_FPU = ""
meta
meta-yocto
meta-yocto-bsp = "dizzy:58863ad092c9a279e305c841dbb4353de2ecfae8"
meta-intel-quark = "master:a314f0ceea986fde42d5d9b0ea449f7a563e9351"
NOTE: Preparing runqueue
NOTE: Executing SetScene Tasks
NOTE: Executing RunQueue Tasks
ERROR: Unable to install packages. Command '/home/bgabor/quark/dizzy/build/tmp/sysroots/i686-linux/usr/bin/smart --quiet --data-dir=/home/bgabor/quark/dizzy/build/tmp/work/quark-poky-linux/core-image-minimal-initramfs/1.0-r0/rootfs/var/lib/smart install -y busybox#i586 initramfs-live-install-efi#i586 run-postinsts#all initramfs-live-install#i586 udev#i586 initramfs-live-boot#quark base-passwd#i586' returned 1:
error: Can't install initramfs-live-install-1.0-r9#i586: no package provides grub
ERROR: Function failed: do_rootfs
ERROR: Logfile of failure stored in: /home/bgabor/quark/dizzy/build/tmp/work/quark-poky-linux/core-image-minimal-initramfs/1.0-r0/temp/log.do_rootfs.29435
ERROR: Task 240 (/home/bgabor/quark/dizzy/meta/recipes-core/images/core-image-minimal-initramfs.bb, do_rootfs) failed with exit code '1'
NOTE: Tasks Summary: Attempted 2529 tasks of which 2527 didn't need to be rerun and 1 failed.
No currently running tasks (2435 of 2531)
Summary: 1 task failed:
/home/bgabor/quark/dizzy/meta/recipes-core/images/core-image-minimal-initramfs.bb, do_rootfs
Summary: There was 1 WARNING message shown.
Summary: There were 2 ERROR messages shown, returning a non-zero exit code.
It does not find grub package, however I can see there are two in the recipes:
bgabor#debian:~/quark/dizzy/build$ bitbake-layers show-recipes | grep "grub" -A 3
Parsing recipes..done.
grub:
meta-intel-quark 0.97+gitAUTOINC+5775f32a62
meta 0.97
meta 2.00
--
grub-conf:
meta-intel-quark 1.0
grub-efi:
meta 2.00
gsettings-desktop-schemas:
meta 3.10.1
Tried to clean and then bitbake again but no result.
What might be the problem here?
First, Poky includes OpenEmbedded Core so you don't need to clone both, just clone Poky.
Second, the meta-intel-iot layer is pretty dead and only formally supports the Daisy release of Poky (see the README). You may have luck using newer releases, but obviously not git master (aka Thud).
https://wiki.yoctoproject.org/wiki/Releases lists the releases. Instead of checking out master of Poky, check out the relevant release branch. I suggest starting with daisy but trying something a little newer such as jethro would be a good idea.

Failed building wheel in Google ML engine

I follow this tutorial to submit machine learning job to Google ML engine. Then I faced the error ImportError: No module named matplotlib.pyplot but solved it by adding the Matplotlib into RequiredPackages in setup.py. Then I face another error which is ImportError: No module named _tkinter, please install the python-tk package. I found this solution and this solution but do not help and give me another error.
Failed building wheel for my-package
Command "/usr/bin/python -u -c "import setuptools, tokenize;file='/tmp/pip-T6kjZl-build/setup.py';f=getattr(tokenize, 'open', open)(file);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, file, 'exec'))" install --record /tmp/pip-4LpzWh-record/install-record.txt --single-version-externally-managed --compile --user --prefix=" failed with error code 1 in /tmp/pip-T6kjZl-build/
Command '['pip', 'install', '--user', '--upgrade', '--force-reinstall', '--no-deps', u'my-package-0.1.1.tar.gz']' returned non-zero exit status 1
My setup.py.
"""Setup script for object_detection."""
import logging
import subprocess
from setuptools import find_packages
from setuptools import setup
from setuptools.command.install import install
class CustomCommands(install):
def RunCustomCommand(self, command_list):
p = subprocess.Popen(
command_list,
stdin=subprocess.PIPE,
stdout=subprocess.PIPE,
stderr=subprocess.STDOUT)
# Can use communicate(input='y\n'.encode()) if the command run requires
# some confirmation.
stdout_data, _ = p.communicate()
logging.info('Log command output: %s', stdout_data)
if p.returncode != 0:
raise RuntimeError('Command %s failed: exit code: %s', (command_list, p.returncode))
def run(self):
self.RunCustomCommand(['apt-get', 'update'])
self.RunCustomCommand(
['apt-get', 'install', '-y', 'python-tk'])
install.run(self)
REQUIRED_PACKAGES = ['Pillow>=1.0', 'Matplotlib>=2.1']
setup(
name='object_detection',
version='0.1',
install_requires=REQUIRED_PACKAGES,
include_package_data=True,
packages=[p for p in find_packages() if
p.startswith('object_detection')],
description='Tensorflow Object Detection Library',
cmdclass={
'install': CustomCommands,
})
The tutorial blog link that you are following is outdated I think. I followed this one
you need to change runtime to 1.9 instead of 1.8 in cloud ml training command
From tensorflow/models/research/
gcloud ml-engine jobs submit training `whoami`_object_detection_pets_`date +%m_%d_%Y_%H_%M_%S` \
--runtime-version 1.9 \
--job-dir=gs://${YOUR_GCS_BUCKET}/model_dir \
--packages dist/object_detection-0.1.tar.gz,slim/dist/slim-0.1.tar.gz,/tmp/pycocotools/pycocotools-2.0.tar.gz \
--module-name object_detection.model_main \
--region us-central1 \
--config object_detection/samples/cloud/cloud.yml \
-- \
--model_dir=gs://${YOUR_GCS_BUCKET}/model_dir \
--pipeline_config_path=gs://${YOUR_GCS_BUCKET}/data/faster_rcnn_resnet101_pets.config
after changing runtime everything worked perfect. No need to change any setup file. No Library errors.

Raspbian dpkg error /var/lib/dpkg/triggers/File

I have just installed raspbian for my raspberry pi and I tried to install fswebcam with sudo apt-get install fswebcam and I get the error:
dpkg: error: syntax error in file triggers file `/var/lib/dpkg/triggers/File'
E: Sub-process /usr/bin/dpkg returned an error code (2)
I then thought about doing an update so did:
sudo apt-get update
sudo apt-get upgrade
This also produces the same error:
dpkg: error: syntax error in file triggers file `/var/lib/dpkg/triggers/File'
E: Sub-process /usr/bin/dpkg returned an error code (2)
The packages download but can not install.
Here is the cat of the file and it does seem wrong:
/usr/sharm/Ijbo0knsvadm-infO
▒e;{▒my
#▒q▒}db+/}sR+sh!reman man-db/uw▒>l▒caMom`O▒m!f)Er
/u3z/locax+#rcre/ma~--d▒
'6sZ/▒%▒Z6/maN ▒sn%ef▒e▒d/mqn`eaN-▒2
-T{3/,HbNarL-m(j▒Y▒gn?dabihF/▒yn/▒o▒u▒7s li▒g▒+b`.p%0;a▒m`f
)▒ks/lib/miG*mk▒En▒c lyb'dib2.-▒*armhf
/As▒5rx▒2E/▒xib-0.8.3c▒▒e"! ,cbomib2n0=zirmh/muk/Scil0ag/d%cAs▒o}sp%d#g/mime/packages miLe)cwrxfst
-u{s▒!▒kJ%/mho▒'qackawe;(S▒a▒'d▒m+/e-info
.5▒rlibog$j▒▒92bufm2.▒?:-1 ▒l▒qders libedc,pixbu&2.-2▒BiHf▒u▒b/diB)Az▒=nifT8▒wOuea▒ih$-gmj▒u▒.#}▒"▒`'2.10.0/loadgR{▒|i"gdN-#j▒▒▒f2.0-0>abolf-usr/s▒ar'/▒o▒6sb▒/n6Bonf+gusr-s``ve?ehosdw▒h{:5'fkod▒ g▒▒4!nO▒ig
/ts▒?{(I▒e->mYIf▒v▒nT1▒f4K▒nf+g
/usr/mi▒o▒rLoliNu▒=▒~u%abHHdlj▒:.0&▒▒ .0m▒mmduleS0▒9kg▒k:.)0:armh/}rr.l▒b/gtk-2.1/▒.10/p▒yL-gdules`liCgtk2.0d0▒D▒-(,+/ts▒'sHe2e-Bxymicatmozs0ndsktop-film,W▒a▒c
/usb/3``r▒/gcknv-defa}~D?▒o▒=▒f2X/5krH3▒u/▒c/,▒▒#~d!|opy(&Cg▒V4
/pSf-shaRe▒wcgnF+skhUi!{ gco~▒s▒5{r>u(a3e/Qo0pl▒r ;p▒f
/u3r.L▒▒prm-diFqX)gnuepb▒`f/gti-;o0/▒.l▒?iLmkve&Ls|▒0gTk▒#-0:qr-`f
w/0▒3~2./im▒od7|e,▒i`guk▒3-`*!Ex▒▒msr/sjarD▒unu!mandJ▒urR/▒yB/▒unu menu
/etc▒ee▒q-/eDH▒▒c }e.=
/Tsr/sha▒e/kcoos▒x▒color xi"▒▒o0-icon-theiesR▒y▒▒uoxc▒f3'gNo▒u onNme-i#on
▒▒e/'
I would prefer not having to re-install raspbian but I will if I must
Thanks
In the end I unfortunately had to format the sd card and reinstall raspbian. The output of the file is a lot more comprehensive now. I will put the contents at a later date.