Pytest "Error: could not load path/to/conftest.py" - pytest

I get the following error when I try to run pytest repo/tests/test_file.py:
$ pytest repo/tests/test_file.py
Traceback (most recent call last):
File "/Users/marlo/anaconda3/envs/venv/lib/python3.6/site-packages/_pytest/config.py", line 329, in _getconftestmodules
return self._path2confmods[path]
KeyError: local('/Users/marlo/repo/tests/test_file.py')
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/Users/marlo/anaconda3/envs/venv/lib/python3.6/site-packages/_pytest/config.py", line 329, in _getconftestmodules
return self._path2confmods[path]
KeyError: local('/Users/marlo/repo/tests')
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/Users/marlo/anaconda3/envs/venv/lib/python3.6/site-packages/_pytest/config.py", line 362, in _importconftest
return self._conftestpath2mod[conftestpath]
KeyError: local('/Users/marlo/repo/conftest.py')
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/Users/marlo/anaconda3/envs/venv/lib/python3.6/site-packages/_pytest/config.py", line 368, in _importconftest
mod = conftestpath.pyimport()
File "/Users/marlo/anaconda3/envs/venv/lib/python3.6/site-packages/py/_path/local.py", line 686, in pyimport
raise self.ImportMismatchError(modname, modfile, self)
py._path.local.LocalPath.ImportMismatchError: ('conftest', '/home/venvuser/venv/conftest.py', local('/Users/marlo/repo/conftest.py'))
ERROR: could not load /Users/marlo/repo/conftest.py
My repo structure is
lib/
-tests/
-test_file.py
app/
-test_settings.py
pytest.ini
conftest.py
...
Other people have run this code fine, and according to this question (and this one), my structure is good and I am not missing any files. I can only conclude that something about my computer or project set-up is not right. If you have any suggestions or insights that I may be missing, please send them my way!
-------------------------------MORE DETAILS------------------------------
test_file.py:
def func(x):
return x + 1
def test_answer():
assert func(3) == 5
pytest.ini:
[pytest]
DJANGO_SETTINGS_MODULE = app.test_settings
python_files = tests.py test_* *_tests.py *test.py

I have docker as well as running pytest outside of docker too, and for me a much lower-impact fix whenever this crops up is to delete all the compiled python files
find . -name \*.pyc -delete

I figured it out and I'll answer in case others have the same issue:
I didn't even take into consideration that I had a docker container (of the same app) in the repo directory and, although I was not running the docker container, it was influencing the filepaths somehow.
To fix this:
I re-cloned the repo from the remote source into a new folder so that nothing from the old repo could "contaminate" it.
Updated my virtual environment with the .yml specifications of the clean repo
$ conda env update --name project --file project.yml
My project uses a postgres database, so I dropped it and created a new one
$ dropdb projectdb
$ createdb projectdb
Since my project uses mongo, I also dropped that database
$ mongo projectdb --eval "db.dropDatabase()"
Installed a clean pytest
$ pip uninstall pytest
$ pip install pytest
...and voilà! I could run pytest.
Many thanks to #hoefling and others who helped me debug.

I was running docker as well, but it seems my problem was different.
I was using an old version of pytest:
platform linux -- Python 3.9.7, pytest-3.7.2, py-1.10.0, pluggy-1.0.0
which stopped working after my ubuntu image was pulling python 3.10 by default.
my solution was to update (and fix) the dockerfile image to use:
FROM python:3.10
instead of python:latest, and update the pytest version as well.

Related

Yocto Build error; Initialising tasks...ERROR: An uncaught exception occurred in runqueue (from Yocto newbies)

The error:
2022-06-23T00:57:52.6893303Z **Initialising tasks...ERROR: An uncaught exception occurred in runqueue**
2022-06-23T00:57:52.6894429Z Traceback (most recent call last):
2022-06-23T00:57:52.6896192Z File "/__w/1/s/msocs-rackmanager-bsp/yocto/bitbake/lib/bb/runqueue.py", line 1524, in RunQueue.execute_runqueue():
2022-06-23T00:57:52.6896660Z try:
2022-06-23T00:57:52.6896927Z > return self._execute_runqueue()
2022-06-23T00:57:52.6897250Z except bb.runqueue.TaskFailure:
2022-06-23T00:57:52.6898244Z File "/__w/1/s/msocs-rackmanager-bsp/yocto/bitbake/lib/bb/runqueue.py", line 1474, in RunQueue._execute_runqueue():
2022-06-23T00:57:52.6899084Z self.rqdata.init_progress_reporter.next_stage()
2022-06-23T00:57:52.6899368Z > self.start_worker()
2022-06-23T00:57:52.6899675Z self.rqdata.init_progress_reporter.next_stage()
2022-06-23T00:57:52.6900270Z File "/__w/1/s/msocs-rackmanager-bsp/yocto/bitbake/lib/bb/runqueue.py", line 1295, in RunQueue.start_worker():
2022-06-23T00:57:52.6900860Z for mc in self.rqdata.dataCaches:
2022-06-23T00:57:52.6901154Z > self.worker[mc] = self._start_worker(mc)
2022-06-23T00:57:52.6901387Z
2022-06-23T00:57:52.6902344Z File "/__w/1/s/msocs-rackmanager-bsp/yocto/bitbake/lib/bb/runqueue.py", line 1268, in RunQueue._start_worker(mc='', fakeroot=False, rqexec=None):
2022-06-23T00:57:52.6902932Z worker.stdin.write(b"<extraconfigdata>" + pickle.dumps(self.cooker.extraconfigdata) + b"</extraconfigdata>")
2022-06-23T00:57:52.6903397Z > worker.stdin.write(b"<workerdata>" + pickle.dumps(workerdata) + b"</workerdata>")
2022-06-23T00:57:52.6904226Z worker.stdin.flush()
2022-06-23T00:57:52.6904497Z BrokenPipeError: [Errno 32] Broken pipe
2022-06-23T00:57:52.6904644Z
2022-06-23T00:57:52.6905449Z ERROR: Running idle function
2022-06-23T00:57:52.6906161Z Traceback (most recent call last):
2022-06-23T00:57:52.6908003Z File "/__w/1/s/msocs-rackmanager-bsp/yocto/bitbake/lib/bb/cooker.py", line 1438, in buildTargetsIdle(server=<ProcessServer(ProcessServer-1, started)>, rq=<bb.runqueue.RunQueue object at 0x7fd16f08e4a8>, abort=False):
2022-06-23T00:57:52.6908771Z try:
2022-06-23T00:57:52.6910081Z > retval = rq.execute_runqueue()
2022-06-23T00:57:52.6910509Z except runqueue.TaskFailure as exc:
2022-06-23T00:57:52.6911385Z File "/__w/1/s/msocs-rackmanager-bsp/yocto/bitbake/lib/bb/runqueue.py", line 1524, in RunQueue.execute_runqueue():
2022-06-23T00:57:52.6911836Z try:
2022-06-23T00:57:52.6912141Z > return self._execute_runqueue()
2022-06-23T00:57:52.6912497Z except bb.runqueue.TaskFailure:
2022-06-23T00:57:52.6913487Z File "/__w/1/s/msocs-rackmanager-bsp/yocto/bitbake/lib/bb/runqueue.py", line 1474, in RunQueue._execute_runqueue():
2022-06-23T00:57:52.6913882Z self.rqdata.init_progress_reporter.next_stage()
2022-06-23T00:57:52.6914155Z > self.start_worker()
2022-06-23T00:57:52.6914634Z self.rqdata.init_progress_reporter.next_stage()
2022-06-23T00:57:52.6915413Z File "/__w/1/s/msocs-rackmanager-bsp/yocto/bitbake/lib/bb/runqueue.py", line 1295, in RunQueue.start_worker():
2022-06-23T00:57:52.6915991Z for mc in self.rqdata.dataCaches:
2022-06-23T00:57:52.6916309Z > self.worker[mc] = self._start_worker(mc)
2022-06-23T00:57:52.6916551Z
2022-06-23T00:57:52.6917175Z File "/__w/1/s/msocs-rackmanager-bsp/yocto/bitbake/lib/bb/runqueue.py", line 1268, in RunQueue._start_worker(mc='', fakeroot=False, rqexec=None):
2022-06-23T00:57:52.6917793Z worker.stdin.write(b"<extraconfigdata>" + pickle.dumps(self.cooker.extraconfigdata) + b"</extraconfigdata>")
2022-06-23T00:57:52.6918746Z > worker.stdin.write(b"<workerdata>" + pickle.dumps(workerdata) + b"</workerdata>")
2022-06-23T00:57:52.6919101Z worker.stdin.flush()
2022-06-23T00:57:52.6919963Z BrokenPipeError: [Errno 32] Broken pipe
2022-06-23T00:57:52.6920269Z
2022-06-23T00:57:52.8226862Z
2022-06-23T00:57:52.8228092Z Summary: There were 11 WARNING messages shown.
2022-06-23T00:57:52.8229138Z Summary: There were 2 ERROR messages shown, returning a non-zero exit code.
The bsp file has these lines:
<remote fetch="https://git.yoctoproject.org/git" name="yocto"/>
<remote fetch="git://git.openembedded.org" name="oe"/>
<remote> fetch="https://git.yoctoproject.org" name="ti"/>
I created a Linux VM using Hyper-V on a Windows system. I ran through the steps manually and created the build without issues. When I add the similar steps onto the Azure Pipeline, I ran into problem. I got this unhandled exception.
I don't have the Yocto code to look as I don't have access to the Azure build machine. I am not sure how to debug this issue and why I am seeing this failure. Can someone provide some suggestions?
So I figured out the issue. In my local VM build, there was no locale issue. When I use the Azure Dev Ops Pipeline, I ran into this error:
Please use a locale setting which supports UTF-8 (such as LANG=en_US.UTF-8).
Python can't change the filesystem locale after loading so we need a UTF-8 when Python starts or things won't work
I tried to fix this error. LANG=en_US.UTF-8 didn't work. I fixed it using something. This error went away but later during Initialising tasks, I got the unhandled exception.
So I searched on the web and came up with this fix for the locale:
sudo apt-get install locales
sudo locale-gen en_US.UTF-8
export LC_ALL=en_US.UTF-8
export LANG=en_US.UTF-8
# sudo dpkg-reconfigure locales
cat "LC_ALL=en_US.UTF-8" > /etc/environment
cat "LANG=en_US.UTF-8" >> /etc/environment
cat /etc/environment > /etc/default/locale
I probably don't need all of them but I wanted to be sure. This got me passed the unhandled exception shown.

How Can I Solve "Failed To Source Bitbake" With Xilinx Petalinux SDK

I am running Ubuntu 16.04 with Xilinx Petalinux 2018.03 SDK. After a number of successful compilations I am now facing this error
$ petalinux-build
[INFO] building project
[INFO] sourcing bitbake
ERROR: Failed to source bitbake
ERROR: Failed to build project
How can I solved this issue?
Another reason to get the errors "ERROR: Failed to source bitbake" as well as "ERROR: Failed to build project" is a possible upgrade of Python on the build machine. The Petalinux SDK requires python v2 (>= 2.7.3) for the 2018.3 edition.
You can check under [YOUR_PROJECT]/build/build.log and you might see a log similar to this one below:
[INFO] building project
[INFO] sourcing bitbake
SDK environment now set up; additionally you may now run devtool to perform development tasks.
Run devtool --help for further details.
OpenEmbedded requires 'python' to be python v2 (>= 2.7.3), not python v3.
Please set up python v2 as your default 'python' interpreter.
ERROR: Failed to source bitbake
ERROR: Failed to build project
To remedy this issue remove the symbolic link under /usr/bin and make sure to create a new one that is pointing to Python 2.7:
sudo rm /usr/bin/python
sudo ln -s /usr/bin/python2.7 /usr/bin/python
First you need to investigate the error a little further, do this:
source /opt/pkg/petalinux/2018.3/settings.sh
It will return something similar to this below:
PetaLinux environment set to '/opt/pkg/petalinux/2018.3'
INFO: Checking free disk space
INFO: Checking installed tools
INFO: Checking installed development libraries
INFO: Checking network and other services
Source the environment setup:
source /opt/pkg/petalinux/2018.3/components/yocto/source/aarch64/environment-setup-aarch64-xilinx-linux
followed by:
devtool --help
In my case I can see more about the actual error:
NOTE: Starting bitbake server...
ERROR: Unable to start bitbake server
ERROR: Last 10 lines of server log for this session (/opt/pkg/petalinux/2018.3/components/yocto/source/aarch64/bitbake-cookerdaemon.log):
Traceback (most recent call last):
File "/opt/pkg/petalinux/2018.3/components/yocto/source/aarch64/layers/core/bitbake/lib/bb/daemonize.py", line 77, in createDaemon
function()
File "/opt/pkg/petalinux/2018.3/components/yocto/source/aarch64/layers/core/bitbake/lib/bb/server/process.py", line 433, in _startServer
self.cooker = bb.cooker.BBCooker(self.configuration, self.featureset)
File "/opt/pkg/petalinux/2018.3/components/yocto/source/aarch64/layers/core/bitbake/lib/bb/cooker.py", line 178, in __init__
self.configwatcher = pyinotify.WatchManager()
File "/opt/pkg/petalinux/2018.3/components/yocto/source/aarch64/layers/core/bitbake/lib/pyinotify.py", line 1764, in __init__
raise OSError(err % self._inotify_wrapper.str_errno())
OSError: Cannot initialize new instance of inotify, Errno=Too many open files (EMFILE)
ERROR: Unable to start bitbake server
ERROR: Last 10 lines of server log for this session (/opt/pkg/petalinux/2018.3/components/yocto/source/aarch64/bitbake-cookerdaemon.log):
Traceback (most recent call last):
File "/opt/pkg/petalinux/2018.3/components/yocto/source/aarch64/layers/core/bitbake/lib/bb/daemonize.py", line 77, in createDaemon
function()
File "/opt/pkg/petalinux/2018.3/components/yocto/source/aarch64/layers/core/bitbake/lib/bb/server/process.py", line 433, in _startServer
self.cooker = bb.cooker.BBCooker(self.configuration, self.featureset)
File "/opt/pkg/petalinux/2018.3/components/yocto/source/aarch64/layers/core/bitbake/lib/bb/cooker.py", line 178, in __init__
self.configwatcher = pyinotify.WatchManager()
File "/opt/pkg/petalinux/2018.3/components/yocto/source/aarch64/layers/core/bitbake/lib/pyinotify.py", line 1764, in __init__
raise OSError(err % self._inotify_wrapper.str_errno())
OSError: Cannot initialize new instance of inotify, **Errno=Too many open files (EMFILE)**
This is pointing to the /proc/sys/fs/inotify/max_user_instances that need to be increased. In my case I went from 128 to 256 by doing this:
sudo su
echo 256 > /proc/sys/fs/inotify/max_user_instances
You need to become root with "su" and change the mac_user_instances.

bitbake fails at the simplest recipe

Just installed Yocto. On a morty branch. Executed the following commands:
cd poky
source oe-init-build-env build-qemuarm
In conf/local.conf changed the name of the machine to MACHINE ?= "qemuarm"
Then executed the following:
$ bitbake core-image-minimal
Loading cache: 100% |##########################################################################################################| Time: 0:00:00
Loaded 1320 entries from dependency cache.
ERROR: Execution of event handler 'sstate_eventhandler2' failed
Traceback (most recent call last):
File "/home/some-user/projects/melp/poky/meta/classes/sstate.bbclass", line 1015, in sstate_eventhandler2(e=<bb.event.ReachableStamps object at 0x7fbc17f2e0f0>):
for l in lines:
> (stamp, manifest, workdir) = l.split()
if stamp not in stamps:
ValueError: not enough values to unpack (expected 3, got 1)
ERROR: Command execution failed: Traceback (most recent call last):
File "/home/some-user/projects/melp/poky/bitbake/lib/bb/command.py", line 101, in runAsyncCommand
self.cooker.updateCache()
File "/home/some-user/projects/melp/poky/bitbake/lib/bb/cooker.py", line 1658, in updateCache
bb.event.fire(event, self.databuilder.mcdata[mc])
File "/home/some-user/projects/melp/poky/bitbake/lib/bb/event.py", line 201, in fire
fire_class_handlers(event, d)
File "/home/some-user/projects/melp/poky/bitbake/lib/bb/event.py", line 124, in fire_class_handlers
execute_handler(name, handler, event, d)
File "/home/some-user/projects/melp/poky/bitbake/lib/bb/event.py", line 96, in execute_handler
ret = handler(event)
File "/home/some-user/projects/melp/poky/meta/classes/sstate.bbclass", line 1015, in sstate_eventhandler2
(stamp, manifest, workdir) = l.split()
ValueError: not enough values to unpack (expected 3, got 1)
It looks like it is a python error. Does anyone know what is the issue? Am I using the wrong version?
Here is the output of python --version
$ python --version
Python 2.7.12
What am I doing wrong?
You realise that Morty is 18 months old and in a few weeks will be longer supported right?
Anyway, looks like the sstate-cache/ somehow is corrupted. Delete your tmp/ and sstate-cache/ directories and try again.

Mongo connector with neo4j not working

I'm running the following command:
mongo-connector -m localhost:27017 -t http://localhost:7474/db/data -d neo4j_doc_manager
But getting following error:
No handlers could be found for logger "mongo_connector.util"<br/>
Traceback (most recent call last):<br/>
File "/usr/local/bin/mongo-connector", line 11, in <module>
sys.exit(main())<br/>
File "/usr/local/lib/python2.7/site-packages/mongo_connector/util.py", line 85, in wrapped<br/>
func(*args, **kwargs)
File "/usr/local/lib/python2.7/site-<br/>packages/mongo_connector/connector.py", line 1041, in main
conf.parse_args()<br/>
File "/usr/local/lib/python2.7/site-packages/mongo_connector/config.py", line 118, in parse_args
option, dict((k, values.get(k)) for k in option.cli_names))
File "/usr/local/lib/python2.7/site-packages/mongo_connector/connector.py", line 824, in apply_doc_managers
module = import_dm_by_name(dm['docManager'])<br/>
File "/usr/local/lib/python2.7/site-packages/mongo_connector/connector.py", line 814, in import_dm_by_name
"vailable doc managers." % full_name)
mongo_connector.errors.InvalidConfiguration: Could not import mongo_connector.doc_managers.neo4j_doc_manager. It could be that this doc manager has been moved out of this project and is maintained elsewhere. Make sure that you have the doc manager installed alongside mongo-connector. Check the README for a list of available doc managers.
NOTE: I'm using neo4j 3.0, os: macosx, python: 2.7**
This appears to be an error caused by failing to pin the py2neo dependency to a specific version. This project was written using py2neo 2.0.7 but failed to pin that version, as py2neo 3.0 is now being pulled in a few things are breaking.
We will push an update for 3.0 soon, in the meantime you can get around this by uninstalling py2neo and installing version 2.0.7:
lyonwj#lyonwjs-MacBook-Pro-2 ~/n/mongo_demo> pip uninstall py2neo
Uninstalling py2neo-3
...
lyonwj#lyonwjs-MacBook-Pro-2 ~/n/mongo_demo> pip install py2neo==2.0.7
Collecting py2neo==2.0.7
Downloading py2neo-2.0.7.tar.gz (251kB)
100% |████████████████████████████████| 256kB 2.6MB/s
Installing collected packages: py2neo
Running setup.py install for py2neo ... done
Successfully installed py2neo-2.0.7
lyonwj#lyonwjs-MacBook-Pro-2 ~/n/mongo_demo>
mongo-connector -m localhost:27017 -t http://localhost:7474/db/data -d neo4j_doc_manager
Logging to mongo-connector.log.
See https://github.com/neo4j-contrib/neo4j_doc_manager/issues/60

Eclipse using multiple Python interpreters with execnet

I'm using the execnet package to allow communication between Python scripts interpreted by different Python interpreters.
The following code (test_execnet.py):
import execnet
for python_version in ('python', 'python3'):
try:
gw = execnet.makegateway("popen//python="+python_version)
ch = gw.remote_exec('channel.send(1/3)')
res = ch.receive()
print(python_version, ': ', res, sep ="")
except:
print('problems with ', python_version)
Runs perfectly in the command-line Terminal, showing the following output:
$ python3 test_execnet.py
python: 0
python3: 0.333333333333
However, if I try to run the same code from within the Eclipse IDE, I get the following error:
'import site' failed; use -v for traceback
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "<string>", line 4, in <module>
File "<string>", line 2, in <module>
File "/Library/Frameworks/Python.framework/Versions/3.1/lib/python3.1/site-packages/execnet/gateway_base.py", line 8, in <module>
import sys, os, weakref
File "/Library/Frameworks/Python.framework/Versions/3.1/lib/python3.1/os.py", line 380, in <module>
from _abcoll import MutableMapping # Can't use collections (bootstrap)
File "/Library/Frameworks/Python.framework/Versions/3.1/lib/python3.1/_abcoll.py", line 54
class Hashable(metaclass=ABCMeta):
^
SyntaxError: invalid syntax
problems with python
problems with python3
NOTE:
Eclipse Version: 3.6.0
PyDev Interpreter configured for the project: python3
"Preferences/Interpreter - Python"'s Python Interpreters:
python (/usr/bin/python)
python3 (/Library/Frameworks/Python.Framework/Versions/3.1/Resources/Python.app/Contents/MacOS/Python
EDIT:
I write a code to show the os.environ like this:
for python_version in ('python', 'python3'):
try:
import os
for item in os.environ:
print(item, '= ', os.environ[item])
except:
print('problems with ', python_version)
I got the following outputs:
eclipse_output.txt
terminal_output.txt
A FileMerge comparison of the files can be found at eclipse_output.txt vs. terminal_output.pdf.
Any hints?
Thanks
seems like pydev does site-customizations and particularly modifies things for interactive/console usage (judging from a very quick skim of http://github.com/aptana/Pydev/blob/master/plugins/org.python.pydev/pysrc/pydev_sitecustomize/sitecustomize.py ). This is not useful or fitting for execnet-mediated processes.
You could try to "del os.environ['PYTHONPATH']" before you invoke execnet.makegateway, or, to be more careful, just delete the sitecustomize part of it.
hth,
holger
'import site' failed; use -v for traceback
I have seen that when python was unable to find its landmark. Which that indicates there is a PYTHONHOME problem.
Check out http://docs.python.org/using/cmdline.html#envvar-PYTHONHOME maybe eclipse is screwing your environment up.
Edit:
Looked at your env dumps, looks like eclipse is definitely messing with PYTHONPATH, which will cause your child python processes to not work correctly.
Basically what you have going on here is eclipse starts a python v2 instance with a PYTHONPATH pointing to the python v2 directories. Then you spawn a python v3 process which tries to load its landmark from the python v2 directories...
You need to find a way to have eclipse not mess with the PYTHONPATH. I am not sure what eclipse is trying to do by doing that, but it is certainly no friend when you want to spawn new python processes.