breaking on unhandled exceptions in pydev/gae - pydev

i am using pydev to develop a google app engine application. i followed the steps mentioned here to configure pydev debugger to break on unhandled exception. i could get it to work on a sample pydev project, but when i try the same steps in my pydev gae project, it doesn't work and gives following error:
pydev debugger: warning: psyco not
available for speedups (the debugger
will still work correctly, but a bit
slower) pydev debugger: starting
...
Traceback (most recent call last):
File "c:\program
files\google\google_appengine\google\appengine\tools\dev_appserver.py",
line 3858, in _HandleRequest
self._Dispatch(dispatcher, self.rfile, outfile, env_dict) File
"c:\program
files\google\google_appengine\google\appengine\tools\dev_appserver.py",
line 3792, in _Dispatch
base_env_dict=env_dict) File "c:\program
files\google\google_appengine\google\appengine\tools\dev_appserver.py",
line 580, in Dispatch
base_env_dict=base_env_dict) File "c:\program
files\google\google_appengine\google\appengine\tools\dev_appserver.py",
line 2918, in Dispatch
self._module_dict) File "c:\program
files\google\google_appengine\google\appengine\tools\dev_appserver.py",
line 2822, in ExecuteCGI
reset_modules = exec_script(handler_path, cgi_path,
hook) File "c:\program
files\google\google_appengine\google\appengine\tools\dev_appserver.py",
line 2702, in ExecuteOrImportScript
exec module_code in script_module.dict File
"C:\Users\siddjain\workspace\rfad\src\main.py",
line 1, in
import pydevd ImportError: No module named pydevd
my debug configuration for gae project is like this:
the sample pydev project where it works is like this and am following same pattern in my gae project:
import pydevd
def f(x,y):
z = y/x;
return z;
def main():
pydevd.set_pm_excepthook()
print f(0,0)
if __name__ == '__main__':
main()
the run config for test project is like this:
the pydevd.py module is under C:\eclipse\plugins\org.python.pydev.debug_2.0.0.2011040403\pysrc. Although this path is not included in the pythonpath for test project, the breaking works in test. i also tried including this path in pythonpath of gae project to see if that fixes my problem, but it didn't

Its still not fixed although following steps got rid of No module named pydevd error:
1. Create a symlink to C:\eclipse\plugins\org.python.pydev.debug_2.0.0.2011040403\pysrc:
src>mklink /d debugger C:\eclipse\plugins\org.python.pydev.debug_2.0.0.2011040403\pysrc
2. Put an empty __init__.py file in C:\eclipse\plugins\org.python.pydev.debug_2.0.0.2011040403\pysrc. i learnt this from some google link that i can't find now.
3. In source code:
import debugger.pydevd as pydevd
after these steps its able to import pydevd, but still doesn't break on uncaught exception.
The development server runs your application on your local computer for testing your application. The server simulates the App Engine datastore, services and sandbox restrictions (hence step 1). although i don't understand why we need to import pydevd since its pydevd who's running our application in the first place!
In the Run->Debug Configurations->Interpreter tab if I click on "see resulting command line..."
C:\Python25\python.exe -u
C:\eclipse\plugins\org.python.pydev.debug_2.0.0.2011040403\pysrc\pydevd.py
--vm_type python --client 127.0.0.1 --port 0 --file "c:\program
files\google\google_appengine\dev_appserver.py"
The PYTHONPATH that will be used is:
C:\eclipse\plugins\org.python.pydev_2.0.0.2011040403\PySrc\pydev_si...

Related

VS Code and pytest: Where is the default junit-xml output path defined and why is there one?

I am trying to run a debug the tests of my Python project inside the VS Code interface.
I followed the instruction from VS Code's website using pytest but when trying to run a test the output fails:
============================= test session starts =============================
platform win32 -- Python 3.8.10, pytest-6.2.4, py-1.10.0, pluggy-0.13.1
rootdir: c:\Users\myUserName\Projects\myProjectName
plugins: localserver-0.5.0
collected 1 item
myProjectName\tests\test_static_analysis.py . [100%]
- generated xml file: C:\Users\MYUSERNAME\AppData\Local\Temp\tmp-26696NTXW7ChqfMEN.xml
-
============================== 1 passed in 0.18s ==============================
Error: Error: cannot open file:///c%3A/Users/myUserName/Projects/myProjectName/C. Detail:
Unable to read file 'c:\Users\myUserName\Projects\myProjectName\C' (Error: Unable to resolve
non-existing file 'c:\Users\myUserName\Projects\myProjectName\C')
Error: Error: cannot open file:///c%3A/Users/myUserName/Projects/myProjectName/C. Detail:
Unable to read file 'c:\Users\myUserName\Projects\myProjectName\C' (Error: Unable to resolve
non-existing file 'c:\Users\myUserName\Projects\myProjectName\C')
Maybe VS Code doesn't have the authorization to write in AppData.
What concerns me is why is VS Code launching pytest with a junit-xml output option?
The command actually executed by VS Code is:
c:; cd 'c:\Users\myUserName\Projects\myProjectName'; & 'C:\ProgramData\Anaconda3\envs\venv-myProjectName\python.exe' 'c:\Users\myUserName\.vscode\extensions\ms-python.python-2021.8.1105858891\pythonFiles\lib\python\debugpy\launcher' '53774' '--' 'c:\Users\myUserName\.vscode\extensions\ms-python.python-2021.8.1105858891\pythonFiles\testlauncher.py' 'c:\Users\myUserName\Projects\myProjectName' 'pytest' '--override-ini' 'junit_family=xunit1' '--rootdir' 'c:\Users\myUserName\Projects\myProjectName' '--junit-xml=C:\Users\MYUSERNAME\AppData\Local\Temp\tmp-26696pYh1w3pF2cXx.xml' './myProjectName/tests/test_static_analysis.py::TestStaticAnalysisVesselForceOnLateralCenter::test_pos_surge_on_vessel'
When going in Settings > python.testing.pytestArgs it is empty.
Where is this output path defined?
How can I change it to be in the local working directory?
Do I need to have a junit-xml output? Is it mandatory for VS Code UI to work?

How Can I Solve "Failed To Source Bitbake" With Xilinx Petalinux SDK

I am running Ubuntu 16.04 with Xilinx Petalinux 2018.03 SDK. After a number of successful compilations I am now facing this error
$ petalinux-build
[INFO] building project
[INFO] sourcing bitbake
ERROR: Failed to source bitbake
ERROR: Failed to build project
How can I solved this issue?
Another reason to get the errors "ERROR: Failed to source bitbake" as well as "ERROR: Failed to build project" is a possible upgrade of Python on the build machine. The Petalinux SDK requires python v2 (>= 2.7.3) for the 2018.3 edition.
You can check under [YOUR_PROJECT]/build/build.log and you might see a log similar to this one below:
[INFO] building project
[INFO] sourcing bitbake
SDK environment now set up; additionally you may now run devtool to perform development tasks.
Run devtool --help for further details.
OpenEmbedded requires 'python' to be python v2 (>= 2.7.3), not python v3.
Please set up python v2 as your default 'python' interpreter.
ERROR: Failed to source bitbake
ERROR: Failed to build project
To remedy this issue remove the symbolic link under /usr/bin and make sure to create a new one that is pointing to Python 2.7:
sudo rm /usr/bin/python
sudo ln -s /usr/bin/python2.7 /usr/bin/python
First you need to investigate the error a little further, do this:
source /opt/pkg/petalinux/2018.3/settings.sh
It will return something similar to this below:
PetaLinux environment set to '/opt/pkg/petalinux/2018.3'
INFO: Checking free disk space
INFO: Checking installed tools
INFO: Checking installed development libraries
INFO: Checking network and other services
Source the environment setup:
source /opt/pkg/petalinux/2018.3/components/yocto/source/aarch64/environment-setup-aarch64-xilinx-linux
followed by:
devtool --help
In my case I can see more about the actual error:
NOTE: Starting bitbake server...
ERROR: Unable to start bitbake server
ERROR: Last 10 lines of server log for this session (/opt/pkg/petalinux/2018.3/components/yocto/source/aarch64/bitbake-cookerdaemon.log):
Traceback (most recent call last):
File "/opt/pkg/petalinux/2018.3/components/yocto/source/aarch64/layers/core/bitbake/lib/bb/daemonize.py", line 77, in createDaemon
function()
File "/opt/pkg/petalinux/2018.3/components/yocto/source/aarch64/layers/core/bitbake/lib/bb/server/process.py", line 433, in _startServer
self.cooker = bb.cooker.BBCooker(self.configuration, self.featureset)
File "/opt/pkg/petalinux/2018.3/components/yocto/source/aarch64/layers/core/bitbake/lib/bb/cooker.py", line 178, in __init__
self.configwatcher = pyinotify.WatchManager()
File "/opt/pkg/petalinux/2018.3/components/yocto/source/aarch64/layers/core/bitbake/lib/pyinotify.py", line 1764, in __init__
raise OSError(err % self._inotify_wrapper.str_errno())
OSError: Cannot initialize new instance of inotify, Errno=Too many open files (EMFILE)
ERROR: Unable to start bitbake server
ERROR: Last 10 lines of server log for this session (/opt/pkg/petalinux/2018.3/components/yocto/source/aarch64/bitbake-cookerdaemon.log):
Traceback (most recent call last):
File "/opt/pkg/petalinux/2018.3/components/yocto/source/aarch64/layers/core/bitbake/lib/bb/daemonize.py", line 77, in createDaemon
function()
File "/opt/pkg/petalinux/2018.3/components/yocto/source/aarch64/layers/core/bitbake/lib/bb/server/process.py", line 433, in _startServer
self.cooker = bb.cooker.BBCooker(self.configuration, self.featureset)
File "/opt/pkg/petalinux/2018.3/components/yocto/source/aarch64/layers/core/bitbake/lib/bb/cooker.py", line 178, in __init__
self.configwatcher = pyinotify.WatchManager()
File "/opt/pkg/petalinux/2018.3/components/yocto/source/aarch64/layers/core/bitbake/lib/pyinotify.py", line 1764, in __init__
raise OSError(err % self._inotify_wrapper.str_errno())
OSError: Cannot initialize new instance of inotify, **Errno=Too many open files (EMFILE)**
This is pointing to the /proc/sys/fs/inotify/max_user_instances that need to be increased. In my case I went from 128 to 256 by doing this:
sudo su
echo 256 > /proc/sys/fs/inotify/max_user_instances
You need to become root with "su" and change the mac_user_instances.

How to solve error on docker:layers_calculator to compute the Merkle tree on private tangle?

I want to setup a private tangle on my own virtual machine with Ubuntu 18.04, 4GB RAM and 20GB memory.
I have follow this instructions: https://docs.iota.org/docs/compass/0.1/how-to-guides/set-up-a-private-tangle. Every command works fine until reach this one: bazel run //docker:layers_calculator.
It shows an error as follows:
Starting local Bazel server and connecting to it...
ERROR: /home/istabraq/compass/third-party/maven_deps.bzl:3:5: Traceback (most recent call last):
File "/home/istabraq/compass/WORKSPACE", line 42
maven_jars()
File "/home/istabraq/compass/third-party/maven_deps.bzl", line 3, in maven_jars
native.maven_jar(<4 more arguments>)
type 'struct' has no method maven_jar()
ERROR: error loading package '': Encountered error while reading extension file 'protobuf_deps.bzl': no such package '#com_google_protobuf_deps//': error loading package 'external': Could not load //external package
ERROR: error loading package '': Encountered error while reading extension file 'protobuf_deps.bzl': no such package '#com_google_protobuf_deps//': error loading package 'external': Could not load //external package
INFO: Elapsed time: 4.743s
INFO: 0 processes.
FAILED: Build did NOT complete successfully (0 packages loaded)
FAILED: Build did NOT complete successfully (0 packages loaded)
How can I solve this problem? what I have missed?
read carefully the message given after running bazel installer:
Make sure you have "/home/yourusername/bin" in your path. You can also activate bash completion by adding the following line to your :
source /home/yourusername/.bazel/bin/bazel-complete.bash
You can check with: "bazel info" or "bazel version"
Unfortunately, there are further errors:
https://github.com/iotaledger/compass/issues/142
I have solve this issue by using this commands:
Step 3: Set up your environment
If you ran the Bazel installer with the --user flag as above, the Bazel executable is installed in your $HOME/bin directory. It’s a good idea to add this directory to your default paths, as follows:
export PATH="$PATH:$HOME/bin"
You can also add this command to your ~/.bashrc or ~/.zshrc file to make it permanent.
reference:
https://docs.bazel.build/versions/master/install-ubuntu.html

Pytest "Error: could not load path/to/conftest.py"

I get the following error when I try to run pytest repo/tests/test_file.py:
$ pytest repo/tests/test_file.py
Traceback (most recent call last):
File "/Users/marlo/anaconda3/envs/venv/lib/python3.6/site-packages/_pytest/config.py", line 329, in _getconftestmodules
return self._path2confmods[path]
KeyError: local('/Users/marlo/repo/tests/test_file.py')
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/Users/marlo/anaconda3/envs/venv/lib/python3.6/site-packages/_pytest/config.py", line 329, in _getconftestmodules
return self._path2confmods[path]
KeyError: local('/Users/marlo/repo/tests')
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/Users/marlo/anaconda3/envs/venv/lib/python3.6/site-packages/_pytest/config.py", line 362, in _importconftest
return self._conftestpath2mod[conftestpath]
KeyError: local('/Users/marlo/repo/conftest.py')
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/Users/marlo/anaconda3/envs/venv/lib/python3.6/site-packages/_pytest/config.py", line 368, in _importconftest
mod = conftestpath.pyimport()
File "/Users/marlo/anaconda3/envs/venv/lib/python3.6/site-packages/py/_path/local.py", line 686, in pyimport
raise self.ImportMismatchError(modname, modfile, self)
py._path.local.LocalPath.ImportMismatchError: ('conftest', '/home/venvuser/venv/conftest.py', local('/Users/marlo/repo/conftest.py'))
ERROR: could not load /Users/marlo/repo/conftest.py
My repo structure is
lib/
-tests/
-test_file.py
app/
-test_settings.py
pytest.ini
conftest.py
...
Other people have run this code fine, and according to this question (and this one), my structure is good and I am not missing any files. I can only conclude that something about my computer or project set-up is not right. If you have any suggestions or insights that I may be missing, please send them my way!
-------------------------------MORE DETAILS------------------------------
test_file.py:
def func(x):
return x + 1
def test_answer():
assert func(3) == 5
pytest.ini:
[pytest]
DJANGO_SETTINGS_MODULE = app.test_settings
python_files = tests.py test_* *_tests.py *test.py
I have docker as well as running pytest outside of docker too, and for me a much lower-impact fix whenever this crops up is to delete all the compiled python files
find . -name \*.pyc -delete
I figured it out and I'll answer in case others have the same issue:
I didn't even take into consideration that I had a docker container (of the same app) in the repo directory and, although I was not running the docker container, it was influencing the filepaths somehow.
To fix this:
I re-cloned the repo from the remote source into a new folder so that nothing from the old repo could "contaminate" it.
Updated my virtual environment with the .yml specifications of the clean repo
$ conda env update --name project --file project.yml
My project uses a postgres database, so I dropped it and created a new one
$ dropdb projectdb
$ createdb projectdb
Since my project uses mongo, I also dropped that database
$ mongo projectdb --eval "db.dropDatabase()"
Installed a clean pytest
$ pip uninstall pytest
$ pip install pytest
...and voilà! I could run pytest.
Many thanks to #hoefling and others who helped me debug.
I was running docker as well, but it seems my problem was different.
I was using an old version of pytest:
platform linux -- Python 3.9.7, pytest-3.7.2, py-1.10.0, pluggy-1.0.0
which stopped working after my ubuntu image was pulling python 3.10 by default.
my solution was to update (and fix) the dockerfile image to use:
FROM python:3.10
instead of python:latest, and update the pytest version as well.

Eclipse using multiple Python interpreters with execnet

I'm using the execnet package to allow communication between Python scripts interpreted by different Python interpreters.
The following code (test_execnet.py):
import execnet
for python_version in ('python', 'python3'):
try:
gw = execnet.makegateway("popen//python="+python_version)
ch = gw.remote_exec('channel.send(1/3)')
res = ch.receive()
print(python_version, ': ', res, sep ="")
except:
print('problems with ', python_version)
Runs perfectly in the command-line Terminal, showing the following output:
$ python3 test_execnet.py
python: 0
python3: 0.333333333333
However, if I try to run the same code from within the Eclipse IDE, I get the following error:
'import site' failed; use -v for traceback
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "<string>", line 4, in <module>
File "<string>", line 2, in <module>
File "/Library/Frameworks/Python.framework/Versions/3.1/lib/python3.1/site-packages/execnet/gateway_base.py", line 8, in <module>
import sys, os, weakref
File "/Library/Frameworks/Python.framework/Versions/3.1/lib/python3.1/os.py", line 380, in <module>
from _abcoll import MutableMapping # Can't use collections (bootstrap)
File "/Library/Frameworks/Python.framework/Versions/3.1/lib/python3.1/_abcoll.py", line 54
class Hashable(metaclass=ABCMeta):
^
SyntaxError: invalid syntax
problems with python
problems with python3
NOTE:
Eclipse Version: 3.6.0
PyDev Interpreter configured for the project: python3
"Preferences/Interpreter - Python"'s Python Interpreters:
python (/usr/bin/python)
python3 (/Library/Frameworks/Python.Framework/Versions/3.1/Resources/Python.app/Contents/MacOS/Python
EDIT:
I write a code to show the os.environ like this:
for python_version in ('python', 'python3'):
try:
import os
for item in os.environ:
print(item, '= ', os.environ[item])
except:
print('problems with ', python_version)
I got the following outputs:
eclipse_output.txt
terminal_output.txt
A FileMerge comparison of the files can be found at eclipse_output.txt vs. terminal_output.pdf.
Any hints?
Thanks
seems like pydev does site-customizations and particularly modifies things for interactive/console usage (judging from a very quick skim of http://github.com/aptana/Pydev/blob/master/plugins/org.python.pydev/pysrc/pydev_sitecustomize/sitecustomize.py ). This is not useful or fitting for execnet-mediated processes.
You could try to "del os.environ['PYTHONPATH']" before you invoke execnet.makegateway, or, to be more careful, just delete the sitecustomize part of it.
hth,
holger
'import site' failed; use -v for traceback
I have seen that when python was unable to find its landmark. Which that indicates there is a PYTHONHOME problem.
Check out http://docs.python.org/using/cmdline.html#envvar-PYTHONHOME maybe eclipse is screwing your environment up.
Edit:
Looked at your env dumps, looks like eclipse is definitely messing with PYTHONPATH, which will cause your child python processes to not work correctly.
Basically what you have going on here is eclipse starts a python v2 instance with a PYTHONPATH pointing to the python v2 directories. Then you spawn a python v3 process which tries to load its landmark from the python v2 directories...
You need to find a way to have eclipse not mess with the PYTHONPATH. I am not sure what eclipse is trying to do by doing that, but it is certainly no friend when you want to spawn new python processes.