environment variable not accessible when start python script by pydev runner - eclipse

My configuration:
ubuntu 16.04 eclipse 4.7 python 2.7.12 PyDev 6.2.0.2017
When running test.py which contains blow code from PyDev:
print os.environ['PATH']
Expected values got printed out,then change 'PATH' to 'LIB_ROOT'
print os.environ['LIB_ROOT']
Got error:
LIB_ROOT=os.environ['LIB_ROOT']
File "/usr/lib/python2.7/UserDict.py", line 40, in __getitem__
raise KeyError(key)
KeyError: 'LIB_ROOT'
And same variable can be accessed from ipython console
and run test.py in bash
python test.py
expected valued got printed out also,can anybody tell me why?

Related

ERROR: gcloud failed to load: No module named 'googlecloudsdk.third_party.apis.binaryauthorization.v1alpha2.binaryauthorization_v1alpha2_messages'

I am trying to install gcloud CLI on macOS 64-bit(x86_64) by following these instructions: https://cloud.google.com/sdk/docs/install
When I run ./google-cloud-sdk/install.sh I get this error:
Welcome to the Google Cloud CLI!
Traceback (most recent call last):
File "/Users/benkula/./google-cloud-sdk/bin/bootstrapping/install.py", line 30, in <module>
from googlecloudsdk import gcloud_main
File "/Users/benkula/google-cloud-sdk/lib/googlecloudsdk/gcloud_main.py", line 37, in <module>
from googlecloudsdk.command_lib.util.apis import yaml_command_translator
File "/Users/benkula/google-cloud-sdk/lib/googlecloudsdk/command_lib/util/apis/yaml_command_translator.py", line 39, in <module>
from googlecloudsdk.command_lib.iam import iam_util
File "/Users/benkula/google-cloud-sdk/lib/googlecloudsdk/command_lib/iam/iam_util.py", line 85, in <module>
binaryauthorization_message_v1alpha2 = core_apis.GetMessagesModule(
File "/Users/benkula/google-cloud-sdk/lib/googlecloudsdk/api_lib/util/apis.py", line 339, in GetMessagesModule
return __import__(api_def.apitools.messages_full_modulepath,
ModuleNotFoundError: No module named 'googlecloudsdk.third_party.apis.binaryauthorization.v1alpha2.binaryauthorization_v1alpha2_messages'
./google-cloud-sdk/bin/gcloud init shows a similar error:
ERROR: gcloud failed to load: No module named 'googlecloudsdk.third_party.apis.binaryauthorization.v1alpha2.binaryauthorization_v1alpha2_messages'
gcloud_main = _import_gcloud_main()
import googlecloudsdk.gcloud_main
from googlecloudsdk.command_lib.util.apis import yaml_command_translator
from googlecloudsdk.command_lib.iam import iam_util
binaryauthorization_message_v1alpha2 = core_apis.GetMessagesModule(
return __import__(api_def.apitools.messages_full_modulepath,
This usually indicates corruption in your gcloud installation or problems with your Python interpreter.
Please verify that the following is the path to a working Python 2.7 or 3.5+ executable:
/Applications/Xcode.app/Contents/Developer/usr/bin/python3
If it is not, please set the CLOUDSDK_PYTHON environment variable to point to a working Python 2.7 or 3.5+ executable.
If you are still experiencing problems, please reinstall the Cloud SDK using the instructions here:
https://cloud.google.com/sdk/
Python executable seems to be where it should be, so I don't think that's the problem. I can see it when I run ls /Applications/Xcode.app/Contents/Developer/usr/bin/.
python3 -V shows Python 3.8.9.
What can I do to install gcloud CLI successfully?
I used this command instead, and it worked for me:
curl https://sdk.cloud.google.com | bash
Found here: https://cloud.google.com/sdk/docs/downloads-interactive.

Cannot run magic functions in ipython terminal

I am using Enthought's Canopy environment on a 64 bit Linux OS. Everything works fine in the Ipython console which is attached with the editor. But when I ipython in the terminal and try to use magic functions, I get the following error.
---------------------------------------------------------------------------
error Traceback (most recent call last)
<ipython-input-3-29a4050aa687> in <module>()
----> 1 get_ipython().show_usage()
/home/shahensha/Development/Canopy/appdata/canopy-1.0.3.1262.rh5-x86_64/lib/python2.7/site-packages/IPython/core/interactiveshell.pyc in show_usage(self)
2931 def show_usage(self):
2932 """Show a usage message"""
-> 2933 page.page(IPython.core.usage.interactive_usage)
2934
2935 def extract_input_lines(self, range_str, raw=False):
/home/shahensha/Development/Canopy/appdata/canopy-1.0.3.1262.rh5-x86_64/lib/python2.7/site-packages/IPython/core/page.pyc in page(strng, start, screen_lines, pager_cmd)
188 if screen_lines <= 0:
189 try:
--> 190 screen_lines += _detect_screen_size(screen_lines_def)
191 except (TypeError, UnsupportedOperation):
192 print(str_toprint, file=io.stdout)
/home/shahensha/Development/Canopy/appdata/canopy-1.0.3.1262.rh5-x86_64/lib/python2.7/site-packages/IPython/core/page.pyc in _detect_screen_size(screen_lines_def)
112 # Proceed with curses initialization
113 try:
--> 114 scr = curses.initscr()
115 except AttributeError:
116 # Curses on Solaris may not be complete, so we can't use it there
/home/shahensha/Development/Canopy/appdata/canopy-1.0.3.1262.rh5-x86_64/lib/python2.7/curses/__init__.pyc in initscr()
31 # instead of calling exit() in error cases.
32 setupterm(term=_os.environ.get("TERM", "unknown"),
---> 33 fd=_sys.__stdout__.fileno())
34 stdscr = _curses.initscr()
35 for key, value in _curses.__dict__.items():
error: setupterm: could not find terminfo database
So, I installed a bare bones iPython shell which is not the one provided by Canopy and tried the same magic functions in there and it works fine.
Have I done something wrong with the installation? Please help
Thanks a lot
shahensha
This is not a solution, but just an observation. My desktop is MacOS-X and I connect to a Centos machine to run Enthought Canopy both 64 bit. I get the same error message as OP if I ssh from iterm2, but not if I use the Terminal app.
I am not sure what the underlying reason is, but may be someone can verify if a similar situation is true for linux. Interestingly I can use either iterm2 or Terminal on the local canopy without any issues.
Update:
I just noticed that the TERM environment variable in iterm2 was set to "xterm" while the Terminal app was showing "xterm-256color". Issuing the command export TERM="xterm-256color" before running the Canopy ipython in terminal solves the issue for me in iterm2.
Problem reproduction:
$ python -c 'import curses; curses.setupterm()'
Traceback (most recent call last):
File "<string>", line 1, in <module>
_curses.error: setupterm: could not find terminfo database
This irc log gave me the idea that this error was to do with libncursesw.
My Canopy version is 1.0.3.1262.rh5-x86_64. I have installed it to ~/src/canopy.
In ~/src/canopy/appdata/canopy-1.0.3.1262.rh5-x86_64/lib we can see that my canopy install has libncursesw.so.5.7.
My machine (Debian Wheezy 64bit) has libncursesw.so.5.9 (in /lib/x86_64-linux-gnu/libncursesw.so.5.9). I made canopy use this. You can toggle the problem on / off by using LD_PRELOAD and pointing at the .so file.
Solution
Replace libncurses.so.5.7 with libncurses.so.5.9:
CANOPYDIR=$HOME/src/canopy
CANOPYLIBS=$CANOPYDIR/appdata/canopy-1.0.3.1262.rh5-x86_64/lib/
SYSTEMLIBS=/lib/x86_64-linux-gnu
cp $SYSTEMLIBS/libncurses.so.5.9 $CANOPYLIBS
ln -sf $CANOPYLIBS/libncurses.so.5.9 $CANOPYLIBS/libncurses.so.5
It appears that Canopy User Python is not your default. See this article:
https://support.enthought.com/entries/23646538-Make-Canopy-s-Python-be-your-default-Python-i-e-on-the-PATH-
Update: Not true here -- instead, see batu's workaround answer.

iptest3 with IPython

I'm brand new to Python programming and trying to get myself a functional base from which I can run things like the IPython Notebook which looks pretty exciting.
Thus far I have both Python 2.7 and 3.3 from python.org installed in OS X 10.6 (Snow Leopard) as well as ActiveTcl 8.5.13. Almost everything that I've tried thus far works as expected. I'm focused on learning 3.3, but want to have the option of using 2.7 too. I read up in several documents that I need to start gaining access to PyPI packages using a Python package manager and that distribute is the one I should use for 3k. So I installed that according to the documentation I found and it seemed to work fine.
I also installed pip as directed, and a number of others.
At this point, I have:
$ pip freeze
distribute==0.6.34
ipython==0.13.1
nose==1.2.1 (installed after IPython)
pexpect==2.4 (installed after IPython)
pyflakes3k==0.4.3
readline==6.2.4.1 (installed after IPython)
At this point, I'm doing this from ipython.org guidance
And when I did $ easy_install pexpect, I got a bunch of errors:
$ easy_install pexpect
Searching for pexpect
Reading http://pypi.python.org/simple/pexpect/
Reading http://pexpect.sourceforge.net/
Reading http://sourceforge.net/project/showfiles.php?group_id=59762
Best match: pexpect 2.4
Downloading http://pypi.python.org/packages/source/p/pexpect/pexpect-2.4.tar.gz#md5=fe82d69be19ec96d3a6650af947d5665
Processing pexpect-2.4.tar.gz
Writing /var/folders/td/td0Sh8EfGFuMCnKex1v+q++++TI/-Tmp-/easy_install-s4dtyy/pexpect-2.4/setup.cfg
Running pexpect-2.4/setup.py -q bdist_egg --dist-dir /var/folders/td/td0Sh8EfGFuMCnKex1v+q++++TI/-Tmp-/easy_install-s4dtyy/pexpect-2.4/egg-dist-tmp-5h5cg4
File "build/bdist.macosx-10.6-intel/egg/fdpexpect.py", line 36
raise ExceptionPexpect, 'The fd argument is not a valid file descriptor.'
^
SyntaxError: invalid syntax
File "build/bdist.macosx-10.6-intel/egg/FSM.py", line 77
return `self.value`
^
SyntaxError: invalid syntax
File "build/bdist.macosx-10.6-intel/egg/pexpect.py", line 82
except ImportError, e:
^
SyntaxError: invalid syntax
zip_safe flag not set; analyzing archive contents...
File "/Library/Frameworks/Python.framework/Versions/3.3/lib/python3.3/site-packages/pexpect-2.4-py3.3.egg/fdpexpect.py", line 36
raise ExceptionPexpect, 'The fd argument is not a valid file descriptor.'
^
SyntaxError: invalid syntax
File "/Library/Frameworks/Python.framework/Versions/3.3/lib/python3.3/site-packages/pexpect-2.4-py3.3.egg/FSM.py", line 77
return `self.value`
^
SyntaxError: invalid syntax
File "/Library/Frameworks/Python.framework/Versions/3.3/lib/python3.3/site-packages/pexpect-2.4-py3.3.egg/pexpect.py", line 82
except ImportError, e:
^
SyntaxError: invalid syntax
Adding pexpect 2.4 to easy-install.pth file
Installed /Library/Frameworks/Python.framework/Versions/3.3/lib/python3.3/site-packages/pexpect-2.4-py3.3.egg
Processing dependencies for pexpect
Finished processing dependencies for pexpect
That looks bad to me (although I don't yet have the expertise to really interpret it), and so I'm not sure if I have a complete install of pexpect.
After installing nose (before pexpect as per URL above), I tried running iptest and iptest3 from the command line, and both failed to find the command, but after I did easy_install ipython again (after nose), I noticed that this install added iptest3 (as well as ipcluster3 and a few other scripts) to my path, and now my bash shell can find iptest3, but when I run it, I get some more bad-looking output:
$ iptest3
Traceback (most recent call last):
File "/Library/Frameworks/Python.framework/Versions/3.3/bin/iptest3", line 9, in <module>
load_entry_point('ipython==0.13.1', 'console_scripts', 'iptest3')()
File "/Library/Frameworks/Python.framework/Versions/3.3/lib/python3.3/site-packages/distribute-0.6.34-py3.3.egg/pkg_resources.py", line 343, in load_entry_point
return get_distribution(dist).load_entry_point(group, name)
File "/Library/Frameworks/Python.framework/Versions/3.3/lib/python3.3/site-packages/distribute-0.6.34-py3.3.egg/pkg_resources.py", line 2308, in load_entry_point
return ep.load()
File "/Library/Frameworks/Python.framework/Versions/3.3/lib/python3.3/site-packages/distribute-0.6.34-py3.3.egg/pkg_resources.py", line 2014, in load
entry = __import__(self.module_name, globals(),globals(), ['__name__'])
File "/Library/Frameworks/Python.framework/Versions/3.3/lib/python3.3/site-packages/IPython/__init__.py", line 43, in <module>
from .config.loader import Config
File "/Library/Frameworks/Python.framework/Versions/3.3/lib/python3.3/site-packages/IPython/config/__init__.py", line 16, in <module>
from .application import *
File "/Library/Frameworks/Python.framework/Versions/3.3/lib/python3.3/site-packages/IPython/config/application.py", line 31, in <module>
from IPython.config.configurable import SingletonConfigurable
File "/Library/Frameworks/Python.framework/Versions/3.3/lib/python3.3/site-packages/IPython/config/configurable.py", line 26, in <module>
from .loader import Config
File "/Library/Frameworks/Python.framework/Versions/3.3/lib/python3.3/site-packages/IPython/config/loader.py", line 27, in <module>
from IPython.utils.path import filefind, get_ipython_dir
File "/Library/Frameworks/Python.framework/Versions/3.3/lib/python3.3/site-packages/IPython/utils/path.py", line 25, in <module>
from IPython.utils.process import system
File "/Library/Frameworks/Python.framework/Versions/3.3/lib/python3.3/site-packages/IPython/utils/process.py", line 27, in <module>
from ._process_posix import _find_cmd, system, getoutput, arg_split
File "/Library/Frameworks/Python.framework/Versions/3.3/lib/python3.3/site-packages/IPython/utils/_process_posix.py", line 22, in <module>
from IPython.external import pexpect
File "/Library/Frameworks/Python.framework/Versions/3.3/lib/python3.3/site-packages/IPython/external/pexpect/__init__.py", line 2, in <module>
import pexpect
File "/Library/Frameworks/Python.framework/Versions/3.3/lib/python3.3/site-packages/pexpect-2.4-py3.3.egg/pexpect.py", line 82
except ImportError, e:
^
SyntaxError: invalid syntax
After doing all that, I noted that my /Library/Frameworks/Python.framework/Versions/Current had been pointing to 2.7, and I guessed that that might be related to my problems and changed the symbolic link to point to 3.3, but iptest3 still fails with the error above.
Any other thoughts on what to do to fix this? It's clear that iptest is pretty important to doing anything else (like IPython Notebook) I want to do.
There is a py3-compatible fork of pexpect called pexpect-u (the u is for unicode, the main difference between the two). You need this to run the pexpect-based parts of IPython on Python 3.
Should be a simple
pip install pexpect-u
side note: pexpect-u is by IPython developer Thomas Kluyver, who did most of the heavy lifting bringing py3 compatibility to IPython.

breaking on unhandled exceptions in pydev/gae

i am using pydev to develop a google app engine application. i followed the steps mentioned here to configure pydev debugger to break on unhandled exception. i could get it to work on a sample pydev project, but when i try the same steps in my pydev gae project, it doesn't work and gives following error:
pydev debugger: warning: psyco not
available for speedups (the debugger
will still work correctly, but a bit
slower) pydev debugger: starting
...
Traceback (most recent call last):
File "c:\program
files\google\google_appengine\google\appengine\tools\dev_appserver.py",
line 3858, in _HandleRequest
self._Dispatch(dispatcher, self.rfile, outfile, env_dict) File
"c:\program
files\google\google_appengine\google\appengine\tools\dev_appserver.py",
line 3792, in _Dispatch
base_env_dict=env_dict) File "c:\program
files\google\google_appengine\google\appengine\tools\dev_appserver.py",
line 580, in Dispatch
base_env_dict=base_env_dict) File "c:\program
files\google\google_appengine\google\appengine\tools\dev_appserver.py",
line 2918, in Dispatch
self._module_dict) File "c:\program
files\google\google_appengine\google\appengine\tools\dev_appserver.py",
line 2822, in ExecuteCGI
reset_modules = exec_script(handler_path, cgi_path,
hook) File "c:\program
files\google\google_appengine\google\appengine\tools\dev_appserver.py",
line 2702, in ExecuteOrImportScript
exec module_code in script_module.dict File
"C:\Users\siddjain\workspace\rfad\src\main.py",
line 1, in
import pydevd ImportError: No module named pydevd
my debug configuration for gae project is like this:
the sample pydev project where it works is like this and am following same pattern in my gae project:
import pydevd
def f(x,y):
z = y/x;
return z;
def main():
pydevd.set_pm_excepthook()
print f(0,0)
if __name__ == '__main__':
main()
the run config for test project is like this:
the pydevd.py module is under C:\eclipse\plugins\org.python.pydev.debug_2.0.0.2011040403\pysrc. Although this path is not included in the pythonpath for test project, the breaking works in test. i also tried including this path in pythonpath of gae project to see if that fixes my problem, but it didn't
Its still not fixed although following steps got rid of No module named pydevd error:
1. Create a symlink to C:\eclipse\plugins\org.python.pydev.debug_2.0.0.2011040403\pysrc:
src>mklink /d debugger C:\eclipse\plugins\org.python.pydev.debug_2.0.0.2011040403\pysrc
2. Put an empty __init__.py file in C:\eclipse\plugins\org.python.pydev.debug_2.0.0.2011040403\pysrc. i learnt this from some google link that i can't find now.
3. In source code:
import debugger.pydevd as pydevd
after these steps its able to import pydevd, but still doesn't break on uncaught exception.
The development server runs your application on your local computer for testing your application. The server simulates the App Engine datastore, services and sandbox restrictions (hence step 1). although i don't understand why we need to import pydevd since its pydevd who's running our application in the first place!
In the Run->Debug Configurations->Interpreter tab if I click on "see resulting command line..."
C:\Python25\python.exe -u
C:\eclipse\plugins\org.python.pydev.debug_2.0.0.2011040403\pysrc\pydevd.py
--vm_type python --client 127.0.0.1 --port 0 --file "c:\program
files\google\google_appengine\dev_appserver.py"
The PYTHONPATH that will be used is:
C:\eclipse\plugins\org.python.pydev_2.0.0.2011040403\PySrc\pydev_si...

Eclipse using multiple Python interpreters with execnet

I'm using the execnet package to allow communication between Python scripts interpreted by different Python interpreters.
The following code (test_execnet.py):
import execnet
for python_version in ('python', 'python3'):
try:
gw = execnet.makegateway("popen//python="+python_version)
ch = gw.remote_exec('channel.send(1/3)')
res = ch.receive()
print(python_version, ': ', res, sep ="")
except:
print('problems with ', python_version)
Runs perfectly in the command-line Terminal, showing the following output:
$ python3 test_execnet.py
python: 0
python3: 0.333333333333
However, if I try to run the same code from within the Eclipse IDE, I get the following error:
'import site' failed; use -v for traceback
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "<string>", line 4, in <module>
File "<string>", line 2, in <module>
File "/Library/Frameworks/Python.framework/Versions/3.1/lib/python3.1/site-packages/execnet/gateway_base.py", line 8, in <module>
import sys, os, weakref
File "/Library/Frameworks/Python.framework/Versions/3.1/lib/python3.1/os.py", line 380, in <module>
from _abcoll import MutableMapping # Can't use collections (bootstrap)
File "/Library/Frameworks/Python.framework/Versions/3.1/lib/python3.1/_abcoll.py", line 54
class Hashable(metaclass=ABCMeta):
^
SyntaxError: invalid syntax
problems with python
problems with python3
NOTE:
Eclipse Version: 3.6.0
PyDev Interpreter configured for the project: python3
"Preferences/Interpreter - Python"'s Python Interpreters:
python (/usr/bin/python)
python3 (/Library/Frameworks/Python.Framework/Versions/3.1/Resources/Python.app/Contents/MacOS/Python
EDIT:
I write a code to show the os.environ like this:
for python_version in ('python', 'python3'):
try:
import os
for item in os.environ:
print(item, '= ', os.environ[item])
except:
print('problems with ', python_version)
I got the following outputs:
eclipse_output.txt
terminal_output.txt
A FileMerge comparison of the files can be found at eclipse_output.txt vs. terminal_output.pdf.
Any hints?
Thanks
seems like pydev does site-customizations and particularly modifies things for interactive/console usage (judging from a very quick skim of http://github.com/aptana/Pydev/blob/master/plugins/org.python.pydev/pysrc/pydev_sitecustomize/sitecustomize.py ). This is not useful or fitting for execnet-mediated processes.
You could try to "del os.environ['PYTHONPATH']" before you invoke execnet.makegateway, or, to be more careful, just delete the sitecustomize part of it.
hth,
holger
'import site' failed; use -v for traceback
I have seen that when python was unable to find its landmark. Which that indicates there is a PYTHONHOME problem.
Check out http://docs.python.org/using/cmdline.html#envvar-PYTHONHOME maybe eclipse is screwing your environment up.
Edit:
Looked at your env dumps, looks like eclipse is definitely messing with PYTHONPATH, which will cause your child python processes to not work correctly.
Basically what you have going on here is eclipse starts a python v2 instance with a PYTHONPATH pointing to the python v2 directories. Then you spawn a python v3 process which tries to load its landmark from the python v2 directories...
You need to find a way to have eclipse not mess with the PYTHONPATH. I am not sure what eclipse is trying to do by doing that, but it is certainly no friend when you want to spawn new python processes.