I have used the following scripts in an iPython notebook to enable autoreload.
import autoreload
%autoreload
The import works find but the magic function throws the following error:
ERROR: Line magic function `%autoreload` not found.
Here are the current version settings as per the notebook 'about' section:
Python 2.7.3 (default, Jun 22 2015, 19:33:41)
[GCC 4.6.3]
IPython 4.1.2 -- An enhanced Interactive Python.
Any pointers on what the issue could be would be appreciated.
Thanks
If you are using this in IPython notebook
This command will load autoreload:
%load_ext autoreload
This line will reload pandas (replace pandas with the module name you would like to reload) every time code is run:
%aimport pandas
If you wish to remove pandas from reloading every time you run code then,
this line will stop pandas from reloading:
%aimport -pandas
Include %autoreload at the top the cell for the magic to happen!
autoreload Reference
Related
I am trying to compile a Python package that I recently migrated from Python2 to Python3.
When running the source code in Python3 it works as expected, so do the source and compiled versions for Python2, but when I cythonize and compile the Python3 package the resulting binaries throw this error when importing a certain module:
dynamic module does not define module export function (PyInit_NKPD)
EDIT:
When I then close the Python interpreter, open it again and import the same model it works.
Interestingly when I put the code on my local drive it imports fine, but when I put it on my server and import it from there I get the above error. And to make matters worse, sometimes it imports fine from the server as well if I just wait a while and try again:
lockjaw:controller frank$ python3.7
Python 3.7.7 (v3.7.7:d7c567b08f, Mar 10 2020, 02:56:16)
[Clang 6.0 (clang-600.0.57)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import common
added to sys.path: /opt/ohufx/transfer/NUBRIDGE_COMPILE/osx/NKPD/src
trying to import model.NukepediaDB...
NukepediaDB imported
trying to import model.NKPD...
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "src/controller/common.py", line 15, in init controller.common
import model.NKPD # this does not import
ImportError: dynamic module does not define module export function (PyInit_NKPD)
>>> import common
added to sys.path: /opt/ohufx/transfer/NUBRIDGE_COMPILE/osx/NKPD/src
trying to import model.NukepediaDB...
trying to import model.NKPD...
NKPD imported
model.NKPD always seems to import fine, but model.NKPD seems to be the stumbling block, even though this may be a red herring. It almost behaves like sometimes the server connection drops out after the first import, though in that case the other incarnations (Python 2/source code) would throw errors as well (not to mention a whole lot of other things would break in my office).
I have reduced the package to three modules with nothing but import and print statements which can be found here (including the resuling C++ and binary files).
The structure is very simple:
Where common is the module I import to test by cd-ing into the src/controller folder, opening Python3.7 and importing common. Common then imports model.NukepediaDB and model.NKPD, the latter causing the trouble.
I cythonized the python files with this line:
/Library/Frameworks/Python.framework/Versions/3.7/bin/python3.7 py2cpp.py
Then compiled with this:
/Library/Frameworks/Python.framework/Versions/3.7/bin/python3.7 setup.py build_ext --inplace
I would love to understand why the import sometimes fails when run from the server, as I have to distribute this package to others and need to ensure it runs reliably.
Thanks,
Frank
I am trying to run my first PySpark script on a Linux VM I configured. The error message I have is KeyError: SPARK_HOME when I run the following:
from os import environ
from pyspark import SparkContext
I momentarily made this error go away by running export SPARK_HOME=~/spark-2.4.3-bin-hadoop2.7. I then ran into a new error error=2, No such file or directory. Searching took me to this page:https://community.cloudera.com/t5/Community-Articles/Tutorial-Install-Configure-iPython-and-create-run-PySpark/ta-p/246400. I then ran export PYSPARK_PYTHON=~/python3*. This brings me back to experiencing the KeyError: SPARK_HOME error.
Honestly, I'm stumbling through this, because it's my first time configuring Spark, and using PySpark. I still don't quite understand the ins-and-outs of pycharm, as well.
I expect to be able to run the following basic sample script on this page: https://medium.com/parrot-prediction/integrating-apache-spark-2-0-with-pycharm-ce-522a6784886f with no issues.
there is a package called findspark here
or you may use below code to set path if not found in environment
import os
if 'SPARK_HOME' not in os.environ:
os.environ['SPARK_HOME'] = 'full_path_to_spark_root'
[code continues]
First of all I'm aware of the question here but I couldn't find a satisfied answer there. I don't want to ignore errors or use comments - I want to have the right settings in eclipse/pydev. My problem is pretty similar to this one.
I'm using Ubuntu 12.04 and installed a virtuenv for python 2.7 in my home directory. After installing several python packages (numpy, scipy, matplotlib, etc.) using pip, I installed eclipse 4.3 with pydev.
If I use the python system interpreter at /usr/bin/python everything works fine (except that I didn't want to use). However, if I try to set up a python interpreter using the virtualenv first I get this warning describe here. After clicking "proceed anyway", it seems to work. So far so good.
However e.g. import numpy as np gives for each np.* call the eclipse/pydev error Undefined variable from import, also the code completion doesn't work properly. It seems to work, e.g. for datetime, but not for numpy, scipy and matplotlib.
Does anybody figured out to configure eclipse correctly?
I already tried to add the numpy path manually to the virtualenv interpreter, but than I get the weird error:
import matplotlib.dates as mpl_dates
File "/home/pydev/myenv-py27/local/lib/python2.7/site-packages/matplotlib/init.py", line 149, in
import sys, os, tempfile
File "/usr/lib/python2.7/tempfile.py", line 34, in
from random import Random as _Random
ImportError: cannot import name Random
I'm starting using Jython. I noted that sys.path is completely different when Jython is executed from the command line than from Eclipse.
Command line
tk:~$ jython
Jython 2.5.3 (2.5:c56500f08d34+, Aug 13 2012, 14:48:36)
[Java HotSpot(TM) Server VM (Oracle Corporation)] on java1.7.0_10
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> print sys.path
['',
'/jproject/extras/2.5.3/Lib',
'__classpath__',
'__pyclasspath__/',
'/jproject/extras/2.5.3/Lib/site-packages']
It's OK, I can import everything from there (import pdb, import csv, etc).
Eclipse
I added jython.jar in Eclipse to the Java Build Path using the "Add JARs" button. I wrote a simple Java class (excerpt):
PythonInterpreter pi = new PythonInterpreter();
pi.exec("import sys");
pi.exec("print sys.path");
Output:
['/jproject/projects/foobar/.metadata/.plugins/org.eclipse.wst.server.core/tmp0/wtpwebapps/MyProject/WEB-INF/lib/Lib',
'__classpath__',
'__pyclasspath__/']
From Python scripts created in the Java project I can't import anything. I always get ImportError: No module named ...
So I created a file called .jython in my home directory containing this:
python.path=/jproject/extras/2.5.3/Lib:/jproject/extras/2.5.3/Lib/site-packages
It seems to fix the import issue. But, is this the right way to proceed?
I think I don't have to do the last step manually and probably I installed Jython badly.
Looks like I have same issue described in my question here:
The solution I've come up is workaround that doesn't require any actions from user to set python.path: Basically I added code that sets python.path to application work directory (user.path) before initializing jython environment.
I recently upgraded my Enthought Python distribution with
sudo easy_install -U ipython
However, when I start up ipython with the ipython command, I get an error:
David-Faux-MacBook-Air:core davidfaux$ ipython
Python 2.7.2 |EPD 7.2-2 (32-bit)| (default, Sep 7 2011, 09:16:50)
Type "copyright", "credits" or "license" for more information.
IPython 0.13.1 -- An enhanced Interactive Python.
? -> Introduction and overview of IPython's features.
%quickref -> Quick reference.
help -> Python's own help system.
object? -> Details about 'object', use 'object??' for extra details.
[TerminalIPythonApp] Error in loading extension: kernmagic
Check your config files in /Users/davidfaux/.ipython/profile_default
---------------------------------------------------------------------------
AttributeError Traceback (most recent call last)
/Library/Frameworks/Python.framework/Versions/7.2/lib/python2.7/site-packages/ipython-0.13.1-py2.7.egg/IPython/core/extensions.pyc in load_extension(self, module_str)
90 __import__(module_str)
91 mod = sys.modules[module_str]
---> 92 return self._call_load_ipython_extension(mod)
93
94 def unload_extension(self, module_str):
/Library/Frameworks/Python.framework/Versions/7.2/lib/python2.7/site-packages/ipython-0.13.1-py2.7.egg/IPython/core/extensions.pyc in _call_load_ipython_extension(self, mod)
122 def _call_load_ipython_extension(self, mod):
123 if hasattr(mod, 'load_ipython_extension'):
--> 124 return mod.load_ipython_extension(self.shell)
125
126 def _call_unload_ipython_extension(self, mod):
/Library/Frameworks/Python.framework/Versions/7.2/lib/python2.7/site-packages/kernmagic/__init__.py in load_ipython_extension(ip)
28
29 def load_ipython_extension(ip):
---> 30 activate(ip)
31 activate_aliases(ip)
/Library/Frameworks/Python.framework/Versions/7.2/lib/python2.7/site-packages/kernmagic/__init__.py in activate(ip, *args)
14 continue
15 magic_name = name[len('magic_'):]
---> 16 ip.shell.define_magic(magic_name, getattr(mymagics, name))
17
18 def activate_aliases(ip, *args):
AttributeError: 'TerminalInteractiveShell' object has no attribute 'shell'
I googled around, and many posts I found said to comment out this line in an ipython configuration file.
c.TerminalIPythonApp.extensions = ['kernmagic']
I find it sketchy to comment out an ipython file though... later, I found that kernmagic fixed the issue with ipython.
How do I upgrade kernmagic in my ipython distribution so that this error goes away?
This worked for me:
hg clone https://bitbucket.org/robertkern/kernmagic
cd kernmagic
python setup.py build
python -c "import setuptools;execfile('setup.py')" bdist_egg
egginst dist/kernmagic-0.0.0-py2.7.egg
You can comment/remove the line in your configuration file, there is no issues with that if you do not use kernmagic. Actually the .ipython/profile_xxx folder are made for people to change things.
Otherwise if you really want to upgrade kern magic, I suggest you uninstall, remove the files and reinstall it from the current source.
Also IIRC the kern magic issue does not come with raw IPython only with EPD, so you shouldn't be afraid of removing it from config files and system.
For those who are not running EPD's distribution of python (or if Rich's solution didn't work):
First of all, I'm running Python 2.7.5 |Anaconda 1.6.0 (x86_64)|, on Mac OS X 10.8.4.
As Matt has stated, it is easiest to simply comment out the line that looks like this:
# A list of dotted module names of IPython extensions to load.
c.TerminalIPythonApp.extensions = ['kernmagic']
I found this in:
~/.ipython/profile_default/ipython_config.py
It should be located around 1/3 of the way down. It's worth pointing out that this is a temporary solution - I have no idea what kernmagic does or if it's an important ipython extension.
Enthought has updated kernmagic, so Enthought's enpkg fixed the same problem for me on EPD 7.3 and ipython 0.13.1:
sudo enpkg kernmagic
I've just created a quick-and-dirty port of kernmagic to python3. It may also work when this problem arises under python-2.x.
Here is a link to the code. All you need to do is drop the entire kernlab folder into your site-packages directory. It seems to be a pure python library, so it ought to work on any platform.
Good luck!
I also faced this problem, I'm on windows environment, I found that line in
C:\Users\Li\.ipython\profile_default\ipython_config.py
As stated, it is easiest to simply comment out the line that looks like this:
# A list of dotted module names of IPython extensions to load.
c.TerminalIPythonApp.extensions = ['kernmagic']