I've placed package dependencies (wheels) of a Kedro project into a <project-root>/deps/*.whl directory. I'm using a venv installed into <project-root>/.venv and manage it using Poetry.
Packages are referenced in pyproject.toml like this (here e.g. local-package):
[tool.poetry.dependencies]
local-package = {path = "deps/local_package-X.Y.Z-py2.py3-none-any.whl"}
The package can be imported in a REPL session:
$ .venv/bin/python
Python 3.8.2 (default, Apr 27 2020, 15:53:34)
[GCC 9.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import local_package
>>>
In VSCode I've activated my venv and am running `kedro jupyter lab` to start jupyter lab. If I try to import the package it cannot be found (running `import local_package` in a notebook cell leads to `ModuleNotFoundError: No module named "local_package"`). If I run `!poetry show | grep local-package` in a notebook cell I get `local-package X.Y.Z deps/local_package-X.Y.Z-py2.py3-none-any.whl`. The Python interpreter shown in Jupyter Lab is the project one. What do I have to do to get this working?
Related
I know the same question already exists here, but regarding Windows and no vitual-env.
I run ubuntu 20.04 with python3-venv and python 3.8.
In python3 command line I can import psycopg2 and connect to a database.
Only VS code gives me an import error on psycopg2.
python 3.8.10 (default, Jun 2 2021, 10:49:15)
[GCC 9.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import psycopg2
>>>
How can I solve this?
Thanks for your help.
I just found the reason.
I did a
pip3 freeze > requirements.txt
git add *; git commit -m "savepoint"; git push
and commited all to my git repository. After that I deleted the whole project directory.
Then I created a new venv project and cloned the previously saved git repository.
pip3 install -r requirements.txt
did install all necessary python modules including psycopg2.
After that VS code could import psycopg2.
I think this was an issue with wrong dependenties in venv cache.
I'm importing debugpy into my code and it works, but VSCode complains that it cannot be imported:
The interpreter shows that I am in the same venv as my terminal:
In my terminal (in VSCode) I can list debugpy and import it at the Python prompt.
(python4uvm_examples) raysalemi#RayProMac python4uvm_examples % pip list | grep debugpy
debugpy 1.4.1
(python4uvm_examples) raysalemi#RayProMac python4uvm_examples % python
Python 3.8.5 (default, Sep 4 2020, 02:22:02)
[Clang 10.0.0 ] :: Anaconda, Inc. on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import debugpy
>>>
How do I get VSCode to see that it can be imported?
they have this bug for some time now. what I do - select another interpreter, then re-select the one I need. usually works. (using Pylance as language server).
First, I may have more of an issue with correct environment selection. I did a fresh install of Linux Mint 19 and apt dist-upgrade... etc... and then used apt-get and pip to install pymysql, pil.intertk and pylint for both python and python3 from the cli...
All is good from cli:
~$ python3 --version
Python 3.6.5
~$ pylint3 --version
No config file found, using default configuration
pylint3 1.8.3,
astroid 1.6.0
Python 3.6.5 (default, Apr 1 2018, 05:46:30)
[GCC 7.3.0]
python3
Python 3.6.5 (default, Apr 1 2018, 05:46:30)
[GCC 7.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import pymysql
>>> import PIL
>>>
After installing VSC.... the only two selection of python environments are 2.7.12 and 3.5.2 ?
I then modified VSC settings with: "python.pythonPath": "/usr/bin/python3"
Which still shows as Python 3.5.2 64-bit within VSC
But here comes my issue that I have tried many suggestions I have found from searching with no luck...
pylint is not finding any module I install. I assume its due to the mix match of version ? But not sure how to resolve that?
When I try to specify the pylint path : "python.linting.pylintPath": "/usr/bin/pylint3"
I get an invalid path error in VSC.. but its a valid path
~$ which pylint3
/usr/bin/pylint3
Any help would be greatly appreciated....
So after some frustrating trail and error I have fixed the issue by uninstalling the Flatpak version of Visual Studio Code listed in the Linux Mint's Software Center... and installing the deb package from the official site.
After installing the deb package... the proper version of python are showing and pylint is working correctly.
If I 'source activate' a non-root environment (in my case "data"), then launch Jupyter notebook, the env seems to switch to root. I can tell because if I try to open a new python notebook, the dropdown under New says Python [Root]. I am also unable to import packages in my env, but not in root.
(data) Edwards-MacBook-Pro:~ mango$ conda list
# packages in environment at /Users/mango/anaconda/envs/data:
#
boto 2.42.0 py35_0
bz2file 0.98 py35_0
cycler 0.10.0 py35_0
freetype 2.5.5 1
gensim 0.12.4 np111py35_0
libpng 1.6.22 0
matplotlib 1.5.1 np111py35_0
mkl 11.3.3 0
numpy 1.11.1 py35_0
openssl 1.0.2i 0
pandas 0.18.1 np111py35_0
pip 8.1.2 py35_0
pyparsing 2.1.4 py35_0
pyqt 4.11.4 py35_4
python 3.5.2 0
python-dateutil 2.5.3 py35_0
pytz 2016.6.1 py35_0
qt 4.8.7 4
readline 6.2 2
requests 2.11.1 py35_0
scikit-learn 0.17.1 np111py35_2
scipy 0.18.1 np111py35_0
seaborn 0.7.1 py35_0
setuptools 27.2.0 py35_0
sip 4.18 py35_0
six 1.10.0 py35_0
smart_open 1.3.4 py35_0
sqlite 3.13.0 0
tk 8.5.18 0
wheel 0.29.0 py35_0
xz 5.2.2 0
zlib 1.2.8 3
(data) Edwards-MacBook-Pro:~ mango$ ipython
Python 3.5.2 |Anaconda custom (x86_64)| (default, Jul 2 2016, 17:52:12)
Type "copyright", "credits" or "license" for more information.
IPython 4.2.0 -- An enhanced Interactive Python.
? -> Introduction and overview of IPython's features.
%quickref -> Quick reference.
help -> Python's own help system.
object? -> Details about 'object', use 'object??' for extra details.
In [1]: import seaborn
---------------------------------------------------------------------------
ImportError Traceback (most recent call last)
<ipython-input-1-085c0287ecb5> in <module>()
----> 1 import seaborn
ImportError: No module named 'seaborn'
In [2]:
The same behavior occurs with gensim, so it is not just seaborn.
I managed to solve the issue. Conda installs the [root] environment with ipython and jupyter. If you create an env, those are not available by default. So when creating and env, either be sure to list those packages explicitly, or clone root. Cloning root may make a bulkier env and it may be less desirable for production, but better for a sandbox context.
I discovered this issue by trying the same test above with python and finding that my packages indicated that I was in the data env. I then decided to try the Anaconda Navigator program with conda install anaconda-navigator. While I like a cli, this GUI based program seems like a better way to manage packages.
I'm using Eclipse kepler & pydev 3.4.1 on Ubuntu 12.04.
I installed ipython 0.12.1 via pip, which was located in /usr/lib/python2.7/dist-packages/, so I added this path to PYTHONPATH via eclipse->preference->PyDev->Interpreters->Python Interpreter->System PYTHONPATH.
By this way, I would start IPython in the interactive console(just press CTRL_ALT_ENTER), the output is :
Python 2.7.3 (default, Feb 27 2014, 20:00:17) Type "copyright",
"credits" or "license" for more information.
IPython 0.12.1 -- An enhanced Interactive Python. ? ->
Introduction and overview of IPython's features. %quickref -> Quick
reference. help -> Python's own help system. object? -> Details
about 'object', use 'object??' for extra details.
PyDev -- Python IDE for Eclipse For help on using PyDev's Console see
http://pydev.org/manual_adv_interactive_console.html
It looks fine, but when I try to run a simple code in the interactive console, say:
from common_function import *
if __name__ == '__main__':
print("hello")
It shows me an ImportError:
File "/git/mycode/python_lab/pylab/simple_test.py", line 7, in
from common_function import *
ImportError: No module named common_function
Here, the simple_test.py and common_function.py are placed in the same src folder, and I find some posts on stackoverflow saying that adding the src folder which contatins all the py files to project's PYTHONPATH would solve this problem, so I tried and it did solve the ImportError, but the IPython is disable, the output of interactive console is :
/usr/bin/python2.7 2.7.3 (default, Feb 27 2014, 20:00:17) [GCC 4.6.3]
PyDev console: using default backend (IPython not available).
runfile('/media/data/yanglin/local/work/git/mycode/python_lab/pylab/simple_test.py')
hello
Any one know how to fix this problem?