VSCode settings for search paths with PEP582 & ipykernel - visual-studio-code

I used python package managers (PyFlow/PDM) which supports PEP582: packages are installed in the local directory {workspace_path}/__pypackages__/.
I added some configs in settings.json to let VSCode know how to find the packages:
{
...,
"python.autoComplete.extraPaths": [
"__pypackages__/3.7lib"
],
"python.analysis.extraPaths": [
"__pypackages__/3.7/lib"
]
}
I wrote some code, and the completion and inline code docs works well with above settings.
# main.py
#%% jupyter cell
import sys
print(sys.path)
import pandas as pd
print(pd.__version__)
#%%
print("hello world")
I ran my main.py in both ways:
Run the file with VSCode (ctrl + opt + N)
Run the jupyter cell with ipykernel (shift + enter). VSCode asked me to install ipykernel at the first time, but it installed with pip, and not installed into __pypackages__.
but it failed and showed: ModuleNotFoundError: No module named 'pandas'.
I check the search path with sys.path, it doesn't contain __pypackages__ for python modules:
['{$HOME}/{workspace}',
'{$HOME}/.vscode/extensions/ms-toolsai.jupyter-2021.5.745244803/pythonFiles',
'{$HOME}/.vscode/extensions/ms-toolsai.jupyter-2021.5.745244803/pythonFiles/lib/python',
'{$HOME}/.pyenv/versions/3.7.10/lib/python37.zip',
'{$HOME}/.pyenv/versions/3.7.10/lib/python3.7',
'{$HOME}/.pyenv/versions/3.7.10/lib/python3.7/lib-dynload',
'',
'{$HOME}/{workspace}/.venv/lib/python3.7/site-packages',
'{$HOME}/{workspace}/.venv/lib/python3.7/site-packages/IPython/extensions',
'{$HOME}/.ipython']
My questions:
How to configure settings.json to add search paths (__pypackages__/3.7lib) for Python modules.
How to configure settings.json to let VSCode Jupyter server launch IPython with ipykernel installed in __pypackages__/3.7lib

"python.autoComplete.extraPaths": [
"${workspaceFolder}/__pypackages__/<major.minor>/lib"
],
"python.analysis.extraPaths": [
"${workspaceFolder}/__pypackages__/<major.minor>/lib"
],
Try this in settings.json
Replace <major.minor> with your python version like 3.10, 3.9, etc

I found two ways to solve the sys.path problem.
The easiest one would be to launch VSCode with the following command:
pdm run -p <path-to-profile> code <path-to-folder-or-workspace-file>
This ensures that PYTHONPATH contains __packages__\... directories for VSCode to use.

Related

Why Does the VS Code Jupyter Extension Keep Timing-out Trying to Find a Kernel That Exists?

I need to set up virtual environments for each language that I use. To do this, I'm running the Ubuntu 20.04 LTS Windows Subsystem for Linux (WSL) on Windows 10. Within WSL, I'm using Anaconda, installed in /usr/local/Anaconda, to create conda virtual environments for each language (i.e. one environment contains all my Python stuff, another contains my R stuff, etc.).
Since WSL doesn't come with a GUI, I'm using Visual Studio Code's (VSCode) Jupyter Notebook Extension to run Jupyter Notebooks to see plots/graphics. So far, I managed to easily create conda environments for Python (with ipython and ipykernel) and R (with IRkernel) and run their code in a notebook via the extension. Each time I set up an environment, the extension is easily able to find the kernel, connect to it and run the code.
However, I've not been able to set up an environment for Julia. I followed the documentation on the Julia website for installing the kernel, which is successfully found by the extension. But, when I try running a cell, the extension says it is trying to connect to the kernel, only for it to timeout and fail.
Here are the steps I have taken so far:
Create a clean conda environment (conda create -n Julia && conda activate Julia)
Install the latest version of Julia (conda install -c conda-forge julia)
Install the latest version of Jupyter (conda install -c conda-forge jupyter)
Install the Julia kernel with the built-in Julia package manager (using Pkg; Pkg.add("IJulia"))
Build the IJulia package (using Pkg; Pkg.build("IJulia"))
Confirm the presence of the Julia kernel (jupyter kernelspec list) which indeed shows the presence of a Julia kernel
Reload the VSCode connection to WSL (Ctrl + Shift + P; >Reload Window)
Shut down WSL via CMD (wsl --shutdown) for changes to take effect and reconnect
After I restart VSCode and WSL, the extension shows an option to use the Julia kernel installed in my conda environment: Julia 1.7.2 (~/.conda/envs/Julia/bin/julia). But when I create a cell and run code in a notebook, the extension creates a popup saying that it is connecting to the kernel and after some time an error message shows up:
Failed to start the Kernel.
Unable to start Kernel `Julia 1.7.2` due to connection timeout.
View Jupyter log for further details
I can also see the kernel spec JSON file in ~/.local/share/jupyter/kernels/julia-1.7/kernel.json
{
"display_name": "Julia 1.7.2",
"argv": [
"/home/USER/.conda/envs/Julia/bin/julia",
"-i",
"--color=yes",
"--project=#.",
"/home/USER/.conda/envs/Julia/share/julia/packages/IJulia/AQu2H/src/kernel.jl",
"{connection_file}"
],
"language": "julia",
"env": {},
"interrupt_mode": "signal"
}
The log file starts showing problems here:
info 17:50:48.378: Process Execution: cwd: ~
cwd: ~
warn 17:50:48.893: StdErr from Kernel Process [91m[1mERROR: [22m[39m
warn 17:50:49.138: StdErr from Kernel Process LoadError:
warn 17:50:49.795: StdErr from Kernel Process ArgumentError: Package IJulia not found in current path:
- Run `import Pkg; Pkg.add("IJulia")` to install the IJulia package.
The extension says it cannot find the IJulia kernel. This perplexes me because I can see the kernel spec in my home directory, the jupyter binary I installed from conda says that its there and the Jupyter Notebook extension can see the kernel. I have no explanation as to why the extension can see the kernel, match up the kernelspec but not be able to connect to it. Help would greatly be appreciated!

Setup vscode MyPy for project with both Python2 and Python3

I have a vscode linux based project that is mainly Python2.7. The project started well before Python3 had momentum.
In vscode settings I have to following to specify the code is for python2.
"python.linting.mypyArgs": [ "--py2" ],
Now, I have to add in some new code/modules/packages that are written in Python3.
Can I configure vscode mypy to use Python3 linting for some files/directories and Python2 linting for other files/directories ? If so, how?

Conda virtual environment for IPython

I'm fairly new to the Python scene. My problem is that when I launch a jupyter notebook from an Anaconda Powershell with my DataScience virtual environment activated, the notebook does not have my virtual environment in it's PATH, and therefore cannot find some packages (like plotly and progress). The same is true when I launch VS Code from Anaconda Navigator with DataScience activated. When I run import plotly in an interactive window, I get ModuleNotFoundError: No module named ‘plotly’. But when I run this line in the terminal within VS Code, it runs without error.
So I have run the following commands in various shell/terminal sessions:
import sys
print(sys.path)
In a VS Code terminal I get:
['', 'C:\\Users\\adiad\\Anaconda3\\envs\\DataScience\\python37.zip', 'C:\\Users\\adiad\\Anaconda3\\envs\\DataScience\\DLLs', 'C:\\Users\\adiad\\Anaconda3\\envs\\DataScience\\lib', 'C:\\Users\\adiad\\Anaconda3\\envs\\DataScience', 'C:\\Users\\adiad\\Anaconda3\\envs\\DataScience\\lib\\site-packages']
In an interactive window in VS Code I get:
['C:\\Users\\adiad\\AppData\\Local\\Temp\\04e2b30c-4fc3-4aa9-9567-3aba17081a73', 'C:\\Users\\adiad\\Anaconda3\\python37.zip', 'C:\\Users\\adiad\\Anaconda3\\DLLs', 'C:\\Users\\adiad\\Anaconda3\\lib', 'C:\\Users\\adiad\\Anaconda3', '', 'C:\\Users\\adiad\\Anaconda3\\lib\\site-packages', 'C:\\Users\\adiad\\Anaconda3\\lib\\site-packages\\win32', 'C:\\Users\\adiad\\Anaconda3\\lib\\site-packages\\win32\\lib', 'C:\\Users\\adiad\\Anaconda3\\lib\\site-packages\\Pythonwin', 'C:\\Users\\adiad\\Anaconda3\\lib\\site-packages\\IPython\\extensions', 'C:\\Users\\adiad\\.ipython']
In a jupyter notebook running in my browser I get:
['C:\\Users\\adiad\\Anaconda3\\envs\\test', 'C:\\Users\\adiad\\Anaconda3\\python37.zip', 'C:\\Users\\adiad\\Anaconda3\\DLLs', 'C:\\Users\\adiad\\Anaconda3\\lib', 'C:\\Users\\adiad\\Anaconda3', '', 'C:\\Users\\adiad\\Anaconda3\\lib\\site-packages', 'C:\\Users\\adiad\\Anaconda3\\lib\\site-packages\\win32', 'C:\\Users\\adiad\\Anaconda3\\lib\\site-packages\\win32\\lib', 'C:\\Users\\adiad\\Anaconda3\\lib\\site-packages\\Pythonwin', 'C:\\Users\\adiad\\Anaconda3\\lib\\site-packages\\IPython\\extensions', 'C:\\Users\\adiad\\.ipython']
The IPython session don't appear to reference my virtual environment. So my question is: what do I need to do make IPython run with same environment as my terminal?
I found the following SO question which seems to answer my question, but I find it hard to believe that everyone is following this practice.
How to start an ipython shell(not notebook) within a conda or virtualenv
Here's my configuration:
conda version : 4.7.12
conda-build version : 3.18.8
python version : 3.7.3.final.0
virtual packages :
base environment : C:\Users\adiad\Anaconda3 (writable)
channel URLs : https://conda.anaconda.org/conda-forge/win-64
https://conda.anaconda.org/conda-forge/noarch
https://repo.anaconda.com/pkgs/main/win-64
https://repo.anaconda.com/pkgs/main/noarch
https://repo.anaconda.com/pkgs/r/win-64
https://repo.anaconda.com/pkgs/r/noarch
https://repo.anaconda.com/pkgs/msys2/win-64
https://repo.anaconda.com/pkgs/msys2/noarch
package cache : C:\Users\adiad\Anaconda3\pkgs
C:\Users\adiad\.conda\pkgs
C:\Users\adiad\AppData\Local\conda\conda\pkgs
envs directories : C:\Users\adiad\Anaconda3\envs
C:\Users\adiad\.conda\envs
C:\Users\adiad\AppData\Local\conda\conda\envs
platform : win-64
user-agent : conda/4.7.12 requests/2.22.0 CPython/3.7.3 Windows/10 Windows/10.0.18362
After doing further digging, my problem ought to be filed under, "knowing enough to be dangerous." My problem was ultimately caused by the fact that the jupyter package hadn't yet been installed in my new environment. So whenever I attempted to launch an IPython session of some kind, either in VS Code or in a browser, the application would look in my environment and see that the IPython packages weren't installed. It would then look to other conda environments and use the "nearest" equivalent, which was the base environment. Hence, most of the packages would load, but not all.
The fix to my problem was:
conda install jupyter
Another simple fix:
Launch CMD.exe prompt on Anaconda Navigator
Install: conda install jupyter
And
Conda install plotly

Visual Studio Code, autopep8 doesn't run

On Windows 10.
I did this:
pip install autopep8
and in vscode user settings I have
"python.linting.pep8Enabled": true,
"python.formatting.provider": "autopep8",
When I run format document, or explicitly invoke autopep8 from the command pallette, the error is:
Error: Command failed: autopep8 c:\tca-backend\lambdas\utilities\NetMenuAPIUtil.py
'autopep8' is not recognized as an internal or external command,
operable program or batch file.
Clearly, vsc wants to invoke autopep8.exe but there is no exe. Just py. So I created a autopep8.bat which works when I test from the command line, but when run from vsc, it inserts the content of the batch file into the top of the document. (Yes, that's as strange as it sounds.)
All other Python-related operations work ok, including the ESLint extension.
VS code Python extension supports source code formatting using either autopep8 (the default), black, or yapf so you don't need to install python formatting tools by yourself.
The way I use formatting is to set a shortcut in vs code.
Go to File -> Preferences -> Keyboard Shortcuts, then search format. Set the shortcut as ctrl + shift + p which is the same as the shortcut of autopep8 or you can set any combination you prefer.
Click the shortcuts in your .py files then you will get the formatted code.
Ref: https://code.visualstudio.com/docs/python/editing
Since you are using VSCode on Windows, please click on the Terminal Section and type
pip install pep8
This will start installing pep8.
for me, the autopep8 doc installation isn't enough, still seeing this error, I had to follow this https://pip.pypa.io/en/latest/user_guide/#user-installs. it works like a charm.
I'm on windows so I used this:
py -m pip install –-user autopep8
You need to add this as a PATH under System Environment Variables:
c:\users\<username>\appdata\roaming\python\python39\site-packages
Then close and restart VS Code. Type autopep8 at a terminal prompt and you should see this:
C:\Foobar>autopep8
usage: autopep8 [-h] [--version] [-v] [-d] [-i] [--global-config filename] [--ignore-local-config] [-r] [-j n] [-p n] [-a] [--experimental] [--exclude globs] [--list-fixes] [--ignore errors] [--select errors]
[--max-line-length n] [--line-range line line] [--hang-closing] [--exit-code]
[files ...]
autopep8: error: incorrect number of arguments
I encountered an error message while trying to do same and below was what I did (in my terminal):
Install or upgrade pep8:
pip install --upgrade autopep8
Navigate to the folder/directory where the file you need formatted is, then use the following command:
autopep8 --in-place file_name
There you have it!
I encountered this same error running WSL: Ubuntu-20.04, well this was my solution:
$ pip --version
This will make you verify that you have python3 pip installed correctly in your distribution, if that's not the case, it will pop up an error:
Command 'pip' not found, but can be installed with:
apt install python3-pip
Just run the indicated command to installed it and then after that, run:
$ python3 -m pip install autopep8
Now everything should be working as it should, including the formatting autpep8.

How do I add python3 kernel to jupyter (IPython)

My Jupyter notebooks installed with python 2 kernel. I do not understand why. I might have messed something up when I did the install. I already have python 3 installed. How can I add it to Jupyter?
Here's a screenshot of what the default Jupyter insalled with python3 -m install jupyter and opened in the browser with jupyter notebooklooks like:
This worked for me on Ubuntu 16.04:
python2 -m pip install ipykernel
python2 -m ipykernel install --user
python3 -m pip install ipykernel
python3 -m ipykernel install --user
Reference to the documentation:
Kernels for Python 2 and 3. Installing the IPython kernel - IPython Docs.
Make sure you have ipykernel installed and use ipython kernel install to drop the kernelspec in the right location for python2. Then ipython3 kernel install for Python3. Now you should be able to chose between the 2 kernels regardless of whether you use jupyter notebook, ipython notebook or ipython3 notebook (the later two are deprecated).
Note that if you want to install for a specific Python executable you can use the following trick:
path/to/python -m ipykernel install <options>
This works when using environments (venv,conda,...) and the <options> let you name your kernel (see --help). So you can do
conda create -n py36-test python=3.6
source activate py36-test
python -m ipykernel install --name py36-test
source deactivate
And now you get a kernel named py36-test in your dropdown menus, along the other ones.
See Using both Python 2.x and Python 3.x in IPython Notebook which has more recent information.
Got ipython notebook with Python2 (on Windows7)
upgrade to Jupyter with pip install -U jupyter
install Python3
install Jupyter again using pip3 install jupyter
install Python3 kernel using ipython3 kernelspec install-self
I finally got 2 working kernels.
for recent versions of jupyter/ipython: use jupyter kernelspec
Full doc: https://ipython.readthedocs.io/en/latest/install/kernel_install.html
list current kernels
$ jupyter kernelspec list
Available kernels:
python2 .../Jupyter/kernels/python2
python3 .../Jupyter/kernels/python3
In my case, the python3 kernel setup was broken because the py3.5 linked was no longer there, replaced by a py3.6
add/remove kernels
Remove:
$ jupyter kernelspec uninstall python3
Add a new one:
Using the Python you wish to add and pointing to the python which runs your jupiter:
$ /path/to/kernel/env/bin/python -m ipykernel install --prefix=/path/to/jupyter/env --name 'python-my-env'
See more examples in https://ipython.readthedocs.io/en/6.5.0/install/kernel_install.html#kernels-for-different-environments
List again:
$ jupyter kernelspec list
Available kernels:
python3 /usr/local/lib/python3.6/site-packages/ipykernel/resources
python2 /Users/stefano/Library/Jupyter/kernels/python2
Doc: https://jupyter-client.readthedocs.io/en/latest/kernels.html#kernelspecs
Details
Kernels available are listed under the kernels folder in Jupyter DATA DIRECTORY (see http://jupyter.readthedocs.io/en/latest/projects/jupyter-directories.html for details).
For instance on macosx that would be /Users/YOURUSERNAME/Library/Jupyter/kernels/
the kernel is simply described by a kernel.json file, eg. for /Users/me/Library/Jupyter/kernels/python3/kernel.json
{
"argv": [
"/usr/local/opt/python3/bin/python3.5",
"-m",
"ipykernel",
"-f",
"{connection_file}"
],
"language": "python",
"display_name": "Python 3"
}
Rather then manipulating that by hand, you can use the kernelspec command (as above). It was previously available through ipython now through jupyter (http://ipython.readthedocs.io/en/stable/install/kernel_install.html#kernels-for-different-environments - https://jupyter-client.readthedocs.io/en/latest/kernels.html#kernelspecs).
$ jupyter kernelspec help
Manage Jupyter kernel specifications.
Subcommands
-----------
Subcommands are launched as `jupyter kernelspec cmd [args]`. For information on
using subcommand 'cmd', do: `jupyter kernelspec cmd -h`.
list
List installed kernel specifications.
install
Install a kernel specification directory.
uninstall
Alias for remove
remove
Remove one or more Jupyter kernelspecs by name.
install-self
[DEPRECATED] Install the IPython kernel spec directory for this Python.
To see all available configurables, use `--help-all`
Kernels for other languages
By the way, not strictly related to this question but there's a lot of other kernels available... https://github.com/jupyter/jupyter/wiki/Jupyter-kernels
Following command works for me, on macOS Sierra(10.12.3)
python3 -m pip install ipykernel
python3 -m ipykernel install --user
check it with
jupyter kernelspec list
Available kernels:
python3 /Users/admin/Library/Jupyter/kernels/python3
python2 /anaconda2/share/jupyter/kernels/python2
If you are using the anaconda distribution, this worked for me (on a macintosh):
$ conda create -n py3k python=3 anaconda
$ source activate py3k
$ ipython kernelspec install-self
Just a note for the last command:
(py3k)Monas-MacBook-Pro:cs799 mona$ ipython kernelspec install-self
[TerminalIPythonApp] WARNING | Subcommand `ipython kernelspec` is deprecated and will be removed in future versions.
[TerminalIPythonApp] WARNING | You likely want to use `jupyter kernelspec` in the future
[InstallNativeKernelSpec] WARNING | `jupyter kernelspec install-self` is DEPRECATED as of 4.0. You probably want `ipython kernel install` to install the IPython kernelspec.
[InstallNativeKernelSpec] Installed kernelspec python3 in /usr/local/share/jupyter/kernels/python3
(py3k)Monas-MacBook-Pro:cs799 mona$ ipython kernel install
Installed kernelspec python3 in /usr/local/share/jupyter/kernels/python3
As tested in OSX Yosemite with the above steps and entering jupter notebook and creating a new notebook in the browser you will see the following screenshot:
I successfully installed python3 kernel on macOS El Capitan (ipython version: 4.1.0) with following commands.
python3 -m pip install ipykernel
python3 -m ipykernel install --user
You can see all installed kernels with jupyter kernelspec list.
More info is available here
open terminal(or cmd for window), then run following commands:
(On window, drop "source" in the second line.)
conda create -n py35 python=3.5
source activate py35
conda install notebook ipykernel
ipython kernel install --user --name=python3.5
I tried some method but It doesnt work, then I found this way.
It worked with me. Hoping it can help.
I had Python 2.7 and wanted to be able to switch to Python 3 inside of Jupyter.
These steps worked for me on a Windows Anaconda Command Prompt:
conda update conda
conda create -n py33 python=3.3 anaconda
activate py33
ipython kernelspec install-self
deactivate
Now after opening ipython notebook with the usual command for Python2.7, Python3.3 is also available when creating a new notebook.
Shortest and best solution from my side
conda create --name py3-ml python=3.6
conda install ipykernel
source activate py3-ml # or conda activate py3-ml
python -m ipykernel install --user --name py3-ml
sudo apt-get install python3-pip python3-dev
pip3 install -U jupyter
For the current Python Launcher
If you have Py3 installed but default to py2
py -3 -m pip install ipykernel
py -3 -m ipykernel install --user
If you have Py2 installed but default to py3
py -2 -m pip install ipykernel
py -2 -m ipykernel install --user
On Ubuntu 14.04 I had to use a combination of previous answers.
First, install pip3
apt-get install python-pip3
Then with pip3 install jupyter
pip3 install jupyter
Then using ipython3 install the kernel
ipython3 kernel install
None of the other answers were working for me immediately on ElementaryOS Freya (based on Ubuntu 14.04); I was getting the
[TerminalIPythonApp] WARNING | File not found: 'kernelspec'
error that quickbug described under Matt's answer. I had to first do:
sudo apt-get install pip3, then
sudo pip3 install ipython[all]
At that point you can then run the commands that Matt suggested; namely: ipython kernelspec install-self and ipython3 kernelspec install-self
Now when I launch ipython notebook and then open a notebook, I am able to select the Python 3 kernel from the Kernel menu.
I managed to install a Python3 kernel besides the Python2. Here is the way I did it:
open a new notebook in Jupyter
copy and run the two cells here: Enable-Python-3-kernel
The latest working link can be found here.
The actual code is:
! mkdir -p ~/.ipython/kernels/python3
%%file ~/.ipython/kernels/python3/kernel.json
{
"display_name": "IPython (Python 3)",
"language": "python",
"argv": [
"python3",
"-c", "from IPython.kernel.zmq.kernelapp import main; main()",
"-f", "{connection_file}"
],
"codemirror_mode": {
"version": 2,
"name": "ipython"
}
}
Adding kernel means you want to use Jupyter Notebook with versions of python which are not showing up in the list.
Simple approach- Start notebook with required python version, suppose I have python3.7 installed then use below command from terminal (cmd) to run notebook:
python3.7 -m notebook
Sometimes instead of python3.7 it's install with alias of py, py3.7, python.
To add specific python into the jupyter kernel, first check the paths of the available python or python3 using the following command
$ where python3
Let's say you have '/usr/local/bin/python3' as one of the paths. To create a kernel for this version of python with name 'name_to_new_kernel' which will showup in jupyter,
$ /usr/local/bin/python3 -m pip install ipykernel
$ /usr/local/bin/python3 -m ipykernel install --user --name name_to_new_kernel
To check if the new kernel with name 'name_to_new_kernel' added to jupyter run the following command
jupyter kernelspec list
The solution is well documented in the official docs: https://ipython.readthedocs.org/en/latest/install/kernel_install.html
I tried the first approach. Since I already had ipykernel installed, simply running python3 -m ipykernel install --user solved the problem.
I was facing the same issue. Whenever I would start jupyter notebook using the following command, I would not see an option to create a new Python 3 Notebook.
I solved the issue by the running the following command.
pip install notebook
After that I started notebook with the same command as before
jupyter notebook
I was able to see the my home directory and create new python3 notebooks.
This answer explains how to create a Python 3, Jupyter 1, and ipykernel 5 workflow with Poetry dependency management. Poetry makes creating a virtual environment for Jupyter notebooks easy. I strongly recommend against running python3 commands. Python workflows that install global dependencies set you up for dependency hell.
Here's a summary of the clean, reliable Poetry workflow:
Install the dependencies with poetry add pandas jupyter ipykernel
Open a shell within the virtual environment with poetry shell
Open the Jupyter notebook with access to all the virtual environment dependencies with jupyter notebook
There are clean Conda workflows as well. Watch out for a lot of the answers in this thread - they'll set you down a path that'll cause a lot of pain & suffering.
INSTALLING MULTIPLE KERNELS TO A SINGLE VIRTUAL ENVIRONMENT (VENV)
Most (if not all) of these answers assume you are happy to install packages globally. This answer is for you if you:
use a *NIX machine
don't like installing packages globally
don't want to use anaconda <-> you're happy to run the jupyter server from the command line
want to have a sense of what/where the kernel installation 'is'.
(Note: this answer adds a python2 kernel to a python3-jupyter install, but it's conceptually easy to swap things around.)
Prerequisites
You're in the dir from which you'll run the jupyter server and save files
python2 is installed on your machine
python3 is installed on your machine
virtualenv is installed on your machine
Create a python3 venv and install jupyter
Create a fresh python3 venv: python3 -m venv .venv
Activate the venv: . .venv/bin/activate
Install jupyterlab: pip install jupyterlab. This will create locally all the essential infrastructure for running notebooks.
Note: by installing jupyterlab here, you also generate default 'kernel specs' (see below) in $PWD/.venv/share/jupyter/kernels/python3/. If you want to install and run jupyter elsewhere, and only use this venv for organizing all your kernels, then you only need: pip install ipykernel
You can now run jupyter lab with jupyter lab (and go to your browser to the url displayed in the console). So far, you'll only see one kernel option called 'Python 3'. (This name is determined by the display_name entry in your kernel.json file.)
Add a python2 kernel
Quit jupyter (or start another shell in the same dir): ctrl-c
Deactivate your python3 venv: deactivate
Create a new venv in the same dir for python2: virtualenv -p python2 .venv2
Activate your python2 venv: . .venv2/bin/activate
Install the ipykernel module: pip install ipykernel. This will also generate default kernel specs for this python2 venv in .venv2/share/jupyter/kernels/python2
Export these kernel specs to your python3 venv: python -m ipykernel install --prefix=$PWD/.venv. This basically just copies the dir $PWD/.venv2/share/jupyter/kernels/python2 to $PWD/.venv/share/jupyter/kernels/
Switch back to your python3 venv and/or rerun/re-examine your jupyter server: deactivate; . .venv/bin/activate; jupyter lab. If all went well, you'll see a Python 2 option in your list of kernels. You can test that they're running real python2/python3 interpreters by their handling of a simple print 'Hellow world' vs print('Hellow world') command.
Note: you don't need to create a separate venv for python2 if you're happy to install ipykernel and reference the python2-kernel specs from a global space, but I prefer having all of my dependencies in one local dir
TL;DR
Optionally install an R kernel. This is instructive to develop a sense of what a kernel 'is'.
From the same dir, install the R IRkernel package: R -e "install.packages('IRkernel',repos='https://cran.mtu.edu/')". (This will install to your standard R-packages location; for home-brewed-installed R on a Mac, this will look like /usr/local/Cellar/r/3.5.2_2/lib/R/library/IRkernel.)
The IRkernel package comes with a function to export its kernel specs, so run: R -e "IRkernel::installspec(prefix=paste(getwd(),'/.venv',sep=''))". If you now look in $PWD/.venv/share/jupyter/kernels/ you'll find an ir directory with kernel.json file that looks something like this:
{
"argv": ["/usr/local/Cellar/r/3.5.2_2/lib/R/bin/R", "--slave", "-e", "IRkernel::main()", "--args", "{connection_file}"],
"display_name": "R",
"language": "R"
}
In summary, a kernel just 'is' an invocation of a language-specific executable from a kernel.json file that jupyter looks for in the .../share/jupyter/kernels dir and lists in its interface; in this case, R is being called to run the function IRkernel::main(), which will send messages back and forth to the Jupiter server. Likewise, the python2 kernel just 'is' an invocation of the python2 interpreter with module ipykernel_launcher as seen in .venv/share/jupyter/kernels/python2/kernel.json, etc.
Here is a script if you want to run all of these instructions in one fell swoop.
Here's a Windows/non command line method I found, which worked for me:
Find the folder where the kernel files are stored (on my machine - C:\ProgramData\jupyter\kernels - note that ProgramData is a hidden folder), create a copy of the existing kernel's folder, change the name and edit the json file within to point to the new kernel's directory. In this json you can also edit the kernel name that is displayed in ipython (e.g. instead of just python 2 you can specify 2.7.9 if you need to further distinguish for some reason).
When you use conda managing your python envs, follow these two steps:
activate py3 (on Windows or source activate py3 on Linux)
conda install notebook ipykernel or just use conda install jupyter
I was getting same error with python-2. I wanted to run python-2 jupyter notebook session but by default I was getting python-3. So easiest work around is open Anaconda terminal for python-2 and type 'jupyter notebook' , it will launch jupyter-notebook session without any problem. Similary it could be tried with python-3
I am pretty sure all you have to do is run
pip3 install jupyter