I prefer using the ipython enhanced REPL over the python and so am specifying as such in the pyspark command line:
Result: jupyter notebook launched (surprising):
PYSPARK_DRIVER_PYTHON=ipython MASTER="local[*]" $SPARK_HOME/bin/pyspark
This approach was working for several years - but at the present it is causing jupyter notebook to be launched.
That result would make sense if the explicit command notebook were also present:
Result: jupyter notebook launched (expected):
PYSPARK_DRIVER_PYTHON=ipython notebook MASTER="local[*]" $SPARK_HOME/bin/pyspark
Given that the notebook is not specified then why is the notebook launched instead of the REPL ?
Note: when using ipython alone on the command line via
Result: ipython REPL launched (expected):
ipython
the REPL is launched (and not the jupyter notebook).
you might have set PYSPARK_DRIVER_PYTHON_OPTS variable to "notebook". this will cause to open jupyter notebook
Related
I want to use the Wolfram Language with Jupyter notebooks, and VSCode is the application of my choice. I did all the procedure to generated a WolframLanguage kernel for Jupyter.
If I launch the Jupyter notebook from terminal by running jupyter notebook in my terminal, a Jupyter notebook opens in my browser and it shows the Wolfram kernel. These are the kernels returned by terminal.
They show up in the browser-version of Jupyter too.
Now I go to VS Code. I start a new Jupyter notebook by using the following command by using shift+command+P.
Create: New Jupyter Notebook
Now the kernel picker prompts me to choose kernels. Not only does it not have the WolframLanguage kernel, but it also shows completely different kernels from what were returned by the terminal.
Please guide me on how do I make my WolframLanguage kernel appear in my VS Code.
Thanks!!!
I've got a notebook that has got a bit unwieldy and I'm doing some refactoring which isn't fun.
I was wondering if it would be possible to execute code in this notebook from the command line for debugging.
Ideally, I would run something like:
run-in-jupyter $notebook file.py
and see the output from the command line. There is an interpreter in jupyterlab that can do this, so this make me think that it is possible.
I have a brief search but couldn't find much
How to run an .ipynb Jupyter Notebook from terminal? I explicitly don't want to do this (I want to run commands in an existing instace)
There is this library but this seems quite involved and some of the results I found on the internet where people not being able to use the library
jupyter console (pip install jupyter-console) connects to a running jupyter kernel from the kernel. Details on running kernels can be found amongst jupyter's run time files, on my box these live in ~/.local/share/jupyter/runtime. You can find the path to the kernel data file corresponding to an open workbook with %config IPKernelApp.connection_file which will look something like ~/.local/share/jupyter/runtime/kernel-55da8a07-b67d-4584-9ec6-f24e4a26cbbd.json.
You can then connect from the command line with
jupyter console --existing ~/.local/share/jupyter/runtime/kernel-55da8a07-b67d-4584-9ec6-f24e4a26cbbd.json
You can pipe commands into it as shown
echo h=87 | jupyter console --existing 55da8a07-b67d-4584-9ec6-f24e4a26cbbd 'h=57' --simple-prompt -y
Is there a way to open an IPython interactive console in pycharm that is connected to an existing running kernel (similar to "python --existing")?
btw: in case it's relevant, in my case, the running kernel is of a Jupiter notebook...
EDIT: To clarify, my question is NOT about how to open an interactive console in PyCharm. It is about how to connect that interactive console to a an existing running (Jupiter notebook) Kernel.
(tl;dr: Use jupyter console --existing in the PyCharm "Terminal" tool window (not the "Python Console" tool window) to connect to an existing iPython kernel running in a local Jupyter Notebook server.)
I can confirm that the comment by #john-moutafis suggesting ipython console --existing is the right idea. The command gives "WARNING | You likely want to use jupyter console in the future" so I tried that.
I have a project using a conda environment as its interpreter. Jupyter Notebook is installed in the conda environment.
I open the Terminal tool window. It automatically activates the conda environment.
I type jupyter notebook. The notebook server starts and a browser window opens.
I create a notebook in the browser, and execute a cell containing foo = "bar".
In PyCharm, I open another Terminal tool window by clicking the plus sign to the left of the terminal pane.
In the new terminal I type jupyter console --existing, and it starts an ipython console session.
At the prompt I type dir(), and foo is among the results, confirming that I'm attached to the same kernel as the notebook.
I don't know how it picks which kernel to connect to when there are multiple kernels running in the notebook server.
Don't type exit in the iPython session if you plan to continue using the notebook, it shuts down the kernel.
Unfortunately, tools like Debug and "Execute Line/Selection in Console", which are available for the "Python Console" tool window, are not available for the "Terminal" tool window. In fact, because the Terminal tool window is a simple tool, and that's where I've run my commands, this solution isn't very integrated with PyCharm. The terminal opens in the project directory and activates the conda environment, and it's conveniently adjacent to the editors and tools of the IDE, but otherwise there's no connection to PyCharm's tools.
If anyone can successfully attach PyCharm's integrated PyDev debugger to a running kernel, please chime in.
I'm using PyCharm 2016.3 on macOS 10.12.3.
The easiest way for me is just to type %qtconsole in a jupyter notebook cell and run it. A qt console will open already connected to the running kennel. No PyCharm involved.
My PyCharm freezes after running any code in .ipynb (Jupyter Notebook) file.
i.e. the cell starts running and after that PyCharm doesnt respond and I have to kill it. No message is shown.
.py scripts are running correctly. Can anyone help?
My versions:
PyCharm Community Edition 2016.2.3 (Ubuntu 16.04)
ipython 5.1.0 py35_0
jupyter 1.0.0 py35_3
notebook 4.2.3 py35_0
It appears that your jupyter kernel is not running.
I have had success by first starting a jupyter notebook from the command line:
$jupyter notebook.
Then go back to the PyCharm jupyter notebook and run a cell. If you get the dialog box to enter the Jupyter Notebook URL, enter the url printed to your command line terminal session after you started the notebook there.
I am using the ipython notebook to code but sometimes I like to connect to the kernel on a terminal to do some debugging. To launch the interactive terminal connected to my kernel I do:
ipython console --existing kernel-715d122b-4902-4c9d-a663-8da4c34c96ee.json --profile notebook
When I do this I do not have access to the %paste magic since it is not loaded in the notebook profile. How can I load the %paste magic in this case?
Thanks.