Jupyter error when starting Kernel. Cell not executing just shows * - jupyter

I use a custom kernel manager, which launches kernels as Docker containers.
Recently updated my system, but can't find any relevant Jupyter versions which may affect this:
Tornado 6.2
JupyterLab Version 3.2.9
Custom Kernels
When starting a new Kernel I get:
[I 2022-07-28 10:17:20.946 ServerApp] Kernel started: b2039b53-2ce0-48f1-b327-2b0393a53ff5
[W 2022-07-28 10:17:20.952 ServerApp] No session ID specified
NOTE: When using the `ipython kernel` entry point, Ctrl-C will not work.
To exit, you will have to explicitly quit this process, by either sending
"quit" from a client, or using Ctrl-\ in UNIX-like environments.
To read more about this, see https://github.com/ipython/ipython/issues/2049
To connect another client to this kernel, use:
--existing /connection-spec
[W 2022-07-28 10:18:20.954 ServerApp] Timeout waiting for kernel_info reply from b2039b53-2ce0-48f1-b327-2b0393a53ff5
[W 2022-07-28 10:18:25.473 ServerApp] Nudge: attempt 10 on kernel b2039b53-2ce0-48f1-b327-2b0393a53ff5
[W 2022-07-28 10:18:30.486 ServerApp] Nudge: attempt 20 on kernel b2039b53-2ce0-48f1-b327-2b0393a53ff5
[W 2022-07-28 10:18:35.499 ServerApp] Nudge: attempt 30 on kernel b2039b53-2ce0-48f1-b327-2b0393a53ff5
[W 2022-07-28 10:18:40.517 ServerApp] Nudge: attempt 40 on kernel b2039b53-2ce0-48f1-b327-2b0393a53ff5
[W 2022-07-28 10:18:45.533 ServerApp] Nudge: attempt 50 on kernel b2039b53-2ce0-48f1-b327-2b0393a53ff5
[W 2022-07-28 10:18:50.548 ServerApp] Nudge: attempt 60 on kernel b2039b53-2ce0-48f1-b327-2b0393a53ff5
[W 2022-07-28 10:18:55.562 ServerApp] Nudge: attempt 70 on kernel b2039b53-2ce0-48f1-b327-2b0393a53ff5
[W 2022-07-28 10:19:00.576 ServerApp] Nudge: attempt 80 on kernel b2039b53-2ce0-48f1-b327-2b0393a53ff5
[W 2022-07-28 10:19:05.592 ServerApp] Nudge: attempt 90 on kernel b2039b53-2ce0-48f1-b327-2b0393a53ff5
[W 2022-07-28 10:19:10.607 ServerApp] Nudge: attempt 100 on kernel b2039b53-2ce0-48f1-b327-2b0393a53ff5
[W 2022-07-28 10:19:15.624 ServerApp] Nudge: attempt 110 on kernel b2039b53-2ce0-48f1-b327-2b0393a53ff5
[W 2022-07-28 10:19:20.637 ServerApp] Nudge: attempt 120 on kernel b2039b53-2ce0-48f1-b327-2b0393a53ff5
[E 2022-07-28 10:19:20.960 ServerApp] Uncaught exception GET /api/kernels/b2039b53-2ce0-48f1-b327-2b0393a53ff5/channels (127.0.0.1)
HTTPServerRequest(protocol='http', host='127.0.0.1:8080', method='GET', uri='/api/kernels/b2039b53-2ce0-48f1-b327-2b0393a53ff5/channels', version='HTTP/1.1', remote_ip='127.0.0.1')
Traceback (most recent call last):
File "/opt/conda/lib/python3.7/site-packages/tornado/websocket.py", line 944, in _accept_connection
await open_result
File "/opt/conda/lib/python3.7/asyncio/tasks.py", line 318, in __wakeup
future.result()
concurrent.futures._base.TimeoutError: Timeout
Working case I see, Websocket is opening successfully and:
[I 2022-07-28 10:23:49.698 ServerApp] Kernel started: 84d2de07-7ed5-46ba-8f85-c2bb583be71d
[W 2022-07-28 10:23:49.704 ServerApp] No session ID specified
NOTE: When using the `ipython kernel` entry point, Ctrl-C will not work.
To exit, you will have to explicitly quit this process, by either sending
"quit" from a client, or using Ctrl-\ in UNIX-like environments.
To read more about this, see https://github.com/ipython/ipython/issues/2049
To connect another client to this kernel, use:
--existing /connection-spec
[W 2022-07-28 10:23:53.058 ServerApp] No channel specified, assuming shell: {'header': {'msg_id': '6101c9180e5f11ed969a42010a800005', 'username': 'test', 'session': '6101cc4c0e5f11ed969a42010a800005', 'data': '2022-07-28T10:23:53.054348', 'msg_type': 'execute_request', 'version': '5.0'}, 'parent_header': {'msg_id': '6101c9180e5f11ed969a42010a800005', 'username': 'test', 'session': '6101cc4c0e5f11ed969a42010a800005', 'data': '2022-07-28T10:23:53.054348', 'msg_type': 'execute_request', 'version': '5.0'}, 'metadata': {}, 'content': {'code': 'print(123)', 'silent': False}}
any suggestions?

Related

Don't start JupyterLab if configuration is invalid

When JupyterLab tries to start, in some occasions, the config file may be invalid, and Jupyterlab starts regardless in default port 8888.
Currently I have a JupyterLab service defined as follows, but I also try to start it manually and got the same results.
An example when config file may be invalid is when a library is not available to be imported.
I want to prevent Jupyter to start and fail gracefully.
[Unit]
Description=Jupyter Service
[Service]
Type=simple
PIDFile=/run/jupyter.pid
ExecStart=/bin/bash --login -c '/opt/conda/bin/jupyter lab --config=/home/jupyter/.jupyter/jupyter_notebook_config.py'
User=jupyter
Group=jupyter
WorkingDirectory=/home/jupyter
Restart=no
[Install]
WantedBy=multi-user.target
(base) root#ab-test1:/home/jupyter/.jupyter# service jupyter status
● jupyter.service - Jupyter Notebook Service
Loaded: loaded (/lib/systemd/system/jupyter.service; enabled; vendor preset: enabled)
Active: active (running) since Wed 2022-02-02 07:53:24 UTC; 3s ago
Main PID: 335 (jupyter-lab)
Tasks: 1 (limit: 4915)
Memory: 64.0M (high: 14.3G max: 14.3G)
CGroup: /system.slice/jupyter.service
└─335 /opt/conda/bin/python3.7 /opt/conda/bin/jupyter-lab --config=/home/jupyter/.jupyter/jupyter_notebook_config.py
Feb 02 07:53:27 ab-test1 bash[335]: Traceback (most recent call last):
Feb 02 07:53:27 ab-test1 bash[335]: File "/opt/conda/lib/python3.7/site-packages/traitlets/config/application.py", line 738, in _load_config_files
Feb 02 07:53:27 ab-test1 bash[335]: config = loader.load_config()
Feb 02 07:53:27 ab-test1 bash[335]: File "/opt/conda/lib/python3.7/site-packages/traitlets/config/loader.py", line 614, in load_config
Feb 02 07:53:27 ab-test1 bash[335]: self._read_file_as_dict()
Feb 02 07:53:27 ab-test1 bash[335]: File "/opt/conda/lib/python3.7/site-packages/traitlets/config/loader.py", line 646, in _read_file_as_dict
Feb 02 07:53:27 ab-test1 bash[335]: exec(compile(f.read(), conf_filename, 'exec'), namespace, namespace)
Feb 02 07:53:27 ab-test1 bash[335]: File "/home/jupyter/.jupyter/jupyter_notebook_config.py", line 3, in <module>
Feb 02 07:53:27 ab-test1 bash[335]: import tensorflow
Feb 02 07:53:27 ab-test1 bash[335]: ModuleNotFoundError: No module named 'tensorflow'
(base) root#ab-test1:/home/jupyter/.jupyter# service jupyter status
● jupyter.service - Jupyter Notebook Service
Loaded: loaded (/lib/systemd/system/jupyter.service; enabled; vendor preset: enabled)
Active: active (running) since Wed 2022-02-02 07:53:24 UTC; 6s ago
Main PID: 335 (jupyter-lab)
Tasks: 2 (limit: 4915)
Memory: 138.6M (high: 14.3G max: 14.3G)
CGroup: /system.slice/jupyter.service
└─335 /opt/conda/bin/python3.7 /opt/conda/bin/jupyter-lab --config=/home/jupyter/.jupyter/jupyter_notebook_config.py
Feb 02 07:53:29 ab-test1 bash[335]: [I 2022-02-02 07:53:29.644 ServerApp] or http://127.0.0.1:8888/lab?token=ead21ce39f979fe916e7e041134da576feb28b839bd99577
Feb 02 07:53:29 ab-test1 bash[335]: [I 2022-02-02 07:53:29.644 ServerApp] Use Control-C to stop this server and shut down all kernels (twice to skip confirmation)
Feb 02 07:53:29 ab-test1 bash[335]: [W 2022-02-02 07:53:29.648 ServerApp] No web browser found: could not locate runnable browser.
Feb 02 07:53:29 ab-test1 bash[335]: [C 2022-02-02 07:53:29.648 ServerApp]
Feb 02 07:53:29 ab-test1 bash[335]:
Feb 02 07:53:29 ab-test1 bash[335]: To access the server, open this file in a browser:
Feb 02 07:53:29 ab-test1 bash[335]: file:///home/jupyter/.local/share/jupyter/runtime/jpserver-335-open.html
Feb 02 07:53:29 ab-test1 bash[335]: Or copy and paste one of these URLs:
Feb 02 07:53:29 ab-test1 bash[335]: http://localhost:8888/lab?token=ead21ce39f979fe916e7e041134da576feb28b839bd99577
Feb 02 07:53:29 ab-test1 bash[335]: or http://127.0.0.1:8888/lab?token=ead21ce39f979fe916e7e041134da576feb28b839bd99577
Logs:
/opt/conda/bin/jupyter-lab --debug --config=/home/jupyter/.jupyter/jupyter_notebook_config.py
[D 2022-02-02 08:09:28.170 ServerApp] Searching ['/home/gogasca', '/home/gogasca/.jupyter', '/home/gogasca/.local/etc/jupyter', '/opt/conda/etc/jupyter', '/usr/local/etc/jupyter', '/etc/jupyter'] for config files
[D 2022-02-02 08:09:28.170 ServerApp] Looking for jupyter_config in /etc/jupyter
[D 2022-02-02 08:09:28.170 ServerApp] Looking for jupyter_config in /usr/local/etc/jupyter
[D 2022-02-02 08:09:28.170 ServerApp] Looking for jupyter_config in /opt/conda/etc/jupyter
[D 2022-02-02 08:09:28.171 ServerApp] Loaded config file: /opt/conda/etc/jupyter/jupyter_config.json
[D 2022-02-02 08:09:28.171 ServerApp] Looking for jupyter_config in /home/gogasca/.local/etc/jupyter
[D 2022-02-02 08:09:28.171 ServerApp] Looking for jupyter_config in /home/gogasca/.jupyter
[D 2022-02-02 08:09:28.171 ServerApp] Looking for jupyter_config in /home/gogasca
[D 2022-02-02 08:09:28.173 ServerApp] Looking for jupyter_notebook_config in /home/jupyter/.jupyter
[E 2022-02-02 08:09:28.174 ServerApp] Exception while loading config file /home/jupyter/.jupyter/jupyter_notebook_config.py
Traceback (most recent call last):
File "/opt/conda/lib/python3.7/site-packages/traitlets/config/application.py", line 738, in _load_config_files
config = loader.load_config()
File "/opt/conda/lib/python3.7/site-packages/traitlets/config/loader.py", line 614, in load_config
self._read_file_as_dict()
File "/opt/conda/lib/python3.7/site-packages/traitlets/config/loader.py", line 646, in _read_file_as_dict
exec(compile(f.read(), conf_filename, 'exec'), namespace, namespace)
File "/home/jupyter/.jupyter/jupyter_notebook_config.py", line 3, in <module>
import tensorflow
ModuleNotFoundError: No module named 'tensorflow'
[D 2022-02-02 08:09:28.177 ServerApp] Paths used for configuration of jupyter_server_config:
/etc/jupyter/jupyter_server_config.json
[D 2022-02-02 08:09:28.178 ServerApp] Paths used for configuration of jupyter_server_config:
/usr/local/etc/jupyter/jupyter_server_config.json
[D 2022-02-02 08:09:28.178 ServerApp] Paths used for configuration of jupyter_server_config:
/opt/conda/etc/jupyter/jupyter_server_config.d/beatrix_jupyterlab.json
/opt/conda/etc/jupyter/jupyter_server_config.d/jupyter-server-proxy-jupyterserverextension.json
/opt/conda/etc/jupyter/jupyter_server_config.d/jupyter_server_mathjax.json
/opt/conda/etc/jupyter/jupyter_server_config.d/jupyterlab.json
/opt/conda/etc/jupyter/jupyter_server_config.d/jupyterlab_git.json
/opt/conda/etc/jupyter/jupyter_server_config.d/jupytext.json
/opt/conda/etc/jupyter/jupyter_server_config.d/nbclassic.json
/opt/conda/etc/jupyter/jupyter_server_config.d/nbdime.json
/opt/conda/etc/jupyter/jupyter_server_config.json
[D 2022-02-02 08:09:28.179 ServerApp] Paths used for configuration of jupyter_server_config:
/home/gogasca/.local/etc/jupyter/jupyter_server_config.json
[D 2022-02-02 08:09:28.180 ServerApp] Paths used for configuration of jupyter_server_config:
/home/gogasca/.jupyter/jupyter_server_config.json
generated new fontManager
/opt/conda/lib/python3.7/site-packages/jupyter_server_mathjax/app.py:40: FutureWarning: The alias `_()` will be deprecated. Use `_i18n()` instead.
help=_("""The MathJax.js configuration file that is to be used."""),
[I 2022-02-02 08:09:32.735 ServerApp] beatrix_jupyterlab | extension was successfully linked.
[W 2022-02-02 08:09:32.737 MathJaxExtension] Config option `kernel_spec_manager_class` not recognized by `MathJaxExtension`.
[W 2022-02-02 08:09:32.738 MathJaxExtension] Config option `kernel_spec_manager_class` not recognized by `MathJaxExtension`.
[W 2022-02-02 08:09:32.743 MathJaxExtension] Config option `kernel_spec_manager_class` not recognized by `MathJaxExtension`.
[D 2022-02-02 08:09:32.744 MathJaxExtension] Config changed: {'JupyterApp': {'kernel_spec_manager_class': 'nb_conda_kernels.CondaKernelSpecManager'}, 'ServerApp': {'config_file': '/home/jupyter/.jupyter/jupyter_notebook_config.py', 'log_level': 'DEBUG', 'jpserver_extensions': <LazyConfigValue {'update': {'beatrix_jupyterlab': True, 'jupyter_server_mathjax': True, 'jupyter_server_proxy': True, 'jupyterlab_git': True, 'jupytext': True, 'nbclassic': True, 'nbdime': True}}>}, 'ExtensionApp': {'log_level': 'DEBUG'}}
[D 2022-02-02 08:09:32.745 ServerApp] Config changed: {'ServerApp': {'config_file': '/home/jupyter/.jupyter/jupyter_notebook_config.py', 'log_level': 'DEBUG', 'jpserver_extensions': <LazyConfigValue value={'jupyterlab': True, 'beatrix_jupyterlab': True, 'jupyter_server_mathjax': True, 'jupyter_server_proxy': True, 'jupyterlab_git': True, 'jupytext': True, 'nbclassic': True, 'nbdime': True}>}, 'ExtensionApp': {'log_level': 'DEBUG'}, 'JupyterApp': {'kernel_spec_manager_class': 'nb_conda_kernels.CondaKernelSpecManager'}}
[I 2022-02-02 08:09:32.746 ServerApp] jupyter_server_mathjax | extension was successfully linked.
[I 2022-02-02 08:09:32.746 ServerApp] jupyter_server_proxy | extension was successfully linked.
[W 2022-02-02 08:09:32.749 LabApp] Config option `kernel_spec_manager_class` not recognized by `LabApp`.
[W 2022-02-02 08:09:32.753 LabApp] Config option `kernel_spec_manager_class` not recognized by `LabApp`.
[W 2022-02-02 08:09:32.760 LabApp] Config option `kernel_spec_manager_class` not recognized by `LabApp`.
[D 2022-02-02 08:09:32.761 LabApp] Config changed: {'JupyterApp': {'kernel_spec_manager_class': 'nb_conda_kernels.CondaKernelSpecManager'}, 'NotebookApp': {}, 'ServerApp': {'config_file': '/home/jupyter/.jupyter/jupyter_notebook_config.py', 'log_level': 'DEBUG', 'jpserver_extensions': <LazyConfigValue value={'jupyterlab': True, 'beatrix_jupyterlab': True, 'jupyter_server_mathjax': True, 'jupyter_server_proxy': True, 'jupyterlab_git': True, 'jupytext': True, 'nbclassic': True, 'nbdime': True}>}, 'ExtensionApp': {'log_level': 'DEBUG'}}
[I 2022-02-02 08:09:32.762 ServerApp] jupyterlab | extension was successfully linked.
[I 2022-02-02 08:09:32.762 ServerApp] jupyterlab_git | extension was successfully linked.
[I 2022-02-02 08:09:32.762 ServerApp] jupytext | extension was successfully linked.
[W 2022-02-02 08:09:32.765 NotebookApp] Config option `kernel_spec_manager_class` not recognized by `NotebookApp`.
[W 2022-02-02 08:09:32.768 NotebookApp] Config option `kernel_spec_manager_class` not recognized by `NotebookApp`.
[W 2022-02-02 08:09:32.774 NotebookApp] Config option `kernel_spec_manager_class` not recognized by `NotebookApp`.
[D 2022-02-02 08:09:32.774 NotebookApp] Config changed: {'JupyterApp': {'kernel_spec_manager_class': 'nb_conda_kernels.CondaKernelSpecManager'}, 'NotebookApp': {'nbserver_extensions': {'nb_conda': True, 'jupyter_http_over_ws': True}}, 'ServerApp': {'config_file': '/home/jupyter/.jupyter/jupyter_notebook_config.py', 'log_level': 'DEBUG', 'jpserver_extensions': <LazyConfigValue value={'jupyterlab': True, 'beatrix_jupyterlab': True, 'jupyter_server_mathjax': True, 'jupyter_server_proxy': True, 'jupyterlab_git': True, 'jupytext': True, 'nbclassic': True, 'nbdime': True}>}, 'ExtensionApp': {'log_level': 'DEBUG'}}
[I 2022-02-02 08:09:32.778 ServerApp] Writing Jupyter server cookie secret to /home/gogasca/.local/share/jupyter/runtime/jupyter_cookie_secret
[D 2022-02-02 08:09:32.782 ServerApp] Paths used for configuration of jupyter_notebook_config:
/home/gogasca/.jupyter/jupyter_notebook_config.json
[D 2022-02-02 08:09:32.782 ServerApp] Paths used for configuration of jupyter_notebook_config:
/etc/jupyter/jupyter_notebook_config.json
[D 2022-02-02 08:09:32.783 ServerApp] Paths used for configuration of jupyter_notebook_config:
/usr/local/etc/jupyter/jupyter_notebook_config.json
[D 2022-02-02 08:09:32.784 ServerApp] Paths used for configuration of jupyter_notebook_config:
/opt/conda/etc/jupyter/jupyter_notebook_config.d/jupyter-server-proxy-notebookserverextension.json
/opt/conda/etc/jupyter/jupyter_notebook_config.d/jupyterlab.json
/opt/conda/etc/jupyter/jupyter_notebook_config.d/jupyterlab_git.json
/opt/conda/etc/jupyter/jupyter_notebook_config.d/jupytext.json
/opt/conda/etc/jupyter/jupyter_notebook_config.d/nbdime.json
/opt/conda/etc/jupyter/jupyter_notebook_config.json
[D 2022-02-02 08:09:32.784 ServerApp] Paths used for configuration of jupyter_notebook_config:
/home/gogasca/.local/etc/jupyter/jupyter_notebook_config.json
[D 2022-02-02 08:09:32.785 ServerApp] Paths used for configuration of jupyter_notebook_config:
/home/gogasca/.jupyter/jupyter_notebook_config.json
[I 2022-02-02 08:09:32.787 ServerApp] nb_conda | extension was found and enabled by nbclassic. Consider moving the extension to Jupyter Server's extension paths.
[I 2022-02-02 08:09:32.787 ServerApp] nb_conda | extension was successfully linked.
[I 2022-02-02 08:09:32.789 ServerApp] jupyter_http_over_ws | extension was found and enabled by nbclassic. Consider moving the extension to Jupyter Server's extension paths.
[I 2022-02-02 08:09:32.789 ServerApp] jupyter_http_over_ws | extension was successfully linked.
[I 2022-02-02 08:09:32.789 ServerApp] nbclassic | extension was successfully linked.
[I 2022-02-02 08:09:32.789 ServerApp] nbdime | extension was successfully linked.
[D 2022-02-02 08:09:32.790 ServerApp] Config changed: {'ExtensionApp': {'log_level': 'DEBUG'}, 'JupyterApp': {'kernel_spec_manager_class': 'nb_conda_kernels.CondaKernelSpecManager'}, 'NotebookApp': {'nbserver_extensions': {'nb_conda': True, 'jupyter_http_over_ws': True}}, 'ServerApp': {'config_file': '/home/jupyter/.jupyter/jupyter_notebook_config.py', 'log_level': 'DEBUG', 'jpserver_extensions': <LazyConfigValue value={'jupyterlab': True, 'beatrix_jupyterlab': True, 'jupyter_server_mathjax': True, 'jupyter_server_proxy': True, 'jupyterlab_git': True, 'jupytext': True, 'nbclassic': True, 'nbdime': True}>}}
[D 2022-02-02 08:09:32.791 ServerApp] Raising open file limit: soft 1024->4096; hard 1048576->1048576
[D 2022-02-02 08:09:32.792 ServerApp] [nb_conda_kernels] refreshing conda info
[I 2022-02-02 08:09:33.136 ServerApp] [nb_conda_kernels] enabled, 1 kernels found
[I 2022-02-02 08:09:33.161 ServerApp] nbclassic | extension was successfully loaded.
[I 2022-02-02 08:09:33.163 ServerApp] beatrix_jupyterlab | extension was successfully loaded.
jupyter_http_over_ws extension initialized. Listening on /http_over_websocket
[I 2022-02-02 08:09:33.164 ServerApp] jupyter_http_over_ws | extension was successfully loaded.
[I 2022-02-02 08:09:33.165 ServerApp] jupyter_server_mathjax | extension was successfully loaded.
[I 2022-02-02 08:09:33.177 ServerApp] jupyter_server_proxy | extension was successfully loaded.
[I 2022-02-02 08:09:33.178 LabApp] JupyterLab extension loaded from /opt/conda/lib/python3.7/site-packages/jupyterlab
[I 2022-02-02 08:09:33.178 LabApp] JupyterLab application directory is /opt/conda/share/jupyter/lab
[I 2022-02-02 08:09:33.183 ServerApp] jupyterlab | extension was successfully loaded.
[I 2022-02-02 08:09:33.188 ServerApp] jupyterlab_git | extension was successfully loaded.
[I 2022-02-02 08:09:33.188 ServerApp] [Jupytext Server Extension] Deriving a JupytextContentsManager from LargeFileManager
[I 2022-02-02 08:09:33.190 ServerApp] jupytext | extension was successfully loaded.
[I 2022-02-02 08:09:33.191 ServerApp] [nb_conda] enabled
[I 2022-02-02 08:09:33.191 ServerApp] nb_conda | extension was successfully loaded.
[D 2022-02-02 08:09:33.253 ServerApp] Using default logger
[D 2022-02-02 08:09:33.254 ServerApp] Using default logger
[D 2022-02-02 08:09:33.254 ServerApp] Using default logger
[D 2022-02-02 08:09:33.254 ServerApp] Using default logger
[D 2022-02-02 08:09:33.254 ServerApp] Using default logger
[D 2022-02-02 08:09:33.254 ServerApp] Using default logger
[I 2022-02-02 08:09:33.257 ServerApp] nbdime | extension was successfully loaded.
[I 2022-02-02 08:09:33.258 ServerApp] Serving notebooks from local directory: /home/gogasca
[I 2022-02-02 08:09:33.258 ServerApp] Jupyter Server 1.13.1 is running at:
[I 2022-02-02 08:09:33.258 ServerApp] http://localhost:8888/lab?token=0be64d1115747cdd0726876d15ece5ba15c4032127b508f1
[I 2022-02-02 08:09:33.258 ServerApp] or http://127.0.0.1:8888/lab?token=0be64d1115747cdd0726876d15ece5ba15c4032127b508f1
[I 2022-02-02 08:09:33.258 ServerApp] Use Control-C to stop this server and shut down all kernels (twice to skip confirmation).
[W 2022-02-02 08:09:33.263 ServerApp] No web browser found: could not locate runnable browser.
[C 2022-02-02 08:09:33.263 ServerApp]
To access the server, open this file in a browser:
file:///home/gogasca/.local/share/jupyter/runtime/jpserver-4572-open.html
Or copy and paste one of these URLs:
http://localhost:8888/lab?token=0be64d1115747cdd0726876d15ece5ba15c4032127b508f1
or http://127.0.0.1:8888/lab?token=0be64d1115747cdd0726876d15ece5ba15c4032127b508f1
You can enforce failure on configuration errors by configuring environment variable TRAITLETS_APPLICATION_RAISE_CONFIG_FILE_ERROR to 1 or true (see docs); on Linux this can be done by prepending the startup command like so:
TRAITLETS_APPLICATION_RAISE_CONFIG_FILE_ERROR=1 jupyter lab
If you would like it to become a new default, there is a feature request marked as good first issue here: https://github.com/jupyterlab/jupyterlab/issues/5716

Use notebook with slurm

I want to be able to connect to our institution's cluster using VS Code Remote SSH without the server running on a compute node instead of the login node. The preferred workflow is to SSH into the login node and then use a command to allocate a job and spin up an interactive shell on a compute node, and then run the jupyter notebook kernel on those node.
I launch jupyter-lab through this bash script
#!/bin/bash
#SBATCH -J jupyterTest
#SBATCH -N 1
#SBATCH --mem=16GB
#SBATCH --time=5:00:00
#SBATCH --output=/home/adufour/work/jupyter.log
#Load necessary modules
module purge
source /home/adufour/.bashrc
conda activate singlecell
#Go to the folder you wanna run jupyter in
cd ~/work
#Start the notebook
jupyter lab --ip=0.0.0.0 --port=8888
Which give me the following log :
[I 2021-02-16 16:14:29.363 ServerApp] jupyterlab | extension was successfully linked.
[I 2021-02-16 16:14:31.551 ServerApp] nbclassic | extension was successfully linked.
[I 2021-02-16 16:14:31.690 LabApp] JupyterLab extension loaded from /work/adufour/anaconda3/envs/singlecell/lib/python3.9/site-packages/jupyterlab
[I 2021-02-16 16:14:31.691 LabApp] JupyterLab application directory is /work/adufour/anaconda3/envs/singlecell/share/jupyter/lab
[I 2021-02-16 16:14:31.698 ServerApp] jupyterlab | extension was successfully loaded.
[I 2021-02-16 16:14:31.714 ServerApp] nbclassic | extension was successfully loaded.
[I 2021-02-16 16:14:31.719 ServerApp] Serving notebooks from local directory: /work/adufour
[I 2021-02-16 16:14:31.719 ServerApp] Jupyter Server 1.3.0 is running at:
[I 2021-02-16 16:14:31.719 ServerApp] http://node126:8888/lab?token=6a1178ff82901e42559423100b13ba12105935432fb0efdd
[I 2021-02-16 16:14:31.719 ServerApp] or http://127.0.0.1:8888/lab?token=6a1178ff82901e42559423100b13ba12105935432fb0efdd
[I 2021-02-16 16:14:31.719 ServerApp] Use Control-C to stop this server and shut down all kernels (twice to skip confirmation).
[C 2021-02-16 16:14:31.744 ServerApp]
To access the server, open this file in a browser:
file:///home/adufour/.local/share/jupyter/runtime/jpserver-108627-open.html
Or copy and paste one of these URLs:
http://node126:8888/lab?token=6a1178ff82901e42559423100b13ba12105935432fb0efdd
or http://127.0.0.1:8888/lab?token=6a1178ff82901e42559423100b13ba12105935432fb0efdd
Unable to connect to VS Code server.
Error in request

Opening Jupyterhub url on kubernetes is so slow(gets js, icon)

I deployed jupyterhub on kubernetes using helm.
and I can login with ID 'admin'
but when I first login, the url doesn't respond or respond after 30~50 seconds later, it seems it fails to get the javascript file or icon.
When I refresh it, it works then.
Is there any problem with the network in my kubernetes cluster?
I'm using GlusterFS Storage Class for Dynamic Provisioning.
This is my config file when install jupyterhub using helm.
proxy:
secretToken: "34999170ac41826f956ee1a757b53ff91ce6efabc3dfe24fcee863955efcc6b9"
The pod's log is like this(with user qqqqq)
[I 2020-12-23 05:22:21.664 SingleUserNotebookApp extension:158] JupyterLab extension loaded from /opt/conda/lib/python3.7/site-packages/jupyterlab
[I 2020-12-23 05:22:21.665 SingleUserNotebookApp extension:159] JupyterLab application directory is /opt/conda/share/jupyter/lab
[I 2020-12-23 05:22:22.015 SingleUserNotebookApp singleuser:561] Starting jupyterhub-singleuser server version 1.1.0
[I 2020-12-23 05:22:22.022 SingleUserNotebookApp notebookapp:1924] Serving notebooks from local directory: /home/jovyan
[I 2020-12-23 05:22:22.022 SingleUserNotebookApp notebookapp:1924] The Jupyter Notebook is running at:
[I 2020-12-23 05:22:22.022 SingleUserNotebookApp notebookapp:1924] http://jupyter-qqqqq:8888/user/qqqqq/
[I 2020-12-23 05:22:22.022 SingleUserNotebookApp notebookapp:1925] Use Control-C to stop this server and shut down all kernels (twice to skip confirmation).
[I 2020-12-23 05:22:22.038 SingleUserNotebookApp singleuser:542] Updating Hub with activity every 300 seconds
[I 2020-12-23 05:22:25.096 SingleUserNotebookApp log:174] 302 GET /user/qqqqq/ -> /user/qqqqq/tree? (#10.233.79.154) 0.93ms
[I 2020-12-23 05:22:25.165 SingleUserNotebookApp log:174] 302 GET /user/qqqqq/ -> /user/qqqqq/tree? (#10.233.93.0) 0.76ms
[I 2020-12-23 05:22:25.185 SingleUserNotebookApp log:174] 302 GET /user/qqqqq/tree? -> /hub/api/oauth2/authorize?client_id=jupyterhub-user-qqqqq&redirect_uri=%2Fuser%2Fqqqqq%2Foauth_callback&response_type=code&state=[secret] (#10.233.93.0) 2.31ms
[I 2020-12-23 05:22:25.561 SingleUserNotebookApp auth:981] Logged-in user {'kind': 'user', 'name': 'qqqqq', 'admin': False, 'groups': [], 'server': '/user/qqqqq/', 'pending': None, 'created': '2020-12-23T05:22:16.257525Z', 'last_activity': '2020-12-23T05:22:25.524384Z', 'servers': None}
[I 2020-12-23 05:22:25.562 SingleUserNotebookApp log:174] 302 GET /user/qqqqq/oauth_callback?code=[secret]&state=[secret] -> /user/qqqqq/tree? (#10.233.93.0) 250.52ms
[I 2020-12-23 05:22:25.654 SingleUserNotebookApp log:174] 200 GET /user/qqqqq/tree? (qqqqq#10.233.93.0) 71.92ms
GET //usr/qqqqq/tree? I'm getting stuck in here.
Thanks for any advice!
try looking for events of hub pod and also for user pods, it must be either taking time in allocating pvc, pulling image (for first time only if not present in machine) or setting up route in jhub. Also check whether you've added any postHook in user images config.
I don't know why but, after I changed kubernetes version from 1.16 to 1.17, it works fine.

Pyspark kernel doesnt start on Jupyterhub

The Jupyterhub Pyspark kernel used to work very well but now will not start or will have Kernel connected status (doesnt go to idle) but will not run any code in cells. It is using localprocess spawner with PAM auth on Centos.In the jupyterhub logs we see these messages:
[I 2020-03-10 15:13:08.644 SingleUserNotebookApp restarter:110] KernelRestarter: restarting kernel (4/5), new random ports
Assertion failed: rc == 0 (src/socket_poller.cpp:41)
[W 2020-03-10 15:13:11.666 SingleUserNotebookApp restarter:100] KernelRestarter: restart failed
[W 2020-03-10 15:13:11.666 SingleUserNotebookApp kernelmanager:127] Kernel 0148c77d-143a-4721-90e7-0d9a41a878c4 died, removing from map.
Any thoughts?
This was resolved by reinstalling sparkmagic in the base conda environment:
(base) [root#]# pip install sparkmagic

Is it possible to use Jupyter Notebook in Google Cloud Shell?

What I have tried:
Start Google Cloud shell
sudo pip install jupyter
jupyter notebook --generate-config
Add the following to ~/.jupyter/jupyter_notebook_config.py
c.NotebookApp.ip = 'localhost'
c.NotebookApp.open_browser = False
c.NotebookApp.port = 8080
c.NotebookApp.allow_origin = '*'
c.ConnectionFileMixin.control_port = 8081
c.ConnectionFileMixin.hb_port = 8082
c.ConnectionFileMixin.iopub_port = 8083
c.ConnectionFileMixin.shell_port = 8084
c.ConnectionFileMixin.stdin_port = 0
Run
jupyter notebook --no-browser --debug
Go to web preview, the home page loads just fine, but when I load a new notebook .ipynb, I get:
A connection to the notebook server could not be established. The
notebook will continue trying to reconnect. Check your network
connection or notebook server configuration.
Error in developer console:
main.min.js:22712 WebSocket connection to
'wss://8080-dot-2103947-dot-devshell.appspot.com/api/kernels/05832e24-2cd6-421c-aa55-16c58eceab94/channels?session_id=83EE7CD1C17942BE827623E5DFF08240'
failed: Error during WebSocket handshake: Unexpected response code:
400
But in Google Cloud Shell, there is no 400 error:
[D 19:36:27.820 NotebookApp] Searching [u'/home/jie', '/home/jie/.jupyter', '/usr/etc/jupyter', '/usr/local/etc/jupyter', '/etc/jupyter'] for config files
[D 19:36:27.821 NotebookApp] Looking for jupyter_config in /etc/jupyter
[D 19:36:27.822 NotebookApp] Looking for jupyter_config in /usr/local/etc/jupyter
[D 19:36:27.861 NotebookApp] Looking for jupyter_config in /usr/etc/jupyter
[D 19:36:27.862 NotebookApp] Looking for jupyter_config in /home/jie/.jupyter
[D 19:36:27.863 NotebookApp] Looking for jupyter_config in /home/jie
[D 19:36:27.865 NotebookApp] Looking for jupyter_notebook_config in /etc/jupyter
[D 19:36:27.866 NotebookApp] Looking for jupyter_notebook_config in /usr/local/etc/jupyter
[D 19:36:27.866 NotebookApp] Looking for jupyter_notebook_config in /usr/etc/jupyter
[D 19:36:27.867 NotebookApp] Looking for jupyter_notebook_config in /home/jie/.jupyter
[D 19:36:27.868 NotebookApp] Loaded config file: /home/jie/.jupyter/jupyter_notebook_config.py
[D 19:36:27.869 NotebookApp] Looking for jupyter_notebook_config in /home/jie
[I 19:36:29.267 NotebookApp] Serving notebooks from local directory: /home/jie
[I 19:36:29.268 NotebookApp] 0 active kernels
[I 19:36:29.268 NotebookApp] The Jupyter Notebook is running at: http://localhost:8080/
[I 19:36:29.268 NotebookApp] Use Control-C to stop this server and shut down all kernels (twice to skip confirmation).
[I 19:36:36.858 NotebookApp] 302 GET /?authuser=0 (127.0.0.1) 1.20ms
[D 19:36:37.525 NotebookApp] Using contents: services/contents
[D 19:36:37.562 NotebookApp] 200 GET /tree (::1) 38.02ms
[D 19:36:38.268 NotebookApp] 200 GET /custom/custom.css (127.0.0.1) 1.60ms
[D 19:36:39.180 NotebookApp] 200 GET /static/services/contents.js?v=20160919193627 (::1) 1.45ms
[D 19:36:39.184 NotebookApp] 200 GET /custom/custom.js?v=20160919193627 (127.0.0.1) 1.47ms
[D 19:36:39.906 NotebookApp] 200 GET /api/config/tree?_=1474285000896 (::1) 3.61ms
[D 19:36:39.910 NotebookApp] 200 GET /api/terminals?_=1474285000900 (127.0.0.1) 1.15ms
[D 19:36:39.915 NotebookApp] 200 GET /api/config/common?_=1474285000897 (::1) 3.59ms
[D 19:36:39.918 NotebookApp] 200 GET /api/terminals?_=1474285000898 (::1) 1.07ms
[D 19:36:39.940 NotebookApp] 200 GET /api/sessions?_=1474285000899 (127.0.0.1) 20.56ms
[D 19:36:39.945 NotebookApp] Native kernel (python2) available from /usr/local/lib/python2.7/dist-packages/ipykernel/resources
[D 19:36:39.945 NotebookApp] Native kernel (python2) available from /usr/local/lib/python2.7/dist-packages/ipykernel/resources
[D 19:36:39.947 NotebookApp] 304 GET /api/kernelspecs (127.0.0.1) 5.75ms
[D 19:36:40.593 NotebookApp] 200 GET /api/contents?type=directory&_=1474285000901 (::1) 4.76ms
[D 19:37:40.160 NotebookApp] 200 GET /api/sessions?_=1474285000902 (127.0.0.1) 1.15ms
[D 19:37:40.162 NotebookApp] 200 GET /api/terminals?_=1474285000903 (::1) 0.65ms
[D 19:37:41.392 NotebookApp] 200 GET /api/contents?type=directory&_=1474285000904 (127.0.0.1) 3.10ms
[D 19:38:40.143 NotebookApp] 200 GET /api/sessions?_=1474285000905 (::1) 1.34ms
[D 19:38:40.209 NotebookApp] 200 GET /api/terminals?_=1474285000906 (127.0.0.1) 1.07ms
[D 19:38:40.766 NotebookApp] 200 GET /api/contents?type=directory&_=1474285000907 (::1) 3.33ms
[D 19:39:14.165 NotebookApp] 200 GET /api/sessions?_=1474285000908 (127.0.0.1) 1.05ms
[D 19:39:14.192 NotebookApp] 200 GET /api/terminals?_=1474285000909 (::1) 0.96ms
[D 19:39:14.761 NotebookApp] 200 GET /api/contents?type=directory&_=1474285000910 (127.0.0.1) 3.22ms
[D 19:40:40.338 NotebookApp] 200 GET /api/sessions?_=1474285000911 (::1) 1.40ms
[D 19:40:40.363 NotebookApp] 200 GET /api/terminals?_=1474285000912 (127.0.0.1) 0.94ms
[D 19:40:40.936 NotebookApp] 200 GET /api/contents?type=directory&_=1474285000913 (::1) 3.39ms
[D 19:42:13.646 NotebookApp] 200 GET /api/sessions?_=1474285000914 (127.0.0.1) 1.41ms
[D 19:42:13.649 NotebookApp] 200 GET /api/terminals?_=1474285000915 (::1) 0.77ms
[D 19:42:15.178 NotebookApp] 200 GET /api/contents?type=directory&_=1474285000916 (127.0.0.1) 5.44ms
[I 19:42:17.736 NotebookApp] Creating new notebook in
[D 19:42:17.738 NotebookApp] Saving /home/jie/Untitled1.ipynb
[D 19:42:17.851 NotebookApp] 201 POST /api/contents (::1) 117.08ms
[D 19:42:18.462 NotebookApp] Using contents: services/contents
[D 19:42:18.494 NotebookApp] 200 GET /notebooks/Untitled1.ipynb?kernel_name=python2 (127.0.0.1) 32.79ms
[D 19:42:19.275 NotebookApp] 304 GET /custom/custom.css (::1) 1.28ms
[D 19:42:19.296 NotebookApp] 200 GET /static/components/MathJax/MathJax.js?config=TeX-AMS_HTML-full,Safe&delayStartupUntil=configured (127.0.0.1) 2.40ms
[D 19:42:20.407 NotebookApp] 200 GET /static/components/MathJax/config/TeX-AMS_HTML-full.js?rev=2.6.0 (::1) 2.29ms
[D 19:42:20.609 NotebookApp] 304 GET /custom/custom.js?v=20160919193627 (127.0.0.1) 1.11ms
[D 19:42:21.246 NotebookApp] 200 GET /api/config/common?_=1474285342417 (::1) 3.12ms
[D 19:42:21.300 NotebookApp] 200 GET /api/contents/Untitled1.ipynb?type=notebook&_=1474285342418 (127.0.0.1) 44.64ms
[D 19:42:21.302 NotebookApp] Native kernel (python2) available from /usr/local/lib/python2.7/dist-packages/ipykernel/resources
[D 19:42:21.303 NotebookApp] Native kernel (python2) available from /usr/local/lib/python2.7/dist-packages/ipykernel/resources
[D 19:42:21.305 NotebookApp] 304 GET /api/kernelspecs (127.0.0.1) 2.81ms
[D 19:42:21.309 NotebookApp] 200 GET /api/config/notebook?_=1474285342416 (::1) 2.74ms
[D 19:42:22.147 NotebookApp] Native kernel (python2) available from /usr/local/lib/python2.7/dist-packages/ipykernel/resources
[D 19:42:22.149 NotebookApp] Starting kernel: ['/usr/bin/python', '-m', 'ipykernel', '-f', u'/home/jie/.local/share/jupyter/runtime/kernel-05832e24-2cd6-421c-aa55-16c58eceab94.json']
[D 19:36:37.562 NotebookApp] 200 GET /tree (::1) 38.02ms
[D 19:42:22.153 NotebookApp] Connecting to: tcp://127.0.0.1:8081
[I 19:42:22.155 NotebookApp] Kernel started: 05832e24-2cd6-421c-aa55-16c58eceab94
[D 19:42:22.155 NotebookApp] Kernel args: {'kernel_name': u'python2', 'cwd': u'/home/jie'}
[D 19:42:22.161 NotebookApp] 201 POST /api/sessions (::1) 87.33ms
[D 19:42:22.165 NotebookApp] 200 GET /api/contents/Untitled1.ipynb/checkpoints?_=1474285342419 (127.0.0.1) 1.60ms
[D 19:42:22.276 NotebookApp] Using contents: services/contents
[W 19:42:22.293 NotebookApp] 404 GET /nbextensions/widgets/notebook/js/extension.js?v=20160919193627 (::1) 19.36ms referer=https://8080-dot-2103947-dot-devshell.appspot
.com/notebooks/Untitled1.ipynb?kernel_name=python2
[D 19:42:22.350 NotebookApp] Native kernel (python2) available from /usr/local/lib/python2.7/dist-packages/ipykernel/resources
[D 19:42:22.351 NotebookApp] Serving kernel resource from: /usr/local/lib/python2.7/dist-packages/ipykernel/resources
[D 19:42:22.356 NotebookApp] 304 GET /kernelspecs/python2/logo-64x64.png (127.0.0.1) 7.08ms
[D 19:42:22.401 NotebookApp] 200 GET /static/style/style.min.css.map (::1) 1.83ms
[D 19:42:22.549 NotebookApp] 200 GET /static/components/MathJax/config/Safe.js?rev=2.6.0 (127.0.0.1) 2.08ms
[D 19:42:23.217 NotebookApp] 200 GET /static/components/MathJax/jax/output/HTML-CSS/fonts/STIX-Web/fontdata.js?rev=2.6.0 (::1) 1.41ms
[D 19:42:23.240 NotebookApp] 200 GET /static/components/MathJax/extensions/Safe.js?rev=2.6.0 (127.0.0.1) 1.25ms
[D 19:42:52.652 NotebookApp] 200 GET /api/sessions?_=1474285000917 (::1) 3.00ms
[D 19:42:52.654 NotebookApp] 200 GET /api/terminals?_=1474285000918 (127.0.0.1) 0.65ms
[D 19:42:53.314 NotebookApp] 200 GET /api/contents?type=directory&_=1474285000919 (::1) 4.03ms
[D 19:44:22.156 NotebookApp] 200 GET /api/contents/Untitled1.ipynb?content=0&_=1474285342420 (127.0.0.1) 1.36ms
[I 19:44:22.762 NotebookApp] Saving file at /Untitled1.ipynb
[D 19:44:22.763 NotebookApp] Saving /home/jie/Untitled1.ipynb
[D 19:44:22.826 NotebookApp] 200 PUT /api/contents/Untitled1.ipynb (::1) 64.32ms
So that leads me to think the web socket requests didn't reach the kernel, they were blocked by Google Cloud Shell. Google App Engine does not provide Socket API for free, is this the reason why I can't get it to work?
Yes. Google Cloud Shell is not meant to stand up a web socket service and is not configured for that. In addition, there are service limits that you'll likely hit if you try to do this, check the Usage Limits and Outgoing Connections sections here
Check out Google Compute Engine and Google Container Engine for this purpose, both are not free past the trial though.