I need to run some code that takes a lot of time, so I did the following:
use ssh to connect to my server, use screen to start jupypter notebook server without web browser.
use the following to redirect localhost:portOnMyMachine to port 8888 on my server:
myName#serverAddress -NL portOnMyMachine:localhost:8888 ubuntu#DNSofMyServer
Open my browser with localhost:portOnMyMachine
And I could edit notebooks on my browser. Every time I need to edit my notebook, I just need to repeat 2 and 3.
However, if I run some large program, I cannot see which line the kernel is working on when I log in again. There is no "" symbol after the line number anymore. I guess it is still running something, because it will put a '' after any new line I want to execute.
So I was wondering if anybody could tell me how to re-open a running notebook and have it in the status before I close it?
Related
Problem
I am trying to connect to my school's computing cluster (aka a linux server with "login node" and "computing node") using VS Code's Remote SSH, but I cannot figure out how to run a command after SSH-ing.
Goal
I simply want to view Python code and test some small lines in a .ipynb jupyter notebook in the computing platform's environment.
Description
Basically, normally in the command line (or mobaXterm of a Windows machine) of my local machine, I first log onto the computing platform's login node with ssh -Y -L PORT:127.0.0.1:PORT username#computing.cluster.ip, and then run srun -t 0-12:00 --pty -p gpu --gres=gpu:1 --x11 --tunnel PORT:PORT /bin/bash to log onto the computing node interactively (shown command allows for port forwarding). The problem is, in VS Code I can only connect to the login node, but after that there's no way for me to run another command and log onto the computing node. The reason I need to get to computing node is that I want to test something with a .ipynb file interactively on VS Code while reading the code, and the login node does not allow me to perform computation.
Failed trials
I've been trying Code-Server, but it does not support .ipynb well (it keeps asking me to install jupyter notebook even though I have installed it in my conda env), possibly because it by default recognizes HPC cluster's Python interpreter which I cannot modify (I can't even select Jupyter kernel in code-server). I also tried to directly use Jupyter Notebook (open Jupyter with port forwarding after getting onto computing node), but reading code on it is much more inconvenient.
Would greatly appreciate your suggestions!
I am mew to loopback . I am running the API in localhost . I created the model and I am able to access it to localhost . But the problem is I can not type anythings in command prompt windows after running the applications ..
Here is the screen shot of the command prompt .
When you start your server you can't type anything on that console.
If you want to use console you have to either stop your server or you have to use another console.
If you want to restart your server then you have to halt it using Ctrl + c command and after that npm start command.
If you want to use the same console for more commands, then open a new console and run your node server over there.
And if you want to clear your warning of "includeSubdomains" then follow these steps :
1)Go to server/middleware.json
2)search for "includeSubdomains" key and replace with "includeSubDomains".
I've done my development of play framework,and I have dist it to be a binary version uploaded to the server.
So,my problem is everytime i open it on ssh , it will shutdown when i logout , how to make it running until i shut it down manually?
You can use screen. It will start a new session that will stay open even if you disconnect from the server. You can then, on the next ssh, reconnect with screen -r -D. There are a ton of useful features such as creating multiple "screens" with ctrl+a and then c (see e.g. screen intro on digitalocean for more) so you can continue working on the server while the app is running
add nohup to your launch command to ensure it is not terminated when you exit your shell.
nohup exec /path/to/my/app/bin/myapp -Dhttp.port=8080
I have a notebook running for some hours, I came back in the morning and at some point the browser crashed. I can see that the kernel is still running and computing.
If I open the browser and open the notebook again, it seems as it opens the file, but does not really connect to the existing running kernel. I judge this by the fact that the notebook does not show the "kernel busy" badge on the top.
What I really would like to do is, is to connect to the running kernel with a qtconsole, interrupt the process, go into debug mode and save the important variables, shut down the kernel properly and continue the process with the data I saved.
Now is there a way of connecting to a running kernel with a qtconsole?
For future reference:
Alternatively one could also wait until the process finished the computation and then hook into the session and save the results (but this works only if the kernel is idle, hence the question is still open):
Just copy the hash from the terminal that started the ipyhton notebook and then link into the session with:
ipython console --existing 08906ba1-7942-4309-9955-712efdeb09f9
or
ipython qtconsole --existing 08906ba1-7942-4309-9955-712efdeb09f9
(08906ba1-7942-4309-9955-712efdeb09f9 being the hash)
I am taking my first steps with ipython notebook and I installed it successfully on a remote server of mine (over SSH) and I started it using the following command:
ipython notebook --ip='*' ---pylab=inline --port=7777
I then checked on http://myserver.sth:7777/ and the notebook was running just fine. I then wanted to close the SSH connection with the server and keep ipython running in the background. When I did this, I couldn't connect to myserver.sth:7777 anymore. Once I connected again to the remote server by SSH, I could connect again to the notebook. I then tried to use screen to start ipython: I created a new screen by screen -S ipy, I started ipython notebook as above and I used Ctrl+A,D to detach the screen and exit to the TTY. I could still connect remotely to the notebook. I then closed the SSH connection and I got a 404 NOT FOUND error when I tried to access my previously stored notebook and I couldn't see it on the list of notebook at http://myserver.sth:7777/. I tried to create a new notebook, but I got a 500 Internal Server Error.
I also tried running ipython notebook with and without using sudo.
Any ideas?
Rather than use screen, perhaps you could switch to an init script or supervisord to keep IPython notebook up and running.
Let's assume you go the supervisord route:
Install supervisord
Install supervisord using your package manager. For ubuntu it's named supervisor.
apt-get install supervisor
If you decide to install supervisor through pip, you'll have to set up its init.d script yourself.
Write a supervisor configuration file for IPython
The configuration file tells supervisor what to run and how.
After you install supervisor, it should have created /etc/supervisor/supervisord.conf. These lines should exist in the file:
[include]
files = /etc/supervisor/conf.d/*.conf
If they contain these lines, you're in good shape. I only show them to demonstrate where it expects new configuration files. Your configuration file can go there, named something like /etc/supervisor/conf.d/ipynb.conf.
Here's a sample configuration that was generated by Chef by an ipython-notebook-cookbook that runs the notebook in a virtualenv:
[program:ipynb]
command=/home/ipynb/.ipyvirt/bin/ipython notebook --profile=cooked
process_name=%(program_name)s
numprocs=1
numprocs_start=0
autostart=true
autorestart=true
startsecs=1
startretries=3
exitcodes=0,2
stopsignal=QUIT
stopwaitsecs=10
user=ipynb
redirect_stderr=false
stdout_logfile=AUTO
stdout_logfile_maxbytes=50MB
stdout_logfile_backups=10
stdout_capture_maxbytes=0
stdout_events_enabled=false
stderr_logfile=AUTO
stderr_logfile_maxbytes=50MB
stderr_logfile_backups=10
stderr_capture_maxbytes=0
stderr_events_enabled=false
environment=HOME="/home/ipynb",SHELL="/bin/bash",USER="ipynb",PATH="/home/ipynb/.ipyvirt/bin:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games",VIRTUAL_ENV="/home/ipynb/.ipyvirt"
directory=/home/ipynb
serverurl=AUTO
The above supervisor config also relies on an IPython notebook configuration (located at /home/ipynb/.ipython/profile_cooked/ipython_notebook_config.py). This makes configuration much easier (as you can also set up your password hash and many other configurables).:
c = get_config()
# Kernel config
# Make matplotlib plots inline
c.IPKernelApp.pylab = 'inline'
# The IP address the notebook server will listen on.
# If set to '*', will listen on all interfaces.
# c.NotebookApp.ip= '127.0.0.1'
c.NotebookApp.ip='*'
# Port to host on (e.g. 8888, the default)
c.NotebookApp.port = 8888 # If you want it on 80, I recommend iptables rules
# Open browser (probably want False)
c.NotebookApp.open_browser = False
Re-read and update, now that you have the configuration file
supervisorctl reread
supervisorctl update
Reality
In reality, I used to use a Chef cookbook to do the entire installation and configuration. However, using configuration management with tiny stuff like this is a bit of overkill (unless you're orchestrating these in automation).
Nowadays I use Docker images for IPython notebook, orchestrating via JupyterHub or tmpnb.