I use vscode on a remote server using the Remote Explorer tool. I am able to use the interactive window but the kernel runs by default on the login node that I connect to. In a terminal I can connect to an interactive compute node using qlogin -q short.qc. How can I make the interactive kernel use a compute node as opposed to the login node?
Related
I am doing some analyses using vscode on a remote server that has got SLURM installed to manage jobs and provide parallel computing. I would like to run each cell in the Jupyter notebook as an interactive job on SLURM the same way my command line code would be run as an interactive SLURM job after I have used srun to request compute nodes. The jobs I need to run on the Jupyter notebook require a lot of memory, so I need to run them using SLURM.
My current work around is to run srun on the terminal and start a python terminal, then I copy and paste the code from each cell of my notebook into the python terminal. I'd really appreciate your help.
It is an old question, but answering as I also came across this problem recentrly.
After you do srun on a terminal, you should be able to ssh directly into your compute node in VScode and use all the capabilities of the compute node in the interactive mode/notebook
The steps I take, for example, are:
in a terminal (e.g. powershell), srun into a node
add that node to your config file, so that you can ssh into it
open vscode and ssh into that node
run code in interactive window/notebook, with access to CPU/GPU of the node
I am using the VSCode extension Remote-SSH to connect to a remote server.
However, on this server, I can only login via the login node. I'm not allowed to run heavy tasks on this node, so I need to enter voltash to get transferred to a usable node.
I can do this by opening a new terminal. However, the current node remains unchanged on Remote-SSH:
Even though the terminal window says I am on n-62-27-20 (the name of the node I'm currently on, in the terminal), the Remote-SSH status bar (bottom left) still is on the login node.
My question is: is there a way to enter this voltash command inside the Remote-SSH extension?
I found this: https://stackoverflow.com/questions/57780250/does-vscode-remote-support-double-ssh however, I can't enter a server IP as I need to use a command (voltash).
Problem
I am trying to connect to my school's computing cluster (aka a linux server with "login node" and "computing node") using VS Code's Remote SSH, but I cannot figure out how to run a command after SSH-ing.
Goal
I simply want to view Python code and test some small lines in a .ipynb jupyter notebook in the computing platform's environment.
Description
Basically, normally in the command line (or mobaXterm of a Windows machine) of my local machine, I first log onto the computing platform's login node with ssh -Y -L PORT:127.0.0.1:PORT username#computing.cluster.ip, and then run srun -t 0-12:00 --pty -p gpu --gres=gpu:1 --x11 --tunnel PORT:PORT /bin/bash to log onto the computing node interactively (shown command allows for port forwarding). The problem is, in VS Code I can only connect to the login node, but after that there's no way for me to run another command and log onto the computing node. The reason I need to get to computing node is that I want to test something with a .ipynb file interactively on VS Code while reading the code, and the login node does not allow me to perform computation.
Failed trials
I've been trying Code-Server, but it does not support .ipynb well (it keeps asking me to install jupyter notebook even though I have installed it in my conda env), possibly because it by default recognizes HPC cluster's Python interpreter which I cannot modify (I can't even select Jupyter kernel in code-server). I also tried to directly use Jupyter Notebook (open Jupyter with port forwarding after getting onto computing node), but reading code on it is much more inconvenient.
Would greatly appreciate your suggestions!
I use Google Cloud Shell to run Jupyter Notebook on instance , but after 1 hours,the connection is disconnect.
And I reconnect to instance, there is nothing, is same as reset instance.
So, please tell me how to set up ,make the connection of cloud shell
keep connect.
Thanks!
Google Cloud Shell auto disconnect because you didn't do anything in the Cloudshell's terminal! So to keep the cloudshell session alive, you can enter the following javascript code to the browser's javascript console
setInterval(function() {document.elementFromPoint(500, 500).click();}, 30000);
That's the trick!
P/S: If you want to use Jupyter Notebook, why you don't use Google Colab instead? It has a pre-installed Jupyter Notebook as well as more CPU and ram. You can use the same js for Colab!
UPDATE: you can try Cloud Shell SDK, then connect to Google Cloud Shell by gcloud cloud-shell ssh, which is much faster than the web interface, allow port forwarding (based on ssh), and have a longer time limit.
You can even connect to it using VSCode.
The virtual machine instance that backs your Cloud Shell session is
not permanently allocated to a Cloud Shell session and terminates if
the session is inactive for an hour.
https://cloud.google.com/shell/docs/limitations
I am late to answer this but try this if not already. Open the sysctl.conf and add below lines.
grep keep /etc/sysctl.conf
sudo vi /etc/sysctl.conf
net.ipv4.tcp_keepalive_time=60
net.ipv4.tcp_keepalive_intvl=60
net.ipv4.tcp_keepalive_probes=5
stop and start your instance.
Answer from a year later
You can use gcloud cli. Just install it, open a terminal and run gcloud cloud-shell ssh --authorize-session, then you can use the terminal.
You can also ssh from VSCode to Google Cloud Shell. Just set the key file to .ssh/google_compute_engine, get the ip of CloudShell machine by curl -L ipconfig.me, and then ssh to that ip with port 6000.
I am using a windows machine and I have several running sessions of the jupyter notebook. Each session display the content in different ports. I would like to know if there is a way of getting a report including the different sessions I am running and the different ports on which they are displaying the content. I could not find that in the documentation.
Usually what you can do is pass the --help flag to the jupyter notebook command and read the informations in particular here
$ jupyter notebook --help
The Jupyter HTML Notebook.
This launches a Tornado based HTML Notebook Server that serves up an
HTML5/Javascript Notebook client.
Subcommands
-----------
Subcommands are launched as `jupyter-notebook cmd [args]`. For information on
using subcommand 'cmd', do: `jupyter-notebook cmd -h`.
list
List currently running notebook servers.
stop
Stop currently running notebook server for a given port
password
Set a password for the notebook server.
Options
-------
...
Oh ! A list sub command. let's try it:
$ jupyter notebook list
Currently running servers:
http://localhost:8888/ :: /Users/myuser/dev/ipython-website
http://localhost:8889/ :: /Users/myuser
.. well that's on macOS, but you should get the same on Windows.
One of the questions is why do you have multiple sessions as the notebok can browse the filesystem. It is likely confusing.
Enjoy.