Connect locally to Jupyter in Sagemaker - jupyter

When I run %connect_info in Jupyterlab on Sagemaker I get session info. and
{"shell_port": ,
"iopub_port": ,
"stdin_port": ,
"control_port": ,
"hb_port": ,
"ip": "",
"key": "",
"transport": "",
"signature_scheme": "",
"kernel_name": ""}
Paste the above JSON into a file, and connect with:
$> jupyter <app> --existing <file>
or, if you are local, you can connect with just:
$> jupyter <app> --existing kernel-052ed888-e682-4786-aa4c-cdb19c6145bf.json
or even just:
$> jupyter <app> --existing
if this is the most recent Jupyter kernel you have started.
But when I run the Jupiter statement against the saved text file, it doesn't connect.
Sometimes, it would be convenient to connect my local VS code app to my notebooks instead of developing in the browser.
Any suggestions are appreciated.

the recommended way to build locally on SageMaker is using the SageMaker Python SDK. See an example here - https://aws.amazon.com/blogs/machine-learning/run-your-tensorflow-job-on-amazon-sagemaker-with-a-pycharm-ide/

Related

How to create a local terminal profile?

In Visual Studio Code's settings, you can easily add custom terminal profiles like this:
"terminal.integrated.profiles.linux": {
"test": {
"path": "bash",
"icon": "terminal-bash",
}
}
This will open a terminal host for the host on which the vscode server is running (local, devcontainer, remote over ssh, etc).
With that said, VS Code is still capable of opening a local terminal on the host regardless of where it the server is running with the workbench.action.terminal.newLocal command:
Is it possible to write a terminal profile to open a local terminal from the usual UI?

Use VSCode explorer with remote server (using SSH)

I need to code / edit files in a remote server using SSH, and I would like to access it with VSCode.
I'm on Windows 10, using "Git Bash" as integrated terminal in VSCode, which means I can connect to the server using VSCode's terminal.
What I'm missing is a way to open files from the terminal to the editor, and even better - interacting with the files using the explorer.
How can this be done?
Since the May, 2nd 2019 announcement of "Remote Development with VS Code", you now officially have:
Visual Studio Code Remote - SSH
The Remote - SSH extension lets you use any remote machine with a SSH server as your development environment.
Since nearly every desktop and server operating system has a SSH server that can be configured, the extension can greatly simplify development and troubleshooting in a wide variety of situations.
You can:
Develop on the same operating system you deploy to or use larger, faster, or more specialized hardware than your local machine.
Quickly swap between different, remote development environments and safely make updates without worrying about impacting your local machine.
Access an existing development environment from multiple machines or locations.
Debug an application running somewhere else such as a customer site or in the cloud.
Q1 2020: VSCode 1.42 improves support for Windows servers, including automatic OS detection.
Is this what you are looking for?
https://marketplace.visualstudio.com/items?itemName=humy2833.ftp-simple#overview
You need to setup sFtp connection to your server.
Install extensions Code Runner and SSH-FS. Add config into your user setting like this:
"code-runner.runInTerminal":true,
"code-runner.fileDirectoryAsCwd": true,
"code-runner.ignoreSelection": true,
"code-runner.saveFileBeforeRun": true,
"files.eol": "\n",
"sshfs.configs": [
{
"label": "label",
//Must use the root direction "/"
"root": "/",
"host": "host",
"port": port,
"username": "name",
"password": "password"
"name": "name"
}]
Login your server account over ssh by the vscode terminal. Then you can edit and run your code on remote server.
Try disabling it and re-enabling it. It works for me.
I was able to log in via putty and ssh from terminal, so it wasn't that. It's just the configuration gets hung up or something.
All of the previous answers require installing a package from the VS Code Marketplace. Here is a solution, that won't even require you to install VS Code.
The Web Based VS Code Editor: Run VS Code on any machine anywhere and access it in the browser.
All you need to do is, first install code-server using:
$ curl -fsSL https://code-server.dev/install.sh | sh
Start code-server using:
$ code-server
After running this you can log on to https://127.0.0.1:8080 and view the web-based VS code.
To keep the service running in the background, use:
$ sudo systemctl restart code-server#$USER
You might also want to edit the configurations, for e.g., run it on 0.0.0.0:8080 instead of 127.0.0.1:8080 or to change the password - use:
$ nano ~/.config/code-server/config.yaml
I have used nano as my text editor, feel free to use your preferred text editor.
For in-depth setup and configuration guide: Setup Guide
Original GitHub repo for more info: code-server

Visual Studio Code - unable to attach debugger to remote Node.js application

here is my debug configuration :
{
"type": "node",
"request": "attach",
"name": "testServer",
"address": "test.server.ip",
"port": 5858,
"localRoot": "${workspaceRoot}/test/server",
"remoteRoot": "~/App/test/server"
}
I have started remote application in debug mode successfully by using below command
node --debug app
Then I start VS Code debugger using testServer configuration. it print error:
Debugging with legacy protocol because Node.js version could not be determined (Error: timeout)
I am using VS Code version 1.16.1 on macOS sierra.
I guess it's not able to connect remote server because it's secured by SSH. But I can't see any configuration related to SSH in VS Code debugger's configuration.
I have already gone through some articles and issues like this
andthis, but no help.
Thanks for any help.
You'll need to start the remote application, and tell node to expose the port remotely with the following command:
node app --inspect=0.0.0.0:5858
I got it working like that on Windows :
Create a new putty session with hostname, your remote server, and go to "Tunnels" under the SSH dropdown, then configure it like it :
Run node debug on your remote host and copy the port it give to you (For me 9229)
Run node debug
Then fill Putty like that :
Fill putty tunnel
You can now save this session, and then open it.
Now, every time you will open this SSH session, everything happening on your remote server on port 9229 will be redirected to your local 9229 port.
In VSCode, the config is now really simple, because it's like you are on local :
{
"type": "node",
"request": "attach",
"name": "Attach",
"port": 9229
},
Hope it helped

Juypter notebook - connect to the same kernel session from a command line ipython?

Suppose I've a Jupyter notebook loaded in a browser. Is that possible to somehow run a command line ipython session which connect to the same kernel insance as the one used by the notebook (i.e they can see the same set of variables)? Thanks.
Yes! After starting/loading an IPython notebook, open up a terminal and connect a command line ipython session using the --existing parameter:
jupyter console --existing
By default, it will connect to the latest started IPython kernel. To select a different kernel to connect to, look in the log of the Jupyter notebook for lines like these:
[I 09:47:54.462 NotebookApp] Kernel started: 06c9ffae-ae9f-4c22-93c0-4eacf23672b1
To connect to this kernel, do:
jupyter console --existing 06c9ffae-ae9f-4c22-93c0-4eacf23672b1
With JupyterLab it's possible to open a linked console by right clicking on any Notebook and selecting New Console for Notebook (as mentioned in #machine_building's comment on this answer). That will show a connected console beside/below the Notebook, which one can interact with like in a terminal with command history etc.

Use Jupyter Notebook on my local computer to run code on a remote computer

I use Jupyter Notebook to run bioinformatic analyses, and I love it. However, it only really plays nice when I run it on my personal computer. However, I regularly do analysis using a remote computer with multiple cores to reduce processing time. I'd like to be able to use the Jupyter Notebook interface on my personal computer while everything is actually running on the remote computer. I generally do this via ssh access to the remote computer within the shell and execute all commands at the command line. I'd love to do this from the Jupyter notebook on my personal computer, rather than from the shell on my personal computer. It is relevant that I don't have sudo access on the remote computer.
So far, I've installed miniconda and jupyter notebook on the remote computer like this:
wget https://repo.continuum.io/miniconda/Miniconda-latest-Linux-x86_64.sh
bash Miniconda3-latest-Linux-x86_64.sh
Once conda is installed properly, I install jupyter notebook via miniconda with this line:
conda install jupyter
This installs successfully. I can then start a jupyter notebook session on the remote machine with the line:
jupyter notebook --no-browser
So far, so good. My next question: How do I have my local jupyter notebook connect to the remote machine, so that I can execute commands on the remote machine using my local jupyter notebook? There is some documentation here, however i have been trying different things for hours, but have failed to succeed.
Can anyone give a straight forward method to connect to my remote server, given that I am this far along? I feel like it should just be a matter of entering url addresses and passwords into my local Jupyter notebook (all of this is so easy via ssh in the shell).
Follow the steps below:
Enable port forwarding on remote machine
ssh -N -f -L 127.0.0.1:8898:127.0.0.1:8898 user#remote-machine.com
Do ssh to your remote machine and then run following command on remote machine
jupyter-notebook --no-browser --port=8898
you will see some thing as shown below
Copy/paste this URL into your browser when you connect for the first time,
to login with a token:
http://localhost:8898/token=eaf2f51f9c053f43d8bd093e76f0cc6301b545549c998fa2&token=eaf2f51f9c053f43d8bd093e76f0cc6301b545549c998fa2
Copy and paste the URL in your local machine browser.
If you want to access Jupyter/Ipython notebook running on a VPS remotely, I wrote a tutorial on the digital ocean community site.
As shown in the guide, after installing and running Ipython Notebook using command line on the server, you can connect to the notebook using SSH tunnelling with Putty (on windows) or the ssh -L command on Unix-like systems (ie Mac and Linux)