How to run Jupyter Notebook on server - server

I've used Jupyter Notebook before on my local computer as well as Jupyter Hub, but now I'm trying to install it on my Ubuntu server so that I can access it from everywhere, but I can't make it work.
I tried using SSH tunneling, but then I don't get the right permissions, and I don't want to make my share open globaly.
I did it like this on the server:
jupyter notebook --no-browser --port=8889
On the local machine I used:
ssh -f user#server -L 8889:localhost:8889 -N
When I create a new Notebook it says:
Unexpected error while saving file: Untitled.ipynb [Errno 13] Permission denied: '/home/user/.local/share'
It does work when I run the Jupyter Notebook as root with sudo, but that's not a recommend way to start Jupyter Notebook.
How can I fix this?

It looks like jupyter is fine but you do not have the rights to write in the folder where you want to write in the terminal:
chmod 666 /home/user/.local/share
or
sudo chmod 666 /home/user/.local/share
/home/user/.local/share is where you are saving
chmod 666 will grant you right to write and read there. You could select instead of 666 other combinations, dependind if you want to give right to write to read and so on. For further info on that http://www.thinkplexx.com/learn/article/unix/command/chmod-permissions-flags-explained-600-0600-700-777-100-etc.
sudo because you may need to have the right to give the rights.

As mentioned in previous answer, it is permissions issue: you can try
sudo chmod 777 /home/user/.local/share

Related

Error: EACCES: permission denied, only in VSCode using Remote SSH

I can find plenty of references to this error, but they all point to permission issues, however my permissions appear to be fine as I can modify this folder using nano and SSH.
I'm trying to use RemoteSSH with a custom user account, user
The server is running Debian 11 and nginx
I have key based auth and I connect to the server. When I try to create or remove a file in the web folder (/var/www/html) I get this error message Error: EACCES: permission denied, <what I was trying to do, i.e. rename a file, or delete a file>
I can do all of these things using a standard SSH connection (openSSH built in to Windows 10)
The owner of /var/www/ is set to www-data (recursively)
user is a member of the group www-data
Do I need to do anything in VSCode to update permissions? Am I missing something else?
Here are the exact commands I used:
sudo adduser user www-data
sudo chown -R www-data:www-data /var/www
sudo chmod -R 0775 /var/www
OK I did a bit more Googling right after posting this, and I was able to fix this by deleting the .vscode-server folder in the home directory.
https://github.com/microsoft/vscode-remote-release/issues/3399#issuecomment-922935448
I'm not sure if there is an easier way to fix this without doing so, or having to do that every time permissions are changed.

are there ways to use VS code plugins in google cloud shell?

I have a few quick navigation plugins such as "block travel" I use all the time. Is there a way to use these in cloud shell?
I imagine there are some restrictions, but even some simple editor plugins can be huge timesavers.
While I'm at it - alt-D to duplicate a line, or transpose lines - some of those seem to be missing and hard to use key remapping to get working, at least within the shell. In general maybe keyboard shortcuts seem to get trapped by the browser or PWA wrapper. I'm using cloudshell as a webapp on a chromebook FWIW, for various secure projects.
I have come up with a solution that covers both aspects of your question
To get Unlimited Persistent Disk:
You can use Google Cloud Storage FUSE
Google Cloud Storage FUSE lets you mount a GCS bucket as a folder to your linux instance. By doing that you get an “unlimited '' persistent disk and it is super simple to set up since gcsfuse is already installed in cloud shell.
1. Create a GCS bucket (you only need to run this once) -- replace BUCKET_NAME with any name:
gsutil mb "gs://BUCKET_NAME/"
2. Create a local directory for mounting -- replace FOLDER_NAME with the chosen directory name:
mkdir /home/[USER]/[FOLDER_NAME]
chmod 777 /home/[USER]/[FOLDER_NAME]
3. Mount the bucket onto the local filesystem (note: you need to re-run this every time Cloud Shell starts)
gcsfuse -o nonempty -file-mode=777 -dir-mode=777 --uid=1000 --debug_gcs [BUCKET_NAME] /home/[USER]/[FOLDER_NAME]
To use third party plugins in cloud shell:
You can use an environment customization script (.customization_environment) as mentioned in the public documentation. It allows you to install additional packages into your Cloud Shell environment when it starts.
For reference, below are the steps to install VS Code plug-in.
Step 1:
To install the VSCode server, run the script named visual_studio_code.sh as below, in the root directory workspace of Cloud Shell Editor.
visual_studio_code.sh file:
export VERSION=`curl -s https://api.github.com/repos/cdr/code-server/releases/latest | grep -oP '"tag_name": "\K(.*)(?=")'`
wget https://github.com/cdr/code-server/releases/download/$VERSION/code-server-3.10.2-linux-amd64.tar.gz
tar -xvzf code-server-3.10.2-linux-amd64.tar.gz
cd code-server-3.10.2-linux-amd64
Run the script using the below command in shell,
./visual_studio_code.sh
if getting permission denied error then run this following command in shell,
chmod +x visual_studio_code.sh
./visual_studio_code.sh
Step 2:
Make a customization script in the root directory workspace in Cloud Shell Editor to start VS Code Server on boot with the below commands :
.customization_environment file :
#!/bin/sh
#.customize_environmnet run in background as root, wait for your user to initialize
sleep 20
sudo -u [USER] /home/[USER]/code-server-3.10.2-linux-amd64/code-server --auth none --port 9090
Step 3:
To view Visual Studio Code server on port 9090 :
Click on Web Preview > Change Port > 9090
If getting a 404 error, remove ‘?authuser=0’ from the url.
Visual Studio Code Server will now be running on the browser!!!
Block travel navigation plugin:
To have the block travel navigation plugin in cloud shell,follow the below commands and run them in shell in root directory:
wget https://github.com/efatsi/block-travel/archive/refs/tags/v1.0.0.tar.gz
tar xzvf v1.0.0.tar.gz
ls
#You will see block-travel-1.0.0
block-travel-1.0.0/keymaps/block-travel.cson --auth none --port 9090
#You might get Permission denied if yes, then follow the next two commands else go to webport view in 9090
chmod +x block-travel-1.0.0/keymaps/block-travel.cson
block-travel-1.0.0/keymaps/block-travel.cson --auth none --port 9090
Open the webport view in 9090, you will be able to navigate through the vs code files using :
Alt+up for block-travel.jumpUp
Alt+shift+up for block-travel.selectUp
Alt+down for block-travel.jumpDown
Alt+shift+down for block-travel.selectDown
WARNING: This should not be considered a long term solution, just a stop gap until this is supported in an easier fashion.
This might not be the greatest idea but it does seem to work for the vim extension I tried in my environment. Probably best to make a request through the in product feedback to get it officially added but until then you can follow these steps.
Upload the .vsix package to your $HOME directory.
Unzip the package into the /google/devshell/editor/theia/plugins directory. This action will not persist so you'll want to add the command to the .customize_environment script actions.
e.g.
sudo unzip vscodevim.vsix -d /google/devshell/editor/theia/plugins/vscode-vim
Now for the questionable part. You'll want to install the pslist package to make life easy so you have access to the rkill command. You probably also want to add this to the .customize_environment file as well since it also will not persist.
sudo apt install pslist
Now we need to get the process id for the editor. Currently this seems to be spawned by a supervisord command, which also spawns the tmux section so we're going to grab the process id that is from the runuser command it spawns (and filter for the theia one just in case).
ps ax | grep runuser | grep "theia start"
Then we can use rkill to kill the process and all of the its children, which will cause supervisord to restart it for us and the plugin should be available.
sudo rkill PID_OF_GREP_OUTPUT
I'm not sure the best way to script the rkill command yet, since I don't know the timing of when it's up vs the .customize_environment execution, so right now I run this each time I start up a new VM.
If anything goes horribly wrong, you should be able to request a restart of the VM from the menu options and get a fresh one.
Cloud Shell offers VS Code editor experience through Theia. Did you try cloud Code editor in the cloud shell? that is exposed through "Open Editor" button on the top right, this will open cloud code editor that gives you VSCode experience. You have all the navigation keys that are available in the editor.

how do I elevate my gcloud scp and ssh commands?

I want to be able to fire commands at my instance with gcloud because it handles auth for me. This works well but how do I run them with sudo/root access?
For example I can copy files to my accounts folder:
gcloud compute scp --recurse myinst:/home/me/zzz /test --zone us-east1-b
But I cant copy to /tmp:
gcloud compute scp --recurse /test myinst:/tmp --zone us-east1-b
pscp: unable to open directory /tmp/.pki: permission denied
19.32.38.265147.log | 0 kB | 0.4 kB/s | ETA: 00:00:00 | 100%
pscp: unable to open /tmp/ks-script-uqygub: permission denied
What is the right way to run "gcloud compute scp" with sudo? Just to be clear, I of course can ssh into the instance and run sudo interactively
Edit: for now im just editing the permissions on the remote host
Just so I'm understanding correctly, are you trying to copy FROM the remote /tmp folder, or TO it? This question sounds like you're trying to copy to it, but the code says you're trying to copy from it.
This has worked for me in the past for copying from my local drive to a remote drive, though I have some concern over running sudo remotely:
gcloud compute scp myfile.txt [gce_user]#myinst:~/myfile.txt --project=[project_name];
gcloud compute ssh [gce_user]#myinst --command 'sudo cp ~/myfile.txt /tmp/' --project=[project_name];
You would reverse the process (and obviously rewrite the direction and sequence of the commands) if you needed to remotely access the contents of /tmp and then copy them down to your local drive.
Hope this helps!

using a conda virtual environment in jupyter notebook

I have read and implemented instructions from earlier posts like:
How to start an ipython shell(not notebook) within a conda or virtualenv
My goal is to use a kernel in ipython which has all conda packages from my virtual environment.
I have a google ubuntu 16.04 machine where I have installed anaconda and a virtual environment in which i installed all my packages..
when i run
python -m ipykernel.kernelspec
i get the following error:
/home/admin/anaconda3/envs/py36ve/lib/python3.6/site-packages/IPython/paths.py:61: UserWarning: IPython dir
'/home/admin/.ipython' is not a writable location, using a temp
directory.
" using a temp directory.".format(ipdir))
[Errno 13] Permission denied: '/usr/local/share/jupyter/kernels/python3'
I tried running with sudo too.. i created a kernel but when i use it then it has none of the packages i installed in the virtual environment..
I do have a similar issue with this when I try to submit my program to a cluster where it doesn't have access to my local directory and it shows the same message. But I don't get Permission denied message and everything is fine by me. But I wanted to address this issue and looked into it and I found that paths.py at line 62 in python package in the case of not writable, it creates a temp directory like the following:
ipdir = tempfile.mkdtemp()
As in tempfile documentation says:
Creates a temporary directory in the most secure manner possible. There are no race conditions in the directory’s creation. The directory is readable, writable, and searchable only by the creating user ID.
It is strange that you do get this but if you want to make it work, find the paths.py and change that to your liking and makes sure it works and replace it with the original.

IPython error executing system commands

If i execute any system commend in ipython2/3 on my ubuntu 16.04 machine, I get permission denied error as follows:
In [1]: ls
--------------------------------------------------------------------------
PermissionError Traceback (most recent call last)
<ipython-input-1-5b21b94dda0e> in <module>()
----> 1 get_ipython().magic('ls ')
I did not need to do anything special in ubuntu 14.04. I have installed ipython2 and ipython3 with
pip2 install ipython --user
pip3 install ipython --user
I also tried installing it system-wide with sudo as
sudo pip2 install ipython
sudo pip3 install ipython
but the problem still persists. I do not get this problem in jupyter notebook. Any clue will be highly appreciated.
When I was setting up my machine, I created a dummy rootUser as the first admin user. Later on, I created another user with the same name, uid and gid as my workplace and added it to sudo. This some how causes permission problems which is really strange. Though I do not know the actual cause of the problem but I did reproduce it with the above steps. I solved this problem by creating the first user with the name same as my workplace. Later i changed the uid and gid and file/directly permissions accordingly. This solved the problem and I can now use ipython2/3 normally.