How do I set a default umask in JupyterHub - jupyter

How do I set a default umask (other than the standard 0022) for individual notebooks/users in JupyterHub?
Use case: I'm using the SystemUserSpawner, which spawns a Docker container for a user, but hooked into the underlying (virtual machine) system users: their notebook home directory matches that of the underlying OS. I've added a small option so that not just users, but also groups match.
I've mounted the base home directory (/home/ usually) to a separate _users folder in the notebook (read-only), so that users can browse each other's home directories for sharing scripts. By default, however, I'd like to have the permissions by group, not world-readable (obviously, users can change that if they'd like), so that different groups can read & share within their group, but not automatically with everyone.
Using an umask setting of 0027 seems to be practical for this, but I can't seem to set it (systemwide): none of the standard OS practices (OS of the Docker container, Ubuntu 18.04) appear to work.
How do I set up a default umask for 0027 for each notebook user?

The single-user notebooks take their configuration from /etc/jupyter/jupyter_notebook_config.py (note: this file lives in the Docker container, not on the host OS).
The last lines of that configuration file are the following:
# Change default umask for all subprocesses of the notebook server if set in
# the environment
if 'NB_UMASK' in os.environ:
os.umask(int(os.environ['NB_UMASK'], 8))
Thus, we can set the environment variable NB_UMASK to set the default umask for the notebook user.
We can set this in the Jupyter Hub configuration file on the host OS. In /etc/jupyterhub/jupyterhub_config.py, add or adjust the c.SystemUserSpawner.environment (or perhaps just c.Spawner.environment, but I'm using a variant of the SystemUserSpanwer) setting to include:
c.SystemUserSpawner.environment = {'NB_UMASK': '0027'}
And that line should be the only thing to set the umask in the entire notebook.
For the record, my full spawner environment is as follows:
c.SystemUserSpawner.environment = {'JUPYTER_ENABLE_LAB': '1', 'GRANT_SUDO': '1', 'NB_UMASK': '0027'}
So that I have a Jupyter Lab environment, and users are able to install further software inside their container as need (sudo apt install <something>).

Related

How to set the Jupyter default user for Pyspark in GCP Dataproc

In a Jupyter notebook connected to a GCP Spark cluster, the cell !pip3 install pyLDAvis==3.2.1 works, but gives a warning:
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager.
It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv
The warning is not unique to pyLDAvis, other packages — even numpy — give the same warning.
Running the notebook as root shouldn't be the default. How can the default user in the notebook be set to singhj rather than root? I have searched through IPython Configuration and customization for any hints.
Configuration: Fresh cluster in GCP Dataproc, default Jupyter notebook, nothing customized.
The Jupyter server in a Dataproc cluster is run by the systemd service defined in the file /usr/lib/systemd/system/jupyter.service.
If you want to change the user it runs as, then you can modify that file and replace the line saying User=root with one saying the name of the user you want (e.g. User=singhj in your example).
Then, once the file has been updated, restart the systemd service by running the following commands as root:
systemctl daemon-reload
systemctl restart jupyter
If you want to automate that, you can write an initialization action to make the change at cluster creation time.

using a conda virtual environment in jupyter notebook

I have read and implemented instructions from earlier posts like:
How to start an ipython shell(not notebook) within a conda or virtualenv
My goal is to use a kernel in ipython which has all conda packages from my virtual environment.
I have a google ubuntu 16.04 machine where I have installed anaconda and a virtual environment in which i installed all my packages..
when i run
python -m ipykernel.kernelspec
i get the following error:
/home/admin/anaconda3/envs/py36ve/lib/python3.6/site-packages/IPython/paths.py:61: UserWarning: IPython dir
'/home/admin/.ipython' is not a writable location, using a temp
directory.
" using a temp directory.".format(ipdir))
[Errno 13] Permission denied: '/usr/local/share/jupyter/kernels/python3'
I tried running with sudo too.. i created a kernel but when i use it then it has none of the packages i installed in the virtual environment..
I do have a similar issue with this when I try to submit my program to a cluster where it doesn't have access to my local directory and it shows the same message. But I don't get Permission denied message and everything is fine by me. But I wanted to address this issue and looked into it and I found that paths.py at line 62 in python package in the case of not writable, it creates a temp directory like the following:
ipdir = tempfile.mkdtemp()
As in tempfile documentation says:
Creates a temporary directory in the most secure manner possible. There are no race conditions in the directory’s creation. The directory is readable, writable, and searchable only by the creating user ID.
It is strange that you do get this but if you want to make it work, find the paths.py and change that to your liking and makes sure it works and replace it with the original.

Can I add (create) shared directories from the host to a running Vagrant box on the fly?

I have a situation where I would like to add shared folders to a running Vagrant box (Ubuntu guest) provided by VirtualBox.
According to the docs, these "Synced Folders" should be defined in the configuration file like so:
config.vm.synced_folder "/mnt/plugdrive15", "/synced/plugdrive15"
But what if I need to share directories that are new on my host? I have this often and I cannot predict their names, so I cannot pre-write them beforehand in the configuration file. I want to automate this.
Using VBoxManage sharedfolder add doesn't work, because as soon as you do vagrant up, only the shares specified in the configuration file (and the default vagrant one) are left.
If all else fails (no better solutions), can Vagrantfile parse a configuration file (in which I specify shares right before vagrant up?
You can using virtual box interface, you have a shared folder option in the VM setting and you can add a new folder while the VM is running
[edit]
there is ofc a command line tool associated : http://www.virtualbox.org/manual/ch04.html#sharedfolders
[edit 2]
Setting the vm name is a provider specific operation, so VBoxManage tool will consider it as well.
config.vm.provider :virtualbox do |vb|
vb.name = "barhost"
end
If you don't specify a name vagrant will generated one, see How to change Vagrant 'default' machine name? for precisions.

ipython notebook on remote server peculiarity

I am taking my first steps with ipython notebook and I installed it successfully on a remote server of mine (over SSH) and I started it using the following command:
ipython notebook --ip='*' ---pylab=inline --port=7777
I then checked on http://myserver.sth:7777/ and the notebook was running just fine. I then wanted to close the SSH connection with the server and keep ipython running in the background. When I did this, I couldn't connect to myserver.sth:7777 anymore. Once I connected again to the remote server by SSH, I could connect again to the notebook. I then tried to use screen to start ipython: I created a new screen by screen -S ipy, I started ipython notebook as above and I used Ctrl+A,D to detach the screen and exit to the TTY. I could still connect remotely to the notebook. I then closed the SSH connection and I got a 404 NOT FOUND error when I tried to access my previously stored notebook and I couldn't see it on the list of notebook at http://myserver.sth:7777/. I tried to create a new notebook, but I got a 500 Internal Server Error.
I also tried running ipython notebook with and without using sudo.
Any ideas?
Rather than use screen, perhaps you could switch to an init script or supervisord to keep IPython notebook up and running.
Let's assume you go the supervisord route:
Install supervisord
Install supervisord using your package manager. For ubuntu it's named supervisor.
apt-get install supervisor
If you decide to install supervisor through pip, you'll have to set up its init.d script yourself.
Write a supervisor configuration file for IPython
The configuration file tells supervisor what to run and how.
After you install supervisor, it should have created /etc/supervisor/supervisord.conf. These lines should exist in the file:
[include]
files = /etc/supervisor/conf.d/*.conf
If they contain these lines, you're in good shape. I only show them to demonstrate where it expects new configuration files. Your configuration file can go there, named something like /etc/supervisor/conf.d/ipynb.conf.
Here's a sample configuration that was generated by Chef by an ipython-notebook-cookbook that runs the notebook in a virtualenv:
[program:ipynb]
command=/home/ipynb/.ipyvirt/bin/ipython notebook --profile=cooked
process_name=%(program_name)s
numprocs=1
numprocs_start=0
autostart=true
autorestart=true
startsecs=1
startretries=3
exitcodes=0,2
stopsignal=QUIT
stopwaitsecs=10
user=ipynb
redirect_stderr=false
stdout_logfile=AUTO
stdout_logfile_maxbytes=50MB
stdout_logfile_backups=10
stdout_capture_maxbytes=0
stdout_events_enabled=false
stderr_logfile=AUTO
stderr_logfile_maxbytes=50MB
stderr_logfile_backups=10
stderr_capture_maxbytes=0
stderr_events_enabled=false
environment=HOME="/home/ipynb",SHELL="/bin/bash",USER="ipynb",PATH="/home/ipynb/.ipyvirt/bin:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games",VIRTUAL_ENV="/home/ipynb/.ipyvirt"
directory=/home/ipynb
serverurl=AUTO
The above supervisor config also relies on an IPython notebook configuration (located at /home/ipynb/.ipython/profile_cooked/ipython_notebook_config.py). This makes configuration much easier (as you can also set up your password hash and many other configurables).:
c = get_config()
# Kernel config
# Make matplotlib plots inline
c.IPKernelApp.pylab = 'inline'
# The IP address the notebook server will listen on.
# If set to '*', will listen on all interfaces.
# c.NotebookApp.ip= '127.0.0.1'
c.NotebookApp.ip='*'
# Port to host on (e.g. 8888, the default)
c.NotebookApp.port = 8888 # If you want it on 80, I recommend iptables rules
# Open browser (probably want False)
c.NotebookApp.open_browser = False
Re-read and update, now that you have the configuration file
supervisorctl reread
supervisorctl update
Reality
In reality, I used to use a Chef cookbook to do the entire installation and configuration. However, using configuration management with tiny stuff like this is a bit of overkill (unless you're orchestrating these in automation).
Nowadays I use Docker images for IPython notebook, orchestrating via JupyterHub or tmpnb.

How to share my remote samba connection for all local users?

Several guys in our office have accounts on local linux(ubuntu) workstation (named local-server). But Just I have the account to access the remote samba server (named remote-server).
For security, I can not tell other guys my account and password information. But I want to share my mount point for them.
For example, I mount remote samba server on this path: /home/samba/. I want everyone on local-server can read/write '/home/samba' directory and its sub-directory. (However, I don't want to modify the remote file's permission mode)
How can I configure it? Thank you very much!
You can set the uid/gid to be different and therefore allow other users to read/write to your share. First create a local group, e.g. shareaccess and assign it to your users:
sudo addgroup shareaccess
sudo usermod -a -G shareaccess user1
Then mount the share:
mount.cifs -ouid=youruser,forceuid,gid=shareaccess,forcegid,file_mode=770,dir_mode=770,credentials=/etc/secret-cred //server/share /home/samba
Files and directory will appear to be owned by youruser:shareaccess locally with permissions ug=rwx.
If the server is a Samba (not Windows) server too and has Unix extensions enabled, file and directory permissions are set according to the server. This might or might not be desirable. You can disable Unix extensions for the mount by adding the nounix option, which will force the modes to be the ones specified at mount-time. Be aware that this will disable all Unix extensions, e.g. symlink support.
References:
mount.cifs man page
if you want to re-attach your mounted samba network shares, you can use the following:
sudo mount -a