I'm new to this kind of things. In my company we have a unix remote server with python/anaconda installed and we would like to switch from jupyter-lab to jupyterhub to be able to let many users login and use jupyter at the same time.
I'm trying to get used to the installation using my account (taht is not admin and is accessed through cyberark).
I installed with conda jupyterhub, create the config file and set the ip address (JupyterHub.ip) and port (JupyterHub.port) so to be able to connect via web browser from the user client.
I have opened the firewall route to this port correctly.
Just to try i have set the authenticator to dummy so to be able to connect using whatever username and pwd.
Using my account i run
jupyterhub
this is the ouput
[pas.datascience_dev#devrmdatasci01 .jupyter]$ jupyterhub [I 2019-06-04 11:43:05.522 JupyterHub app:2120] Using Authenticator: jupyterhub.auth.DummyAuthenticator-1.0.0 [I 2019-06-04 11:43:05.522 JupyterHub app:2120] Using Spawner: jupyterhub.spawner.LocalProcessSpawner-1.0.0 [I 2019-06-04 11:43:05.526 JupyterHub app:1257] Loading cookie_secret from /home/pas.datascience_dev/.jupyter/jupyterhub_cookie_secret [I 2019-06-04 11:43:05.557 JupyterHub proxy:460] Generating new CONFIGPROXY_AUTH_TOKEN [W 2019-06-04 11:43:05.559 JupyterHub app:1532] No admin users, admin interface will be unavailable. [W 2019-06-04 11:43:05.559 JupyterHub app:1534] Add any administrative users to c.Authenticator.admin_users in config. [I 2019-06-04 11:43:05.559 JupyterHub app:1563] Not using whitelist. Any authenticated user will be allowed. [I 2019-06-04 11:43:05.611 JupyterHub app:2337] Hub API listening on http://127.0.0.1:8081/hub/ [W 2019-06-04 11:43:05.613 JupyterHub proxy:642] Running JupyterHub without SSL. I hope there is SSL termination happening somewhere else... [I 2019-06-04 11:43:05.613 JupyterHub proxy:645] Starting proxy # http://10.207.2.10:8689/ 11:43:06.217 - info: [ConfigProxy] Proxying http://10.207.2.10:8689 to (no default) 11:43:06.220 - info: [ConfigProxy] Proxy API at http://127.0.0.1:8001/api/routes [I 2019-06-04 11:43:06.684 JupyterHub proxy:319] Checking routes [I 2019-06-04 11:43:06.685 JupyterHub proxy:399] Adding default route for Hub: / => http://127.0.0.1:8081 [I 2019-06-04 11:43:06.688 JupyterHub app:2422] JupyterHub is now running at http://10.207.2.10:8689/
I copy the url and paste it to my local desktop browser and correctly get the login page.
I insert user: "aaa" and pwd "abc" (i'm using the dummy auth so it should be accepted) but i get spqwn failed with this log:
[I 2019-06-04 11:46:11.157 JupyterHub base:663] User logged in: aaa [I 2019-06-04 11:46:11.160 JupyterHub log:174] 302 POST /hub/login?next= -> /hub/spawn (aaa#127.0.0.1) 10.15ms [E 2019-06-04 11:46:11.243 JupyterHub user:626] Unhandled error starting aaa's server: 'getpwnam(): name not found: aaa' [I 2019-06-04 11:46:11.261 JupyterHub log:174] 302 GET /hub/spawn -> /hub/spawn-pending/aaa (aaa#127.0.0.1) 56.58ms ERROR:asyncio:Task exception was never retrieved future: <Task finished coro=<BaseHandler.spawn_single_user() done, defined at /data/pyconda36/lib/python3.6/site-packages/jupyterhub/handlers/base.py:697> exception=KeyError('getpwnam(): name not found: aaa',)> Traceback (most recent call last): File "/data/pyconda36/lib/python3.6/site-packages/jupyterhub/handlers/base.py", line 889, in spawn_single_user timedelta(seconds=self.slow_spawn_timeout), finish_spawn_future File "/data/pyconda36/lib/python3.6/site-packages/jupyterhub/handlers/base.py", line 807, in finish_user_spawn await spawn_future File "/data/pyconda36/lib/python3.6/site-packages/jupyterhub/user.py", line 642, in spawn raise e File "/data/pyconda36/lib/python3.6/site-packages/jupyterhub/user.py", line 546, in spawn url = await gen.with_timeout(timedelta(seconds=spawner.start_timeout), f) File "/data/pyconda36/lib/python3.6/site-packages/jupyterhub/spawner.py", line 1377, in start env = self.get_env() File "/data/pyconda36/lib/python3.6/site-packages/jupyterhub/spawner.py", line 1326, in get_env env = self.user_env(env) File "/data/pyconda36/lib/python3.6/site-packages/jupyterhub/spawner.py", line 1313, in user_env home = pwd.getpwnam(self.user.name).pw_dir KeyError: 'getpwnam(): name not found: aaa' [E 2019-06-04 11:46:11.344 JupyterHub pages:284] Previous spawn for aaa failed: 'getpwnam(): name not found: aaa' [E 2019-06-04 11:46:11.350 JupyterHub log:166] { "Cookie": "jupyterhub-hub-login=[secret]; user-id=[secret]; csrf-token=[secret]; _xsrf=[secret]; username-10-207-2-10-8687=[secret]; username-10-207-2-10-8686=[secret]; jupyterhub-session-id=[secret]", "Accept-Language": "it-IT,it;q=0.9,en-US;q=0.8,en;q=0.7", "Accept-Encoding": "gzip, deflate", "Referer": "http://10.207.2.10:8689/hub/login", "Accept": "text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8", "User-Agent": "Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/64.0.3282.186 Safari/537.36", "Upgrade-Insecure-Requests": "1", "Cache-Control": "max-age=0", "Connection": "close", "Host": "10.207.2.10:8689" } [E 2019-06-04 11:46:11.350 JupyterHub log:174] 500 GET /hub/spawn-pending/aaa (aaa#127.0.0.1) 12.16ms
If i try to login using the username from which i'm launching the jupyterhub so user = "pas.datascience_dev" and use random pwd i get to a charging bar but then spawn failed with this error
Spawn failed: Server at http://127.0.0.1:55345/user/pas.datascience_dev/ didn't respond in 30 seconds
and this log
[I 2019-06-04 11:47:16.755 JupyterHub base:663] User logged in: pas.datascience_dev
[I 2019-06-04 11:47:16.757 JupyterHub log:174] 302 POST /hub/login?next= -> /hub/spawn (pas.datascience_dev#127.0.0.1) 8.06ms
[I 2019-06-04 11:47:16.822 JupyterHub spawner:1387] Spawning jupyterhub-singleuser --port=55345
Failed to set groups [Errno 1] Operation not permitted
Traceback (most recent call last):
File "/data/pyconda36/bin/jupyterhub-singleuser", line 8, in
from jupyterhub.singleuser import main
File "/data/pyconda36/lib/python3.6/site-packages/jupyterhub/singleuser.py", line 41, in
from notebook.notebookapp import (
File "/data/pyconda36/lib/python3.6/site-packages/notebook/notebookapp.py", line 40, in
ioloop.install()
File "/data/pyconda36/lib/python3.6/site-packages/zmq/eventloop/ioloop.py", line 210, in install
assert (not ioloop.IOLoop.initialized()) or
AttributeError: type object 'IOLoop' has no attribute 'initialized'
[I 2019-06-04 11:47:17.801 JupyterHub log:174] 302 GET /hub/spawn -> /hub/spawn-pending/pas.datascience_dev (pas.datascience_dev#127.0.0.1) 1008.91ms
ERROR:asyncio:Task exception was never retrieved
future: <Task finished coro=<BaseHandler.spawn_single_user() done, defined at /data/pyconda36/lib/python3.6/site-packages/jupyterhub/handlers/base.py:697> exception=HTTPError()>
Traceback (most recent call last):
File "/data/pyconda36/lib/python3.6/site-packages/jupyterhub/handlers/base.py", line 889, in spawn_single_user
timedelta(seconds=self.slow_spawn_timeout), finish_spawn_future
tornado.util.TimeoutError: Timeout
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/data/pyconda36/lib/python3.6/site-packages/jupyterhub/handlers/base.py", line 922, in spawn_single_user
% (status, spawner._log_name),
tornado.web.HTTPError: HTTP 500: Internal Server Error (Spawner failed to start [status=1]. The logs for pas.datascience_dev may contain details.)
[W 2019-06-04 11:47:48.040 JupyterHub user:678] pas.datascience_dev's server never showed up at http://127.0.0.1:55345/user/pas.datascience_dev/ after 30 seconds. Giving up
[E 2019-06-04 11:47:48.055 JupyterHub gen:974] Exception in Future <Task finished coro=<BaseHandler.spawn_single_user..finish_user_spawn() done, defined at /data/pyconda36/lib/python3.6/site-packages/jupyterhub/handlers/base.py:800> exception=TimeoutError("Server at http://127.0.0.1:55345/user/pas.datascience_dev/ didn't respond in 30 seconds",)> after timeout
Traceback (most recent call last):
File "/data/pyconda36/lib/python3.6/site-packages/tornado/gen.py", line 970, in error_callback
future.result()
File "/data/pyconda36/lib/python3.6/site-packages/jupyterhub/handlers/base.py", line 807, in finish_user_spawn
await spawn_future
File "/data/pyconda36/lib/python3.6/site-packages/jupyterhub/user.py", line 654, in spawn
await self._wait_up(spawner)
File "/data/pyconda36/lib/python3.6/site-packages/jupyterhub/user.py", line 701, in _wait_up
raise e
File "/data/pyconda36/lib/python3.6/site-packages/jupyterhub/user.py", line 669, in _wait_up
http=True, timeout=spawner.http_timeout, ssl_context=ssl_context
File "/data/pyconda36/lib/python3.6/site-packages/jupyterhub/utils.py", line 234, in wait_for_http_server
timeout=timeout,
File "/data/pyconda36/lib/python3.6/site-packages/jupyterhub/utils.py", line 177, in exponential_backoff
raise TimeoutError(fail_message)
TimeoutError: Server at http://127.0.0.1:55345/user/pas.datascience_dev/ didn't respond in 30 seconds
[I 2019-06-04 11:47:48.058 JupyterHub log:174] 200 GET /hub/api/users/pas.datascience_dev/server/progress (pas.datascience_dev#127.0.0.1) 30176.86ms
What I'm missing or doing wrong ?
After i'll be able to use this i will give the whole package to systems admin and make them install it as root so to be able to manage multiple users
Use:
pip install notebook
or:
conda install notebook
Related
I am trying to run google cloud python sdk from inside a k8 pod, running on google compute engine. There is a service account attached to the VM, which is giving it access to the secrets manager. I am able to access secrets manager from the host, however running the python sdk from k8 pod complains of not able to access the metadata service
>>> secret_id = 'unskript_test'
>>> name = client.secret_path(project_id, secret_id)
>>> response = client.get_secret(request={"name": name})
Traceback (most recent call last):
File "/opt/conda/lib/python3.7/site-packages/google/api_core/grpc_helpers.py", line 67, in error_remapped_callable
return callable_(*args, **kwargs)
File "/opt/conda/lib/python3.7/site-packages/grpc/_channel.py", line 946, in __call__
return _end_unary_response_blocking(state, call, False, None)
File "/opt/conda/lib/python3.7/site-packages/grpc/_channel.py", line 849, in _end_unary_response_blocking
raise _InactiveRpcError(state)
grpc._channel._InactiveRpcError: <_InactiveRpcError of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Getting metadata from plugin failed with error: Failed to retrieve http://metadata.google.internal/computeMetadata/v1/instance/service-accounts/default/?recursive=true from the Google Compute Enginemetadata service. Compute Engine Metadata server unavailable"
debug_error_string = "{"created":"#1630634901.103779641","description":"Getting metadata from plugin failed with error: Failed to retrieve http://metadata.google.internal/computeMetadata/v1/instance/service-accounts/default/?recursive=true from the Google Compute Enginemetadata service. Compute Engine Metadata server unavailable","file":"src/core/lib/security/credentials/plugin/plugin_credentials.cc","file_line":90,"grpc_status":14}"
>
metadata.google.internal doesnt get resolved from the k8 pod
jovyan#jovyan-25ca6c8c-157d-49e5-9366-f9d57fcb7a9f:~$ wget http://metadata.google.internal/computeMetadata/v1/instance/service-accounts/default/?recursive=true
--2021-09-03 02:11:19-- http://metadata.google.internal/computeMetadata/v1/instance/service-accounts/default/?recursive=true
Resolving metadata.google.internal (metadata.google.internal)... failed: Name or service not known.
wget: unable to resolve host address ‘metadata.google.internal’
However, host is able to resolve it
ubuntu#gcp-test-proxy:~$ wget http://metadata.google.internal/computeMetadata/v1/instance/service-accounts/default/?recursive=true
--2021-09-03 02:11:27-- http://metadata.google.internal/computeMetadata/v1/instance/service-accounts/default/?recursive=true
Resolving metadata.google.internal (metadata.google.internal)... 169.254.169.254
Connecting to metadata.google.internal (metadata.google.internal)|169.254.169.254|:80... connected.
HTTP request sent, awaiting response... 403 Forbidden
2021-09-03 02:11:27 ERROR 403: Forbidden.
How can i make the pod resolve metadata.google.internal?
I've set up a test MongoDb Atlas account, just one of the free ones and using the below code on my Ubuntu box I can successfully create users and also search for them.
When I try the exact same thing on my MacBook Air I get an SSL handshake as below.
pymongo.errors.ServerSelectionTimeoutError: SSL handshake failed: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:833),SSL handshake failed: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:833),SSL handshake failed: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:833)
import pymongo
client = pymongo.MongoClient("mongodb+srv://MYUSERNAME:MYPASSWORD#cluster0-ABCDEF.mongodb.net/test")
db = client.johnny
collection = db.myjohnnytest
example = {'name' : 'Johnny',
'email' : 'johnny#test.net'}
user_id = collection.insert_one(example).inserted_id
I found this answer which seemed like it was the key:
Stack Overflow Answer
But I have since tried this and whether I follow it exactly or activate my virtualenv I get the following error:
Could not find an activated virtualenv (required).
Traceback (most recent call last):
File "", line 44, in
File "", line 25, in main
File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/subprocess.py", line 291, in check_call
raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['/Library/Frameworks/Python.framework/Versions/3.6/bin/python3.6', '-E', '-s', '-m', 'pip', 'install', '--upgrade', 'certifi']' returned non-zero exit status 3.
logout
Saving session...
...copying shared history...
...saving history...truncating history files...
...completed.
By default, PyMongo is configured to require a certificate from the server when TLS is enabled. This is configurable using the ssl_cert_reqs option. To disable this requirement pass ssl.CERT_NONE as a keyword parameter:
>>> uri = 'mongodb://example.com/?ssl=true&ssl_cert_reqs=CERT_NONE'
>>> client = pymongo.MongoClient(uri)
Have you update the CA bundle for Python?
From a bash shell/ terminal:
open "/Applications/Python 3.6/Install Certificates.command"
or Python 3.7 etc
MongoDB Documentation for Python
Stackoverflow article on MacOS, Python and CA
I am showing 2 ways of solving this:
(easiest)
Adding ...&ssl=true&ssl_ca_certs=/path/to/cert.pem to the mongodb url
(recommended)
Configuring python SSL connection.
$ python3 -c "import ssl; print(ssl.get_default_verify_paths())
DefaultVerifyPaths(cafile='/Library/Frameworks/Python.framework/Versions/3.6/etc/openssl/cert.pem', capath=None, openssl_cafile_env='SSL_CERT_FILE', openssl_cafile='/Library/Frameworks/Python.framework/Versions/3.6/etc/openssl/cert.pem', openssl_capath_env='SSL_CERT_DIR', openssl_capath='/Library/Frameworks/Python.framework/Versions/3.6/etc/openssl/certs')
Then make sure you add your cert.pem file to the specified path. In my case, I should add it to the folder /Library/Frameworks/Python.framework/Versions/3.6/etc/openssl/
I am trying to run jupyter hub for multiuser mode. I installed jupyterhub from PIP.
[root#ip-of-machine hadoop]# echo $PATH
/usr/local/bin/jupyterhub-singleuser:/usr/local/bin/jupyterhub:/sbin:/bin:/usr/sbin:/usr/bin:/opt/aws/bin
When I tun jupyterhub I get an error saying
500 : Internal Server Error
Failed to start your server. Please contact admin.
I ran it as
[root#ip-of-machine hadoop]# /usr/local/bin/jupyterhub -f ./jupyterhub/jupyterhub_config.py --no-ssl
The logs contain
[I 2016-05-03 17:13:09.406 JupyterHub spawner:465] Spawning jupyterhub-singleuser --user=jupyter --port=40770 --cookie-name=jupyter-hub-token-jupyter --base-url=/user/jupyter --hub-host= --hub-prefix=/hub/ --hub-api-url=http://127.0.0.1:8081/hub/api --ip=127.0.0.1
[E 2016-05-03 17:13:09.433 JupyterHub user:237] Unhandled error starting jupyter's server: [Errno 2] No such file or directory: 'jupyterhub-singleuser'
But there is singleuser in path. I explicitly added it to path and exported.
Any clues? Thanks.
PATH should contain a list of directories containing executables that should be found, not the executables themselves.
So instead of
/usr/local/bin/jupyterhub-singleuser:/usr/local/bin/jupyterhub:/sbin:/bin:/usr/sbin:/usr/bin:/opt/aws/bin
your PATH should be:
/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin:/opt/aws/bin
which should find both jupyterhub and jupyterhub-singleuser.
I am using a xmpppy library to connect with XMPP server... I was able to connect with server but while authentication get error "Plugging ignored: another instance already plugged."
What does this error mean and how can I resolve it??
In [37]: c.isConnected()
Out[37]: ''
In [38]: jid = xmpp.protocol.JID('gathole#localhost')
In [39]: c.auth(jid.getNode(),'password', resource=jid.getResource())
DEBUG: sasl start Plugging <xmpp.auth.SASL instance at 0x108fc2710> into <xmpp.client.Client instance at 0x109003c68>
DEBUG: sasl error Plugging ignored: another instance already plugged.
Traceback (most recent call last):
File "/Users/gathole/.virtualenvs/driveu/lib/python2.7/site-packages/IPython/terminal/interactiveshell.py", line 459, in interact
line = self.raw_input(prompt)
File "/Users/gathole/.virtualenvs/driveu/lib/python2.7/site-packages/IPython/terminal/interactiveshell.py", line 528, in raw_input
line = py3compat.cast_unicode_py2(self.raw_input_original(prompt))
KeyboardInterrupt
DEBUG: gen_auth start Plugging <xmpp.auth.NonSASL instance at 0x108fc2710> into <xmpp.client.Client instance at 0x109003c68>
DEBUG: gen_auth error Plugging ignored: another instance already plugged.
Traceback (most recent call last):
File "/Users/gathole/.virtualenvs/driveu/lib/python2.7/site-packages/IPython/terminal/interactiveshell.py", line 459, in interact
line = self.raw_input(prompt)
File "/Users/gathole/.virtualenvs/driveu/lib/python2.7/site-packages/IPython/terminal/interactiveshell.py", line 528, in raw_input
line = py3compat.cast_unicode_py2(self.raw_input_original(prompt))
KeyboardInterrupt
It requires change the in XMPP ejabberd server config. Change the line {hosts, ["localhost"]} with {hosts, ["localhost", "server-domain", "server-ip-address"]} in the ejabberd.cfg file.
Restart the server and create another user under new hosts with the server domain or server ip.
I'm trying to set up a Jupyterhub instance to serve IPython notebooks on a Google Compute Engine. However, when running jupyterhub i am faced with an error regarding sockets:
[E 2015-08-31 10:27:55.617 JupyterHub app:1097]
Traceback (most recent call last):
File "/home/esten/anaconda3/envs/py3k/lib/python3.3/site- packages/jupyterhub/app.py", line 1095, in launch_instance_async
yield self.start()
File "/home/esten/anaconda3/envs/py3k/lib/python3.3/site-packages/jupyterhub/app.py", line 1027, in start
self.http_server.listen(self.hub_port, address=self.hub_ip)
File "/home/esten/anaconda3/envs/py3k/lib/python3.3/site-packages/tornado/tcpserver.py", line 126, in listen
sockets = bind_sockets(port, address=address)
File "/home/esten/anaconda3/envs/py3k/lib/python3.3/site-packages/tornado/netutil.py", line 187, in bind_sockets
sock.bind(sockaddr)
OSError: [Errno 99] Cannot assign requested address
The address/port assigned by the config file is localhost/8081, and binding a socket like below works perfectly fine
import socket
s = socket.socket()
s.bind(("localhost", 8081))
Does jupyterhub look somewhere else for the information or is something done differently when binding the socket through my own code?
This seems to be a problem with GCE not supporting ipv6.
I found this link explaining that enabling ipv6 solved the issue on another machine.
Running using --ip solved the issue:
jupyter notebook --ip="*"