Watch long term process in Ipython notebook - ipython

Question

For now, after reconnect ipython notebook is not able to print any output from running kernel. I tried following work-around:
import sys, time
with open('foo.log','w') as sys.stdout:
for i in range(5):
print i
time.sleep(1)
everything is nice, I can see the process with tail -f foo.log, but at the end whole thing crashes and I get following error:
ERROR:tornado.general:Uncaught exception, closing connection.
Traceback (most recent call last):
File "/usr/lib/python2.7/dist-packages/zmq/eventloop/zmqstream.py", line 407, in _run_callback
callback(*args, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/tornado/stack_context.py", line 275, in null_wrapper
return fn(*args, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/ipykernel/kernelbase.py", line 260, in dispatcher
return self.dispatch_shell(stream, msg)
File "/usr/local/lib/python2.7/dist-packages/ipykernel/kernelbase.py", line 218, in dispatch_shell
sys.stdout.flush()
ValueError: I/O operation on closed file
ERROR:tornado.general:Uncaught exception, closing connection.
Traceback (most recent call last):
File "/usr/lib/python2.7/dist-packages/zmq/eventloop/zmqstream.py", line 433, in _handle_events
self._handle_recv()
File "/usr/lib/python2.7/dist-packages/zmq/eventloop/zmqstream.py", line 465, in _handle_recv
self._run_callback(callback, msg)
File "/usr/lib/python2.7/dist-packages/zmq/eventloop/zmqstream.py", line 407, in _run_callback
callback(*args, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/tornado/stack_context.py", line 275, in null_wrapper
return fn(*args, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/ipykernel/kernelbase.py", line 260, in dispatcher
return self.dispatch_shell(stream, msg)
File "/usr/local/lib/python2.7/dist-packages/ipykernel/kernelbase.py", line 218, in dispatch_shell
sys.stdout.flush()
ValueError: I/O operation on closed file
ERROR:tornado.application:Exception in callback None
Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/tornado/ioloop.py", line 883, in start
handler_func(fd_obj, events)
File "/usr/local/lib/python2.7/dist-packages/tornado/stack_context.py", line 275, in null_wrapper
return fn(*args, **kwargs)
File "/usr/lib/python2.7/dist-packages/zmq/eventloop/zmqstream.py", line 433, in _handle_events
self._handle_recv()
File "/usr/lib/python2.7/dist-packages/zmq/eventloop/zmqstream.py", line 465, in _handle_recv
self._run_callback(callback, msg)
File "/usr/lib/python2.7/dist-packages/zmq/eventloop/zmqstream.py", line 407, in _run_callback
callback(*args, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/tornado/stack_context.py", line 275, in null_wrapper
return fn(*args, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/ipykernel/kernelbase.py", line 260, in dispatcher
return self.dispatch_shell(stream, msg)
File "/usr/local/lib/python2.7/dist-packages/ipykernel/kernelbase.py", line 218, in dispatch_shell
sys.stdout.flush()
ValueError: I/O operation on closed file

Answer 1

I have finally managed to log messages from jupyter into file and flush memory immediately:
class InstantLogger():
def __init__(self, f):
self.f = open(f,'w')
def __getattr__(self,name):
return object.__getattribute__(self.f, name)
def write(self, x):
self.f.write(x)
self.f.flush()
def flush(self):
self.f.flush()
def close(self):
self.f.close()
than I can use it as:
old_stdout = sys.stdout
sys.stdout = InstantLogger('file.log')
print ....
sys.stdout.close()
sys.stdout = old_stdout