I am trying to figure out a way to capture logs rendered through users interaction with the kernel in Jupyter Lab.
After you jupyter lab --generate-config there will be configuration file with a few logging-related options available:
c.Application.log_datefmt = '%Y-%m-%d %H:%M:%S'
c.Application.log_format = '[%(name)s]%(highlevel)s %(message)s'
c.Application.log_level = 30
With these, it seems to generate some verbose output in the terminal but can someone point me in direction to help understand the following:
How can I get this to log into a file?
This configuration seems to be generating logging handlers which could be altered in the notebook code easily - is there a way to get a lower end logger that can not be altered in user code?
Is this a right way to capture kernel logs (I am looking for something that would capture all the inputs and maybe even outputs sent kernel way)?
Related
I am trying to create an extension for jupyter-lab that interacts with a running ipython kernel, running either in a console or a notebook.
As a minimal example I am trying to get two "hello world" type programs. The first should print a message to stdout of the running kernel that is input in the extension. The second should log a message to the console of the extension that is input in the running ipython kernel.
To understand jupyter-lab extension I started with the astronomy picture of the day! tutorial. Although this covers the basics of extensions this does not address interacting with a running kernel.
From "really confused with jupyter notebook lab extensions and ipywidgets"! it seems I should be looking at the Comms module however this is documentation for Jupyter-notebook and not for Jupyter-lab. As far as I understand, Jupyter-lab is sufficiently different under the hood that this should not work.
Could someone explain to me how to create an extension that interacts with a running kernel and what the appropriate documentation/tutorial is to look at if it exists?
I want my function to do one thing if it's called from the IPython notebook and another thing if its called from a console or library code. In particular I'm making a progress bar with the desired following behavior:
Notebook: Immediately return a widget
Console: Block and dump information to sys.stdout
Is there a flag somewhere I can check to determine if the user called my function from a notebook or otherwise?
You can't detect that the frontend is a notebook with perfect precision, because an IPython Kernel can have one or more different Jupyter frontends with different capabilities (terminal console, qtconsole, notebook, etc.). You can, however, identify that it is a Kernel and not plain terminal IPython:
import sys
def is_kernel():
if 'IPython' not in sys.modules:
# IPython hasn't been imported, definitely not
return False
from IPython import get_ipython
# check for `kernel` attribute on the IPython instance
return getattr(get_ipython(), 'kernel', None) is not None
Since the notebook is so much more popular than other frontends, this indicates that the frontend is probably a notebook, but not conclusively. So you want to be sure there's a way out for the more primitive frontends.
I learned this trick from Using IPython console along side IPython notebook . But when I connect this kernel using console, the %load magic will load file into a pager (like you you do 'man thecodefile.py' in console), not to the input line.
Anyone know how to change these behavior back to default?
You ask two question here, please ask them separately.
As for !su ausername there is no way to forward stdin through subprocess.
On Centos 6.2, trying to get the kernel log redirected to the serial console, I came across an issue where agetty seems to be respawning every few keypresses.
That is, I get a login prompt in the middle of typing (after logging in).
In order to investigate the issue further, I'm looking for the location of agetty logs, but to no avail. Where and how can I see log messages for respawned agetty process?
The "diagnostics" section of the "agetty" command manpage states:
Depending on how the program was configured, all diagnostics are written to
the console device or reported via the syslog(3) facility. Error messages are
produced if the port argument does not specify a terminal device; if there is
no utmp entry for the current process (System V only); and so on.
The syslog facility by default writes the "/var/log/messages" file, but it can be configured to write another file by editing its configuration file "/etc/syslog.conf".
Finally, if the error you get is "respawning too fast", you should check your "/etc/inittab" file, as described here.
Howdy, I am trying to run matlab remotely on windows via OpenSSH installed with Cygwin, but launching matlab in windows without the GUI seems to be impossible.
If i am logged in locally, I can launch matlab -nodesktop -nodisplay -r script, and matlab will launch up a stripped down GUI and do the command.
However, this is impossible to do remotely via ssh, as, matlab needs to display the GUI.
Does anyone have any suggestions or work arounds?
Thanks,
Bob
Short story: is your script calling exit()? Are you using "-wait"?
Long story: I think you're fundamentally out of luck if you want to interact with it, but this should work if you just want to batch jobs up. Matlab on Windows is a GUI application, not a console application, and won't interact with character-only remote connectivity. But you can still launch the process. Matlab will actually display the GUI - it will just be in a desktop session on the remote computer that you have no access to. But if you can get it to do your job without further input, this can be made to work, for some value of "work".
Your "-r script" switch is the right direction. But realize that on Windows, Matlab's "-r" behavior is to finish the script and then go back to the GUI, waiting for further input. You need to explicitly include an "exit()" call to get your job to finish, and add try/catches to make sure that exit() gets reached. Also, you should use a "-logfile" switch to capture a copy of all the command window output to a log file so you can see what it's doing (since you can't see the GUI) and have a record of prior runs.
Also, matlab.exe is asynchronous by default. Your ssh call will launch Matlab and return right away unless you add the "-wait" switch. Check the processes on the machine you're sshing to; Matlab may actually be running. Add -wait if you want it to block until finished.
One way to do this stuff just use -r to call to a standard job wrapper script that initializes your libraries and paths, runs a job, and does cleanup and exit. You'll also want to make a .bat wrapper that sets up the -logfile switch to point to a file with the job name, timestamp, and other info in it. Something like this at the M-code level.
function run_batch_job(jobname)
try
init_my_matlab_library(); % By calling classpath(), javaclasspath(), etc
feval(jobname); % assumes jobname is an M-file on the path
catch err
warning('Error occurred while running job %s: %s', jobname, err.message)
end
try
exit();
catch err
% Yes, exit() can throw errors
java.lang.System.exit(1); % Scuttle the process hard to make sure job finishes
end
% If your code makes it to here, your job will hang
I've set up batch job systems using this style in Windows Scheduler, Tidal, and TWS before. I think it should work the same way under ssh or other remote access.
A Matlab batch system on Windows like this is brittle and hard to manage. Matlab on Windows is fundamentally not built to be a headless batch execution system; assumptions about an interactive GUI are pervasive in it and hard to work around. Low-level errors or license errors will pop up modal dialog boxes and hang your job. The Matlab startup sequence seems to have race conditions. You can't set the exit status of MATLAB.exe. There's no way of getting at the Matlab GUI to debug errors the job throws. The log file may be buffered and you lose output near hangs and crashes. And so on.
Seriously consider porting to Linux. Matlab is much more suitable as a batch system there.
If you have the money or spare licenses, you could also use the Matlab Distributed Computing toolbox and server to run code on remote worker nodes. This can work for parallelization or for remote batch jobs.
There are two undocumented hacks that reportedly fix a similar problem - they are not guarantied to solve your particular problem but they are worth a try. Both of them depend on modifying the java.opts file:
-Dsun.java2d.pmoffscreen=false
Setting this option fixes a problem of extreme GUI slowness when launching Matlab on a remote Linux/Solaris computer.
-Djava.compiler=NONE
This option disables the Java just-in-time compiler (JITC). Note that it has no effect on the Matlab interpreter JITC. It has a similar effect to running Matlab with the '–nojvm' command-line option. Note that this prevents many of Matlab's GUI capabilities. Unfortunately, in some cases there is no alternative. For example, when running on a remote console or when running pre-2007 Matlab releases on Intel-based Macs. In such cases, using the undocumented '-noawt' command-line option, which enables the JVM yet prevents JAVA GUI, is a suggested compromise.
Using putty use ssh -X remote "matlab" it should work