graceful interrupt of while loop in ipython notebook - ipython

I'm running some data analysis in ipython notebook. A separate machine collects some data and saves them to a server folder, and my notebook scans this server periodically for new files, and analyzes them.
I do this in a while loop that checks every second for new files. Currently I have set it up to terminate when some number of new files are analyzed. However, I want to instead terminate upon a keypress.
I have tried try-catching a keyboard interrupt, as suggested here: How to kill a while loop with a keystroke?
but it doesn't seem to work with ipython notebook (I am using Windows).
Using openCV's keywait does work for me, but I was wondering if there are alternative methods without having to import opencv.
I have also tried implementing a button widget that interrupts the loop, as such:
from ipywidgets import widgets
import time
%pylab inline
button = widgets.Button(description='Press to stop')
display(button)
class Mode():
def __init__(self):
self.value='running'
mode=Mode()
def on_button_clicked(b):
mode.value='stopped'
button.on_click(on_button_clicked)
while True:
time.sleep(1)
if mode.value=='stopped':
break
But I see that the loop basically ignores the button presses.

You can trigger a KeyboardInterrupt in a Notebook via the menu "Kernel --> Interrupt".
So use this:
try:
while True:
do_something()
except KeyboardInterrupt:
pass
as suggested here and click this menu entry.

Related

programmatically press an enter key after starting .exe file in Matlab

In Matlab I can start external .exe files that sometime have a pop up that requires an enter key pressed. For example:
system('C:\Program Files (x86)\WinZip\WINZIP32.EXE')
will start Winzip, and then in order to use it you need to pass the "buy now" pop up window by pressing enter.
Now my problem is not with winzip, I only gave it as an example (i use winrar anyway :).
How can I programmatically press an enter key in Matlab in such cases ? (I use win 7)
Can an event listener be used to solve that?
EDIT: The java.awt.Robot class indeed works on explorer, but not on any software that has a pop up window with an OK button that needs to be pressed. I don't know why it doesn't work for that. I gave the winzip example because I assume everybody has winzip/winrar installed in their machine. The actual software I have is different and irrelevant for the question.
There is a way using Java from Matlab, specifically the java.awt.Robot class. See here.
Apparently there are two types of programs, regarding the way they work when called from Matlab with system('...'):
For some programs, Matlab waits until the program has finished before running the next statement. This happens for example with WinRAR (at least in my Windows 7 machine).
For other programs this doesn't happen, and Matlab proceeds with the next statement right after the external program has been started. An example of this type is explorer (the standard Windows file explorer).
Now, it is possible to return execution to Matlab immediately even for type 1 programs: just add & at the end of the string passed to system. This is standard in Linux Bash shell, and it also works in Windows, as discussed here.
So, you would proceed as follows:
robot = java.awt.Robot;
command = '"C:\Program Files (x86)\WinRAR\WinRAR"'; %// external program; full path
system([command ' &']); %// note: ' &' at the end
pause(5) %// allow some time for the external program to start
robot.keyPress (java.awt.event.KeyEvent.VK_ENTER); %// press "enter" key
robot.keyRelease (java.awt.event.KeyEvent.VK_ENTER); %// release "enter" key
If your applications are only on Windows platform, you can try using .net objects.
The SendWait method of the SendKeys objects allows to send virtually any key, or key combination, to the application which has the focus, including the "modifier" keys like Alt, Shift, Ctrl etc ...
The first thing to do is to import the .net library, then the full syntax to send the ENTER key would be:
NET.addAssembly('System.Windows.Forms');
System.Windows.Forms.SendKeys.SendWait('{ENTER}'); %// send the key "ENTER"
If you only do it once the full syntax is OK. If you plan to make extensive use of the command, you can help yourself with an anonymous helper function.
A little example with notepad
%% // import the .NET assembly and define helper function
NET.addAssembly('System.Windows.Forms');
sendkey = #(strkey) System.Windows.Forms.SendKeys.SendWait(strkey) ;
%% // prepare a few things to send to the notepad
str1 = 'Hello World' ;
str2 = 'OMG ... my notepad is alive' ;
file2save = [pwd '\SelfSaveTest.txt'] ;
if exist(file2save,'file')==2 ; delete(file2save) ; end %// this is just in case you run the test multiple times.
%% // go for it
%// write a few things, save the file then close it.
system('notepad &') ; %// Start notepad, without matlab waiting for the return value
sendkey(str1) %// send a full string to the notepad
sendkey('{ENTER}'); %// send the {ENTER} key
sendkey(str2) %// send another full string to the notepad
sendkey('{! 3}'); %// note how you can REPEAT a key send instruction
sendkey('%(FA)'); %// Send key combination to open the "save as..." dialog
pause(1) %// little pause to make sure your hard drive is ready before continuing
sendkey(file2save); %// Send the name (full path) of the file to save to the dialog
sendkey('{ENTER}'); %// validate
pause(3) %// just wait a bit so you can see you file is now saved (check the titlebar of the notepad)
sendkey('%(FX)'); %// Bye bye ... close the Notepad
As explained in the Microsoft documentation the SendKeys class may have some timing issues sometimes so if you want to do complex manipulations (like Tab multiple times to change the button you actually want to press), you may have to introduce a pause in your Matlab calls to SendKeys.
Try without first, but don't forget you are managing a process from another without any synchronization between them, so timing all that can require a bit of trial and error before you get it right, at least for complex sequences (simple one should be straightforward).
In my case above for example I am running all my data from an external hard drive with an ECO function which puts it into standby, so when I called the "save as..." dialog, it takes time for it to display because the HDD has to wake up. If I didn't introduce the pause(1), sometimes the file path would be imcomplete (the first part of the path was send before the dialog had the focus).
Also, do not forget the & character when you execute the external program. All credit to Luis Mendo for highlighting it. (I tend to forget how important it is because I use it by default. I only omit it if I have to specifically wait for a return value from the program, otherwise I let it run on its own)
The special characters have a special code. Here are a few:
Shift +
Control (Ctrl) ^
Alt %
Tab {TAB}
Backspace {BACKSPACE}, {BS}, or {BKSP}
Validation {ENTER} or ~ (a tilde)
Ins Or Insert {INSERT} or {INS}
Delete {DELETE} or {DEL}
Text Navigation {HOME} {END} {PGDN} {PGUP}
Arrow Keys {UP} {RIGHT} {DOWN} {LEFT}
Escape {ESC}
Function Keys {F1} ... {F16}
Print Screen {PRTSC}
Break {BREAK}
The full list from Microsoft can be found here
There is a small javascript utility that simulates keystrokes like this on the Windows javascript interpreter.
Just create a js file with following code:
var WshShell = WScript.CreateObject("WScript.Shell");
WshShell.SendKeys(WScript.Arguments(0));
then call it from Matlab after the necessary timeout like this:
system('c:\my\js\file\script.js {Enter}');
Can't test here now, but I think this should work...
If you need to run a console-only program in a context that permits full DOS redirection, you can create a file called, say, CR.txt containing a carriage return and use the '<' notation to pipe the value into the program.
This only works if you can provide all the keyboard input can be recorded in the file. It fails dismally if the input has to vary based on responses.
An alternative is to duplicate the input (and possibly output) stream(s) for the program and then pipe data into and out of the program. This is more robust and can permit dynamic responses to the data, but will also likely require substantial effort to implement a robot user to the application.
Rog-O-Matic is an example of a large application completely controlled by a program that monitors screen output and simulates keyboard input to play an early (1980s) ASCII graphic adventure game.
The other responses will be required for GUI-based applications.
Python package pywinauto can wait any dialog and click buttons automatically. But it's capable for native and some .NET applications only. You may have problems with pressing WPF button (maybe QT button is clickable - not checked), but in such case code like app.DialogTitle.wait('ready').set_focus(); app.DialogTitle.type_keys('{ENTER}') may help. Your case is quite simple and probably some tricks with pywinauto are enough. Is your "app with popup" 64-bit or 32-bit?
wait and wait_not functions have timeout parameter. But if you need precisely listener with potentially infinite loop awaiting popups, good direction is global Windows hooks (pyHook can listen mouse and keybd events, but cannot listen dialog opening). I'll try to find my prototype that can detect new windows. It uses UI Automation API event handlers... and... ops... it requires IronPython. I still don't know how to set UI Automation handler with COM interface from standard CPython.
EDIT (2019, January): new module win32hooks was implemented in pywinauto a while ago. Example of usage is here: examples/hook_and_listen.py.

Pydev interactive IPython consoles freezes on Pandas dataframe completition?

I'm using large dataframe from Pandas in the interactive IPython console in Pydev/Eclipse. Every time I try to access dataframe attributes or methods, the console freeze for a few seconds after I type df.. Apparently it looks for completitions and maybe accidently calls some lengthy operation in the background.
Is there a way to avoid that?
You can control much of the behaviour by setting Eclipse Preferences. You can start to disable all and try re-enable things you need.
PyDev->Editor->Code Analysis
PyDev->Editor->Code Completion
What could be your problem
Request completion on '.'
Other Strategies
Disable Code Analysis within special modules by using a special
comments in source.
Disable Automatic Code Analysis and trigger it with a shortcut (show all pydev shortcut with SHIFT+CTRL+L)

Execute/Run a single line in IPython rather entire Cell

I have often this problem, when I'm slicing or subsetting data that I want to view/print [df.head()] the data and look into into it before writing next line of my code.
For this case, every time, I have to run the whole block(cell) in ipython, even if I have some logic written I had to comment that block and execute my print line alone.
Is there a feature where you can select a single line and execute it.
I often start a qtconsole attached to the kernel. You can do that as follows:
Create a new cell.
In the new cell, type %qtconsole and execute that cell.
Delete the new cell.
Once you have a qtconsole that is attached to the notebook kernel. You can print
the value of variables there.

Configure pointer focus properties of Matlab's command window

I'm running Matlab 2013a, under Linux, using Xmonad (using the XMonad.Config.Xfce package).
This problem occurs whether the command window is docked or not.
The command window prompt does not get the keyboard focus unless the pointer is located
over the command window.
Is there a way to get the Matlab command window to have focus behaviour just like other normal windows, like a terminal?
Most important: I'd like to have the keyboard focus follow the window focus,
and not require any special positioning of the pointer, so that I can just "Alt-Tab" around my windows and have the command window get the keyboard focus. All of the resources I've found so far relate to programmatic control of focus; I'm just trying to improve my user experience in an interactive session.
To get keyboard focus on the Command Window, include the following in your xmonad.hs
import XMonad.Hooks.SetWMName
import XMonad.Hooks.ManageHelpers
and configure your ManageHook as follows
myManageHook = composeAll . concat $
[ [appName =? a --> doCenterFloat | a <- myFloatAS ]
, (your other hooks)
] where
myFloatAS = ["MATLAB"]
Finally, include setWMName "LG3D" in your startupHook. See here for a full xmonad.hs configuration which uses this (this is where I found the solution). If you have other Java apps that don't get focus as they should you can add them to the myFloatAS list.
It's a problem in the built-in java.
If i run:
export MATLAB_JAVA=/usr/lib/jvm/java-7-openjdk/jre
matlab -desktop
Matlab works as expected.
I ran into this problem, running MATLAB2014a. I set up setWMName "LG3D" but still i couldn't get focus on my window. I had to click on the focused window to get the cursor, and sometimes the situation was even worse and I had to click on random places till i get my cursor back. This wouldn't happen on MATLAB2010. What worked for me was to use the native version of java as describe above.
In the end, i used the following bash script to start matlab8:
#!/bin/bash
export MATLAB_JAVA=/usr/lib/jvm/java-7-openjdk-amd64/jre/
/usr/local/bin/matlab8 -desktop -nosplash

pydev Ipython console freezing when paging

If in the pydev 2.5 with Ipython console 0.11 I write:
import numpy
numpy?
I get the documentation for numpy and as it's longer than the screen it gets paged showing the message:
---Return to continue, q to quit---
However no matter what keys I press it'll stay there. typically I'll have to restart the console to continue.
Is this a bug?
It seems that the console is not sending the key press to actually perform the paging.
A workaround is to redefine the ipython paging function to never actually page.
Add the following code to the 'Initial interpreter commands' (PyDev -> Interactive Console)
from IPython.core import page
def nopage(strng, start=0, screen_lines=0, pager_cmd=None):
print(strng)
page.page = nopage