I am looking for a way to save 'everything' in the matlab session when it is stopped for debugging.
Example
function funmain
a=1;
if a>1
funsub(1)
end
funsub(2)
end
function funsub(c)
b = c + 1;
funsubsub(c)
end
function funsubsub(c)
c = c + 2; %Line with breakpoint
end
When I finally reach the line with the breakpoint, I can easily navigate all workspaces and see where all function calls are made.
The question
How can I preserve this situation?
When debugging nested programs that take a long time to run, I often find myself waiting for a long time to reach a breakpoint. And sometimes I just have to close matlab, or want to try some stuff and later return to this point, so therefore finding a way to store this state would be quite desirable. I work in Windows Server 2008, but would prefer a platform independant solution that does not require installation of any software.
What have I tried
1. Saving all variables in the workspace: This works sometimes, but often I will also need to navigate other workspaces
2. Saving all variables in the calling workspace: This is already better as I can run the lowest function again, but may still be insufficient. Doing this for all nested workspaces is not very convenient, and navigating the saved workspaces may be even worse.
Besides the mentioned inconveniences, this also doesn't allow me to see the exact route via which the breakpoint is reached. Therefore I hope there is a better solution!
Code structure example
The code looks a bit like this
function fmain
fsub1()
fsub2()
fsub3()
end
function fsub1
fsubsub11
fsubsub12
...
fsubsub19
end
function fsub2
fsubsub21
fsubsub22
...
fsubsub29
end
function fsub3
fsubsub31
fsubsub32
...
fsubsub39
end
function fsubsub29
fsubsubsub291
fsubsubsub292% The break may occur in here
...
fsubsubsub299
The break can of course occur anywhere, and normally I would be able to navigate the workspace and all those above it.
Checkpointing
What you're looking to implement is known as checkpointing code. This can be very useful on pieces of code that run for a very long time. Let's take a very simple example:
f=zeros(1e6,1);
for i=1:1e6
f(i) = g(i) + i*2+5; % //do some stuff with f, not important for this example
end
This would obviously take a while on most machines so it would be a pain if it ran half way, and then you had to restart. So let's add a checkpoint!
f=zeros(1e6,1);
i=1; % //start at 1
% //unless there is a previous checkpoint, in which case skip all those iterations
if exist('checkpoint.mat')==2
load('checkpoint.mat'); % //this will load f and i
end
while i<1e6+1
f(i) = g(i) + i*2+5;
i=i+1;
if mod(i,1000)==0 % //let's save our state every 1000 iterations
save('checkpoint.mat','f','i');
end
end
delete('checkpoint.mat') % //make sure to remove it when we're done!
This allows you to quit your code midway through processing without losing all of that computation time. Deciding when and how often to checkpoint is the balance between performance and lost time!
Sample Code Implementation
Your Sample code would need to be updated as follows:
function fmain
sub1done=false; % //These really wouldn't be necessary if each function returns
sub2done=false; % //something, you could just check if the return exists
sub3done=false;
if exist('checkpoint_main.mat')==2, load('checkpoint_main.mat');end
if ~sub1done
fprintf('Entering fsub1\n');
fsub1()
fprintf('Finished with fsub1\n');
sub1done=true;
save('checkpoint_main.mat');
end
if ~sub2done
fprintf('Entering fsub2\n');
fsub2()
fprintf('Finished with fsub2\n');
sub2done=true;
save('checkpoint_main.mat');
end
if ~sub3done
fprintf('Entering fsub3\n');
fsub3()
fprintf('Finished with fsub3\n');
sub3done=true;
save('checkpoint_main.mat');
end
delete('checkpoint_main.mat');
end
function fsub2
subsub21_done=false;subsub22_done=false;...subsub29_done=false;
if exist('checkpoint_fsub2')==2, load('checkpoint_fsub2');end
if ~subsub21_done
fprintf('\tEntering fsubsub21\n');
fsubsub21
fprintf('\tFinished with fsubsub21\n');
subsub21_done=true;
save('checkpoint_fsub2.mat');
end
...
if ~subsub29_done
fprintf('\tEntering fsubsub29\n');
fsubsub29
fprintf('\tFinished with fsubsub29\n');
subsub29_done=true;
save('checkpoint_fsub2.mat');
end
delete('checkpoint_fsub2.mat');
end
function fsubsub29
subsubsub291_done=false;...subsubsub299_done=false;
if exist('checkpoint_fsubsub29.mat')==2,load('checkpoint_fsubsub29.mat');end
if ~subsubsub291_done
fprintf('\t\tEntering fsubsubsub291\n');
fsubsubsub291
fprintf('\t\tFinished with fsubsubsub291\n');
subsubsub291_done=true;
save('checkpoint_fsubsub29.mat');
end
if ~subsubsub292_done
fprintf('\t\tEntering fsubsubsub292\n');
fsubsubsub292% The break may occur in here
fprintf('\t\tFinished with fsubsubsub292\n')
subsubsub292_done=true;
save(checkpoint_fsubsub29.mat');
end
delete('checkpoint_fsubsub29.mat');
end
So in this structure if you restarted the program after it was killed it would resume back to the last saved checkpoint. So for example if the program died in subsubsub291, the program would skip fsub1 altogether, just loading the result. And then it would skip subsub21 all the way down to subsub29 where it would enter subsub29. Then it would skip subsubsub291 and enter 292 where it left off, having loaded all of the variables in that workspace and in previous workspaces. So if you backed out of 292 into 29 you would have the same workspace as if the code just ran. Note that this will also print a nice tree structure as it enters and exits functions to help debug execution order.
Reference:
https://wiki.hpcc.msu.edu/pages/viewpage.action?pageId=14781653
After a bit of googling, I found that using putvar (custom function from here: http://au.mathworks.com/matlabcentral/fileexchange/27106-putvar--uigetvar ) solved this.
Related
I frequently find myself in the situation, where I insert a lot of keyboard commands into my code to debug it. However, I would like to have a bit more flexibility. That's why I started writing my stahp function.
function stahp()
disp('Stahp requested.');
dbstack
disp('You can either');
disp(' 1 abort.');
disp(' 2 continue.');
disp(' 3 debug.');
in = input('Choose what to do:\n');
switch(in)
case 1
error('Stahp.');
case 2
disp('Continue.')
case 3
disp('Debug.');
keyboard % <------------------------------------ Here is my problem
otherwise
stahp();
end
end
The idea is, to let the user choose, what he wants to do (continue, abort, debug, maybe something else in the future). However, I would prefer to not start the debug mode inside the stahp function, but right after stepping out.
For example, when running
function test_stahp
a = 1
stahp()
b = 2
end
I would like to enter the debug mode right before b=2. I assume, that dbstep out can somehow be used, but the way I tried it so far, you still have to step manually out of stahp(). I am also aware, that the recursive call of stahp() in otherwise might complicate things, but I could remove that part.
Any help is very much appreciated. Thank you.
You can use dbstack to get the current stack and then retrieve the name of the calling function and line number that called stahp. Using this info, you can use the dbstop to create a breakpoint in the function that performed the function call. The code sample below sets the breakpoint in test_stahp on the b=2 line.
function stahp()
disp('Stahp requested.');
dbstack
disp('You can either');
disp(' 1 abort.');
disp(' 2 continue.');
disp(' 3 debug.');
in = input('Choose what to do:\n');
r=dbstack;
switch(in)
case 1
dbclear(r(2).name,num2str(r(2).line+1));
error('Stahp.');
case 2
dbclear(r(2).name,num2str(r(2).line+1));
disp('Continue.')
case 3
disp('Debug.');
dbstop(r(2).name,num2str(r(2).line+1))
otherwise
stahp();
end
end
When an out-of-memory error is raised in a parfor, is there any way to kill only one Matlab slave to free some memory instead of having the entire script terminate?
Here is what happens by default when an out-of-memory error occurs in a parfor: the script terminated, as shown in the screenshot below.
I wish there was a way to just kill one slave (i.e. removing a worker from parpool) or stop using it to release as much memory as possible from it:
If you get a out of memory in the master process there is no chance to fix this. For out of memory on the slave, this should do it:
The simple idea of the code: Restart the parfor again and again with the missing data until you get all results. If one iteration fails, a flag (file) is written which let's all iterations throw an error as soon as the first error occurred. This way we get "out of the loop" without wasting time producing other out of memory.
%Your intended iterator
iterator=1:10;
%flags which indicate what succeeded
succeeded=false(size(iterator));
%result array
result=nan(size(iterator));
FLAG='ANY_WORKER_CRASHED';
while ~all(succeeded)
fprintf('Another try\n')
%determine which iterations should be done
todo=iterator(~succeeded);
%initialize array for the remaining results
partresult=nan(size(todo));
%initialize flags which indicate which iterations succeeded (we can not
%throw erros, it throws aray results)
partsucceeded=false(size(todo));
%flag indicates that any worker crashed. Have to use file based
%solution, don't know a better one. #'
delete(FLAG);
try
parfor falseindex=1:sum(~succeeded)
realindex=todo(falseindex);
try
% The flag is used to let all other workers jump out of the
% loop as soon as one calculation has crashed.
if exist(FLAG,'file')
error('some other worker crashed');
end
% insert your code here
%dummy code which randomly trowsexpection
if rand<.5
error('hit out of memory')
end
partresult(falseindex)=realindex*2
% End of user code
partsucceeded(falseindex)=true;
fprintf('trying to run %d and succeeded\n',realindex)
catch ME
% catch errors within workers to preserve work
partresult(falseindex)=nan
partsucceeded(falseindex)=false;
fprintf('trying to run %d but it failed\n',realindex)
fclose(fopen(FLAG,'w'));
end
end
catch
%reduce poolsize by 1
newsize = matlabpool('size')-1;
matlabpool close
matlabpool(newsize)
end
%put the result of the current iteration into the full result
result(~succeeded)=partresult;
succeeded(~succeeded)=partsucceeded;
end
After quite bit of research, and a lot of trial and error, I think I may have a decent, compact answer. What you're going to do is:
Declare some max memory value. You can set it dynamically using the MATLAB function memory, but I like to set it directly.
Call memory inside your parfor loop, which returns the memory information for that particular worker.
If the memory used by the worker exceeds the threshold, cancel the task that worker was working on. Now, here it get's a bit tricky. Depending on the way you're using parfor, you'll either need to delete or cancel either the task or worker. I've verified that it works with the code below when there is one task per worker, on a remote cluster.
Insert the following code at the beginning of your parfor contents. Tweak as necessary.
memLimit = 280000000; %// This doesn't have to be in parfor. Everything else does.
memData = memory;
if memData.MemUsedMATLAB > memLimit
task = getCurrentTask();
cancel(task);
end
Enjoy! (Fun question, by the way.)
One other option to consider is that since R2013b, you can open a parallel pool with 'SpmdEnabled' set to false - this allows MATLAB worker processes to die without the whole pool being shut down - see the doc here http://www.mathworks.co.uk/help/distcomp/parpool.html . Of course, you still need to arrange somehow to shutdown the workers.
I have a loop that has a command I want to execute at a specific time, say 3 seconds from now. But I don't want to add a delay because I want my rest of the statements and the loop to execute continuously without delay. Is there any way I can store statements in a buffer to execute at a required time?
You need to set up a timer object. The "Display message using Timer"-Example shows exactly what you need.
Have you tried tic and toc? Like this:
tic
executeFlag = 1;
while(1) % Your loop here.
if (toc > 3) && (executeFlag)
timedThing(); % The thing to run on a timer.
executeFlag = 0;
end
everythingElse(); % Everything else you need to do.
end
It's not pretty, but it will do the job. You can remove the executeFlag if you want the timed thing to run after a certain time.
Let me know if this doesn't do it for you, and I'll take another shot at it.
I would like to be able to terminate my current running scripts( functions ) by calling a command in the code. Return would only terminate the current function not entire script. Therefore return is not the one.
What I am looking for is a command which does exactly what CTRL + C do.
I have already seen this: how to stop execution and noticed that no one has yet provided a proper answer for this question in there either.
ultimately I want to terminate the entire running scripts upon closing a figure:
hFig = figure('CloseRequestFcn',{#closeHandler});
.
.
.
function closeHandler (src,evnt)
CTRL+C <--- I am looking for such a command
end
PS. function error() will not work either: Try this:
function terminateInCode()
hFig = figure('CloseRequestFcn',{#closeHandler});
while(1)
plot(10*rand,10*rand,'+');
pause(0.1);
end;
function closeHandler (src,evnt)
delete(hFig);
error('program terminated!');
end
end
Here is a sample function with example based on yuk's answer. Components include:
Insure the command window has focus to receive the CTRL+C
Use a timer to release CTRL+C after the break has occurred
Use a Java robot to press CTRL+C
Sample function is below:
function terminateExecution
%terminateExecution Emulates CTRL-C
% terminateExecution Stops operation of a program by emulating a
% CTRL-C press by the user.
%
% Running this function
%
%Example:
%for ix = 1:100
% disp(ix)
% if ix>20
% terminateExecution;
% end
%end
%1) request focus be transferred to the command window
% (H/T http://undocumentedmatlab.com/blog/changing-matlab-command-window-colors/)
cmdWindow = com.mathworks.mde.cmdwin.CmdWin.getInstance();
cmdWindow.grabFocus();
%2) Wait for focus transfer to complete (up to 2 seconds)
focustransferTimer = tic;
while ~cmdWindow.isFocusOwner
pause(0.1); %Pause some small interval
if (toc(focustransferTimer) > 2)
error('Error transferring focus for CTRL+C press.')
end
end
%3) Use Java robot to execute a CTRL+C in the (now focused) command window.
%3.1) Setup a timer to relase CTRL + C in 1 second
% Try to reuse an existing timer if possible (this would be a holdover
% from a previous execution)
t_all = timerfindall;
releaseTimer = [];
ix_timer = 1;
while isempty(releaseTimer) && (ix_timer<= length(t_all))
if isequal(t_all(ix_timer).TimerFcn, #releaseCtrl_C)
releaseTimer = t_all(ix_timer);
end
ix_timer = ix_timer+1;
end
if isempty(releaseTimer)
releaseTimer = timer;
releaseTimer.TimerFcn = #releaseCtrl_C;
end
releaseTimer.StartDelay = 1;
start(releaseTimer);
%3.2) Press press CTRL+C
pressCtrl_C
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
function pressCtrl_C
import java.awt.Robot;
import java.awt.event.*;
SimKey=Robot;
SimKey.keyPress(KeyEvent.VK_CONTROL);
SimKey.keyPress(KeyEvent.VK_C);
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
function releaseCtrl_C(ignore1, ignore2)
import java.awt.Robot;
import java.awt.event.*;
SimKey=Robot;
SimKey.keyRelease(KeyEvent.VK_CONTROL);
SimKey.keyRelease(KeyEvent.VK_C);
Not sure it will work, just an idea. How about to emulate keyboard key press from MATLAB?
You can try either java.awd.Robot:
import java.awt.Robot;
import java.awt.event.*;
SimKey=Robot;
SimKey.keyPress(KeyEvent.VK_CONTROL);
SimKey.keyPress(KeyEvent.VK_C);
or WScript.Shell and SendKeys.
Unfortunately, it seems it cannot be done:
Mathworks
There is no way to programmatically issue a Ctrl+C in MATLAB besides using the keyboard's Ctrl+C combination.
As an alternative, you can use the ERROR command to force an error that will exit the code. For example:
error('Program terminated for a specific reason')
Here's an alternative that uses undocumented Matlab calls to place the key event directly into the command window. The method to do so is protected; this uses reflection to unprotect it.
Unlike #yuk and #Persuit's answers, this does not seem to have issues with the control key sticking. Additionally, it will always post directly to the command window without any race conditions or other issues of ensuring focus. And, I think that it fires deterministically -- it'll execute immediately.
The one caveat is that it uses an undocumented call to retreive the handle of the command window instance. This varies slightly by release as it is dependent upon the window frame layout. Some of Yair Altman's (undocumentedmatlab.com) work on the file exchange has more robust functions to grab this in a more general fashion; this code should work with most modern releases of Matlab (Tested on R2011a, both Mac & Win).
function interrupt
import java.awt.event.KeyEvent
import java.util.Calendar
import java.lang.reflection.*
cmdwin = handle(com.mathworks.mde.cmdwin.CmdWin.getInstance().getComponent(0).getComponent(0).getComponent(0),'CallbackProperties');
argSig = javaArray('java.lang.Class',1);
argSig(1) = java.lang.Class.forName('java.awt.event.KeyEvent');
method = cmdwin.getClass().getDeclaredMethod('processKeyEvent',argSig);
method.setAccessible(true);
cal = Calendar.getInstance();
args = javaArray('java.lang.Object',1);
args(1) = KeyEvent(cmdwin,KeyEvent.KEY_PRESSED,cal.getTime().getTime(),KeyEvent.CTRL_DOWN_MASK,KeyEvent.VK_C,KeyEvent.CHAR_UNDEFINED);
method.invoke(cmdwin,args);
You can use the function error. You will go back to matlab.
It will produce an error, but that is also what usually happen when you press CTRL+C, in a matlab script.
You should add some kind of message like error('Interrupted by user');
It is not exactly what you are asking for, but considering your example, your problem can be solved like that:
function terminateInCode()
hFig = figure('CloseRequestFcn',{#closeHandler});
stop=0;
while(~stop)
plot(10*rand,10*rand,'+');
pause(0.1);
end;
function closeHandler (src,evnt)
delete(hFig);
stop=1;
end
end
Try the return statement. It will push you out of a function.
If you want to terminate it completely you need to use ERROR.
You could always use EXIT if its really that disastrous.
I have a matlab script which executes 5 algorithm sequentially. All these 5 algorithms needs to run for 10 different initialization.
Whenever there is an error in i-th initialization, the script exit with an error message. I fix the issue(say, data issue) and start running the script again which executes from the first initialization.
I dont want to my code to run for previously executed initialization. ( from 1 run to i-1 the run)
One way is to reassign the value of index to start from i, which in turn require to modify the scrip everytime again and again.
Is there any way to restart the script from the i-th initialization onwards which dont require to modify the script?
I suggest that you use try and catch, and check which indexes succeeded.
function errorIndexes = myScript(indexes)
errorIndexes = [];
errors = {};
for i = indexes
try
%Do something
catch me
errorIndexes(end+1) = i;
errors{end+1} = me;
end
end
end
On the outside you should have a main file like that:
function RunMyScript()
if exist('unRunIndexes.mat','file')
unRunIndexes= load('unRunIndexes.mat');
else
unRunIndexes= 1:n;
end
unRunIndexes= myScript( indexes)
save('unRunIndexes.mat',unRunIndexes);
end
Another technique you might like to consider is checkpointing. I've used something similar with long-running (more than one day) loops operating in an environment where the machine could become unavailable at any time, e.g. distributed clusters of spare machines in a lab.
Basically, you check to see if a 'checkpoint' file exists before you start your loop. If it does than that suggests the loop did not finish successfully last time. It contains information on where the loop got up to as well as any other state that you need to get going again.
Here's a simplified example:
function myFunction()
numIter = 10;
startIter = 1;
checkpointFilename = 'checkpoint.mat';
% Check for presence of checkpoint file suggesting the last run did not
% complete
if exist(checkpointFilename, 'file')
s = load(checkpointFilename);
startIter = s.i;
fprintf('Restarting from iteration %d\n', startIter);
end
for i = startIter:numIter
fprintf('Starting iteration %d\n', i);
expensiveComputation();
save(checkpointFilename, 'i');
end
% We succefully finished. Let's delete our checkpoint file
delete(checkpointFilename);
function expensiveComputation()
% Pretend to do lots of work!
pause(1);
end
end
Running and breaking out with ctrl-c part way through looks like this:
>> myFunction
Starting iteration 1
Starting iteration 2
Starting iteration 3
Starting iteration 4
Operation terminated by user during myFunction/expensiveComputation (line 27)
In myFunction (line 18)
expensiveComputation();
>> myFunction
Restarting from iteration 4
Starting iteration 4
Starting iteration 5
...
You can type (in the command line):
for iter=l:n,
%%% copy - paste your code inside the loop
end