I've tried to set a maximum execution time for matlab function, but none of them succeeded. I have a matlab file loading a model setting from COMSOL (using COMSOL with Matlab livelink).
%filename ‘mymodel.m'
fff = 0;
store{fff+1,1} = 'out1';
store{fff+1,2} = 'out2';
for i = 1:2
for j = 1:2
fff = fff+1;
%load Model parameter setting
model_parameter_script_file
if i == 1;model.param.set('x1',val1);end
if i == 2;model.param.set('x1',val2);end
if j == 1;model.param.set('x2',val1);end
if j == 2;model.param.set('x2',val2);end
try
%load model solution script from COMSOL
model_soulution_script_file
%store table calculated from the solution into 'store'
%catch null values
catch
store{fff+1,1} = 'null';
store{fff+1,2} = 'null';
end
end
end
The issue is that I cannot figure out a way to tell the computer to terminate the for loop if anyone one of them exceeds the maximum execution time. Is it possible to do with Matlab Parallel Computing Toolbox? This could be used to accelerate program execution, but not sure if it can kill a thread when it finds that particular job exceeds the maximum running time. Thanks a lot for your help in advance.
Related
I am simulating a Dual Active Bridge with a digital control system in PLECS (similar to Simulink). What I want to do is to run a sequence of simulations where I change two variables related to the controller(Kp and Ki), to do this parameter sweep I have to create a simulation script in Octave. I have followed the official tutorial (https://www.youtube.com/watch?v=RB3EdvIE7ds&ab_channel=Plexim), however I didn't manage to achieve an appropriate sweep of the variables.
I expect an output result from majority of the parameter sweeps, however I don't get an output from most of them*. The issues I am seeing:
Some values aren't imported correctly.
Sometimes are real-time result presents an output waveform, however when the trace is automatically saved it disappears.
What I have tried:
I have tried changing the initialization value of Kp and Ki to 0, then to 1. This resulted in invalid results where the system essentially does nothing.
Changing the data in the .csv file to be in general format rather than scientific.
What I think is happening:
the "plecs('simulate', ...)" commands are running individually causing 2 separate simulations to be executed. I don't know how I would solve this issue, but would I have to combine the Kp and Ki structures together and use that as an argument of "plecs('simulate', ...)" function?
mdl = plecs('get', '', 'CurrentCircuit');
scopepath = [mdl '/Output'];
% clear all previous traces in scope 'Scope' in the current model
plecs('scope', scopepath, 'ClearTraces');
potentialStructure = struct('Kp', 0.0000819); %default value KP
integralStructure = struct('Ki', -0.0000505); %default value Ki
potStructure = struct('ModelVars', potentialStructure); %KP simStruct with 'ModelVars
intStructure = struct('ModelVars', integralStructure); %KI simStruct with 'ModelVars
Kpd = dlmread ('Digital Controller PI Values.csv',',', 'A2:A194'); %read first column
Kid = dlmread ('Digital Controller PI Values.csv',',', 'B2:B194'); %read second column
for ix = 1:length(Kpd) %how many loops - same as number of rows
potStructure.ModelVars.Kp = Kpd(ix); %set value of KP
intStructure.ModelVars.Ki = Kid(ix); %set value of KI
plecs('simulate', potStructure); %override the Kp value in the model during each simulation run
plecs('simulate', intStructure); %override the Ki value in the model during each simulation run
%hold and label trace
plecs('scope', scopepath, 'HoldTrace', ['Kp = ' mat2str(Kpd(ix)) ' Ki = ' mat2str(Kid(ix))]);
end
[Simulation Results, Code, Data][1]
*The invalid results are the ones that can be seen as a straight line at 0 in the image attached
[1]: https://i.stack.imgur.com/SXHfW.png
The issue lied how I was using the struct() function. I could set multiple variables and with set value to the single struct. This in turn solved my issue where I was executing 2 simulations in a single loop.
mdl = plecs('get', '', 'CurrentCircuit');
scopepath = [mdl '/Output'];
% clear all previous traces in scope 'Scope' in the current model
plecs('scope', scopepath, 'ClearTraces');
mdlVars = struct('Kp', 0.0000819, 'Ki', -0.0000505 ); %default value KP
simStruct = struct('ModelVars', mdlVars); %KP simStruct with 'ModelVars
Kpd = dlmread ('Digital Controller PI Values.csv',',', 'A2:A194'); %read first column
Kid = dlmread ('Digital Controller PI Values.csv',',', 'B2:B194'); %read second column
for ix = 1:length(Kpd) %how many loops - same as number of rows
simStruct.ModelVars.Kp = Kpd(ix); %set value of KP
simStruct.ModelVars.Ki = Kid(ix); %set value of KI
plecs('simulate', simStruct); %override the Kp value in the model during each simulation run
%hold and label trace
plecs('scope', scopepath, 'HoldTrace', ['Kp = ' mat2str(Kpd(ix)) ' Ki = ' mat2str(Kid(ix))]);
end
I created a MATLAB code using Psychtoolbox to make an experiment.
Everything works as I intended but it seems the initial loading of the experiment takes too long. The task is a simple yes/no response task whether the target word('probe') appeared in the previous set of word stimuli.
I put basic intro text as an image and then wait for any keypress to start the experiment but it will take about 40 seconds to actually begin the first trial after any keystroke. I want to make it work without any delay. It should start its first trial immediately after any keystroke.
I checked the timestops with GetSecs() on numerous positions in the code and it was not anything to do with loading stimuli or initial setting of the experiment before the for loop I attached below.
To make things look simpler, I changed some of the variables into actual numbers I used. I can gurantee that it is not due to large stimuli size since it is only 1500 words. Once the for loop starts, it goes smoothly but it takes 40 seconds to actually start the first trial so I think it is something to do with a specific function in the for loop or the way I built it.
Please let me know if anything is too vague or unclear. I will do my best to make things read better.
Edit: I minimalized the code leaving only the function names used in Psychtoolbox. I left the functions I used in between loops to let you know if they could cause any delay. It will not be possible to run this without Psychtoolbox installed so I guess you can briefly examine the structure of the code.
for trial = 1:250
for i = 1:6
DrawFormattedText();
Screen();
WaitSecs(0.5);
end
DrawFormattedText();
flipTime = Screen();
WaitSecs(0.5);
DrawFormattedText();
flipTime = Screen();
rt = 0;
resp = 0;
while GetSecs - flipTime < 3
clear keyCode;
RestrictKeysForKbCheck();
[keyIsDown,secs,keyCode] = KbCheck;
respTime = GetSecs;
pressedKeys = find(keyCode);
% ESC key quits the experiment
if keyCode(KbName('ESCAPE')) == 1
clear all
close all
sca
return
end
% Check for response keys
if ~isempty(pressedKeys)
for i = 1:2
if KbName(i) == pressedKeys(1)
resp = i;
rt = respTime - flipTime;
end
end
end
% Exit loop once a response is recorded
if rt > 0
break;
end
end
if rt == 0 || rt > 3 % 3 second limit for subjects to react to the probe stimuli
DrawFormattedText();
Screen();
WaitSecs(1);
end
Screen();
vbl = Screen();
WaitSecs(1);
% Record the trial data into output data matrix
respMat{1, trial} = trial;
respMat{2, trial} = resp;
respMat{3, trial} = rt;
end
So I have a matlab.m file script. When the file runs. It generates a vector. I want to save that vector and rerun the script all over again. How do I put a loop on the entire script file and create a vector_{i}, where the index enters the name of the file? I would post the code but it wont work without the data on my desktop.
[data,labels]=xlsread('C:\Users\Hilbert\Desktop\matlab\matlabdata_abe.xlsx');
gdp=log(data(:,1)./lagmatrix(data(:,1),1)) %GDP
ip=log(data(:,2)./lagmatrix(data(:,2),1)) %IP
tnx=data(:,3) %TNX
m2=log(data(:,4)./lagmatrix(data(:,4),1)) %M2
cpi=log(data(:,5)./lagmatrix(data(:,5),1)) %CPI
ffed=log(data(:,6)./lagmatrix(data(:,6),1)) %FedFund
Dgdp=gdp
inflation=cpi
Dm2=m2
ffr_=ffed
data=[Dgdp(54:length(cpi)), inflation(54:length(cpi)), Dm2(54:length(cpi)), ffr_(54:length(cpi)) ];
data_L1=lagmatrix(data,1)
data_L2=lagmatrix(data,2)
data_L3=lagmatrix(data,3)
data_L4=lagmatrix(data,4)
mat=[ones(1,size(data_L1',2));data_L1';data_L2';data_L3';data_L4']
mat=mat(:,5:end)
X=[data';data_L1';data_L2';data_L3']
X=X(:,5:end)
mat=mat';
X=X'
Fhat=(inv(mat'*mat) * mat'*X)';
nobs=size(data,1)
p=4
yhat= mat*Fhat'
yhat=yhat(:,1:4)
data_sample=data(5:nobs,:)
res=data_sample - yhat
res_{loopindexnumber}=res %saves the vector and re-runs the entire cost again the idea is to bootstrap the data by running many simulations and saving the residual vector
Make the script a function. And then execute the function in a loop how many times you want. For example:
function res = my_function(k)
% your script goes here.
% the function is saved in my_function.m file
% some calucations producing return_vector using k parmeter
res = return_vector
Later on, just run a for loop over the function and store the results to a cell array:
for k = 1:10
A{k} = my_function(k)
end
Make the script a function. And then execute the function in a loop how many times you want. For example:
function res = my_function(k)
% your script goes here.
% the function is saved in my_function.m file
% some calucations producing return_vector using k parmeter
res = return_vector
Later on, just run a for loop over the function.
for k = 1:10
assignin('base', ['A_', num2str(k)], my_function(k))
end
I run a lot of long simulations in Matlab, typically taking from a couple of minutes to a couple of hours, so to speed things up I decided to run the simulations simultaneously using a parfor loop.
arglist = [arg1, arg2, arg3, arg4];
parfor ii = 1:size(arglist, 2)
myfun(arglist(ii));
end
Everything worked just fine, except for one thing: the progress printing. Since each simulation takes a lot of time, I usually print progress using something like
prevlength = 0;
for ii = 1:tot_iter
% Some calculations here
msg = sprintf('Working on %d of %d, %.2f percent done', ii, tot_iter, ii/tot_iter);
fprintf(repmat('\b', 1, prevlength))
fprintf(msg);
prevlength = numel(msg);
end
but, as could be expected, when doing this inside a parfor loop, you get chaos.
I have googled a lot in search of a solution and have found a bunch of "parfor progress printers" like this one. However, all of them print the progress of the entire parfor loop instead of showing how far each of the individual iterations have come. Since I only have about 4-8 iterations in the parfor loop, but each iteration takes about an hour, this approach isn't very helpful to me.
The ideal solution for me would be something that looks like this
Working on 127 of 10000, 1.27 percent done
Working on 259 of 10000, 2.59 percent done
Working on 3895 of 10000, 38.95 percent done
Working on 1347 of 10000, 13.47 percent done
that is, each simulation gets one line showing how far it has run. I'm not sure though if this is possible at all, I can at least not imagine any way to do this.
Another way would be to do something like this
Sim 1: 1.27% Sim 2: 2.59% Sim 3: 38.95% Sim 4: 13.47%
that is, show all the progresses on the same line. To do this, you would need to keep track of what position on the line each simulation is to write on and write there, without erasing the other progresses. I can't figure out how this would be done, is this possible to do?
If there is some other solution to my problem (showing the progress of each individual iteration) that I haven't thought of, I'd be happy to hear about it.
Since this is the first time I ask a question here on SO it is quite possible that there is something that I have missed; if so, please feel free to comment below.
Edit
After receiving this answer, I thought that I should share how I used it to solve my problem since I didn't use it exactly as in the answer, in case someone else encounters the same problem.
Here is a small test program with basically the same structure as my program, making use of the progress bar (parfor_progress) mentioned in the answer:
function parfor_progress_test()
cpus = feature('numCores');
matlabpool('open', cpus);
cleaner = onCleanup(#mycleaner);
args = [1000, 1000, 1000, 1000];
m = sum(args);
parfor_progress(m);
parfor ii = 1:size(args,2)
my_fun(args(ii));
end
parfor_progress(0);
end
function my_fun(N)
for ii = 1:N
pause(rand*0.01);
parfor_progress;
end
end
function mycleaner
matlabpool close;
fclose all;
end
Simple Progress Bar
Something like a progress bar could be done similar to this...
Before the parfor loop :
fprintf('Progress:\n');
fprintf(['\n' repmat('.',1,m) '\n\n']);
And during the loop:
fprintf('\b|\n');
Here we have m is the total number of iterations, the . shows the total number of iterations and | shows the number of iterations completed. The \n makes sure the characters are printed in the parfor loop.
Progress Bar and Percentage Completion
Otherwise you could try this: http://www.mathworks.com/matlabcentral/fileexchange/32101-progress-monitor--progress-bar--that-works-with-parfor
It will display a progress bar and percentage completion but can be easily modified to just include the percentage completion or progress bar.
This function amends a character to a file on every iteration and then reads the number of characters written to that file which indicates the number of iterations completed. This file accessing method is allowed in parfor's.
Assuming you correctly add the above to your MATLAB path somehow, you can then use the following:
arglist = [arg1, arg2, arg3, arg4];
parfor_progress(size(arglist, 2)); % Set the total number of iterations
parfor ii = 1:size(arglist, 2)
myfun(arglist(ii));
parfor_progress; % Increment the progress counter
end
parfor_progress(0); % Reset the progress counter
Time to Completion and Percentage Completion
There is also a function called showTimeToCompletion() which is available from: https://www.soundzones.com/software/sound-zone-tools/
and works alongside parfor_progress. This function allows you to print a detailed summary of the progress of a parfor loop (or any loop for that matter) which contains the start time, length of time running, estimated finish time and percentage completion. It makes smart use of the \b (backspace) character so that the command window isn't flooded with text. Although not strictly a progress 'bar' it is perhaps more informative.
The third example in the header of the function file,
fprintf('\t Completion: ');
showTimeToCompletion; startTime=tic;
len=1e2;
p = parfor_progress( len );
parfor i = 1:len
pause(1);
p = parfor_progress;
showTimeToCompletion( p/100, [], [], startTime );
end
outputs the following to the command window:
Completion: 31.00%
Remaining: 00:00:23
Total: 00:00:33
Expected Finish: 3:30:07PM 14-Nov-2017
Useful for estimating the completion of a running simulation, especially one that may take hours or days.
Starting in R2013b, you can use PARFEVAL to evaluate your function asynchronously and have the client display progress updates. (Obviously this approach is not quite as simple as adding stuff to your PARFOR loop). There's an example here.
The Diary property of the Future returned by PARFEVAL is updated continuously while processing, so that might also be useful if you have a small number of large tasks.
Starting in R2017a, you can use parallel.pool.DataQueue and afterEach to implement a waitbar for parfor, like so:
if isempty(gcp('nocreate'))
parpool('local', 3);
end
dq = parallel.pool.DataQueue;
N = 10;
wb = waitbar(0, 'Please wait...');
% Use the waitbar's UserData to track progress
wb.UserData = [0 N];
afterEach(dq, #(varargin) iIncrementWaitbar(wb));
afterEach(dq, #(idx) fprintf('Completed iteration: %d\n', idx));
parfor idx = 1:N
pause(rand());
send(dq, idx);
end
close(wb);
function iIncrementWaitbar(wb)
ud = wb.UserData;
ud(1) = ud(1) + 1;
waitbar(ud(1) / ud(2), wb);
wb.UserData = ud;
end
after exploring #Edric answer I found that there is an example in the Matlab documentation that exactly implements a waitbar for a pareval loop. Check help FetchNext
N = 100;
for idx = N:-1:1
% Compute the rank of N magic squares
F(idx) = parfeval(#rank, 1, magic(idx));
end
% Build a waitbar to track progress
h = waitbar(0, 'Waiting for FevalFutures to complete...');
results = zeros(1, N);
for idx = 1:N
[completedIdx, thisResult] = fetchNext(F);
% store the result
results(completedIdx) = thisResult;
% update waitbar
waitbar(idx/N, h, sprintf('Latest result: %d', thisResult));
end
% Clean up
delete(h)
I am running a parfor loop in Matlab that takes a lot of time and I would like to know how many iterations are left. How can I get that info?
I don't believe you can get that information directly from MATLAB, short of printing something with each iteration and counting these lines by hand.
To see why, recall that each parfor iteration executes in its own workspace: while incrementing a counter within the loop is legal, accessing its "current" value is not (because this value does not really exist until completion of the loop). Furthermore, the parfor construct does not guarantee any particular execution order, so printing the iterator value isn't helpful.
cnt = 0;
parfor i=1:n
cnt = cnt + 1; % legal
disp(cnt); % illegal
disp(i); % legal ofc. but out of order
end
Maybe someone does have a clever workaround, but I think that the independent nature of the parfor iterations belies taking a reliable count. The restrictions mentioned above, plus those on using evalin, etc. support this conclusion.
As #Jonas suggested, you could obtain the iteration count via side effects occurring outside of MATLAB, e.g. creating empty files in a certain directory and counting them. This you can do in MATLAB of course:
fid = fopen(['countingDir/f' num2str(i)],'w');
fclose(fid);
length(dir('countingDir'));
Try this FEX file: http://www.mathworks.com/matlabcentral/fileexchange/32101-progress-monitor--progress-bar--that-works-with-parfor
You can easily modify it to return the iteration number instead of displaying a progress bar.
Something like a progress bar could be done similar to this...
Before the parfor loop :
fprintf('Progress:\n');
fprintf(['\n' repmat('.',1,m) '\n\n']);
And during the loop:
fprintf('\b|\n');
Here we have m is the total number of iterations, the . shows the total number of iterations and | shows the number of iterations completed. The \n makes sure the characters are printed in the parfor loop.
With Matlab 2017a or later you can use a data queue or a pollable data queue to achieve this. Here's the MathWorks documentation example of how to do a progress bar from the first link :
function a = parforWaitbar
D = parallel.pool.DataQueue;
h = waitbar(0, 'Please wait ...');
afterEach(D, #nUpdateWaitbar);
N = 200;
p = 1;
parfor i = 1:N
a(i) = max(abs(eig(rand(400))));
send(D, i);
end
function nUpdateWaitbar(~)
waitbar(p/N, h);
p = p + 1;
end
end
End result :
If you just want to know how much time is left approximately, you can run the program once record the max time and then do this
tStart = tic;
parfor i=1:n
tElapsed = toc(tStart;)
disp(['Time left in min ~ ', num2str( ( tMax - tElapsed ) / 60 ) ]);
...
end
I created a utility to do this:
http://www.mathworks.com/matlabcentral/fileexchange/48705-drdan14-parforprogress