Matlab: Print progress from parfor loop - matlab

Question

I run a lot of long simulations in Matlab, typically taking from a couple of minutes to a couple of hours, so to speed things up I decided to run the simulations simultaneously using a parfor loop.
arglist = [arg1, arg2, arg3, arg4];
parfor ii = 1:size(arglist, 2)
myfun(arglist(ii));
end
Everything worked just fine, except for one thing: the progress printing. Since each simulation takes a lot of time, I usually print progress using something like
prevlength = 0;
for ii = 1:tot_iter
% Some calculations here
msg = sprintf('Working on %d of %d, %.2f percent done', ii, tot_iter, ii/tot_iter);
fprintf(repmat('\b', 1, prevlength))
fprintf(msg);
prevlength = numel(msg);
end
but, as could be expected, when doing this inside a parfor loop, you get chaos.
I have googled a lot in search of a solution and have found a bunch of "parfor progress printers" like this one. However, all of them print the progress of the entire parfor loop instead of showing how far each of the individual iterations have come. Since I only have about 4-8 iterations in the parfor loop, but each iteration takes about an hour, this approach isn't very helpful to me.
The ideal solution for me would be something that looks like this
Working on 127 of 10000, 1.27 percent done
Working on 259 of 10000, 2.59 percent done
Working on 3895 of 10000, 38.95 percent done
Working on 1347 of 10000, 13.47 percent done
that is, each simulation gets one line showing how far it has run. I'm not sure though if this is possible at all, I can at least not imagine any way to do this.
Another way would be to do something like this
Sim 1: 1.27% Sim 2: 2.59% Sim 3: 38.95% Sim 4: 13.47%
that is, show all the progresses on the same line. To do this, you would need to keep track of what position on the line each simulation is to write on and write there, without erasing the other progresses. I can't figure out how this would be done, is this possible to do?
If there is some other solution to my problem (showing the progress of each individual iteration) that I haven't thought of, I'd be happy to hear about it.
Since this is the first time I ask a question here on SO it is quite possible that there is something that I have missed; if so, please feel free to comment below.
Edit
After receiving this answer, I thought that I should share how I used it to solve my problem since I didn't use it exactly as in the answer, in case someone else encounters the same problem.
Here is a small test program with basically the same structure as my program, making use of the progress bar (parfor_progress) mentioned in the answer:
function parfor_progress_test()
cpus = feature('numCores');
matlabpool('open', cpus);
cleaner = onCleanup(#mycleaner);
args = [1000, 1000, 1000, 1000];
m = sum(args);
parfor_progress(m);
parfor ii = 1:size(args,2)
my_fun(args(ii));
end
parfor_progress(0);
end
function my_fun(N)
for ii = 1:N
pause(rand*0.01);
parfor_progress;
end
end
function mycleaner
matlabpool close;
fclose all;
end

Answer 1

Simple Progress Bar
Something like a progress bar could be done similar to this...
Before the parfor loop :
fprintf('Progress:\n');
fprintf(['\n' repmat('.',1,m) '\n\n']);
And during the loop:
fprintf('\b|\n');
Here we have m is the total number of iterations, the . shows the total number of iterations and | shows the number of iterations completed. The \n makes sure the characters are printed in the parfor loop.
Progress Bar and Percentage Completion
Otherwise you could try this: http://www.mathworks.com/matlabcentral/fileexchange/32101-progress-monitor--progress-bar--that-works-with-parfor
It will display a progress bar and percentage completion but can be easily modified to just include the percentage completion or progress bar.
This function amends a character to a file on every iteration and then reads the number of characters written to that file which indicates the number of iterations completed. This file accessing method is allowed in parfor's.
Assuming you correctly add the above to your MATLAB path somehow, you can then use the following:
arglist = [arg1, arg2, arg3, arg4];
parfor_progress(size(arglist, 2)); % Set the total number of iterations
parfor ii = 1:size(arglist, 2)
myfun(arglist(ii));
parfor_progress; % Increment the progress counter
end
parfor_progress(0); % Reset the progress counter
Time to Completion and Percentage Completion
There is also a function called showTimeToCompletion() which is available from: https://www.soundzones.com/software/sound-zone-tools/
and works alongside parfor_progress. This function allows you to print a detailed summary of the progress of a parfor loop (or any loop for that matter) which contains the start time, length of time running, estimated finish time and percentage completion. It makes smart use of the \b (backspace) character so that the command window isn't flooded with text. Although not strictly a progress 'bar' it is perhaps more informative.
The third example in the header of the function file,
fprintf('\t Completion: ');
showTimeToCompletion; startTime=tic;
len=1e2;
p = parfor_progress( len );
parfor i = 1:len
pause(1);
p = parfor_progress;
showTimeToCompletion( p/100, [], [], startTime );
end
outputs the following to the command window:
Completion: 31.00%
Remaining: 00:00:23
Total: 00:00:33
Expected Finish: 3:30:07PM 14-Nov-2017
Useful for estimating the completion of a running simulation, especially one that may take hours or days.

Answer 2

Starting in R2013b, you can use PARFEVAL to evaluate your function asynchronously and have the client display progress updates. (Obviously this approach is not quite as simple as adding stuff to your PARFOR loop). There's an example here.
The Diary property of the Future returned by PARFEVAL is updated continuously while processing, so that might also be useful if you have a small number of large tasks.

Answer 3

Starting in R2017a, you can use parallel.pool.DataQueue and afterEach to implement a waitbar for parfor, like so:
if isempty(gcp('nocreate'))
parpool('local', 3);
end
dq = parallel.pool.DataQueue;
N = 10;
wb = waitbar(0, 'Please wait...');
% Use the waitbar's UserData to track progress
wb.UserData = [0 N];
afterEach(dq, #(varargin) iIncrementWaitbar(wb));
afterEach(dq, #(idx) fprintf('Completed iteration: %d\n', idx));
parfor idx = 1:N
pause(rand());
send(dq, idx);
end
close(wb);
function iIncrementWaitbar(wb)
ud = wb.UserData;
ud(1) = ud(1) + 1;
waitbar(ud(1) / ud(2), wb);
wb.UserData = ud;
end

Answer 4

after exploring #Edric answer I found that there is an example in the Matlab documentation that exactly implements a waitbar for a pareval loop. Check help FetchNext
N = 100;
for idx = N:-1:1
% Compute the rank of N magic squares
F(idx) = parfeval(#rank, 1, magic(idx));
end
% Build a waitbar to track progress
h = waitbar(0, 'Waiting for FevalFutures to complete...');
results = zeros(1, N);
for idx = 1:N
[completedIdx, thisResult] = fetchNext(F);
% store the result
results(completedIdx) = thisResult;
% update waitbar
waitbar(idx/N, h, sprintf('Latest result: %d', thisResult));
end
% Clean up
delete(h)