This question already has an answer here:
parfor in matlab. sliced variable and nested loop
(1 answer)
Closed 7 years ago.
Consider the following code in MATLAB:
function parallelProblem
N = 10;
A = rand(N);
parfor i=1:N
for k=2:N
A(i,k) = f(A(i,k-1));
end
end
end
function y=f(x)
y = x;
end
This is a summary of the problematic code I'm working on. Basically the idea is the following: I have to variables iand kand I can perform my computation with no communication between different i, but communication between different values of kis required.
Therefore I want to parallelize the loop over i. However, for the code above I get the error
parallelProblem
Error: File: parallelProblem.m Line: 6 Column: 9
The variable A in a parfor cannot be classified.
See Parallel for Loops in MATLAB, "Overview".
Hovering over the word parfor (which is underlined) gives
The PARFOR loop can not run due to the way the variable 'A' is used.
and hovering over f(A(i,k-1)) gives
Valid indices for 'A' are restricted in PARFOR loops
and
In PARFOR loop, variable 'A' is indexed in different ways, potentially causing dependencies between iterations.
From an intuitive point of view, I see no reason why the code should be working in parallel. Is there any way, I can modify my code to get the desired result?
The problem is that you are overwriting A. Loops in a parfor are not executed in order and MATLAB sees that you are overwriting the values of A, and then using them. You can easily fix this using an auxiliary variable:
function parallelProblem
N = 5;
A = magic(N);
Aresult=[];
parfor i=1:N,N
b=[];
for k=2:N
b(k) = f(A(i,k-1));
end
Aresult(i,:)=b;
end
end
Related
I am trying to use a for loop inside of a parfor loop in Matlab.
The for loop is equivalent to the ballode example in here.
Inside the for loop a function ballBouncing is called which is a system of 6 differential equations.
So, what I am trying to do is to use 500 different sets of parameter values for the ODE system and run it, but for each parameter set, a sudden impulse is added, which is handled through the code in 'for' loop.
However, I don't understand how to implement this using a parfor and a for loop as below.
I could run this code by using two for loops but when the outer loop is made to be a parfor it gives the errors,
the PARFOR loop cannot run due to the way variable results is used,
the PARFOR loop cannot run due to the way variable y0 is used and
Valid indices for results are restricted in PARFOR loops
results=NaN(500,100);
x=rand(500,10);
parfor j=1:500
bouncingTimes=[10,50];%at time 10 a sudden impulse is added
refine=2;
tout=0;
yout=y0;%initial conditions of ODE system
paras=x(j,:);%parameter values for the ODE
for i=1:2
tfinal=bouncingTimes(i);
[t,y]=ode45(#(t,y)ballBouncing(t,y,paras),tstart:1:tfinal,y0,options);
nt=length(t);
tout=[tout;t(2:nt)];
yout=[yout;y(2:nt,:)];
y0(1:5)=y(nt,1:5);%updating initial conditions with the impulse
y0(6)=y(nt,6)+paras(j,10);
options = odeset(options,'InitialStep',t(nt)-t(nt-refine),...
'MaxStep',t(nt)-t(1));
tstart =t(nt);
end
numRows=length(yout(:,1));
results(1:numRows,j)=yout(:,1);
end
results;
Can someone help me to implement this using a parfor outer loop.
Fixing the assignment into results is relatively straightforward - what you need to do is ensure you always assign a whole column. Here's how I would do that:
% We will always need the full size of results in dimension 1
numRows = size(results, 1);
parfor j = ...
yout = ...; % variable size
yout(end:numRows, :) = NaN; % Expand if necessary
results(:, j) = yout(1:numRows, 1); % Shrink 'yout' if necessary
end
However, y0 is harder to deal with - the iterations of your parfor loop are not order-independent because of the way you're passing information from one iteration to the next. parfor can only handle loops where the iterations are order-independent.
I'm trying to run this code in Matlab
a = ones(4,4);
b=[1,0,0,1;0,0,0,1;0,1,0,0;0,0,0,0];
b(:,:,2)=[0,1,1,0;1,1,1,0;1,0,1,1;1,1,1,1];
parfor i = 1:size(b,3)
c = b(:,:,i)
a(c) = i;
end
but get the error:
Error: The variable a in a parfor cannot be classified.
See Parallel for Loops in MATLAB, "Overview".
There are restrictions in how you can write into arrays inside the body of a parfor loop. In general, you will need to use sliced arrays.
The reason behind this issue is that Matlab needs to prevent that different worksers access the same data, leading to unpredictable results (as the timely order in which the parfor loops through i is not detemined).
So, although in your example the workers don't operate on the same entries of a, due to the way how you index a (with an array of logicals), it is currently not possible for Matlab to decide if this is the case or not (in other words, Matlab cannot classify a).
Edit: For completeness I add some code that is equivalent to your example, although I assume that your actual problem involves more complicated logical indexing?
a = ones(4,4,4);
parfor i = 1:size(a,1)
a(i, :, :) = zeros(4, 4) + i; % this is sliced indexing
end
Edit: As the OP example was modified, the above code is not equivalent to the example anymore.
Suppose I have two functions written on different scripts, say function1.m and function2.m The two computation in the two functions are independent (Some inputs may be the same, say function1(x,y) and function2(x,z) for example). However, running sequentially, say ret1 = function1(x,y); ret2 = function2(x,z); may be time consuming. I wonder if it is possible to run it in parfor loop:
parfor i = 1:2
ret(i) = run(['function' num2str(i)]); % if i=1,ret(1)=function1 and i=2, ret(2)=function2
end
Is it possible to write it in parfor loop?
Your idea is correct, but the implementation is wrong.
Matlab won't let you use run within parfor as it can't make sure it's a valid way to use parfor (i.e. no dependencies between iterations). The proper way to do that is to use functions (and not scrips) and an if statement to choose between them:
ret = zeros(2,1);
parfor k = 1:2
if k==1, ret(k) = f1(x,y); end
if k==2, ret(k) = f2(x,z); end
end
here f1 and f2 are some functions that return a scalar value (so it's suitable for ret(k) and each instance of the loop call a different if statement.
You can read here more about how to convert scripts to functions.
The rule of thumb for a parfor loop is that each iteration must be standalone. More accurately,
The body of the parfor-loop must be independent. One loop iteration
cannot depend on a previous iteration, because the iterations are
executed in a nondeterministic order.
That means that every iteration must be one which can be performed on its own and produce the correct result.
Therefore, if you have code that says, for instance,
parfor (i = 1:2)
function1(iterator,someNumber);
function2(iterator,someNumber);
end
there should be no issue with applying parfor.
However, if you have code that says, for instance,
persistentValue = 0;
parfor (i = 1:2)
persistentValue = persistentValue + function1(iterator,someNumber);
function2(iterator,persistentValue);
end
it would not be usable.
Yes. It is possible.
Here's an example:
ret = zeros(2,1);
fHandles = {#min, #max};
x = 1:10;
parfor i=1:2
ret(i) = fHandles{i}(x);
end
ret % show the results.
Whether this is a good idea or not, I don't know. There is overhead to setting up the parallel processing that may or may not make it worthwhile for you.
Typically the more iterations you have computed, the more value you get from setting up a parfor loop as the iterations are sliced-up and sent non-deterministically to the separate cores for processing. So you're getting use of 2 cores right now, but if you have many functions this may improve things.
The order that the iterations are run is not guaranteed (it could be that one core gets assigned a range of values for i, but we do not know if it those values are taken in order or randomly), so your code can't depend on other iterations of the loop.
In general, the MATLAB editor is pretty at flagging these issues ahead of time.
EDIT
Here's a proof of concept for a variable number of arguments to your different functions
ret = zeros(2,1);
fHandles = {#min, #max};
x = 1:10; % x is a 1x10 vector
y = rand(20); % y is a 20x20 matrix
z = 1; % z is a scalar value
fArgs = {{x};
{y,z}}; %wrap your arguments up in a cell
parfor i=1:2
ret(i) = fHandles{i}([fArgs{i}{:}]); %calls the function with its variable sized arguments here
end
ret % show the output
Again, this is just proof-of-concept. There are big warnings showing up in MATLAB about having to broadcast fArgs across all of the cores.
The following code works, but if I change for into parfor, it gave an error
Index exceeds matrix dimensions
This is my code
a=zeros(3,1);
for t=1:2
ind=randsample(3,2)
a=pf(a,ind)
end
function a=pf(a,ind)
a(ind)=a(ind)+2;
end
How can I get this code working without the error?
You are seeing the error because you are misusing parfor in your code. You haven't read the relevant documentation enough, and you seem to believe that parfor is magic fairy dust that makes your computation faster, regardless of computation. Well, I have bad news.
Let's take a closer look at your example:
a = zeros(3,1);
% usual for
disp('before for')
for t=1:2
ind = randsample(3,2);
a = pf(a,ind);
disp(a); % add printing line
end
% parfor
disp('before parfor')
parfor t=1:2
ind = randsample(3,2);
a = pf(a,ind);
disp(a); % add printing line
end
The output:
before for
2
2
0
2
4
2
before parfor
Error: The variable a is perhaps intended as a reduction variable, but is actually an uninitialized temporary.
See Parallel for Loops in MATLAB, "Temporary Variables Intended as Reduction Variables".
As you can see, in the latter case there are no prints inside the parfor, so it doesn't even get run. See also the warning about the type of variables. The variable a is being misidentified by the execution engine because what you are doing to it doesn't make any sense.
So what to do instead? You need to formulate your problem in a way that is compatible with parfor. This will, alas, depend on what exactly you're doing to your matrix. For your specific case of incrementing random elements, I suggest that you gather the increments separately in the loop, and sum them up afterwards:
a = zeros(3,1); % only needed for size; assumed that it exists already
numiters = 2;
increments = zeros([size(a), numiters]); % compatible with a proper 2d array too
parfor t=1:numiters
ind = randsample(3,2);
% create an auxiliary increment array so that we can use a full slice of 'increments'
new_contrib = zeros(size(a));
new_contrib(ind) = 2;
increments(:,t) = new_contrib;
disp(increments(:,t)); % add printing line
end
% collect increments along last axis
a = sum(increments,ndims(increments));
disp(a)
Output:
2
0
2
2
2
0
4
2
2
Note the lack of warnings and the presence of a meaningful answer. Refactoring the loop this way transparently signals MATLAB what the variables are doing, and that increments is being filled up by independent iterations of the parfor loop. This is the way in which parfor can "speed up calculations", a very specific and controlled way that implies restrictions on the logistics used inside the loop.
n = 2;
a=zeros(3,1);
ind=zeros(3,2,n);
for ii = 1:n
ind(:,:,ii) = randsample(3,2);
end
for t=1:n
a=pf(a,ind(:,:,t));
end
function a=pf(a,ind)
a(ind)=a(ind)+2;
end
The above gets the randsample out of the loop, which is probably the issue here. Note that randsample does not support direct 3D matrix creation, so I initialised that in a loop.
parfor is a convenient way to distribute independent iterations of intensive computations among several "workers". One meaningful restriction is that parfor-loops cannot be nested, and invariably, that is the answer to similar questions like there and there.
Why parallelization across loop boundaries is so desirable
Consider the following piece of code where iterations take a highly variable amount of time on a machine that allows 4 workers. Both loops iterate over 6 values, clearly hard to share among 4.
for row = 1:6
parfor col = 1:6
somefun(row, col);
end
end
It seems like a good idea to choose the inner loop for parfor because individual calls to somefun are more variable than iterations of the outer loop. But what if the run time for each call to somefun is very similar? What if there are trends in run time and we have three nested loops? These questions come up regularly, and people go to extremes.
Pattern needed for combining loops
Ideally, somefun is run for all pairs of row and col, and workers should get busy irrespectively of which iterand is being varied. The solution should look like
parfor p = allpairs(1:6, 1:6)
somefun(p(1), p(2));
end
Unfortunately, even if I knew which builtin function creates a matrix with all combinations of row and col, MATLAB would complain with an error The range of a parfor statement must be a row vector. Yet, for would not complain and nicely iterate over columns. An easy workaround would be to create that matrix and then index it with parfor:
p = allpairs(1:6, 1:6);
parfor k = 1:size(pairs, 2)
row = p(k, 1);
col = p(k, 2);
somefun(row, col);
end
What is the builtin function in place of allpairs that I am looking for? Is there a convenient idiomatic pattern that someone has come up with?
MrAzzman already pointed out how to linearise nested loops. Here is a general solution to linearise n nested loops.
1) Assuming you have a simple nested loop structure like this:
%dummy function for demonstration purposes
f=#(a,b,c)([a,b,c]);
%three loops
X=cell(4,5,6);
for a=1:size(X,1);
for b=1:size(X,2);
for c=1:size(X,3);
X{a,b,c}=f(a,b,c);
end
end
end
2) Basic linearisation using a for loop:
%linearized conventional loop
X=cell(4,5,6);
iterations=size(X);
for ix=1:prod(iterations)
[a,b,c]=ind2sub(iterations,ix);
X{a,b,c}=f(a,b,c);
end
3) Linearisation using a parfor loop.
%linearized parfor loop
X=cell(4,5,6);
iterations=size(X);
parfor ix=1:prod(iterations)
[a,b,c]=ind2sub(iterations,ix);
X{ix}=f(a,b,c);
end
4) Using the second version with a conventional for loop, the order in which the iterations are executed is altered. If anything relies on this you have to reverse the order of the indices.
%linearized conventional loop
X=cell(4,5,6);
iterations=fliplr(size(X));
for ix=1:prod(iterations)
[c,b,a]=ind2sub(iterations,ix);
X{a,b,c}=f(a,b,c);
end
Reversing the order when using a parfor loop is irrelevant. You can not rely on the order of execution at all. If you think it makes a difference, you can not use parfor.
You should be able to do this with bsxfun. I believe that bsxfun will parallelise code where possible (see here for more information), in which case you should be able to do the following:
bsxfun(#somefun,(1:6)',1:6);
You would probably want to benchmark this though.
Alternatively, you could do something like the following:
function parfor_allpairs(fun, num_rows, num_cols)
parfor i=1:(num_rows*num_cols)
fun(mod(i-1,num_rows)+1,floor(i/num_cols)+1);
end
then call with:
parfor_allpairs(#somefun,6,6);
Based on the answers from #DanielR and #MrAzzaman, I am posting two functions, iterlin and iterget in place of prod and ind2sub that allow iteration over ranges also if those do not start from one. An example for the pattern becomes
rng = [1, 4; 2, 7; 3, 10];
parfor k = iterlin(rng)
[plate, row, col] = iterget(rng, k);
% time-consuming computations here %
end
The script will process the wells in rows 2 to 7 and columns 3 to 10 on plates 1 to 4 without any workers idling while more wells are waiting to be processed. In hope that this helps someone, I deposited iterlin and iterget at the MATLAB File Exchange.