I have a problem with MathWorks Parallel Computing Toolbox in Matlab. See my code below
for k=1:length(Xab)
n1=length(Z)*(k-1)+1:length(Z)*k;
MX_j(1,n1)=JXab{k};
MY_j(1,n1)=JYab{k};
MZ_j(1,n1)=Z;
end
for k=length(Xab)+1:length(Xab)+length(Xbc)
n2=length(Z)*(k-1)+1:length(Z)*k;
MX_j(1,n2)=JXbc{k-length(Xab)};
MY_j(1,n2)=JYbc{k-length(Yab)};
MZ_j(1,n2)=Z;
end
for k=length(Xab)+length(Xbc)+1:length(Xab)+length(Xbc)+length(Xcd)
n3=length(Z)*(k-1)+1:length(Z)*k;
MX_j(1,n3)=JXcd{k-length(Xab)-length(Xbc)};
MY_j(1,n3)=JYcd{k-length(Yab)-length(Ybc)};
MZ_j(1,n3)=Z;
end
for k=length(Xab)+length(Xbc)+length(Xcd)+1:length(Xab)+length(Xbc)+length(Xcd)+length(Xda)
n4=length(Z)*(k-1)+1:length(Z)*k;
MX_j(1,n4)=JXda{k-length(Xab)-length(Xbc)-length(Xcd)};
MY_j(1,n4)=JYda{k-length(Yab)-length(Ybc)-length(Ycd)};
MZ_j(1,n4)=Z;
end
If I change the for-loop to parfor-loop, matlab warns me that MX_j is not an efficient variable. I have no idea how to solve this and how to make these for loops compute in parallel?
For me, it looks like you can combine it to one loop. Create combined cell arrays.
JX = cat(2,JXab, JXbc, JXcd, JXda);
JY = cat(2,JYab, JYbc, JYcd, JYda);
Check for the right dimension here. If your JXcc arrays are column arrays, use cat(1,....
After doing that, one single loop should do it:
n = length(Xab)+length(Xbc)+length(Xcd)+length(Xda);
for k=1:n
k2 = length(Z)*(k-1)+1:length(Z)*k;
MX_j(1,k2)=JX{k};
MY_j(1,k2)=JY{k};
MZ_j(1,k2)=Z;
end
Before parallizing anything, check if this still valid. I haven't tested it. If everything's nice, you can switch to parfor.
When using parfor, the arrays must be preallocated. The following code could work (untested due to lack of test-data):
n = length(Xab)+length(Xbc)+length(Xcd)+length(Xda);
MX_j = zeros(1,n*length(Z));
MY_j = MX_j;
MZ_j = MX_j;
parfor k=1:n
k2 = length(Z)*(k-1)+1:length(Z)*k;
MX_j(1,k2)=JX{k};
MY_j(1,k2)=JY{k};
MZ_j(1,k2)=Z;
end
Note: As far as I can see, the parfor loop will be much slower here. You simply assign some values... no calculation at all. The setup of the worker pool will take 99.9% of the total execution time.
Related
I have a code that goes like this which I want to run using parpool:
result = zeros(J,K)
for k = 1:K
for j = 1:J
build(:,1) = old1(:,j,k)
build(:,2) = old2(:,j,k)
result(j,k) = call_function(build); %Takes a long time to run
end
end
It takes a long time to run this code and I have to run this multiple times for my simulation so I want to run the outermost loop (k = 1:K) in parallel in MATLAB.
From what I have read, I cannot use parfor since all each function uses the same variables old1 and old2. I could use spmd and distribute my matrices old1 and old2. But I read this creates as many copies of the variable as the workers and I do not want this to happen. I could use drange. But I am not sure how it exactly works. I am finding it difficult to actually use what I have been reading in MATLAB references. Any resource and pointers would be of great help!
Constraints are as follows:
Must not create multiple copies of the variables old1, old2. But I can slice it across workers as each iteration doesn't require other iterations.
Have to distribute for the outermost loop only. For ease of accessing data outside this block of code.
Thank you.
old1 and old2 can be used, I think. Initialize as constants using:
old1 = parallel.pool.Constant(old1);
old2 = parallel.pool.Constant(old2);
Have you seen this post?
https://www.mathworks.com/help/distcomp/improve-parfor-performance.html
I'm trying to run this code in Matlab
a = ones(4,4);
b=[1,0,0,1;0,0,0,1;0,1,0,0;0,0,0,0];
b(:,:,2)=[0,1,1,0;1,1,1,0;1,0,1,1;1,1,1,1];
parfor i = 1:size(b,3)
c = b(:,:,i)
a(c) = i;
end
but get the error:
Error: The variable a in a parfor cannot be classified.
See Parallel for Loops in MATLAB, "Overview".
There are restrictions in how you can write into arrays inside the body of a parfor loop. In general, you will need to use sliced arrays.
The reason behind this issue is that Matlab needs to prevent that different worksers access the same data, leading to unpredictable results (as the timely order in which the parfor loops through i is not detemined).
So, although in your example the workers don't operate on the same entries of a, due to the way how you index a (with an array of logicals), it is currently not possible for Matlab to decide if this is the case or not (in other words, Matlab cannot classify a).
Edit: For completeness I add some code that is equivalent to your example, although I assume that your actual problem involves more complicated logical indexing?
a = ones(4,4,4);
parfor i = 1:size(a,1)
a(i, :, :) = zeros(4, 4) + i; % this is sliced indexing
end
Edit: As the OP example was modified, the above code is not equivalent to the example anymore.
I have a question about parallelizing code in MATLAB. I use MATLAB 2017a.
Let's say I have a cell array:
A = { A1, ..., A10}
and these matrices are quite big ( size > 10000 ). Now I want to start manipulating these matrices in a parallelpool. In fact, ther first worker needs only A1, the second worker needs only A2 and so on.
I have now this code;
parfor i = 1:10
matrix = A{i};
blabla = manipulate(Ai);
save(blabla);
end
I think that MATLAB gives every worker all the matrices in A but this is not really needed. Is there a way to say:
"Give i-th worker only matrix Ai"?
Based on the documentation for variables in parfor loops, specifically sliced variables, it appears as though the cell array A in your example meets the criteria to be treated as a sliced variable by default. You shouldn't have to do anything special. You may want to confirm that all the listed criteria are met, and take a look at each variable to see how they are used both inside and outside your parfor loop.
You want spmd blocks. That way, you explicitly handle the slicing of the parallel data, rather than letting Matlab do it automagically with the parfor block.
parpool('myprofile',10)
spmd
i = labindex;
B = foo(A{i});
end
for i = 1:10
bar(B{i});
end
Suppose I have two functions written on different scripts, say function1.m and function2.m The two computation in the two functions are independent (Some inputs may be the same, say function1(x,y) and function2(x,z) for example). However, running sequentially, say ret1 = function1(x,y); ret2 = function2(x,z); may be time consuming. I wonder if it is possible to run it in parfor loop:
parfor i = 1:2
ret(i) = run(['function' num2str(i)]); % if i=1,ret(1)=function1 and i=2, ret(2)=function2
end
Is it possible to write it in parfor loop?
Your idea is correct, but the implementation is wrong.
Matlab won't let you use run within parfor as it can't make sure it's a valid way to use parfor (i.e. no dependencies between iterations). The proper way to do that is to use functions (and not scrips) and an if statement to choose between them:
ret = zeros(2,1);
parfor k = 1:2
if k==1, ret(k) = f1(x,y); end
if k==2, ret(k) = f2(x,z); end
end
here f1 and f2 are some functions that return a scalar value (so it's suitable for ret(k) and each instance of the loop call a different if statement.
You can read here more about how to convert scripts to functions.
The rule of thumb for a parfor loop is that each iteration must be standalone. More accurately,
The body of the parfor-loop must be independent. One loop iteration
cannot depend on a previous iteration, because the iterations are
executed in a nondeterministic order.
That means that every iteration must be one which can be performed on its own and produce the correct result.
Therefore, if you have code that says, for instance,
parfor (i = 1:2)
function1(iterator,someNumber);
function2(iterator,someNumber);
end
there should be no issue with applying parfor.
However, if you have code that says, for instance,
persistentValue = 0;
parfor (i = 1:2)
persistentValue = persistentValue + function1(iterator,someNumber);
function2(iterator,persistentValue);
end
it would not be usable.
Yes. It is possible.
Here's an example:
ret = zeros(2,1);
fHandles = {#min, #max};
x = 1:10;
parfor i=1:2
ret(i) = fHandles{i}(x);
end
ret % show the results.
Whether this is a good idea or not, I don't know. There is overhead to setting up the parallel processing that may or may not make it worthwhile for you.
Typically the more iterations you have computed, the more value you get from setting up a parfor loop as the iterations are sliced-up and sent non-deterministically to the separate cores for processing. So you're getting use of 2 cores right now, but if you have many functions this may improve things.
The order that the iterations are run is not guaranteed (it could be that one core gets assigned a range of values for i, but we do not know if it those values are taken in order or randomly), so your code can't depend on other iterations of the loop.
In general, the MATLAB editor is pretty at flagging these issues ahead of time.
EDIT
Here's a proof of concept for a variable number of arguments to your different functions
ret = zeros(2,1);
fHandles = {#min, #max};
x = 1:10; % x is a 1x10 vector
y = rand(20); % y is a 20x20 matrix
z = 1; % z is a scalar value
fArgs = {{x};
{y,z}}; %wrap your arguments up in a cell
parfor i=1:2
ret(i) = fHandles{i}([fArgs{i}{:}]); %calls the function with its variable sized arguments here
end
ret % show the output
Again, this is just proof-of-concept. There are big warnings showing up in MATLAB about having to broadcast fArgs across all of the cores.
parfor is a convenient way to distribute independent iterations of intensive computations among several "workers". One meaningful restriction is that parfor-loops cannot be nested, and invariably, that is the answer to similar questions like there and there.
Why parallelization across loop boundaries is so desirable
Consider the following piece of code where iterations take a highly variable amount of time on a machine that allows 4 workers. Both loops iterate over 6 values, clearly hard to share among 4.
for row = 1:6
parfor col = 1:6
somefun(row, col);
end
end
It seems like a good idea to choose the inner loop for parfor because individual calls to somefun are more variable than iterations of the outer loop. But what if the run time for each call to somefun is very similar? What if there are trends in run time and we have three nested loops? These questions come up regularly, and people go to extremes.
Pattern needed for combining loops
Ideally, somefun is run for all pairs of row and col, and workers should get busy irrespectively of which iterand is being varied. The solution should look like
parfor p = allpairs(1:6, 1:6)
somefun(p(1), p(2));
end
Unfortunately, even if I knew which builtin function creates a matrix with all combinations of row and col, MATLAB would complain with an error The range of a parfor statement must be a row vector. Yet, for would not complain and nicely iterate over columns. An easy workaround would be to create that matrix and then index it with parfor:
p = allpairs(1:6, 1:6);
parfor k = 1:size(pairs, 2)
row = p(k, 1);
col = p(k, 2);
somefun(row, col);
end
What is the builtin function in place of allpairs that I am looking for? Is there a convenient idiomatic pattern that someone has come up with?
MrAzzman already pointed out how to linearise nested loops. Here is a general solution to linearise n nested loops.
1) Assuming you have a simple nested loop structure like this:
%dummy function for demonstration purposes
f=#(a,b,c)([a,b,c]);
%three loops
X=cell(4,5,6);
for a=1:size(X,1);
for b=1:size(X,2);
for c=1:size(X,3);
X{a,b,c}=f(a,b,c);
end
end
end
2) Basic linearisation using a for loop:
%linearized conventional loop
X=cell(4,5,6);
iterations=size(X);
for ix=1:prod(iterations)
[a,b,c]=ind2sub(iterations,ix);
X{a,b,c}=f(a,b,c);
end
3) Linearisation using a parfor loop.
%linearized parfor loop
X=cell(4,5,6);
iterations=size(X);
parfor ix=1:prod(iterations)
[a,b,c]=ind2sub(iterations,ix);
X{ix}=f(a,b,c);
end
4) Using the second version with a conventional for loop, the order in which the iterations are executed is altered. If anything relies on this you have to reverse the order of the indices.
%linearized conventional loop
X=cell(4,5,6);
iterations=fliplr(size(X));
for ix=1:prod(iterations)
[c,b,a]=ind2sub(iterations,ix);
X{a,b,c}=f(a,b,c);
end
Reversing the order when using a parfor loop is irrelevant. You can not rely on the order of execution at all. If you think it makes a difference, you can not use parfor.
You should be able to do this with bsxfun. I believe that bsxfun will parallelise code where possible (see here for more information), in which case you should be able to do the following:
bsxfun(#somefun,(1:6)',1:6);
You would probably want to benchmark this though.
Alternatively, you could do something like the following:
function parfor_allpairs(fun, num_rows, num_cols)
parfor i=1:(num_rows*num_cols)
fun(mod(i-1,num_rows)+1,floor(i/num_cols)+1);
end
then call with:
parfor_allpairs(#somefun,6,6);
Based on the answers from #DanielR and #MrAzzaman, I am posting two functions, iterlin and iterget in place of prod and ind2sub that allow iteration over ranges also if those do not start from one. An example for the pattern becomes
rng = [1, 4; 2, 7; 3, 10];
parfor k = iterlin(rng)
[plate, row, col] = iterget(rng, k);
% time-consuming computations here %
end
The script will process the wells in rows 2 to 7 and columns 3 to 10 on plates 1 to 4 without any workers idling while more wells are waiting to be processed. In hope that this helps someone, I deposited iterlin and iterget at the MATLAB File Exchange.