I have made for loop parralel in matlab, but I get warning that some of my arrays are broadcasted which lead to unnecessary communication overhead. I'm new in matlab and don't know how to solve this issue. Would someone help me with this?
A and Y is broadcasted in this code
parfor k=1:length(gamma)
Lambda=gamma(k);
tmp=zeros(nfolds,Num_Tasks);
for p=1:length(omega)
OmegA= omega(p)
for Fold=1:size(Fold_indices,2)
% Creat train and test fold
A_Train=A(logical(Fold_indices(:,Fold)),1:end); # A is broadcasted
%size(A_Train)
Y_Train=Y(logical(Fold_indices(:,Fold)),1:Num_Tasks); # Y is broadcated
A_Test=A(~logical(Fold_indices(:,Fold)),1:end);
Y_Test=Y(~logical(Fold_indices(:,Fold)),1:Num_Tasks);
coff=Estimate_x(Y_Train,A_Train,Lambda,OmegA,Binding_matrix) ;
% Do the prediction on Kth f;old and compute the error
% coff
%sum((A_Test*coff-Y_Test).^2) ./ size(A_Test,1)
tmp(Fold,1:end)=sum((A_Test*coff-Y_Test).^2) ./ size(A_Test,1);
tmp;
%Coefficents{:,Fold}=coff;
coff
end
In_Fold_Error{1,p}=tmp;
%In_Fold_Error{2,k}= Lambda;
Coefficents{:,p}=coff;
end
CVErr_twoparam{1,k}=In_Fold_Error;
Coefficents_twoparam=Coefficents;
end
Unless you can compute A_Train and Y_Train directly in the parfor loop, as opposed to using subscripting in order to get their values from A and Y, there's nothing you can do. Your source of information is unique, so these values will be broadcasted.
If you have enough memory, maybe it would be a better idea to compute A_Train and Y_Train as cell arrays of matrices in a normal for, and then use these sequentially pre-computed values inside the parfor. Like this you don't send the entire matrices A and Y to workers.
Related
I am trying to use a for loop inside of a parfor loop in Matlab.
The for loop is equivalent to the ballode example in here.
Inside the for loop a function ballBouncing is called which is a system of 6 differential equations.
So, what I am trying to do is to use 500 different sets of parameter values for the ODE system and run it, but for each parameter set, a sudden impulse is added, which is handled through the code in 'for' loop.
However, I don't understand how to implement this using a parfor and a for loop as below.
I could run this code by using two for loops but when the outer loop is made to be a parfor it gives the errors,
the PARFOR loop cannot run due to the way variable results is used,
the PARFOR loop cannot run due to the way variable y0 is used and
Valid indices for results are restricted in PARFOR loops
results=NaN(500,100);
x=rand(500,10);
parfor j=1:500
bouncingTimes=[10,50];%at time 10 a sudden impulse is added
refine=2;
tout=0;
yout=y0;%initial conditions of ODE system
paras=x(j,:);%parameter values for the ODE
for i=1:2
tfinal=bouncingTimes(i);
[t,y]=ode45(#(t,y)ballBouncing(t,y,paras),tstart:1:tfinal,y0,options);
nt=length(t);
tout=[tout;t(2:nt)];
yout=[yout;y(2:nt,:)];
y0(1:5)=y(nt,1:5);%updating initial conditions with the impulse
y0(6)=y(nt,6)+paras(j,10);
options = odeset(options,'InitialStep',t(nt)-t(nt-refine),...
'MaxStep',t(nt)-t(1));
tstart =t(nt);
end
numRows=length(yout(:,1));
results(1:numRows,j)=yout(:,1);
end
results;
Can someone help me to implement this using a parfor outer loop.
Fixing the assignment into results is relatively straightforward - what you need to do is ensure you always assign a whole column. Here's how I would do that:
% We will always need the full size of results in dimension 1
numRows = size(results, 1);
parfor j = ...
yout = ...; % variable size
yout(end:numRows, :) = NaN; % Expand if necessary
results(:, j) = yout(1:numRows, 1); % Shrink 'yout' if necessary
end
However, y0 is harder to deal with - the iterations of your parfor loop are not order-independent because of the way you're passing information from one iteration to the next. parfor can only handle loops where the iterations are order-independent.
I am experimenting with MATLAB SPDM. However, I have the following problem to solve:
I am running a quite long algorithm and I would like to save the progress along the way in case the power gets cut, someone unplugs the power plug or memory error.
The loop has 144 iterations that take each around 30 minutes to complete => 72h. A lot of problems can occur in that interval.
Of course, I have the distributed computing toolbox on my machine. The computer has 4 physical cores. I run MATLAB R2016a.
I do not really want to use a parfor loop because I concatenate results and have dependency across iterations. I think SPMD is the best choice for what I want to do.
I'll try to describe what I want as best as I can:
I want to be able to save at a set iteration of the loop the results so far, and I want to save the results by worker.
Below is a Minimum (non)-Working Example. The last four lines should be put in a different .m file. This function, called within a parfor loop, allows to save intermediate iterations. It is working properly in other routines that I use. The error is at line 45 (output_save). Somehow, I would like to "pull" the composite object into a "regular" object (cell/structure).
My hunch is that I do not quite understand how Composite objects work and especially how they can be saved into "regular" objects (cells, structures, etc).
% SPMD MWE
% Clear necessary things
clear output output2 output_temp iter kk
% Useful thing that will be used later on
Rorder=perms(1:4);
% Stem of the file to save the data to
stem='MWE_MATLAB_spmd';
% Create empty cells where the results of the kk loop will be stored
output1{1,1}=[];
output2{1,2}=[];
% Start the parpool
poolobj=gcp;
% Define which worker/lab will do which iteration
iterperworker=ceil(size(Rorder,1)/poolobj.NumWorkers);
for i=1:poolobj.NumWorkers
if i<poolobj.NumWorkers
itertodo{1,i}=1+(iterperworker)*(i-1):iterperworker*i;
else
itertodo{1,i}=1+(iterperworker)*(i-1):size(Rorder,1);
end
end
%Start the spmd
% try
spmd
iter=1;
for kk=itertodo{1,labindex}
% Print which iteration is done at the moment
fprintf('\n');
fprintf('Ordering %d/%d \r',kk,size(Rorder,1));
for j=1:size(Rorder,2)
output_temp(1,j)=Rorder(kk,j).^j; % just to populate a structure
end
output.output1{1,1}=cat(2,output.output1{1,1},output_temp); % Concatenate the results
output.output2{1,2}=cat(2,output.output1{1,2},0.5*output_temp); % Concatenate the results
labindex_save=labindex;
if mod(iter,2)==0
output2.output=output; % manually put output in a structure
dosave(stem,labindex_save,output2); % Calls the function that allows me to save in parallel computing
end
iter=iter+1;
end
end
% catch me
% end
% Function to paste in another m-file
% function dosave(stem,i,vars)
% save(sprintf([stem '%d.mat'],i),'-struct','vars')
% end
A Composite is created only outside an spmd block. In particular, variables that you define inside an spmd block exist as a Composite outside that block. When the same variable is used back inside an spmd block, it is transformed back into the original value. Like so:
spmd
x = labindex;
end
isa(x, 'Composite') % true
spmd
isa(x, 'Composite') % false
isequal(x, labindex) % true
end
So, you should not be transforming output using {:} indexing - it is not a Composite. I think you should simply be able to use
dosave(stem, labindex, output);
I want to paralyze my forloop in Matlab.I use parfor function for that, but I get error because the way I used variable inside of the loop. would someone help me to fix that. I'm new in matlab.
Here is part of my try.
Here is part of problematic part:
CV_err=zeros(length(gamma), (Num_Tasks + 1));
parfor k=1:length(gamma)
#block of code
#
CV_err(k,1:Num_Tasks)= sum(In_Fold_Error)./size(In_Fold_Error,1);
CV_err(k,Lambda_location)= Lambda;
CV_err(k,(Num_Tasks +2))= sum(CV_err(k,1:Num_Tasks))/Num_Tasks;
end
Error: parfor loop can not run due to way CV_err is used.
CV_err is indexed in different ways, potentially causing dependencies
Seems that valid indices are restricted in parfor .
While your variable is sliced, you only access the k-th row in the k-th iteration, the code analyser does not understand it. Give a little help to matlab, first put all data into a vector and then write all at once to the sliced variable.
CV_err=zeros(length(gamma), (Num_Tasks + 2));
parfor k=1:length(gamma)
%block of code
%
temp=zeros(1,(Num_Tasks + 2));
temp(1,1:Num_Tasks)= sum(In_Fold_Error)./size(In_Fold_Error,1);
temp(1,Lambda_location)= Lambda;
temp(1,(Num_Tasks +2))= sum(temp(1,1:Num_Tasks))/Num_Tasks;
CV_err(k,:)=temp;
end
The limitation is explained in the documentation:
Form of Indexing. Within the list of indices for a sliced variable, one of these indices is of the form i, i+k, i-k, k+i, or k-i, where i is the loop variable and k is a constant or a simple (nonindexed) broadcast variable; and every other index is a scalar constant, a simple broadcast variable, a nested for-loop index, colon, or end.
Source
To fix pre-allocation, don't pre-allocate. You're just telling to MATLAB how it should split the work among workers; parfor doesn't like that.
The answer is: don't make loops change common variables, write your results separately, grow cell arrays instead of matrices, i.e.
clear CV_err;
parfor k=1:length(gamma)
%// here your other code
this_CV_err = zeros(Num_Tasks+2,1);
this_CV_err(1:Num_Tasks) = sum(In_Fold_Error)./size(In_Fold_Error,1);
this_CV_err(Lambda_location) = Lambda;
this_CV_err(Num_Tasks+2) = mean(this_CV_err(1:Num_Tasks));
CV_err{k} = this_CV_err;
end;
I have a problem with the following code. I want to store all the values I am creating in the for loop below so that I can make a plot of it. I have tried several things, but nothing works. Does anyone know a simple method to create a vector of the results and then plot them?
dx=0.1;
t=1;
e=1;
for x=-1:dx:1
lower_bound=-100;
upper_bound=x/(sqrt(4*t*e));
e=1;
u=(1/sqrt(pi))*quad(#integ,lower_bound,upper_bound);
plot(x,u)
hold on
end
hold off
I would like to use as much of this matlab code as possible.
dx=0.1;
t=1;
e=1;
xval=[-1:dx:1].';
upper_bound = zeros(numel(xval),1);
u = zeros(numel(xval),1);
for ii=1:numel(xval)
x = xval(ii)
lower_bound=-100;
upper_bound(ii,1)=x/(sqrt(4*t*e));
u(ii,1)=(1/sqrt(pi))*quad(#integ,lower_bound,upper_bound(ii));
end
figure;
plot(xval,u)
by adding the (ii) behind your statements it saves your variables in an array. I did not use that on your lower_bound since it is a constant.
Note that I first created an array xval and called that with integers in ii, since subscriptindices must be positive integers in MATLAB. I also initialised both upper_bound and u by creating a zero matrix before the loop executes. This is handy since extending an existing vector is very memory and time consuming in MATLAB and since you know how big they will get (same number of elements as xval) you might as well use that.
I also got the plot call outside the loop, to prevent you from plotting 21 blue lines in 1 plot.
I am implementing an Sum of Square Distances based disparity Map function in Matlab for computer vision. Currently the code has a nested for loop and runs very slow. Any suggestions on vectorizing it to make it more efficient? Thanks
%im1 and im2 are images and win1, win2 are window sizes
for i=win1+1:1:bottom-win1
parfor j=win2+1:1:right-win2
%j=[win2+1:bottom-win2];
template=im1(i-win1:i+win1,j-win2:j+win2);
arg1=conv2(im2.^2,ones(size(template))/2,'same');
arg2=conv2(im2,rot90(template,2),'same');
arg=arg1-arg2;
[xj]=find(arg==min(arg(:)));
disparityMap(i,j)=1-xj(1);
end
end
Three suggestions to try to speed things up:
move the parfor to the outer loop to reduce the overhead of the parallel construct ;
compute im2.^2 once before the loop and save its value in a temporary variable as it does not depend on the loop variables there is no need to compute it again and again, and actually
move the whole computation of arg1 out of the loops as it only depends on the size of template and not its value, and if I see correctly, the size is constant ;
replace the [xj]=find(arg==min(arg(:))); construct with something along the lines of [tmp, ind] = min(arg(:)) ; xj=ind2sub(size(arg), ind) to avoid the call to find and rescan the matrix while the indices can be computed simply.
Untested, but it should give you a start
arg1=conv2(im2.^2,ones([2*win1+1, 2*win2+1])/2,'same');
parfor i=win1+1:1:bottom-win1
for j=win2+1:1:right-win2
%j=[win2+1:bottom-win2];
template=im1(i-win1:i+win1,j-win2:j+win2);
arg2=conv2(im2,rot90(template,2),'same');
arg=arg1-arg2;
[tmp, ind] = min(arg(:)) ;
xj=ind2sub(size(arg), ind);
disparityMap(i,j)=1-xj(1);
end
end
Also make sure the number of workers is chosen appropriately, and try to compile the code to mex to see if there is improvement.