i have written a function in matlab, which is to calculate a cost function for an array:
function [F1val, Com]=F1(Community,NeighMat)
global FlattedAdjMat;
Com=zeros(numel(Community),3);
Com(:,1)=Community'; % The First row of Com= Community's Nodes
% Calculating INTERNAL Edges of Community
for j=1:numel(Com(:,1))
Com(j,2)=sum(FlattedAdjMat((Community),Com(j,1)));% Internal Degrees of Node j
end
F1val=(nansum(Com(:,3)./((Com(:,2)+Com(:,3)))));
end
But i have 2 problem with the line Com(j,2)=sum(FlattedAdjMat((Community),Com(j,1))),
when i try to execute it in parallel using parfor:
parfor iii=1:5
[z,p]=F1(Community,NeighMat)
end
this error occurs in the line: Index exceeds matrix dimensions
while in normal case(not Parallel) there is no problem
it is very time consuming and slow down the speed.
NeighMat is a weighted adjacency matrix,
Community is an array of the matrix indexes,
FlattedAdjMat is adjacency matrix.
whould you please help me?
sample data:
for ii=1:10
NeighMat{ii}=randi(10,1,10)
end
Community=[1,5,6,8,9];`
global FlattedAdjMat
FlattedAdjMat=randi([0,1],10,10)
You have a problem with global variable. This problem is well discussed here.
I rewrite Your code a bit and it works for me perfect (Matlab 2017b Win64)
close all
clear all
clc
%% SAMPLE DATA
for ii=1:10
NeighMat{ii}=randi(10,1,10);
end
Community=[1,5,6,8,9];
FlattedAdjMat=randi([0,1],10,10);
%% BODY
parfor iii=1:5
[z,p]=F1(Community,NeighMat,FlattedAdjMat)
end
%% FUNCTION
function [F1val, Com]=F1(Community,NeighMat,FlattedAdjMat)
Com=zeros(numel(Community),3);
Com(:,1)=Community'; % The First row of Com= Community's Nodes
% Calculating INTERNAL Edges of Community
for j=1:numel(Com(:,1))
Com(j,2)=sum(FlattedAdjMat((Community),Com(j,1)));% Internal Degrees of Node j
end
F1val=(nansum(Com(:,3)./((Com(:,2)+Com(:,3)))));
end
Related
I'm using parfor, unfortunately two of my variables, x and y, which are both matrices, get broadcasted and don't know how to avoid it. I've read about it in the MATLAB help, but couldn't figure out a solution. How can I prevent broadcasting of x and y?
Here is my code :
parfor k=1:length(Lambda)
lambda=Lambda(k);
for p=1:length(Gamma)
gamma=Gamma(p);
for Fold=1:size(Fold_indices,2)
x_Train=x(logical(Fold_indices(:,Fold)),1:end);
Y_Train=y(logical(Fold_indices(:,Fold)),1:Num_Tasks);
% Do sth with x_Train and Y_train
end
end
end
I've tried to slice broadcasted data(x) into a cell array and it didn't solve the problem a well.
B=cell(1,J);
% Fill each entry of B with a matrix
% ...do it here
....
parfor k=1:length(Lambda)
lambda=Lambda(k);
for p=1:length(Gamma)
gamma=Gamma(p);
for Fold=1:J)
x_Train=B{1,J};
% Do sth with x_Train and Y_train
end
end
end
Interestingly, when I assign the broadacaste variable(B) to other varible(D), then it stops getting brodcated.
B=cell(1,J);
% Fill each entry of B with a matrix
% ...do it here
....
parfor k=1:length(Lambda)
D=B;
lambda=Lambda(k);
for p=1:length(Gamma)
gamma=Gamma(p);
for Fold=1:J)
x_Train=B{1,J};
% Do sth with x_Train and Y_train
end
end
end
First off: you need to broadcast. Each worker is a separate MATLAB instance, and it needs the data. Sending data to a worker's MATLAB instance is called broadcasting. So there's no preventing it when you use parallel computing, it's the core of it even.
Second: you can't avoid broadcasting x and y in its entirety here, since you use all of it in each separate parfor iteration. Avoiding broadcasting as a whole requires you to not need all of the matrix in each loop iteration, in which case you can slice your variables, as is presented in this answer; i.e. you'll have to rewrite your code in such a manner that you do not require all of x and y to be on each separate worker.
I am attempting to speed up my MATLAB code by using parfor, however, I am doing it incorrectly. My code is rather simple, I am fitting some data using MATLAB's built-in mle function by using varying initial guesses for the mean (mm) and variance (vv). onestagepdf2 is my probability density function.
Here is the code snippit:
mm=linspace(.1, 1, 2); % mean
vv=linspace(.1, 2, 2); % variance
N=length(mm);
n=length(vv);
pd=zeros(n*N,2);
ld = NaN*ones(n*N,1);
options = statset('MaxIter',10000, 'MaxFunEvals',10000);
parfor i=1:N % pick a mean
m=mm(i);
parfor j=1:n % pick a variance
v=vv(j);
x0=[m,v];
[p,conf1]=mle(data,'pdf',#onestagepdf2,'start',x0, 'upperbound', [Inf Inf],'lowerbound',[0 0],'options',options)
pd(n*(i-1)+j,:)=p; % store parameter values from mle
l=onestagepdf2(data,p(1),p(2)); % evaluate pdf with parameter values
ld(n*(i-1)+j)=sum(log(l)); % store likelihood value
end
end
The error that I receive is:
'The variable pd in a parfor cannot be classified.'
pd = zeros(n, N, 2); %initialise culprits
ld= zeros(n,N);
parfor ii=1:N % pick a mean
m=mm(ii);
for jj=1:n % Parallellise the second parfor
v=vv(jj);
x0=[m,v];
[p,conf1]=mle(data,'pdf',#onestagepdf2,'start',x0, 'upperbound', [Inf Inf],'lowerbound',[0 0],'options',options)
pd(ii, jj, :) = p;=p; % store parameter values from mle
l=onestagepdf2(data,p(1),p(2)); % evaluate pdf with parameter values
ld(ii,jj)=sum(log(l)); % store likelihood value
end
end
Your pd was indeed the culprit, as #Trilarion stated, for the error you got. Probably ld also isn't too great, with the same syntax, so initialise that as well. This happened because parfor wants to know what the size of all variables within the loop is before executing. That this is a fixed maximum size is unknown to MATLAB, since you are using the loop variables to "change" the size.
Probably you had "orange wiggles" below those two lines (like spell-check errors) when you ran this as a for loop saying that "pd appears to be growing in size each iteration. Please consider preallocating for speed". This is required by parfor, since the order of iteration is not sequential it is impossible to grow arrays this way.
Secondly, you cannot nest parfor loops. You can use things like a function with a parfor and run that within the parfor, but that won't get you a speed-up since you are already using all your workers.
See Saving time and memory using parfor in Matlab? for more general information on parfor especially on speed.
You want a sliced, output variable but Matlab is not clever enough to detect that n*(i-1)+j is actually reasonable and won't interfere with an asynchronous evaluation.
Just do it as separate dimensions
pd = zeros(n, N, 2);
...
% in the loop
pd(i, j, :) = p;
That will work.
Please note, that Matlab does not allow nested parfors. However, you also do not need them if N is larger than the number of workers. See also the documentation.
I have a code to find the pdf's approximation of a vector based on the formula for kernel estimation:
I implemented this formula in the code below (see previous question). However, that code takes long time to run (two loops are used). Could you see the below code and help me to make it faster?
This is the code:
function pdf_est=KDE()
close all;
%%Random values of 20 pixels, range=[1 256]
data=randi([1 256],1,20)-1; %// changed: "-1"
%% Estimate histogram%%%%%
pdf_est=zeros(1,256);
z=256;
for i=0:z-1 %// changed_ subtracted 1
for j=1:length(data)
pdf_est(i+1)=pdf_est(i+1)+Gaussian(i-data(j)); %// changed: "+1" (twice)
end
end
%% Plot real histogram 1 to 256; binsize=1;
hold on
plot(0:255, imhist(uint8(data))./length(data),'r'); %// changed: explicit x axis
%% Plot histogram estimation
plot(0:255, pdf_est./length(data),'b'); %// changed: explicit x axis
hold off
function K=Gaussian(x)
sigma=1; %// change? Set as desired
K=1./(sqrt(2*pi)*sigma)*exp(-x^2./(2*sigma^2));
You can get rid of both of those nasty nested loops and then use the hardcoded sigma to have a mega-reduced vectorized solution -
pdf_est = sum(1./(sqrt(2*pi))*exp(-bsxfun(#minus,[0:z-1]',data).^2/2),2)
Or if you would like to have the flexibility to have sigma around, use this -
sum(1./(sqrt(2*pi)*sigma)*exp(-bsxfun(#minus,[0:z-1]',data).^2/(2*sigma^2)),2)
That's all really there is!
Quick tests put this to speedup the original code by 10x!
I was wondering if anyone could look at this code. I'm trying to do a 3x3 gaussian kernel without using any matlab built-in functions. (Yes I am aware of all the built in functions of gaussian and convolution ie fspecial)
the result gives me a white image instead. not sure what's the problem with it.
clc;
close all;
clear all;
img=imread('image.jpg');
figure,imshow(img);
img2=zeros(size(img)+2);
newImg=zeros(size(img));
for rgb=1:3
for x=1:size(img,1)
for y=1:size(img,2)
img2(x+1,y+1,rgb)=img(x,y,rgb);
end
end
end
for i= 1:size(img2,1)-2
for j=1:size(img2,2)-2
window=zeros(9,1);
inc=1;
for x=1:3
for y=1:3
window(inc)=img2(i+x-1,j+y-1,rgb);
inc=inc+1;
end
end
kernel=[1;2;1;2;4;2;1;2;1];
% kernel=[0;1;0;1;-4;1;0;1;0];
med=window.*kernel;
disp(med);
med=sum(med);
med=floor(med);
newImg(i,j,:)=med;
end
end
newImg=uint8(newImg);
figure,imshow(newImg);
Thanks.
I guess you should normalize the kernel. I mean, devide each element by the sum of their value (16), to make their sum equal to 1:
kernel=[1;2;1;2;4;2;1;2;1]/16;
I have a 3 dimensional grid, in which for each point of the grid I want to calculate a time dependent function G(t) for a large number of time steps and then summing the G function for each grid point. Using 4 for loops the execution time is becoming very large, so I am trying to avoid this using parfor.
a part of my code:
for i=1:50
for j=1:50
for k=1:25
x_in=i*dx;
y_in=j*dy;
z_in=k*dz;
%dx,dy, dz are some fixed values
r=sqrt((xx-x_in).^2+(yy-y_in).^2+(zz-z_in).^2);
%xx,yy,zz are 50x50x25 matrices generated from meshgrid
% r is a 3d matrix which produced from a 3 for-loop, for all the points of grid
parfor q=1:100
t=0.5*q;
G(q)=((a*p)/(t.^1.5)).*(exp(-r.^2/(4*a*t)));
% a,p are some fixed values
end
GG(i,j,k)=sum(G(:));
end
end
end
When I am using parfor the execution time becomes larger, and I am not sure why this is happening. Maybe I am not so familiar with sliced and indexed variables on a parfor loop.
My pc processor has 8 threads and ram memory ddr3 8GB
Any help will be great.
Thanks
As has been discussed in a previous question, parfor comes with an overhead. Therefore, loop that is too simple will execute more slowly with parfor.
In your case, the solution may be to parallelize the outermost loops.
%# preassign GG
GG = zeros(50,50,25);
%# loop over indices into GG
parfor idx = 1:(50*50*25)
[i,j,k] = ind2sub([50 50 25],idx);
x_in=i*dx;
y_in=j*dy;
z_in=k*dz;
%dx,dy, dz are some fixed values
r=sqrt((xx-x_in).^2+(yy-y_in).^2+(zz-z_in).^2);
%xx,yy,zz are 50x50x25 matrices generated from meshgrid
% r is a 3d matrix which produced from a 3 for-loop, for all the points of grid
for q=1:100
t=0.5*q;
G(q)=((a*p)/(t.^1.5)).*(exp(-r.^2/(4*a*t)));
% a,p are some fixed values
end
GG(idx)=sum(G(:));
end