I am using a parfor in my code, I read all parfor limitations and I think I am doing everything right:
for it = 1:maxiter
rep_it = it;
parfor Fo = allFo
rep_Fo = Fo;
Fmax = 2*Fo;
find_rep = find(rep_Fo==allFo) ;
[Fac, c, F_est,loss] = AutoTen(info.Data,Fmax,2);
[Fac, F_est_baseline3] = AutoTenBaseline(info.Data,Fmax,1);
[Fac, F_est_baseline3] = AutoTenBaseline(info.Data,Fmax,2);
est_rank(find_rep,rep_it) = F_est;
est_rank_baseline1(find_rep,rep_it) = F_est_baseline3;
est_rank_baseline2(find_rep,rep_it) = F_est_baseline3;
end
end
But I get the error:
Error: The variable est_rank in a parfor cannot be classified
Any help is appreciated how to solve this.
Your problem comes from the find_rep and rep_it variables that are used for indexing. If you want est_rank to be sliceable, you have to use the Fo index for it like:
est_rank(Fo, rep_it) = F_est;
otherwise you (i.e. MATLAB) cannot guarantee that you won't try to access the same cell in the variable from two parallel processes.
If I got your intention correctly, then this fix should do it:
for it = 1:maxiter
parfor Fo = 1:numel(allFo)
Fmax = 2*allFo(Fo);
[Fac, c, F_est,loss] = AutoTen(info.Data,Fmax,2);
[Fac, F_est_baseline3] = AutoTenBaseline(info.Data,Fmax,1);
[Fac, F_est_baseline3] = AutoTenBaseline(info.Data,Fmax,2);
est_rank(Fo,it) = F_est;
est_rank_baseline1(Fo,it) = F_est_baseline3;
est_rank_baseline2(Fo,it) = F_est_baseline3;
end
end
Let me know if it did the trick ;)
Related
Hello I want to use parallelize my MATLAB code to run High computing server. It is code to make image database for Deep learning. To parallelize the code I found the I have to for parfor loop. But I used that with the first loop or with the second loop it shows me error parfor cannot be run due to the variable imdb and image_counter. Anyone please help me to change the code to work with parfor
for i = 1:length(cur_images)
X = sprintf('image Numb: %d ',i);
disp(X)
cur_image = load(cur_images{i,:});
cur_image=(cur_image.Image.crop);
%----------------------------------------------
cur_image = imresize(cur_image, image_size);
if(rgb < 1)
imdb.images.data(:,:,1,image_counter) = cur_image;
else
imdb.images.data(:,:,1,image_counter) = cur_image(:,:,1);
imdb.images.data(:,:,2,image_counter) = cur_image(:,:,2);
imdb.images.data(:,:,3,image_counter) = cur_image(:,:,3);
imdb.images.data(:,:,4,image_counter) = cur_image(:,:,4);
imdb.images.data(:,:,5,image_counter) = cur_image(:,:,5);
imdb.images.data(:,:,6,image_counter) = cur_image(:,:,6);
imdb.images.data(:,:,7,image_counter) = cur_image(:,:,7);
imdb.images.data(:,:,8,image_counter) = cur_image(:,:,8);
imdb.images.data(:,:,9,image_counter) = cur_image(:,:,9);
imdb.images.data(:,:,10,image_counter) = cur_image(:,:,10);
end
imdb.images.set( 1,image_counter) = set;
image_counter = image_counter + 1;
end
The main problem here is that you can't assign to fields of a structure inside parfor in the way that you're trying to do. Also, your outputs need to be indexed by the loop variable to qualify as "sliced" - i.e. don't use image_counter. Putting this together, you need something more like:
% Make a numeric array to store the output.
data_out = zeros([image_size, 10, length(cur_images)]);
parfor i = 1:length(cur_images)
cur_image = load(cur_images{i, :});
cur_image=(cur_image.Image.crop);
cur_image = imresize(cur_image, image_size);
% Now, assign into 'data_out'. A little care needed
% here.
if rgb < 1
data_tmp = zeros([image_size, 10]);
data_tmp(:, :, 1) = cur_image;
else
data_tmp = cur_image;
end
data_out(:, :, :, i) = data_tmp;
end
imdb.images.data = data_out;
Im trying to make a loop for doing the same operation to a lot of .mov files in matlab. The code i have right now looks like this:
close all
clear all
clc
movFiles = dir('*.mov');
numFiles = length(movFiles);
mydata = cell(1,numFiles);
% mydata = zeros(numFiles);
for k = 1:numFiles
mydata{1,k} = VideoReader(movFiles(k).name);
end
for k=1:numFiles
bk_downsample = 5; %The downsample factor for frame averaging
%disp('Opening video...') %lower number =longer computation time
vob = mydata;
frame = vob.read(inf); %Reads to end = vob knows the number of frames
vidHeight = vob.Height;
vidWidth = vob.Width;
nFrames = vob.NumberOfFrames;
%% First-iteration background frame
background_frame = double(frame*0);
disp('Calculating background...')
for k = 1:bk_downsample:nFrames
background_frame = background_frame + double(read(vob, k));
disp(k/(nFrames)*100)
end
%background_frame = uint8(bk_downsample*background_frame/(nFrames));
background_frame = bk_downsample*background_frame/(nFrames);
%imshow(background_frame)
%% Second-iteration background frame
%This section re-calculates the background frame while attempting to
%minimize the effect of moving objects in the calculation
background_frame2 = double(frame*0);
pixel_sample_density = im2bw(double(frame*0));
diff_frame = double(frame*0);
stream_frame = diff_frame(:,:,1);
bk_downsample = 10;
figure
hold on
for k = 1:bk_downsample:nFrames
diff_frame = imabsdiff(double(read(vob, k)), background_frame);
diff_frame = 1-im2bw(uint8(diff_frame),.25);
pixel_sample_density = pixel_sample_density + diff_frame;
stream_frame = stream_frame + (1-diff_frame)/(nFrames/bk_downsample);
nonmoving = double(read(vob, k));
nonmoving(:,:,1) = nonmoving(:,:,1).*diff_frame;
nonmoving(:,:,2) = nonmoving(:,:,2).*diff_frame;
nonmoving(:,:,3) = nonmoving(:,:,3).*diff_frame;
background_frame2 = background_frame2 + nonmoving;
%pause
disp(k/(nFrames)*100)
end
background_frame2(:,:,1) = background_frame2(:,:,1)./pixel_sample_density;
background_frame2(:,:,2) = background_frame2(:,:,2)./pixel_sample_density;
background_frame2(:,:,3) = background_frame2(:,:,3)./pixel_sample_density;
imshow(uint8(background_frame2))
%imshow(stream_frame)
filename = ['Ring_' num2str(k) '_background_' num2str(img) '.jpg'];
imwrite((uint8(background_frame2)),filename)
end
I know that the error starts with vob=mydata; but im not sure how to correct it, hope that someone is able to help me since it would save me a lot of time in my data-analysis.
Have a great day! :)
Your code doesn't make much sense... You're creating a cell array:
mydata = cell(1,numFiles);
%// . . .
mydata{1,k} = . . .
but however you try to access it like a structure:
vob = mydata;
frame = vob.read(inf);
If I'd guess, then your error stems from you forgetting to index in the cell array, i.e.:
vob = mydata{k};
Other programming oddity I noticed in your code is the fact you're using the same looping variable k, in two nested for lops, the outer one being on k=1:numFiles and the inner ones being on k=1:bk_downsample:nFrames; don't do that unless you're trying to drive yourself crazy while figuring out why your for loop executes only once. Name them k1 for the outer loop and k2 for the inner loops and you'll be happier.
I'm no expert in video processing, but for me it looks like your line should be:
vob=mydata{1,k};
That's why that error shows up, because you are treating a cell of structs as if it was a single struct.
I'm unable to execute the following code:
parallel_mode = true; %To process multiple images at the same time
parallel_degree = 4; %Number of threads that will be created
if parallel_mode
if matlabpool('size') == 0
matlabpool(parallel_degree);
elseif matlabpool('size') ~= parallel_degree
matlabpool close;
matlabpool(parallel_degree);
end
end
%Loading dictionary
try
load(dictionary_path);
catch
error(['Was impossible to load the dictionary in path: ' dictionary_path ', Please, check the path. ' ...
'Maybe you should use dictionary_training function to create it.'])
end
%% Processing test images
test_images = dir([test_im_path, pattern]);
num_test_images = size(test_images,1);
%Pre-allocating memory to speed-up
estimated_count = zeros(1,num_test_images);
true_count = zeros(1,num_test_images);
estimated_upper_count = zeros(1,num_test_images);
estimated_lower_count = zeros(1,num_test_images);
true_upper_count = zeros(1,num_test_images);
true_lower_count = zeros(1,num_test_images);
%Calculating dimensions of the image subregion where we can count
im_test = imread([test_im_path test_images(1).name]);
dis = round(patch_size/2);
dim_x = dis:size(im_test,2)-dis+1;
dim_y = dis:size(im_test,1)-dis+1;
toGaussian = fspecial('gaussian', hsize, sigma);
parfor a=1:num_test_images
disp(['Processing image #' num2str(a) ' of ' num2str(num_test_images) '...']);
im_test = imread([test_im_path test_images(a).name]);
[~, name, extension] = fileparts(test_images(a).name);
im_ground_truth = imread([ground_truth_path name 'dots' extension]);
disp('Extracting features...');
features = extract_features(im_test, features_type, dic_signal, sparsity, patch_size, mean_rem_flag);
features = full(features); %ND-sparse arrays are not supported.
%Re-arranging features
features = reshape(features', size(dim_y,2), size(dim_x,2), dic_size);
%Normalizing features
max_factors_3D = repmat(max_factors_depth, [size(features,1), size(features,2)]);
max_offset_3D = repmat(max_offset_depth, [size(features,1), size(features,2)]);
features = (features-max_offset_3D)./max_factors_3D;
%%Some stuff
....
end
When i execute it i get:
Undefined function or variable 'dic_signal'.
when it arrives to the extract_features function. However, in the single thread version (with for instead of parfor) it works correctly.
Someone can give me any hint?.
Thank you.
EDIT:
dic_signal is defined and correctly loaded in load(dictionary_path);
I suspect the load command doesn't load the variables in the workers workspace, only on your MATLAB instance workspace, which is why it works in a normal for loop, but not with a parfor loop. You might want to try instead:
pctRunOnAll load(dictionary_path)
to ensure the correct data is loaded into the workspace of each of the workers.
I managed to advance with a problem I was having and am looking for advice on the structure of my outer for loop. My inner loop works correctly. I want to repeat the 11 calculations 6 times using a different height (value of h) for each iteration. On the iterations of the J loop h increases to it's original value + 2*h. POI in the K loop depends on h and is defined but has been omitted for readability. Can update with the full script if necessary. Should the definition of POI be within the nested loop?
h = 0.5985;
GroundDistance = 18.75;
SouthAngle = 57;
TreeHeight = 14;
POI = TreeHeight-h;
BuildingElevationAng = atand(POI/GroundDistance);
a=0.265;
TreeLengthRHS = 15.89+a;
h = 0.5985;
X(1,1)=h;
for k = 2:6;
X(k) = X(k-1) + (X(1)*2);
end
for J = 1:6
h=h+(2*h);
for K = 1:11
TreeLengthTest(K) = TreeLengthRHS + (2*(K-1)*a);
TreeLengthLHS = 75 - TreeLengthTest(K);
AngleBx(K) = atand(TreeLengthLHS/GroundDistance);
AngleCx(K) = atand(TreeLengthTest(K)/GroundDistance); %wasTreeLengthRHS
DistanceAx(K) = GroundDistance/cosd(SouthAngle);
DistanceBx(K) = GroundDistance/cosd(AngleBx(K));
DistanceCx(K) = GroundDistance/cosd(AngleCx(K));
AltAngleA(K) = atand(POI/DistanceAx(K));
AltAngleB(K) = atand(POI/DistanceBx(K));
AltAngleC(K) = atand(POI/DistanceCx(K));
AzimuthA = 0;
AzimuthB = (-AngleBx)-SouthAngle;
AzimuthC = AngleCx-SouthAngle;
end
end
Any help is greatly appreciated.
Well, there are a couple things I'm not sure of, but I'll give a shot at an answer, and see if this helps. The definition of POI does not have to be inside the inner loop, because POI is constant for each of those 11 calculations. However, I think you're defining h a little bit incorrectly, because you'll never get to use the value of h that you define initially. Since you already have the X vector, what if you try your nested for loops like:
for J = 1:6
POI = TreeHeight - X(J);
for K = 1:11
...All your code here...
end
end
I am trying to parallize two of my for-loops and run it on a remote cluster.
I am using matlabpool open local 12 at the beginning with matlabpool close at the end. The problem I am running into is that my parfor-loop cannot use my matric properly and I am not sure how I would rewrite it so that it works.
H = hadamard(n);
H = [H;-H];
P = setdiff(P,H,'rows');
[r,c] = size(P);
A = zeros(n,r);
parfor i=1:r
for j=1:n
d = P(i,:) + H(j,:);
A(j,i) = sum(d(:) ~= 0);
end
end
and:
u2Had = cell(2,r);
parfor i =1:r
u2Had{1,i} = min(A(:,i));
MinHadIndex = find(A(:,i) == u2Had{1,i});
u2Had{2,i} = MinHadIndex;
end
Those are the two segments of the code I am trying to parallize. Any help is much appreciated and if I need to add anymore information please ask.
I don't know what your problem is in the first part as it works fine (perhaps if you defined P better)
regarding the second part, you can only send information to and from parloops in narrow cases.
Here change your code to the following:
u2HadT = cell(1,r);
parfor i =1:r
temp = min(A(:,i));
MinHadIndex = find(A(:,i) == temp);
u2HadT{i} = {temp;MinHadIndex};
end
u2Had = cell(2,r);
for i =1:r
u2Had{1,i} = u2HadT{i}(1);
u2Had{2,i} = u2HadT{i}(2);
end