This question already has an answer here:
parfor doesn't consider information about vectors which are used in it
(1 answer)
Closed 5 years ago.
I want to parallelize some part of my code in matlab. for example below part:
v1=[1,3,6,8];
ggx=5.*ones(15,14);
gax=ones(15,14);
parfor i = 1:length(v1)
m = v1(i);
if m > 1
gax(1:m-1,m-1) = ggx(1:m-1,m-1);
end
if m<nn
gax(m+1:end,m) = ggx(m+1:end,m);
end
end
But there is an error:
Error: The variable gax in a parfor cannot be classified.See Parallel for Loops in MATLAB, "Overview".
Do anyone knows how can I remove the error? Other useful information is that v1 is an increasing vector which doesn't contain any repeated element.
As mentioned in the error message, you must follow the Sliced Variable rule.
Both gax and gay breaks rules of Fixed Index Listing and Form of Indexing. Also, you can this example A(i,20:30,end) % 20:30 not scalar as an example of not sliced variables in the document.
Therefore, you should change the all part of the parfor to getting a proper parallel computation. In the other words, you must design a proper parallel algorithm which you can parallelized the method, based on the loop variable.
Type of First-Level Indexing — The first level of indexing is either parentheses, (), or braces, {}.
Fixed Index Listing — Within the first-level parentheses or braces, the list of indices is the same for all occurrences of a given variable.
Form of Indexing — Within the list of indices for the variable, exactly one index involves the loop variable.
Shape of Array — The array maintains a constant shape. In assigning to a sliced variable, the right side of the assignment cannot be [] or '', because these operators attempt to delete elements.
Related
I want to paralyze my forloop in Matlab.I use parfor function for that, but I get error because the way I used variable inside of the loop. would someone help me to fix that. I'm new in matlab.
Here is part of my try.
Here is part of problematic part:
CV_err=zeros(length(gamma), (Num_Tasks + 1));
parfor k=1:length(gamma)
#block of code
#
CV_err(k,1:Num_Tasks)= sum(In_Fold_Error)./size(In_Fold_Error,1);
CV_err(k,Lambda_location)= Lambda;
CV_err(k,(Num_Tasks +2))= sum(CV_err(k,1:Num_Tasks))/Num_Tasks;
end
Error: parfor loop can not run due to way CV_err is used.
CV_err is indexed in different ways, potentially causing dependencies
Seems that valid indices are restricted in parfor .
While your variable is sliced, you only access the k-th row in the k-th iteration, the code analyser does not understand it. Give a little help to matlab, first put all data into a vector and then write all at once to the sliced variable.
CV_err=zeros(length(gamma), (Num_Tasks + 2));
parfor k=1:length(gamma)
%block of code
%
temp=zeros(1,(Num_Tasks + 2));
temp(1,1:Num_Tasks)= sum(In_Fold_Error)./size(In_Fold_Error,1);
temp(1,Lambda_location)= Lambda;
temp(1,(Num_Tasks +2))= sum(temp(1,1:Num_Tasks))/Num_Tasks;
CV_err(k,:)=temp;
end
The limitation is explained in the documentation:
Form of Indexing. Within the list of indices for a sliced variable, one of these indices is of the form i, i+k, i-k, k+i, or k-i, where i is the loop variable and k is a constant or a simple (nonindexed) broadcast variable; and every other index is a scalar constant, a simple broadcast variable, a nested for-loop index, colon, or end.
Source
To fix pre-allocation, don't pre-allocate. You're just telling to MATLAB how it should split the work among workers; parfor doesn't like that.
The answer is: don't make loops change common variables, write your results separately, grow cell arrays instead of matrices, i.e.
clear CV_err;
parfor k=1:length(gamma)
%// here your other code
this_CV_err = zeros(Num_Tasks+2,1);
this_CV_err(1:Num_Tasks) = sum(In_Fold_Error)./size(In_Fold_Error,1);
this_CV_err(Lambda_location) = Lambda;
this_CV_err(Num_Tasks+2) = mean(this_CV_err(1:Num_Tasks));
CV_err{k} = this_CV_err;
end;
I ran into a weird problem when I used structure arrays inside a parfor loop. Below is my main function:
comm_struct=struct('field1',0,'field2',zeros(10,1),'field3',sparse(zeros(10,10)));
a(1:100,1)=comm_struct;
b(1:10,1)=comm_struct;
disp(size(b))
parfor ct=1:12
b=max_select_2(b,a);
end
where max_select_2 is
function y=max_select_2(b,a)
if size(b,1)~=10
error('size of b is not 10 but %d',size(b,1)),
end
d(1:110,1)=struct('field1',0,'field2',zeros(10,1),'field3',sparse(zeros(10,10)));
d(1:10,1)=b;
d(11:110,1)=a;
[~,sort_idx]=sort([d.field1],'descend');
y=d(sort_idx(1:10));
If I replace the parfor with |for| and ran on my local matlab, no problem, but if I use parfor and run this on a cluster the output is
10 1
which is expected but I also got the error message
"size of b is not 10 but 100"
which means for some reason when b is passed to the function max_select_2, its size changed from 10 to 100? I doubt that it has something to do with how matlab handles the structure array but I don't really know what's going on and how to fix it...
Any help is appreciated! Thanks!
Li
The accumulation function you are using is not allowed (reference)
Unless f is a known noncommutative built-in, it is assumed to be commutative. There is currently no way to specify a user-defined, noncommutative function in parfor.
From my understanding of your code, I recommend to use ; (concatenate) to accumulate and sort outside the parfor loop. Another alternative would be a sliced matrix where each iteration is allowed to fill one column of 10 elements, finally sorting and reshaping it to a vector outside the loop.
This question already has answers here:
Dynamic variables matlab
(2 answers)
Closed 8 years ago.
I was wondering how I can use a for loop to create multiple matrix when given a certain number.
Such as If 3 was given I would need three matricies called: C1, C2 and C3.
k = 3
for i = 1:K
C... = [ ]
end
Not sure how to implement this.
The first thing coming in mind is the eval function mentioned by Dennis Jaheruddin, and yes, it is bad practice. So does the documentation say:
Why Avoid the eval Function?
Although the eval function is very powerful and flexible, it not
always the best solution to a programming problem. Code that calls
eval is often less efficient and more difficult to read and debug than
code that uses other functions or language constructs. For example:
MATLAB® compiles code the first time you run it to enhance performance for future runs. However, because code in an eval
statement can change at run time, it is not compiled.
Code within an eval statement can unexpectedly create or assign to a variable already in the current workspace, overwriting existing
data.
Concatenating strings within an eval statement is often difficult to read. Other language constructs can simplify the syntax in your
code.
The "safer" alternative is the function assignin:
The following will do exactly what you want:
letter = 'C';
numbers = 1:3;
arrayfun(#(x) assignin('base',[letter num2str(x)],[]),numbers)
I know cases where you need to create variables likes this, but in most cases it is better and more convenient to use cell arrays or structs.
The trick is to use a cell array:
k=3
C=cell(k,1)
for t=1:k
C{t}= rand(t)
end
If each matrix is of equal size, you would probably just want a three dimensional matrix rather than a cell array.
You now have one cell array,but can access the matrices like this:
C{2} %Access the second matrix.
The use of eval is almost inevitable in this case:
k = 3;
for i = 1:k
eval(sprintf('C%d = [];', i));
end;
Mind you that generating variable names for data storage rather than indexing them (numerically -- see the solution of Dennis Jaheruddin based on cell arrays -- or creating a dynamic struct with named fields that store the data) is almost always a bad idea. Unless you do so to cope up with some weird scripts that you don't want to / cannot modify that need some variables already created in the global workspace.
I am trying to use MATLAB in order to generate a variable whose elements are either 0 or 1. I want to define this variable using some kind of concatenation (equivalent of Java string append) so that I can add as many 0's and 1's according to some upper limit.
I can only think of using a for loop to append values to an existing variable. Something like
variable=1;
for i=1:N
if ( i%2==0)
variable = variable.append('0')
else
variable = variable.append('1')
i=i+1;
end
Is there a better way to do this?
In MATLAB, you can almost always avoid a loop by treating arrays in a vectorized way.
The result of pseudo-code you provided can be obtained in a single line as:
variable = mod((1:N),2);
The above line generates a row vector [1,2,...,N] (with the code (1:N), use (1:N)' if you need a column vector) and the mod function (as most MATLAB functions) is applied to each element when it receives an array.
That's not valid Matlab code:
The % indicates the start of a comment, hence introducing a syntax error.
There is no append method (at least not for arrays).
Theres no need to increment the index in a for loop.
Aside of that it's a bad idea to have Matlab "grow" variables, as memory needs to be reallocated at each time, slowing it down considerably. The correct approach is:
variable=zeros(N,1);
for i=1:N
variable(i)=mod(i,2);
end
If you really do want to grow variables (some times it is inevitable) you can use this:
variable=[variable;1];
Use ; for appending rows, use , for appending columns (does the same as vertcat and horzcat). Use cat if you have more than 2 dimensions in your array.
function letter=read_char(imgn, num_letters)
global templates
comp=[ ];
for n=1:num_letters
sem=corr2(imgn, templates{1,n});
comp=[comp sem];
end
vd=find(comp==max(comp));
Can someon please explain what the 'FOR' loop does and how 'vd' is calculated?
Also, what is the variable 'comp' and what will it looks like as the array also contains itself and another variable 'sem' which is calculated from the corr2 function.
thanks
The for loop creates a loop variable n that starts at 1 for the first loop and is incremented by 1 for each successive loop until it reaches num_letters. This means the loop will execute num_letters times.
The variable comp is first initialized to the empty matrix []. Within the loop, a 2-D correlation coefficient is computed between the matrix imgn and another matrix templates{1,n} (indexed from a cell array) using the function CORR2. The correlation coefficient sem is appended to the array comp using horizontal concatenation. This will ultimately result in comp being a 1-by-num_letters array of correlation coefficients.
The variable vd stores the array indices where values in comp are equal to the maximum value found in comp. This is done using the functions MAX and FIND and the comparison operator ==.
In the future, I would urge you to first use the online documentation to try and help you better understand how MATLAB works. It is very good documentation. I've learned most of what I know from it. ;)