The following loop results in an error in C_mat and B_mat:
%previously defined
N_RIPETIZIONI=2;
K=201;
parfor n=1:N_RIPETIZIONI*K
[r,k]=ind2sub([N_RIPETIZIONI,K],n);
B=B_mat{r};
C=C_mat{r};
end
The warning says:
The entire array or structure B_mat is a broadcast variable. This might result in unnecessary communication overhead.
The same for C_mat.
How can I fix it so that the indices of B_mat and C_mat are no more broadcast variables?
The issue is that the way you index B_mat (i.e. not using n), every thread in the parfor requires the entirety of B_mat to run. The big bottleneck in parfor code is transferring copies of the data to each node.
MATLAB is basically telling you that if you were to do this, you may actually have slower code than otherwise. Its not that B_mat is some type of variable called "broadcast", its that the way you wrote the code, each n in parfor requires a copy of B_mat.
I assume this is not your real code, so we can't really help you fix it, but hopefully this explains it.
Related
I am trying to use a table within a parfor loop in MATLAB. This gives me "Transparency violation error. See Parallel Computing Toolbox about transparency" I'm trying to build this table so I can make a prediction using a trained classifier from the MATLAB classification learner app (trainedClassifier.prefictFcn(T))...so either I need to build a table within the parfor loop or need some alternative to a table that I can still feed into the classifier.
parfor i=1:100
acheck=1;
bcheck=2;
ccheck=3;
T=table(acheck,bcheck,ccheck);
end
This solution works for your particular problem:
parfor i=1:100
acheck=1;
bcheck=2;
ccheck=3;
T(i,:)=table([acheck,bcheck,ccheck]);
end
Note that in your original program you just overwrite existing values and end up with a one row table. I assumed that that was not intended. Actually, that would be the outcome of a for.
Also, since this is a parfor and T is created inside the loop (as well as acheck, etc.) using just T creates nothing at all. The variable is a temporary one, visible to each process locally and destroyed in global scope (more can be found here).
To fix both overwriting and accessibility the program assigns the each set of variables to each row of T. If square brackets are omitted, the program throws a transparency error. Unfortunately, I do not know why is that but it may be that the operations done by the table data-structure cause that. Maybe someone else will know the answer, for now this seem to solve your problem though.
while using Matlab parfor I came across the following behaviour
parpool(2)
parfor j=1:100
v = j+1;
clear v
end
> Error in ==> parallel_function>make_general_channel/channel_general at 886
> Transparency violation error.
I looked into it, and indeed one is not allowed to use clear within parfor.
My question is why. v is created inside every specific worker, and so it does not interfere with other workers.
Matlab uses static code analyzer to understand how the body of parfor loop interacts with main workspace, i.e. which variables need to be transferred to workers and back. A number of functions, such as eval, evalc, evalin, assignin (with the workspace argument specified as 'caller'), load (unless the output is assigned to a variable), save and clear can modify workspace in ways that cannot be predicted by the static analyzer. There is no way to ensure integrity of the workspace when multiple workers are operating on it, and such functions are used.
Important thing to realize is that when you use a command syntax to invoke a function, such as clear v, the argument is passed as a string literal, meaning there is no way for the static analyzer to understand which variable you are trying to clear, hence no way to figure out the effect the command will have on the workspace.
As suggested in documentation, the workaround to free up most of the memory used by a variable inside parfor is: v = [];
I am now trying to do parallel computing in Matlab and want to use parfor loop to improve the efficiency. The problem is I can guarantee that each loop is independent with each other but I finally need to update a global variable (maybe called broadcast variable in Matlab), when I want to assign some value to it there is a problem says it can't be classified. If I still want to do it in this Matlab, how can I solve this problem or is there any other way I can try to improve the efficiency?
The code is like this:
Atoms(1:nOfAtomsInTwoDim,:)=TwoDimAtoms;
odd_type=TwoDimAtoms;
even_type=TwoDimAtoms;
even_type(:,1)=TwoDimAtoms(:,1)+LatticeSpacing/2;
even_type(:,2)=TwoDimAtoms(:,2)+LatticeSpacing/2;
parfor i=2:1:nOflayers+1
temp_type=TwoDimAtoms;
if mod(i,2)
temp_type=odd_type;
temp_type(:,3)=TwoDimAtoms(:,3)+(i-1)*LatticeSpacing/2;
else
temp_type=even_type;
temp_type(:,3)=TwoDimAtoms(:,3)+(i-1)*LatticeSpacing/2;
end
iBegin=(i-1)*nOfAtomsInTwoDim+1;
iEnd=i*nOfAtomsInTwoDim;
Atoms(iBegin,iEnd,:)=temp_type;
end
Your code isn't executable which makes it slightly tricky to work out what's going on, and as #PetrH points out I assume your indexing expression at the end is intended to be Atoms(iBegin:iEnd,:).
To make this work in parfor, you need to arrange for Atoms to be sliced (broadcast variables are inputs to the parfor loop which are constant and the same for each iteration). In other words, your indexing expression needs to be something more like
parfor i = ...
...
Atoms(i, :) = ...;
end
Having said all that, if this is your entire parfor loop, I would concentrate instead on vectorising things rather than applying parfor. It appears that the amount of work inside the parfor loop is rather small and it is unlikely to give you much benefit, whereas my guess is that vectorisation should give you much better speedup.
I am calling external functions in my parfor loop as follows.
parfor idx = 1:2
import com.comsol.model.*
import com.comsol.model.util.*
model = ModelUtil.create('Model');
model.modelNode.create('comp1');
model.geom.create('geom1', 2);
model.geom('geom1').feature.create('sq1', 'Square');
model.geom('geom1').feature('sq1').set('size', '0.03125');
model.geom('geom1').feature('sq1').setIndex('pos', '0', 0);
model.geom('geom1').feature('sq1').setIndex('pos', '0', 1);
model.geom('geom1').run;
end
Error: MATLAB cannot determine whether "ModelUtil" refers to a function or variable.
See Parallel for Loops in MATLAB, "Unambiguous Variable Names".
After reading the "Unambiguous Variable Names" part in the MATLAB parfor documentation, I pretty much understand why this error occurs. However, I have no idea how to fix it.
I encountered the same problem with .Net objects. Like you, I found it works perfectly in non-parallel mode and the problem only occurs with parfor.
This is not a question of paths, this is purely a Matlab syntax/parser problem.
As a workaround, I wrapped the operations in a separate Matlab function so that within the scope of the parfor it is no longer ambiguous. Within the (otherwise unnecessary) function it has no trouble resolving the class, even when called from within parfor.
Vector's solution didn't work for me. The thing is that the different workers don't see the library you are using: you have to update the javapath in the parfor with javaaddpath.
I have this piece of code which I want to run in parallel. One of the arguments passed to the function containing the parfor loop is a function handle which then is executed inside the parfor loop. Like this
[X] = randstep( X,params,roomfun )
...
parfor i=1:N
while ~ok
X(:,i) = A*X(:,i);
if roomfun(X(:,i))
ok = 1;
end
end
end
However, MATLAB complains about roomfun, saying it is indexed but not sliced. This is of course not the case since it is function which can be executed fine without the other loop iterations.
Is there any way I can tell matlab this is a function or maybe another way I can organize the loops in order to get this running in parallel?
First of all, this is only a warning, not an error. So the parfor loop should run just fine.
Secondly, the warning about something being indexed but not sliced comes from mlint, which, while pretty good, doesn't understand everything, so if it interprets something wrong, don't sweat it.
Thirdly, in R2012b, I don't even get the warning anymore (mlint has gotten smarter), which further shows that there's nothing to worry about.
Finally, if there really was a slicing problem, it's still not an issue that would prevent the loop from running in parallel - all the warning says is that an array is sent in its entirety to the workers, which can slow down the parfor loop in case the array is large.