The following code:
matlabpool('open','local',2)
parfor i=1:5
proc = System.Diagnostics.Process;
end
results in an error:
Error: MATLAB cannot determine whether "System" refers to a function or variable.
However, when I execute the parfor loop again (after the error), it runs through! I found a couple of similar questions, but I wasn't able to implement the suggested solutions.
MATLAB parfor - cannot determine whether "ModelUtil" refers to a function or variable?
MATLAB using parfor (parallel computing toolbox) and custom packages with +
I can't wrap my mind around why the loop runs the second time. If I then call
matlabpool close
and execute the whole script again, the error appears once again. So it only happens the first time after the pool was initiated. Any ideas?
This is because any variable or function you use in a parfor loop must be defined explicitly in the code at parsing time. If there is any ambiguity, Matlab opt to throw an error rather than messing up by assuming.
Just define an anonymous function that create the object you want before the parfor loop, then you can use it as will within the parfor loop.
This run fine on my machine (Matlab R2013a):
getSystemProcess = #() System.Diagnostics.Process ;
parfor i=1:5
proc = getSystemProcess();
end
Read this Matlab chapter for more informations on how the variable/function names are interpreted in a parfor loop: Unambiguous Variable Names
Related
In the following parfor loop, Matlab says the variable 'sf' cannot be classified. However, the way it is defined inside the innermost loop doesn't seem to affect parfor. Could you tell me the issue and show me how this code snippet should be modified?
parfor ii=1:1:10000
for jj=1:200
for kk=1:80
sf{kk}=fit([kk*dKy;(kk+1)*dKy],[result{kk}(ii);result{kk+1}(ii)],'exp1','lower',[kk*dKy,result{kk}(ii)]);
fun=#(t) sf{kk}(t).*cos(Ky(kk).*t);
result2{ii}(jj)=0;
result2{ii}(jj)=result2{ii}(jj)+integral(fun,Ky(kk),Ky(kk+1),'ArrayValued',true)/(2*pi);
end
end
end
I would like to run a Simulink model in a parfor loop on many cores with different data. However, I could not get the sim results when I use parfor whereas I can obtain them while using only for loop.
It simply get [t,u] from workspace1, consider a transfer function n{1}/d{1} and then calculates the EqFracInt to workspace2.
The problematic part of my code is
...
parfor ieq=1:1
assignin('base','t',t);
assignin('base','u',u);
assignin('base','n',n);
assignin('base','d',d);
assignin('base','T_end',T_end);
[simout] = sim('RespSpecFrac', [0 T_end], simset('ReturnWorkspaceOutputs','on'));
PGRs = simout.get('EqFracInt');
end
I could not get the PGRs values. Could you please explain the error to me and how to solve it?
The cause you can't get PGRs values is that is a temporary variable. Since in a parfor loop each iteration is independent, any variable assigned directly inside the loop without indexing is used only in that specific iteration and destroyed.
In order to get the desired values, you need to transform PGRs in either a sliced variable (by indexing it) or in a reduction variable (by, for example, concatenating it). Try to modify the last line of your loop following one of these examples:
PGRs(ieq, :) = simout.get('EqFracInt'); % sliced variable
or
PGRs = [PGRs; simout.get('EqFracInt')]; % reduction variable
The specific implementation will depend on the shape of the expected outputs EqFracInt, of course. You can see more about types of variables inside a parfor loop in the documentation.
I am calling external functions in my parfor loop as follows.
parfor idx = 1:2
import com.comsol.model.*
import com.comsol.model.util.*
model = ModelUtil.create('Model');
model.modelNode.create('comp1');
model.geom.create('geom1', 2);
model.geom('geom1').feature.create('sq1', 'Square');
model.geom('geom1').feature('sq1').set('size', '0.03125');
model.geom('geom1').feature('sq1').setIndex('pos', '0', 0);
model.geom('geom1').feature('sq1').setIndex('pos', '0', 1);
model.geom('geom1').run;
end
Error: MATLAB cannot determine whether "ModelUtil" refers to a function or variable.
See Parallel for Loops in MATLAB, "Unambiguous Variable Names".
After reading the "Unambiguous Variable Names" part in the MATLAB parfor documentation, I pretty much understand why this error occurs. However, I have no idea how to fix it.
I encountered the same problem with .Net objects. Like you, I found it works perfectly in non-parallel mode and the problem only occurs with parfor.
This is not a question of paths, this is purely a Matlab syntax/parser problem.
As a workaround, I wrapped the operations in a separate Matlab function so that within the scope of the parfor it is no longer ambiguous. Within the (otherwise unnecessary) function it has no trouble resolving the class, even when called from within parfor.
Vector's solution didn't work for me. The thing is that the different workers don't see the library you are using: you have to update the javapath in the parfor with javaaddpath.
Why can't I use the parfor in this piece of code?
parfor i=1:r
for j=1:N/r
xr(j + (N/r) * (i-1)) = x(i + r * (j-1));
end
end
This is the error:
Error: The variable xr in a parfor cannot be classified.
See Parallel for Loops in MATLAB, "Overview".
The issue here is that of improper indexing of the sliced array. parfor loops are run asynchronously, meaning the order in which each iteration is executed is random. From the documentation:
MATLAB workers evaluate iterations in no particular order, and independently of each other. Because each iteration is independent, there is no guarantee that the iterations are synchronized in any way, nor is there any need for this.
You can easily verify the above statement by typing the following in the command line:
parfor i=1:100
i
end
You'll see that the ordering is arbitrary. Hence if you split a parallel job between different workers, one worker has no way of telling if a different iteration has finished or not. Hence, your variable indexing cannot depend on past/future values of the iterator.
Let me demonstrate this with a simple example. Consider the Fibonacci series 1,1,2,3,5,8,.... You can generate the first 10 terms of the series easily (in a naïve for loop) as:
f=zeros(1,10);
f(1:2)=1;
for i=3:10
f(i)=f(i-1)+f(i-2);
end
Now let's do the same with a parfor loop.
f=zeros(1,10);
f(1:2)=1;
parfor i=3:10
f(i)=f(i-1)+f(i-2);
end
??? Error: The variable f in a parfor cannot be classified.
See Parallel for Loops in MATLAB, "Overview"
But why does this give an error?
I've shown that iterations are executed in an arbitrary order. So let's say that a worker gets the loop index i=7 and the expression f(i)=f(i-1)+f(i-2);. It is now supposed to execute the expression and return the results to the master node. Now has iteration i=6 finished? Is the value stored in f(6) reliable? What about f(5)? Do you see what I'm getting at? Supposing f(5) and f(6) are not done, then you'll incorrectly calculate that the 7th term in the Fibonnaci series is 0!
Since MATLAB has no way of telling if your calculation can be guaranteed to run correctly and reproduce the same result each time, such ambiguous assignments are explicitly disallowed.
I have a problem using the Parallel Toolbox from Matlab. Indeed, I want to untar a series of archive in a parfor loop, and it seems that neither untar nor system are working. They do not cause an error, they simply do not produce any result.
The very same code works without any problem as soon as I deactivate the parallelism.
Is there a reference that lists the functions that cannot be used in parfor loops? I couldn't find it easily in the parallel toolbox documentation.
system should work correctly inside a PARFOR loop - providing that the executable you invoke does not require user input.
>> matlabpool('size')
ans =
3
>> parfor ii=1:2, system('pwd'), end
/tmp
ans =
0
/tmp
ans =
0
The main constraints on functions which cannot be used directly inside a PARFOR loop body relate to "workspace transparency" - you cannot use functions that modify the workspace such as assignin, load, clear etc. See this page for more on that. (You can of course call a function from the body of a PARFOR loop which calls load etc.)
You can try something like this.
function parallel_stuff
parfor i = 1:10
b = my_untar(a)
end
end
function b = my_untar(a)
b = untar(a)
end