Variable classification error while storing structure within parfor - matlab

I have a function which runs a parfor loop. Within the loop, I call another function which generates a structure as a result. I need to store all the structures.
function myFunction(arguments)
% do some preliminary calcultions
parfor i = 1:num_sim % number of simulations
name = sprintf('result_%i',i)
% do some calculations and generate a structure as a result called "struct_result"
total_results.(name) = struct_result
end
end
This gives me an error message:
The variable total_results in a parfor cannot be classified.
How can I store the structure "struct_result" from all the simulations? It is a nested structure.

The problem here is that you are assigning to part of total_results during the loop, but not in a "sliced" manner. It's probably simpler to collect up the names and values separately, and then use cell2struct after the parfor loop, like this:
N = 10;
names = cell(1, N);
results = cell(1, N);
parfor i = 1:10
name = sprintf('result_%i',i)
names{i} = name;
results{i} = struct('a', rand(i), 'b', rand(i));
end
total_results = cell2struct(results, names, 2);
EDIT another way of slicing the outputs (as suggested in the comments) is to use
parfor i = 1:10
total_results(i).result = struct('a', rand(i), 'b', rand(i));
end
In this case, this works because the first-level indexing expression is a "sliced" expression.

Related

Function call with variable number of input arguments when number of input arguments is not explicitly known

I have a variable pth which is a cell array of dimension 1xn where n is a user input. Each of the elements in pth is itself a cell array and length(pth{k}) for k=1:n is variable (result of another function). Each element pth{k}{kk} where k=1:n and kk=1:length(pth{k}) is a 1D vector of integers/node numbers of again variable length. So to summarise, I have a variable number of variable-length vectors organised in a avriable number of cell arrays.
I would like to try and find all possible intersections when you take a vector at random from pth{1}, pth{2}, {pth{3}, etc... There are various functions on the File Exchange that seem to do that, for example this one or this one. The problem I have is you need to call the function this way:
mintersect(v1,v2,v3,...)
and I can't write all the inputs in the general case because I don't know explicitly how many there are (this would be n above). Ideally, I would like to do some thing like this;
mintersect(pth{1}{1},pth{2}{1},pth{3}{1},...,pth{n}{1})
mintersect(pth{1}{1},pth{2}{2},pth{3}{1},...,pth{n}{1})
mintersect(pth{1}{1},pth{2}{3},pth{3}{1},...,pth{n}{1})
etc...
mintersect(pth{1}{1},pth{2}{length(pth{2})},pth{3}{1},...,pth{n}{1})
mintersect(pth{1}{1},pth{2}{1},pth{3}{2},...,pth{n}{1})
etc...
keep going through all the possible combinations, but I can't write this in code. This function from the File Exchange looks like a good way to find all possible combinations but again I have the same problem with the function call with the variable number of inputs:
allcomb(1:length(pth{1}),1:length(pth{2}),...,1:length(pth{n}))
Does anybody know how to work around this issue of function calls with variable number of input arguments when you can't physically specify all the input arguments because their number is variable? This applies equally to MATLAB and Octave, hence the two tags. Any other suggestion on how to find all possible combinations/intersections when taking a vector at random from each pth{k} welcome!
EDIT 27/05/20
Thanks to Mad Physicist's answer, I have ended up using the following which works:
disp('Computing intersections for all possible paths...')
grids = cellfun(#(x) 1:numel(x), pth, 'UniformOutput', false);
idx = cell(1, numel(pth));
[idx{:}] = ndgrid(grids{:});
idx = cellfun(#(x) x(:), idx, 'UniformOutput', false);
idx = cat(2, idx{:});
valid_comb = [];
k = 1;
for ii = idx'
indices = reshape(num2cell(ii), size(pth));
selection = cellfun(#(p,k) p{k}, pth, indices, 'UniformOutput', false);
if my_intersect(selection{:})
valid_comb = [valid_comb k];
endif
k = k+1;
end
My own version is similar but uses a for loop instead of the comma-separated list:
disp('Computing intersections for all possible paths...')
grids = cellfun(#(x) 1:numel(x), pth, 'UniformOutput', false);
idx = cell(1, numel(pth));
[idx{:}] = ndgrid(grids{:});
idx = cellfun(#(x) x(:), idx, 'UniformOutput', false);
idx = cat(2, idx{:});
[n_comb,~] = size(idx);
temp = cell(n_pipes,1);
valid_comb = [];
k = 1;
for k = 1:n_comb
for kk = 1:n_pipes
temp{kk} = pth{kk}{idx(k,kk)};
end
if my_intersect(temp{:})
valid_comb = [valid_comb k];
end
end
In both cases, valid_comb has the indices of the valid combinations, which I can then retrieve using something like:
valid_idx = idx(valid_comb(1),:);
for k = 1:n_pipes
pth{k}{valid_idx(k)} % do something with this
end
When I benchmarked the two approaches with some sample data (pth being 4x1 and the 4 elements of pth being 2x1, 9x1, 8x1 and 69x1), I got the following results:
>> benchmark
Elapsed time is 51.9075 seconds.
valid_comb = 7112
Elapsed time is 66.6693 seconds.
valid_comb = 7112
So Mad Physicist's approach was about 15s faster.
I also misunderstood what mintersect did, which isn't what I wanted. I wanted to find a combination where no element present in two or more vectors, so I ended writing my version of mintersect:
function valid_comb = my_intersect(varargin)
% Returns true if a valid combination i.e. no combination of any 2 vectors
% have any elements in common
comb_idx = combnk(1:nargin,2);
[nr,nc] = size(comb_idx);
valid_comb = true;
k = 1;
% Use a while loop so that as soon as an intersection is found, the execution stops
while valid_comb && (k<=nr)
temp = intersect(varargin{comb_idx(k,1)},varargin{comb_idx(k,2)});
valid_comb = isempty(temp) && valid_comb;
k = k+1;
end
end
Couple of helpful points to construct a solution:
This post shows you how to construct a Cartesian product between arbitrary arrays using ndgrid.
cellfun accepts multiple cell arrays simultaneously, which you can use to index specific elements.
You can capture a variable number of arguments from a function using cell arrays, as shown here.
So let's get the inputs to ndgrid from your outermost array:
grids = cellfun(#(x) 1:numel(x), pth, 'UniformOutput', false);
Now you can create an index that contains the product of the grids:
index = cell(1, numel(pth));
[index{:}] = ndgrid(grids{:});
You want to make all the grids into column vectors and concatenate them sideways. The rows of that matrix will represent the Cartesian indices to select the elements of pth at each iteration:
index = cellfun(#(x) x(:), index, 'UniformOutput', false);
index = cat(2, index{:});
If you turn a row of index into a cell array, you can run it in lockstep over pth to select the correct elements and call mintersect on the result.
for i = index'
indices = num2cell(i');
selection = cellfun(#(p, i) p{i}, pth, indices, 'UniformOutput', false);
mintersect(selection{:});
end
This is written under the assumption that pth is a row array. If that is not the case, you can change the first line of the loop to indices = reshape(num2cell(i), size(pth)); for the general case, and simply indices = num2cell(i); for the column case. The key is that the cell from of indices must be the same shape as pth to iterate over it in lockstep. It is already generated to have the same number of elements.
I believe this does the trick. Calls mintersect on all possible combinations of vectors in pth{k}{kk} for k=1:n and kk=1:length(pth{k}).
Using eval and messing around with sprintf/compose a bit. Note that typically the use of eval is very much discouraged. Can add more comments if this is what you need.
% generate some data
n = 5;
pth = cell(1,n);
for k = 1:n
pth{k} = cell(1,randi([1 10]));
for kk = 1:numel(pth{k})
pth{k}{kk} = randi([1 100], randi([1 10]), 1);
end
end
% get all combs
str_to_eval = compose('1:length(pth{%i})', 1:numel(pth));
str_to_eval = strjoin(str_to_eval,',');
str_to_eval = sprintf('allcomb(%s)',str_to_eval);
% use eval to get all combinations for a given pth
all_combs = eval(str_to_eval);
% and make strings to eval in intersect
comp = num2cell(1:numel(pth));
comp = [comp ;repmat({'%i'}, 1, numel(pth))];
str_pattern = sprintf('pth{%i}{%s},', comp{:});
str_pattern = str_pattern(1:end-1); % get rid of last ,
strings_to_eval = cell(length(all_combs),1);
for k = 1:size(all_combs,1)
strings_to_eval{k} = sprintf(str_pattern, all_combs(k,:));
end
% and run eval on all those strings
result = cell(length(all_combs),1);
for k = 1:size(all_combs,1)
result{k} = eval(['mintersect(' strings_to_eval{k} ')']);
%fprintf(['mintersect(' strings_to_eval{k} ')\n']); % for debugging
end
For a randomly generated pth, the code produces the following strings to evaluate (where some pth{k} have only one cell for illustration):
mintersect(pth{1}{1},pth{2}{1},pth{3}{1},pth{4}{1},pth{5}{1})
mintersect(pth{1}{1},pth{2}{1},pth{3}{1},pth{4}{2},pth{5}{1})
mintersect(pth{1}{1},pth{2}{1},pth{3}{1},pth{4}{3},pth{5}{1})
mintersect(pth{1}{1},pth{2}{1},pth{3}{2},pth{4}{1},pth{5}{1})
mintersect(pth{1}{1},pth{2}{1},pth{3}{2},pth{4}{2},pth{5}{1})
mintersect(pth{1}{1},pth{2}{1},pth{3}{2},pth{4}{3},pth{5}{1})
mintersect(pth{1}{2},pth{2}{1},pth{3}{1},pth{4}{1},pth{5}{1})
mintersect(pth{1}{2},pth{2}{1},pth{3}{1},pth{4}{2},pth{5}{1})
mintersect(pth{1}{2},pth{2}{1},pth{3}{1},pth{4}{3},pth{5}{1})
mintersect(pth{1}{2},pth{2}{1},pth{3}{2},pth{4}{1},pth{5}{1})
mintersect(pth{1}{2},pth{2}{1},pth{3}{2},pth{4}{2},pth{5}{1})
mintersect(pth{1}{2},pth{2}{1},pth{3}{2},pth{4}{3},pth{5}{1})
mintersect(pth{1}{3},pth{2}{1},pth{3}{1},pth{4}{1},pth{5}{1})
mintersect(pth{1}{3},pth{2}{1},pth{3}{1},pth{4}{2},pth{5}{1})
mintersect(pth{1}{3},pth{2}{1},pth{3}{1},pth{4}{3},pth{5}{1})
mintersect(pth{1}{3},pth{2}{1},pth{3}{2},pth{4}{1},pth{5}{1})
mintersect(pth{1}{3},pth{2}{1},pth{3}{2},pth{4}{2},pth{5}{1})
mintersect(pth{1}{3},pth{2}{1},pth{3}{2},pth{4}{3},pth{5}{1})
mintersect(pth{1}{4},pth{2}{1},pth{3}{1},pth{4}{1},pth{5}{1})
mintersect(pth{1}{4},pth{2}{1},pth{3}{1},pth{4}{2},pth{5}{1})
mintersect(pth{1}{4},pth{2}{1},pth{3}{1},pth{4}{3},pth{5}{1})
mintersect(pth{1}{4},pth{2}{1},pth{3}{2},pth{4}{1},pth{5}{1})
mintersect(pth{1}{4},pth{2}{1},pth{3}{2},pth{4}{2},pth{5}{1})
mintersect(pth{1}{4},pth{2}{1},pth{3}{2},pth{4}{3},pth{5}{1})
As Madphysicist pointed out, I misunderstood the initial structure of your initial cell array, however the point stands. The way to pass an unknown number of arguments to a function is via comma-separated-list generation, and your function needs to support it by being declared with varargin. Updated example below.
Create a helper function to collect a random subcell from each main cell:
% in getRandomVectors.m
function Out = getRandomVectors(C) % C: a double-jagged array, as described
N = length(C);
Out = cell(1, N);
for i = 1 : length(C)
Out{i} = C{i}{randi( length(C{i}) )};
end
end
Then assuming you already have an mintersect function defined something like this:
% in mintersect.m
function Intersections = mintersect( varargin )
Vectors = varargin;
N = length( Vectors );
for i = 1 : N; for j = 1 : N
Intersections{i,j} = intersect( Vectors{i}, Vectors{j} );
end; end
end
Then call this like so:
C = { { 1:5, 2:4, 3:7 }, {1:8}, {2:4, 3:9, 2:8} }; % example double-jagged array
In = getRandomVectors(C); % In is a cell array of randomly selected vectors
Out = mintersect( In{:} ); % Note the csl-generator syntax
PS. I note that your definition of mintersect differs from those linked. It may just be you didn't describe what you want too well, in which case my mintersect function is not what you want. What mine does is produce all possible intersections for the vectors provided. The one you linked to produces a single intersection which is common to all vectors provided. Use whichever suits you best. The underlying rationale for using it is the same though.
PS. It is also not entirely clear from your description whether what you're after is a random vector k for each n, or the entire space of possible vectors over all n and k. The above solution does the former. If you want the latter, see MadPhysicist's solution on how to create a cartesian product of all possible indices instead.

How to correct "Valid indices for 'variable' are restricted in PARFOR loops" error in matlab

I am trying to set up a parfor nested loop in MatLab R2016a as below.
N = size(A,1);
M = size(v,1);
in = zeros(N*M,1);
parfor i=1:N
for j=1:M
k = (i-1)*M+j;
if sqrt(sum((A(i,:)-v(j,:)).^2))<=tol
in(k) = i;
end
end
end
However, I am getting the following error Valid indices for 'in' are restricted in PARFOR loops. Is there some way I can correct this because both arrays A and v are considerably large, over 40,000 rows for A and 8,000 v? The variable tol is 0.0959.
The problem is that MATLAB doesn't recognize that the variable k is slicing the matrix in correctly. The solution should be to index in using i and j separately:
N = size(A,1);
M = size(v,1);
in = zeros(M,N);
parfor i=1:N
for j=1:M
if sqrt(sum((A(i,:)-v(j,:)).^2))<=tol
in(j,i) = i;
end
end
end
in = in(:); % reshape to a column vector, as the output in the question's code
The other alternative, but it requires more intermediate memory, is to compute this without a loop at all:
A = reshape(A,1,N,[]);
v = reshape(v,M,1,[]);
in = sum(bsxfun(#minus,A,v).^2,3) < tol*tol;
in = in(:);
(Or something similar to that, I have not run this code... Please let me know, or fix the post, if there is a typo or other mistake.)
N = size(A,1);
M = size(v,1);
in = cell(N,1);
parfor i=1:N
s=v;
p=zeros(1:M,1);
for j=1:M
k = (i-1)*M+j;
if sqrt(sum((A(i,:)-s(j,:)).^2))<=tol
p(k) = i;
end
end
in{i}=single(p);
end
in=cell2mat(in);
in=reshape(in,[N*M,1]);
sometime matlab doesnt recognize a variable in parfor loop as "sliced variable"
, a sliced variable is a variable that has a reference out of parfor loop and each of its element only accessed by a single worker (in parfor parallel workers)
so you could use a temporary variable and collect results after the parfor loop,
NOTE 1: It is better to vectorise code in the older versions because loops didn't use to be as good as they are now in R2017 referring to (this).
NOTE 2: if most elements of "in" are zero try using "sparse matrix" which could save a lot of memory;

Sliced Variables in PARFOR loop: Sequential to Parallel Conversion in MATLAB

I have a code in MATLAB in which I'm running monte-carlo simulations using parfor instead of simple for loop to convert the code from sequential to parallel. Following is the piece of code which is inside the parfor loop.
But MATLAB gives an error saying "Valid indices for local_Q_mega_sub_seed are restricted in parfor loop". Suggested action says to "Fix the index" and it suggests to use "Sliced Variables". I've been struggling to use this concept. I have read https://blogs.mathworks.com/loren/2009/10/02/using-parfor-loops-getting-up-and-running/#12 and https://www.mathworks.com/matlabcentral/answers/123922-sliced-variables-in-parfor-loop-restricted-indexing along with MATLAB documentation https://www.mathworks.com/help/distcomp/sliced-variable.html and https://www.mathworks.com/help/distcomp/parfor.html but I'm not getting it right.
Could anyone please let me know how can I use sliced variables in the given piece of code so that I could get an idea?
index_f = 1;
subseed_step = (sub_seed_transmitted_at/fs_local)*sintablen_mega_frequency;
for i = 1 : fs_local
local_Q_mega_sub_seed(i) = SINTAB(round(index_f));
local_I_mega_sub_seed(i) = COSTAB(round(index_f));
index_f = index_f + subseed_step;
if index_f>sintablen_mega_frequency
index_f = index_f - sintablen_mega_frequency;
end
You're not showing enough context here, but I bet the problem here is similar to this one:
parfor ii = 1:10
for jj = 1:10
tmp(jj) = rand
end
out(ii) = sum(tmp);
end
In this case, the parfor machinery cannot categorically prove that the way tmp is being used is independent of the order of iterations of the parfor loop. This is because it appears as though values assigned to tmp in one iteration of the parfor loop are still being used in the next iteration.
Fortunately, there's a very simple workaround for this case - convince parfor that you are not doing anything dependent on the order of evaluation of the iterations of the loop by resetting the variable. In the simple case above, this means:
parfor ii = 1:10
% reset 'tmp' at the start of each parfor loop iteration
tmp = [];
for jj = 1:10
tmp(jj) = rand
end
out(ii) = sum(tmp);
end

Assigning dynamic values to variables in base workspace in MATLAB

load( lapFileSource, 'UntitledMeta_Data' );%My MetaData
universal={'TestType';'TestApparatus';'TestSystem Location';
'Configuration';'Wire condition';'Wire Type';'Circuit';};
u=11;
for o=drange(1:u)
if strcmp('',MetaData{o})
universal{o}='Null';
else
universal{o}=MetaData{o};
end
assignin('base','universal{o}',MetaData{o})
end
I am getting error to assignin the variable in workplace.
You need to read the documentation for the functions you use. From the documentation for assignin:
% The var input must be the array name only; it cannot contain array indices.
And, even more explicitly:
% assignin does not assign to specific elements of an array.
% The following statement generates an error:
X = 1:8;
assignin('base', 'X(3:5)', -1);
You can either shift your assignin call outside of the for loop and pass your full array to the base workspace, or you can follow the advice of the documentation and utilize evalin.
For example:
function trialcode
n = 5;
myarray = cell(1, 5);
for ii = 1:n;
if mod(ii,2)
myarray{ii} = 'odd';
else
myarray{ii} = 'even';
end
mycmd = sprintf('evalinarray{%u} = ''%s'';', ii, myarray{ii});
evalin('base', mycmd);
end
assignin('base','assigninarray', myarray)
end
The resulting arrays, evalinarray and assigninarray, are equivalent. Of the two, I would recommend using assignin as it is a lot more robust and explicit than the evalin approach.

How to use a variable outside a PARFOR loop in MATLAB?

In MATLAB, I have a variable proba and I have a parfor loop as showing below:
parfor f = 1:N
proba = (1/M)*ones(1, M);
% rest of the code
end
pi_proba = proba;
MATLAB said that: "The temporary variable 'proba' is used after the PARFOR loop, but its value is nondeterministic"
I do not understand how to correct this error. I need to use a parallel loop and I need proba after the loop. How to do this?
When using parfor the classes are classified according to these categories. Make sure every variable matches one of these categories. For non-writing access to proba a Broadcast-Variable would be the best choice:
proba = (1/M)*ones(1, M);
parfor f = 1:N
% rest of the code
end
pi_proba = proba;
In case of writing access within the loop, a sliced variable is nessecary:
proba=cell(1,N)
parfor f = 1:N
%now use proba{f} inside the loop
proba{f}=(1/M)*ones(1, M);
% rest of the code
end
%get proba from whatever iteration you want
pi_proba = proba{N};