Good day MATLAB pros,
I have a long list of (single value) variables in my workspace that has to be stored into an array for every loop of execution.
Here's a simple example:
Variables in the workspace:
a = 1;
b = 2.2;
c = 3.4;
d = [0.5 0.7 1 1.2 1.5];
e = [-15 -10 -5 0 5 10 15 20];
serial = {'WTZ00151'};
model = {'336F'};
NameList = {'a';'serial';'model'};
1) Here, I'm saving only the single value variables into Data structure, however what I'd like to do is for every loop, save the single values into an array in Data structure.
varList = who;
Data = struct;
fields = fieldnames(Data, '-full');
fieldSizes = structfun(#(field) length(field),Data);
% removing arrays from structure
for lst = 1:length(fieldSizes)
if fieldSizes(lst) > 1
Data = rmfield(Data,fields(lst));
end
end
Data =
Data: [1x1 struct]
a: 1
b: 2.2000
c: 3.4000
index: 10
model: {'336F'}
serial: {'WTZ00151'}
So if I run this in a loop, for i = 1:5, Data should look like this:
Data =
Data: [1x1 struct]
a: [1 1 1 1 1]
b: [1x5 double]
c: [1x5 double]
index: [10 10 10 10 10]
model: {1x5 cell}
serial: {1x5 cell}
Any ideas on how to code the for loop?
2) Since there are too many variables in the workspace & I have a long list of variables that needs storing, instead of using who to save ALL variables to the structure (and then filtering out the unwanted), how could I use a list of variable names (imported from a text file: NameList) to call out what needs to be stored? Using the variable name from NameList does not call out the structure values.
Much appreciated,
It's not immediately clear what part of your code actually is creating your data structure. There are several ways to create a struct from your array of variable names.
One way is to save the relevant variables to a file and load them back into a struct
save('tmp.mat', NameList{:});
Data = load('tmp.mat');
Another option (not recommended) is to use eval
for k = 1:numel(NameList)
Data.(NameList{k}) = eval(NameList{k});
end
As far as storing data from multiple iterations, I personally would recommend storing the data into an array of struct rather than a struct of arrays. You should be able to store each Data instance in an array using k as an index as shown below:
allData(k) = Data;
If you decide you really want a struct of arrays, you can always convert it afterwards.
fields = fieldnames(allData);
output = struct();
for k = 1:numel(fields)
% Try to just concatenate the values together
try
values = [allData.(fields{k})];
% If that failed, concatenate them within a cell array
catch
values = {allData.(fields{k})};
end
% Store this in a single output struct
output.(fields{k}) = values;
end
Related
I am trying to vectorize the following data structure in Matlab but I cannot find/code an efficient way.
A = 1x2 struct array with fields: [a , b , c]
A(1) = a: 1 , b: 2 , c: [1x1 struct]
A(1).c = key: 5
A(2) = a: 1 , b: [] , c: [1x3 struct]
A(2).c = 1x3 struct array with fields: [key , key2]
A(2).c(1).key = 3
A(2).c(2).key = 4
A(2).c(3).key = 7
A(2).c(1).key2 = 10
A(2).c(2).key2 = []
A(2).c(3).key2 = 17
I know. This is a highly inefficient data structure. That's why I am trying to vectorize it with index, so the final structure will look like
A = 1x1 structure with fields [a , b , c , b_index , c_index]
A.a = [1 1]
A.b = [2]
A.b_index = [1]
A.c = 1x1 structure with fields [key key2 key2_index]
A.c_index = [1 2 2 2]
A.c.key = [5 3 4 7]
A.c.key2 = [10 17]
A.c.key2_index = [2 4]
My attempt 1:
I've first tried parfor at each level (for this example, specifically: A, c, key 3 levels) with a survey first to see if it is empty, what data it contains, do I need to index this field. and then vertcat(x.(fieldname)) if it is not a structure leaf. But if it is, I've package it up as a cell and recursively push it down to be vectorized.
That works, but it unfortunately takes too long. When I did a profile on it, it showed the mex distribution function that's taking up all the time. I'm guessing that's because I am doing parfor at every level, hence MATLAB has to index and distribute to each worker very frequently at every level.
My attempt 2:
I've tried to do a parfor survey of the structure completely first. Use a uint8 value for each field. And then at the combination stage, I use vertcat to check the survey results first to see if I need to index and if I need to do cat(3,...) for the data field. But that is memory inefficient and slow at the survey stage. And it doesn't speed up much at the combination stage. Though indexing becomes much easier.
I guess my questions are
1. How can I code it in a way that parfor only index and distribute the whole array once so my first attempt can be more efficient, or is my second attempt a better idea?
2. What is a good general approach to the problem?
My two cents for your 2. question: Matlab's parfor works faster on simple arrays/matrices. This is due to the fact that arrays are allocated contagiously in memory and thus enable faster access and computation. So, instead of having complex structures, I would suggest using simpler arrays etc. if you're more concerned with the performance of your program and not with the readability.
Say that I have two cell arrays, A and B, that contain string values. I wish to populate a struct S such I generate every possible combination of S.valueinA.valueinB = 1. I am currently trying to accomplish this with two nested for-loops that iterate through every possible combination and wanted to ask if there is a more efficient way to solve this problem in MATLAB.
If you want to make dynamic field names in structures, I don't see how else you can do it without two for loops. Let's say we have two cell arrays A and B that consist of string entries.
For my example, apologies for the strings inside these arrays in advance as I couldn't think of anything better at the moment!
Is this what you're trying to achieve?
S = struct();
A = {'hello', 'my', 'name', 'is', 'ray'};
B = {'i', 'am', 'doing', 'awesome'};
for idx = 1 : numel(A)
for idx2 = 1 : numel(B)
S.(A{idx}).(B{idx2}) = 1;
end
end
This creates a nested structure S such that for each element in A, this becomes a field in S where this field is another structure that contains fields with names coming from all elements in B.
If we displayed S, we get:
>> S
S =
hello: [1x1 struct]
my: [1x1 struct]
name: [1x1 struct]
is: [1x1 struct]
ray: [1x1 struct]
If we accessed the hello field of S, we get:
>> S.hello
ans =
i: 1
am: 1
doing: 1
awesome: 1
Similarly, if we accessed the my field, we get:
>> S.my
ans =
i: 1
am: 1
doing: 1
awesome: 1
Therefore, if we want to get the hello field followed by the am field, we do:
>> S.hello.am
ans =
1
I have defined a structures array as such:
oRandVecs = struct('vV',{[],[]},...
'ind_min',{[],[]},...
'mean',{[],[]},...
'vV_descending',{[],[]},...
'largest_diff',{[],[]});
oRandVecs(1).vV and oRandVecs(2).vv both get column vectors assigned to them. However, the problem is that the output is as follows:
>> oRandVecs(1)
ans =
vV: [4x1 double]
ind_min: 2
mean: 6.5500
vV_descending: [4x1 double]
largest_diff: 2.8000
Instead of actually showing the vector, it only describes its type.
What am I to do?
The reason why is because it's simply too big to display on the screen with that structure :) If you want to actually display it, use dot notation to display your data.
In other words, do this:
disp(oRandVecs(1).vV);
You can also do that with the other variable:
disp(oRandVecs(1).vV_descending);
The answer by rayryeng is probably the way to go.
An alternative is to convert from structure to cell and then use celldisp:
celldisp(struct2cell(oRandVecs(1)))
Example:
>> oRandVecs = struct('vV',{[],[]},...
'ind_min',{[],[]},...
'mean',{[],[]},...
'vV_descending',{[],[]},...
'largest_diff',{[],[]}); %// define empty struct
>> oRandVecs(1).vV = (1:4).'; %'// fill some fields: column vector, ...
>> oRandVecs(1).mean = 5:7; %// ... row vector, ...
>> oRandVecs(1).vV_descending = randn(2,3); %// ... matrix
>> celldisp(struct2cell(oRandVecs(1)))
ans{1} =
1
2
3
4
ans{2} =
[]
ans{3} =
5 6 7
ans{4} =
0.016805198757746 0.236095190511728 0.735153386198679
2.162769508502985 -0.158789830267017 0.661856091557715
ans{5} =
[]
I'm reading in a mat file which has variables with a counter e.g. a1, a2, a3, b, c1 and c2.
I want to put the data into a structure with field names "a", "b" and "c" where
a=[a1;a2;a3], b=b and c = [c1; c2].
The algorithm needs to be able to do this, without knowing what the maximum value of the counter of each variable is. How can I do this?
First, you would load your mat file data using the load function, placing it in a structure. Let's say that gave you the following sample data:
>> s = struct('a1',1:5,'b',[],'a2',1:5,'a3',1:5,'c1',1:3,'c2',3:5)
s =
a1: [1 2 3 4 5]
b: []
a2: [1 2 3 4 5]
a3: [1 2 3 4 5]
c1: [1 2 3]
c2: [3 4 5]
Then, we'll order the fields alphabetically using orderfields, find the field names using fieldnames, use regexprep to strip all trailing numbers off of the field names, then get the unique substrings using unique:
>> s = orderfields(s);
>> [strs, inds] = unique(regexprep(fieldnames(s), '\d*$', ''), 'last')
strs =
'a'
'b'
'c'
inds =
3
4
6
Using the indices returned by unique, we can calculate how many times each substring appears by doing diff([0; inds]). Then, we group the structure data into a cell array using these counts and struct2cell and mat2cell:
>> data = mat2cell(struct2cell(s), diff([0; inds]))
data =
{3x1 cell}
{1x1 cell}
{2x1 cell}
Notice that we have a cell array of cell arrays. If you know for sure that each set of entries for each substring will concatenate properly (as in our example), you can concatenate them using cellfun and cell2mat as follows:
>> data = cellfun(#cell2mat, data, 'UniformOutput', false)
data =
[3x5 double]
[]
[2x3 double]
Now we have a cell array of matrices, and a new structure can be made using cell2struct:
>> snew = cell2struct(data, strs)
snew =
a: [3x5 double]
b: []
c: [2x3 double]
I have a structure of arrays StockInfo in Matlab. The fields of the structure StockInfo are as follows:
StockInfo =
Name: {10x1 cell}
Values: [10x6 double]
Return: [10x1 double]
I need to sort StockInfo based on the field Return, so that each array in the struct is sorted accordingly. Any idea how to do it?
As I mentioned in the comment above, you question is unclear. I think you are confusing structures and structure arrays. This post might be of help.
That said, here is an example to show what I think you meant to do.
First I create a structure array with some random data:
% cell array of 10 names
names = arrayfun(#(k) randsample(['A':'Z' 'a':'z' '0':'9'],k), ...
randi([5 10],[10 1]), 'UniformOutput',false);
% 10x6 matrix of values
values = rand(10,6);
% 10x1 vector of values
returns = randn(10,1);
% 10x1 structure array
StockInfo = struct('Name',names, 'Values',num2cell(values,2), ...
'Return',num2cell(returns));
The created variable is a an array of structures:
>> StockInfo
StockInfo =
10x1 struct array with fields:
Name
Values
Return
where each element is a structure with the following fields:
>> StockInfo(1)
ans =
Name: 'Pr3N4LTEi'
Values: [0.7342 0.1806 0.7458 0.8044 0.6838 0.1069]
Return: -0.3818
Next can sort this struct array by the "return" field (each struct has a corresponding scalar value):
[~,ord] = sort([StockInfo.Return]);
StockInfo = StockInfo(ord);
The result is that the array is now sorted by the "return" values in ascending order:
>> [StockInfo.Return]
ans =
Columns 1 through 8
-0.3818 0.4289 -0.2991 -0.8999 0.6347 0.0675 -0.1871 0.2917
Columns 9 through 10
0.9877 0.3929
You can sort structure arrays based on fields with the FileExchange function nestedSortStruct (link).
B = nestedSortStruct(A, 'Return');
A solution with built-in functions only could be:
[~, ix] = sort(StockInfo.Return);
StockInfo = struct(...
'Name', {StockInfo.Name{ix}}, ...
'Values', StockInfo.Values(ix), ...
'Return', StockInfo.Return(ix));
Replace ~ with any unused identifier if your Matlab is older and does not support unused output arguments.