I have a series of 11 large matrices corresponding to earthquake data. I want to draw data from individual matrices in a loop. I will use dummy matrices data1 and data2 to illustrate my problem.
load('data1');
load('data2'); %data1 and data2 are large matrices
matrixname={data1 data2};
for j=1:2
matrix=matrixname(j);
latitude=matrix(:,1);
longitude=matrix(:,2);
geoshow(latitude,longitude);
end
So in my loop I want to extract columns from different matrices depending on the index. But I cannot figure out how to do this - I get error messages saying that the index exceeds the matrix dimensions.
Appreciate the help!
As mentioned, use {} brackets for cell arrays. Also, if you want to store the latitude and longitude into new matrices then add the 'j' term to your matrices so that you don't overwrite the data in your for loop.
Also when writing for loops, it's better to use size/length/numel instead of a fixed variable in case the number of files your analysing changes.
So taking your code;
for j=1:size(matrixname,2)
matrix = []; % reformat matrix in case of size differences
matrix = matrixname{j}; % gets the jth matrix
latitude(j) = matrix(:,1); % outputs column 1
longitude(j) = matrix(:,2); % outputs column 2
geoshow(latitude(j),longitude(j)); % runs function on jth set of data
end
Remember, [] brackets are for matrices. Using the wrong brackets is what gave you the error, as pointed out above.
To get the matrix you need from a cell array (which is what {data1 data2} is), you need to use cell2mat():
matrix=cell2mat(matrixname(j));
Since data1 and data2 are matrices, when you do:
matrixname={data1 data2};
You don't get an array of doubles as you're expecting, but a cell array. That's ok and good since data1 and data2 doesn't have the same size (number of rows or columns).
Then, inside the for loop, when accessing one of the original matrices (data1 or data2) from matrixname, you should convert it back to an array of doubles. The simplest and fastest way of doing it is by:
matrix = matrixname{j};
Look at the difference from your code: I'm using curly brackets {} instead of parenthesis (). Doing that way, matrix is a array of doubles. Doing matrix=matrixname(j), matrix is a cell array.
Finally, it's always good to clear temporary variables inside a loop. Then, your code should like:
load('data1');
load('data2'); %data1 and data2 are large matrices
matrixname = {data1 data2};
for j=1:2
matrix=matrixname{j};
latitude=matrix(:,1);
longitude=matrix(:,2);
geoshow(latitude,longitude);
clear matrix latitude longitude
end
For example, if:
data1 = [1 1; 2 2; 3 3];
data2 = [10 10; 20 20; 30 30; 40 40];
matrixname = {data1 data2};
matrixname{1} gives you exactly data1 and matrixname{2} gives you data2.
Related
I want to generate a 3D cell array called timeData so that timeData(:,:,a) for some integer a is an nx1 matrix of data, and the number of rows n varies with the value of a in a 1:1 correspondence. To do this, I am generating a 2D array of data called data that is nx1. This assignment statement takes place within a for loop as follows:
% Before iterating, I define an array of indices where I want to store the
% data sets in timeData. This choice of storage location is for
% organizational purposes.
A = [2, 5, 9, 21, 34, 100]; % Notice they are in ascending order, but have
% gaps that have no predictability.
sizeA = size(A);
numIter = A(1);
for m = 1:numIter % numIter is the number of data sets that I need to store
% in timeData
% At this point, some code that is entirely irrelevant to my question
% generates a nx1 array of data. One example of this data array is below.
data = [1.1;2.3;5.5;4.4]; % This is one example of what data could be. Its
% number of rows, n, changes each iteration, as
% do its contents.
B = size(data);
timeData(1:B(1),1,A(m)) = num2cell(data);
end
This code does put all contents of data in the appropriate locations within timeData as I want. However, it also adds {0x0 double} rows to all 2D arrays of timeData(:,:,a) for any a whose corresponding number of rows n was not the largest number of rows. Thus, there are many of these 2D arrays that have 10 to a couple hundred 0-valued rows that I don't want. For values of a that did not have a corresponding data set, the content of timeData(:,:,a) is an nx1 array of {0x0 double}.
I need to iterate over the contents of timeData in subsequent code, and I need to be able to find the size of the data set that is in timeData(:,:,a) without somehow discounting all the {0x0 double}.
How can I modify my assignment statement to fix this?
Edit: Desired output of the above example is the following with n = 5. Let this data set be represented by a = 9.
timeData(:,:,9) = {[1.1]}
{[2.3]}
{[5.5]}
{[8.6]}
{[4.4]}
Now, consider the possibility that a previous or subsequent value of the A matrix had a data set with n = 7, and n = 7 is the largest data set (largest n value). timeData(:,:,9) outputs like so in my code:
timeData(:,:,9) = {[1.1]}
{[2.3]}
{[5.5]}
{[8.6]}
{[4.4]}
{[0x0 double]}
{[0x0 double]}
#Dev-iL, as I understand it, your answer gives me the ability to delete the cells that have {[0x0 double]} in them (this is what I mean by "discounting"). This is a good plan B, but is there a way to prevent the {[0x0 double]} cells from showing up in the first place?
Edit 2: Update to the above statement "your answer gives me the ability to delete the cells that have {[0x0 double]} in them (this is what I mean by "discounting")". The cellfun(#isempty... ) function makes the {[0x0 double]}cells go to {[0x0 cell]}, it does not remove them. In other words, size(timeData(:,:,9)) is the same before and after the command is performed. This is not what I want. I want size(timeData(:,:,9)) to be 5x1 no matter what n is for any other value of a.
Edit 3: I just realized that the most desired output would be the following:
timeData(:,:,9) = {[1.1;2.3;5.5;8.6;4.4]} % An n x 1 column matrix within
% the cell.
but I can work with this outcome or the outcome as described above.
Unfortunately, I don't understand the structure of your dataset, which is why I can't suggest a better assignment method. However, I'd like to point out an operation that can you help deal with your data after it's been created:
cellfun(#isempty,timeData);
What the above does is return a logical array the size of timeData, indicating which cells contain something "empty". Typically, an array of arbitrary datatype is considered "empty" when it has at least one dimension that is equal to 0.
How can you use it to your advantage?
%% Example 1: counting non-empty cells:
nData = sum(~cellfun(#isempty,timeData(:)));
%% Example 2: assigning empty cells in place of empty double arrays:
timeData(cellfun(#isempty,timeData)) = {{}};
I have a dataset of points represented by a 2D vector (X).
Each point belongs to a categorical data (Y) represented by an integer value(from 1 to 4).
I want to plot each point with a different symbol depending on its class.
Toy example:
X = randi(100,10,2); % 10 points ranging 1:100 in 2D space
Y = randi(4,10,1); % class of the points (1 to 4)
I create a vector of symbols for each class:
S = {'bx' 'rx' 'b.' 'r.'};
Then I try:
plot(X(:,1), X(:,2), S(Y))
Error using plot
Invalid first data argument
How can I assign to each point of X a different symbol based on the value of Y?
Of curse I can use a loop for each class and plot the different classes one by one. But is there a method to directly plot each class with a different symbol?
No need for a loop, use gscatter:
X = randi(100,10,2); % 10 points ranging 1:100 in 2D space
Y = randi(4,10,1); % class of the points (1 to 4)
color = 'brbr';
symbol = 'xx..';
gscatter(X(:,1),X(:,2),Y,color,symbol)
and you will get:
If X has many rows, but there are only a few S types, then I suggest you check out the second approach first. It's optimized for speed instead of readability. It's about twice as fast if the vector has 10 elements, and more than 200 times as fast if the vector has 1000 elements.
First approach (easy to read):
Regardless of approach, I think you need a loop for this:
hold on
arrayfun(#(n) plot(X(n,1), X(n,2), S{Y(n)}), 1:size(X,1))
Or, to write the loop in the "conventional way":
hold on
for n = 1:size(X,1)
plot(X(n,1), X(n,2), S{Y(n)})
end
Second approach (gives same plot as above):
If your dataset is large, you can sort [Y_sorted, sort_idx] = sort(Y), then use sort_idx to index X, like this: X_sorted = X(sort_idx);. After this, you split X_sorted into 4 groups, one for each of the individual Y-values, using histc and mat2cell. Then you loop over the four groups and plot each one individually.
This way you only need to loop through four values, regardless of the number of elements in your data. This should be a lot faster if the number of elements is high.
[Y_sorted, Y_index] = sort(Y);
X_sorted = X(Y_index, :);
X_cell = mat2cell(X_sorted, histc(Y,1:numel(S)));
hold on
for ii = 1:numel(X_cell)
plot(X_cell{ii}(:,1),X_cell{ii}(:,2),S{ii})
end
Benchmarking:
I did a very simple benchmarking of the two approaches using timeit. The result shows that the second approach is a lot faster:
For 10 elements:
First approach: 0.0086
Second approach: 0.0037
For 1000 elements:
First approach = 0.8409
Second approach = 0.0039
Assuming i have a series of column-vectors with different length, what would be the best way, in terms of computation time, to join all of them into one matrix where the size of it is determined by the longest column and the elongated columns cells are all filled with NaN's.
Edit: Please note that I am trying to avoid cell arrays, since they are expensive in terms of memory and run time.
For example:
A = [1;2;3;4];
B = [5;6];
C = magicFunction(A,B);
Result:
C =
1 5
2 6
3 NaN
4 NaN
The following code avoids use of cell arrays except for the estimation of number of elements in each vector and this keeps the code a bit cleaner. The price for using cell arrays for that tiny bit of work shouldn't be too expensive. Also, varargin gets you the inputs as a cell array anyway. Now, you can avoid cell arrays there too, but it would most probably involve use of for-loops and might have to use variable names for each of the inputs, which isn't too elegant when creating a function with unknown number of inputs. Otherwise, the code uses numeric arrays, logical indexing and my favourite bsxfun, which must be cheap in the market of runtimes.
Function Code
function out = magicFunction(varargin)
lens = cellfun(#(x) numel(x),varargin);
out = NaN(max(lens),numel(lens));
out(bsxfun(#le,[1:max(lens)]',lens)) = vertcat(varargin{:}); %//'
return;
Example
Script -
A1 = [9;2;7;8];
A2 = [1;5];
A3 = [2;6;3];
out = magicFunction(A1,A2,A3)
Output -
out =
9 1 2
2 5 6
7 NaN 3
8 NaN NaN
Benchmarking
As part of the benchmarking, we are comparing our solution to #gnovice's solution that was mostly based on using cell arrays. Our intention here to see that after avoiding cell arrays, what speedups we are getting if there's any. Here's the benchmarking code with 20 vectors -
%// Let's create row vectors A1,A2,A3.. to be used with #gnovice's solution
num_vectors = 20;
max_vector_length = 1500000;
vector_lengths = randi(max_vector_length,num_vectors,1);
vs =arrayfun(#(x) randi(9,1,vector_lengths(x)),1:numel(vector_lengths),'uni',0);
[A1,A2,A3,A4,A5,A6,A7,A8,A9,A10,A11,A12,A13,A14,A15,A16,A17,A18,A19,A20] = vs{:};
%// Maximally cell-array based approach used in linked #gnovice's solution
disp('--------------------- With #gnovice''s approach')
tic
tcell = {A1,A2,A3,A4,A5,A6,A7,A8,A9,A10,A11,A12,A13,A14,A15,A16,A17,A18,A19,A20};
maxSize = max(cellfun(#numel,tcell)); %# Get the maximum vector size
fcn = #(x) [x nan(1,maxSize-numel(x))]; %# Create an anonymous function
rmat = cellfun(fcn,tcell,'UniformOutput',false); %# Pad each cell with NaNs
rmat = vertcat(rmat{:});
toc, clear tcell maxSize fcn rmat
%// Transpose each of the input vectors to get column vectors as needed
%// for our problem
vs = cellfun(#(x) x',vs,'uni',0); %//'
[A1,A2,A3,A4,A5,A6,A7,A8,A9,A10,A11,A12,A13,A14,A15,A16,A17,A18,A19,A20] = vs{:};
%// Our solution
disp('--------------------- With our new approach')
tic
out = magicFunction(A1,A2,A3,A4,A5,A6,A7,A8,A9,A10,...
A11,A12,A13,A14,A15,A16,A17,A18,A19,A20);
toc
Results -
--------------------- With #gnovice's approach
Elapsed time is 1.511669 seconds.
--------------------- With our new approach
Elapsed time is 0.671604 seconds.
Conclusions -
With 20 vectors and with a maximum length of 1500000, the speedups are between 2-3x and it was seen that the speedups have increased as we have increased the number of vectors. The results to prove that are not shown here to save space, as we have already used quite a lot of it here.
If you use a cell matrix you won't need them to be filled with NaNs, just write each array into one column and the unused elements stay empty (that would be the space efficient way). You could either use:
cell_result{1} = A;
cell_result{2} = B;
THis would result in a size 2 cell array which contains all elements of A,B in his elements. Or if you want them to be saved as columns:
cell_result(1,1:numel(A)) = num2cell(A);
cell_result(2,1:numel(B)) = num2cell(B);
If you need them to be filled with NaN's for future coding, it would be the easiest to find the maximum length you got. Create yourself a matrix of (max_length X Number of arrays).
So lets say you have n=5 arrays:A,B,C,D and E.
h=zeros(1,n);
h(1)=numel(A);
h(2)=numel(B);
h(3)=numel(C);
h(4)=numel(D);
h(5)=numel(E);
max_No_Entries=max(h);
result= zeros(max_No_Entries,n);
result(:,:)=NaN;
result(1:numel(A),1)=A;
result(1:numel(B),2)=B;
result(1:numel(C),3)=C;
result(1:numel(D),4)=D;
result(1:numel(E),5)=E;
I have a set of data that is <106x25 double> but this is inside a struct and I want to extract the data into a matrix. I figured a simple FOR loop would accomplish this but I have hit a road block quite quickly in my MATLAB knowledge.
This is the only piece of code I have, but I just don't know enough about MATLAB to get this simple bit of code working:
>> x = zeros(106,25); for i = 1:106, x(i,:) = [s(i).surveydata]; end
??? Subscripted assignment dimension mismatch.
's' is a very very large file (in excess of 800MB), it is a <1 x 106 struct>. Suffice it to say, I just need to access a small portion of this which is s.surveydata where most rows are a <1 x 25 double> (a row vector IIRC) and some of them are empty and solely return a [].
s.surveydata obviously returns the results for all of the surveydata contained where s(106).surveydata would return the result for the last row. I therefore need to grab s(1:106).surveydata and put it into a matrix x. Is creating the matrix first by using x = zeros(106,25) incorrect in this situation?
Cheers and thanks for your time!
Ryan
The easiest, cleanest, and fastest way to write all the survey data into an array is to directly catenate it, using CAT:
x = cat(1,s.surveydata);
EDIT: note that if any surveydata is empty, x will have fewer rows than s has elements. If you need x to have the same amount of rows as s has elements, you can do the following:
%# find which entries in s have data
%# note that for the x above, hasData(k) contains the
%# element number in s that the k-th row of x came from
hasData = find(arrayfun(#(x)~isempty(x.surveydata),s));
%# initialize x to NaN, so as to not confuse the
%# real data with missing data entries. The call
%# to hasData when indexing makes this robust to an
%# empty first entry in s
x = NaN(length(s),length(s(hasData(1)).surveydata);
%# fill in only the rows of x that contain data
x(hasData,:) = cat(1,s(hasData).surveydata);
No, creating an array of zeroes is not incorrect. In fact it's a good idea. You don't have to declare variables in Matlab before using them, but for loops, pre-allocating has speed benefits.
x = zeros(size(s), size(s(1)));
for i = 1:106
if ~isempty(s(i).surveydata)
x(i, :) = s(i).surveydata;
end
end
Should accomplish what you want.
EDIT: Since OP indicated that some rows are empty, I accounted for that like he said.
what about this?
what s is?
if s(i).surveydata is scalar:
x = zeros(106,25);
for i = 1:106
x(i,1) = [s(i).surveydata];
end
I am guessing that is what you want tough it is not clear at all :
if s(i).surveydata is row vector:
x = zeros(106,25);
for i = 1:106
x(i,:) = [s(i).surveydata];
end
if s(i).surveydata is column vector:
x = zeros(106,25);
for i = 1:106
x(i,:) = [s(i).surveydata]';
end
Hi I have problem with matrix..
I have many .txt files with different number of rows but have the same number of column (1 column)
e.g. s1.txt = 1234 rows
s2.txt = 1200 rows
s2.txt = 1100 rows
I wanted to combine the three files. Since its have different rows .. when I write it to a new file I got this error = Index exceeds matrix dimensions.
How I can solved this problem? .
You can combine three matrices simply by stacking them: Assuming that s1, etc are the matrices you read in, you can make a new one like this:
snew = [s1; s2; s3];
You could also use the [] style stacking without creating the new matrix variable if you only need to do it once.
You have provided far too little information for an accurate diagnosis of your problem. Perhaps you have loaded the data from your files into variables in your workspace. Perhaps s1 has 1 column and 1234 rows, etc. Then you can concatenate the variables into one column vector like this:
totalVector = [s1; s2; s3];
and write it out to a file with a save() statement.
Does that help ?
Let me make an assumption that this question is connecting with your another question, and you want to combine those matrices by columns, leaving empty values in columns with fewer data.
In this case this code should work:
BaseFile ='s';
n=3;
A = cell(1,n);
for k=1:n
A{k} = dlmread([BaseFile num2str(k) '.txt']);
end
% create cell array with maximum number of rows and n number of columns
B = cell(max(cellfun(#numel,A)),n);
% convert each matrix in A to cell array and store in B
for k=1:n
B(1:numel(A{k}),k) = num2cell(A{k});
end
% save the data
xlswrite('output.txt',B)
The code assumes you have one column in each file, otherwise it will not work.