I have a couple of matrices (1800 x 27) that represent subjects and their recordings (3 minutes equivalent for each of 27 subjects). Each column represents a subject.
I need to do intercorrelation between subjects, let's say to correlate F to G, G to H, and H to F for all 27 subjects.
I use CORR command corr(B) where B is a matrix and it returns the next example:
1 0.07 -0.05 0.10 0.04 0.12
0.07 1 -0.02 -0.08 0.17 0.03
-0.05 -0.02 1 0.04 0.16 0.13
0.10 -0.08 0.04 1 -0.04 0.34
0.04 0.18 0.16 -0.04 1 0.13
How can I adjust the code to exclude self-correlation (eg F to F) so I won't get "1" numerals?
(it's present in each row/column)
I have to perform some transformations afterwards, like Fisher Z-Transformation, which returns inf for each "1", and as result, I can't use further calculations.
Related
I am trying to optimise the running time of my code by getting rid of some for loops. However, I have a variable that is incremented in each iteration in which sometimes the index is repeated. I provide here a minimal example:
a = [1 4 2 2 1 3 4 2 3 1]
b = [0.5 0.2 0.3 0.4 0.1 0.05 0.7 0.3 0.55 0.8]
c = [3 5 7 9]
for i = 1:10
c(a(i)) = c(a(i)) + b(i)
end
Ideally, I would like to compute it by writting:
c(a) = c(a) + b
but obviously it would not give me the same results since I have to recalculate the value for the same index several times so this way to vectorise it would not work.
Also, I am working in Matlab or Octave in case that this is important.
Thank you very much for any help, I am not sure that it is possible to be vectorise.
Edit: thank you very much for your answers so far. I have discovered accumarray, which I did not know before and also understood why changing the for loop between Matlab and Octave was giving me such different times. I also understood my problem better. I gave a too simple example which I thought I could extend, however, what if b was a matrix?
(Let's forget about c at the moment):
a = [1 4 2 2 1 3 4 2 3 1]
b =[0.69 -0.41 -0.13 -0.13 -0.42 -0.14 -0.23 -0.17 0.22 -0.24;
0.34 -0.39 -0.36 0.68 -0.66 -0.19 -0.58 0.78 -0.23 0.25;
-0.68 -0.54 0.76 -0.58 0.24 -0.23 -0.44 0.09 0.69 -0.41;
0.11 -0.14 0.32 0.65 0.26 0.82 0.32 0.29 -0.21 -0.13;
-0.94 -0.15 -0.41 -0.56 0.15 0.09 0.38 0.58 0.72 0.45;
0.22 -0.59 -0.11 -0.17 0.52 0.13 -0.51 0.28 0.15 0.19;
0.18 -0.15 0.38 -0.29 -0.87 0.14 -0.13 0.23 -0.92 -0.21;
0.79 -0.35 0.45 -0.28 -0.13 0.95 -0.45 0.35 -0.25 -0.61;
-0.42 0.76 0.15 0.99 -0.84 -0.03 0.27 0.09 0.57 0.64;
0.59 0.82 -0.39 0.13 -0.15 -0.71 -0.84 -0.43 0.93 -0.74]
I understood now that what I would be doing is rowSum per group, and given that I am using Octave I cannot use "splitapply". I tried to generalise your answers, but accumarray would not work for matrices and also I could not generalise #rahnema1 solution. The desired output would be:
[0.34 0.26 -0.93 -0.56 -0.42 -0.76 -0.69 -0.02 1.87 -0.53;
0.22 -1.03 1.53 -0.21 0.37 1.54 -0.57 0.73 0.23 -1.15;
-0.20 0.17 0.04 0.82 -0.32 0.10 -0.24 0.37 0.72 0.83;
0.52 -0.54 0.02 0.39 -1.53 -0.05 -0.71 1.01 -1.15 0.04]
that is "equivalent" to
[sum(b([1 5 10],:))
sum(b([3 4 8],:))
sum(b([6 9],:))
sum(b([2 7],:))]
Thank you very much, If you think I should include this in another question instead of adding the edit I will do so.
Original question
It can be done with accumarray:
a = [1 4 2 2 1 3 4 2 3 1];
b = [0.5 0.2 0.3 0.4 0.1 0.05 0.7 0.3 0.55 0.8];
c = [3 5 7 9];
c(:) = c(:) + accumarray(a(:), b(:));
This sums the values from b in groups defined by a, and adds that to the original c.
Edited question
If b is a matrix, you can use
full(sparse(repmat(a, 1, size(b,1)), repelem(1:size(b,2), size(b,1)), b))
or
accumarray([repmat(a, 1, size(b,1)).' repelem(1:size(b,2), size(b,1)).'], b(:))
Matrix multiplication and implicit expansion and can be used (Octave):
nc = numel(c);
c += b * (1:nc == a.');
For input of large size it may be more memory efficient to use sparse matrix:
nc = numel(c);
nb = numel(b);
c += b * sparse(1:nb, a, 1, nb, nc);
Edit: When b is a matrix you can extend this solution as:
nc = numel(c);
na = numel(a);
out = sparse(a, 1:na, 1, nc, na) * b;
I am able to plot a trisurf chart, but surf does not work.
What am I doing wrong?
pkg load statistics;
figure (1,'name','Matrix Map');
colormap('hot');
t = dlmread('C:\Map3D.csv');
tx =t(:,1);ty=t(:,2);tz=t(:,3);
tri = delaunay(tx,ty);
handle = surf(tx,ty,tz); #This does NOT work
#handle = trisurf(tri,tx,ty,tz); #This does work
`error: surface: rows (Z) must be the same as length (Y) and columns (Z) must be the same as length
(X)
My data is in a CSV (commas not shown here)
1 2 -0.32
2 2 0.33
3 2 0.39
4 2 0.09
5 2 0.14
1 2.5 -0.19
2 2.5 0.13
3 2.5 0.15
4 2.5 0.24
5 2.5 0.33
1 3 0.06
2 3 0.44
3 3 0.36
4 3 0.45
5 3 0.51
1 3.5 0.72
2 3.5 0.79
3 3.5 0.98
4 3.5 0.47
5 3.5 0.55
1 4 0.61
2 4 0.13
3 4 0.44
4 4 0.47
5 4 0.58
1 4.5 0.85
surf error message is different in Matlab or in Octave.
Error message from Matlab:
Z must be a matrix, not a scalar or vector.
The problem is pretty clear here since you specified Z (for you tz) as a vector.
Error message from Octave:
surface: rows (Z) must be the same as length (Y) and columns (Z) must be the same as length (X)
You are wrong here since on your example, columns (Z) = 1, but length (X) = 26, so here is the mistake.
One of the consequences of that is that with surf you cannot have "holes" or undefined points on your grid. On your case you have a X-grid from 1 to 5 and a Y-grid from 2 to 4.5 but point of coordinate (2, 4.5) is not defined.
#Luis Mendo, Matlab and Octave do allow the prototype surf(matrix_x, matrix_y, matrix_z) but the third argument matrix_z still have to be a matrix (not a scalar or vector). Apparently, a matrix of only one line or column is not considered as a matrix.
To solve the issue, I suggest something like:
tx = 1:5; % tx is a vector of length 5
ty = 2:0.5:4.5; % ty is a vector of length 6
tz = [-0.32 0.33 0.39 0.09 0.14;
-0.19 0.13 0.15 0.24 0.33;
0.06 0.44 0.36 0.45 0.51;
0.72 0.79 0.98 0.47 0.55;
0.61 0.13 0.44 0.47 0.58;
0.85 0. 0. 0. 0.]; % tz is a matrix of size 6*5
surf(tx,ty,tz);
Note that I had to invent some values at the points where your grid was not defined, I put 0. but you can change it with the value you prefer.
I have a huge amount of data in MATLAB (350695x5).
An example is like this:
z = [
1.79 0.16 0.16 21.39 21.50
1.83 0.16 0.16 21.39 22.40
1.92 0.16 0.16 21.39 22.00
2.07 0.16 0.16 21.39 22.00
2.36 0.15 0.15 21.39 21.08
2.96 0.13 0.13 21.39 21.04
3.21 0.13 0.13 21.39 23.00
3.72 0.12 0.12 21.39 24.00
3.87 0.11 0.11 21.39 21.39
4.14 0.10 0.10 21.39 22.00
4.14 0.10 0.10 21.39 21.50
4.16 0.10 0.10 21.39 21.39]
and I need to sort it in the following way:
based on 1 column from 1-2, 2-3, 3-4
and find mean values in the range (0-1, 1-2, 2-3, 3-4) for 2,3,4 columns
the result should look like this:
1 0.16 0.16 21.39 21.97
2 0.15 0.15 21.39 21.49
3 0.12 0.12 21.39 22.68
4 0.10 0.10 21.39 21.63
The problem is that I cannot sort it in a proper way.
The part of the solution can be described by
[ii jj] = ndgrid(z(:,1)+1,1:size(z,2)-1) %should sort first column from 0-1,1-2, 2-3, 3-4
z23 = z(:,2:end)
out = [unique(z(:,1)),accumarray([ii(:),jj(:)],z23(:),[],#mean)], %find mean value
Try this:
idx = floor(z(:, 1));
sub = [idx z(:, 2:5)];
[xx, yy] = ndgrid(idx, 1:size(sub, 2));
out = accumarray([xx(:) yy(:)], sub(:), [], #mean)
out =
1.0000 0.1600 0.1600 21.3900 21.9667
2.0000 0.1467 0.1467 21.3900 21.3733
3.0000 0.1200 0.1200 21.3900 22.7967
4.0000 0.1000 0.1000 21.3900 21.6300
Results don't match exactly with yours. I'm not sure I understand exactly what you wanted, but the code I wrote calculates the average on ranges 1 <= x < 2, 2 <= x < 3, and so on.
Use logical indexing to find the values in z where a certain range applies, e.g.:
i01 = (z >= 0) & (z < 1); % Find logical indices
z01 = z(i01); % Get values from 0 up to 1 (but not including 1)
Then, calculation of the mean is easy: mu_z01 = mean(z01);. Of course, the same method can be applied to the other ranges from 1 to 2, 2 to 3, et cetera.
I have a matrix which is 36 x 2, but I want to seperate the elements to give me 18, 2 x 2 matrices from top to bottom. E.g. if I have a matrix:
1 2
3 4
5 6
7 8
9 10
11 12
13 14
... ...
I want to split it into seperate matrices:
M1 = 1 2
3 4
M2 = 5 6
7 8
M3 = 9 10
11 12
..etc.
maybe the following sample code could useful:
a=rand(36,2);
b=reshape(a,2,2,18)
then with the 3rd index of b you can access your 18 2x2 matrices, eg. b(:,:,2) gives the second 2x2 matrix.
I think that the direct answer to your question is:
sampledata = [...
0.01 0.02 0.03 0.04 0.05 0.06 0.07 0.08 0.09 0.10 0.11 0.12 0.13 0.14 0.15 0.16 0.17 0.18 1.01 1.02 1.03 1.04 1.05 1.06 1.07 1.08 1.09 1.10 1.11 1.12 1.13 1.14 1.15 1.16 1.17 1.18; ...
0.19 0.20 0.21 0.22 0.23 0.24 0.25 0.26 0.27 0.28 0.29 0.30 0.31 0.32 0.33 0.34 0.35 0.36 1.19 1.20 1.21 1.22 1.23 1.24 1.25 1.26 1.27 1.28 1.29 1.30 1.31 1.32 1.33 1.34 1.35 1.36];
for ix = 1:(size(sampledata,2)/2)
assignin('base',['M' sprintf('%02d',ix)], sampledata(:,ix*2+[-1 0]))
end
This creates 18 variables, named 'M01' through 'M18', with pieces of the sampledata matrix.
However, please don't use dynamic variable names like this. It will complicate every other piece of code that it touches. Use a cell array, a 3D array (as suggested by #Johannes_Endres +1 BTW), or structure. Anything that removes the need for you to write something like this later on:
%PLEASE DO NOT USE THIS
%ALSO DO NOT BACK YOURSELF INTO A CORNER WHERE YOU HAVE TO DO IT IN THE FUTURE
varNames = who('M*');
for ix = 1:length(varNames )
str = ['result(' num2str(ix) ') = some_function(' varNames {ix} ');'];
eval(str);
end
I've seen code like this, and it is slow and extremely cumbersome to maintain, not to mention the headache and pain to your internal beauty-meter.
x = reshape(1:36*2,[2 36])'
k = 1
for i = 1:2:35
eval(sprintf('M%d = x(%d:%d,:);',k,i,i+1));
k = k+1;
end
I need to sort out few small matrices from 1 huge raw matrix ...according to sorting 1st column (1st column contain either 1, 2, or 3)...
if 1st column is 1, then randomly 75% of the 1 save in file A1, 25% of the 1 save in file A2.
if 1st column is 2, then randomly 75% of the 2 save in file B1, 25% of the 2 save in file B2.
if 1st column is 3, then randomly 75% of the 3 save in file C1, 25% of the 3 save in file C2.
how am i going to write the code?
Example:
a raw matrix has 15 rows x 6 columns:
7 rows are 1 in 1st column, 5 rows are 2 in 1st column, and 3 rows are 3 in 1st column.
1 -0.05 -0.01 0.03 0.07 0.11
1 -0.4 -0.36 -0.32 -0.28 -0.24
1 0.3 0.34 0.38 0.42 0.46
1 0.75 0.79 0.83 0.87 0.91
1 0.45 0.49 0.53 0.57 0.61
1 0.8 0.84 0.88 0.92 0.96
1 0.05 0.09 0.13 0.17 0.21
2 0.5 0.54 0.58 0.62 0.66
2 0.4 0.44 0.48 0.52 0.56
2 0.9 0.94 0.98 1.02 1.06
2 0.85 0.89 0.93 0.97 1.01
2 0.75 0.79 0.83 0.87 0.91
3 0.36 0.4 0.44 0.48 0.52
3 0.6 0.64 0.68 0.72 0.76
3 0.4 0.44 0.48 0.52 0.56
7 rows got 1 in 1st column, randomly take out 75% of 7 rows (which is 7*0.75=5.25) to be new matrix (5rows x 6 columns), the rest of 25% become another new matrix
5 rows got 2 in 1st column, randomly take out 75% of 5 rows (which is 5*0.75=3.75) to be new matrix (4rows x 6 columns), the rest of 25% become another new matrix
3 rows got 3 in 1st column, randomly take out 75% of 3 rows (which is 3*0.75=2.25) to be new matrix (2rows x 6 columns), the rest of 25% become another new matrix
Result:
A1=
1 -0.4 -0.36 -0.32 -0.28 -0.24
1 0.3 0.34 0.38 0.42 0.46
1 0.75 0.79 0.83 0.87 0.91
1 0.8 0.84 0.88 0.92 0.96
1 -0.05 -0.01 0.03 0.07 0.11
B1=
2 0.9 0.94 0.98 1.02 1.06
2 0.85 0.89 0.93 0.97 1.01
2 0.5 0.54 0.58 0.62 0.66
2 0.75 0.79 0.83 0.87 0.91
C1=
3 0.36 0.4 0.44 0.48 0.52
3 0.4 0.44 0.48 0.52 0.56
here is one possible solution to your problem using the function randperm:
% Create matrices
firstcol=ones(15,1);
firstcol(8:12)=2;
firstcol(13:15)=3;
mat=[firstcol rand(15,5)];
% Sort according to first column
A=mat(mat(:,1)==1,:);
B=mat(mat(:,1)==2,:);
C=mat(mat(:,1)==3,:);
% Randomly rearrange lines
A=A(randperm(size(A,1)),:);
B=B(randperm(size(B,1)),:);
C=C(randperm(size(C,1)),:);
% Select first 75% lines (rounding)
A1=A(1:round(0.75*size(A,1)),:);
A2=A(round(0.75*size(A,1))+1:end,:);
B1=B(1:round(0.75*size(B,1)),:);
B1=B(round(0.75*size(B,1))+1:end,:);
C1=C(1:round(0.75*size(C,1)),:);
C1=C(round(0.75*size(C,1))+1:end,:);
Hope it helps.