I am trying to find and remove correlated columns from data with the function below
function [ correlated ] = correlated( data, threshold )
% data is m x m matrix
% threshold is correlation threshold
c=corr(data);
[i,j] = find(c>threshold);
A = [i,j];
A=A(find(arrayfun(#(i)length(unique(A(i,:))),1:size(A,1))==size(A,2)),:);
% as A also includes 1 1; 2 2; 3 3; etc so I used above line that I found somewhere
% 6 4
% 8 4
% 4 6
% 8 6
% 4 8
% 6 8
% 14 11
% 11 14
% it should not contain both 6 4; and 4 6; how do I remove all such rows?
end
it should not contain both 6 4; and 4 6; how do I remove all such rows?
uniqueA = unique(sort(A,2), 'rows');
Related
I am trying to match RGB values in an image.
% R G B
RGBset = [ 3 9 12;
4 8 13;
11 13 13;
8 3 2]
img(:,:,1) = [1 2 3
6 5 4
7 9 8
10 11 12];
img(:,:,2) = [3 4 8;
6 7 8;
11 10 9;
12 13 14];
img(:,:,3)= [3 7 2;
4 9 10;
5 11 12;
6 13 14]
In this image, only one RGB value matches from the RGBset which is [11,13,13], so the expected output is:
[0 0 0;
0 0 0;
0 0 0;
0 1 0]; % reshape(img(4,2,:),1,3) = [11, 13 13] is available in RGBset
% no other RGB value is present in the image
I have made this code but it is very slow for larger images.
matched= zeros(img(:,:,1));
for r=1:size(img(:,:,1),1)
for c=1:size(img(:,:,2),2)
matched(r,c)=ismember(reshape(img(r,c,:),1,3),RGBset,'rows');
end
end
What will be the solution with more speed?
We can reduce each RGB triplet into a scalar each and we will do this for both RGBset and img. This would reduce them to 2D and 1D matrices respectively. We are calling this as dimensionality-reduction. With this reduced data, we are achieving memory efficiency and that hopefully should lead to performance boost.
Thus, a solution with those bases covered would look something like this -
% Scaling array for dim reduction
s = [256^2, 256, 1].';
% Reduce dims for RGBset and img
RGBset1D = RGBset*s;
img1D = reshape(img,[],3)*s;
% Finally use find membership and reshape to 2D
out = reshape(ismember(img1D, RGBset1D), size(img,1), []);
Benchmarking for the vectorized solutions
Benchmarking code -
% R G B
RGBset = [ 3 9 12;
4 8 13;
11 13 13;
8 3 2]
% Setup inputs
img = randi(255, 2000, 2000, 3);
img(3,2,:) = RGBset(4,:);
% Luis's soln
disp('--------------------- Reshape + Permute ------------------')
tic
img2 = reshape(permute(img, [3 1 2]), 3, []).';
matched = ismember(img2, RGBset, 'rows');
matched = reshape(matched, size(img,1), []);
toc
% Proposed in this post
disp('--------------------- Dim reduction ------------------')
tic
s = [256^2, 256, 1].';
RGBset1D = RGBset*s;
img1D = reshape(img,[],3)*s;
out = reshape(ismember(img1D, RGBset1D), size(img,1), []);
toc
Benchmarking output -
--------------------- Reshape + Permute ------------------
Elapsed time is 3.101870 seconds.
--------------------- Dim reduction ------------------
Elapsed time is 0.031589 seconds.
You can replace the loops by permute and reshape:
img2 = reshape(permute(img, [3 1 2]), 3, []).';
matched = ismember(img2, RGBset, 'rows');
matched = reshape(matched, size(img,1), []);
This creates a 3-column matrix img2 in which each row corresponds to one pixel from img. That way ismember(..., 'rows') can be applied in a vectorized manner. The obtained result is then reshaped as needed.
You can just loop over the colours, which is much quicker than looping over each pixel.
% Set up your colours into the 3rd dimension, so they match along the same axis
RGB3D = reshape(RGBset,[],1,3);
% Loop over them
for ii = 1:size(RGB3D, 1)
% See if all 3 members of the colour match any pixel
matched = all(ismember(img, RGB3D(ii,:,:)),3)
if any(matched)
disp(matched)
disp(['matched color: ' num2str(ii)]);
% do something else with the matched pixels
end
end
I would like to extract (square) sub-matrices from a matrix, randomly with a size between some min and max values.
I would also like to retain the row and column indices which were selected.
Are there any built-in functions for this purpose? I would appreciate any general algorithm to achieve the desired results.
Just for an example without considering the square matrix constraint:
Input:
| 1 2 3 4 5 6 7 8 <- column indices
--+----------------
1 | 4 3 1 4 0 1 0 1
2 | 2 0 1 5 6 3 2 0
3 | 5 5 0 6 7 8 9 0
4 | 2 3 5 6 7 9 0 1
^
Row indices
Output sample:
| 1 4 5 7 8 <- column indices
--+----------
1 | 4 4 0 0 1
2 | 2 5 6 2 0
4 | 2 6 7 0 1
^
Row indices
Edit: I would like to have a function like this
matrixsample(numberOfSamples, minSize, maxSize, satisfyingFunction)
the samples should vary their size between the minSize and maxSize.
If numberOfSamples = 10, minSize = 2 and maxSize = 6 then output should be randomly selected rows and columns like:
sampleMatrix1 2x2 sampleMatrix2 3x3 sampleMatrix3 5x5 ... sampleMatrix7 6x6 ... sampleMatrx10 4x4
satisfyingFunction could test any attribute of the matrix; like being non-singular, rank > x, etc.
In MATLAB, you will find the randperm function useful for selecting random rows/columns.
To get a randomly sized sub-matrix, use randi([minVal, maxVal]) to get a random integer between minVal and maxVal.
Getting a submatrix from random (ordered) combination of rows and columns:
M = randi([0,9],4,8); % 4x8 matrix of random 1 digit integers, your matrix here!
nRows = randi([2, 4]); % The number of rows to extract, random between 2 and 4
nCols = randi([5, 7]); % The number of columns to extract, random between 5 and 7
idxRows = sort(randperm(size(M,1), nRows)); % Random indices of rows
idxCols = sort(randperm(size(M,2), nCols)); % Random indices of columns
output = M(idxRows, idxCols); % Select the sub-matrix
If you want to make the sub-matrix square then simply use the same value for nRows and nCols.
Showing this method works with your example input/output values:
M = [4 3 1 4 0 1 0 1; 2 0 1 5 6 3 2 0; 5 5 0 6 7 8 9 0; 2 3 5 6 7 9 0 1];
idxRows = [1 2 4]; idxCols = [1 4 5 7 8];
output = M(idxRows, idxCols)
% >> 4 4 0 0 1
% 2 5 6 2 0
% 2 6 7 0 1
Edit in response to extended question scope:
You can package the above up into a short function, which returns the row and column indices, as well as the submatrix.
function output = getsubmatrix(M, nRows, nCols)
% Get a submatrix of random rows/columns
idxRows = sort(randperm(size(M,1), nRows));
idxCols = sort(randperm(size(M,2), nCols));
submatrix = M(idxRows, idxCols);
output = {submatrix, idxRows, idxCols};
end
Then you can use this in some sampling function as you described:
function samples = matrixsample(matrix, numberOfSamples, minSize, maxSize, satisfyingFunction)
% output SAMPLES which contains all sample matrices, with row & column indices w.r.t MATRIX
samples = cell(numberOfSamples,1);
% maximum iterations trying to satisfy SATISFYINGCONDITION
maxiters = 100;
for ii = 1:numberOfSamples
iters = 0; submatrixfound = false; % reset loop exiting conditions
nRows = randi([minSize, maxSize]); % get random submatrix size
nCols = nRows; % Square matrix
while iters < maxiters && submatrixfound == false
% Get random submatrix, and the indices
submatrix = getsubmatrix(matrix, nRows,nCols);
% satisfyingFunction MUST RETURN BOOLEAN
if satisfyingFunction(submatrix{1})
samples{ii} = submatrix; % If satisfied, assign to output
submatrixfound = true; % ... and move on!
end
iters = iters + 1;
end
end
end
Test example:
s = matrixsample(magic(10), 5, 2, 8, #(M)max(M(:)) < 90)
If the table is given, you will do it by randsample:
minRowVal = 3;
maxRowVal = 4;
minColVal = 2;
maxColVal = 4;
kRow = randi([minRowVal maxRowVal]) ;
kCol = randi([minColVal maxColVal]);
table(sort(randsample(size(table,1),kRow)),sort(randsample(size(table,2),kCol))
choose some sample by the given size of kRow and kCol, then select the information from the table.
I want to replace each element by the minimum of its row, other than the element itself.
Example: input In = [1 2 3; 4 5 6; 7 8 9], output out = [2 1 1; 5 4 4; 8 7 7]
EDIT: without for loop, unless computationally more efficient
You can use a new function movmin introduced in MATLAB R2016a to solve this with a moving minimum:
In = [1 2 3; 4 5 6; 7 8 9]; % Sample data
C = size(In, 2); % Get the number of columns
out = movmin(In(:, [2:C 1:(C-1)]), [0 C-2], 2, 'Endpoints', 'discard')
out =
2 1 1
5 4 4
8 7 7
The above works by first indexing the columns of In to create wrap-around copies of the matrix, then slides a window of size C-1 along each row, computing the minimum. The 'Endpoints', 'discard' option discards results where the window extends past the edges of the matrix.
I did it with two calls to min. you can do similarly with sort(In,2):
% input matrix
In = [1 2 3; 4 5 6; 7 8 9];
% compute minimum for each row
[val,mincols] = min(In,[],2);
% generate matrix made of minimum value of each row
Out = repmat(val,[1 size(In,2)]);
% find indexes of minimum values
minrows = 1:size(In,1);
minidxs = sub2ind(size(In),minrows,mincols');
% replace minimum values with infs
In(minidxs) = inf;
% find next minimum values
val = min(In,[],2);
% set original minimum elements to next minimum values
Out(minidxs) = val
Consider the following example
A =
6 9 4 7 10
3 6 5 5 9
10 4 9 8 6
10 6 3 4 6
6 3 3 8 6
6 4 4 4 5
5 10 8 5 7
10 10 8 8 7
5 7 8 9 9
3 3 6 3 9
[~,Inx] =max(A, [],2)
Inx =
5
5
1
1
4
1
2
1
4
5
The above code returns index of the maximum number along each column like in first row the max number is 10 in 5th row so Inx(1) = 5
Can we do the same thing for find ? like for example if I want to find a specific number in each row lets say 8
>> find(A == 8)
ans =
27
28
29
33
35
38
I will get the indices but not row wise like we get for the max() is there some way to do minipulate find to get that ? or else some other way
Update : I know we can use [row,col,v] = find(___) but there is one problem with that it only returns for the rows where the value is present
You can simply convert A to a logical matrix where it's 1 if equal to 8 and 0 otherwise. Then you can use the second output of max to find the column which contains the first 8 in each row. If a row doesn't contain any 8's the first output of max will be a 0 and the second output will be 1.
You can multiply the first and second outputs to zero-out these rows that didn't have an 8.
[has8, col] = max(A == 8, [], 2);
% Zero out any rows that didn't contain an 8
result = has8 .* col;
% 0
% 0
% 4
% 0
% 4
% 0
% 3
% 3
% 3
% 0
If you'd rather have NaN's than 0's, you could do the following which exploits the fact that 0/0 == NaN
result = has8 .* col ./ has8;
% NaN
% NaN
% 4
% NaN
% 4
% NaN
% 3
% 3
% 3
% NaN
Inx = zeros(size(A,1),1); % initialize with default values
[row,col,v] = find(A==8); % find the proper indices
Inx(row) = col; % insert values
I have a m-by-n matrix and I want to shift each row elements k no. of times (" one resultant matrix for each one shift so a total of k matrices corresponding to each row shifts ")(k can be different for different rows and 0<=k<=n) and want to index all the resultant matrices corresponding to each individual shift.
Eg: I have the matrix: [1 2 3 4; 5 6 7 8; 2 3 4 5]. Now, say, I want to shift row1 by 2 times (i.e. k=2 for row1) and row2 by 3times (i.e. k=3 for row2) and want to index all the shifted versions of matrices (It is similar to combinatorics of rows but with limited and diffeent no. of shifts to each row).
Can someone help to write up the code? (please help to write the general code but not for the example I mentioned here)
I found the following question useful to some extent, but it won't solve my problem as my problem looks like a special case of this problem:
Matlab: How to get all the possible different matrices by shifting it's rows (Update: each row has a different step)
See if this works for you -
%// Input m-by-n matrix
A = rand(2,5) %// Edit this to your data
%// Initialize shifts, k for each row. The number of elements would be m.
sr = [2 3]; %// Edit this to your data
[m,n] = size(A); %// Get size
%// Get all the shits in one go
sr_ind = arrayfun(#(x) 0:x,sr,'un',0); %//'
shifts = allcomb(sr_ind{:},'matlab')'; %//'
for k1 = 1:size(shifts,2)
%// Get shift to be used for each row for each iteration
shift1 = shifts(:,k1);
%// Get circularly shifted column indices
t2 = mod(bsxfun(#minus,1:n,shift1),n);
t2(t2==0) = n;
%// Get the linear indices and use them to index into input to get the output
ind = bsxfun(#plus,[1:m]',(t2-1)*m); %//'
all_matrices = A(ind) %// outputs
end
Please note that this code uses MATLAB file-exchange code allcomb.
If your problem in reality is not more complex than what you showed us, it can be done by a double loop. However, i don't like my solution, because you would need another nested loop for each row you want to shift. Also it generates all shift-combinations from your given k-numbers, so it has alot of overhead. But this can be a start:
% input
m = [1 2 3 4; 5 6 7 8; 2 3 4 5];
shift_times = {0:2, 0:3}; % 2 times for row 1, 3 times for row 2
% desird results
desired_matrices{1} = [4 1 2 3; 5 6 7 8; 2 3 4 5];
desired_matrices{2} = [3 4 1 2; 5 6 7 8; 2 3 4 5];
desired_matrices{3} = [1 2 3 4; 8 5 6 7; 2 3 4 5];
desired_matrices{4} = [4 1 2 3; 8 5 6 7; 2 3 4 5];
desired_matrices{5} = [3 4 1 2; 8 5 6 7; 2 3 4 5];
% info needed:
[rows, cols] = size(m);
count = 0;
% make all shift combinations
for shift1 = shift_times{1}
% shift row 1
m_shifted = m;
idx_shifted = [circshift([1:cols]',shift1)]';
m_shifted(1, :) = m_shifted(1, idx_shifted);
for shift2 = shift_times{2}
% shift row 2
idx_shifted = [circshift([1:cols]',shift2)]';
m_shifted(2, :) = m_shifted(r_s, idx_shifted);
% store them
store{shift1+1, shift2+1} = m_shifted;
end
end
% store{i+1, j+1} stores row 1 shifted by i and row 2 shifted by j
% example
all(all(store{2,1} == desired_matrices{1})) % row1: 1, row2: 0
all(all(store{2,2} == desired_matrices{4})) % row1: 1, row2: 1
all(all(store{3,2} == desired_matrices{5})) % row1: 2, row2: 1