I have a matrix (89x42) of 0's and 1's that I'd like to multiply combinations of rows together.
For example, for matrix
input = [1 0 1
0 0 0
1 1 0];
and with 2 combinations, I want an output of
output = [0 0 0; % (row1*row2)
1 0 0; % (row1*row3)
0 0 0] % (row2*row3)
Which rows to multiply is dictated by "n Choose 2" (nCk), or all possible combinations of the rows n taken k at a time. In this case k=2.
Currently I am using a loop and it works fine for the 89C2 combinations of rows, but when I run it with 89C3 it takes far too long too run.
What would be the most efficient way to do this program so I can do more than 2 combinations?
You can do it using nchoosek and element-wise multiplication.
inp = [1 0 1; 0 0 0; 1 1 0]; %Input matrix
C = nchoosek(1:size(inp,1),2); %Number of rows taken 2 at a time
out = inp(C(:,1),:) .* inp(C(:,2),:); %Multiplying those rows to get the desired output
Several things you can do:
Use logical ("binary") arrays (or even sparse logical arrays) instead of double arrays.
Use optimized combinatorical functions.
bitand or and instead of times (where applicable).
Vectorize:
function out = q44417404(I,k)
if nargin == 0
rng(44417404);
I = randi(2,89,42)-1 == 1;
k = 3;
end
out = permute(prod(reshape(I(nchoosek(1:size(I,1),k).',:).',size(I,2),k,[]),2),[3,1,2]);
I'm unsure how to phrase the question, but I think an example will help. Suppose I have a vector y = [3;1;4;1;6]. I want to create the matrix Y =
[0 0 1 0 0 0;
1 0 0 0 0 0;
0 0 0 1 0 0;
1 0 0 0 0 0;
0 0 0 0 0 1]
↑ ↑ ↑ ↑ ↑ ↑
1 2 3 4 5 6
where the element on each column is one or zero corresponding to the value in the vector.
I found that I could do it using
Y = []; for k = 1:max(y); Y = [Y (y==k)]; end
Can I do it without a for loop (and is this method more efficient if y has thousands of elements)?
Thanks!
Your method is not efficient because you're growing the size of Y in the loop which is not a good programming practice. Here is how your code can be fixed:
Ele = numel(y);
Y= zeros(Ele, max(y));
for k = 1:Ele
Y (k,y(k))= 1;
end
And here is an alternative approach without a loop:
Ele = numel(y); %Finding no. of elements in y
Y= zeros(Ele, max(y)); % Initiailizing the matrix of the required size with all zeros
lin_idx = sub2ind(size(Y), 1:Ele, y.'); % Finding linear indexes
Y(lin_idx)=1 % Storing 1 in those indexes
You can use bsxfun:
result = double(bsxfun(#eq, y(:), 1:max(y)));
If you are running the code on Matlab version R2016b or later, you can simplify the syntax to
result = double(y(:)==(1:max(y)));
Another approach, possibly more efficient, is to fill in the values directly using accumarray:
result = accumarray([(1:numel(y)).' y(:)], 1);
I found another solution:
E = eye(max(y));
Y = E(y,:);
Another solution:
Y = repmat(1:max(y), size(y)) == repmat(y, 1, max(y))
I have cell array A of dimension m * k.
I want to keep the rows of A unique up to an order of the k cells.
The "tricky" part is "up to an order of the k cells": consider the k cells in the ith row of A, A(i,:); there could be a row j of A, A(j,:), that is equivalent to A(i,:) up to a re-ordering of its k cells, meaning that for example if k=4it could be that:
A{i,1}=A{j,2}
A{i,2}=A{j,3}
A{i,3}=A{j,1}
A{i,4}=A{j,4}
What I am doing at the moment is:
G=[0 -1 1; 0 -1 2; 0 -1 3; 0 -1 4; 0 -1 5; 1 -1 6; 1 0 6; 1 1 6; 2 -1 6; 2 0 6; 2 1 6; 3 -1 6; 3 0 6; 3 1 6];
h=7;
M=reshape(G(nchoosek(1:size(G,1),h),:),[],h,size(G,2));
A=cell(size(M,1),2);
for p=1:size(M,1)
A{p,1}=squeeze(M(p,:,:));
left=~ismember(G, A{p,1}, 'rows');
A{p,2}=G(left,:);
end
%To find equivalent rows up to order I use a double loop (VERY slow).
indices=[];
for j=1:size(A,1)
if ismember(j,indices)==0 %if we have not already identified j as a duplicate
for i=1:size(A,1)
if i~=j
if (isequal(A{j,1},A{i,1}) || isequal(A{j,1},A{i,2}))...
&&...
(isequal(A{j,2},A{i,1}) || isequal(A{j,2},A{i,2}))...
indices=[indices;i];
end
end
end
end
end
A(indices,:)=[];
It works but it is too slow. I am hoping that there is something quicker that I can use.
I'd like to propose another idea, which has some conceptual resemblance to erfan's. My idea uses hash functions, and specifically, the GetMD5 FEX submission.
The main task is how to "reduce" each row in A to a single representative value (such as a character vector) and then find unique entries of this vector.
Judging by the benchmark vs. the other suggestions, my answer doesn't perform as well as one of the alternatives, but I think its raison d'être lies in the fact that it is completely data-type agnostic (within the limitations of the GetMD51), that the algorithm is very straightforward to understand, it's a drop-in replacement as it operates on A, and that the resulting array is exactly equal to the one obtained by the original method. Of course this requires a compiler to get working and has a risk of hash collisions (which might affect the result in VERY VERY rare cases).
Here are the results from a typical run on my computer, followed by the code:
Original method timing: 8.764601s
Dev-iL's method timing: 0.053672s
erfan's method timing: 0.481716s
rahnema1's method timing: 0.009771s
function q39955559
G=[0 -1 1; 0 -1 2; 0 -1 3; 0 -1 4; 0 -1 5; 1 -1 6; 1 0 6; 1 1 6; 2 -1 6; 2 0 6; 2 1 6; 3 -1 6; 3 0 6; 3 1 6];
h=7;
M=reshape(G(nchoosek(1:size(G,1),h),:),[],h,size(G,2));
A=cell(size(M,1),2);
for p=1:size(M,1)
A{p,1}=squeeze(M(p,:,:));
left=~ismember(G, A{p,1}, 'rows');
A{p,2}=G(left,:);
end
%% Benchmark:
tic
A1 = orig_sort(A);
fprintf(1,'Original method timing:\t\t%fs\n',toc);
tic
A2 = hash_sort(A);
fprintf(1,'Dev-iL''s method timing:\t\t%fs\n',toc);
tic
A3 = erfan_sort(A);
fprintf(1,'erfan''s method timing:\t\t%fs\n',toc);
tic
A4 = rahnema1_sort(G,h);
fprintf(1,'rahnema1''s method timing:\t%fs\n',toc);
assert(isequal(A1,A2))
assert(isequal(A1,A3))
assert(isequal(numel(A1),numel(A4))) % This is the best test I could come up with...
function out = hash_sort(A)
% Hash the contents:
A_hashed = cellfun(#GetMD5,A,'UniformOutput',false);
% Sort hashes of each row:
A_hashed_sorted = A_hashed;
for ind1 = 1:size(A_hashed,1)
A_hashed_sorted(ind1,:) = sort(A_hashed(ind1,:));
end
A_hashed_sorted = cellstr(cell2mat(A_hashed_sorted));
% Find unique rows:
[~,ia,~] = unique(A_hashed_sorted,'stable');
% Extract relevant rows of A:
out = A(ia,:);
function A = orig_sort(A)
%To find equivalent rows up to order I use a double loop (VERY slow).
indices=[];
for j=1:size(A,1)
if ismember(j,indices)==0 %if we have not already identified j as a duplicate
for i=1:size(A,1)
if i~=j
if (isequal(A{j,1},A{i,1}) || isequal(A{j,1},A{i,2}))...
&&...
(isequal(A{j,2},A{i,1}) || isequal(A{j,2},A{i,2}))...
indices=[indices;i];
end
end
end
end
end
A(indices,:)=[];
function C = erfan_sort(A)
STR = cellfun(#(x) num2str((x(:)).'), A, 'UniformOutput', false);
[~, ~, id] = unique(STR);
IC = sort(reshape(id, [], size(STR, 2)), 2);
[~, col] = unique(IC, 'rows');
C = A(sort(col), :); % 'sort' makes the outputs exactly the same.
function A1 = rahnema1_sort(G,h)
idx = nchoosek(1:size(G,1),h);
%concatenate complements
M = [G(idx(1:size(idx,1)/2,:),:), G(idx(end:-1:size(idx,1)/2+1,:),:)];
%convert to cell so A1 is unique rows of A
A1 = mat2cell(M,repmat(h,size(idx,1)/2,1),repmat(size(G,2),2,1));
1 - If more complicated data types need to be hashed, one can use the DataHash FEX submission instead, which is somewhat slower.
Stating the problem: The ideal choice in identifying unique rows in an array is to use C = unique(A,'rows'). But there are two major problems here, preventing us from using this function in this case. First is that you want to count in all the possible permutations of each row when comparing to other rows. If A has 5 columns, it means checking 120 different re-arrangements per row! Sounds impossible.
The second issue is related to unique itself; It does not accept cells except cell arrays of character vectors. So you cannot simply pass A to unique and get what you expect.
Why looking for an alternative? As you know, because currently it is very slow:
With nested loop method:
------------------- Create the data (first loop):
Elapsed time is 0.979059 seconds.
------------------- Make it unique (second loop):
Elapsed time is 14.218691 seconds.
My solution:
Generate another cell array containing same cells, but converted to string (STR).
Find the index of all unique elements there (id).
Generate the associated matrix with the unique indices and sort rows (IC).
Find unique rows (rows).
Collect corresponding rows of A (C).
And this is the code:
disp('------------------- Create the data:')
tic
G = [0 -1 1; 0 -1 2; 0 -1 3; 0 -1 4; 0 -1 5; 1 -1 6; 1 0 6; ...
1 1 6; 2 -1 6; 2 0 6; 2 1 6; 3 -1 6; 3 0 6; 3 1 6];
h = 7;
M = reshape(G(nchoosek(1:size(G,1),h),:),[],h,size(G,2));
A = cell(size(M,1),2);
for p = 1:size(M,1)
A{p, 1} = squeeze(M(p,:,:));
left = ~ismember(G, A{p,1}, 'rows');
A{p,2} = G(left,:);
end
STR = cellfun(#(x) num2str((x(:)).'), A, 'UniformOutput', false);
toc
disp('------------------- Make it unique (vectorized):')
tic
[~, ~, id] = unique(STR);
IC = sort(reshape(id, [], size(STR, 2)), 2);
[~, col] = unique(IC, 'rows');
C = A(sort(col), :); % 'sort' makes the outputs exactly the same.
toc
Performance check:
------------------- Create the data:
Elapsed time is 1.664119 seconds.
------------------- Make it unique (vectorized):
Elapsed time is 0.017063 seconds.
Although initialization needs a bit more time and memory, this method is extremely faster in finding unique rows with the consideration of all permutations. Execution time is almost insensitive to the number of columns in A.
It seems that G is a misleading point.
Here is result of nchoosek for a small number
idx=nchoosek(1:4,2)
ans =
1 2
1 3
1 4
2 3
2 4
3 4
first row is complement of the last row
second row is complement of one before the last row
.....
so if we extract rows {1 , 2} from G then its complement will be rows {3, 4} and so on. In the other words if we assume number of rows of G to be 4 then G(idx(1,:),:) is complement of G(idx(end,:),:).
Since rows of G are all unique then all A{m,n}s always have the same size.
A{p,1} and A{p,2} are complements of each other. and size of unique rows of A is size(idx,1)/2
So no need to any loop or further comparison:
h=7;
G = [0 -1 1; 0 -1 2; 0 -1 3; 0 -1 4; 0 -1 5; 1 -1 6; 1 0 6; ...
1 1 6; 2 -1 6; 2 0 6; 2 1 6; 3 -1 6; 3 0 6; 3 1 6];
idx = nchoosek(1:size(G,1),h);
%concatenate complements
M = [G(idx(1:size(idx,1)/2,:).',:), G(idx(end:-1:size(idx,1)/2+1,:).',:)];
%convert to cell so A1 is unique rows of A
A1 = mat2cell(M,repmat(h,size(idx,1)/2,1),repmat(size(G,2),2,1));
Update: Above method works best however if the idea is to get A1 from A other than G I suggest following method based of erfan' s. Instead of converting array to string we can directly work with the array:
STR=reshape([A.'{:}],numel(A{1,1}),numel(A)).';
[~, ~, id] = unique(STR,'rows');
IC = sort(reshape(id, size(A, 2),[]), 1).';
[~, col] = unique(IC, 'rows');
C1 = A(sort(col), :);
Since I use Octave I can not currently run mex file then I cannot test Dev-iL 's method
Result:
erfan method (string): 4.54718 seconds.
rahnema1 method (array): 0.012639 seconds.
Online Demo
I need to replace the zeros (or NaNs) in a matrix with the previous element row-wise, so basically I need this Matrix X
[0,1,2,2,1,0;
5,6,3,0,0,2;
0,0,1,1,0,1]
To become like this:
[0,1,2,2,1,1;
5,6,3,3,3,2;
0,0,1,1,1,1],
please note that if the first row element is zero it will stay like that.
I know that this has been solved for a single row or column vector in a vectorized way and this is one of the nicest way of doing that:
id = find(X);
X(id(2:end)) = diff(X(id));
Y = cumsum(X)
The problem is that the indexing of a matrix in Matlab/Octave is consecutive and increments columnwise so it works for a single row or column but the same exact concept cannot be applied but needs to be modified with multiple rows 'cause each of raw/column starts fresh and must be regarded as independent. I've tried my best and googled the whole google but coukldn’t find a way out. If I apply that same very idea in a loop it gets too slow cause my matrices contain 3000 rows at least. Can anyone help me out of this please?
Special case when zeros are isolated in each row
You can do it using the two-output version of find to locate the zeros and NaN's in all columns except the first, and then using linear indexing to fill those entries with their row-wise preceding values:
[ii jj] = find( (X(:,2:end)==0) | isnan(X(:,2:end)) );
X(ii+jj*size(X,1)) = X(ii+(jj-1)*size(X,1));
General case (consecutive zeros are allowed on each row)
X(isnan(X)) = 0; %// handle NaN's and zeros in a unified way
aux = repmat(2.^(1:size(X,2)), size(X,1), 1) .* ...
[ones(size(X,1),1) logical(X(:,2:end))]; %// positive powers of 2 or 0
col = floor(log2(cumsum(aux,2))); %// col index
ind = bsxfun(#plus, (col-1)*size(X,1), (1:size(X,1)).'); %'// linear index
Y = X(ind);
The trick is to make use of the matrix aux, which contains 0 if the corresponding entry of X is 0 and its column number is greater than 1; or else contains 2 raised to the column number. Thus, applying cumsum row-wise to this matrix, taking log2 and rounding down (matrix col) gives the column index of the rightmost nonzero entry up to the current entry, for each row (so this is a kind of row-wise "cummulative max" function.) It only remains to convert from column number to linear index (with bsxfun; could also be done with sub2ind) and use that to index X.
This is valid for moderate sizes of X only. For large sizes, the powers of 2 used by the code quickly approach realmax and incorrect indices result.
Example:
X =
0 1 2 2 1 0 0
5 6 3 0 0 2 3
1 1 1 1 0 1 1
gives
>> Y
Y =
0 1 2 2 1 1 1
5 6 3 3 3 2 3
1 1 1 1 1 1 1
You can generalize your own solution as follows:
Y = X.'; %'// Make a transposed copy of X
Y(isnan(Y)) = 0;
idx = find([ones(1, size(X, 1)); Y(2:end, :)]);
Y(idx(2:end)) = diff(Y(idx));
Y = reshape(cumsum(Y(:)), [], size(X, 1)).'; %'// Reshape back into a matrix
This works by treating the input data as a long vector, applying the original solution and then reshaping the result back into a matrix. The first column is always treated as non-zero so that the values don't propagate throughout rows. Also note that the original matrix is transposed so that it is converted to a vector in row-major order.
Modified version of Eitan's answer to avoid propagating values across rows:
Y = X'; %'
tf = Y > 0;
tf(1,:) = true;
idx = find(tf);
Y(idx(2:end)) = diff(Y(idx));
Y = reshape(cumsum(Y(:)),fliplr(size(X)))';
x=[0,1,2,2,1,0;
5,6,3,0,1,2;
1,1,1,1,0,1];
%Do it column by column is easier
x=x';
rm=0;
while 1
%fields to replace
l=(x==0);
%do nothing for the first row/column
l(1,:)=0;
rm2=sum(sum(l));
if rm2==rm
%nothing to do
break;
else
rm=rm2;
end
%replace zeros
x(l) = x(find(l)-1);
end
x=x';
I have a function I use for a similar problem for filling NaNs. This can probably be cutdown or sped up further - it's extracted from pre-existing code that has a bunch more functionality (forward/backward filling, maximum distance etc).
X = [
0 1 2 2 1 0
5 6 3 0 0 2
1 1 1 1 0 1
0 0 4 5 3 9
];
X(X == 0) = NaN;
Y = nanfill(X,2);
Y(isnan(Y)) = 0
function y = nanfill(x,dim)
if nargin < 2, dim = 1; end
if dim == 2, y = nanfill(x',1)'; return; end
i = find(~isnan(x(:)));
j = 1:size(x,1):numel(x);
j = j(ones(size(x,1),1),:);
ix = max(rep([1; i],diff([1; i; numel(x) + 1])),j(:));
y = reshape(x(ix),size(x));
function y = rep(x,times)
i = find(times);
if length(i) < length(times), x = x(i); times = times(i); end
i = cumsum([1; times(:)]);
j = zeros(i(end)-1,1);
j(i(1:end-1)) = 1;
y = x(cumsum(j));
I have a Matrix:
1 2 3
4 5 6
7 8 1
How may I use matlab to find this:
for 1st row: row3
for 2nd row: ---
for 3rd row: row1
I want to have row indices for each row witch have common elements.
Consider this
A = [1 2 3; %Matrix A is a bit different from yours for testing
4 5 6;
7 8 1;
1 2 7;
4 5 6];
[row col] =size(A)
answers = zeros(row,row); %matrix of answers,...
%(i,j) = 1 if row_i and row_j have an equal element
for i = 1:row
for j = i+1:row %analysis is performed accounting for
% symmetry constraint
C = bsxfun(#eq,A(i,:),A(j,:)'); %Tensor comparison
if( any(C(:)) ) %If some entry is non-zero you have equal elements
answers(i,j) = 1; %output
end
end
end
answers = answers + answers'; %symmetric
The output here is
answers =
0 0 1 1 0
0 0 0 0 1
1 0 0 1 0
1 0 1 0 0
0 1 0 0 0
of course the answers matrix is symmetric because your relation is.
The solution proposed by Acorbe can be quite slow if you have many rows and/or long rows. I have checked that in most cases the two solutions that I present below should be considerably faster. If your matrix contains just few different values (relative to the size of the matrix) then this should work pretty fast:
function rowMatches = find_row_matches(A)
% Returns a cell array with row matches for each row
c = unique(A);
matches = false(size(A,1), numel(c));
for i = 1:numel(c)
matches(:, i) = any(A == c(i), 2);
end
rowMatches = arrayfun(#(j) ...
find(any(matches(:, matches(j,:)),2)), 1:size(A,1), 'UniformOutput', false);
This other alternative might be faster when you have short rows, i.e. when size(A,2) is small:
function answers = find_answers(A)
% Returns an "answers" matrix like in Acorbe's solution
c = unique(A);
answers = false(size(A,1), size(A,1));
idx = 1:size(A,1);
for i = 1:numel(c)
I = any(A == c(i), 2);
uMatch = idx(I);
answers(uMatch, uMatch) = true;
isReady = all(A <= c(i), 2);
if any(isReady),
idx(isReady) = [];
A = A(~isReady,:);
end
end
Depending on what you are planning to do with this output it could be redundant to have a match for "3rd row: row1".
You already have this match earlier in your output in the form of "1st row: row3"