Faster way to group data in the same quartile range - matlab

Question

Consider a column of a 10 x 10 matrix K, say K(:,1)
I would like to create a 10x4 binary matrix which tells us which quarter range the row entry belongs to. For example
ith row of binary matirx : [ 1 0 0 0 ] => K(i,1)<prctile(K(:,1),25)
My code:
%%%
K = randi(10,10);
BINMAT = zeros(size(K,1),4);
y_1 = prctile(K(:,1),25) ;
ID_1 = find(K(:,1) < y_1);
BINMAT(ID_1,1)=1;
y_2 = prctile(K(:,1),50);
ID_2 = find(( K(:,1) > y_1 & K(:,1) < y_2 ));
BINMAT(ID_2,2)=1;
y_3 = prctile(K(:,1),75);
ID_3 = find(( K(:,1) > y_2 & K(:,1) < y_3 ));
BINMAT(ID_3,3)=1;
y_4 = prctile(K(:,1),100);
ID_4 = find((K(:,1) > y_3 & K(:,1) < y_4 ));
BINMAT(ID_4,4)=1;
%%%
If I have to do this not just for one column but for a set of columns, say A = [ 1 2 5 6], and BINMAT should have 16 columns (4 for each column of K) .Is there a faster way to do this?

Answer 1

You can use a for loop that iterates over the desired column indexes given by A:
K = randi(10,10);
A = [1 2 5 6]; % columns in K to process
BINMAT = zeros(size(K,1), 4*length(A));
cnt = 0; % helper
for col_indx = A
y_1 = prctile(K(:,col_indx),25) ;
ID_1 = find(K(:,col_indx) < y_1);
BINMAT(ID_1, 4*cnt + 1) = 1;
y_2 = prctile(K(:,col_indx),50);
ID_2 = find(( K(:,col_indx) > y_1 & K(:,col_indx) < y_2 ));
BINMAT(ID_2, 4*cnt + 2)=1;
y_3 = prctile(K(:,col_indx),75);
ID_3 = find(( K(:,col_indx) > y_2 & K(:,col_indx) < y_3 ));
BINMAT(ID_3, 4*cnt + 3)=1;
y_4 = prctile(K(:,col_indx),100);
ID_4 = find((K(:,col_indx) > y_3 & K(:,col_indx) < y_4 ));
BINMAT(ID_4, 4*cnt + 4)=1;
cnt = cnt + 1;
end
I have noticed that many of the rows of BINMAT contain only zeros because the code you posted does not take values equal to y_1, y_2, y_3 and y_4 into account. I think you should use K(:,col_indx) >= y_1 ... and so on.

Answer 2

Another suggestion:
K = randi(10,10)
p = 25:25:100;
Y = prctile(K, p);
Y = [zeros(1, size(Y, 2)) ;Y];
BINMAT = zeros(size(K, 1), length(p), size(K, 2));
for j = 1:size(K, 2)
for i = 1:length(p)
BINMAT(Y(i, j) <= K(:,j) & K(:, j) <= Y(i+1, j), i, j) = 1;
end
end
Then, BINMAT(:, :, i) is the binary matrix, as you defined it, for K(:, i).

Answer 3

Percentile is, at its heart, the position of an element in the sorted list. So using sort directly will provide the most efficient solution, since you want multiple percentiles out of multiple columns.
First we need a way to assign fixed bins to the sorted positions. Here's the vector that I think prctile uses, but since 10 doesn't split evenly into 4 bins, it's somewhat arbitrary. (in other words, do you assign element 3 to the 0-25% bin or the 25%-50% bin)? floor(4*(0.5+(0:9).')/10)+1
Now we just need to sort each column, and assign the sort position of each original element to one of those positions. The second output of sort does most of the work:
K = randi(10,10);
A = [1 2 5 6]; % columns in K to process
BINMAT = zeros(size(K,1), 4*length(A));
bins = floor(4*(0.5+(0:9).')/10)+1;
[sortedK, idx] = sort(K(:,A));
% The k'th element of idx belongs to the c(k) bin. So now generate the output.
% We need to offset to the correct block of BINMAT for each column
offset_bins = bsxfun(#plus, bins, 4*(0:length(A)-1));
BINMAT(sub2ind(size(BINMAT), idx, offset_bins)) = 1;