Vectorizing set operations for string-valued cell arrays in MATLAB/Octave - matlab

Question

I have a large data set, X, comprising demographic information of survey respondents. The data is largely categorical, so each row in X contains a bunch of string-valued features such as gender, race, interests, etc for a single respondent. Each column of X is a single response category. I have loaded this data set into a big cell array in MATLAB/Octave (testing on both). I would like to measure the Jaccard distance between each sample and every other sample in the data set. Basically what I want to do is this:
dist = zeros(size(X,1)); % Initialize my distance matrix
for ii = 1:size(X,1)
for jj = ii:size(X,1) % Only need the upper triangle since dist is symmetric
% Find the Jaccard distance between the ii-th and jj-th respondent
dist(ii,jj) = 1 - numel(intersect(X(ii,:), X(jj,:))) / numel(union(X(ii,:), X(jj,:)));
end
end
Except obviously I want to vectorize the code. I have tried using cellfun and bsxfun to vectorize, but when I do something like:
res = cellfun('intersect', X, X, 'UniformOutput', false);
I get a cell array the same size as X, wherein the (i,j)-element is equivalent to intersect(X(i,j), X(i,j)); basically the unique characters in the (i-j)-cell. This does not help me. When I try:
res = bsxfun('intersect', X, X);
I get one long cell array containing (I think) all of the unique values that any cell in X takes. This does not help me either.
I would like a solution that enables me to vectorize the code at the beginning of this discussion. If it is easier to do so, a code that finds the subset of X with the minimum (or maximum) Jaccard distance from any one row in X would be exactly what I need.
Thanks in advance!
EDIT: Changed the loop code to only calculate the upper triangle of dist. Still takes far too long, and the fact that it is non-vectorized bugs me on a philosophical level.
EDIT: The first element of X, given by typing X(1,:) is:
ans =
{
[1,1] = Non - U.S. Citizen
[1,2] = Denied
[1,3] = M
[1,4] = CHINA
[1,5] = Full Time
[1,6] = D-Asian American or Pacific Islander
[1,7] =
[1,8] =
[1,9] = MSME
[1,10] =
}
This is just testing data for developing the algorithm while I wait on my actual survey results, but the survey results will have a similar form.
EDIT: More data from X, but in CSV form, is as follows:
Non - U.S. Citizen,Denied,M,INDIA,Full Time,E-Other,,,MSME,
Non - U.S. Citizen,Denied,F,INDIA,Full Time,D-Asian American or Pacific Islander,,,MSME,DESIGN
Non - U.S. Citizen,Denied,M,INDIA,Full Time,E-Other,,,MS,
Non - U.S. Citizen,Denied,M,IRAN,Full Time,B-Caucasian American Non-Hispanic,,,PhD,NANO
Non - U.S. Citizen,Left Without Degree,M,JORDAN,Full Time,E-Other,,,,
Non - U.S. Citizen,Denied,F,IRAN,Full Time,E-Other,,,PhD,BIOENG
,Not Attending,M,,Full Time,,,,PhD,
Non - U.S. Citizen,Not Attending,F,IRAN,Full Time,I-International Student,,,PhD,
Non - U.S. Citizen,Denied,M,BANGLADESH,Full Time,E-Other,,,PhD,NANO
Non - U.S. Citizen,Denied,M,BANGLADESH,Full Time,E-Other,,,MS,

Answer 1

This might be a workaround, I'll illustrate on a single row of data:
a={'Non - U.S. Citizen','Denied','M','INDIA','Full Time','E-Other','','','MSME',''}
Sum each cell element, this casts the strings to doubles and sum thier value. It will work assuming the odds for a non unique sum result are slim (if not there's a trick you can implement, but I doubt it'll actually happen):
b=cellfun(#sum,a,'un',0)
now you have a single number per cell element, you can use cell2mat to get a matrix and \ or pdist etc ...