How do I determine where a word appears over multiple sentences? - matlab

Question

I have the following code that tells me how many times a word appears in a sentence. Specifically, I have a 1D cell array of sentence strings and a 1D cell array of words that I want to use to search within each sentence. This code is a 2D cell array where each row and column combination tells me how many times I see a particular word (column) appear in a sentence (row). In other words:
Out = NaN(numel(sentences), numel(out_words));
for i = 1:numel(out_words)
Out(:,i) = cellfun(#(x) numel(strfind(x, out_words{i})), sentences);
end
display(Out)
What I would like now is a 1D cell array Out where each element describes a word and within this element is a vector that tells you which sentences the word appears in.
For example, if the word is trees and trees is assigned an ID of 1, a potential vector within the cell element of Out{1} could be [1,5,8], which means that the word trees appeared in sentences 1, 5, and 8. Is there a way to do this in MATLAB?

Answer 1

I will reiterate what I asked in the comments so people on the StackOverflow community will know what the question is really asking:
The OP wants a 1D cell array Out where each element describes a word and within this element is a vector that tells you which sentences the word appears in.
For example, if the word is trees and trees is assigned an ID of 1, a potential vector within the cell element of Out{1} could be [1,5,8], which means that the word trees appeared in sentences 1, 5, and 8.
One easy way to do this would be to loop over every word that you have and use strfind and see which elements in the output are non-empty. For those locations that are non-empty, this would determine where the word has occurred in a particular sentence. Let's do an example. I'm going to choose six sentences to be the first six lyrics of Bruce Cockburn's Lovers in a Dangerous Time. I'll declare this to be in a cell array of strings called sentences. To be sure we can find words correctly and not worry about case, we will convert all sentences to lower case with lower:
sentences = {'Don''t the hours grow shorter as the days go by',
'You never get to stop and open your eyes'
'One day you''re waiting for the sky to fall'
'The next you''re dazzled by the beauty of it all'
'When you''re lovers in a dangerous time'
'Lovers in a dangerous time'};
sentences = lower(sentences);
Next, I'm going to declare a words array that determines which words I want to find over all of the sentences:
words = {'you', 'one', 'dangerous', 'beauty', 'go', 'to'};
As such, the code you want is very simply:
Out = cell(numel(words), 1); %// Declare empty array of cells for each word
for idx = 1 : numel(words) %// For each word...
K = strfind(sentences, words{idx}); %// See which sentences have these words
ind = cellfun('isempty', K); %// Determine which locations are EMPTY
Out{idx} = find(~ind); %// To find those locations that are non-empty, we need to find those entries that are 0, so search for the inverse
end
Let's go through the above code slowly. We first declare a cell array of elements (1D) that is as long as the total number of words we have. Next, for each word, we use strfind to determine whether we can find that particular word in all of the sentences. strfind will return a cell array where each element in this array tells you the starting index (or indices if there is more than one occurrence) of where we have found this word. If an element in this cell array is empty, this means that we did not find the word in the sentence.
Now, what we're going to do next is search within this cell array for any entries that are empty. This can be done with cellfun and the output will be a logical vector where 1 means it's empty and 0 means it's non-empty. We want to find those locations that are non-empty, and so we use find to search for locations that are non-empty. These locations ultimately determine whether we have found that word in that sentence.
As such, if we run with the above example, this is what we get:
>> celldisp(Out)
Out{1} =
2
Out{2} =
3
Out{3} =
5
6
Out{4} =
4
Out{5} =
1
Out{6} =
2
3
This means that for the first word, you, we have found this word in sentence #2, which is: 'you never get to stop and open your eyes'. Next, the second word is one, and we have found this in sentence #3, which reads: 'one day you're waiting for the sky to fall'. The next word is dangerous, and we have found this in sentences #5 and #6, which read: 'when you're lovers in a dangerous time' and
'lovers in a dangerous time'. You can follow along with the rest of the cell array and you can verify that what each cell element gives you are the sentence IDs that tell you where each word appeared.
In your comments, you want to go further and make an associative array where you specify the string you want and the output will be the sentence IDs of the sentences that contain these words. You can use a containers.Map class to do that for you. Specifically:
out_dict = containers.Map(words, Out);
Now, you can do:
>> out_dict('dangerous')
ans =
5
6
If you want to input in multiple words, use the values method:
vals = values(out_dict, {'you', 'dangerous', 'go'})
celldisp(vals)
vals{1} =
2
vals{2} =
5
6
vals{3} =
1
If you want to display all of the words, just do:
vals = values(out_dict, words);
celldisp(vals)
vals{1} =
2
vals{2} =
3
vals{3} =
5
6
vals{4} =
4
vals{5} =
1
vals{6} =
2
3
BTW, I will reiterate this for you. Please consider accepting the answers that were provided to you in your previous questions. This signifies to the StackOverflow community that you no longer need help for your questions. Since you like reading answers, you can read this to help you figure out how to accept answers: How to update and accept answers
Good luck!