500000x2 array, find rows meeting specific requirements of 1st and 2nd column, MATLAB - matlab

Question

I'm facing a dead end here..
I have collected a huge amount of data and I have isolated only the information that I'm interested in, into a 500K x 2 array of pairs.
1st column contains an ID of, let's say, an Access Point.
2nd column contains a string.
There might be multiple occurrences of an ID in the 1st column, and there can be anything written in the 2nd column. Remember, those are pairs in each row.
What I need to find in those 500K pairs:
I want to find all the IDs, or even the rows, that have 'hello' written in the 2nd column, AND as an additional requirement, there must be more than 2 occurrences of this 'pair'.
Even better want to save how many times this happens, if this happens more than 2 times.
so for example:
col1 (IDs): [ 1, 2, 6, 2, 1, 2, 3, 1]
col2 (str): [ 'hello', 'go', 'hello', 'piz', 'hello', 'da', 'mn', 'hello']
so the data that I ask is :
[ 1, 3 ] , which means, ID=1 , 3 occurences of id=1 with str='hello'

Answer 1

I tried to benchmark it to see if it could do 500.000 rows in a reasonable time.
generate some test data (in total about 60MB)
V = 1+round(rand(5E5,1).*1E4);
H = cell(1,length(V));
for ct = 1:length(H)
switch floor(rand(1)*10)
case 0
H{ct} = 'hello';
case 1
H{ct} = 'go';
case 2
H{ct} = 'piz';
case 3
H{ct} = 'da';
case 4
H{ct} = 'mn';
case 5
H{ct} = 'ds';
case 6
H{ct} = 'wf';
case 7
H{ct} = 'sf';
case 8
H{ct} = 'as';
case 9
H{ct} = 'sg';
end
end
The analysis
tic
a=ismember(H,{'hello'});
M = accumarray(V(a),1);
idx = find(M>1);
result = [idx,M(idx)];
toc
Elapsed time is 0.011699 seconds.
Alternative method with a loop
tic
M=zeros(max(V),1);
for ct = 1:length(H)
if strcmp(H{ct},'hello')
M(V(ct))=M(V(ct))+1;
end
end
idx = find(M>1);
result1 = [idx,M(idx)];
toc
Elapsed time is 0.192560 seconds.

Answer 2

Their are many possible solutions. Here is one: use a map structure. The key set of the map contains the ID's (where "hello" appears in the second column), and the value set contains the number of occurrences.
Run over the second column. When you find "hello", check if the corresponding ID is already a key in the map structure. If true, add +1 to the value associated to that key. Else, add a new pair (key,value) = (the ID, 1).
When finished, remove all the pairs from the map whose values are less or equal than 2. The remaining map is what you are looking for.
Matlab map: https://es.mathworks.com/help/matlab/map-containers.html