Finding bad words from large list of email addressess using PHP -Mongodb - mongodb

Question

I have large list of email addressses from a file. It comes around 1 million email ids. I have list of bad words like spam,junk etc, it consist of 20,000+ bad words.
I need to validate email ids. If bad words is present any where in email id it will be marked as invalid.
For example;
testspam#gmail.com - invalid
newuser#desspam.com - invalid
I would like to know which will be fastest comparison method as array looping will take time.
I tried following methods
//$keyword_list- array of bad words;
//$check_key- the email id which need to validate
$arrays = array_chunk($keyword_list, 2000);
for($i=0;$i<count($arrays);$i++)
{
if (preg_match('/'.implode('|', $arrays[$i]).'/', $check_key, $matches)){
return 1;
}
}
The above method is taking more time when comparing 1 million data.
Next we tried with the following code and this also takes more time
//$contain = bad words separated by '|'
// $str - the email id which need to validate
if(stripos($contain,"|") !== false)
{
$s = preg_split('/[|]+/i',$contain);
$len = sizeof($s);
for($i=0;$i < $len;$i++)
{
if(stripos($str,$s[$i]) !== false)
{
return(true);
}
}
}
if(stripos($str,$contain) !== false)
{
return(true);
}
return(false);
Finally I had tried Mongodb Text Search. It works fast with the following issues
If 'Hell' is the word in my bad list and my email id is like
head#e-hellinglysussex.sch.uk, then the Mongodb Text Search wont matches it.
Here is the code I used;
$ret = $db->command( array("text" =>$section, "search" => $keyword_string, "limit"=>$cnt_finalnonmatch));
where $section = Collection name,
$keyword_string = Comparing keywords string separated by space, for eg "Hell Spam Junk" etc,
$cnt_finalnonmatch = total number of comparing email ids
Please help me to solve this issue.

Answer 1

I am not entirely sure, but I suspect that the problem is that 'Hell' is not equal to 'hell' when you search for text since mongodb is case sensitive.
The solution should be to force all the strings and word to be lowercase (or uppercase)

Answer 2

We have used Mongodb 'like' to solve this issue;
$keywords = $key['keyword']; // Keywords need to compare
$regexObj = new MongoRegex("/".$keywords."/i"); // MongoRegex function declration
$where = array($section => $regexObj); // $section is the collection name
$resultset = $info->find($where);