Map Reduce does not emit large data sets - mongodb

Question

I am facing issues in map reduce, whenever expected result data set is large it returns nothing, it works for small data sets like for 40 thousand documents. Following is the code and problem understanding. See, I used this code
search = "breaking bad f"
var emit = function(a,b){
print(a);
}
map = function() {
if(this.torrent_name.indexOf(search) > -1){
emit(this._id, this.torrent_name);
}
}
reduce = function(key,values){
return values;
}
res = db.torrents.mapReduce(map,reduce,{out: { inline: 1 },query:{$text:{$search:search}},scope:{search:search},sort:{'seeders':-1}})
printjson(res);
Now the result to this job is:
{
"results" : [ ],
"timeMillis" : 503,
"counts" : {
"input" : 39859,
"emit" : 0,
"reduce" : 0,
"output" : 0
},
"ok" : 1
}
which makes sense because map reduce input is same as answer to below query
db.torrents.find({$text:{$search:"breaking bad f"}}).count()
output => 39859
Now the main issue come when I change the search string in map reduce job to "breaking bad s", the result shown is
{
"results" : [ ],
"timeMillis" : 329,
"counts" : {
"input" : 0,
"emit" : 0,
"reduce" : 0,
"output" : 0
},
"ok" : 1
}
which does not makes any because map reduce input is not equal to answer of below query
db.torrents.find({$text:{$search:"breaking bad s"}}).count()
output => 71484
From above results I come to conclusion that there is come memory issue but I don't know where and why. Please help.

Answer 1

Your process here is flawed in a number of ways.
Text search does not work like that
You are asking a $text search query to match on partial words such as
"breaking bad s"
"breaking bad f"
In each case, the "s" and the "f" here are ignored as they are not a whole word. So the only terms looked for are "breaking" and "bad". And I do mean "terms" here a opposed to a "phrase" as in "breaking bad", as the syntax you are using does not do that, but only looks for the terms instead.
They "might" be the phrase, but generally they will not be if the data being searched contains "breaking" or "bad" in any other combination.
I don't know where you think those counts are coming from, but it certainly has nothing to do with the "non-word" that is appended there.
The mapper is also wrong
Following on from above, since what you are actually matching here is "breaking" and "bad" as individual words, then it makes sense to only check that those "words" are present in the string. They will be of course, but the test is wrong and should be written like this:
map = function() {
if ( /breaking|bad/.test(this.torrent_name) ) {
emit(null,1);
}
};
Reducer is wrong as well
More to the point, besides the "emits" failing earlier, the reducer would never be called at all. With mapReduce the idea is that all "common" _id values are sent to the reducer for "reduction" to a single value and then returned.
The way you had this writen you are just trying to pass out values which is actually an array. If the reducer had fired, then this produces a "big" error, in that you can only return a "single" value. That is in fact why it is called a "reduce" stage, because you want to "reduce" the grouped data down to a single common point.
So again we rewrite this to something logical:
var reduce = function(key,values) {
return Array.sum(values);
};
Also noting here that what comes "out" of the emitted data from the mapper must be the same structure and type as what comes "out" of the reducer as well.
This is because in order to handle "large data", mapReduce does not process the same grouped key "all at once", but rather in small "chunks". So data that comes "out" of a reducer, can end up going back "in" as one of the values to further reduce.
So finally if you run this:
res = db.torrents.mapReduce(
map,
reduce,
{
"out": { "inline": 1 },
"query": { "$text":{ "$search" :search } }
}
);
You may actually just get a sane response that tells you it did something.
But as you develop this further, take note of what is said above and fully read the documentation, which also explains the points as descrived above.