Mongo compound index is not chosen - mongodb

Question

I have the following schema:
{
score : { type : Number},
edges : [{name : { type : String }, rank : { type : Number }}],
disable : {type : Boolean},
}
I have tried to build index for the following query:
db.test.find( {
disable: false,
edges: { $all: [
{ $elemMatch:
{ name: "GOOD" } ,
},
{ $elemMatch:
{ name: "GREAT" } ,
},
] },
}).sort({"score" : 1})
.limit(40)
.explain()
First try
When I created the index name "score" :
{
"edges.name" : 1,
"score" : 1
}
The 'explain' returned :
{
"cursor" : "BtreeCursor score",
....
}
Second try
when I changed "score" to:
{
"disable" : 1,
"edges.name" : 1,
"score" : 1
}
The 'explain' returned :
"clauses" : [
{
"cursor" : "BtreeCursor name",
"isMultiKey" : true,
"n" : 0,
"nscannedObjects" : 304,
"nscanned" : 304,
"scanAndOrder" : true,
"indexOnly" : false,
"nChunkSkips" : 0,
"indexBounds" : {
"edges.name" : [
[
"GOOD",
"GOOD"
]
]
}
},
{
"cursor" : "BtreeCursor name",
"isMultiKey" : true,
"n" : 0,
"nscannedObjects" : 304,
"nscanned" : 304,
"scanAndOrder" : true,
"indexOnly" : false,
"nChunkSkips" : 0,
"indexBounds" : {
"edges.name" : [
[
"GOOD",
"GOOD"
]
]
}
}
],
"cursor" : "QueryOptimizerCursor",
....
}
Where the 'name' index is :
{
'edges.name' : 1
}
Why is the mongo refuses to use the 'disable' field in the index?
(I've tried other fields too except from 'disable' but got the same problem)

Answer 1

It looks like the query optimiser is choosing the most efficient index and the index on edges.name works best. I recreated your collection by inserting a couple documents according to your schema. I then created the compound index below.
db.test.ensureIndex({
"disable" : 1,
"edges.name" : 1,
"score" : 1
});
When running an explain on the query you specified, the index was used.
> db.test.find({ ... }).sort({ ... }).explain()
{
"cursor" : "BtreeCursor disable_1_edges.name_1_score_1",
"isMultiKey" : true,
...
}
However, as soon as I added the index on edges.name, the query optimiser chose that index for the query.
> db.test.find({ ... }).sort({ ... }).explain()
{
"cursor" : "BtreeCursor edges.name_1",
"isMultiKey" : true,
...
}
You can still hint the other index though, if you want the query to use the compound index.
> db.test.find({ ... }).sort({ ... }).hint("disable_1_edges.name_1_score_1").explain()
{
"cursor" : "BtreeCursor disable_1_edges.name_1_score_1",
"isMultiKey" : true,
...
}
[EDIT: Added possible explanation related to the use of $all.]
Note that if you run the query without $all, the query optimiser uses the compound index.
> db.test.find({
"disable": false,
"edges": { "$elemMatch": { "name": "GOOD" }}})
.sort({"score" : 1}).explain();
{
"cursor" : "BtreeCursor disable_1_edges.name_1_score_1",
"isMultiKey" : true,
...
}
I believe the issue here is that you are using $all. As you can see in the result of your explain field, there are clauses, each pertaining to one of the values you are searching with $all. So the query has to find all pairs of disable and edges.name for each of the clauses. My intuition is that these two runs with the compound index make it slower than a query that looks directly at edges.name and then weeds out disable. This might be related to this issue and this issue, which you might want to look into.