Why does MongoDB count query sometimes uses IXSCAN, explain shows COUNT_SCAN? - mongodb

Question

I found that a count query with two parameters was taking longer than expected on our production database. I added an index (which took a few hours, this collection has over 100 million documents) that had both fields, which improved the results of my .explain() call from IXSCAN to COUNT_SCAN.
Looking at the logs now, I see that there are still tons of IXSCAN planSummaries for this count query:
2019-07-17T13:02:34.561+0000 I COMMAND [conn25293] command
DatabaseName.CollectionName command: count { count: "CollectionName",
query: { userId: "5a4f82d4e4b09d5e0cdbae15", status: "FINISHED" } }
planSummary: IXSCAN { userId: 1 } keysExamined:299 docsExamined:299
numYields:7 reslen:44 locks:{ Global: { acquireCount: { r: 16 } },
Database: { acquireCount: { r: 8 } }, Collection: { acquireCount: { r:
8 } } } protocol:op_query 124ms
There is an index on the userId field, but I don't understand why this count query isn't hitting my new index. Here's the explain results:
db.CollectionName.explain().count({ userId: "59adb07de4b00782f7620c11", status: "FINISHED" })
/* 1 */
{
"queryPlanner" : {
"plannerVersion" : 1,
"namespace" : "DatabaseName.CollectionName",
"indexFilterSet" : false,
"parsedQuery" : {
"$and" : [
{
"status" : {
"$eq" : "FINISHED"
}
},
{
"userId" : {
"$eq" : "59adb07de4b00782f7620c11"
}
}
]
},
"winningPlan" : {
"stage" : "COUNT",
"inputStage" : {
"stage" : "COUNT_SCAN",
"keyPattern" : {
"userId" : 1.0,
"status" : 1.0
},
"indexName" : "idx_userId_status",
"isMultiKey" : false,
"multiKeyPaths" : {
"userId" : [],
"status" : []
},
"isUnique" : false,
"isSparse" : false,
"isPartial" : false,
"indexVersion" : 1,
"indexBounds" : {
"startKey" : {
"userId" : "59adb07de4b00782f7620c11",
"status" : "FINISHED"
},
"startKeyInclusive" : true,
"endKey" : {
"userId" : "59adb07de4b00782f7620c11",
"status" : "FINISHED"
},
"endKeyInclusive" : true
}
}
},
"rejectedPlans" : []
},
"serverInfo" : {
"host" : "ip-10-114-1-8",
"port" : 27017,
"version" : "3.4.16",
"gitVersion" : "0d6a9242c11b99ddadcfb6e86a850b6ba487530a"
},
"ok" : 1.0
}
Checking the index stats, I see that it has been used quite a bit
{
"name" : "idx_userId_status",
"key" : {
"userId" : 1.0,
"status" : 1.0
},
"host" : "ip-address:27017",
"accesses" : {
"ops" : NumberLong(541337),
"since" : ISODate("2019-07-16T14:34:25.281Z")
}
}
I'm assuming that this means it is used sometimes, but for some reason not used at other times. Why would that be the case?

Answer 1

In my understanding of query planning in MongoDB, the DB keeps some kind of cache of query planning to be able to chose the right one without much thinking.
My guess is, in the case of the IXSCAN, the DB thought that using this one or the other wouldn't make that much of a difference.
That being said, you can still use the explain(true) (or more exactly explain("allPlansExecution") that will try to execute all possible plans. And then if you analyze the executionTimeMillis, you might see some difference that would explain the choice of the query planning.
Hope this helps :)