I am using maxTime to terminate the queries in mongo if it exceeds 1000ms.
Is there a way to find out if the query got terminated due to maxtime or it couldn't return any results due to match issues.
MongoDB will throw exception if query timeout.
Like this:
error: { "$err" : "operation exceeded time limit", "code" : 50 }
Hence you are able to distinguish between the exception case and no result case.
Read more here.
Related
Posting this here for anyone else encountering this error while running a query on MongoDb 5.0 and not able to find the root cause.
Sample Error:
{"ok":0,"code":5107200,"codeName":"Location5107200","$clusterTime":{"clusterTime":"7083090928751083534","signature":{"hash":"GDi3A/dasddadfssfsdfsdfddfs=","keyId":"2314243432443433"}},"operationTime":"7083090928751083534","name":"MongoError"}
Root cause: Review if there are any invalid values being passed to the $skip stage in your operation pipeline.
In my case, under certain scenarios we were passing a null value to $skip, causing this not a descriptive error from MongoDB.
I am getting the 'Sort operation used more than the maximum 33554432 bytes of RAM' error on some query. However what is even more troublesome, is that I can't even run it with .explain():
> db.collection.find({...}, {...}).limit(5000).sort({...})
Error: error: {
"ok" : 0,
"errmsg" : "errmsg: \"Sort operation used more than the maximum 33554432 bytes of RAM. Add an index, or specify a smaller limit.\"",
"code" : 96,
"codeName" : "OperationFailed"
}
> db.collection.find({...}, {...}).limit(5000).sort({...}).explain()
2019-07-22T09:15:38.246+0000 E QUERY [thread1] Error: explain failed: {
"ok" : 0,
"errmsg" : "errmsg: \"Sort operation used more than the maximum 33554432 bytes of RAM. Add an index, or specify a smaller limit.\"",
"code" : 96,
"codeName" : "OperationFailed"
} :
_getErrorWithCode#src/mongo/shell/utils.js:25:13
throwOrReturn#src/mongo/shell/explainable.js:31:1
constructor/this.finish#src/mongo/shell/explain_query.js:172:24
DBQuery.prototype.explain#src/mongo/shell/query.js:521:12
#(shell):1:1
Tried to recreate the index but nothing changed.
What could be happening here?
Edit: To clarify, the sort field is indexed and in the correct sort order. I was wondering why the explain fails, which seemed odd to me and I thought there might be data corruption going on here. How are we meant to diagnose a problematic query with explain if it fails on what we are trying to diagnose?
Edit 2: Upon further diagnosis I could literally pinpoint it to .limit(4308) works and .limit(4309) barfs. However there is nothing wrong with the 4309th record...
Furthermore this happens in one env and not the other that are identical expect for data.
For any time travelers from the future:
RE .explain(), seems to be just a quirk in Mongo. To see the query plan the limit must be reduced. I guess as silly as this sounds Mongo actually runs the query and then shows the query plan...
Worth noting that this is Mongo 3.4. Might have changed by now...
Our performance problem came down to having a huge subobject property (let's call it .metaData). However since we know it's problematic we didn't include it in the projection. But - it does appear in the find criteria as {metaData: {$exists: true}}. I guess mongo fetches the whole thing and keeps it in memory only to do {$exists: true} on it. That led the query to blow up the 32M memory limit eventhough the actual result requires much less memory and the sort field is indexed.
So we live to write more bugs another day...
I am trying to convert a huge set of dates to ISODate objects and storing it in the same collection and field. I am using the following query:
db.collection.find().forEach(function(element){
element.StartTime = ISODate(element.StartTime);
element.StopTime = ISODate(element.StopTime);
db.collection.save(element);
});
The query ran for about 10 minutes, and then gave an error:
2017-06-15T15:48:10.419+0200 E QUERY [thread1] Error: invalid ISO date :
ISODate#src/mongo/shell/types.js:65:1
#(shell):2:23
DBQuery.prototype.forEach#src/mongo/shell/query.js:501:1
#(shell):1:1
I had a look into the entries in the DB and it looks like it did convert a lot of the data set, but I am having troubles now fixing the error and finding the location where it stopped. What I tried is using Studio3T to find where either "StartTime" or "StopTime" has a value of 2017-06-15T15:48:10.419+0200 or at least starts with 2017-06, but as I expected (since there shouldn't be any data from June 2017 in there), it can't find anything.
Now when I run the query again, it gives the error immediately.
My question is if it's possible to find the document in the collection responsible for this error or what the cause might be. Are there recommendations for bulk operations like this preventing these errors?
I been testing a mapreduction operation over huge data set of about 25 million documents. The reduce function is correct and works fine when tested on a small data set.
I get the following error while running the mapreduce over large set
"errmsg" : "exception: Exceeded depth limit of 150 when converting js object to BSON. Do you have a cycle?",
"code" : 17279,
"ok" : 0
I have run into a problem where I have exceeded the allowed BSON size of 16MB and I am getting this error now whenever I try to do something on my collection.
Now my question is, how do I repair and solve the problem?
How do I check whether it is an individual document within my collection, or the collection itself exceeding the limit
How do I remove the offending document? I just keep getting this error whenever I try doing something with this collection now.
I have already tried db.repairDatabase(), but just keep getting the same error:
"errmsg" : "exception: BSONObj size: 1718558820 (0x666F2064) is invalid. Size must be between 0 and 16793600(16MB) First element: ...: ?type=32",
"code" : 10334,
"ok" : 0
Look at the size. It's obviously not a size, it's four ASCII characters. Go and find your bug.