Is there a way to run explain with runCommand? I have the following query:
db.runCommand({geoNear:"Locations", near:[50,50], spherical:true})
How can I run an explain on it? I want to get the execution time.
As far as I know, explain is a method on cursors. You, however, can enable the integrated mongodb profiler:
db.setProfilingLevel(2); // log all operations
db.setProfilingLevel(1, 50); // log all operations longer than 50msecs
This will log details like nscanned, nreturned to a capped collection called system.profile, but does not provide as much detail as an explain() call does.
In this case, however, I think it might be possible to change the runCommand to a $near-query instead? That would give you full access to explain.
I guess we can't do explain for runCommand. Some of the runCommand give you the stats automatically ( eg. distinct command : db.runCommand({distinct : 'test', key : 'a'}) )
But, you can take help of the query profiler.
db.setProfilingLevel(2)
Once you run that query, switch off the profiler, and check the system.profile collection for this query.
According to explain docs you can simply:
db.runCommand({
explain: {geoNear:"Locations", near:[50,50], spherical:true}
})
Both answers (from mnemosyn and Abhishek Kumar) are correct.
However, if you just want to see the execution time (like me), this is provided in the results along with some extra stats:
...
"stats" : {
"time" : 2689,
"btreelocs" : 0,
"nscanned" : 1855690,
"objectsLoaded" : 979,
"avgDistance" : 0.006218027001875209,
"maxDistance" : 0.006218342348749806
},
"ok" : 1
Well, I should have looked closer before posting the question :).
Maybe not a runCommand but I use something like this:
expStats = function() {
var exp = db.collection.find({"stats.0.age" : 10},{"stats.age" : 1}).explain("allPlansExecution");
print ("totalDocsExamined = "+exp.executionStats.totalDocsExamined);
print ("nReturned = "+exp.executionStats.nReturned);
return print ("execTime = "+exp.executionStats.executionTimeMillis);
}
Related
I'm struggling with a seemingly simple query in mongodb.
I have a job collection that has objects like:
{
"_id" : ObjectId("5995c1fc3c2a353a782ee51b"),
"stages" : [
{
"start" : ISODate("2017-02-02T22:06:26Z"),
"end" : ISODate("2017-02-03T22:06:26Z"),
"name" : "stage_one"
},
{
"start" : ISODate("2017-02-03T22:06:26Z"),
"end" : ISODate("2017-02-07T20:34:01Z"),
"name" : "stage_two"
}
]
}
I want to get a job whose second stage does not have an end time, i.e. end is null or not defined.
According to the mongo docs on querying for null and querying an array of embedded documents, it would seem the correct query should be:
db.job.findOne({'stages.1.end': null})
However, running that query returns me the job above which does have a non-null end date. In fact, if I run the query with count instead of findOne, I see that all jobs are returned - no filtering is done at all.
For completeness, here is the output from an example on a fresh mongo instance:
So in this example, I would expect db.job.findOne({'stages.1.end': null}) to return nothing since there is only one document and its second stage has a non-null end date.
This feels like the sort of issue where it's just me being an idiot and if so, I apologise profusely.
Thanks in advance for your help and let me know if you need any more details!
EDIT:
After some more experimentation, I think I can achieve what I want with the following:
db.job.find({$or: [{'stages.1.end': { $type: 10 }}, {'stages.1.end': {$exists: false}}]})
While this gets the job done, it doesn't feel like the simplest way and I still don't understand why the original query doesn't work. If anyone could shed some light on this it'd be much appreciated.
While running a query i want a result of that query into another file
My query is running successfully in mongo terminal.
db.questions.find({"question" : {$regex : ".*.*"}},{question :1,_id:0,id:1});
{ "id" : 0, "question" : "Amount was debited from my account, but ticket was not
generated. What should I do now?" }
{ "id" : 1, "question" : "How safe is goCash?" }
{ "id" : 2, "question" : "How referral program and goCash works?" }
Now check following image:
here
By this i can write in any type of file by just adding >> filename.extension
Now the main problem is with this one:
When i use find instead of findOne and use a regular expression, it shows unexpected token
Anyone knows how i can modify it to get the desired result.
You are using "(qoutes) inside the command line. Try using {'question' : { \$regex : '.*'}}.
Further the db.collection.find() method returns a cursor. Redirecting it to a file will just write the cursor json.To access the documents, you need to iterate the cursor.
mongo localhost/database -eval "var cursor = {'question' : { \$regex : '.*'}}; while(cursor.hasNext()){ printjson( cursor.next())}"
All the examples in the blog return result as json format.
You can also use any of the mongo-drivers to write your result to a file.
I'm trying to find records in MongoDB that are created outside a specific period. The query to search for records inside a specific period is pretty straightforward:
db.test.find({"Published":{'$gt':"2011-08-02", '$lt':"2011-08-06"}})
So naturally, I tried this for "outside" a specific range:
db.test.find({'$not':{"Published":{'$gt':"2011-08-02", '$lt':"2011-08-06"}}})
But this returns an empty result, while there are definately records published then.
What query should I use instead? Can anyone help me? I'm using raw mongo queries.
Thanks in advance
--- UPDATE ---
I found that the following query works, but it doesn't look like the perfect solution:
db.test.find(
{'$or': [
{"Published":{'$lt':"2011-02-02"}},
{"Published":{'$gt':"2011-08-06"}}
]}
)
Is there a cleaner way to do it?
You are putting the $not in the wrong place. Try this:
db.test.find({"Published":{ $not:{$gt:"2011-08-02", $lt:"2011-08-06"} } })
For details, see the MongoDB docs about the $notoperator.
Edit as because of the comment this solution would not work:
> db.dates.find()
{ "_id" : ObjectId("5492d46ef6226b581c80c0a2"), "a" : 1, "date" : "2011-08-04" }
{ "_id" : ObjectId("5492d4e2f6226b581c80c0a3"), "a" : 2, "date" : "2011-08-07" }
> db.dates.find({date:{$not:{$gt:"2011-08-02",$lt:"2011-08-06"}}})
{ "_id" : ObjectId("5492d4e2f6226b581c80c0a3"), "a" : 2, "date" : "2011-08-07" }
I have define 2 reconds in 2 collections separately in one mongo db like this
order:
{
"_id" : ObjectId("53142f58c781abdd1d836fcd"),
"number" : "order1",
"user" : ObjectId("53159bd7d941aba0621073e3")
}
user
{
"_id" : ObjectId("53159bd7d941aba0621073e3"),
"name" : "user1",
"gender" : "male"
}
when I use this command in console, it can not execute
db.orders.find({user: db.user['_id']}) or db.orders.find({user: user['_id']}),
is there anything wrong? Thanks!
Another way is to:
> var user = db.users.findOne({name: 'user1'});
> db.orders.find({user: user._id});
This way can be a bit more flexible especially if you want to return orders for multiple users etc.
Taking your comment:
Thanks, but actually, I want to search all the orders of all users, not only one user. So what would I do? Thanks
db.users.find().forEach(function(doc){
printJson(db.orders.find({user: doc._id}));
});
I think you want:
db.order.find({'user':db.users.findOne({'name':'user1'})._id})
All you need to do is to make a query and check the output before inserting it into the query.
i.e. check that:
db.users.findOne({'name':'user1'})._id
has the output:
ObjectId("53159bd7d941aba0621073e3")
If you want to run larger queries, you're going to have to change your structure. Remember that Mongodb doesn't do joins so you'll need to do create a user document that looks like this:
user
{
"name":"user1",
"gender":"male",
"orders":[
{
'number':'order1'
}
]
}
You can then update this using:
db.user.update({'_id':ObjectId('blah')}, {'orders':{$push:{'number':'order2'}})
This will then start you with order tracking.
Mongo will be able to find using the following:
db.user.find({'orders.numbers':'order2'})
returning the full user record
Maybe you can help this solution:
use orders
db.getCollection('order').find({},{'_id':1})
db.getCollection('user').find({},{'_id':1})
This question is updated with a simpler explanation.
For my data the following Mongo CLI query takes 208ms. This query retrieves ALL the data for the 18 requested objects.
db.videos.find({avg_rating: {$gt: 1}, poster_large_thumb: {$exists:true}, release_date: {$lte: ISODate("2000-12-31")}}).sort({release_date:-1, avg_rating:-1, title:1}).skip(30).limit(18).pretty().explain()
{
"cursor" : "BasicCursor",
"nscanned" : 76112,
"nscannedObjects" : 76112,
"n" : 48,
"scanAndOrder" : true,
"millis" : 208,
"nYields" : 0,
"nChunkSkips" : 0,
"isMultiKey" : false,
"indexOnly" : false,
"indexBounds" : {
}
}
However with Mongoid when I do the query it creates a criteria object that has no actual data. I pass this criteria to a method that iterates it to create a JSON data structure. Since the data is needed each object must be retrieved from the DB. Here is the criteria returning query:
#videos = Video.order_by(release_date: -1, avg_rating: -1, title: 1).where(:avg_rating.gt => 1, :poster_large_thumb.exists => 1, :release_date.lte => start_date).skip(skip*POSTERS_PER_ROW).limit(limit*POSTERS_PER_ROW)
When I iterate #videos, each object takes over 240ms to retrieve from the DB, this from some debug output.
Getting one object:
2013-12-18 00:43:52 UTC
2013-12-18 00:43:52 UTC
0.24489331245422363
Assuming that if I got all the data in the Video.order_by(...) query it would take 208ms total how can I force it to do retrieval in one query instead of getting each objects individually?
Something here is causing the entire retrieval to take a couple of orders of magnitude more than the Mongo CLI.
Responses:
skip().limit() queries tend to get slower and slower on MongoDB side. As skip walks through the docs, more info see here https://stackoverflow.com/a/7228190/534150
The multiple identical queries look like to me a N+1 type of issue. That means that probably, somewhere in your view, you have a loop that calls a property that is lazy loaded, so it sends the query over and over again. Those problems are usually tricky to find, but to track them you need to have the end-to-end trace, which you are probably the only one that can do that, as you have access to the source code.
the Mongoid side looks correct to me.
Thanks for the ideas, I think #Arthur gave the important hint. No one could have answered because it looks like the problem was in another bit of code--in how I access the criteria.
Given the following query, which produces a criteria:
#videos = Video.order_by(release_date: 1, avg_rating: 1, title: -1).where(:release_date.ne => 0,:avg_rating.gt => 1, :poster_large_thumb.exists => 1, :release_date.gt => start_date).skip(skip*POSTERS_PER_ROW).limit(limit*POSTERS_PER_ROW).only(:_id, :poster_large_thumb, :title)
In a couple nested block I'm grabbing values with a line of code like this:
video_full = #videos[((row_index)*POSTERS_PER_ROW)+column_index]
This random access notation seems to be the problem. It seems to execute the full Moped query for every individual object so POSTERS_PER_ROW*num_rows times.
If I grab all the videos before the loops with this bit of code:
#videos.each do |video|
videos_full.push video
end
Then grab the values from an array instead of the criteria like this:
video_full = videos_full[((row_index)*POSTERS_PER_ROW)+column_index]
I get only one Moped query of 248ms, all objects are retrieved with that one query. This is a huge speed up. the query time goes from num_rows* POSTERS_PER_ROW*248ms to just 248ms
This is a big lesson learned for me so if someone can point to the docs that describe this effect and the rules to follow I'd sure appreciate it.