I want to log my query times for each time when a query made.
I'm using mongodb on playframework. Simply I'm thinkinig to substract start and end time.For ex:
a=currenttime();
madequert();
querytime=currenttime()-a;
Is there any better way?
You probably want to use Mongo's DB profiler. That way you'll keep that of your code (less work to maintain it) and it will give you more options to check Mongo behaviour.
Related
I want to run aggregation on my large data sets. (It's about 361K documents) and Insert them to another collection.
I getting this error:
I tried to increase Max Time but it has maximum and it's not enough for my data sets. I found https://docs.mongodb.com/manual/reference/method/cursor.noCursorTimeout/ but it seems noCursorTimeout only apply on find not aggregation.
please tell me how I can disable cursor timeout or another solution to do this.
I am no MongoDB expert but will interpret what I know.
MongoDB Aggregation Cursors don't have a mechanism to adjust Batch Size or set Cursor Timeouts.
Therefore there is no direct way to alter this and the timeout of an aggregation query solely depends on the cursorTimeoutMillis parameter of the MongoDB or mongos` instance. Its default timeout value is 10 minutes.
Your only option is to change this value by the below command.
use admin
db.runCommand({setParameter:1, cursorTimeoutMillis: 1800000})
However, I strongly advise you against using this command. That's because it's a safety mechanism built into MongoDB. It automatically deletes queries that are running idle for more than 10 minutes, so that there is a lesser load in the MongoDB server. If you change this parameter (say to 30 minutes), MongoDB will allow idle queries to be running in the background for those 30 minutes, which will not only make all the new queries slower to execute, but also increase load and memory on the MongoDB side.
You have a couple of workarounds. Reduct the amount of documents if working on MongoDB Compass or copy and run the commands on Mongo Shell (I had success so far with this method).
I want to kill all the queries that are taking too long to execute. I have tried socketTimeOut but it is not the good way to do the same and the expected result is not achieved.
Read this for explanation http://mongodb.github.io/node-mongodb-native/2.2/reference/faq/. There is also an option of maxTimeMS, but we cannot make it general and pass it in mongoClient URI options. It is query specific. Is there any other parameter that I can set for the terminating all the queries that take more time to execute.
Any settings in mongo config can also help, but it should work for all the types of queries like insert,find,update, remove. I have also seen db.currentOp() option, but I want to know if it can be done in a better way and if db.currentOp() is the only option, how can I use it to find and terminate a query with Java Driver.
Thank you
I have gone through all the documentations, there is no way we can implement it.
The thing is maxTimeMS is implemented for each query but not on the Global URI level.
I am new to the world of NoSQL databases and MongoDB.
I am trying to access the query execution time in Mongodb. I plan to do CURD operations with a set of 1000,10000 and 100000 records.I would also like to run few other random queries. I want to execute these queries through a java class. Can any one kindly suggest me the best way to get the response times from the java class file. I understand Database Profiler for mongodb is a good way, but would like to execute profiler commands through java.Could any one suggest how to execute Profiler commands from java class for each query.
P.S: Kindly suggest if there is any better way. Any suggestion would be of a great help.
Using the Mongo Java client you would invoke find() in the usual way and then append a query modifier like so:
MongoCollection<...> collection = ...
FindIterable<...> results = collection.find(query).modifiers(new Document(QueryOperators.EXPLAIN, true));
This will return you Mongo's profiler document for that query.
Plenty of details here on the format, content etc of that document.
If you are interested in profiling you might also need to hint (i.e. force Mongo to use a specific index), this is done in much the same way, for example:
MongoCollection<...> collection = ...
FindIterable<...> results = collection.find().modifiers(new Document(QueryOperators.HINT, indexName));
How could we make MongoDB report errors for queries that don't use indices?
We end up creating indices for every query anyway so it would be great if MongoDB would report missing indices for us. Also it would be convenient to be able to configure the restriction on a connection basis. This way indices wouldn't come into our way when working from MongoDB shell.
The notablescan ( http://docs.mongodb.org/manual/reference/parameters/#param.notablescan ) option for the MongoDB binary (mongod.exe or mongod depending on your OS) allows you to stop any query, with an emitted log error, which does not use an index at all.
This option will not stop inefficient queries though so that part will still need to be discovered manually by you.
When performing a query in MongoDb, I need to obtain a total count of all matches, along with the documents themselves as a limited/paged subset.
I can achieve the goal with two queries, but I do not see how to do this with one query. I am hoping there is a mongo feature that is, in some sense, equivalent to SQL_CALC_FOUND_ROWS, as it seems like overkill to have to run the query twice. Any help would be great. Thanks!
EDIT: Here is Java code to do the above.
DBCursor cursor = collection.find(searchQuery).limit(10);
System.out.println("total objects = " + cursor.count());
I'm not sure which language you're using, but you can typically call a count method on the cursor that's the result of a find query and then use that same cursor to obtain the documents themselves.
It's not only overkill to run the query twice, but there is also the risk of inconsistency. The collection might change between the two queries, or each query might be routed to a different peer in a replica set, which may have different versions of the collection.
The count() function on cursors (in the MongoDB JavaScript shell) really runs another query, You can see that by typing "cursor.count" (without parentheses), so it is not better than running two queries.
In the C++ driver, cursors don't even have a "count" function. There is "itcount", but it only loops over the cursor and fetches all results, which is not what you want (for performance reasons). The Java driver also has "itcount", and there the documentation says that it should be used for testing only.
It seems there is no way to do a "find some and get total count" consistently and efficiently.