For MongoDB and Mongoose, does this cause 2 queries because of the where clause? Like findAndModify does because it returns the whole document before modifying?
Model.where({ _id: id }).update({ title: 'words' })
Nope, but neither does findAndModify, as in both cases the entire command is executed atomically by the MongoDB server.
To confirm, you can see the commands that Mongoose is executing by adding the following to your code:
mongoose.set('debug', true);
Related
Context
I have a big collection with millions of documents which is constantly updated with production workload. When performing a query, I have noticed that a document can be returned multiple times; My workload tries to migrate the documents to a SQL system which is set to allow unique row ids, hence it crashes.
Problem
Because the collection is so big and lots of users are updating it after the query is started, iterating over the cursor's result may give me documents with the same id (old and updated version).
What I'v tried
const cursor = db.collection.find(query, {snapshot: true});
while (cursor.hasNext()) {
const doc = cursor.next();
// do some stuff
}
Based on old documentation for the mongo driver (I'm using nodejs but this is applicable to any official mongodb driver), there is an option called snapshot which is said to avoid what is happening to me. Sadly, the driver returns an error indicating that this option does not exists (It was deprecated).
Question
Is there a way to iterate through the documents of a collection in a safe fashion that I don't get the same document twice?
I only see a viable option with aggregation pipeline, but I want to explore other options with standard queries.
Finally I got the answer from a mongo changelog page:
MongoDB 3.6.1 deprecates the snapshot query option.
For MMAPv1, use hint() on the { _id: 1} index instead to prevent a cursor from returning a document more than once if an intervening write operation results in a move of the document.
For other storage engines, use hint() with { $natural : 1 } instead.
So, from my code example:
const cursor = db.collection.find(query).hint({$natural: 1});
while (cursor.hasNext()) {
const doc = cursor.next();
// do some stuff
}
I want to insert one document into collection1 and after successfully inserting the document, I want to insert another document into collection2. One of the fields for the document in collection2 will be the _id of the document just inserted into collection1.
I am using a callback:
db.collection1.insert(<document>,function(err,doc)){
db.collection2.insert({collection1_id: doc[0]._id, <field>:<value>})
However, it seems that callback is not available without Node.js.
Is there any workaround?
Callbacks are part of the Node.js async API and are not supported in the mongo shell (as at MongoDB 4.0). However, you can always write the equivalent without callbacks.
The mongo shell's insertOne() method will return an insertedId field with the _id value of the inserted document, so you can either save or reference this value.
For example:
db.collection2.insertOne({
collection1_id: db.collection1.insertOne({}).insertedId,
field: 'value'
})
Hello I'm new to Mongodb, I am currently trying to update the last document in a query result but having trouble doing so.
I know how to get the last document using
db.collection.find().sort({$natural:-1}).limit(1)
but how do I update this? I tried doing:
db.collection.update(db.collection.find().sort({$natural:-1}).limit(1))
But that didn't work. And I don't think:
db.collection.update(query,update,option).sort({$natural:-1}).limit(1))
would do what i want. I checked the official documentation but couldn't find anything on querying only for the last document.
You can use findAndModify to perform an update that requires sorting to identify the document to update:
db.test.findAndModify({
query: {},
sort: {$natural: -1},
update: {$set: {foo: 'bar'}}
})
You can also do it this way:
var id = db.collection.find().sort({$natural:-1}).limit(1)[0]['_id'];
db.collection.update({_id: id},{...})
I'm unable to use maxTimeMS parameter with Mongo 2.6 and Pymongo 2.7.1
As per the documentation on this page Official Mongodb Aggregation Page the aggregation method should return a Cursor object. However, when I run the query locally on a mongod instance (2.6+) with pymongo 2.7.1, I get a dict object!
In [14]: obj = coll.aggregate({'$group': {'_id': '$l', 'n': {'$sum':
1}}})
In [15]: type(obj) Out[15]: dict
Can anyone help me understand what is happening here?
Yes, you can use maxTimeMS with pymongo aggregation.
c.foo.bar.aggregate([], maxTimeMS=1000)
{u'ok': 1.0, u'result': []}
If you want a cursor:
for result in c.foo.bar.aggregate([], cursor={}, maxTimeMS=1000):
... print result
The aggregate command didn't support cursors before MongoDB 2.6 so it had to be added as an option to avoid breaking existing applications.
This is covered in the driver documentation where it is described that in order to return a cursor, you need to specify the arguments in addition to the pipeline in your .aggregate() method:
cursor = coll.aggregate([{'$group': { '_id': '$l', 'n': {'$sum': 1} }}],cursor={})
Note that the returned object here is a CommandCursor and not a cursor.
This is because various modifiers such as .limit() and .skip() and other options do not apply in the context of an aggregation result. As such $maxTimeMS is not a valid option for this type of cursor.
In addition, it would not do what you think it will even where valid. The reason being that the "cursor" execution is only counted "after" the "aggregation pipeline" execution is complete, so in this case, just fetching the results.
Look at the .currentOp() and .killOp() implementations for other ways to control long running aggregation tasks.
I know this may be a silly question, but I read on an e-book that there is an upsert option in MongoDB insert. I couldn't find proper documentation about this. Can someone educate me about this?
Since upsert is defined as operation that "creates a new document when no document matches the query criteria" there is no place for upsertsin insert command. It is an option for the update command. If you execute command like below it works as an update, if there is a document matching query, or as an insert with document described by update as an argument.
db.collection.update(query, update, {upsert: true})
MongoDB 3.2 adds replaceOne:
db.collection.replaceOne(query, replacement, {upsert: true})
which has similar behavior, but its replacement cannot contain update operators.
As in the links provided by PKD, db.collection.insert() provides no upsert possibility. Instead, mongo insert inserts a new document into a collection. Upsert is only possible using db.collection.update() and db.collection.save().
If you happen to pass a document to db.collection.insert() which is already in the collection and thus has an _id similar to an existing _id, it will throw a duplicate key exception.
For upserting a singe document using the java driver:
FindOneAndReplaceOptions replaceOptions = new FindOneAndReplaceOptions();
replaceOptions.upsert(true);
collection.findOneAndReplace(
Filters.eq("key", "value"),
document,
replaceOptions
);
Although uniqueness should be ensured from Filters.eq("key", "value") otherwise there is a possibility of adding multiple documents. See this for more