I can't find this in the documentation in any of the obvious places.
I'd like to know if it is possible to know if Mongo executed an insert or update in the upsert operation?
Yes there is, on a safe call (or getLastError) the update function will return an array with an upsert field and a updatedExisting field.
You can read the PHP version of this here: http://php.net/manual/en/mongocollection.insert.php towards the bottom.
As it says within the documentation on upserted:
If an upsert occured, this field will contain the new record's _id field. For upserts, either this field or updatedExisting will be present (unless an error occurred).
So upserted contains the _id of the new record if a insert was done or it will increment updatedExisting if it updated a record.
I am sure a similar thing appears in all drivers.
Edit
It will actually be a boolean in the updatedExisting field of true or false
For reference only, in node.js:
collection.update( source, target, { upsert: true }, function(err, result, upserted) {
...
});
For reference only, in node.js using Mongoose 3.6:
model.update( findquery, updatequery, { upsert: true }, function(err, numberAffected, rawResponse) {
...
});
Where rawResponse looks like this when it has updated an existing document:
{ updatedExisting: true,
n: 1,
connectionId: 222,
err: null,
ok: 1 }
And it looks like this when it has created a new document:
{ updatedExisting: false,
upserted: 51eebc080eb3e2208a630d8e,
n: 1,
connectionId: 222,
err: null,
(Both cases would return numberAffected = 1)
The Answer was taken from "MongoDB Applied Design Patterns" Book
!determine whether an upsert was an insert
or an update
Using MongoDB driver 3.5.9 under Node.js, I found that there are these properties we are interested in after using updateOne with { upsert: true }:
{
modifiedCount: 0,
upsertedId: null,
upsertedCount: 0,
matchedCount: 1
}
When upsert inserted something, we will get upsertedCount > 0 and upsertedId will hold the newly inserted document ID. When upsert modified something, we will get modifiedCount > 0.
The tutorial for all CRUD operations is here https://mongodb.github.io/node-mongodb-native/3.6/tutorials/crud/
Related
I want to depict the following use case using MongoDb:
I want to read from a collection and memorize that particular point in time.
When writing the next time to that collection, I want to not be able to write a new document, if another document has been added to that collection in between.
Using a timestamp property on the documents would be ok.
Is this possible?
One trick is use findAndModify
Assume at the time of reading, your most recent timestamp on a document is oldTimestamp:
db.collection.findAndModify({
query: {timestamp: {$gt: oldTimestamp}},
new: true, // Return modified / inserted document
upsert: true, // Update if match found, insert otherwise
update: {
$setOnInsert: {..your document...}
}
})
This will not insert your document if another document is inserted between your read and write operation.
However, this won't let you know that the document is inserted or not directly.
You should compare returned document with your proposed document to find that out.
In case using nodejs driver, the correct pattern should be:
collection.findAndModify(criteria[, sort[, update[, options]]], callback)
According to the example, our query should be:
db.collection('test').findAndModify(
{timestamp: {$gt: oldTimestamp}}, // query, timestamp is a property of your document, often set as the created time
[['timestamp','desc']], // sort order
{$setOnInsert: {..your document..}}, // replacement, replaces only the field "hi"
{
new: true,
upsert: true
}, // options
function(err, object) {
if (err){
console.warn(err.message); // returns error if no matching object found
}else{
console.dir(object);
}
});
});
This can be achieved, using a timestamp property in every document. You can take a look at the Mongoose Pre Save path validation hook . Using this hook, you can write something like this.
YourSchema.path('timestamp').validate(function(value, done) {
this.model(YourSchemaModelName).count({ timestamp: {$gt : value} }, function(err, count) {
if (err) {
return done(err);
}
// if count exists and not zero hence document is found with greater timestamp value
done(!count);
});
}, 'Greater timestamp already exists');
Sounds like you'll need to do some sort of optimistic locking at the collection level. I understand you are writing new documents but never updating existing ones in this collection?
You could add an index on the timestamp field, and your application would need to track the latest version of this value. Then, before attempting a new write you could lookup the latest value from the collection with a query like
db.collection.find({}, {timestamp: 1, _id:0}).sort({timestamp:-1}).limit(1)
which would project just the maximum timestamp value using a covered query which is pretty efficient.
From that point on, it's up to your application logic to handle the 'conflict'.
as the title says, I want to perform a find (one) for a document, by _id, and if doesn't exist, have it created, then whether it was found or was created, have it returned in the callback.
I don't want to update it if it exists, as I've read findAndModify does. I have seen many other questions on Stackoverflow regarding this but again, don't wish to update anything.
I am unsure if by creating (of not existing), THAT is actually the update everyone is talking about, it's all so confuzzling :(
Beginning with MongoDB 2.4, it's no longer necessary to rely on a unique index (or any other workaround) for atomic findOrCreate like operations.
This is thanks to the $setOnInsert operator new to 2.4, which allows you to specify updates which should only happen when inserting documents.
This, combined with the upsert option, means you can use findAndModify to achieve an atomic findOrCreate-like operation.
db.collection.findAndModify({
query: { _id: "some potentially existing id" },
update: {
$setOnInsert: { foo: "bar" }
},
new: true, // return new doc if one is upserted
upsert: true // insert the document if it does not exist
})
As $setOnInsert only affects documents being inserted, if an existing document is found, no modification will occur. If no document exists, it will upsert one with the specified _id, then perform the insert only set. In both cases, the document is returned.
Driver Versions > 2
Using the latest driver (> version 2), you'll use findOneAndUpdate as findAndModify was deprecated. The new method takes 3 arguments, the filter, the update object (which contains your default properties, that should be inserted for a new object), and options where you have to specify the upsert operation.
Using the promise syntax, it looks like this:
const result = await collection.findOneAndUpdate(
{ _id: new ObjectId(id) },
{
$setOnInsert: { foo: "bar" },
},
{
returnOriginal: false,
upsert: true,
}
);
const newOrUpdatedDocument = result.value;
Its a bit dirty, but you can just insert it.
Be sure that the key has a unique index on it (if you use the _id it's ok, it's already unique).
In this way if the element is already present it will return an exception that you can catch.
If it isn't present, the new document will be inserted.
Updated: a detailed explanation of this technique on the MongoDB Documentation
Here's what I did (Ruby MongoDB driver):
$db[:tags].update_one({:tag => 'flat'}, {'$set' => {:tag => 'earth' }}, { :upsert => true })}
It will update it if it exists, and insert it if it doesn't.
Considering a simple mongo document structure:
{ _id, firstTime, lastTime }
The client needs to insert a document with a known ID, or update an existing document. The 'lastTime' should always be set to some latest time. For the 'firstTime', if a document is being inserted, then the 'firstTime' should be set to current time. However, if the document is already created, then 'firstTime' remain unchanged. I would like to do it purely with upserts (to avoid look ups).
I've crawled the http://www.mongodb.org/display/DOCS/Updating, but I just don't see how that particular operation can be done.
I don't believe this is something unreasonable, there are $push and $addToSet operations that effectively do that on array fields, just nothing that would do the same on simple fields. It's like there should be something like $setIf operation.
I ran into the exact same problem and there was no simple solution for <2.4 however since 2.4 the $setOnInsert operator let's you do exactly that.
db.collection.update( <query>,
{ $setOnInsert: { "firstTime": <TIMESTAMP> } },
{ upsert: true }
)
See the 2.4 release notes of setOnInsert for more info.
I ran into a very similar problem when attempting to upsert documents based on existing content--maybe this solution will work for you also:
Try removing the _id attribute from your record and only use it in the query portion of your update (you'll have to translate from pymongo speak...)
myid = doc.get('_id')
del doc['_id']
mycollection.update({'_id':myid}, {'$set':doc}, upsert=True)
If you will trigger the following code 2 subsequent times, it will first set both firstVisit and lastVisit on document insert (and will return upsertedId in the response) and on the second it will only update lastVisit (and will return modifiedCount: 1).
Tested with Mongo 4.0.5 though I believe should be working with older versions.
db.collection.updateOne(
{_id: 1},
{
$set: {
lastVisit: Date.now()
},
$setOnInsert: {
firstVisit: Date.now()
}
},
{ upsert: true }
);
There's no way to do this with just one upsert. You'd have to do it as 2 operations - first try to insert the document, if it already exists the insert will fail due to duplicate key violation on the _id index. Then you do an update operation to set the lastTime to now.
I have a collected named foo hypothetically.
Each instance of foo has a field called lastLookedAt which is a UNIX timestamp since epoch. I'd like to be able to go through the MongoDB client and set that timestamp for all existing documents (about 20,000 of them) to the current timestamp.
What's the best way of handling this?
Regardless of the version, for your example, the <update> is:
{ $set: { lastLookedAt: Date.now() / 1000 } }
However, depending on your version of MongoDB, the query will look different. Regardless of version, the key is that the empty condition {} will match any document. In the Mongo shell, or with any MongoDB client:
$version >= 3.2:
db.foo.updateMany( {}, <update> )
{} is the condition (the empty condition matches any document)
3.2 > $version >= 2.2:
db.foo.update( {}, <update>, { multi: true } )
{} is the condition (the empty condition matches any document)
{multi: true} is the "update multiple documents" option
$version < 2.2:
db.foo.update( {}, <update>, false, true )
{} is the condition (the empty condition matches any document)
false is for the "upsert" parameter
true is for the "multi" parameter (update multiple records)
This code will be helpful for you
Model.update({
'type': "newuser"
}, {
$set: {
email: "abc#gmail.com",
phoneNumber:"0123456789"
}
}, {
multi: true
},
function(err, result) {
console.log(result);
console.log(err);
})
I have been using MongoDB .NET driver for a little over a month now. If I were to do it using .NET driver, I would use Update method on the collection object. First, I will construct a query that will get me all the documents I am interested in and do an Update on the fields I want to change. Update in Mongo only affects the first document and to update all documents resulting from the query one needs to use 'Multi' update flag. Sample code follows...
var collection = db.GetCollection("Foo");
var query = Query.GTE("No", 1); // need to construct in such a way that it will give all 20K //docs.
var update = Update.Set("timestamp", datetime.UtcNow);
collection.Update(query, update, UpdateFlags.Multi);
You can use updateMany() methods of mongodb to update multiple document
Simple query is like this
db.collection.updateMany(filter, update, options)
For more doc of uppdateMany read here
As per your requirement the update code will be like this:
User.updateMany({"created": false}, {"$set":{"created": true}});
here you need to use $set because you just want to change created from true to false. For ref. If you want to change entire doc then you don't need to use $set
I need to insert a document if it doesn't exist. I know that the "upsert" option can do that, but I have some particular needs.
First I need to create the document with its _id field only, but only if it doesn't exist already. My _id field is a number generated by me (not an ObjectId). If I use the "upsert" option then I get "Mod on _id not allowed"
db.mycollection.update({ _id: id }, { _id: id }, { upsert: true });
I know that we can't use the _id in a $set.
So, my question is: If there any way to a "create if doesn't exists" atomically in mongodb?
EDIT:
As proposed by #Barrie this works (using nodejs and mongoose):
var newUser = new User({ _id: id });
newUser.save(function (err) {
if (err && err.code === 11000) {
console.log('If duplicate key the user already exists', newTwitterUser);
return;
}
console.log('New user or err', newTwitterUser);
});
But I still wonder if it is the best way to do it.
I had the same problem, but found a better solution for my needs. You can use that same query style if you simply remove the _id attribute from the update object. So if at first you get an error with this:
db.mycollection.update({ _id: id }, {$set: { _id: id, name: 'name' }}, { upsert: true });
instead use this:
db.mycollection.update({ _id: id }, {$set: { name: 'name' }}, { upsert: true });
This is better because it works for both insert and update.
UPDATE: Upsert with _id can be done without $setOnInsert, as explaind by #Barrie above.
The trick is to use $setOnInsert:{_id:1} with upsert, that way the _id is only written to if it's an insert, and never for updates.
Only, there was a bug preventing this from working until v2.6 - I just tried it on 2.4 and it's not working.
The workaround I use is having another ID field with a unique index. Eg. $setOnInsert:{myId:1}.
You can just use insert(). If the document with the _id you specify already exists, the insert() will fail, nothing will be modified - so "create if it doesn't exist" is what it's already doing by default when you use insert() with a user-created _id.
Please note that $setOnInsert don't work easily when you upsert a simple key => value object (not $set or other).
I need to use that (in PHP):
public function update($criteria , $new_object, array $options = array()){
// In 2.6, $setOnInsert with upsert == true work with _id field
if(isset($options['upsert']) && $options['upsert']){
$firstKey = array_keys($new_object)[0];
if(strpos($firstKey, '$')===0){
$new_object['$setOnInsert']['_id'] = $this->getStringId();
}
//Even, we need to check if the object exists
else if($this->findOne($criteria, ['_id'])===null){
//In this case, we need to set the _id
$new_object['_id'] = $this->getStringId();
}
}
return parent::update($criteria, $new_object, $options);
}