Upserts in mongodb when using custom _id values - mongodb

I need to insert a document if it doesn't exist. I know that the "upsert" option can do that, but I have some particular needs.
First I need to create the document with its _id field only, but only if it doesn't exist already. My _id field is a number generated by me (not an ObjectId). If I use the "upsert" option then I get "Mod on _id not allowed"
db.mycollection.update({ _id: id }, { _id: id }, { upsert: true });
I know that we can't use the _id in a $set.
So, my question is: If there any way to a "create if doesn't exists" atomically in mongodb?
EDIT:
As proposed by #Barrie this works (using nodejs and mongoose):
var newUser = new User({ _id: id });
newUser.save(function (err) {
if (err && err.code === 11000) {
console.log('If duplicate key the user already exists', newTwitterUser);
return;
}
console.log('New user or err', newTwitterUser);
});
But I still wonder if it is the best way to do it.

I had the same problem, but found a better solution for my needs. You can use that same query style if you simply remove the _id attribute from the update object. So if at first you get an error with this:
db.mycollection.update({ _id: id }, {$set: { _id: id, name: 'name' }}, { upsert: true });
instead use this:
db.mycollection.update({ _id: id }, {$set: { name: 'name' }}, { upsert: true });
This is better because it works for both insert and update.

UPDATE: Upsert with _id can be done without $setOnInsert, as explaind by #Barrie above.
The trick is to use $setOnInsert:{_id:1} with upsert, that way the _id is only written to if it's an insert, and never for updates.
Only, there was a bug preventing this from working until v2.6 - I just tried it on 2.4 and it's not working.
The workaround I use is having another ID field with a unique index. Eg. $setOnInsert:{myId:1}.

You can just use insert(). If the document with the _id you specify already exists, the insert() will fail, nothing will be modified - so "create if it doesn't exist" is what it's already doing by default when you use insert() with a user-created _id.

Please note that $setOnInsert don't work easily when you upsert a simple key => value object (not $set or other).
I need to use that (in PHP):
public function update($criteria , $new_object, array $options = array()){
// In 2.6, $setOnInsert with upsert == true work with _id field
if(isset($options['upsert']) && $options['upsert']){
$firstKey = array_keys($new_object)[0];
if(strpos($firstKey, '$')===0){
$new_object['$setOnInsert']['_id'] = $this->getStringId();
}
//Even, we need to check if the object exists
else if($this->findOne($criteria, ['_id'])===null){
//In this case, we need to set the _id
$new_object['_id'] = $this->getStringId();
}
}
return parent::update($criteria, $new_object, $options);
}

Related

MongoDB - how to only update field if field does not exist

How can I update a mongo document with the following requirements:
Find a document by email property:
If the document exists:
If both retrieved and new document have property A, keep property A (the retrieved one).
If retrieved document property A is null or undefined or doesn't exist, update using property A of the new object.
If the document doesn't exist
Insert the new document.
The findOneAndUpdate seems not to convey the both 3 of the requirements. Thanks.
My recommendation is to go the following path:
db.getCollection('<some-collection>').update(
{ email: 'someguy#email.com' },
{
$set: {
name: "some guy",
username: someguy,
tel: '1234'
}
},
{ upsert: true }
);
Check upsert documentation:
https://docs.mongodb.com/manual/reference/method/db.collection.update/#upsert-option
Lets go through your requirements now:
3. If the document doesn't exist, insert the new document.
Yes, it will insert new document to collection if it doesnt find the document by email. Resulting document will be combination of find condition + $set + autogenerated _id, so it will look something like this:
{
_id: ObjectId(...)
email: 'someguy#email.com'
name: "some guy",
username: someguy,
tel: '1234'
}
2. If retrieved document property A is null or undefined or doesn't exist, update using property A of the new object.
All properties provided in $set will unconditionally be persisted in the database, which also covers your requirement of updating null/undefined values
3. If both retrieved and new document have property A, keep property A (the retrieved one).
If both newly provided A and database A are the same, we dont have a problem.
If As are different, dont you want to store the new A value?
If you are afraid of nulls/undefined values, you can omit them before providing object to $set.
What is the use-case for you not wanting to update database property with newly provided value?
One use-case i can see is that you want to pass createdAt in case you are creating new record, but dont want to update that value for existing records.
If thats the case, and you know those properties in advance, you can use $setOnInsert update operator. https://docs.mongodb.com/manual/reference/operator/update/#id1
So your update query can look like this:
db.getCollection('<some-collection>').update(
{ email: 'someguy#email.com' },
{
$set: {
name: "some guy",
username: someguy,
tel: '1234'
},
$setOnInsert: {
createdAt: new Date(),
updatedAt: new Date()
}
},
{ upsert: true }
);
I hope this helps!
You need not retrieve the document for updating the property A. You can use the update API of mongo to do so. Please find the psuedo code below:
db.<collection>.update({
"$or": [
{ "PropertyA": { "$exists": false } },
{ "PropertyA": null }
]
}, {$set: {"PropertyA": "NewValue"}});
The above code is for one property, but I think you can figure out how to scale it up.
Hope this helps !!

Using MongoDB mongo go driver for counters persistent collection [duplicate]

as the title says, I want to perform a find (one) for a document, by _id, and if doesn't exist, have it created, then whether it was found or was created, have it returned in the callback.
I don't want to update it if it exists, as I've read findAndModify does. I have seen many other questions on Stackoverflow regarding this but again, don't wish to update anything.
I am unsure if by creating (of not existing), THAT is actually the update everyone is talking about, it's all so confuzzling :(
Beginning with MongoDB 2.4, it's no longer necessary to rely on a unique index (or any other workaround) for atomic findOrCreate like operations.
This is thanks to the $setOnInsert operator new to 2.4, which allows you to specify updates which should only happen when inserting documents.
This, combined with the upsert option, means you can use findAndModify to achieve an atomic findOrCreate-like operation.
db.collection.findAndModify({
query: { _id: "some potentially existing id" },
update: {
$setOnInsert: { foo: "bar" }
},
new: true, // return new doc if one is upserted
upsert: true // insert the document if it does not exist
})
As $setOnInsert only affects documents being inserted, if an existing document is found, no modification will occur. If no document exists, it will upsert one with the specified _id, then perform the insert only set. In both cases, the document is returned.
Driver Versions > 2
Using the latest driver (> version 2), you'll use findOneAndUpdate as findAndModify was deprecated. The new method takes 3 arguments, the filter, the update object (which contains your default properties, that should be inserted for a new object), and options where you have to specify the upsert operation.
Using the promise syntax, it looks like this:
const result = await collection.findOneAndUpdate(
{ _id: new ObjectId(id) },
{
$setOnInsert: { foo: "bar" },
},
{
returnOriginal: false,
upsert: true,
}
);
const newOrUpdatedDocument = result.value;
Its a bit dirty, but you can just insert it.
Be sure that the key has a unique index on it (if you use the _id it's ok, it's already unique).
In this way if the element is already present it will return an exception that you can catch.
If it isn't present, the new document will be inserted.
Updated: a detailed explanation of this technique on the MongoDB Documentation
Here's what I did (Ruby MongoDB driver):
$db[:tags].update_one({:tag => 'flat'}, {'$set' => {:tag => 'earth' }}, { :upsert => true })}
It will update it if it exists, and insert it if it doesn't.

mongodb upsert with condition

is there a way with mongodb to bulk upsert but choose what to do when inserting and when updating.
for example:
I have this document in the mongodb:
{name:"david", last_name:"family_name", published:true}
now arrives a new document that is like this:
{name:"david", last_name:"family_name_changed", published:false}
now I want to replace the document that is in the DB (document 1)
that will only update the last_name field (or any other field) but I dont want it to update the published field. (ever)
what is the way to do that?
thank you
You can use $set to update a specific field, for instance:
db.<name of your collection>.update(<query>,
{
$set:
{
last_name: <value to set>
}
}, { multi: true } );

Upsert instead of save mongodb

I have two collections(so and temp_so).
My query:
db.getCollection('temp_sale_order').find({},'orderItemId').
forEach(function(element) { db.getCollection('sale_order').save(element); });
i have a document with orderItemId 123 in temp_so. I want to check if a document with orderItemId exist in so then the query should update the whole document in so. If the document with orderItemId does not exist in so, it should create one.
The query should do this for all the documents in temp_so.
My current query just insert the data in so and doesn't check if the document with orderItemId exists or not in so. How to rectify it?
In SQL this can be written as:
insert into so ( select * from temp_so where orderItemId not in ( select orderItemId from so));
Update so set col = select col from temp_so where temp_so.orderItemId = so.orderItemId.
*update for all the columns*
Technically speaking the .save() action is already a "wrapper" around an "upsert", it's just only ever looking at the _id field only, and where present.
So you just need to be explicit about which field you are looking up. And also removing the existing _id:
db.getCollection('temp_sale_order').find({},{ 'orderItemId': 1 }).
forEach(function(element) {
delete element._id;
db.getCollection('sale_order').update(
{ "orderItemId": element.orderItemId },
element,
{ "upsert": true }
);
});
You might also consider that you don't want to replace the whole document and would rather just "insert" the first occurance only. Which is what $setOnInsert does here:
db.getCollection('temp_sale_order').find({},{ 'orderItemId': 1}).
forEach(function(element) {
delete element._id;
db.getCollection('sale_order').update(
{ "orderItemId": element.orderItemId },
{ "$setOnInsert": { "orderItemid": element.orderItemId } },
{ "upsert": true }
);
});
So that basically means that any alteration within $setOnInsert will not actually be applied unless this is actually an "upsert" and a new document is created. If it is just a "match" then nothing in the document gets changed.

MongoDB: update every document on one field

I have a collected named foo hypothetically.
Each instance of foo has a field called lastLookedAt which is a UNIX timestamp since epoch. I'd like to be able to go through the MongoDB client and set that timestamp for all existing documents (about 20,000 of them) to the current timestamp.
What's the best way of handling this?
Regardless of the version, for your example, the <update> is:
{ $set: { lastLookedAt: Date.now() / 1000 } }
However, depending on your version of MongoDB, the query will look different. Regardless of version, the key is that the empty condition {} will match any document. In the Mongo shell, or with any MongoDB client:
$version >= 3.2:
db.foo.updateMany( {}, <update> )
{} is the condition (the empty condition matches any document)
3.2 > $version >= 2.2:
db.foo.update( {}, <update>, { multi: true } )
{} is the condition (the empty condition matches any document)
{multi: true} is the "update multiple documents" option
$version < 2.2:
db.foo.update( {}, <update>, false, true )
{} is the condition (the empty condition matches any document)
false is for the "upsert" parameter
true is for the "multi" parameter (update multiple records)
This code will be helpful for you
Model.update({
'type': "newuser"
}, {
$set: {
email: "abc#gmail.com",
phoneNumber:"0123456789"
}
}, {
multi: true
},
function(err, result) {
console.log(result);
console.log(err);
})
I have been using MongoDB .NET driver for a little over a month now. If I were to do it using .NET driver, I would use Update method on the collection object. First, I will construct a query that will get me all the documents I am interested in and do an Update on the fields I want to change. Update in Mongo only affects the first document and to update all documents resulting from the query one needs to use 'Multi' update flag. Sample code follows...
var collection = db.GetCollection("Foo");
var query = Query.GTE("No", 1); // need to construct in such a way that it will give all 20K //docs.
var update = Update.Set("timestamp", datetime.UtcNow);
collection.Update(query, update, UpdateFlags.Multi);
You can use updateMany() methods of mongodb to update multiple document
Simple query is like this
db.collection.updateMany(filter, update, options)
For more doc of uppdateMany read here
As per your requirement the update code will be like this:
User.updateMany({"created": false}, {"$set":{"created": true}});
here you need to use $set because you just want to change created from true to false. For ref. If you want to change entire doc then you don't need to use $set