Mongoose/MongoDB throwing duplicate key error on save? - mongodb

According to MongoDB's documentation a call to save will create a new document, or update an existing document if _id is provided. Mongoose's documentation is less detailed and does not go into whether it will insert or update.
I am attempting to use Mongoose's save function to update a document, but I keep getting the error:
{"error":{"name":"MongoError","code":11000,"err":"insertDocument ::
caused by :: 11000 E11000 duplicate key error index:
staging.participants.$_id _ dup key: { :
ObjectId('5515a34ed65073ec234b5c5f') }"}}
Does Mongoose's save function perform an upsert like MongoDB's save function or is it just performing an insert?

What defines whether the save will be an insert or an update is the isNew flag, as you can see here.
This flag is set automatically to false when the document instance is returned from a find query (or any of its variations). If you are instantiating the document manually, try setting this flag to false before saving it:
var instance = new Model({ '_id': '...', field: '...' });
instance.isNew = false;
instance.save(function(err) { /* ... */ });
There is also an init function, that will initialize the document and automatically set isNew to false:
var data = { '_id': '...', field: '...' };
var instance = new Model();
instance.init(data, {}, function(err) {
instance.save(function(err) { /* ... */ })
});

Related

Mongoose: atomic FindOne-Or-Insert(), do not update existing instance if found

In Mongoose, I am seeking to perform atomically a way to Model.FindOne-Or-Insert(), similar functionality and signature as currently available Model.FindOneAndUpdate() except if an instance exists (i.e. matches filter) then do not update using provided object but return instance found as is, and if not exists (i.e. no match with filter) then insert object and return new instance.
I could not find a way using Model.FindOneAndUpdate() not to perform an update to an existing instance by trying out variances to its options and not providing fields to object that preferred not to update if instance exists.
So, my current non-atomic workaround is Model.FindOne() and if not found then perform Document.save().
const Foo = DB.model('foo', FooSchema)
async function findOneAndUpdateFoo(jsonFoo, next) {
const filter = {
deletedAt: null
}
if (jsonFoo.dsAccountId) {
filter.dsAccountId = jsonFoo.dsAccountId
}
if (jsonIntegration.dsUserId) {
filter.dsUserId = jsonIntegration.dsUserId
}
if (jsonFoo.providerId) {
filter.providerId = jsonFoo.providerId
}
const fooDoc = {
name: jsonFoo.name,
dsAccountId: jsonFoo.dsAccountId,
dsUserId: jsonFoo.dsUserId,
providerId: jsonFoo.providerId,
providerName: jsonFoo.providerName,
// Most of these fields could be empty
accessToken: jsonFoo.accessToken,
refreshToken: jsonFoo.refreshToken,
scope: jsonFoo.scope,
tokenType: jsonFoo.tokenType,
expiresAt: jsonFoo.expiresAt
}
return await Foo.findOneAndUpdate(
filter, // find a document with that filter
fooDoc, // document to insert when nothing was found
{ upsert: true, new: true, runValidators: true } // options
)
.catch(next)
}
Suggestions? Thank you
You can use $setOnInsert in your update parameter so that it will only apply in the insert case; with the update becoming a no-op in the case where the document already exists:
return await Foo.findOneAndUpdate(
filter, // find a document with that filter
{$setOnInsert: fooDoc}, // document to insert when nothing was found
{ upsert: true, new: true, runValidators: true }
)
Note that you should also create a unique index over the fields included in your filter and then handle the possibility of a duplicate error. See this post for the details why.

FindAndModify: get newDocument along with info whether doc was inserted or updated

Using findAndModify:
I need to get the new resulting document
I need to know if an insert or update was done
var newUpdate = {
$set : newData,
$setOnInsert: {created_at: new Date()}
};
var options = {
upsert :true,
new: true,
};
collectionDriver.findAndModify(colName, query, newUpdate, options, function(err,resultDoc)
{
if (err) {
} else {
}
});
I get the new document, how can I know if an insert/update was happened?
One possible, though not the most efficient way, can be to check for presence of record before issuing the modify command because if you need the new record, then you cannot check if it previously existed.

Got duplicate key error dup key: { : undefined }

I have an array field called udids in Meteor.users schema, which should contains unique elements. This is how I defined the index using SimpleSchema and Collection2:
new SimpleSchema({
...
udids: {
type: Array,
index: true,
unique: true,
optional: true,
sparse: true,
},
'udids.$': {
type: String,
},
...
})
However, when I start the app, I got this error: E11000 duplicate key error collection: meteor.users index: c2_udids dup key: { : undefined }.
I tried searching for the documents with udids = undefined in the database: db.users.find({ udids: { $type: 6 } }) ($type: 6 for undefined value) but it returns nothing.
The error message is a bit unclear so I had to guess the reason why. I found out that the current database already has some users with udids = []. I'm writing a migration script to unset this field from those users. Hopefully this will help others who have the same problem as me.
I've not tested this, but it should ideally work.
Used Meteor.users as a collection name. You may want to replace it with whichever collection you want to run the validation against.
Made use of custom function to find at least one doc which contains the field's value in udids.
If you don't have access to the collection on the client side, then you can edit the custom function and have it handled asynchronously.
new SimpleSchema({
...
'udids': {
optional: true,
type: [String], // an array of string eg. ['A','B']
custom: function() {
// this.value = current field's value
// Below mongo query will check to see if there is at least one
// mongo doc whose udids contains the currently entered field's value.
// if found, then return the error message.
if (Meteor.users.findOne({
udids: {
$in: [this.value]
}
})) {
return "duplicateMsg";
}
},
...
});
SimpleSchema.messages({ duplicateMsg:"Udid already exists"});

Lokijs: Inserting document with unique key violation

I am running the latest, as of 5.3.2016, minified Lokijs from Lokijs.org and NW.js v0.12.3-win-x64. I have a document already saved in Lokijs:
"collections":[{"name":"admins","data":[{"username":"erik","meta":{"revision":1,"created":1459028934981,"version":0,"updated":1462333795190},"$loki":1}],"idIndex":[1],"binaryIndices":{},"constraints":null,"uniqueNames":["username"],"transforms":{},"objType":"admins","dirty":true,"cachedIndex":null,"cachedBinaryIndex":null,"cachedData":null,"transactional":false,"cloneObjects":false,"cloneMethod":"parse-stringify","asyncListeners":false,"disableChangesApi":true,"autoupdate":false,"ttl":{"age":null,"ttlInterval":null,"daemon":null},"maxId":2,"DynamicViews":[],"events":{"insert":[null],"update":[null],"pre-insert":[],"pre-update":[],"close":[],"flushbuffer":[],"error":[null],"delete":[null],"warning":[null]},"changes":[],"username":{"name":"username","regExp":{}}}.
I am trying to generate an error when I attempt to insert a duplicate key value. I have added a unique constraint on the 'username' key in this collection and have verified the collection.uniqueNames array contains 'username'.
When I run the code below, as expected, no additional documents are inserted into the collection.data array and the database is saved. However, no errors are generated. Also, when I console.log the document object after the insert method has run, it changes to:
Object {username: "erik", meta: Object, $loki: 2}.
When I change the key value to something else, the unique document is then inserted and saved properly.
How do I go about generating an error when attempting to insert a document containing key(s) that violate unique constraints? Thank you.
insertDocument: function(objParameters) {
var collection = objParameters.insert.collection;
collection.ensureUniqueIndex('username');
var document = {username: ''};
document.username = 'erik';
collection.on('error', function(err) {
return console.log(err);
});
collection.insert(document);
return thisModule.$body.triggerHandler('app.database.save');
}
EDIT: loki.db to test clone
{"filename":"loki.db","collections":[{"name":"test","data":[{"name":"erik","meta":{"revision":0,"created":1462493328062,"version":0},"$loki":1}],"idIndex":[1],"binaryIndices":{},"constraints":null,"uniqueNames":["name"],"transforms":{},"objType":"test","dirty":true,"cachedIndex":null,"cachedBinaryIndex":null,"cachedData":null,"transactional":false,"cloneObjects":true,"cloneMethod":"parse-stringify","asyncListeners":false,"disableChangesApi":true,"autoupdate":false,"ttl":{"age":null,"ttlInterval":null,"daemon":null},"maxId":2,"DynamicViews":[],"events":{"insert":[null],"update":[null],"pre-insert":[],"pre-update":[],"close":[],"flushbuffer":[],"error":[null],"delete":[null],"warning":[null]},"changes":[]}],"databaseVersion":1.1,"engineVersion":1.1,"autosave":false,"autosaveInterval":5000,"autosaveHandle":null,"options":{},"persistenceMethod":"fs","persistenceAdapter":null,"verbose":false,"events":{"init":[null],"loaded":[],"flushChanges":[],"close":[],"changes":[],"warning":[]},"ENV":"NODEJS"}
Code to test clone:
var loki = require('lokijs-1.3.min.js');
var db = new loki();
var collection = db.addCollection('test', {
clone: true,
unique: 'name'
});
collection.on('error', function(error) {
return console.log(error);
});
collection.insert({ name: 'erik'});
collection.insert({ name: 'erik'});
db.saveDatabase();
If you don't use clone: true then you need to call coll.update(document) to force index recomputation, which will trigger the error.

Invoke db.eval in FindAndModify using MongoDB C# Client

I have the following Document:
{
"_id": 100,
"Version": 1,
"Data": "Hello"
}
I have a function which return a number from a sequence:
function getNextSequence(name) {
var ret = db.Counter.findAndModify(
{
query: { _id: name },
update: { $inc: { value: 1 } },
new: true,
upsert: true
}
);
return ret.value;
}
I can use this for optimistic concurrency by performing the following Mongo command:
db.CollectionName.findAndModify({
query: { "_id" : NumberLong(100), "Version" : 1 },
update: { "$set" : {
"Data": "Here is new data!",
"Version" : db.eval('getNextSequence("CollectionName")') }
},
new: true
}
);
This will update the document (as the _id and Version) match, with the new Data field, and also the new number out of the eval call.
It also returns a modified document, from which I can retrieve the new Version value if I want to make another update later (in the same 'session').
My problem is:
You cannot create an Update document using the MongoDB C# client that will serialize to this command.
I used:
var update = Update.Combine(
new UpdateDocument("$set", doc),
Update.Set(versionMap.ElementName, new BsonJavaScript("db.eval('getNextSequence(\"Version:CollectionName\")')")))
);
If you use what I first expected to perform this task, BsonJavascript, you get the following document, which incorrectly sets Version to a string of javascript.
update: { "$set" : {
"Data": "Here is new data!",
"Version" : { "$code" : "db.eval('getNextSequence(\"Version:CollectionName\")')" }
}
}
How can I get MongoDB C# client to serialize an Update document with my db.eval function call in it?
I have tried to add a new BsonValue type in my assembly which I would serialize down to db.eval(''); However there is a BsonType enum which I cannot modify, without making a mod to MongoDB which I would not like to do incase of any issues with the change, compatibility etc.
I have also tried simply creating the Update document myself as a BsonDocument, however FindAndModify will only accept an IMongoUpdate interface which a simply a marker that at present I find superfluous.
I have just tried to construct the command manually by creating a BsonDocument myself to set the Value: db.eval, however I get the following exception:
A String value cannot be written to the root level of a BSON document.
I see no other way now than drop down to the Mongo stream level to accomplish this.
So I gave up with trying to get Mongo C# Client to do what I needed and instead wrote the following MongoDB function to do this for me:
db.system.js.save(
{
_id : "optimisticFindAndModify" ,
value : function optimisticFindAndModify(collectionName, operationArgs) {
var collection = db.getCollection(collectionName);
var ret = collection.findAndModify(operationArgs);
return ret;
}
}
);
This will get the collection to operate over, and execute the passed operationArgs in a FindAndModify operation.
Because I could not get the shell to set a literal value (ie, not a "quoted string") on a javascript object, I had to to this in my C# code:
var counterName = "Version:" + CollectionName;
var sequenceJs = string.Format("getNextSequence(\"{0}\")", counterName);
var doc = entity.ToBsonDocument();
doc.Remove("_id");
doc.Remove(versionMap.ElementName);
doc.Add(versionMap.ElementName, "SEQUENCEJS");
var findAndModifyDocument = new BsonDocument
{
{"query", query.ToBsonDocument()},
{"update", doc},
{"new", true},
{"fields", Fields.Include(versionMap.ElementName).ToBsonDocument() }
};
// We have to strip the quotes from getNextSequence.
var findAndModifyArgs = findAndModifyDocument.ToString();
findAndModifyArgs = findAndModifyArgs.Replace("\"SEQUENCEJS\"", sequenceJs);
var evalCommand = string.Format("db.eval('optimisticFindAndModify(\"{0}\", {1})');", CollectionName, findAndModifyArgs);
var modifiedDocument = Database.Eval(new EvalArgs
{
Code = new BsonJavaScript(evalCommand)
});
The result of this is that I can now call my Sequence Javascript, the getNextSequence function, inside the optimisticFindAndModify function.
Unforunately I had to use a string replace in C# as again there is no way of setting a BsonDocument to use the literal type db.eval necessary, although Mongo Shell likes it just fine.
All is now working.
EDIT:
Although, if you really want to push boundaries, and are actually awake, you will realize this same action can be accomplished by performing an $inc on the Version field.... and none of this is necessary....
However: If you want to follow along to the MongoDB tutorial on how they to say to implement concurrency, or you just want to use a function in a FindAndModify, this will help you. I know I'll probably refer back to it a few times in this project!