I'm running into an issue with Cosmos DB where the behavior of a query with {upsert: true} and $setOnInsert where the insert values are applied every time regardless of if the operation was an insert or an update.
The results of the following example query when ran against Cosmos DB and MongoDB show a difference in the final value of defaultQty.
db.products.remove({})
// WriteResult({ "nRemoved" : 1 })
db.products.insert({ _id: 1, item: "apple", price: 0.05, defaultQty: 50})
// WriteResult({ "nInserted" : 1 })
db.products.find({})
// { "_id" : 1, "item" : "apple", "price" : 0.05, "defaultQty" : 50 }
sleep(100)
db.products.update(
{ _id: 1 },
{ $set: { price: 0.10 }, $setOnInsert: { defaultQty: 100 }},
{ upsert: true }
)
// WriteResult({ "nMatched" : 1, "nUpserted" : 0, "nModified" : 1 })
db.products.find({})
// { "_id" : 1, "item" : "apple", "price" : 0.1, "defaultQty" : 100 }
Here is a screen shot of the comparison results side-by-side in Studio 3T.
Has anyone experienced this?
Thanks!
This issue is now fixed awaiting deployment. You can track progress here https://feedback.azure.com/forums/599059-azure-cosmos-db-mongodb-api/suggestions/20017141-bug-fix-during-upsert-operation-setoninsert-is-b
We Will post update once deployment is completed.
thanks!
Yes, this is a known issue that will be fixed shortly in Azure Cosmos DB.
Related
After scouring the documentation and posts online, there is one thing I have never been clear about with Mongo.
When you are attempting to write documents in bulk to a collection like the example below, is it ever possible that you would get some documents that write successfully, but some that don't?
db.products.insertMany( [
{ item: "card", qty: 15 },
{ item: "envelope", qty: 20 },
{ item: "stamps" , qty: 30 }
] );
In other words, could you ever get into a situation where you would create documents for the card and envelope items, but not for the stamps item?
I am trying to improve the performance of some of my company's processes and there is some debate in my team as to what kind of error scenarios can really arise from bulk inserts or updates, so if anyone has a clear answer, that would be fantastic. I know that generally mongo queries are not transactional unless you explicitly state so, but this is one area where it just wasn't clear.
Have a look at this example:
db.products.insertMany([
{ _id: 1, item: "card", qty: 15 },
{ _id: 2, item: "envelope", qty: 20 },
{ _id: 1, item: "stamps", qty: 30 }
]);
uncaught exception: BulkWriteError({
"writeErrors" : [
{
"index" : 2,
"code" : 11000,
"errmsg" : "E11000 duplicate key error collection: so.products index: _id_ dup key: { _id: 1.0 }",
"op" : {
"_id" : 1,
"item" : "stamps",
"qty" : 30
}
},
],
"writeConcernErrors" : [ ],
"nInserted" : 2,
"nUpserted" : 0,
"nMatched" : 0,
"nModified" : 0,
"nRemoved" : 0,
"upserted" : [ ]
}) :
Behavior should be obvious. Note, by default the documents are inserted in the same order as in your command, unless you specify option ordered: false
i have question
{
"_id" : ObjectId("6167c0c907fe8867d48970ce"),
"userMappingId" : "6167bf7307fe8867d48970cd",
"audioUrl" : "abcs2.com/kou.mp3",
"audioDuration" : 11,
"dayId" : 4,
"periodId" : "61652f30919d4616107a4896",
"ctime" : ISODate("2021-10-14T05:31:53.569Z"),
"_class" : "org.kou.xixi"
}
if i want to change ctime how i read the code ?
i have use this
> db.xabs.updateOne({ "_id" : "6167c0c907fe8867d48970ce" },{$set:{"ctime" : "2021-10-13T05:31:53.569Z"})
or use
db.xabs.updateOne({ "userMappingId" : "6167bf7307fe8867d48970cd" },{$set:{"ctime" : "2021-10-13T05:31:53.569Z"})
but my data still cannot change to the new one
Change your _id into ObjectId.
_id: ObjectId("6167c0c907fe8867d48970ce")
db.collection.update({
"_id": ObjectId("6167c0c907fe8867d48970ce")
},
{
$set: {
"ctime": "2021-10-13T05:31:53.569Z"
}
})
As I can see you are finding the record with the id you need to cast the string to ObjectId and second you need to also cast the timestamp using ISODate like following:
db.xabs.updateOne(
{
_id: ObjectId("6167c0c907fe8867d48970ce")
},
{
$set: {
ctime: ISODate("2021-10-13T05:31:53.569Z")
}
}
);
db. xabs.update({ "_id": new ObjectId("6167c0c907fe8867d48970ce")},{ $set: {"ctime": ISODate("2021-10-13T05:31:53.569Z")}});
WriteResult({ "nMatched" : 1, "nUpserted" : 0, "nModified" : 1 })
after i use this code i can run it... it is from documentation for mongoDb Shell 2.6.10
thanks for you all for help me for this issue
Trying to understand why/if mongoose is updating my documents even though no data is changed?
If I save a new document with the query below it will return this in the console.log(item)
{ n: 1,
nModified: 0,
upserted: [ { index: 0, _id: 5f3d35c386aeb6c6fb35fa79 } ],
ok: 1 }
Query
Product.updateOne(
{productName: product.productName},
{$set: newProduct},
{upsert: true}
).then((item) => {
console.log(item);
}).catch((e) => {
console.log('Insert error', e);
});
If i rerun the same query again i get this back. This indicates that the document has been modified but the data is the same, there is no new data that has been inserted.
{ n: 1, nModified: 1, ok: 1 }
I've noticed if i remove the stores array, delete the document, insert it again and rerun the query I get { n: 1, nModified: 0, ok: 1 } back in the console.log(item)
I run the same querys, the same amout of time, but when having an array in the object i get this { n: 1, nModified: 1, ok: 1 } and when not having an array a get this { n: 1, nModified: 0, ok: 1 }
It seems that when having an array the document gets modified regardless if the data is changed.
Example 1
Gives { n: 1, nModified: 1, ok: 1 }
const newProduct = {
ean: product.ean,
productName: product.productName,
lowestPrice: product.productPrice,
mainCategory: categories.mainCategory,
group: categories.group,
subCategory: categories.subCategory,
subSubCategory: subSubCat,
stores: [{
name: "foobar",
}],
};
Example 2
Gives { n: 1, nModified: 0, ok: 1 }
const newProduct = {
ean: product.ean,
productName: product.productName,
lowestPrice: product.productPrice,
mainCategory: categories.mainCategory,
group: categories.group,
subCategory: categories.subCategory,
subSubCategory: subSubCat,
};
Is it me who misunderstands the operation below or whats going on?
What i want to do is:
1.insert if the document don't exists based on productName,
2.if something differs in the document stored in the database and the newProduct, update the document.
3.If nothing differs, do nothing
Product model
const ProductSchema = new Schema({
ean: String,
productName: String,
mainCategory: String,
subCategory: String,
group: String,
subSubCategory: String,
lowestPrice: Number,
isPopular: Boolean,
description: String,
stores: [
{
name: String,
},
],
});
Edit: As its pretty hard to explain I created a small repo that shows the issue.
https://github.com/gameatrix/mongo_array
The database still performs the update.
For example, let's conditionally upsert a value:
MongoDB Enterprise ruby-driver-rs:PRIMARY> db.foo.update({a:42},{a:42},{upsert:true})
WriteResult({
"nMatched" : 0,
"nUpserted" : 1,
"nModified" : 0,
"_id" : ObjectId("5f3d4ed509fcd40c9f092690")
})
MongoDB Enterprise ruby-driver-rs:PRIMARY> db.foo.update({a:42},{a:42},{upsert:true})
WriteResult({ "nMatched" : 1, "nUpserted" : 0, "nModified" : 1 })
The first write was an insert, the second write was an update. The second write did not change any data but the database performed a write.
You can verify there was a write by using a change stream in another shell instance:
MongoDB Enterprise ruby-driver-rs:PRIMARY> db.foo.watch()
{ "_id" : { "_data" : "825F3D4ED5000000012B022C0100296E5A1004FAC29486D5A3459A8726349007F2E43E46645F696400645F3D4ED509FCD40C9F0926900004" }, "operationType" : "insert", "clusterTime" : Timestamp(1597853397, 1), "fullDocument" : { "_id" : ObjectId("5f3d4ed509fcd40c9f092690"), "a" : 42 }, "ns" : { "db" : "test", "coll" : "foo" }, "documentKey" : { "_id" : ObjectId("5f3d4ed509fcd40c9f092690") } }
{ "_id" : { "_data" : "825F3D4ED6000000012B022C0100296E5A1004FAC29486D5A3459A8726349007F2E43E46645F696400645F3D4ED509FCD40C9F0926900004" }, "operationType" : "replace", "clusterTime" : Timestamp(1597853398, 1), "fullDocument" : { "_id" : ObjectId("5f3d4ed509fcd40c9f092690"), "a" : 42 }, "ns" : { "db" : "test", "coll" : "foo" }, "documentKey" : { "_id" : ObjectId("5f3d4ed509fcd40c9f092690") } }
By definition an upsert either modifies documents that match the condition or inserts new documents. You are always going to have a write when upserting.
2.if something differs in the document stored in the database and the newProduct, update the document.
The bolded part is not how MongoDB (and most databases, as far as I know) work. Whether a write is performed does not depend on whether the data being written is the same as what is already in the database.
I am trying to update a collection value with _id 5af5968e17abd4901f4ecfdb.
Query executed:
db.genericcrosssell.config.update(
{_id: '5af5968e17abd4901f4ecfdb'},
{$set: {
"HOTEL_ISSUANCE_EMAIL:{"USE_QUOTATION":1,"CAR_RENTAL":0},
"HOTEL_MY_BOOKING":{"USE_QUOTATION":1,"CAR_RENTAL":0},
"HOTEL_PUSH_NOTIFICATION":{"USE_QUOTATION":0,"CAR_RENTAL":0},
"HOTEL_ISSUING_TRANSITION":{"USE_QUOTATION":0,"CAR_RENTAL":0},
"HOTEL_UPCOMING_TRIP":{"USE_QUOTATION":1,"CAR_RENTAL":0}
}}
)
Getting output as:
WriteResult({ "nMatched" : 0, "nUpserted" : 0, "nModified" : 0 })
On executing following query, I am getting response.
db.genericcrosssell.config.find().pretty()
Output:
{
"_id" : ObjectId("5af5968e17abd4901f4ecfdb"),
"FLIGHT_ISSUANCE_EMAIL" : {
"USE_QUOTATION" : 1
},
"HOTEL_ISSUANCE_EMAIL" : {
"USE_QUOTATION" : 1,
"CAR_RENTAL" : 100
}
}
Quoted only 2 values here.
You must pass objectid while passing the value:
Try this:
db.genericcrosssell.config.update({_id: ObjectId('5af5968e17abd4901f4ecfdb')},
{$set: {"HOTEL_ISSUANCE_EMAIL":{"USE_QUOTATION":1,"CAR_RENTAL":0},"HOTEL_MY_BOOKING":{"USE_QUOTATION":1,"CAR_RENTAL":0},"HOTEL_PUSH_NOTIFICATION":{"USE_QUOTATION":0,"CAR_RENTAL":0},"HOTEL_ISSUING_TRANSITION":{"USE_QUOTATION":0,"CAR_RENTAL":0},"HOTEL_UPCOMING_TRIP":{"USE_QUOTATION":1,"CAR_RENTAL":0}}})
I need to ensure combination of some fields to be unique , so i found out we can use indexes to achieve it , like :
db.user.ensureIndex({'fname':1,'lname':1},{unique:true})
However in my scenario i allow the user to add email or phone or even both . So how do i achieve the uniqueness in this case . I tried like setting 2 indexed like
db.user.ensureIndex({'fname':1,'lname':1,'email':1},{unique:true})
And
db.user.ensureIndex({'fname':1,'lname':1,'mobile':1},{unique:true})
this dint work when i try to add a document with email unique , like below
> db.user.insert({fname:'tony',lname:'stark',email:'aa#gmail.com'})
WriteResult({ "nInserted" : 1 })
> db.user.insert({fname:'tony',lname:'stark',email:'xyz#gmail.com'})
WriteResult({
"nInserted" : 0,
"writeError" : {
"code" : 11000,
"errmsg" : "E11000 duplicate key error index: temp.user.$fname_1_lname_1_mobile_1 dup key: { : \"tony\", : \"stark\", : null }"
}
})
I guess the error is because of the second index ,since both the inserts have null value for the mobile field . I then tried this
db.user.ensureIndex({'fname':1,'lname':1,'email':1,'mobile':1},{unique:true})
And that dint help ,is there any way to do this with mongodb or check the uniqueness with code otherwise .
Update
I need to ensure uniqueness in this case too :
db.user.insert({fname:'tony',lname:'stark',email:'abc#gmail.com'})
<<insert>>
db.user.insert({fname:'tony',lname:'stark',email:'abc#gmail.com',mobile:554545435})
<<dont insert>>
Update 2
Even Unique partial Index didn't help
> db.users.createIndex({ "fname": 1, "lname": 1, "email":1 },
... { unique: true, partialFilterExpression:{
... email: { $exists: true, $gt : { $type : 10 } } } })
{
"createdCollectionAutomatically" : true,
"numIndexesBefore" : 1,
"numIndexesAfter" : 2,
"ok" : 1
}
> db.users.createIndex({ "fname": 1, "lname": 1, "mobile":1 },
... { unique: true, partialFilterExpression:{
... mobile: { $exists: true, $gt : { $type : 10 } } } })
{
"createdCollectionAutomatically" : false,
"numIndexesBefore" : 2,
"numIndexesAfter" : 3,
"ok" : 1
}
Here is the query :
>db.users.insert({fname:"tony",lname:"stark",mobile:"123",email:"123#gmail.com"})
WriteResult({ "nInserted" : 1 })
>db.users.insert({fname:"tony",lname:"stark",mobile:"123",email:"123#gmail.com"})
WriteResult({ "nInserted" : 1 })
Mongo version - v3.2.0
I have no experience with mongodb much neither have used indexes in other db's , i guess it's kind of noob question .Thanks in advance