Bulk insert.update in mongodb - mongodb

Currently in my MongoDB collection I have this data
currentDataList = [{'id':1,'name':test1},{'id':2,'name':test2}]
but now I have a new data list with huge amount of data like
newDataList = [{'id':1,'name':a1},{'id':2,'name':a2},{'id':3,'name':a3}.....{'id':9999,'name':a9999}]
and I need to update this newDataList in my collection with insert if not exist/update if exist manner.
I do not want to update the documents one by one because it will take time.
Can anyone please suggest some solution for bulk insert/update in this case.

Update in bulk:
If you want to update in bulk you can use db.runCommand() with update command.
Command example:
db.runCommand({
update: "collectionName",
updates: [
{
q: { id: 3 },
u: { $set: { id: 3, name: "a3", newFields: "newField" } },
multi: true,
upsert: true
}, // the list goes here
{
q: { id: 11 },
u: { $set: { id: 11, name: "a11" } },
multi: true,
upsert: true
}
],
ordered: false,
writeConcern: { w: "majority", wtimeout: 5000 }
});
Update one by one:
Alternately you can run the query for each item in the list using the .update() method with upsert set to true.
mongodb playground example
operation example:
db.collection.update({
id: 3
},
{
$set: {
id: 3,
name: "a3",
newField: "newField",
}
},
{
upsert: true,
multi: true
})

Related

MongoDB - Update the value of one field with the value of another nested field

I am trying to run a MongoDB query to update the value of one field with the value of another nested field. I have the following document:
{
"name": "name",
"address": "address",
"times": 10,
"snapshots": [
{
"dayTotal": 2,
"dayHit": 2,
"dayIndex": 2
},
{
"dayTotal": 3,
"dayHit": 3,
"dayIndex": 3
}
]
}
I am trying like this:
db.netGraphMetadataDTO.updateMany(
{ },
[{ $set: { times: "$snapshots.$[elem].dayTotal" } }],
{
arrayFilters: [{"elem.dayIndex":{"$eq": 2}}],
upsert: false,
multi: true
}
);
but got an error:
arrayFilters may not be specified for pipeline-syle updates
You can't use arrayFilters with aggregation pipeline for update query at the same time.
Instead, what you need to do:
Get the dayTotal field from the result 2.
Take the first matched document from the result 3.
Filter the document from snapshots array.
db.netGraphMetadataDTO.updateMany({},
[
{
$set: {
times: {
$getField: {
field: "dayTotal",
input: {
$first: {
$filter: {
input: "$snapshots",
cond: {
$eq: [
"$$this.dayIndex",
2
]
}
}
}
}
}
}
}
}
],
{
upsert: false,
multi: true
})
Demo # Mongo Playground

MongoDB $inc value by 1 and set max

i have a question, I need to increment a field in DB by 1, thats fine i know how to do it, but i need to set max to 5, i mean it like if value in DB is 5, dont update value to 6, i have this code:
collectionName.update({
name: { $in: namesField },
}, { $inc: { fieldName: 1 } }, { multi: true });
i tried this one and many others, but does not work:
collectionName.update({
name: { $in: namesField },
}, { fieldName: { $lt: 6 }, { $inc: { fieldName: 1 } }, { multi: true });
Thank you for every idea
It is just a matter of finding only relevant documents, i.e. with fieldName smaller than 5, and then update only them, as you tried. But the finding part should be on the first argument, since the update syntax expecting query, update, options, as can be seen here
collectionName.update({
name: { $in: namesField }, fieldName: { $lt: 5 }}, { $inc: { fieldName: 1 } }, { multi: true });

Update Multiple Sub Doc By array of sub doc _id's in mongodb

I am trying to update multiple sub documents by given array of sub documents id's. I tried multiple approaches but it's not working.
In my scenario i need to update multiple sub documents by given array of id's. Here is my query as below:
Approach 1. (No elements were updating)
var updated = await ModelName.update(
{
'subDocArray._id' : { $in: req.body.elementId }
},
{
$set: {
'subDocArray.$[elem].abc': req.body.abcValue,
'subDocArray.$[elem].xyz': req.body.xyzValue
},
},{ "arrayFilters": [{ "elem._id": { $in: req.body.elementId } }], "multi": true, "upsert": true }
).lean().exec();
Approach 2: (Only First occurred element is updating)
var updated = await ModelName.update(
{
'subDocArray._id' : { $in: req.body.elementId }
},
{
$set: {
'subDocArray.$.abc': req.body.abcValue,
'subDocArray.$.xyz': req.body.xyzValue
},
},{ multi: true}
).exec();
Here req.body.elementId is array of sub doc id's.
Approach 1 was almost right. I was passing array of elementId's are which are in string format so i converted them in ObjectId form and then it works.
var arrOfObjectId = [];
req.body.elementId.forEach(elem => {
arrOfObjectId.push(Types.ObjectId(elem))
});
To find the difference between both of the array i printed both in console which were showing like below:
console.log(req.body.elementId)
Result: ['xxxxxxxxxxxxxxxxxxxxxxxx','yyyyyyyyyyyyyyyyyyyyyyyy'] //WRONG
console.log(arrOfObjectId)
Result: [ObjectId('xxxxxxxxxxxxxxxxxxxxxxxx'),ObjectId('yyyyyyyyyyyyyyyyyyyyyyyy')] //RIGHT
var updated = await ModelName.update(
{
'subDocArray._id' : { $in: arrOfObjectId }
},
{
$set: {
'subDocArray.$[elem].abc': req.body.abcValue,
'subDocArray.$[elem].xyz': req.body.xyzValue
},
},{ "arrayFilters": [{ "elem._id": { $in: arrOfObjectId } }], "multi": true, "upsert": true }
).lean().exec();

$unset not working on array of objects?

I have a collection with documents like below:
{
title: "whatever",
answers: [
{id: 10, question: ObjectId("54380350a52147000aad4e9b")},
{id: 13, question: ObjectId("54380350a52147000aad4e9c")},
{id: 33}
]
}
I'm have attempted to unset the question attribute using the commands below:
db.participants.update({}, {$unset: {"answers.question": ""}}, {upsert: false, multi:true} )
and
db.participants.update({}, {$unset: {"answers.question": 1}}, {upsert: false, multi:true} )
Both of these spit out: WriteResult({ "nMatched" : 628795, "nUpserted" : 0, "nModified" : 0 }) when completed, but no documents are updated (as the message suggests). Any idea why my update isn't working?
Your update is not working because answers is an array. So you should use the positional operator:
db.participants.update({ 'answers.question': { $exists: true } }, { $unset: { 'answers.$.question': '' } }, { upsert: false, multi: true });
However it updates only one item in the array. This is a limitation of MongoDB, see SERVER-1243.
You can use a forEach to iterate through the items of the array:
db.eval(function () {
db.participants.find({ 'answers.question': { $exists: true } }).forEach(function (participant) {
participant.answers.forEach(function (answer) {
delete answer.question;
});
db.participants.update({ _id: participant._id }, { $set: { answers: participant.answers } });
});
});

Mongoose unique index on subdocument

Let's say I have a simple schema:
var testSchema = new mongoose.Schema({
map: { type: [ mongoose.Schema.Types.Mixed ], default: [] },
...possibly something else
});
Now let's ensure that pairs (_id, map._id) are unique.
testSchema.index({ _id: 1, 'map._id': 1 }, { unique: true });
Quick check using db.test.getIndexes() shows that it was created.
{
"v" : 1,
"unique" : true,
"key" : {
"_id" : 1,
"map._id" : 1
},
"name" : "_id_1_map._id_1",
"ns" : "test.test",
"background" : true,
"safe" : null
}
The problem is, this index is ignored and I can easily create multiple subdocuments with the same map._id. I can easily execute following query multiple times:
db.maps.update({ _id: ObjectId("some valid id") }, { $push: { map: { '_id': 'asd' } } });
and end up with following:
{
"_id": ObjectId("some valid id"),
"map": [
{
"_id": "asd"
},
{
"_id": "asd"
},
{
"_id": "asd"
}
]
}
What's going on here? Why can I push conflicting subdocuments?
Long story short: Mongo doesn't support unique indexes for subdocuments, although it allows creating them...
This comes up in google so I thought I'd add an alternative to using an index to achieve unique key constraint like functionality in subdocuments, hope that's OK.
I'm not terribly familiar with Mongoose so it's just a mongo console update:
var foo = { _id: 'some value' }; //Your new subdoc here
db.yourCollection.update(
{ '_id': 'your query here', 'myArray._id': { '$ne': foo._id } },
{ '$push': { myArray: { foo } })
With documents looking like:
{
_id: '...',
myArray: [{_id:'your schema here'}, {...}, ...]
}
The key being that you ensure update will not return a document to update (i.e. the find part) if your subdocument key already exists.
First objectId length in mongodb must be 24. Then you can turn off _id, and rename _id as id or others,and try $addToSet. Good luck.
CoffeeScript example:
FromSchema = new Schema(
source: { type: String, trim: true }
version: String
{ _id: false }//to trun off _id
)
VisitorSchema = new Schema(
id: { type: String, unique: true, trim: true }
uids: [ { type: Number, unique: true} ]
from: [ FromSchema ]
)
//to update
Visitor.findOneAndUpdate(
{ id: idfa }
{ $addToSet: { uids: uid, from: { source: source, version: version } } }
{ upsert: true }
(err, visitor) ->
//do stuff