MongoDB Need to $push array in nested foreach loops - mongodb

Question

I have 3 collections: parents, children and links with subset of data like:
Parents:
{ "_id": 1, "PID" : 1, "Pname" : "Joe", "Sal" : 20000 },
{ "_id": 2, "PID" : 2, "Pname" : "Jim", "Sal" : 14100 },
{ "_id": 3, "PID" : 3, "Pname" : "Bob", "Sal" : 13500 },
{ "_id": 4, "PID" : 4, "Pname" : "Amy", "Sal" : 12000 },
{ "_id": 5, "PID" : 5, "Pname" : "George", "Sal" : 10000 }
Children:
{ "_id" : 1, "CID" : 1, "Cname" : "Ronney", "Age" : 10 },
{ "_id" : 2, "CID" : 2, "Cname" : "Mo", "Age" : 11 },,
{ "_id" : 3, "CID" : 3, "Cname" : "Adam", "Age" : 13 },
{ "_id" : 4, "CID" : 4, "Cname" : "Eve", "Age" : 21 },
{ "_id" : 5, "CID" : 5, "Cname" : "Johny", "Age" : 19 },
{ "_id" : 6, "CID" : 6, "Cname" : "Sammy", "Age" : 25 },
{ "_id" : 7, "CID" : 7, "Cname" : "Sammy", "Age" : 23 }
Links:
{ "_id" : 1, "PID" : 1, "CID" : 1 },
{ "_id" : 2, "PID" : 1, "CID" : 3 },
{ "_id" : 3, "PID" : 2, "CID" : 5 },
{ "_id" : 4, "PID" : 2, "CID" : 7 },
{ "_id" : 5, "PID" : 2, "CID" : 2 },
{ "_id" : 6, "PID" : 4, "CID" : 4 },
{ "_id" : 7, "PID" : 5, "CID" : 6 }
I need to $push an array of children names into to the parents collection using the links collections which tie parent id to child id. So, for example Parent 1 will have:
{ "_id" :1, "PID" : 1, "Pname" : "Joe", "Sal" : 20000, “Children” : [“Ronny”, ”Adam”]} }
I think I can use a nested foreach loops to achieve this, but I am confused about how.
Any help would be greatly appreciated!

Answer 1

You can forego the usage of "nested loops" with any MongoDB version supporting the $lookup aggregation pipeline operation. This will allow the "joining" of the data from multiple sources into one result set.
Given the collections "parents", "children" and "links", you want to perform two $lookup operations followed by $unwind statements to get the related data from "links" first and then join that to the "children".
Finally for convenience you can $group to get the child names into an array for each parent:
db.parents.aggregate([
{ "$lookup": {
"from": "links",
"localField": "PID",
"foreignField": "PID",
"as": "children"
}},
{ "$unwind": "$children" },
{ "$lookup": {
"from": "children",
"localField": "children.CID",
"foreignField": "CID",
"as": "children"
}},
{ "$unwind": "$children" },
{ "$group": {
"_id": "$_id",
"children": { "$push": "$children.Cname" }
}}
])
Which on your data sample gives output like:
{ "_id" : 4, "children" : [ "Eve" ] }
{ "_id" : 5, "children" : [ "Sammy" ] }
{ "_id" : 2, "children" : [ "Johny", "Sammy", "Mo" ] }
{ "_id" : 1, "children" : [ "Ronney", "Adam" ] }
Now you want to use that data output as a basis to loop and update the appropriate "parent" documents. Taking into consideration that the actual data can and will likely be much larger:
let ops = [];
db.parents.aggregate([
{ "$lookup": {
"from": "links",
"localField": "PID",
"foreignField": "PID",
"as": "children"
}},
{ "$unwind": "$children" },
{ "$lookup": {
"from": "children",
"localField": "children.CID",
"foreignField": "CID",
"as": "children"
}},
{ "$unwind": "$children" },
{ "$group": {
"_id": "$_id",
"children": { "$push": "$children.Cname" }
}}
]).forEach(doc => {
ops = [
...ops,
{
"updateOne": {
"filter": { "_id": doc._id },
"update": {
"$push": { "Children": { "$each": doc.children } }
}
}
}
];
if ( ops.length >= 500 ) {
db.parents.bulkWrite(ops);
ops = [];
}
});
if ( ops.length != 0 ) {
db.parents.bulkWrite(ops);
ops = [];
}
Which then ammends your data as :
{ "_id" : 1, "PID" : 1, "Pname" : "Joe", "Sal" : 20000, "Children" : [ "Ronney", "Adam" ] }
{ "_id" : 2, "PID" : 2, "Pname" : "Jim", "Sal" : 14100, "Children" : [ "Johny", "Sammy", "Mo" ] }
{ "_id" : 3, "PID" : 3, "Pname" : "Bob", "Sal" : 13500 }
{ "_id" : 4, "PID" : 4, "Pname" : "Amy", "Sal" : 12000, "Children" : [ "Eve" ] }
{ "_id" : 5, "PID" : 5, "Pname" : "George", "Sal" : 10000, "Children" : [ "Sammy" ] }
The alternate to using the $push operator would be to use $set instead to write the whole array at once. But it is generally safter to $push or $addToSet which accounts for the "possibility" that an array may already exist at the location, and the intent is to "add" rather than "overwrite.