Join $graphLookup result with other collection - mongodb

I'm working on a hierarchical structure that stores a binary tree.
Let's say I have two collections: users and nodes.
users collection stores personal information and nodes stores the structure of the tree using the Parent References pattern: https://docs.mongodb.com/manual/tutorial/model-tree-structures-with-parent-references/
Users:
[{
"_id": {
"$oid": "600365521599912a5c814e5e"
},
"nombre": "Andres",
"correo": "oachica#gmail.com"
},{
"_id": {
"$oid": "600365e9ccf1e51b2cab341f"
},
"nombre": "Andres",
"correo": "cachi777_#hotmail.com"
},{
"_id": {
"$oid": "6004591536a40941f48121f9"
},
"nombre": "Laura",
"correo": "w.l777#hotmail.com"
},{
"_id": {
"$oid": "6004596936a40941f48121fb"
},
"nombre": "Javi",
"correo": "jocta#hotmail.com"
},{
"_id": {
"$oid": "60047cf23f3f1a0d647cb2c7"
},
"nombre": "Lina",
"correo": "lvelas#hotmail.com"
}]
nodos:
[{
"_id": {
"$oid": "60035d0a1599912a5c814e58"
},
"idUsuario": "600365521599912a5c814e5e",
"nodoPadre": ""
},{
"_id": {
"$oid": "60047e6874cab54a7088ca56"
},
"idUsuario": "600365e9ccf1e51b2cab341f",
"nodoPadre": {
"$oid": "60035d0a1599912a5c814e58"
}
},{
"_id": {
"$oid": "60047f42c89add3c20cff990"
},
"idUsuario": "6004591536a40941f48121f9",
"nodoPadre": {
"$oid": "60047e6874cab54a7088ca56"
}
},{
"_id": {
"$oid": "60047f5dc89add3c20cff991"
},
"idUsuario": "6004596936a40941f48121fb",
"nodoPadre": {
"$oid": "60047f42c89add3c20cff990"
}
},{
"_id": {
"$oid": "600480de9fd6a42b40679e6d"
},
"idUsuario": "60047cf23f3f1a0d647cb2c7",
"nodoPadre": {
"$oid": "60047f5dc89add3c20cff991"
}
}]
Each document in nodos has corresponding document in users bound by _id and idUsuario.
Generally a document in nodes collection has a parent node in the same collection bound by nodoPadre field.
I'm able to get childs of a node using $graphLookup aggregation:
As you can se I got the childs of a node. Now I need to put the personal information in each child in the result array "hijos" as shown below:
Thanks for your help.

$graphLookup as per your requirement
$unwind deconstruct hijos array
$addFields convert hijos.idUsuario to object id because its an string, if it is already in object id then remove this stage
$lookup with users collection
$unwind deconstruct hijos.idUsuario array
$addFields to remove hijos if it is blank {} object
$group by _id and reconstruct hijos array
db.nodes.aggregate([
{
"$graphLookup": {
"from": "nodes",
"startWith": "$_id",
"connectFromField": "_id",
"connectToField": "nodoPadre",
"as": "hijos",
"maxDepth": 4
}
},
{
$unwind: {
path: "$hijos",
preserveNullAndEmptyArrays: true
}
},
{ $addFields: { "hijos.idUsuario": { $toObjectId: "$hijos.idUsuario" } } },
{
"$lookup": {
"from": "users",
"localField": "hijos.idUsuario",
"foreignField": "_id",
"as": "hijos.idUsuario"
}
},
{
$unwind: {
path: "$hijos.idUsuario",
preserveNullAndEmptyArrays: true
}
},
{
$addFields: {
hijos: {
$cond: [{ $eq: ["$hijos", {}] }, "$$REMOVE", "$hijos"]
}
}
},
{
$group: {
_id: "$_id",
hijos: { $push: "$hijos" },
idUsuario: { $first: "$idUsuario" },
nodoPadre: { $first: "$nodoPadre" }
}
}
])
Playground

Related

Mongodb aggregation lookup to add field in each array with condition

I have 3 collections.
User:
{
"_id":ObjectId("60a495cdd4ba8b122899d415"),
"email":"br9#gmail.com",
"username":"borhan"
}
Panel:
{
"_id": ObjectId("60a495cdd4ba8b122899d417"),
"name": "borhan",
"users": [
{
"role": "admin",
"joined": "2021-05-19T04:35:47.474Z",
"status": "active",
"_id": ObjectId("60a495cdd4ba8b122899d418"),
"user": ObjectId("60a495cdd4ba8b122899d415")
},
{
"role": "member",
"joined": "2021-05-19T04:35:47.474Z",
"status": "active",
"_id": ObjectId("60a49600d4ba8b122899d41a"),
"user": ObjectId("60a34e167958972d7ce6f966")
}
],
}
Team:
{
"_id":ObjectId("60a495e0d4ba8b122899d419"),
"title":"New Teams",
"users":[
ObjectId("60a495cdd4ba8b122899d415")
],
"panel":ObjectId("60a495cdd4ba8b122899d417")
}
I want to receive a output from querying Panel colllection just like this:
{
"_id": ObjectId("60a495cdd4ba8b122899d417"),
"name": "borhan",
"users": [
{
"role": "admin",
"joined": "2021-05-19T04:35:47.474Z",
"status": "active",
"_id": ObjectId("60a495cdd4ba8b122899d418"),
"user": ObjectId("60a495cdd4ba8b122899d415"),
"teams":[
{
"_id":ObjectId("60a495e0d4ba8b122899d419"),
"title":"New Teams",
"users":[
ObjectId("60a495cdd4ba8b122899d415")
],
"panel":ObjectId("60a495cdd4ba8b122899d417")
}
]
},
{
"role": "member",
"joined": "2021-05-19T04:35:47.474Z",
"status": "active",
"_id": ObjectId("60a49600d4ba8b122899d41a"),
"user": ObjectId("60a34e167958972d7ce6f966")
}
],
}
I mean i want to add teams field (which is array of teams that user is existed on it) to each user in Panel collection
Here is my match query in mongoose to select specific panel:
panel_model.aggregate([
{
$match: {
users: {
$elemMatch: {user: ObjectId("60a495cdd4ba8b122899d415"), role:"admin"}
}
}
},
])
Is it possible to get my output with $lookup or $addFields aggregations?
You need to join all three collections,
$unwind to deconstruct the array
$lookup there are two kind of lookups which help to join collections. First I used Multiple-join-conditions-with--lookup, and I used standrad lookup to join Users and Teams collections.
$match to match the user's id
$expr - when you use $match inside lookup, u must use it.
$set to add new fields
$group to we already destructed using $unwind. No we need to restructure it
here is the code
db.Panel.aggregate([
{ $unwind: "$users" },
{
"$lookup": {
"from": "User",
"let": { uId: "$users.user" },
"pipeline": [
{
$match: {
$expr: {
$eq: [ "$_id", "$$uId" ]
}
}
},
{
"$lookup": {
"from": "Team",
"localField": "_id",
"foreignField": "users",
"as": "teams"
}
}
],
"as": "users.join"
}
},
{
"$set": {
"users.getFirstElem": {
"$arrayElemAt": [ "$users.join", 0 ]
}
}
},
{
$set: {
"users.teams": "$users.getFirstElem.teams",
"users.join": "$$REMOVE",
"users.getFirstElem": "$$REMOVE"
}
},
{
"$group": {
"_id": "$_id",
"name": { "$first": "name" },
"users": { $push: "$users" }
}
}
])
Working Mongo playground
Note : Hope the panel and user collections are in 1-1 relationship. Otherwise let me know

MongoDB: Populate reference in $group when aggregating?

I have a collection that I need to group by year. My aggregation pipeline is as such:
const WorkHistoryByYear = await this.workHistroyModel.aggregate([
{
$group: {
_id: '$year',
history: {
$push: '$$ROOT'
}
}
}
]);
Which works as expected, returning:
[{
"_id": 2003,
"history": [
{
"_id": "600331b3d84ac418877a0e5a",
"tasksPerformed": [
"5fffb180a477c4f78ad67331",
"5fffb18aa477c4f78ad67332"
],
"year": 2003
},
{
"_id": "600331dcd84ac418877a0e5c",
"tasksPerformed": [
"5fffb180a477c4f78ad67331"
],
"year": 2003
}
]
}]
but I'd like to populate a field if possible.
The WorkHistory schema has a field, tasksPerformed which is an array of ObjectId references. Here is the Task schema:
export const TaskSchema = new Schema({
active: {
type: Schema.Types.Boolean,
default: true,
},
title: {
type: Schema.Types.String,
required: true,
},
order: {
type: Schema.Types.Number,
index: true,
}
});
Is it possible to populate the referenced models within the aggregation? $lookup seems to be what I need, but I have yet to get that to work when following the documentation.
I don't do a lot of database work, so I'm having some difficulty finding the right operator(s) to use, and I've seen similar questions, but not a definitive answer.
Edit:
After adding the code from #varman's answer, my return is now:
{
"_id": 2003,
"history": {
"_id": "600331b3d84ac418877a0e5a",
"tasksPerformed": [
"5fffb180a477c4f78ad67331",
"5fffb18aa477c4f78ad67332"
],
"year": 2003,
"history": {
"tasksPerformed": []
}
}
}
I converted the ObjectId references to strings in an effort to help the matching, but I'm still coming up short.
You can do the lookup to join both collections
$unwind to deconstruct the array. (Array to Objects)
$lookup to join collections
$group to reconstruct the deconstructed array again
The script for the above result is
db.workHistory.aggregate([
{
"$unwind": "$history"
},
{
"$lookup": {
"from": "taskSchema",
"localField": "history.tasksPerformed",
"foreignField": "_id",
"as": "history.tasksPerformed"
}
},
{
"$group": {
"_id": "$_id",
"history": {
"$push": "$history"
}
}
}
])
Working Mongo playground
But before grouping, you have collection look like this
db={
"workHistory": [
{
"_id": 2003,
"history": [
{
"_id": "600331b3d84ac418877a0e5a",
"tasksPerformed": [
"5fffb180a477c4f78ad67331",
"5fffb18aa477c4f78ad67332"
],
"year": 2003
},
{
"_id": "600331dcd84ac418877a0e5c",
"tasksPerformed": [
"5fffb180a477c4f78ad67331"
],
"year": 2003
}
]
}
],
"taskSchema": [
{
"_id": "5fffb180a477c4f78ad67331",
"active": true,
"title": "first"
},
{
"_id": "5fffb18aa477c4f78ad67332",
"active": true,
"title": "second"
}
]
}
Since $unwind is expensive, we could have done aggregation
db.workHistory.aggregate([
{
"$lookup": {
"from": "taskSchema",
"localField": "tasksPerformed",
"foreignField": "_id",
"as": "tasksPerformed"
}
},
{
$group: {
_id: "$year",
history: {
$push: "$$ROOT"
}
}
}
])
Working Mongo playground
you could also do it without an unwind:
db.WorkHistory.aggregate(
[
{
$lookup: {
from: "Tasks",
localField: "tasksPerformed",
foreignField: "_id",
as: "tasksPerformed"
}
},
{
$group: {
_id: "$year",
history: { $push: "$$ROOT" }
}
}
])

Referring to a different collection from an existing one and counting from the same collection

I have to collections A and B in which the documents of A contains the object ids which are present in the B in the fields centre and gcentre. What I'm currently is outputting the result which contains the name of the parent centre by referring from collection A's object id to B and then referring the gcentre's id to find the child and centre and count the documents assigned via javascript post-processing. Been new to the aggregation pipeline, I don't know how to refer via object id and that sort of counting of records. Is it possible with the aggregation pipeline? I have tried with $lookup but it doesn't seem to give the output as expected.
Documents in collection A:
{
"_id": {
"$oid": "5bbafa98d8c77251bea30f8c"
},
"parentCentre": {
"$oid": "1cbafa99d8c77251bea30f11"
},
"childCentre": {
"$oid": "5cbafa99d8c77251bea30f8d"
},
},
{
"_id": {
"$oid": "5bbafa98d8c77251bea30f8c"
},
"parentCentre": {
"$oid": "1cbafa99d8c77251bea30f11"
},
"childCentre": {
"$oid": "5cbafa99d8c77251bea30f8d"
},
},
{
"_id": {
"$oid": "5bbafa98d8c77251bea30f8c"
},
"parentCentre": {
"$oid": "1cbafa99d8c77251bea30f11"
},
"childCentre": {
"$oid": "5cbafa99d8c77251bea30f8d"
},
},
{
"_id": {
"$oid": "5bbafa98d8c77251bea30f8c"
},
"parentCentre": {
"$oid": "1cbafa99d8c77251bea30f21"
},
"childCentre": {
"$oid": "5cbafa99d8c77251bea30f6d"
},
}
Documents in collection B:
{
"_id": {
"$oid": "1cbafa99d8c77251bea30f11"
},
"Type": "Parent",
"Name": "Kris Labs"
},
{
"_id": {
"$oid": "1cbafa99d8c77251bea30f21"
},
"Type": "Parent",
"Name": "DEX Labs"
},
{
"_id": {
"$oid": "5cbafa99d8c77251bea30f8d"
},
"Type": "Child",
"Name": "Mili Labs"
},
{
"_id": {
"$oid": "5cbafa99d8c77251bea30f6d"
},
"Type": "Child",
"Name": "Max Labs"
}
Result:
{
"parentCentreName":"Kris Labs",
"Records":{
{
childCentreName: "Max Labs",
recordCount: 3
}
}
},
{
"parentCentreName":"DEX Labs",
"Records":{
{
childCentreName: "Mili Labs",
recordCount: 1
}
}
}
You can use the following aggregation query:
db.A.aggregate([
{
$group: {
_id: {
p: "$parentCentre",
c: "$childCentre"
},
count: {
$sum: 1
}
}
},
{
$group: {
_id: "$_id.p",
Records: {
$push: {
childCentreName: "$_id.c",
recordCount: "$count"
}
}
}
},
{
$unwind: "$Records"
},
{
"$lookup": {
"from": "B",
"localField": "_id",
"foreignField": "_id",
"as": "p"
}
},
{
"$lookup": {
"from": "B",
"localField": "Records.childCentreName",
"foreignField": "_id",
"as": "c"
}
},
{
$unwind: "$c"
},
{
$unwind: "$p"
},
{
$project: {
"parentCentreName": "$p.Name",
"Records.childCentreName": "$c.Name",
"Records.recordCount": 1,
_id: 0
}
},
{
$group: {
_id: "$parentCentreName",
"Records": {
$push: "$Records"
}
}
},
{
$project: {
"parentCentreName": "$_id",
"Records": 1,
_id: 0
}
}
])
MongoDB Playground

Grouping in MongoDb using aggregate

I am a beginner to MongoDB and I found the Aggregate function hard to understand.
I read many topics and tried many things, however I couldn't get the results I am looking for.
Actually, I have two schema as:
1) Faculty.js
const FacultySchema = new Schema({
name: {
type: String,
required: true
}
});
2) Semester.js
const SemesterSchema = new Schema({
name: {
type: String,
required: true
},
faculty: {
type: Schema.Types.ObjectId,
ref: 'faculties'
}
});
Semester collection
[
{
"_id": ObjectId("5bf82da745209d0d48a91b62"),
"name": "1st Semester",
"faculty": ObjectId("5bf7f39a1972dd0b6c74de7d"),
"__v": 0
},
{
"_id": ObjectId("5bf8c3f945209d0d48a91b63"),
"name": "2nd Semester",
"faculty": ObjectId("5bf7f39a1972dd0b6c74de7d"),
"__v": 0
},
{
"_id": ObjectId("5bf8c3fe45209d0d48a91b64"),
"name": "3rd Semester",
"faculty": ObjectId("5bf7f39a1972dd0b6c74de7d"),
"__v": 0
},
{
"_id": ObjectId("5bf8c40345209d0d48a91b65"),
"name": "4th Semester",
"faculty": ObjectId("5bf7f39a1972dd0b6c74de7d"),
"__v": 0
}
]
What I want to group is all those semesters as an array having same faculty id in one place.
Something like:
[
{faculty: "BBA", semesters: ['first', 'second', 'third']},
{faculty: "BCA", semesters: ['first', 'second', 'third']}
];
How can I achieve this??
You can use $group aggregation to first find the distinct faculties and then $lookup to get the names of the faculties from the Faculties collection
Semester.aggregate([
{ "$group": {
"_id": "$faculty",
"semesters": { "$push": "$name" }
}},
{ "$lookup": {
"from": "faculties",
"let": { "facultyId": "$_id" },
"pipeline": [
{ "$match": { "$expr": { "$eq": ["$_id", "$$facultyId"] }}}
],
"as": "faculty"
}},
{ "$project": {
"semesters": 1, "faculty": { "$arrayElemAt": ["$faculty.name", 0] }
}}
])
Or you can use $lookup first and then $grouping the distinct names
Semester.aggregate([
{ "$lookup": {
"from": "Faculty",
"let": { "facultyId": "$_id" },
"pipeline": [
{ "$match": { "$expr": { "$eq": ["$_id", "$$facultyId"] }}}
],
"as": "faculty"
}},
{ "$unwind": "$faculty" },
{ "$group": {
"_id": "$faculty.name",
"semesters": { "$push": "$name" }
}},
{ "$project": {
"semesters": 1, "faculty": "$_id", "_id": 0 }
}}
])

Selecting all objects from complex model

I have aggregation pipeline stage:
$project: {
'school': {
'id': '$_id',
'name': '$name',
'manager': '$manager'
},
'students': '$groups.students',
'teachers': '$groups.teachers'
}
Need something like this:
{
'users': // manager + students + teachers
}
Tried:
{
'users': {
$push: {
$each: ['$school.manager', '$students', '$teachers']
}
}
}
I'm presuming that "students" and "teachers" are both arrays here and located under a common sub-document heading like so:
{
"_id": 123,
"name": "This school",
"manager": "Bill"
"groups": {
"teachers": ["Ted"],
"students": ["Missy"]
}
}
So in order to get all of those in a singular array such as "users" then it depends on your MongoDB version and the "uniqueness" of your data. For true "sets" and where you have MongoDB 2.6 or greater available, there is the $setUnion operator, albeit with an additional level of $group to make "manager" and array:
db.collection.aggregate([
{ "$group": {
"_id": { "_id": "$_id", "name": "$name" },
"manager": { "$push": "$manager" },
"groups": { "$first": "$groups" }
}},
{ "$project": {
"users": {
"$setUnion": [ "$manager", "$groups.teachers", "$groups.students" ]
}
}}
])
Or otherwise where that operator is not available or there is a "unique" problem then there is this way to handle "combining":
db.collection.aggregate([
{ "$group": {
"_id": { "_id": "_id", "name": "$name" },
"manager": { "$push": "$manager" },
"teachers": { "$first": "$groups.teachers" },
"students": { "$first": "$groups.students" },
"type": { "$first": { "$const": ["M","T","S"] } }
}},
{ "$unwind": "$type" },
{ "$project": {
"users": {
"$cond": [
{ "$eq": [ "$type", "M" ] },
"$manager",
{ "$cond": [
{ "$eq": [ "$type", "T" ] },
"$teachers",
"$students"
]}
]
}
}},
{ "$unwind": "$users" },
{ "$group": {
"_id": "$_id",
"users": { "$push": "$users" }
}}
])
This essentially "tags" each field by a "type" for which the document is copied in the pipeline. Then placed into a single "users" field depending on which "type" matched. The single array then from the resulting three documents from each original can then be safely "unwound" and combined in a final $group operation.
So "sets" are your fastest option where available or where not available or not unique you can use the later technique in order to combine these to a single list.