Many To Many with embedded pivot list - mongodb

I am trying to select all users which references a group that references a permission that has "valid" set to true (possibly multiple). To achieve this I am planning to use an aggregation with a lookup.
db={
"users": [
{
"_id": "1",
"groups": [
"2"
]
},
{
"_id": "2",
"groups": [
"1"
]
}
],
"group": [
{
"_id": "1",
"permissions": [
"12",
"3"
]
},
{
"_id": "2",
"permissions": [
"3",
"2"
]
}
],
"permission": [
{
"_id": "12",
"valid": true
},
{
"_id": "3",
"valid": true
},
{
"_id": "2",
"valid": true
}
]
}
I can't think of an efficient/simple way to do this.
I have 2 ideas:
have a lookup from user to group with a pipeline that does a lookup from group to permission. The problem there is I would need to do a $match to see if the user's group list contains the groups id (which I don't know if its possible)
do a simple lookup from user to group than unwind the joined field and do a second lookup using the joined field from group to permission. Than at the end do a $match to see if any contain "valid" true and group the results back to get the original users. This idea doesnt sound very efficient though. (https://mongoplayground.net/p/JB9Yb3a65Cn)
Any ideas or inputs on this?

you can use direct array of object without $unwind in lookup as localFields,
second lookup will replace group field with permissions response because it is not needed in next stage
db.users.aggregate([
{
$lookup: {
from: "group",
localField: "groups",
foreignField: "_id",
as: "group"
}
},
{
$lookup: {
from: "permission",
localField: "group.permissions",
foreignField: "_id",
as: "group"
}
},
{ $match: { "group.valid": true } },
{ $unset: "group" }
])
Playground

Related

In MongoDB - trying to perform a recursive lookup between 2 collections where 1 collection has a nested relationship within that collection

The scenario is I have 2 collections - 1 with a list of users and 1 with a list of groups.
"users": [
{
"_id": 1,
"username": "Fred Smith",
"level": 12,
},
{
"_id": 2,
"username": "Bob Brown",
"level": 20,
},
{
"_id": 3,
"username": "Joe Blogs",
"level": 1
}
],
A group holds a list of users but crucially also potentially holds groups within the same list as well.
"groups": [
{
"_id": 1,
"groupname": "admin",
"members": ["Fred Smith"]
},
{
"_id": 2,
"groupname": "users",
"members": [
"Fred Smith",
"Bob Brown",
"contractors"
]
},
{
"_id": 3,
"groupname": "contractors",
"members": ["Joe Blogs"]
},
{
"_id": 4,
"groupname": "all",
"members": ["users"]
}]
Given a query of a specific username I need to return all groups that user is a member of.
So far I have the $graphLookup working in the following playground - but I am not sure of the next step to take into account the nested group name inside that list of users. There is also the possibility the groups may nested further groups.
db.users.aggregate([
{
$match: {
"username": "Joe Blogs"
}
},
{
"$graphLookup": {
"from": "groups",
"startWith": "$username",
"connectFromField": "username",
"connectToField": "members",
"as": "groups",
"maxDepth": 10
}
},
{
$project: {
"username": 1,
"groups": "$groups.groupname"
}
}
]
https://mongoplayground.net/p/GBRJp42bdjt
[
{
"_id": 3,
"groups": [
"contractors"
],
"username": "Joe Blogs"
}
]
In this example I have queried Fred who is a member of the contractors group but the contractors group is also a member of the users group and the users group is a member of the all group.
So I would need the query to return both contractors (which it does now) but also users and all.
It feels like given an initial array of groups in this query I then need to do a different query with the array of groups to find out what groups they are a member of.
I could modify the structure if required to make this easier but the nesting of groups is a requirement.
Any help appreciated.
As an update I have created another query which given an array of groups will return a normalised list of all the nested groups - https://mongoplayground.net/p/9NGG_wIJkGX
Now I just need to work out how to feed the results of the first query to the second query - if anyone has an idea I would appreciate it.
Your first playground is actually very close already. You just need to use groupname in connectFromField in the $graphLookup to traverse the collection. At the last $project stage, use $reduce with $setUnion to extract all the groups from the $graphLookup result.
db.users.aggregate([
{
$match: {
"username": "Joe Blogs"
}
},
{
"$graphLookup": {
"from": "groups",
"startWith": "$username",
"connectFromField": "groupname",
"connectToField": "members",
"as": "groups",
"maxDepth": 10
}
},
{
$project: {
"username": 1,
"groups": {
"$reduce": {
"input": "$groups",
"initialValue": [],
"in": {
"$setUnion": [
"$$value",
[
"$$this.groupname"
]
]
}
}
}
}
}
])
Mongo Playground

Mongodb aggregation lookup to add field in each array with condition

I have 3 collections.
User:
{
"_id":ObjectId("60a495cdd4ba8b122899d415"),
"email":"br9#gmail.com",
"username":"borhan"
}
Panel:
{
"_id": ObjectId("60a495cdd4ba8b122899d417"),
"name": "borhan",
"users": [
{
"role": "admin",
"joined": "2021-05-19T04:35:47.474Z",
"status": "active",
"_id": ObjectId("60a495cdd4ba8b122899d418"),
"user": ObjectId("60a495cdd4ba8b122899d415")
},
{
"role": "member",
"joined": "2021-05-19T04:35:47.474Z",
"status": "active",
"_id": ObjectId("60a49600d4ba8b122899d41a"),
"user": ObjectId("60a34e167958972d7ce6f966")
}
],
}
Team:
{
"_id":ObjectId("60a495e0d4ba8b122899d419"),
"title":"New Teams",
"users":[
ObjectId("60a495cdd4ba8b122899d415")
],
"panel":ObjectId("60a495cdd4ba8b122899d417")
}
I want to receive a output from querying Panel colllection just like this:
{
"_id": ObjectId("60a495cdd4ba8b122899d417"),
"name": "borhan",
"users": [
{
"role": "admin",
"joined": "2021-05-19T04:35:47.474Z",
"status": "active",
"_id": ObjectId("60a495cdd4ba8b122899d418"),
"user": ObjectId("60a495cdd4ba8b122899d415"),
"teams":[
{
"_id":ObjectId("60a495e0d4ba8b122899d419"),
"title":"New Teams",
"users":[
ObjectId("60a495cdd4ba8b122899d415")
],
"panel":ObjectId("60a495cdd4ba8b122899d417")
}
]
},
{
"role": "member",
"joined": "2021-05-19T04:35:47.474Z",
"status": "active",
"_id": ObjectId("60a49600d4ba8b122899d41a"),
"user": ObjectId("60a34e167958972d7ce6f966")
}
],
}
I mean i want to add teams field (which is array of teams that user is existed on it) to each user in Panel collection
Here is my match query in mongoose to select specific panel:
panel_model.aggregate([
{
$match: {
users: {
$elemMatch: {user: ObjectId("60a495cdd4ba8b122899d415"), role:"admin"}
}
}
},
])
Is it possible to get my output with $lookup or $addFields aggregations?
You need to join all three collections,
$unwind to deconstruct the array
$lookup there are two kind of lookups which help to join collections. First I used Multiple-join-conditions-with--lookup, and I used standrad lookup to join Users and Teams collections.
$match to match the user's id
$expr - when you use $match inside lookup, u must use it.
$set to add new fields
$group to we already destructed using $unwind. No we need to restructure it
here is the code
db.Panel.aggregate([
{ $unwind: "$users" },
{
"$lookup": {
"from": "User",
"let": { uId: "$users.user" },
"pipeline": [
{
$match: {
$expr: {
$eq: [ "$_id", "$$uId" ]
}
}
},
{
"$lookup": {
"from": "Team",
"localField": "_id",
"foreignField": "users",
"as": "teams"
}
}
],
"as": "users.join"
}
},
{
"$set": {
"users.getFirstElem": {
"$arrayElemAt": [ "$users.join", 0 ]
}
}
},
{
$set: {
"users.teams": "$users.getFirstElem.teams",
"users.join": "$$REMOVE",
"users.getFirstElem": "$$REMOVE"
}
},
{
"$group": {
"_id": "$_id",
"name": { "$first": "name" },
"users": { $push: "$users" }
}
}
])
Working Mongo playground
Note : Hope the panel and user collections are in 1-1 relationship. Otherwise let me know

MongoDB add field with keys from object

I have the following two collections:
{
"organizations": [
{
"_id": "1",
"name": "foo",
"users": { "1": "admin", "2": "member" }
},
{
"_id": "2",
"name": "bar",
"users": { "1": "admin" }
}
],
"users": [
{
"_id": "1",
"name": "john smith"
},
{
"_id": "2",
"name": "bob johnson"
}
]
}
The following query works to merge the users into members when I just use an array of the user ids to match, however, the users prop is an object.
{
"collection": "organizations",
"command": "aggregate",
"query": [
{
"$lookup": {
"from": "users",
"localField": "users",
"foreignField": "_id",
"as": "members"
}
}
]
}
What I'm hoping to do is lookup by id then create a members array from the results with the user object including the role (value of the users objects:
{
"_id": "1",
"name": "foo",
"users": {
"1": "admin",
"2": "member"
},
"members": [
{
"_id": "1",
"name": "john smith",
"role": "admin"
},
{
"_id": "2",
"name": "bob johnson",
"role": "user"
}
]
}
Here's the sandbox I have setup: https://mongoplayground.net/p/yhRpeRvJf3u
You really need to change your schema design, this will cause the performance on retrieving data,
$addFields to add new field usersArray convert users object to array using $objectToArray, the format will be k(key) and v(value),
$lookup to join users collection, set localField name to usersArray.k
$addFields, remove usersArray field using $$REMOVE,
$map iterate loop of members array and $reduce to iterate loop of usersArray and get matching role as per _id and merge current fields and role field using $mergeObjects
db.organizations.aggregate([
{
$addFields: {
usersArray: {
$objectToArray: "$users"
}
}
},
{
"$lookup": {
"from": "users",
"localField": "usersArray.k",
"foreignField": "_id",
"as": "members"
}
},
{
$addFields: {
usersArray: "$$REMOVE",
members: {
$map: {
input: "$members",
as: "m",
in: {
$mergeObjects: [
"$$m",
{
role: {
$reduce: {
input: "$usersArray",
initialValue: "",
in: { $cond: [{ $eq: ["$$this.k", "$$m._id"] }, "$$this.v", "$$value"] }
}
}
}
]
}
}
}
}
}
])
Playground
First of all, the problem with your query is you want to use a KEY to do the $lookup, then the members field always gonna be empty.
You are trying to use users as local field, but users is an object, so you need the key (users.1, users.2, ... )
To do this you need to use $objectToArray, which create an object array with two fields: k and v for key and value. So now, you can $lookup with the field users.k.
To get the query you need $unwind before $lookup because you also want the users filed into the new document.
With the new object created using $objectToArray, you can do $unwind to get the values in differents documents. And then $lookup to get the "join".
Here, localField uses the value k created by $objectToArray (the object key).
After that, $set to add the field with the role and $group again into one document.
Ive used _id to get the values without changes between stages, and into members push the members in each collection.
And then, $project to output the values you want. In this case, tha calues "stored" into _id and the array members in "one level" using $reduce.
So, the query you need I think is this:
db.organizations.aggregate([
{
"$match": {
"_id": "1"
}
},
{
"$set": {
"usersArray": {
"$objectToArray": "$users"
}
}
},
{
"$unwind": "$usersArray"
},
{
"$lookup": {
"from": "users",
"localField": "usersArray.k",
"foreignField": "_id",
"as": "members"
}
},
{
"$set": {
"members.role": "$usersArray.v"
}
},
{
"$group": {
"_id": {
"_id": "$_id",
"users": "$users",
"name": "$name"
},
"members": {
"$push": "$members"
}
}
},
{
"$project": {
"members": {
"$reduce": {
"input": "$members",
"initialValue": [],
"in": {
"$concatArrays": [
"$$value",
"$$this"
]
}
}
},
"users": "$_id.users",
"name": "$_id.name",
"_id": "$_id._id"
}
}
])
Example here

Mongo DB - query nested arrays of objects in referenced document

I have few referenced collections: Employees, Roles, Schools...
I need to query the database, to list all employees that are employed by a certain school. That means all Employees, that has certain School ID in their corresponding role document must be returned in the list or array of results.
So far I have tried to find it like this:
const employees = mongoose.schema("Employees");
employees.find({})
.populate({
"path": "roles"
})
.then(function(docs){
docs.filter(function(){
// write my filters here....
})
});
but this still is inefficient and I can't make it work.
It has to be a smarter way...
This is my Employee document, which references a document in Roles Collection:
{
"_id" : { "$oid" : "57027f1b9522d363243abr42" },
"assignedRoles": [
{
"type": "teacher",
"schools": [
{
"schoolId": "57027f1b9522d3632423rf32",
"classes" : ["math", "science"],
"active": true
},
{
"schoolId": "57027f1b9522d36324252fs2",
"classes" : ["science"],
"active": true
},
{
"schoolId": "57027f1b9522d36324242f35",
"classes" : ["math"],
"active": true
},
]
},
{
"type": "manager",
"schools": [
{
"schoolId": "57027f1b9522d3632423rf32",
"active": true
},
{
"schoolId": "57027f1b9522d36324252fs2",
"active": true
}
]
}
{
"type": "principal",
"schools": [
{
"schoolId": "57027f1b9522d3632423rf32",
"active": true
}
]
}
],
"rolesMeta": "Meta Info for the Roles"
}
The following is the list of Schools - irrelevant for the task, I am just adding that for the completion:
{
"_id": { "$oid" : "57027f1b9522d3632423rf32" },
"name": "G.Washington HS",
"district": "six",
"state": "New Mexico"
},
{
"_id": { "$oid" : "57027f1b9522d36324252fs2" },
"name": "A. Lincoln HS",
"district": "six",
"state": "New Mexico"
},
{
"_id": { "$oid" : "57027f1b9522d36324242f35" },
"name": "T. Roosvelt HS",
"district": "four",
"state": "New Mexico"
}
The Resolution to the problem is to use MongoDB Aggregate framework to be able to drill down and "flatten" the array structure.
Here is how I have resolved it:
db.employees.aggregate([
// I use $project to specify which field I want selected from the collection
{
$project: {_id: 1, "roles": 1}
},
// $lookup helps me get results and populate the 'role' field from the Roles collection. This returns an array
{
$lookup: {
"from": "roles",
"localField": "roles",
"foreignField": "_id",
"as": "roles"
}
},
// $unwind is self explanatory - it flattens the array and creates multiple objects, instances of the parent object for each array element.
{
$unwind: "$roles"
},
// now repeat for each array in this document
{
$unwind: "$roles.assignedRoles"
},
// repeat again, since I need to look in the schools array.
{
$unwind: "$roles.assignedRoles.schools"
},
// The following line will return all records that the school ID checks out.
{
$match: {"roles.assignedRoles.schools.schoolId": school}
},
// $group is a necessary cleanup task so we avoid repeating employee records... Nicely Done! :)
{
$group: {
"_id": "$_id",
"roles": {"$push": "$roles"},
}
}
])
.exec()
.then(function (docs) {
// here you process the resulted docs.
});

How to combine multiple collections and merge the joined documents into a single document

Here is the problem, I am unable to get the following result. Please look into the piece of json and help me out.
This is my data:
[
{
"user_id": "65asdfksadjfk3u4",
"lat": 23.4343,
"long": 15.2382
}
]
Currently my result is:
[
{
"_id": "65asdfksadjfk3u4",
"name": "Srini",
"age": 26,
"some other key": "some other values"
}
]
I need to get the collection from the user_id and add it to the same array object. As you can notice both lat and long are being removed in my current result.
[
{
"_id": "65asdfksadjfk3u4",
"name": "Srini",
"age": 26,
"some other keys": "some other values",
"lat": 23.4343,
"long": 15.2382
}
]
You can append the $lookup stage to join the current pipeline results with the users collections by the user_id fields and then use $mergeObjects in the $replaceRoot to merge the joined documents from users and the current results:
db.collection.aggregate([
/* current pipeline here */
{ "$lookup": {
"from": "users",
"localField": "_id",
"foreignField": "user_id",
"as": "user"
} },
{ "$replaceRoot": {
"newRoot": {
"$mergeObjects": [
{ "$arrayElemAt": [ "$user", 0 ] },
"$$ROOT"
]
}
} },
{ "$project": { "user": 0, "user_id": 0 } }
]);