In MongoDB - trying to perform a recursive lookup between 2 collections where 1 collection has a nested relationship within that collection - mongodb

The scenario is I have 2 collections - 1 with a list of users and 1 with a list of groups.
"users": [
{
"_id": 1,
"username": "Fred Smith",
"level": 12,
},
{
"_id": 2,
"username": "Bob Brown",
"level": 20,
},
{
"_id": 3,
"username": "Joe Blogs",
"level": 1
}
],
A group holds a list of users but crucially also potentially holds groups within the same list as well.
"groups": [
{
"_id": 1,
"groupname": "admin",
"members": ["Fred Smith"]
},
{
"_id": 2,
"groupname": "users",
"members": [
"Fred Smith",
"Bob Brown",
"contractors"
]
},
{
"_id": 3,
"groupname": "contractors",
"members": ["Joe Blogs"]
},
{
"_id": 4,
"groupname": "all",
"members": ["users"]
}]
Given a query of a specific username I need to return all groups that user is a member of.
So far I have the $graphLookup working in the following playground - but I am not sure of the next step to take into account the nested group name inside that list of users. There is also the possibility the groups may nested further groups.
db.users.aggregate([
{
$match: {
"username": "Joe Blogs"
}
},
{
"$graphLookup": {
"from": "groups",
"startWith": "$username",
"connectFromField": "username",
"connectToField": "members",
"as": "groups",
"maxDepth": 10
}
},
{
$project: {
"username": 1,
"groups": "$groups.groupname"
}
}
]
https://mongoplayground.net/p/GBRJp42bdjt
[
{
"_id": 3,
"groups": [
"contractors"
],
"username": "Joe Blogs"
}
]
In this example I have queried Fred who is a member of the contractors group but the contractors group is also a member of the users group and the users group is a member of the all group.
So I would need the query to return both contractors (which it does now) but also users and all.
It feels like given an initial array of groups in this query I then need to do a different query with the array of groups to find out what groups they are a member of.
I could modify the structure if required to make this easier but the nesting of groups is a requirement.
Any help appreciated.
As an update I have created another query which given an array of groups will return a normalised list of all the nested groups - https://mongoplayground.net/p/9NGG_wIJkGX
Now I just need to work out how to feed the results of the first query to the second query - if anyone has an idea I would appreciate it.

Your first playground is actually very close already. You just need to use groupname in connectFromField in the $graphLookup to traverse the collection. At the last $project stage, use $reduce with $setUnion to extract all the groups from the $graphLookup result.
db.users.aggregate([
{
$match: {
"username": "Joe Blogs"
}
},
{
"$graphLookup": {
"from": "groups",
"startWith": "$username",
"connectFromField": "groupname",
"connectToField": "members",
"as": "groups",
"maxDepth": 10
}
},
{
$project: {
"username": 1,
"groups": {
"$reduce": {
"input": "$groups",
"initialValue": [],
"in": {
"$setUnion": [
"$$value",
[
"$$this.groupname"
]
]
}
}
}
}
}
])
Mongo Playground

Related

MongoDB Aggregate - Match documents with id and give result which id not exist on collection B

Collection A:
[{
"_id": 1,
"operation":"SEC",
"name":"x"
},{
"_id": 2,
"operation": "SEC",
"name": "y"
},
{
"_id": 3,
"operation": "SEC",
"name": "z"
}]
Collection B:
[{
"user": 1,
"operation":"SEC",
"name":"x",
"date": "2022-10-25"
},{
"user": 2,
"operation":"SEC",
"name":"y",
"date": "2022-10-25"
}
]
Expected output:
[
{
"_id": 3,
"operation": "SEC",
"name": "z"
}
]
I have two collections and I want to match from the first collection to the second collection by date and want to get only those that are not in the second collection.
You can use the following aggregation pipeline in order to achieve your desired outpu:
[
{
"$lookup": {
"from": "collectionB",
"localField": "_id",
"foreignField": "user",
"as": "collectionB"
}
},
{
$match: {
collectionB: {
$size: 0
}
}
},
{
$project: {
collectionB: 0
}
}
]
Please note that this is an efficient solution. You probably should add a $match step at the beginning in order to limit your results.

Many To Many with embedded pivot list

I am trying to select all users which references a group that references a permission that has "valid" set to true (possibly multiple). To achieve this I am planning to use an aggregation with a lookup.
db={
"users": [
{
"_id": "1",
"groups": [
"2"
]
},
{
"_id": "2",
"groups": [
"1"
]
}
],
"group": [
{
"_id": "1",
"permissions": [
"12",
"3"
]
},
{
"_id": "2",
"permissions": [
"3",
"2"
]
}
],
"permission": [
{
"_id": "12",
"valid": true
},
{
"_id": "3",
"valid": true
},
{
"_id": "2",
"valid": true
}
]
}
I can't think of an efficient/simple way to do this.
I have 2 ideas:
have a lookup from user to group with a pipeline that does a lookup from group to permission. The problem there is I would need to do a $match to see if the user's group list contains the groups id (which I don't know if its possible)
do a simple lookup from user to group than unwind the joined field and do a second lookup using the joined field from group to permission. Than at the end do a $match to see if any contain "valid" true and group the results back to get the original users. This idea doesnt sound very efficient though. (https://mongoplayground.net/p/JB9Yb3a65Cn)
Any ideas or inputs on this?
you can use direct array of object without $unwind in lookup as localFields,
second lookup will replace group field with permissions response because it is not needed in next stage
db.users.aggregate([
{
$lookup: {
from: "group",
localField: "groups",
foreignField: "_id",
as: "group"
}
},
{
$lookup: {
from: "permission",
localField: "group.permissions",
foreignField: "_id",
as: "group"
}
},
{ $match: { "group.valid": true } },
{ $unset: "group" }
])
Playground

MongoDB add field with keys from object

I have the following two collections:
{
"organizations": [
{
"_id": "1",
"name": "foo",
"users": { "1": "admin", "2": "member" }
},
{
"_id": "2",
"name": "bar",
"users": { "1": "admin" }
}
],
"users": [
{
"_id": "1",
"name": "john smith"
},
{
"_id": "2",
"name": "bob johnson"
}
]
}
The following query works to merge the users into members when I just use an array of the user ids to match, however, the users prop is an object.
{
"collection": "organizations",
"command": "aggregate",
"query": [
{
"$lookup": {
"from": "users",
"localField": "users",
"foreignField": "_id",
"as": "members"
}
}
]
}
What I'm hoping to do is lookup by id then create a members array from the results with the user object including the role (value of the users objects:
{
"_id": "1",
"name": "foo",
"users": {
"1": "admin",
"2": "member"
},
"members": [
{
"_id": "1",
"name": "john smith",
"role": "admin"
},
{
"_id": "2",
"name": "bob johnson",
"role": "user"
}
]
}
Here's the sandbox I have setup: https://mongoplayground.net/p/yhRpeRvJf3u
You really need to change your schema design, this will cause the performance on retrieving data,
$addFields to add new field usersArray convert users object to array using $objectToArray, the format will be k(key) and v(value),
$lookup to join users collection, set localField name to usersArray.k
$addFields, remove usersArray field using $$REMOVE,
$map iterate loop of members array and $reduce to iterate loop of usersArray and get matching role as per _id and merge current fields and role field using $mergeObjects
db.organizations.aggregate([
{
$addFields: {
usersArray: {
$objectToArray: "$users"
}
}
},
{
"$lookup": {
"from": "users",
"localField": "usersArray.k",
"foreignField": "_id",
"as": "members"
}
},
{
$addFields: {
usersArray: "$$REMOVE",
members: {
$map: {
input: "$members",
as: "m",
in: {
$mergeObjects: [
"$$m",
{
role: {
$reduce: {
input: "$usersArray",
initialValue: "",
in: { $cond: [{ $eq: ["$$this.k", "$$m._id"] }, "$$this.v", "$$value"] }
}
}
}
]
}
}
}
}
}
])
Playground
First of all, the problem with your query is you want to use a KEY to do the $lookup, then the members field always gonna be empty.
You are trying to use users as local field, but users is an object, so you need the key (users.1, users.2, ... )
To do this you need to use $objectToArray, which create an object array with two fields: k and v for key and value. So now, you can $lookup with the field users.k.
To get the query you need $unwind before $lookup because you also want the users filed into the new document.
With the new object created using $objectToArray, you can do $unwind to get the values in differents documents. And then $lookup to get the "join".
Here, localField uses the value k created by $objectToArray (the object key).
After that, $set to add the field with the role and $group again into one document.
Ive used _id to get the values without changes between stages, and into members push the members in each collection.
And then, $project to output the values you want. In this case, tha calues "stored" into _id and the array members in "one level" using $reduce.
So, the query you need I think is this:
db.organizations.aggregate([
{
"$match": {
"_id": "1"
}
},
{
"$set": {
"usersArray": {
"$objectToArray": "$users"
}
}
},
{
"$unwind": "$usersArray"
},
{
"$lookup": {
"from": "users",
"localField": "usersArray.k",
"foreignField": "_id",
"as": "members"
}
},
{
"$set": {
"members.role": "$usersArray.v"
}
},
{
"$group": {
"_id": {
"_id": "$_id",
"users": "$users",
"name": "$name"
},
"members": {
"$push": "$members"
}
}
},
{
"$project": {
"members": {
"$reduce": {
"input": "$members",
"initialValue": [],
"in": {
"$concatArrays": [
"$$value",
"$$this"
]
}
}
},
"users": "$_id.users",
"name": "$_id.name",
"_id": "$_id._id"
}
}
])
Example here

How to find document that contains array with two equal values?

I have chats's collection with participants array
[{
"_id": ObjectId("5d12b2a10507cfe0bad6d93c"),
"participants": [{
"_id": ObjectId("5ce4af580507cfe0ba1c6f5b"),
"firstname": "John",
"lastname": "Anderson",
"icon": "/assets/images/avatars/small/2.jpg"
},
{
"_id": ObjectId("5ce4af580507cfe0ba1c6f5b"),
"firstname": "John",
"lastname": "Anderson",
"icon": "/assets/images/avatars/small/2.jpg"
}
]
}, {
"_id": ObjectId("5d1124a50507cfe0baba7909"),
"participants": [{
"_id": ObjectId("5ce4af580507cfe0ba1c6f5b"),
"firstname": "John",
"lastname": "Anderson",
"icon": "/assets/images/avatars/small/2.jpg"
},
{
"_id": ObjectId("5ce54cb80507cfe0ba25d74b"),
"firstname": "Milosh",
"lastname": "Jersi",
"icon": "/assets/images/avatars/small/3.jpg"
}
]
}]
I fetch it by
req.db.collection('chats').findOne({'participants._id': {$all: [req.userID, new mongo.ObjectID(req.params.to)]}});
where userID is also ObjectID and equals.
Usually it have different participants, but our user can also send messages to itself, is allowed option in many social networks.
So in this situation, our user "John Anderson" sent message to himself and we inserted chat document for it.
And now i have problem, how to get document with equal array values
{'participants._id': { '$all': [ 5ce4af580507cfe0ba1c6f5b, 5ce4af580507cfe0ba1c6f5b] }}
// return every chat contains our id in atleast one field, but we need both to be equal
// same for $in
{'participants._id': { '$eq': [ 5ce4af580507cfe0ba1c6f5b, 5ce4af580507cfe0ba1c6f5b] }}
// return nothing
what else can I do ?
you can achieve this with the aggregation framework, using a $group stage. First, group by chat._id and use $addToSet to keep only unique users in a new array, ant then add a filter to keep only the documents with one participant:
db.collection.aggregate([
{
"$unwind": "$participants"
},
{
"$group": {
"_id": "$_id",
"participants": {
"$addToSet": "$participants._id"
}
}
},
{
"$match": {
"participants": {
"$size": 1
}
}
}
])
result:
[
{
"_id": ObjectId("5d12b2a10507cfe0bad6d93c"),
"participants": [
ObjectId("5ce4af580507cfe0ba1c6f5b")
]
}
]
you can try it online: mongoplayground.net/p/plB-gsNIxRd

How to combine multiple collections and merge the joined documents into a single document

Here is the problem, I am unable to get the following result. Please look into the piece of json and help me out.
This is my data:
[
{
"user_id": "65asdfksadjfk3u4",
"lat": 23.4343,
"long": 15.2382
}
]
Currently my result is:
[
{
"_id": "65asdfksadjfk3u4",
"name": "Srini",
"age": 26,
"some other key": "some other values"
}
]
I need to get the collection from the user_id and add it to the same array object. As you can notice both lat and long are being removed in my current result.
[
{
"_id": "65asdfksadjfk3u4",
"name": "Srini",
"age": 26,
"some other keys": "some other values",
"lat": 23.4343,
"long": 15.2382
}
]
You can append the $lookup stage to join the current pipeline results with the users collections by the user_id fields and then use $mergeObjects in the $replaceRoot to merge the joined documents from users and the current results:
db.collection.aggregate([
/* current pipeline here */
{ "$lookup": {
"from": "users",
"localField": "_id",
"foreignField": "user_id",
"as": "user"
} },
{ "$replaceRoot": {
"newRoot": {
"$mergeObjects": [
{ "$arrayElemAt": [ "$user", 0 ] },
"$$ROOT"
]
}
} },
{ "$project": { "user": 0, "user_id": 0 } }
]);