I have a database with some users, who belong to teams. Each team has a leader. Each user has a subject.
I want to collate teams by the leader's subject.
My data looks like this:
db={
"teams": [
{
_id: "t1",
members: [
{
"_id": "u1",
"leader": true
},
{
"_id": "u2"
},
{
"_id": "u3"
}
],
},
{
_id: "t2",
members: [
{
"_id": "u2",
"leader": true
},
{
"_id": "u4"
}
],
},
{
_id: "t3",
members: [
{
"_id": "u1",
"leader": true
},
{
"_id": "u4"
}
],
},
{
_id: "t4",
members: [
{
"_id": "u2",
"leader": true
}
],
},
],
"users": [
{
"_id": "u1",
"subject": "history"
},
{
"_id": "u2",
"subject": "maths"
},
{
"_id": "u3",
"subject": "geography"
},
{
"_id": "u4",
"subject": "french"
}
]
}
The result I want is:
{
"history": ["t1", "t3"],
"maths": ["t2", "t4"]
}
I have an aggregation that gets me the _id of every leader, and from there I can get the result I want in stages, by first finding the subject of every leader, then going back through the projects and assigning a subject to each project based on the identify of the leader. It works but it is inelegant and I think it will be slow. It seems to me there should be some better way to do this, maybe something like a join?
Is there a nifty way to get the result I want from a single MongoDB operation?
Here is a Mongo Playground with my data:
https://mongoplayground.net/p/SIJv9-hVNzJ
Many thanks for any help.
Edit: my test data are confusing because '_id' is used in both collections, making it hard to unpack the answer. Here is an updated Mongo Playground that uses different key names for each collection and helped me to understand the perfect answer.
Yes, you should join your collections on users._id with a $lookup, and then transform value to key with $arrayToObject (introduced in Mongodb 3.4.4)
Here is a possible way to do this :
db.teams.aggregate([
{
"$unwind": "$members"
},
{
"$match": {
"members.leader": true
}
},
{
"$lookup": {
"from": "users",
"localField": "members._id",
"foreignField": "_id",
"as": "users"
}
},
{
"$unwind": "$users"
},
{
"$group": {
"_id": "$users.subject",
"team": {
"$push": "$_id"
}
}
},
{
"$replaceRoot": {
"newRoot": {
"$arrayToObject": [
[
{
k: "$_id",
v: "$team"
}
]
]
}
}
}
])
try it online: mongoplayground.net/p/TuEpMzHkI-0
Related
I have a two collections.
OrgStructure (visualise this as a tree structure)
Example Document:
{
"id": "org1",
"nodes": [
{
"nodeId": "root",
"childNodes": ["child1"]
},
{
"nodeId": "child1",
"childNodes": ["child2"]
},
{
"nodeId": "child2",
"childNodes": []
}
]
}
Activity
Example Document:
[
{
"id":"A1",
"orgUnit": "root"
},
{
"id":"A2",
"orgUnit": "child1"
},
{
"id":"A3",
"orgUnit": "child2"
}
]
Now my expectation is to group activities by orgUnit such a way that by considering the child nodes as well.
Here i don't want to do a lookup and i need to consider one OrgStructure document as an input, so that i can construct some condition using the document such a way that the query will return the below result.
Expected result
[
{
"_id": "root",
"activities": ["A1","A2","A3"]
},
{
"_id": "child1",
"activities": ["A2","A3"]
},
{
"_id": "child2",
"activities": ["A3"]
}
]
So im ecpecting an aggregate query something like this
{
"$group": {
"_id": {
"$switch": {
"branches": [
{
"case": {"$in": ["$orgUnit",["root","child1","child2"]]},
"then": "root"
},
{
"case": {"$in": ["$orgUnit",["child1","child2"]]},
"then": "child1"
},
{
"case": {"$in": ["$orgUnit",["child2"]]},
"then": "child2"
}
],
"default": null
}
}
}
}
Thanks in advance!
You will need 2 steps:
create another collection nodes for recursive lookup. The original OrgStructure is hard to perform $graphLookup
db.OrgStructure.aggregate([
{
"$unwind": "$nodes"
},
{
"$replaceRoot": {
"newRoot": "$nodes"
}
},
{
$out: "nodes"
}
])
Perform $graphLookup on nodes collection to get all child nodes. Perform $lookup to Activity and do some wrangling.
db.nodes.aggregate([
{
"$graphLookup": {
"from": "nodes",
"startWith": "$nodeId",
"connectFromField": "childNodes",
"connectToField": "nodeId",
"as": "nodesLookup"
}
},
{
"$lookup": {
"from": "Activity",
"let": {
nodeId: "$nodesLookup.nodeId"
},
"pipeline": [
{
$match: {
$expr: {
$in: [
"$orgUnit",
"$$nodeId"
]
}
}
},
{
$group: {
_id: "$id"
}
}
],
"as": "activity"
}
},
{
$project: {
_id: "$nodeId",
activities: "$activity._id"
}
}
])
Here is the Mongo playground for your reference.
I have a query as below, what it does it creates a link between two documents and find the last order date and users details like email, phone, etc. but on large data set it shows me timeout error any help would be much appreciated, and thanks in advance for the help
db.users.aggregate([
{
"$lookup": {
"from": "orders",
"let": {
"id": "$_id"
},
"pipeline": [
{
"$addFields": {
"owner": {
"$toObjectId": "$owner"
}
}
},
{
"$match": {
$expr: {
$eq: [
"$owner",
"$$id"
]
}
}
},
],
"as": "orders"
}
},
{
"$unwind": {
path: "$orders",
preserveNullAndEmptyArrays: false,
includeArrayIndex: "arrayIndex"
}
},
{
"$group": {
"_id": "$_id",
"order": {
"$last": "$orders.createdAt"
},
"userInfo": {
"$mergeObjects": {
name: "$name",
email: "$email",
phone: "$phone",
orderCount: "$orderCount"
}
}
}
},
{
"$project": {
name: "$userInfo.name",
email: "$userInfo.email",
phone: "$userInfo.phone",
orderCount: "$userInfo.orderCount",
lastOrder: "$order",
}
}
]
)
my documents look like the following for orders
{
"_id": ObjectId("607fbeeb0a752a66a7af40eb"),
"address": {
"loc": [
-1,
3
],
"_id": "5d35d55d3d081f486d0d401c",
"apartment": "",
"description": "ACcdg dfef"
},
"approvedAt": ISODate("2021-04-21T11:28:05.295+05:30"),
"assignedAt": null,
"billingAddress": {
"description": ""
},
"createdAt": ISODate("2021-04-21T11:28:04.449+05:30"),
"creditCard": "",
"deliveryDate": "04/21/21",
"deliveryDateObj": ISODate("2021-04-21T12:27:58.746+05:30"),
"owner": "609bd5831b912947ea51a9ac",
"products": [
"5a070c079b"
],
"updatedAt": ISODate("2021-04-21T11:28:05.295+05:30"),
}
and for users, it is like below
{
"_id": ObjectId("609bd5831b912947ea51a9ac"),
"updatedAt": ISODate("2021-05-12T18:47:55.291+05:30"),
"createdAt": ISODate("2021-05-12T18:47:55.213+05:30"),
"email": "1012#gmail.com",
"phone": "123",
"dob": "1996-04-10",
"password": "",
"stripeID": "",
"__t": "Customer",
"name": {
"first": "A",
"last": "b"
},
"orderCount": 1,
"__v": 0,
"forgottenPassword": ""
}
convert _id to string in lookup's let and you can remove $addFields from lookup pipeline
add $project stage in lookup pipeline and show only required fields
$project to show required fields and get last / max createdAt date use $max, you don't need to $unwind and $group operation
db.users.aggregate([
{
$lookup: {
from: "orders",
let: { id: { $toString: "$_id" } },
pipeline: [
{ $match: { $expr: { $eq: ["$owner", "$$id"] } } },
{
$project: {
_id: 0,
createdAt: 1
}
}
],
"as": "orders"
}
},
{
$project: {
email: 1,
name: 1,
orderCount: { $size: "$orders" },
phone: 1,
lastOrder: { $max: "$orders.createdAt" }
}
}
])
Playground
SUGGESTION:
You can save owner id in orders as objectId instead of string and whenever new order arrive store it as objectId, you can prevent conversation operator $toString operation
create an index in owner field to make lookup process faster.
I have figured out that after using createIndex for the owner field which is used to compare the owner in the orders from the users _id filed, so just after adding an db.orders.createIndex({ owner: 1 }), the query will run much faster and smoother
I have 3 collections.
User:
{
"_id":ObjectId("60a495cdd4ba8b122899d415"),
"email":"br9#gmail.com",
"username":"borhan"
}
Panel:
{
"_id": ObjectId("60a495cdd4ba8b122899d417"),
"name": "borhan",
"users": [
{
"role": "admin",
"joined": "2021-05-19T04:35:47.474Z",
"status": "active",
"_id": ObjectId("60a495cdd4ba8b122899d418"),
"user": ObjectId("60a495cdd4ba8b122899d415")
},
{
"role": "member",
"joined": "2021-05-19T04:35:47.474Z",
"status": "active",
"_id": ObjectId("60a49600d4ba8b122899d41a"),
"user": ObjectId("60a34e167958972d7ce6f966")
}
],
}
Team:
{
"_id":ObjectId("60a495e0d4ba8b122899d419"),
"title":"New Teams",
"users":[
ObjectId("60a495cdd4ba8b122899d415")
],
"panel":ObjectId("60a495cdd4ba8b122899d417")
}
I want to receive a output from querying Panel colllection just like this:
{
"_id": ObjectId("60a495cdd4ba8b122899d417"),
"name": "borhan",
"users": [
{
"role": "admin",
"joined": "2021-05-19T04:35:47.474Z",
"status": "active",
"_id": ObjectId("60a495cdd4ba8b122899d418"),
"user": ObjectId("60a495cdd4ba8b122899d415"),
"teams":[
{
"_id":ObjectId("60a495e0d4ba8b122899d419"),
"title":"New Teams",
"users":[
ObjectId("60a495cdd4ba8b122899d415")
],
"panel":ObjectId("60a495cdd4ba8b122899d417")
}
]
},
{
"role": "member",
"joined": "2021-05-19T04:35:47.474Z",
"status": "active",
"_id": ObjectId("60a49600d4ba8b122899d41a"),
"user": ObjectId("60a34e167958972d7ce6f966")
}
],
}
I mean i want to add teams field (which is array of teams that user is existed on it) to each user in Panel collection
Here is my match query in mongoose to select specific panel:
panel_model.aggregate([
{
$match: {
users: {
$elemMatch: {user: ObjectId("60a495cdd4ba8b122899d415"), role:"admin"}
}
}
},
])
Is it possible to get my output with $lookup or $addFields aggregations?
You need to join all three collections,
$unwind to deconstruct the array
$lookup there are two kind of lookups which help to join collections. First I used Multiple-join-conditions-with--lookup, and I used standrad lookup to join Users and Teams collections.
$match to match the user's id
$expr - when you use $match inside lookup, u must use it.
$set to add new fields
$group to we already destructed using $unwind. No we need to restructure it
here is the code
db.Panel.aggregate([
{ $unwind: "$users" },
{
"$lookup": {
"from": "User",
"let": { uId: "$users.user" },
"pipeline": [
{
$match: {
$expr: {
$eq: [ "$_id", "$$uId" ]
}
}
},
{
"$lookup": {
"from": "Team",
"localField": "_id",
"foreignField": "users",
"as": "teams"
}
}
],
"as": "users.join"
}
},
{
"$set": {
"users.getFirstElem": {
"$arrayElemAt": [ "$users.join", 0 ]
}
}
},
{
$set: {
"users.teams": "$users.getFirstElem.teams",
"users.join": "$$REMOVE",
"users.getFirstElem": "$$REMOVE"
}
},
{
"$group": {
"_id": "$_id",
"name": { "$first": "name" },
"users": { $push: "$users" }
}
}
])
Working Mongo playground
Note : Hope the panel and user collections are in 1-1 relationship. Otherwise let me know
I have two collections name listings and moods.
listings sample:
{
"_id": ObjectId("5349b4ddd2781d08c09890f3"),
"name": "Hotel Radisson Blu",
"moods": [
ObjectId("507f1f77bcf86cd799439010"),
ObjectId("507f1f77bcf86cd799439011")
]
}
moods sample:
{
"_id": ObjectId("507f1f77bcf86cd799439011"),
"name": "Sports"
},
{
"_id": ObjectId("507f1f77bcf86cd799439010"),
"name": "Spanish Food"
},
{
"_id": ObjectId("507f1f77bcf86cd799439009"),
"name": "Action"
}
I need this record.
{
"_id": ObjectId("507f1f77bcf86cd799439011"),
"name": "Sports",
"count": 1
},
{
"_id": ObjectId("507f1f77bcf86cd799439010"),
"name": "Spanish Food",
"count": 1
},
{
"_id": ObjectId("507f1f77bcf86cd799439009"),
"name": "Action",
"count": 0
}
I need this type of record. I have no idea about aggregate.
You can do it using aggregate(),
$lookup to join collection listings
$match pipeline to check moods _id in listings field moods array
db.moods.aggregate([
{
"$lookup": {
"from": "listings",
"as": "count",
let: { id: "$_id" },
pipeline: [
{
"$match": {
"$expr": { "$in": ["$$id", "$moods"] }
}
}
]
}
},
$addFields to add count on the base of $size of array count that we got from above lookup
{
$addFields: {
count: { $size: "$count" }
}
}
])
Playground
did this work:
db.collection.aggrate().count()
Try to combine the functions, it might work.
I have aggregation pipeline stage:
$project: {
'school': {
'id': '$_id',
'name': '$name',
'manager': '$manager'
},
'students': '$groups.students',
'teachers': '$groups.teachers'
}
Need something like this:
{
'users': // manager + students + teachers
}
Tried:
{
'users': {
$push: {
$each: ['$school.manager', '$students', '$teachers']
}
}
}
I'm presuming that "students" and "teachers" are both arrays here and located under a common sub-document heading like so:
{
"_id": 123,
"name": "This school",
"manager": "Bill"
"groups": {
"teachers": ["Ted"],
"students": ["Missy"]
}
}
So in order to get all of those in a singular array such as "users" then it depends on your MongoDB version and the "uniqueness" of your data. For true "sets" and where you have MongoDB 2.6 or greater available, there is the $setUnion operator, albeit with an additional level of $group to make "manager" and array:
db.collection.aggregate([
{ "$group": {
"_id": { "_id": "$_id", "name": "$name" },
"manager": { "$push": "$manager" },
"groups": { "$first": "$groups" }
}},
{ "$project": {
"users": {
"$setUnion": [ "$manager", "$groups.teachers", "$groups.students" ]
}
}}
])
Or otherwise where that operator is not available or there is a "unique" problem then there is this way to handle "combining":
db.collection.aggregate([
{ "$group": {
"_id": { "_id": "_id", "name": "$name" },
"manager": { "$push": "$manager" },
"teachers": { "$first": "$groups.teachers" },
"students": { "$first": "$groups.students" },
"type": { "$first": { "$const": ["M","T","S"] } }
}},
{ "$unwind": "$type" },
{ "$project": {
"users": {
"$cond": [
{ "$eq": [ "$type", "M" ] },
"$manager",
{ "$cond": [
{ "$eq": [ "$type", "T" ] },
"$teachers",
"$students"
]}
]
}
}},
{ "$unwind": "$users" },
{ "$group": {
"_id": "$_id",
"users": { "$push": "$users" }
}}
])
This essentially "tags" each field by a "type" for which the document is copied in the pipeline. Then placed into a single "users" field depending on which "type" matched. The single array then from the resulting three documents from each original can then be safely "unwound" and combined in a final $group operation.
So "sets" are your fastest option where available or where not available or not unique you can use the later technique in order to combine these to a single list.