MongoDB - query references 2 deep of ObjectIDs - mongodb

I've inherited a Azure Cosmos database with a MongoDB API. There is extensive use of "discriminators" so a single collection has many different models.
I am trying to query a document three levels deep based on document ids (ObjectId())
Parent Group
{
_id: ObjectId(),
__type: "ParentGroup",
name: "group 1",
subgroups: [
...ObjectIds,
],
}
Sub Group
{
_id: ObjectId(),
__type: "SubGroup",
name: "a text name",
members: [
...ObjectIds,
],
}
Member
{
_id: ObjectId(),
__type: "Member",
name: "string",
email: "",
induction: Date,
}
Examples I've seen deal with nested documents NOT references
Is it possible to query the Member documents and return?
[
{
parentGroup,
subgroups: [
{sub group, members: [...members]},
{sub group, members: [...members]},
{sub group, members: [...members]},
]
},
]
After reading the comments and further reading i've got this. Its almost there but I think the limitation of MongoDB will prevent the solution being in a single query. The goal is to return ParentGroups->Subgroups->Members Where Members have an "induction" value of "whatever". I am either returning ALL ParentGroups or nothing at all
db.development.aggregate([
{
$match: {
__type: "ParentGroup", $expr: {
$gt: [
{ $size: "$subgroups" }, 0
]
}
}
},
{
$lookup: {
from: "development",
localField: "subgroups",
foreignField: "_id",
as: "subgroups"
}
},
{
$unwind: {
path: "$subgroups",
// preserveNullAndEmptyArrays: true
}
},
{
$lookup: {
from: "development",
localField: "subgroups.members",
foreignField: "_id",
as: "subgroups.members"
}
}
])

Solution that worked for me:
db.development.aggregate([
{
$match: {
__type: "ParentGroup",
},
},
{
$lookup: {
from: "development",
localField: "subgroups",
foreignField: "_id",
as: "subgroups",
},
},
{
$unwind: {
path: "$subgroups",
preserveNullAndEmptyArrays: true,
},
},
{
$lookup: {
from: "development",
localField: "subgroups.members",
foreignField: "_id",
as: "subgroups.activities_x",
},
},
{
$unwind: {
path: "$subgroups.members",
preserveNullAndEmptyArrays: true,
},
},
{
$match: { "subgroups.members.meta": { $exists: true } },
},
{
$project: {
_id: 1,
__type: 1,
name: 1,
subgroups: {
_id: 1,
__type: 1,
name: 1,
members: {
_id: 1,
__type: 1,
name: 1,
meta: 1,
},
},
},
},
]);

Related

Mongodb lookup for array of ids with nested array of objects in other collection

I am new to mongo db i am trying to find a lookup for two collection
one collection is users which has tags like
{
_id: "fdkjkjs",
first_name: "",
last_name: "",
role: "admin",
tags:
[
{ _id: "tag_1_id", name: "Tag 1" },
{ _id: "tag_2_id", name: "Tag 2" },
{ _id: "tag_3_id", name: "Tag 3" },
{ _id: "tag_4_id", name: "Tag 4" }
]
}
and a post collection is as below
{
_id: "fdkjkjs",
title: "",
slug: "",
tags: ["tag_1_id", tag_3_id]
}
So I want to get all the tags in post list API with the names that are in users collection.
so result i wanted belike
[{
_id: "fdkjkjs",
title: "",
slug: "",
tags: ["tag_1_id", tag_3_id],
selectedTags: [
{ _id: "tag_1_id", name: "Tag 1" },
{ _id: "tag_3_id", name: "Tag 3" }
],
}]
Method 1
db.posts.aggregate([
{
"$lookup": {
"from": "users",
"localField": "tags",
"foreignField": "tags._id",
"as": "selectedTags"
}
},
{
"$set": {
"selectedTags": {
"$filter": {
"input": { "$first": "$selectedTags.tags" },
"as": "item",
"cond": { $in: [ "$$item._id", "$tags" ] }
}
}
}
}
])
mongoplayground
Method 2
db.posts.aggregate([
{
$lookup: {
from: "users",
let: { tags_post: "$tags" },
pipeline: [
{
"$unwind": "$tags"
},
{
$match: {
$expr: {
$in: [ "$tags._id", "$$tags_post" ]
}
}
},
{
"$replaceWith": "$tags"
}
],
as: "selectedTags"
}
}
])
mongoplayground
Method 3
db.posts.aggregate([
{
$lookup: {
from: "users",
localField: "tags",
foreignField: "tags._id",
let: { tags_post: "$tags" },
pipeline: [
{
"$unwind": "$tags"
},
{
$match: {
$expr: {
$in: [ "$tags._id", "$$tags_post" ]
}
}
},
{
"$replaceWith": "$tags"
}
],
as: "selectedTags"
}
}
])
mongoplayground
Method 2 arrange data first and then lookup, while method 3 lookup first and then arrange data. Though 2 and 3 looks similar, I think method 3 is faster than method 2.

MongoDB: Optimal joining of one to many relationship

Here is a hypothetical case of orders and products.
'products' collection
[
{
"_id": "61c53eb76eb2dc65de621bd0",
"name": "Product 1",
"price": 80
},
{
"_id": "61c53efca0a306c3f1160754",
"name": "Product 2",
"price": 10
},
... // truncated
]
'orders' collection:
[
{
"_id": "61c53fb7dca0579de038cea8", // order id
"products": [
{
"_id": "61c53eb76eb2dc65de621bd0", // references products._id
"quantity": 1
},
{
"_id": "61c53efca0a306c3f1160754",
"quantity": 2
},
]
}
]
As you can see, an order owns a list of product ids. When I pull an order's details I also need the product details combined like so:
{
_id: ObjectId("61c53fb7dca0579de038cea8"),
products: [
{
_id: ObjectId("61c53eb76eb2dc65de621bd0"),
quantity: 1,
name: 'Product 1',
price: 80
},
{
_id: ObjectId("61c53efca0a306c3f1160754"),
quantity: 2,
name: 'Product 2',
price: 10
},
... // truncated
]
}
Here is the aggregation pipleline I came up with:
db.orders.aggregate([
{
$match: {_id: ObjectId('61c53fb7dca0579de038cea8')}
},
{
$unwind: {
path: "$products"
}
},
{
$lookup: {
from: 'products',
localField: 'products._id',
foreignField: '_id',
as: 'productDetail'
}
},
{
$unwind: {
path: "$productDetail"
}
},
{
$group: {
_id: "$_id",
products: {
$push: {$mergeObjects: ["$products", "$productDetail"]}
}
}
}
])
Given how the data is organized I'm doubting if the pipeline stages are optimal and could do better (possibility of reducing the number of stages, etc.). Any suggestions?
As already mentioned in comments the design is poor. You can avoid multiple $unwind and $group, usually the performance should be better with this:
db.orders.aggregate([
{ $match: { _id: "61c53fb7dca0579de038cea8" } },
{
$lookup: {
from: "products",
localField: "products._id",
foreignField: "_id",
as: "productDetail"
}
},
{
$project: {
products: {
$map: {
input: "$products",
as: "product",
in: {
$mergeObjects: [
"$$product",
{
$first: {
$filter: {
input: "$productDetail",
cond: { $eq: [ "$$this._id", "$$product._id" ] }
}
}
}
]
}
}
}
}
}
])
Mongo Playground

MongoDB aggregate returning only specific fields

I have the following code:
const profiles = await Profile.aggregate([
{
$lookup: {
from: "users",
localField: "user",
foreignField: "_id",
as: "user",
},
},
{
$unwind: "$user",
},
{
$match: {
"user.name": {
$regex: q.trim(),
$options: "i",
},
},
},
{
$skip: req.params.page ? (req.params.page - 1) * 10 : 0,
},
{
$limit: 11,
},
{
$group: {
_id: "$_id",
skills:{skills}
user: { name: "$name" },
user: { avatar: "$avatar" },
},
},
]);
I want to return only specific fields like skills _id and user.name and user.avatar, but this doesn't work. I'm pretty sure that the problem is in $group. I want to receive only these fields
[
{
_id: 5ef78d005d23020ca847aa76,
skills: [ 'asd' ],
user: {
_id: 5ef78c7c5d23020ca847aa75,
name: 'Simeon Lazarov',
avatar: 'uploads\\1593286096227 - background.jpg',
}
}
]
You can make use of $project to get specific fields.
After grouping add the below:
{
$project: {_id:1, skills:1, user:1}
}
Projection value of 0 means that the field needs to be excluded, Value 1 represents inclusion of the field.
Document reference: https://docs.mongodb.com/manual/reference/operator/aggregation/project/

mongodb - summations of array length with same ids

I am creating a platform where people can share their memes. On one page I want to show them who are the most popular members on the platform. so, there is a collection of 'meme' and 'user'
for example,
There is two content with same ids:
{
_id: 1,
username: "name",
bio: "bio",
image: "url",
};
memes
{
_id: 0,
user_id: 1,
image: "meme1.jpg",
likes: [
{
user_id: 4
}
]
},
{
_id: 1,
user_id: 1,
image: "meme2.jpg",
likes: [
{
user_id: 5
},
{
user_id: 6
}
]
}
and I want to output something like this way
{
user_id:1,
username:"name"
likes:3,
}
I wrote this query using aggregate functions but I am not understanding how to identify ids are the same or not?
meme
.aggregate([
{
$lookup: {
from: "users",
localField: "user_id",
foreignField: "_id",
as: "userDetails",
},
},
{
$project: {
user_id: "$user_id",
username: "$userDetails.username",
likes: {
$size: "$likes",
},
},
},
{
$sort: { likes: 1 },
},
])
.exec()
.then((result) => {
console.log(result);
});
It will be easier to start query with users.
You can use $sum, $map, $size aggregations to get the total likes, and add it using $addFields.
db.users.aggregate([
{
$lookup: {
from: "memes",
localField: "_id",
foreignField: "user_id",
as: "userDetails"
}
},
{
$addFields: {
"likes": {
"$sum": {
"$map": {
"input": "$userDetails",
"in": {
"$size": "$$this.likes"
}
}
}
}
}
},
{
$project: {
_id: 0,
user_id: "$_id",
username: 1,
likes: 1
}
}
])
Playground
Result:
[
{
"likes": 3,
"user_id": 1,
"username": "name"
}
]
You could project the length of the likes-array and group each projection by the user_id and cound the results. Something like this should work:
db.getCollection('memes').aggregate([{
$lookup: {
from: "users",
localField: "user_id",
foreignField: "_id",
as: "userDetails"
}
}, {
"$project": {
"user_id": 1,
"likesSize": {
"$size": "$likes"
}
}
}, {
$group: {
_id: "$user_id",
"count": {
"$sum": "$likesSize"
}
}
}
])
The above query should return:
{
"_id" : 1,
"count" : 3
}

mongodb how to use aggregate like populate

The code example is in Mongo Playground
https://mongoplayground.net/p/W4Qt4oX0ZRP
Assume the following documents
[
{
_id: "5df1e6f75de2b22f8e6c30e8",
user: {
name: "Tom",
sex: 1,
age: 23
},
dream: [
{
label: "engineer",
industry: "5e06b16fb0670d7538222909",
type: "5e06b16fb0670d7538222951",
},
{
label: "programmer",
industry: "5e06b16fb0670d7538222909",
type: "5e06b16fb0670d7538222951",
}
],
works: [
{
name: "any engineer",
company: "5dd7fd51b0ae1837a08d00c8",
skill: [
"5dc3998e2cf66bad16efd61b",
"5dc3998e2cf66bad16efd61e"
],
},
{
name: "any programmer",
company: "5dd7fd9db0ae1837a08d00e2",
skill: [
"5dd509e05de2b22f8e67e1b7",
"5dd509e05de2b22f8e67e1bb"
],
}
]
}
]
I tried to use aggregate $lookup $unwind
db.coll.aggregate([
{
$unwind: {
path: "$dream",
}
},
{
$lookup: {
from: "industry",
localField: "dream.industry",
foreignField: "_id",
as: "dream.industry"
},
},
{
$unwind: {
path: "$dream.industry",
}
},
{
$lookup: {
from: "type",
localField: "dream.type",
foreignField: "_id",
as: "dream.type"
},
},
{
$unwind: {
path: "$dream.type",
}
},
{
$unwind: {
path: "$works",
}
},
{
$lookup: {
from: "company",
localField: "works.company",
foreignField: "_id",
as: "works.company"
},
},
{
$unwind: {
path: "$works.company",
}
},
{
$lookup: {
from: "skill",
localField: "works.skill",
foreignField: "_id",
as: "works.skill"
},
},
])
Executing the above code did not get the desired result!
This is what i expect
{
_id: "5df1e6f75de2b22f8e6c30e8",
user: {
name: 'Tom',
sex: 1,
age: 23
},
dream: [
{
label: 'engineer',
industry: {
_id: "5e06b16fb0670d7538222909", // Industry doc _id
name: 'IT',
createdAt: "2019-12-28T01:35:44.070Z",
updatedAt: "2019-12-28T01:35:44.070Z"
},
type: {
_id: "5e06b16fb0670d7538222951", // Type doc _id
name: 'job',
createdAt: "2019-12-28T01:35:44.070Z",
updatedAt: "2019-12-28T01:35:44.070Z"
},
},
{
label: 'programmer',
industry: {
_id: "5e06b16fb0670d7538222909", // Industry doc _id
name: 'IT',
createdAt: "2019-12-28T01:35:44.070Z",
updatedAt: "2019-12-28T01:35:44.070Z"
},
type: {
_id: "5e06b16fb0670d7538222951", // Type doc _id
name: 'job',
createdAt: "2019-12-28T01:35:44.070Z",
updatedAt: "2019-12-28T01:35:44.070Z"
}
}
],
works: [
{
name: 'any engineer',
company: {
_id: "5dd7fd51b0ae1837a08d00c8", // Company doc _id
name: 'alibaba',
area: 'CN',
},
skill: [
{
_id: "5dc3998e2cf66bad16efd61b", // Skill doc _id
name: 'Java'
},
{
_id: "5dc3998e2cf66bad16efd61e", // Skill doc _id
name: 'Php'
},
]
},
{
name: 'any programmer',
company: {
_id: "5dd7fd9db0ae1837a08d00e2", // Company doc _id
name: 'microsoft',
area: 'EN',
},
skill: [
{
_id: "5dd509e05de2b22f8e67e1b7", // Skill doc _id
name: 'Golang'
},
{
_id: "5dd509e05de2b22f8e67e1bb", // Skill doc _id
name: 'Node.js'
}
]
},
]
}
The expected result is dream is an array, works is an array, and dream.industry changed from ObjectId to document, dream.type changed from ObjectId to document, works.company changed from ObjectId to document
When I use populate, I can do it easily
Model.find()
.populate('dream.industry')
.populate('dream.type')
.populate('works.company')
.populate('works.skill')
.lean()
I refer to the following questions
mongoose aggregate lookup array (Almost the same as my question, But not resolved)
$lookup on ObjectId's in an array
hope to get everyone's help, thank you!
To make it easier i would not change the current pipeline but just add a $group stage to end of it in order to re-structure the data.
{
$group: {
_id: "$_id",
user: {$first: "$user"},
dream: {$addToSet: "$dream"},
works: {$addToSet: "$works"}
}
}
With that said if you are using Mongo version 3.6+ i do recommend you use the "newer" version of $lookup to re-write your pipeline to be a bit more efficient by avoiding all these $unwind's.