How to $lookup/populate an embedded document that is inside an array?
Below is how my schema is looking like.
const CommentSchema = new mongoose.Schema({
commentText:{
type:String,
required: true
},
arrayOfReplies: [{
replyText:{
type:String,
required: true
},
replier: [{
type: mongoose.Schema.Types.ObjectId,
ref: 'User',
required: true,
}],
}],
});
How can I get query results that look like below:
[
{
commentText: 'comment text',
arrayOfReplies: [
{
replyText: 'replyText',
replier: {
username:"username"
bio: 'bio'
}
}
]
}
]
I am trying to populate the replier field inside the array arrayOfReplies. I have tried several variations of the aggregation query below. The ones that have come close to what I am trying to achieve have one short-coming. The comments that do not have replies have an arrayOfReplies array that has an empty object. I.e arrayOfReplies: [{}], essentially meaning that the array is not empty.
I have tried using add fields, $mergeObjects among other pipeline operators but to no avail.
How to $lookup/populate the replier document that is inside the arrayOfReplies array?
Below is a template of the main part of my aggregation query, minus trying populate the replier document.
Comment.aggregate([
{$unwind: {"path": '$arrayOfReplies', "preserveNullAndEmptyArrays": true }},
{$lookup:{from:"users",localField:"$arrayOfReplies.replier",foreignField:"_id",as:"replier"}},
{$unwind: {"path": "$replier", "preserveNullAndEmptyArrays": true }},
{$group: {
_id : '$_id',
commentText:{$first: '$commentText'},
userWhoPostedThisComment:{$first: '$userWhoPostedThisComment'},
arrayOfReplies: {$push: '$arrayOfReplies' },
}},
After your lookup stage, each document will have
{
commentText: "text",
arrayOfReplies: <single reply, with replier ID>
replier: [<looked up replier data>]
}
Use an $addFields stage to move that replier data inside the reply object before the group, like:
{$addFields: {"arrayOfReplies.replier":"$replier"}}
Then your group stage will rebuild arrayOfReplies like you want.
You can use the following aggregate:
Playground
Comment.aggregate([
{
$unwind: {
"path": "$arrayOfReplies",
"preserveNullAndEmptyArrays": true
}
},
{
$lookup: {
from: "users",
localField: "arrayOfReplies.replier",
foreignField: "_id",
as: "replier"
}
},
{
$addFields: {
"arrayOfReplies.replier": {
$arrayElemAt: [
"$replier",
0
]
}
}
},
{
$project: {
"replier": 0
}
},
{
$group: {
_id: "$_id",
"arrayOfReplies": {
"$push": "$arrayOfReplies"
},
commentText: {
"$first": "$commentText"
}
}
}
]);
All the answers provided did not solve this issue as stated in the question.
I am trying to populate the replier field inside the array
arrayOfReplies. I have tried several variations of the aggregation
query below. The ones that have come close to what I am trying to
achieve have one short-coming. The comments that do not have replies
have an arrayOfReplies array that has an empty object. I.e
arrayOfReplies: [{}], essentially meaning that the array is not empty.
I wanted an aggregation that returns an empty array (not an array with an empty object) when the array is empty.
I was able to achieve what I wanted by using the code below:
arrayOfReplies:
{$cond:{
if: { $eq: ['$arrayOfReplies', {} ] },
then: "$$REMOVE",
else: {
_id : '$arrayOfReplies._id',
replyText:'$arrayOfReplies.replyText',
}
}}
If you combine the code above with #SuleymanSah's answer you get the full working code.
Related
So I have 2 collections (tenants and campaigns) and I'm trying to compose a query to return 1 tenant and 1 campaign. As an input, there is a tenant domain and campaign slug. Since I first need the tenant _id to query the campaign (based on both tenantId and slug), aggregation seems more performative option (than making 2 consecutive queries).
Technically speaking, I know how to do that:
[{
$match: { 'domains.name': '<DOMAIN_HERE>' },
}, {
$lookup: {
from: 'campaigns',
localField: '_id',
foreignField: 'tenantId',
as: 'campaign',
pipeline: [{
$match: { slug: '<SLUG_HERE>' },
}],
},
}]
which returns:
{
_id: ObjectId('...'),
campaign: [{
_id: ObjectId('...'),
}],
}
But it feels very uncomfortable, because for one the campaign is returned as a field of tenant and for other the campaign is returned as a single item in an array. I know, I can process and better format the result programmatically afterwards. But is there any way to „hack“ the aggregation to achieve a result that looks more like this?
{
tenant: {
_id: ObjectId('...'),
},
campaign: {
_id: ObjectId('...'),
},
}
This is just a simplified example, in reality this aggregation query is a bit more complicated (across more collections, upon few of which I need to perform a very similar query), so it's not just about this one simple query. So the ability to return an aggregated document as a separate object, rather than an array field on parent document would be quite helpful - if not, the world won't fall apart :)
To all those whom it may concern...
Thanks to answers from some good samaritans here, I've figured it out as a combination of $addFields, $project and $unwind. Extending my original aggregation query, the final pipeline would look like this:
[{
$match: { 'domains.name': '<DOMAIN_HERE>' },
}, {
$addFields: { tenant: '$$ROOT' },
}, {
$project: { _id: 0, tenant: 1 },
}, {
$lookup: {
from: 'campaigns',
localField: 'tenant._id',
foreignField: 'tenantId',
as: 'campaign',
pipeline: [{
$match: { slug: '<SLUG_HERE>' },
}],
},
}, {
$unwind: {
path: '$campaign',
preserveNullAndEmptyArrays: true,
},
}]
Thanks for the help! 😊
In this case:
const PostSchema = new mongoose.Schema({
"content": {
type: String,
required: true
},
"user": {
type: mongoose.Schema.Types.ObjectId,
required: true,
ref: "User"
},
"created": {
type: Date,
default: Date.now()
},
"comments": [{
type: mongoose.Schema.Types.ObjectID,
ref: 'Comment'
}]
})
I want to be able to get 10 comments at a time, but I see no way to do that without having to get all the comments every time.
You can use uncorrelated lookup to join collections and limit to 10. Here is an example, I used String for _id for easy understanding.
$lookup - there are two lookup, I used here uncorrelated lookup where you can do parallel aggregation in joining collection. $match helps to conditionally join documents. $expr is a must to use inside the $match when you use uncorrelated lookup. $limit helps to limit the documents. If you need you can add more stages to perform aggregation inside the pipeline
Here is the script
db.PostSchema.aggregate([
{
"$lookup": {
"from": "Comment",
let: {
cId: "$comments"
},
"pipeline": [
{
$match: {
$expr: {
_id: {
in: [
"$$cId"
]
}
}
}
},
{
$limit: 10
}
],
"as": "comments"
}
}
])
Working Mongo playground
I have two collections first one is
user_profile collection
const userProfileSchema = mongoose.Schema({
phone_number: {
type: String,
required: false,
},
primary_skills: [
{
skill_id: {
type: mongoose.Schema.Types.ObjectId,
ref: 'Skill'
},
years: Number,
}
]
});
sample data
{
"phone_number":"222",
"primary_skills":[{skill_id:1,years:12},{skill_id:2,years:13}]
}
in the primary_skills the key skill_id is mapped with another collection named skills
skills collection
const skillSchema = mongoose.Schema({
name: {
type: String,
required: true,
unique:true,
},
});
sample data
[
{
id:1,
name:'php'
},
{
id:2,
name:'java'
}
]
I want to fetch all values in the user_profile collection along with the respective skills name
expected output:
{
"phone_number":"222",
"primary_skills":[{
name:"php",skill_id:1,years:12
},{
name:"java",skill_id:2,years:13}
]
}
I found a similar thread to my question MongoDB lookup when foreign field is an array of objects but it's doing the opposite of what I want
This is the query I tried
profile.aggregate([{
$lookup:{
from:'skills',
localField:'primary_skills.skill_id',
foreignField:'_id',
'as':'primary_skills'
}
}])
This works fine but it didn't contain the years key
You need to do it with $unwind and $group,
$unwind primary_skills because its an array and we need to lookup sub document wise
db.user_profile.aggregate([
{
$unwind: "$primary_skills"
},
$lookup to join primary_skills, that you have already did
{
$lookup: {
from: "skills",
localField: "primary_skills.skill_id",
foreignField: "id",
as: "primary_skills.name"
}
},
$unwind primary_skills.name that we have stored join result, its array and we are unwinding to do object
{
$unwind: {
path: "$primary_skills.name"
}
},
$addFields replace field name that we have object and we need only name
{
$addFields: {
"primary_skills.name": "$primary_skills.name.name"
}
},
$group by _id because we have unwind and we need to combine all documents
{
$group: {
_id: "$_id",
phone_number: {
$first: "$phone_number"
},
primary_skills: {
$push: "$primary_skills"
}
}
}
])
Playground: https://mongoplayground.net/p/bDmrOwmASn5
I have an array of review objects like this :
"reviews": {
"author": "5e9167c5303a530023bcae42",
"rate": 5,
"spoiler": false,
"content": "This is a comment This is a comment This is a comment.",
"createdAt": "2020-04-12T16:08:34.966Z",
"updatedAt": "2020-04-12T16:08:34.966Z"
},
What I want to achieve is to lookup the author field and get the user data, but the problem is that the lookup I am trying to use only returns this to me:
Code :
.lookup({
from: 'users',
localField: 'reviews.author',
foreignField: '_id',
as: 'reviews.author',
})
Response :
Any way to get the author's data in that field? That's where the author's Id is.
Try to execute below query on your database :
db.reviews.aggregate([
/** unwind in general is not needed for `$lookup` for if you wanted to match lookup result with specific elem in array is needed */
{
$unwind: { path: "$reviews", preserveNullAndEmptyArrays: true },
},
{
$lookup: {
from: "users",
localField: "reviews.author",
foreignField: "_id",
as: "author", // Pull lookup result into 'author' field
},
},
/** Update 'reviews.author' field in 'reviews' object by checking if 'author' field got a match from 'users' collection.
* If Yes - As lookup returns an array get first elem & assign(As there will be only one element returned -uniques),
* If No - keep 'reviews.author' as is */
{
$addFields: {
"reviews.author": {
$cond: [
{ $ne: ["$author", []] },
{ $arrayElemAt: ["$author", 0] },
"$reviews.author",
],
},
},
},
/** Group back the documents based on '_id' field & push back all individual 'reviews' objects to 'reviews' array */
{
$group: {
_id: "$_id",
reviews: { $push: "$reviews" },
},
},
]);
Test : MongoDB-Playground
Note : Just in case if you've other fields in document along with reviews that needs to be preserved in output then starting at $group use these stages :
{
$group: {
_id: "$_id",
data: {
$first: "$$ROOT"
},
reviews: {
$push: "$reviews"
}
}
},
{
$addFields: {
"data.reviews": "$reviews"
}
},
{
$project: {
"data.author": 0
}
},
{
$replaceRoot: {
newRoot: "$data"
}
}
Test : MongoDB-Playground
Note : Try to keep queries to run on lesser datasets maybe by adding $match as first stage to filter documents & also have proper indexes.
you should use populate('author') method of mongoose on the request to the server which gets the id of that author and adds the user data to the response of mongoose
and dont forget to set your schema in a way that these two collections are connected
in your review schema you should add ref to the schema which the author user is saved
author: { type: Schema.Types.ObjectId, ref: 'users' },
You can follow this code
$lookup:{
from:'users',
localField:'reviews.author',
foreignField:'_id',
as:'reviews.author'
}
**OR**
> When You find the doc then use populate
> reviews.find().populate("author")
an abbreviated schema:
const ThingSchema = new mongoose.Schema({
_id: {
type: String,
},
widgets: [{
user: {
type: Schema.Types.ObjectId,
ref: 'user',
},
lastViewedAt: {
type: Date,
},
}],
}, { _id: false });
Given a userId, how would you get back all Things that a user belongs to where user is a sub-doc in an array of deeply nested objects?
I have gone between several approaches found on answers to similar "find in array of object" questions suggesting .where, $in, and .populate match not being able to get more than the full, unfiltered, collection of Things or nada. I welcome feedback as to why the particular methods I tried above would be a poor/good choice for the end goal of the filtered result I am after.
Update:
A partial solution I have makes use of aggregate. First I had to $unwind the widgets array. In the resulting collection generated by $unwind I then $matched on the user's _id.
There was a gotcha in that mongoose does not autocast _ids into mongo ObjectIds in aggregate calls unlike with other queries. I had to explicitly convert my _id string with mongoose.Types.ObjectId().
const things = await Thing.aggregate([
{ $unwind: '$widgets' },
{ $match: { 'widgets._id': mongoose.Types.ObjectId(userId) } }
]);
The only issue here is that, in the process of unwinding the widgets array, I have lost that full collection of users and only get the single user who's ID I am searching on in that collection now. I now would need to do something like grab the Things IDs and run an additional query and .populate() the users again.
I had similar results with this particularly verbose go at $filter
const things = await Thing.aggregate([
{ "$match": {
"widgets._id": { "$in": [ mongoose.Types.ObjectId(userId) ] }
}},
{ "$project": {
"widgets": {
"$filter": {
"input": "$widgets",
"as": "user",
"cond": { "$or": [{ "$eq": [ "$$user._id", mongoose.Types.ObjectId(userId) ] }] },
}
}
}}
]);