The code example is in Mongo Playground
https://mongoplayground.net/p/W4Qt4oX0ZRP
Assume the following documents
[
{
_id: "5df1e6f75de2b22f8e6c30e8",
user: {
name: "Tom",
sex: 1,
age: 23
},
dream: [
{
label: "engineer",
industry: "5e06b16fb0670d7538222909",
type: "5e06b16fb0670d7538222951",
},
{
label: "programmer",
industry: "5e06b16fb0670d7538222909",
type: "5e06b16fb0670d7538222951",
}
],
works: [
{
name: "any engineer",
company: "5dd7fd51b0ae1837a08d00c8",
skill: [
"5dc3998e2cf66bad16efd61b",
"5dc3998e2cf66bad16efd61e"
],
},
{
name: "any programmer",
company: "5dd7fd9db0ae1837a08d00e2",
skill: [
"5dd509e05de2b22f8e67e1b7",
"5dd509e05de2b22f8e67e1bb"
],
}
]
}
]
I tried to use aggregate $lookup $unwind
db.coll.aggregate([
{
$unwind: {
path: "$dream",
}
},
{
$lookup: {
from: "industry",
localField: "dream.industry",
foreignField: "_id",
as: "dream.industry"
},
},
{
$unwind: {
path: "$dream.industry",
}
},
{
$lookup: {
from: "type",
localField: "dream.type",
foreignField: "_id",
as: "dream.type"
},
},
{
$unwind: {
path: "$dream.type",
}
},
{
$unwind: {
path: "$works",
}
},
{
$lookup: {
from: "company",
localField: "works.company",
foreignField: "_id",
as: "works.company"
},
},
{
$unwind: {
path: "$works.company",
}
},
{
$lookup: {
from: "skill",
localField: "works.skill",
foreignField: "_id",
as: "works.skill"
},
},
])
Executing the above code did not get the desired result!
This is what i expect
{
_id: "5df1e6f75de2b22f8e6c30e8",
user: {
name: 'Tom',
sex: 1,
age: 23
},
dream: [
{
label: 'engineer',
industry: {
_id: "5e06b16fb0670d7538222909", // Industry doc _id
name: 'IT',
createdAt: "2019-12-28T01:35:44.070Z",
updatedAt: "2019-12-28T01:35:44.070Z"
},
type: {
_id: "5e06b16fb0670d7538222951", // Type doc _id
name: 'job',
createdAt: "2019-12-28T01:35:44.070Z",
updatedAt: "2019-12-28T01:35:44.070Z"
},
},
{
label: 'programmer',
industry: {
_id: "5e06b16fb0670d7538222909", // Industry doc _id
name: 'IT',
createdAt: "2019-12-28T01:35:44.070Z",
updatedAt: "2019-12-28T01:35:44.070Z"
},
type: {
_id: "5e06b16fb0670d7538222951", // Type doc _id
name: 'job',
createdAt: "2019-12-28T01:35:44.070Z",
updatedAt: "2019-12-28T01:35:44.070Z"
}
}
],
works: [
{
name: 'any engineer',
company: {
_id: "5dd7fd51b0ae1837a08d00c8", // Company doc _id
name: 'alibaba',
area: 'CN',
},
skill: [
{
_id: "5dc3998e2cf66bad16efd61b", // Skill doc _id
name: 'Java'
},
{
_id: "5dc3998e2cf66bad16efd61e", // Skill doc _id
name: 'Php'
},
]
},
{
name: 'any programmer',
company: {
_id: "5dd7fd9db0ae1837a08d00e2", // Company doc _id
name: 'microsoft',
area: 'EN',
},
skill: [
{
_id: "5dd509e05de2b22f8e67e1b7", // Skill doc _id
name: 'Golang'
},
{
_id: "5dd509e05de2b22f8e67e1bb", // Skill doc _id
name: 'Node.js'
}
]
},
]
}
The expected result is dream is an array, works is an array, and dream.industry changed from ObjectId to document, dream.type changed from ObjectId to document, works.company changed from ObjectId to document
When I use populate, I can do it easily
Model.find()
.populate('dream.industry')
.populate('dream.type')
.populate('works.company')
.populate('works.skill')
.lean()
I refer to the following questions
mongoose aggregate lookup array (Almost the same as my question, But not resolved)
$lookup on ObjectId's in an array
hope to get everyone's help, thank you!
To make it easier i would not change the current pipeline but just add a $group stage to end of it in order to re-structure the data.
{
$group: {
_id: "$_id",
user: {$first: "$user"},
dream: {$addToSet: "$dream"},
works: {$addToSet: "$works"}
}
}
With that said if you are using Mongo version 3.6+ i do recommend you use the "newer" version of $lookup to re-write your pipeline to be a bit more efficient by avoiding all these $unwind's.
Related
I've inherited a Azure Cosmos database with a MongoDB API. There is extensive use of "discriminators" so a single collection has many different models.
I am trying to query a document three levels deep based on document ids (ObjectId())
Parent Group
{
_id: ObjectId(),
__type: "ParentGroup",
name: "group 1",
subgroups: [
...ObjectIds,
],
}
Sub Group
{
_id: ObjectId(),
__type: "SubGroup",
name: "a text name",
members: [
...ObjectIds,
],
}
Member
{
_id: ObjectId(),
__type: "Member",
name: "string",
email: "",
induction: Date,
}
Examples I've seen deal with nested documents NOT references
Is it possible to query the Member documents and return?
[
{
parentGroup,
subgroups: [
{sub group, members: [...members]},
{sub group, members: [...members]},
{sub group, members: [...members]},
]
},
]
After reading the comments and further reading i've got this. Its almost there but I think the limitation of MongoDB will prevent the solution being in a single query. The goal is to return ParentGroups->Subgroups->Members Where Members have an "induction" value of "whatever". I am either returning ALL ParentGroups or nothing at all
db.development.aggregate([
{
$match: {
__type: "ParentGroup", $expr: {
$gt: [
{ $size: "$subgroups" }, 0
]
}
}
},
{
$lookup: {
from: "development",
localField: "subgroups",
foreignField: "_id",
as: "subgroups"
}
},
{
$unwind: {
path: "$subgroups",
// preserveNullAndEmptyArrays: true
}
},
{
$lookup: {
from: "development",
localField: "subgroups.members",
foreignField: "_id",
as: "subgroups.members"
}
}
])
Solution that worked for me:
db.development.aggregate([
{
$match: {
__type: "ParentGroup",
},
},
{
$lookup: {
from: "development",
localField: "subgroups",
foreignField: "_id",
as: "subgroups",
},
},
{
$unwind: {
path: "$subgroups",
preserveNullAndEmptyArrays: true,
},
},
{
$lookup: {
from: "development",
localField: "subgroups.members",
foreignField: "_id",
as: "subgroups.activities_x",
},
},
{
$unwind: {
path: "$subgroups.members",
preserveNullAndEmptyArrays: true,
},
},
{
$match: { "subgroups.members.meta": { $exists: true } },
},
{
$project: {
_id: 1,
__type: 1,
name: 1,
subgroups: {
_id: 1,
__type: 1,
name: 1,
members: {
_id: 1,
__type: 1,
name: 1,
meta: 1,
},
},
},
},
]);
So I have 2 models user & form.
User Schema
firstName: {
type: String,
required: true,
},
lastName: {
type: String,
required: true,
},
email: {
type: String,
required: true,
}
Form Schema
approvalLog: [
{
attachments: {
type: [String],
},
by: {
type: ObjectId,
},
comment: {
type: String,
},
date: {
type: Date,
},
},
],
userId: {
type: ObjectId,
required: true,
},
... other form parameters
When returning a form, I'm trying to aggregate the user info of every user in the approvalLog into their respective objects as below.
{
...other form info
approvalLog: [
{
attachments: [],
_id: '619cc4953de8413b548f61a6',
by: '619cba9cd64af530448b6347',
comment: 'visit store for disburement',
date: '2021-11-23T10:38:13.565Z',
user: {
_id: '619cba9cd64af530448b6347',
firstName: 'admin',
lastName: 'user',
email: 'admin#mail.com',
},
},
{
attachments: [],
_id: '619cc4ec3ea3e940a42b2d01',
by: '619cbd7b3de8413b548f61a0',
comment: '',
date: '2021-11-23T10:39:40.168Z',
user: {
_id: '619cbd7b3de8413b548f61a0',
firstName: 'sam',
lastName: 'ben',
email: 'sb#mail.com',
},
},
{
attachments: [],
_id: '61a9deab8f472c52d8bac095',
by: '61a87fd93dac9b209096ed94',
comment: '',
date: '2021-12-03T09:08:59.479Z',
user: {
_id: '61a87fd93dac9b209096ed94',
firstName: 'john',
lastName: 'doe',
email: 'jd#mail.com',
},
},
],
}
My current code is
Form.aggregate([
{
$lookup: {
from: 'users',
localField: 'approvalLog.by',
foreignField: '_id',
as: 'approvedBy',
},
},
{ $addFields: { 'approvalLog.user': { $arrayElemAt: ['$approvedBy', 0] } } },
])
but it only returns the same user for all objects. How do I attach the matching user for each index?
I've also tried
Form.aggregate([
{
$lookup: {
from: 'users',
localField: 'approvalLog.by',
foreignField: '_id',
as: 'approvedBy',
},
},
{
$addFields: {
approvalLog: {
$map: {
input: { $zip: { inputs: ['$approvalLog', '$approvedBy'] } },
in: { $mergeObjects: '$$this' },
},
},
},
},
])
This adds the right user to their respective objects, but I can only add the to the root object and not a new one.
You can try the approach,
$map to iterate loop of approvalLog
$filter to iterate loop of approvedBy array and search for user id by
$arrayElemAt to get first element from above filtered result
$mergeObjects to merge current object properties of approvalLog and filtered user
$$REMOVE don't need approvedBy now
await Form.aggregate([
{
$lookup: {
from: "users",
localField: "approvalLog.by",
foreignField: "_id",
as: "approvedBy"
}
},
{
$addFields: {
approvalLog: {
$map: {
input: "$approvalLog",
as: "a",
in: {
$mergeObjects: [
"$$a",
{
user: {
$arrayElemAt: [
{
$filter: {
input: "$approvedBy",
cond: { $eq: ["$$a.by", "$$this._id"] }
}
},
0
]
}
}
]
}
}
},
approvedBy: "$$REMOVE"
}
}
])
Playground
The second approach using $unwind,
$unwind deconstruct the approvalLog array
$lookup with user collection
$addFields and $arrayElemAt to get first element from lookup result
$group by _id and reconstruct the approvalLog array and get first value of other required properties
await Form.aggregate([
{ $unwind: "$approvalLog" },
{
$lookup: {
from: "users",
localField: "approvalLog.by",
foreignField: "_id",
as: "approvalLog.user"
}
},
{
$addFields: {
"approvalLog.user": {
$arrayElemAt: ["$approvalLog.user", 0]
}
}
},
{
$group: {
_id: "$_id",
approvalLog: { $push: "$approvalLog" },
userId: { $first: "$userId" },
// add your other properties like userId
}
}
])
Playground
I'm trying to do an aggregation on two collections that has a linkage between them, and I need to access information in an array of objects in one of those collections.
Here are the schemas:
User Schema:
{
_id: ObjectId,
username: String,
password: String,
associatedEvents: [
{
event_id: ObjectId,
isCreator: boolean,
access_level: String,
}
]
}
Event Schema:
{
_id: ObjectId,
title: String,
associated_users: [
{
user_id: ObjectId
}
]
}
I'm attempting to get the users associated to an event for a specific user, and then get their access level information. Here's the aggregation I have:
const eventsJoined = await Event.aggregate([
{
$match: {
$expr: { $in: [id, "$associatedUserIds"] },
},
},
{
$lookup: {
from: "users",
localField: "associatedUserIds",
foreignField: "_id",
as: "user_info",
},
},
{ $unwind: "$user_info" },
{
$unwind: {
path: "$user_info.associatedEvents",
preserveNullAndEmptyArrays: true,
},
},
{
$group: {
_id: "$_id",
title: { $first: "$title" },
description: { $first: "$description" },
startDate: { $first: "$startdate" },
userInfo: { $first: "$user_info" },
usersAssociatedEvents: { $push: "$user_info.associatedEvents" },
},
},
{
$project: {
title: 1,
description: 1,
startDate: 1,
userInfo: 1,
usersAssociatedEvents: "$usersAssociatedEvents",
},
},
]);
And this is the result I'm getting:
[
{
_id: 609d5ad1ef4cdbeb32987739,
title: 'hello',
description: 'desc',
startDate: null,
usersAssociatedEvents: [ [Object] ]
}
]
As you can see, the query is already aggregating the correct data. But the last thing that's tripping me up is the fact that the aggregation is [ [Object] ] for usersAssociatedEvents instead of the actual contents of the object. Any idea on why that would be?
I have the following code:
const profiles = await Profile.aggregate([
{
$lookup: {
from: "users",
localField: "user",
foreignField: "_id",
as: "user",
},
},
{
$unwind: "$user",
},
{
$match: {
"user.name": {
$regex: q.trim(),
$options: "i",
},
},
},
{
$skip: req.params.page ? (req.params.page - 1) * 10 : 0,
},
{
$limit: 11,
},
{
$group: {
_id: "$_id",
skills:{skills}
user: { name: "$name" },
user: { avatar: "$avatar" },
},
},
]);
I want to return only specific fields like skills _id and user.name and user.avatar, but this doesn't work. I'm pretty sure that the problem is in $group. I want to receive only these fields
[
{
_id: 5ef78d005d23020ca847aa76,
skills: [ 'asd' ],
user: {
_id: 5ef78c7c5d23020ca847aa75,
name: 'Simeon Lazarov',
avatar: 'uploads\\1593286096227 - background.jpg',
}
}
]
You can make use of $project to get specific fields.
After grouping add the below:
{
$project: {_id:1, skills:1, user:1}
}
Projection value of 0 means that the field needs to be excluded, Value 1 represents inclusion of the field.
Document reference: https://docs.mongodb.com/manual/reference/operator/aggregation/project/
in my project a user can create products. each user have a reference to all of its products and each product have a reference to its user.
both the user and the product have a 'name' field.
i need to get all of the users products array, and in that array i want to have the product name and the
user name that created it (and only those fields and no others).
for example:
Users:
{ _id: 1, name: 'josh', productIds: [1,3]}
{ _id: 2, name: 'sheldon', productIds: [2]}
Products:
{ _id: 1, name: 'table', price: 45, userId: 1}
{ _id: 2, name: 'television', price: 25 userId: 2}
{ _id: 3, name: 'chair', price: 14 userId: 1}
i want to get the following result:
{ _id: 1, name: 'josh',
products: {
{ _id: 1, name: 'table', user: { _id: 1, name: 'josh' },
{ _id: 3, name: 'chair', user: { _id: 1, name: 'josh' },
}
}
{ _id: 2, name: 'sheldon',
products: {
{ _id: 2, name: 'television', userId: { _id: 2, name: 'sheldon' }
}
}
i tried the following query that didn't fill the inner userId and left it with only the id (no name):
User.aggregate([
{
$lookup:
{
from: 'products',
localField: 'productIds',
foreignField: '_id',
as: 'products'
}
}
i also tried the following, which did the same as the first query except it only retried the first product for each user:
User.aggregate([
{
$lookup:
{
from: 'products',
localField: 'productIds',
foreignField: '_id',
as: 'products'
}
},
{
$unwind: {
path: "$products",
preserveNullAndEmptyArrays: true
}
},
{
$lookup: {
from: "user",
localField: "products.userId",
foreignField: "_id",
as: "prodUsr",
}
},
{
$group: {
_id : "$_id",
products: { $push: "$products" },
"doc": { "$first": "$$ROOT" }
}
},
{
"$replaceRoot": {
"newRoot": "$doc"
}
}
Product:
const schema = new Schema(
{
name: {
type: String,
required: true
},
price: {
type: Number,
required: true
},
userId: {
type: Schema.Types.ObjectId,
ref: 'User',
required: true
},
}
);
module.exports = mongoose.model('Product', schema);
User:
const schema = new Schema(
{
name: {
type: String,
required: true,
unique: true
},
productIds: [{
type: Schema.Types.ObjectId,
ref: 'Product',
require: false
}],
{ timestamps: true }
);
module.exports = mongoose.model('User', schema);
any help will be highly appreciated
It looks like a perfect scenario for $lookup with custom pipeline and another nested $lookup. The inner one allows you to handle product-> user relationship while the outer one handles user -> product one:
db.Users.aggregate([
{
$project: {
productIds: 0
}
},
{
$lookup: {
from: "Products",
let: { user_id: "$_id" },
pipeline: [
{
$match: {
$expr: {
$eq: [ "$userId", "$$user_id" ]
}
}
},
{
$lookup: {
from: "Users",
localField: "userId",
foreignField: "_id",
as: "user"
}
},
{
$unwind: "$user"
},
{
$project: {
"user.productIds": 0,
"price": 0,
"userId": 0
}
}
],
as: "products"
}
}
])
Mongo Playground