mongoose recursive nesting - mongodb

in my project a user can create products. each user have a reference to all of its products and each product have a reference to its user.
both the user and the product have a 'name' field.
i need to get all of the users products array, and in that array i want to have the product name and the
user name that created it (and only those fields and no others).
for example:
Users:
{ _id: 1, name: 'josh', productIds: [1,3]}
{ _id: 2, name: 'sheldon', productIds: [2]}
Products:
{ _id: 1, name: 'table', price: 45, userId: 1}
{ _id: 2, name: 'television', price: 25 userId: 2}
{ _id: 3, name: 'chair', price: 14 userId: 1}
i want to get the following result:
{ _id: 1, name: 'josh',
products: {
{ _id: 1, name: 'table', user: { _id: 1, name: 'josh' },
{ _id: 3, name: 'chair', user: { _id: 1, name: 'josh' },
}
}
{ _id: 2, name: 'sheldon',
products: {
{ _id: 2, name: 'television', userId: { _id: 2, name: 'sheldon' }
}
}
i tried the following query that didn't fill the inner userId and left it with only the id (no name):
User.aggregate([
{
$lookup:
{
from: 'products',
localField: 'productIds',
foreignField: '_id',
as: 'products'
}
}
i also tried the following, which did the same as the first query except it only retried the first product for each user:
User.aggregate([
{
$lookup:
{
from: 'products',
localField: 'productIds',
foreignField: '_id',
as: 'products'
}
},
{
$unwind: {
path: "$products",
preserveNullAndEmptyArrays: true
}
},
{
$lookup: {
from: "user",
localField: "products.userId",
foreignField: "_id",
as: "prodUsr",
}
},
{
$group: {
_id : "$_id",
products: { $push: "$products" },
"doc": { "$first": "$$ROOT" }
}
},
{
"$replaceRoot": {
"newRoot": "$doc"
}
}
Product:
const schema = new Schema(
{
name: {
type: String,
required: true
},
price: {
type: Number,
required: true
},
userId: {
type: Schema.Types.ObjectId,
ref: 'User',
required: true
},
}
);
module.exports = mongoose.model('Product', schema);
User:
const schema = new Schema(
{
name: {
type: String,
required: true,
unique: true
},
productIds: [{
type: Schema.Types.ObjectId,
ref: 'Product',
require: false
}],
{ timestamps: true }
);
module.exports = mongoose.model('User', schema);
any help will be highly appreciated

It looks like a perfect scenario for $lookup with custom pipeline and another nested $lookup. The inner one allows you to handle product-> user relationship while the outer one handles user -> product one:
db.Users.aggregate([
{
$project: {
productIds: 0
}
},
{
$lookup: {
from: "Products",
let: { user_id: "$_id" },
pipeline: [
{
$match: {
$expr: {
$eq: [ "$userId", "$$user_id" ]
}
}
},
{
$lookup: {
from: "Users",
localField: "userId",
foreignField: "_id",
as: "user"
}
},
{
$unwind: "$user"
},
{
$project: {
"user.productIds": 0,
"price": 0,
"userId": 0
}
}
],
as: "products"
}
}
])
Mongo Playground

Related

MongoDB - query references 2 deep of ObjectIDs

I've inherited a Azure Cosmos database with a MongoDB API. There is extensive use of "discriminators" so a single collection has many different models.
I am trying to query a document three levels deep based on document ids (ObjectId())
Parent Group
{
_id: ObjectId(),
__type: "ParentGroup",
name: "group 1",
subgroups: [
...ObjectIds,
],
}
Sub Group
{
_id: ObjectId(),
__type: "SubGroup",
name: "a text name",
members: [
...ObjectIds,
],
}
Member
{
_id: ObjectId(),
__type: "Member",
name: "string",
email: "",
induction: Date,
}
Examples I've seen deal with nested documents NOT references
Is it possible to query the Member documents and return?
[
{
parentGroup,
subgroups: [
{sub group, members: [...members]},
{sub group, members: [...members]},
{sub group, members: [...members]},
]
},
]
After reading the comments and further reading i've got this. Its almost there but I think the limitation of MongoDB will prevent the solution being in a single query. The goal is to return ParentGroups->Subgroups->Members Where Members have an "induction" value of "whatever". I am either returning ALL ParentGroups or nothing at all
db.development.aggregate([
{
$match: {
__type: "ParentGroup", $expr: {
$gt: [
{ $size: "$subgroups" }, 0
]
}
}
},
{
$lookup: {
from: "development",
localField: "subgroups",
foreignField: "_id",
as: "subgroups"
}
},
{
$unwind: {
path: "$subgroups",
// preserveNullAndEmptyArrays: true
}
},
{
$lookup: {
from: "development",
localField: "subgroups.members",
foreignField: "_id",
as: "subgroups.members"
}
}
])
Solution that worked for me:
db.development.aggregate([
{
$match: {
__type: "ParentGroup",
},
},
{
$lookup: {
from: "development",
localField: "subgroups",
foreignField: "_id",
as: "subgroups",
},
},
{
$unwind: {
path: "$subgroups",
preserveNullAndEmptyArrays: true,
},
},
{
$lookup: {
from: "development",
localField: "subgroups.members",
foreignField: "_id",
as: "subgroups.activities_x",
},
},
{
$unwind: {
path: "$subgroups.members",
preserveNullAndEmptyArrays: true,
},
},
{
$match: { "subgroups.members.meta": { $exists: true } },
},
{
$project: {
_id: 1,
__type: 1,
name: 1,
subgroups: {
_id: 1,
__type: 1,
name: 1,
members: {
_id: 1,
__type: 1,
name: 1,
meta: 1,
},
},
},
},
]);

Merge $lookup value inside objects nested in array mongoose

So I have 2 models user & form.
User Schema
firstName: {
type: String,
required: true,
},
lastName: {
type: String,
required: true,
},
email: {
type: String,
required: true,
}
Form Schema
approvalLog: [
{
attachments: {
type: [String],
},
by: {
type: ObjectId,
},
comment: {
type: String,
},
date: {
type: Date,
},
},
],
userId: {
type: ObjectId,
required: true,
},
... other form parameters
When returning a form, I'm trying to aggregate the user info of every user in the approvalLog into their respective objects as below.
{
...other form info
approvalLog: [
{
attachments: [],
_id: '619cc4953de8413b548f61a6',
by: '619cba9cd64af530448b6347',
comment: 'visit store for disburement',
date: '2021-11-23T10:38:13.565Z',
user: {
_id: '619cba9cd64af530448b6347',
firstName: 'admin',
lastName: 'user',
email: 'admin#mail.com',
},
},
{
attachments: [],
_id: '619cc4ec3ea3e940a42b2d01',
by: '619cbd7b3de8413b548f61a0',
comment: '',
date: '2021-11-23T10:39:40.168Z',
user: {
_id: '619cbd7b3de8413b548f61a0',
firstName: 'sam',
lastName: 'ben',
email: 'sb#mail.com',
},
},
{
attachments: [],
_id: '61a9deab8f472c52d8bac095',
by: '61a87fd93dac9b209096ed94',
comment: '',
date: '2021-12-03T09:08:59.479Z',
user: {
_id: '61a87fd93dac9b209096ed94',
firstName: 'john',
lastName: 'doe',
email: 'jd#mail.com',
},
},
],
}
My current code is
Form.aggregate([
{
$lookup: {
from: 'users',
localField: 'approvalLog.by',
foreignField: '_id',
as: 'approvedBy',
},
},
{ $addFields: { 'approvalLog.user': { $arrayElemAt: ['$approvedBy', 0] } } },
])
but it only returns the same user for all objects. How do I attach the matching user for each index?
I've also tried
Form.aggregate([
{
$lookup: {
from: 'users',
localField: 'approvalLog.by',
foreignField: '_id',
as: 'approvedBy',
},
},
{
$addFields: {
approvalLog: {
$map: {
input: { $zip: { inputs: ['$approvalLog', '$approvedBy'] } },
in: { $mergeObjects: '$$this' },
},
},
},
},
])
This adds the right user to their respective objects, but I can only add the to the root object and not a new one.
You can try the approach,
$map to iterate loop of approvalLog
$filter to iterate loop of approvedBy array and search for user id by
$arrayElemAt to get first element from above filtered result
$mergeObjects to merge current object properties of approvalLog and filtered user
$$REMOVE don't need approvedBy now
await Form.aggregate([
{
$lookup: {
from: "users",
localField: "approvalLog.by",
foreignField: "_id",
as: "approvedBy"
}
},
{
$addFields: {
approvalLog: {
$map: {
input: "$approvalLog",
as: "a",
in: {
$mergeObjects: [
"$$a",
{
user: {
$arrayElemAt: [
{
$filter: {
input: "$approvedBy",
cond: { $eq: ["$$a.by", "$$this._id"] }
}
},
0
]
}
}
]
}
}
},
approvedBy: "$$REMOVE"
}
}
])
Playground
The second approach using $unwind,
$unwind deconstruct the approvalLog array
$lookup with user collection
$addFields and $arrayElemAt to get first element from lookup result
$group by _id and reconstruct the approvalLog array and get first value of other required properties
await Form.aggregate([
{ $unwind: "$approvalLog" },
{
$lookup: {
from: "users",
localField: "approvalLog.by",
foreignField: "_id",
as: "approvalLog.user"
}
},
{
$addFields: {
"approvalLog.user": {
$arrayElemAt: ["$approvalLog.user", 0]
}
}
},
{
$group: {
_id: "$_id",
approvalLog: { $push: "$approvalLog" },
userId: { $first: "$userId" },
// add your other properties like userId
}
}
])
Playground

mongoose : Users who have not logged in since last 5 days

I have 2 schemas
userSchema:
const userSchema = new mongoose.Schema({
firstName: {
type: String,
},
lastName: {
type: String,
},
username: {
type: String,
required: true,
},
password: {
type: String,
required: true,
},
tokens: {
type: [
{
token: String,
},
],
},
});
useractivity:
const userActivitySchema = new mongoose.Schema({
ip: String,
ua: String,
date: {
type: Date,
default: Date.now,
},
user: {
type: mongoose.Schema.Types.ObjectId,
ref: "User",
},
});
I am adding a new entry in useractivity every time user logged into the system.
I wanted to featch list of users who haven't logIn in last 5 days for that I write query :
const dateToCheck = new Date();
dateToCheck.setDate(
dateToCheck.getDate() - process.env.LEASS_ACTIVE_USER_DAYS
);
const userActivity = await UserActivity.aggregate([
{ $match: { date: { $lt: dateToCheck } } },
{
$lookup: {
from: "users",
localField: "user",
foreignField: "_id",
as: "userdata",
},
},
{ $unwind: { path: "$userdata" } },
{
$project: {
date: 1,
"userdata.firstName": 1,
},
},
]);
with this, I am able to get date and firstName of users who haven't logged in last 5 days.
But the issue with this is as there are multiple records in useractivity, above query returns duplicate users in the result.
I tried using { $group: { _id: { user: "$user" } } } to group the result by foreign key but it returns [] in this case.
Any suggestions will be helpfull
Try this out, you already have access to the unique id of the user from your lookup, so use that to group your documents. Additionally, I am storing all the dates in an array from the grouping
UPDATE: Also attaching mongo playground link where I tried it
const userActivity = await UserActivity.aggregate([
{ $match: { date: { $lt: dateToCheck } } },
{
$lookup: {
from: "users",
localField: "user",
foreignField: "_id",
as: "userdata",
},
},
{ $unwind: { path: "$userdata" } },
{
$project: {
date: 1,
"userdata.firstName": 1,
"user_id": "$userdata._id"
},
},
{
$group: {
"_id": {
"user_id": "$user_id",
"user_name": "$userdata.firstName"
},
"dates": {
"$push": "$date"
}
}
}
]);

mongodb how to use aggregate like populate

The code example is in Mongo Playground
https://mongoplayground.net/p/W4Qt4oX0ZRP
Assume the following documents
[
{
_id: "5df1e6f75de2b22f8e6c30e8",
user: {
name: "Tom",
sex: 1,
age: 23
},
dream: [
{
label: "engineer",
industry: "5e06b16fb0670d7538222909",
type: "5e06b16fb0670d7538222951",
},
{
label: "programmer",
industry: "5e06b16fb0670d7538222909",
type: "5e06b16fb0670d7538222951",
}
],
works: [
{
name: "any engineer",
company: "5dd7fd51b0ae1837a08d00c8",
skill: [
"5dc3998e2cf66bad16efd61b",
"5dc3998e2cf66bad16efd61e"
],
},
{
name: "any programmer",
company: "5dd7fd9db0ae1837a08d00e2",
skill: [
"5dd509e05de2b22f8e67e1b7",
"5dd509e05de2b22f8e67e1bb"
],
}
]
}
]
I tried to use aggregate $lookup $unwind
db.coll.aggregate([
{
$unwind: {
path: "$dream",
}
},
{
$lookup: {
from: "industry",
localField: "dream.industry",
foreignField: "_id",
as: "dream.industry"
},
},
{
$unwind: {
path: "$dream.industry",
}
},
{
$lookup: {
from: "type",
localField: "dream.type",
foreignField: "_id",
as: "dream.type"
},
},
{
$unwind: {
path: "$dream.type",
}
},
{
$unwind: {
path: "$works",
}
},
{
$lookup: {
from: "company",
localField: "works.company",
foreignField: "_id",
as: "works.company"
},
},
{
$unwind: {
path: "$works.company",
}
},
{
$lookup: {
from: "skill",
localField: "works.skill",
foreignField: "_id",
as: "works.skill"
},
},
])
Executing the above code did not get the desired result!
This is what i expect
{
_id: "5df1e6f75de2b22f8e6c30e8",
user: {
name: 'Tom',
sex: 1,
age: 23
},
dream: [
{
label: 'engineer',
industry: {
_id: "5e06b16fb0670d7538222909", // Industry doc _id
name: 'IT',
createdAt: "2019-12-28T01:35:44.070Z",
updatedAt: "2019-12-28T01:35:44.070Z"
},
type: {
_id: "5e06b16fb0670d7538222951", // Type doc _id
name: 'job',
createdAt: "2019-12-28T01:35:44.070Z",
updatedAt: "2019-12-28T01:35:44.070Z"
},
},
{
label: 'programmer',
industry: {
_id: "5e06b16fb0670d7538222909", // Industry doc _id
name: 'IT',
createdAt: "2019-12-28T01:35:44.070Z",
updatedAt: "2019-12-28T01:35:44.070Z"
},
type: {
_id: "5e06b16fb0670d7538222951", // Type doc _id
name: 'job',
createdAt: "2019-12-28T01:35:44.070Z",
updatedAt: "2019-12-28T01:35:44.070Z"
}
}
],
works: [
{
name: 'any engineer',
company: {
_id: "5dd7fd51b0ae1837a08d00c8", // Company doc _id
name: 'alibaba',
area: 'CN',
},
skill: [
{
_id: "5dc3998e2cf66bad16efd61b", // Skill doc _id
name: 'Java'
},
{
_id: "5dc3998e2cf66bad16efd61e", // Skill doc _id
name: 'Php'
},
]
},
{
name: 'any programmer',
company: {
_id: "5dd7fd9db0ae1837a08d00e2", // Company doc _id
name: 'microsoft',
area: 'EN',
},
skill: [
{
_id: "5dd509e05de2b22f8e67e1b7", // Skill doc _id
name: 'Golang'
},
{
_id: "5dd509e05de2b22f8e67e1bb", // Skill doc _id
name: 'Node.js'
}
]
},
]
}
The expected result is dream is an array, works is an array, and dream.industry changed from ObjectId to document, dream.type changed from ObjectId to document, works.company changed from ObjectId to document
When I use populate, I can do it easily
Model.find()
.populate('dream.industry')
.populate('dream.type')
.populate('works.company')
.populate('works.skill')
.lean()
I refer to the following questions
mongoose aggregate lookup array (Almost the same as my question, But not resolved)
$lookup on ObjectId's in an array
hope to get everyone's help, thank you!
To make it easier i would not change the current pipeline but just add a $group stage to end of it in order to re-structure the data.
{
$group: {
_id: "$_id",
user: {$first: "$user"},
dream: {$addToSet: "$dream"},
works: {$addToSet: "$works"}
}
}
With that said if you are using Mongo version 3.6+ i do recommend you use the "newer" version of $lookup to re-write your pipeline to be a bit more efficient by avoiding all these $unwind's.

Check follow status in aggregate mongo / mongoose

I have this schema for users where followers/followed is array and the reference the same schema
var userSchema = new Schema({
username: { type: String, unique: true, trim: true, required: true },
password: { type: String, required: true },
followers: [{ type: Schema.Types.ObjectId, ref: "users" }],
followed: [{ type: Schema.Types.ObjectId, ref: "users" }],
registered: { type: Date, default: Date.now },
admin: { type: Number, default: 0 }
});
What I am looking for to return the follow status, if the _id is contains in followed array give me for example follow_status: 1
[
{
$match: { username: new RegExp(username, "i") }
},
{
$unwind: "$followers"
},
{
$lookup: {
from: "users",
localField: "followers",
foreignField: "_id",
as: "info"
}
},
{
$unwind: "$info"
},
{
$project: {
info: {
_id: 1,
username: 1,
avatar: { $ifNull: ["$avatar", ""] },
fullname: { $ifNull: ["$fullname", ""] }
}
}
},
{
$replaceRoot: { newRoot: "$info" }
},
{
$limit: 1000
}
]
Current pipeliens result
[
{
"_id": "5a906653f52e66c9c7a23cb6",
"username": "User1"
},
{
"_id": "5a908eb564a726cf8ec7e0a3",
"username": "User2"
}
]