Mongodb mongoose query get count of documents by reference - mongodb

I have 2 collections in mongodb. articles and tags.
In articles, there can be multiple tags.
Following is the article schema:
const mongoose = require('mongoose');
const articleSchema = new mongoose.Schema({
title: {
type: String,
required: true,
trim: true
},
tags: [{
type: mongoose.Schema.Types.ObjectId,
required: true,
ref: 'Tag'
}]
}, {
timestamps: true
});
const Article = mongoose.model('Article', articleSchema);
module.exports = Article;
Following is the tag schema:
const mongoose = require('mongoose');
const tagSchema = new mongoose.Schema({
title: {
type: String,
required: true,
trim: true,
unique: true
}
}, {
timestamps: true
});
const Tag = mongoose.model('Tag', tagSchema);
module.exports = Tag;
From these collections I wanted to show a simple column chart which shows
how many articles are there against a tag.
I am trying to get data in format like this:
const data = [
{ title: 'Javascript', count: 20 },
{ title: 'ReactJs', count: 12 },
{ title: 'NodeJs', count: 5 }
];
I have tried aggregate, $lookup but not able to find solution.
Also tried this answer
Following I have tried but its not giving desired output.
const result = await Tag.aggregate([
{
$lookup:
{
from: "articles",
localField: "_id",
foreignField: "tags",
as: "articles"
}
}
])
It gives output like this, it returns articles array against tag but I need count of articles only.
[{
"_id": "5f6f39c64250352ec80b0e10",
"title": "ReactJS",
articles: [{ ... }, { ... }]
},{
"_id": "5f6f40325716952d08a6813c",
"title": "Javascript",
articles: [{ ... }, { ... },{ ... }, { ... }]
}]
If anyone knows solution, please let me know. Thank you.

$lookup with articles collection
$project to show required fields and get total size of articles array using $size
const result = await Tag.aggregate([
{
$lookup: {
from: "articles",
localField: "_id",
foreignField: "tags",
as: "articles"
}
},
{
$project: {
_id: 0,
title: 1,
count: { $size: "$articles" }
}
}
])
Playground

Related

How can I populate nested array objects in mongoDB or mongoose?

I have an orders collection where each order has the following shape:
{
"_id": "5252875356f64d6d28000001",
"lineItems": [
{ productId: 'prod_007', quantity: 3 },
{ productId: 'prod_003', quantity: 2 }
]
// other fields omitted
}
I also have a products collection, where each product contains a unique productId field.
How can I populate each lineItem.productId with a matching product from the products collection? Thanks! :)
EDIT: orderSchema and productSchema:
const orderSchema = new Schema({
checkoutId: {
type: String,
required: true,
},
customerId: {
type: String,
required: true,
},
lineItems: {
type: [itemSubSchema],
required: true,
},
});
const itemSubSchema = new Schema(
{
productId: {
type: String,
required: true,
},
quantity: {
type: Number,
required: true,
},
},
{ _id: false }
);
const productSchema = new Schema({
productId: {
type: String,
required: true,
},
name: {
type: String,
required: true,
},
imageURL: {
type: String,
required: true,
},
price: {
type: Number,
default: 0,
},
});
I don't know the exact output you want but I think this is what you are looking for:
The trick here is to use $lookup in an aggregation stage.
First $unwind to deconstruct the array and can merge each id with the other collection.
Then the $lookup itself. This is like a join in SQL. It merges the desired objects with same ids.
Then recreate the population using $mergeObjects to get properties from both collections.
And last re-group objects to get the array again.
db.orders.aggregate([
{
"$unwind": "$lineItems"
},
{
"$lookup": {
"from": "products",
"localField": "lineItems.productId",
"foreignField": "_id",
"as": "result"
}
},
{
"$set": {
"lineItems": {
"$mergeObjects": [
"$lineItems",
{
"$first": "$result"
}
]
}
}
},
{
"$group": {
"_id": "$_id",
"lineItems": {
"$push": "$lineItems"
}
}
}
])
Example here
With this query you have the same intial data but "filled" with the values from the other collection.
Edit: You can also avoid one stage, maybe it is clear with the $set stage but this example do the same as it merge the objects in the $group stage while pushing to the array.
You can use the Mongoose populate method either when you query your documents or as middleware. However, Mongoose only allows normal population on the _id field.
const itemSubSchema = new Schema({
product: {
type: mongoose.Schema.Types.ObjectId,
ref: 'productSchema',
}
});
const order = await orderSchema.find().populate('lineItems.$*.product');
// special populate syntax necessary for nested documents
Using middleware you would still need to reconfigure your item schema to save the _id from products. But this method would automatically call populate each time you query items:
itemSubSchema.pre('find', function(){
this.populate('product');
});
You could also declare your item schema within your order schema to reduce one layer of joining data:
const orderSchema = new Schema({
lineItems: [{
type: {
quantity: {type: Number, required: true},
product: {
type: mongoose.Schema.Types.ObjectId,
required: true,
ref: 'productSchema',
}
},
required: true,
}]
});
const orders = orderSchema.find().populate('lineItems');

MongoDB Realm: Query a Relationship

here is my problem:
export const PostSchema = {
name: "Post",
primaryKey: "_id",
properties: {
_id: "string",
comments: "Comment[]"
},
};
export const CommentSchema = {
name: "Comment",
primaryKey: "_id",
properties: {
_id: "string",
post: "Post"
},
};
// Original Query:
const posts = realm.objects("Post");
console.log(posts);
// Original Query query results in:
[
{
_id: "post_1",
comments: [],
},
{
_id: "post_2",
comments: [],
},
...
];
// I'm looking for a query with the following result:
[
{
_id: "post_1",
comments: [
{
_id: "Only the first (newest) comment post_1",
},
],
},
{
_id: "post_2",
comments: [
{
_id: "Only the first (newest) comment post_2",
},
],
},
...
];
//My current solution therefor is:
let data = realm.objects("Post").map((post) => {
return {
...post.toJSON(),
comments: m.linkingObjects("Comment", "post").slice(0, 1),
};
});
I'm not sure if this is the right/optimal solution.
If I understood correctly, then I make an extra request for the first comment for each post?
So if I have 20 posts, that's 20 extra requests? That would be very impractical?
Is there a better way?
I also tried Inverse Relationships.
But with this method I always get all the comments for a post which is a lot of unnecessary data?
THX for any help :-)
db.posts.aggregate([
{
$lookup: {
from: "comments",
localField: "comments",
foreignField: "_id",
as: "comments",
pipeline: [
{ $sort: { _id: 1 } },
{ $limit: 1 }
]
}
}
])
mongoplayground

Mongoose query on multiple populated fields

I have three collections i.e Events, News and FuneralNews.
I have another Notifications collection that is just a combination of all three and contains the reference Id of either/one of the three collections.
I want to fetch only those Notifcations whose eventId OR newsId OR funeralNewsId field isActive is true
EventsSchema:
var EventsSchema = new Schema({
...
isActive: Boolean
});
FuneralNewsSchema:
var FuneralNewsSchema = new Schema({
...
isActive: Boolean
});
NewsSchema:
var NewsSchema = new Schema({
...
isActive: Boolean
});
NotificationSchema:
var NotificationSchema = new Schema({
type: {
type: String,
required: true
},
creationDate: {
type: Date,
default: Date.now
},
eventId: {type: Schema.Types.ObjectId, ref: 'Events'},
newsId: {type: Schema.Types.ObjectId, ref: 'News'},
funeralNewsId: {type: Schema.Types.ObjectId, ref: 'FuneralNews'},
organisationId: {type: Schema.Types.ObjectId, ref: 'Organization'}
});
This was my query before I need a check on isActive property of referenced collection:
let totalItems;
const query = { organisationId: organisationId };
Notification.find(query)
.countDocuments()
.then((count) => {
totalItems = count;
return Notification.find(query, null, { lean: true })
.skip(page * limit)
.limit(limit)
.sort({ creationDate: -1 })
.populate("eventId")
.populate("newsId")
.populate("funeralNewsId")
.exec();
})
.then((notifications, err) => {
if (err) throw new Error(err);
res.status(200).json({ notifications, totalItems });
})
.catch((err) => {
next(err);
});
Now I don't know how to check on isActive field of three populated collections prior population.
I have seen other questions like this and this but being a newbie can't edit it according to my use-case. Any help would be highly appreciated
use $lookup for each objectId refrence
then group by _id of null to get data and add myCount as total number put original data to array
and use unwind to destruct array and use addField
model
.aggregate([
{
$lookup: {
from: "Events", // events collection name
localField: "eventId",
foreignField: "_id",
as: "events",
},
},
{
$lookup: {
from: "FuneralNews", //FuneralNews collection name
localField: "funeralNewsId",
foreignField: "_id",
as: "funeralnews",
},
},
{
$lookup: {
from: "News", // news collection name
localField: "newsId",
foreignField: "_id",
as: "news",
},
},
{
$match: {
$or: [
{ "news.isActive": true },
{ "events.isActive": true },
{ "funeralnews.isActive": true },
],
},
},
{
$group: {
_id: null,
myCount: {
$sum: 1,
},
root: {
$push: "$$ROOT",
},
},
},
{
$unwind: {
path: "$root",
},
},
{
$addFields: {
"root.total": "$myCount",
},
},
{
$replaceRoot: {
newRoot: "$root",
},
},
{
$sort: {
creationDate: -1,
},
},
])
.skip(page * limit)
.limit(limit);

cross collection query in mongoose with $lookup

I am building a question-answers database. I have three schemas, User, Question and Replies(answers). My problem begins where I want to query just the questions that the user has not already answered. but when I run this query I get the question with an empty replies array. should I populate it somehow? here's where I got:
Query:
let getQuestions = (req, res)=>{
Question.aggregate([
{
$match: {isGlobal: true}
},
{
$lookup: {
from: 'Reply',
localField: '_id',
foreignField: 'questionID',
as: 'replies'
}
},
// {
// $match: {}
// }
]).then(foundQuestions=> console.log(foundQuestions))
};
User Schema (simplified):
const mongoose = require('mongoose');
const userSchema = mongoose.Schema({
firstName: String,
lastName: String,
email: String,
questionReplies: [
{
type: mongoose.Schema.Types.ObjectId,
ref: 'Reply'
}
],
}, {timestamps: true});
module.exports = mongoose.model('User', userSchema);
Question Schema:
const mongoose = require('mongoose');
const questionSchema = mongoose.Schema({
title: String,
description: String,
isGlobal: Boolean,
options: [
{
number: Number,
title: String,
}
]
}, {timestamps: true});
module.exports = mongoose.model('Question', questionSchema);
Reply Schema:
const mongoose = require('mongoose');
const replySchema = mongoose.Schema({
questionID: {
type: mongoose.Schema.Types.ObjectId,
ref: 'Question',
required: true
},
email: {type: String, required: true},
answerID: {type: String, required: true},
number: Number,
title: String,
}, {timestamps: true});
module.exports = mongoose.model('Reply', replySchema);
my replies collection has this document inside:
"questionID" : ObjectId("5c6f6867cbff9c2a9004eb6d")
and I have a question with this ID:
"_id" : ObjectId("5c6f6867cbff9c2a9004eb6d"),
(any opnions on improving database design are welcome too).
Try below:
let getQuestions = (req, res)=>{
Question.aggregate([
{
$match: {isGlobal: true}
},
{
$lookup: {
from: 'Reply',
localField: '_id',
foreignField: 'questionID',
as: 'replies'
}
},
{
$match: { "replies": { $eq: [] } }
}
]).then(foundQuestions=> console.log(foundQuestions))
};
So you will get question which does not have any replies.
I solved my problem and I post here for future references.
the from field used in $lookup refers to the collection name created by mongodb, not the Model name used in mongoose. So the correct query is:
$lookup: {
from: 'replies',
localField: '_id',
foreignField: 'questionID',
as: 'replies'
}
and then I added
{
$match: { "replies": { $eq: [] } }
}
for finding just the questions with no answers.

Mongoose populate and sort by length struggle

I have the following mongoose schema:
const userSchema = new mongoose.Schema({
email: { type: String, unique: true },
fragments: [{type: mongoose.Schema.Types.ObjectId, ref: 'Fragment'}]
}, { timestamps: true, collection: 'user' });
And
const fragmentSchema = new mongoose.Schema({
text: String,
owner: {type: mongoose.Schema.Types.ObjectId, ref: 'User'},
}, { timestamps: true, collection: 'fragment' });
In the data, I have a reference in the Fragment, but not in the User:
User:
{
"_id" : ObjectId("58373e571cbccb010012bfcd"),
"email" : "email#example.com",
// no "fragments": [ObjectId('58075ce37b7f2f01002b718f')] etc.
}
Fragment:
{
"_id" : ObjectId("58075ce37b7f2f01002b718f"),
"text" : "Donc, il faut changer de méthode",
"owner" : ObjectId("58075ce27b7f2f01002b717f")
}
I would like to query users sorted by the count of number of fragments, and I can't achieve this ...
First, I'd like to make this work:
User.find({_id: '58075ce27b7f2f01002b717f'})
.populate('fragments').exec(console.log)
returns
{
_id: 58075ce27b7f2f01002b717f,
email: 'bububg#hotmail.fr',
fragments: []
}
while I should have at least the above fragment included.
And regarding the sorted query, here's where I am now:
User.aggregate([
{ "$project": {
"email": 1,
"fragments": 1,
"nbFragments": { "$size": { "$ifNull": [ "$fragments", [] ] } }
}},
{ "$sort": { "nbFragments": -1 } }
], console.log)
At least it runs, but all the nbFragments fields are set to 0. This might be related to the fact that .populate('fragments') doesn't work but I can't be sure.
Thanks for the help, I did not expect so much trouble using Mongodb...
EDIT: thanks #Veeram, unfortunately your solution is not working:
User.find({}).find({_id: '58075ce27b7f2f01002b717f'}).populate('fragments').exec(console.log)
[ { _id: 58075ce27b7f2f01002b717f,
email: 'email#example.com',
// no fragments entry
} ]
while I updated my schema:
userSchema.virtual('fragments', {
ref: 'Fragment',
localField: '_id',
foreignField: 'owner',
options: { sort: { number: 1 }}, // Added sort just as an example
});
And regarding the aggregate, with:
User.aggregate([{
$lookup: {
from: 'Fragment',
localField: '_id',
foreignField: 'owner',
as: 'fragments'
}
}, { "$project": {
"email": 1,
"fragments": 1,
"nbFragments": {
"$size": { "$ifNull": [ "$fragments", [] ] } }
}}, { "$sort": { "nbFragments": -1 } }
]).exec(console.log)
I get:
{
_id: 58075ce27b7f2f01002b717f,
email: 'email#example.com',
fragments: [] // fragments are always empty while they shouldn't!
}
Tested with following data
User:
{
"_id" : ObjectId("58373e571cbccb010012bfcd"),
"email" : "email#example.com"
}
Fragment:
{
"_id" : ObjectId("58075ce37b7f2f01002b718f"),
"text" : "Donc, il faut changer de méthode",
"owner" : ObjectId("58373e571cbccb010012bfcd")
}
Response
[{"_id":"58373e571cbccb010012bfcd","email":"email#example.com","fragments":[{"_id":"58075ce37b7f2f01002b718f","text":"Donc, il faut changer de méthode","owner":"58373e571cbccb010012bfcd"}],"nbFragments":1}]
You define schema to use owner to populate the fragments also called virtual population. http://mongoosejs.com/docs/populate.html
const userSchema = new mongoose.Schema({
email: { type: String, unique: true }
}, { timestamps: true, collection: 'user' });
var User = mongoose.model("User", userSchema);
const fragmentSchema = new mongoose.Schema({
text: String,
owner: {type: mongoose.Schema.Types.ObjectId, ref: 'User'},
}, { timestamps: true, collection: 'fragment' });
var Fragment = mongoose.model("Fragment", fragmentSchema);
userSchema.virtual('fragments', {
ref: 'Fragment',
localField: '_id',
foreignField: 'owner',
options: { sort: { text: -1 }}, // Added sort just as an example
});
This will now work as expected, but I don't know a way to sort on some dynamic field like count of number of fragments in mongoose. I don't think it is possible
User.find({_id: '58373e571cbccb010012bfcd'})
.populate('fragments').exec(function (err, user) {
console.log(JSON.stringify(user));
});
Okay now for dynamic sorting, you have to use alternative raw mongo query with a $lookup (equivalent of populate).
const userSchema = new mongoose.Schema({
email: { type: String, unique: true }
}, { timestamps: true, collection: 'user' });
var User = mongoose.model("User", userSchema);
const fragmentSchema = new mongoose.Schema({
text: String,
owner: {type: mongoose.Schema.Types.ObjectId, ref: 'User'},
}, { timestamps: true, collection: 'fragment' });
User.aggregate([{
$lookup: {
from: 'fragment',
localField: '_id',
foreignField: 'owner',
as: 'fragments'
}
}, { "$project": {
"email": 1,
"fragments": 1,
"nbFragments": {
"$size": { "$ifNull": [ "$fragments", [] ] } }
}}, { "$sort": { "nbFragments": -1 } }
]).exec(function (err, user) {
console.log(JSON.stringify(user));
})