I have two collections:
Users: [{
_id: 'xxx',
name: 'xxx',
},
...
]
Posts: [{
_id: 'xxx',
userId: 'xxx',
},
...
]
So a user can have multiple posts. I want to get users with the number of posts that each user has. If the user doesn't have any post, it should load 0.
So the result will be:
[{
_id: 'xxx',
name: 'xxx',
numberOfPosts: 'xxx'
},
...
]
Below query is what I have wrote:
$lookup: {
from: 'Posts',
let: {
userId: '$_id',
},
pipeline: [{
$match: {
$expr: {
$eq: ["$userId", "$$userId"]
}
}
}, {
$count: 'posts'
}, {
$project: {
posts: {
$cond: [{
$ifNull: ['$posts', true]
},
'$posts',
0
]},
}
}],
as: 'posts'
}
It doesn't give me 0 if a user doesn't have any post. What was wrong in the query?
You can try simple way,
use $lookup without pipeline
$addFields to count the number of posts using $size
db.Users.aggregate([
{
$lookup: {
from: "Posts",
localField: "_id",
foreignField: "userId",
as: "numberOfPosts"
}
},
{
$addFields: {
numberOfPosts: {
$size: "$numberOfPosts"
}
}
}
])
Playground
Related
I've inherited a Azure Cosmos database with a MongoDB API. There is extensive use of "discriminators" so a single collection has many different models.
I am trying to query a document three levels deep based on document ids (ObjectId())
Parent Group
{
_id: ObjectId(),
__type: "ParentGroup",
name: "group 1",
subgroups: [
...ObjectIds,
],
}
Sub Group
{
_id: ObjectId(),
__type: "SubGroup",
name: "a text name",
members: [
...ObjectIds,
],
}
Member
{
_id: ObjectId(),
__type: "Member",
name: "string",
email: "",
induction: Date,
}
Examples I've seen deal with nested documents NOT references
Is it possible to query the Member documents and return?
[
{
parentGroup,
subgroups: [
{sub group, members: [...members]},
{sub group, members: [...members]},
{sub group, members: [...members]},
]
},
]
After reading the comments and further reading i've got this. Its almost there but I think the limitation of MongoDB will prevent the solution being in a single query. The goal is to return ParentGroups->Subgroups->Members Where Members have an "induction" value of "whatever". I am either returning ALL ParentGroups or nothing at all
db.development.aggregate([
{
$match: {
__type: "ParentGroup", $expr: {
$gt: [
{ $size: "$subgroups" }, 0
]
}
}
},
{
$lookup: {
from: "development",
localField: "subgroups",
foreignField: "_id",
as: "subgroups"
}
},
{
$unwind: {
path: "$subgroups",
// preserveNullAndEmptyArrays: true
}
},
{
$lookup: {
from: "development",
localField: "subgroups.members",
foreignField: "_id",
as: "subgroups.members"
}
}
])
Solution that worked for me:
db.development.aggregate([
{
$match: {
__type: "ParentGroup",
},
},
{
$lookup: {
from: "development",
localField: "subgroups",
foreignField: "_id",
as: "subgroups",
},
},
{
$unwind: {
path: "$subgroups",
preserveNullAndEmptyArrays: true,
},
},
{
$lookup: {
from: "development",
localField: "subgroups.members",
foreignField: "_id",
as: "subgroups.activities_x",
},
},
{
$unwind: {
path: "$subgroups.members",
preserveNullAndEmptyArrays: true,
},
},
{
$match: { "subgroups.members.meta": { $exists: true } },
},
{
$project: {
_id: 1,
__type: 1,
name: 1,
subgroups: {
_id: 1,
__type: 1,
name: 1,
members: {
_id: 1,
__type: 1,
name: 1,
meta: 1,
},
},
},
},
]);
I've built 3 queries and I am trying to optimize one of them because in my opinion is the best way to do it. Unfortunately, I don't get the performance I expected.
Fastest query:
Simple lookups only, but you need to unset a lot of data in the last stage. Imagine having to do that for 20 or 30 fields you do not want. If you use $project you will mess up with the todos collection data, so unset is the only way to do it.
Execution time for my set of data is: 176ms with my data set offcourse
db.todos
.aggregate([
{
$lookup: {
from: 'users',
localField: 'user_id',
foreignField: '_id',
as: 'user',
},
},
{
$unwind: {
path: '$user',
preserveNullAndEmptyArrays: true,
},
},
{
$lookup: {
from: 'user_data',
localField: 'user.data_id',
foreignField: '_id',
as: 'user.data',
},
},
{
$unwind: {
path: '$user.data',
preserveNullAndEmptyArrays: true,
},
},
{ $unset: ['user._id', 'user.data_id', 'user.deleted', 'user.active', 'user.data._id'] },
])
.explain();
My Favorite:
Advanced nested lookups. I like it cause it is clean and clear but I don't think this one uses indexes probably because $expr is used even if it is done on _id.
Execution time for my set of data is: 713ms
db.todos
.aggregate([
{
$lookup: {
from: 'users',
let: { user_id: '$user_id' },
pipeline: [
{ $match: { $expr: { $eq: ['$_id', '$$user_id'] } } },
{
$lookup: {
from: 'user_data',
let: { data_id: '$data_id' },
pipeline: [
{ $match: { $expr: { $eq: ['$_id', '$$data_id'] } } },
{ $project: { _id: 0, name: 1 } },
],
as: 'data',
},
},
{ $unwind: '$data' },
{ $project: { _id: 0, email: 1, data: 1 } },
],
as: 'user',
},
},
{
$unwind: {
path: '$user',
preserveNullAndEmptyArrays: true,
},
},
])
.explain();
Worst one:
Advanced lookups(not nested). Slowest query.
Execution time for my set of data is: 777ms
db.todos
.aggregate([
{
$lookup: {
from: 'users',
let: { user_id: '$user_id' },
pipeline: [
{ $match: { $expr: { $eq: ['$_id', '$$user_id'] } } },
{ $project: { _id: 0, email: 1, data_id: 1 } },
],
as: 'user',
},
},
{
$unwind: {
path: '$user',
preserveNullAndEmptyArrays: true,
},
},
{
$lookup: {
from: 'user_data',
let: { data_id: '$user.data_id' },
pipeline: [
{ $match: { $expr: { $eq: ['$_id', '$$data_id'] } } },
{ $project: { _id: 0, name: 1 } },
],
as: 'user.data',
},
},
{
$unwind: {
path: '$user.data',
preserveNullAndEmptyArrays: true,
},
},
{ $unset: ['user.data_id'] },
])
.explain();
I was surprised (being a Mongo beginner) that the way I thought about building queries is not the most effective way. Is there a way to improve the second option or am I stuck with the first one?
A todo
{"_id":{"$oid":"612156ec810895cfe9f406bd"},"user_id":{"$oid":"61214827810895cfe9f406ac"},"title":"todo 1"}
A user
{"_id":{"$oid":"61216d36810895cfe9f40713"},"active":true,"deleted":false,"data_id":{"$oid":"61216d42810895cfe9f40716"},"email":"test2#test2.com"}
A user data
{"_id":{"$oid":"6121488f810895cfe9f406ad"},"name":{"first":"Fanny","last":"Lenny"}}
Thank you
I have 2 collections, the first storing the animes watched by each user, their status etc:
const listSchema = mongoose.Schema({
user: {
type: Schema.Types.ObjectId,
ref: 'user'
},
animes: [{
_id: false,
anime: {
type: Schema.Types.ObjectId,
ref: 'anime',
unique: true
},
status: String,
episodes: Number,
rating: Number
}]
});
and the second storing all animes and the relevant information.
I want to show that second collection but adding status and episodes fields that are filled from the list of the logged in user.
I tried the following:
Anime.aggregate([
{
$lookup: {
'from': 'lists',
localField: '_id',
foreignField: 'animes.anime',
'as': 'name'
},
},
{
$unwind: '$animes'
},
{
$project:{
status:'$lists.animes.status'
}
}
]
)
but it returns an empty array.
I also don't know how to filter by userId seeing how the _id is in the foreign collection.
You can use below aggregation
{ "$lookup": {
"from": "lists",
"let": { "id": "$_id" },
"pipeline": [
{ "$match": { "$expr": { "$in": ["$$id", "$animes.anime"] }}},
{ "$unwind": "$animes" },
{ "$match": { "$expr": { "$eq": ["$animes.anime", "$$id"] }}}
],
"as": "name"
}}
-
Hi , i have an aggregation query with lookup, i need to project specific fields from this lookup and slice them. This is what I've done so far.
{
$lookup: {
from: 'users',
localField: 'users',
foreignField: '_id',
as: 'users',
}},
I've added the unwind statement
{
$unwind: {
path: '$users',
preserveNullAndEmptyArrays: true
}},
I've added the group statement
{
$group: {
_id: {
_id: '$_id',
createdAt: '$createdAt',
updatedAt: '$updatedAt'
},
users: {
$addToSet: '$users',
}
}
},
And to project specific fields in array of users i did:
{
$project: {
_id: '$_id._id',
createdAt: '$_id.createdAt',
updatedAt: '$_id.updatedAt',
// users: {
// $slice: [
// "$users",
// skip,
// limit
// ]
// },
users: {
$map: {
input: '$users',
as: 'user',
in: {
email: '$$user.email',
name: '$$user.name',
username: '$$user.username',
updatedAt: '$$user.updatedAt'
}
}
}
}},
My question is , How can i use $slice in this scope ?
I don't know how 'legit' is this but , I've added fields to $addToSet statement in $group , so now i can use $slice with mapped fields.
{
$group: {
_id: {
_id: '$_id',
createdAt: '$createdAt',
updatedAt: '$updatedAt',
},
users: {
$addToSet: {
_id: '$users._id',
email: '$users.email',
name: '$users.name',
username: '$users.username',
updatedAt: '$users.updatedAt'
}
}
}
}
Now i can easily do $slice in $project statement.
{
$project: {
_id: '$_id._id',
users: {
$slice: [
"$users",
skip,
limit
]
}
},
}
If someone has a better solution , i would like to know.
I have two collections :
Student
{
_id: ObjectId("657..."),
name:'abc'
},
{
_id: ObjectId("593..."),
name:'xyz'
}
Library
{
_id: ObjectId("987..."),
book_name:'book1',
issued_to: [
{
student: ObjectId("657...")
},
{
student: ObjectId("658...")
}
]
},
{
_id: ObjectId("898..."),
book_name:'book2',
issued_to: [
{
student: ObjectId("593...")
},
{
student: ObjectId("594...")
}
]
}
I want to make a Join to Student collection that exists in issued_to array of object field in Library collection.
I would like to make a query to student collection to get the student data as well as in library collection, that will check in issued_to array if the student exists or not if exists then get the library document otherwise not.
I have tried $lookup of mongo 3.6 but I didn`t succeed.
db.student.aggregate([{$match:{_id: ObjectId("593...")}}, $lookup: {from: 'library', let: {stu_id:'$_id'}, pipeline:[$match:{$expr: {$and:[{"$hotlist.clientEngagement": "$$stu_id"]}}]}])
But it thorws error please help me in regard of this. I also looked at other questions asked at stackoverflow like. question on stackoverflow,
question2 on stackoverflow but these are comapring simple fields not array of objects. please help me
I am not sure I understand your question entirely but this should help you:
db.student.aggregate([{
$match: { _id: ObjectId("657...") }
}, {
$lookup: {
from: 'library',
localField: '_id' ,
foreignField: 'issued_to.student',
as: 'result'
}
}])
If you want to only get the all book_names for each student you can do this:
db.student.aggregate([{
$match: { _id: ObjectId("657657657657657657657657") }
}, {
$lookup: {
from: 'library',
let: { 'stu_id': '$_id' },
pipeline: [{
$unwind: '$issued_to' // $expr cannot digest arrays so we need to unwind which hurts performance...
}, {
$match: { $expr: { $eq: [ '$issued_to.student', '$$stu_id' ] } }
}, {
$project: { _id: 0, "book_name": 1 } // only include the book_name field
}],
as: 'result'
}
}])
This might not be a very good answer, but if you can change your schema of Library to:
{
_id: ObjectId("987..."),
book_name:'book1'
issued_to: [
ObjectId("657..."),
ObjectId("658...")
]
},
{
_id: "ObjectId("898...")",
book_name:'book2'
issued_to: [
ObjectId("593...")
ObjectId("594...")
]
}
Then when you do:
{
$lookup: {
from: 'student',
localField: 'issued_to',
foreignField: '_id',
as: 'issued_to_students', // this creates a new field without overwriting your original 'issued_to'
}
},
You should get, based on your example above:
{
_id: ObjectId("987..."),
book_name:'book1'
issued_to_students: [
{ _id: ObjectId("657..."), name: 'abc', ... },
{ _id: ObjectId("658..."), name: <name of this _id>, ... }
]
},
{
_id: "ObjectId("898...")",
book_name:'book2'
issued_to: [
{ _id: ObjectId("593..."), name: 'xyz', ... },
{ _id: ObjectId("594..."), name: <name of this _id>, ... }
]
}
You need to $unwind the issued_to from library collection to match the issued_to.student with _id
db.student.aggregate([
{ "$match": { "_id": mongoose.Types.ObjectId(id) } },
{ "$lookup": {
"from": Library.collection.name,
"let": { "studentId": "$_id" },
"pipeline": [
{ "$unwind": "$issued_to" },
{ "$match": { "$expr": { "$eq": [ "$issued_to.student", "$$studentId" ] } } }
],
"as": "issued_to"
}}
])