Mongodb - count from a sub collection inside aggregate lookup - mongodb

I have the following mongo db query that works:
.collection('commentsPosts')
.aggregate(
[
{$match:{
commentedAt: {
$lte: from,
},
...(postId && { postId }),
}
},
{$lookup:
{
from:"users",
localField:"commentedBy",
foreignField:"_id",
as:"commenterDetails"
// NEED TO LOOK UP "posts" collection here to find number of posts by commenterDetails.userId = posterId in posts table
},
}
]
)
I need to go one more level down to TO LOOK UP count of posts to find number of posts where commenterDetails.userId = posterId in posts collection
I am looking for a way to add the working query below to the query above.
const userPostCount = await req.db.collection('posts').countDocuments({
creatorId: userId,
});

You can use nested $lookups like this: https://mongoplayground.net/p/_il8aiilLh5
db.comments.aggregate([
{$lookup:
{
from:"users",
let: {userId: "$commentedBy"},
as:"commenterDetails",
pipeline: [
{$match: {$expr: {
$eq: ["$$userId", "$_id"],
}}},
{$lookup: {
from: "posts",
as: "posts",
let: {posterId: "$_id"},
pipeline: [
{$match: {$expr: {
$eq: ["$posterId", "$$posterId"]
}}},
{$count: "total"},
]
}},
{$addFields: {
posts: {$arrayElemAt: ['$posts', 0]}
}},
]
},
},
{$addFields: {
commenterDetails: {$arrayElemAt: ['$commenterDetails', 0]}
}}
])

Related

Find MongoDB documents that are not contained across arrays

MongoDB Collection A contains documents with an array with some document ids of collection B:
Collection A:
{
some_ids_of_b: ["id1", ...]
}
Collection B:
{
_id: "id1"
},
{
_id: "id2"
},
...
How do I query all documents from B whose _ids are NOT in contained in the some_ids_of_b arrays of documents of A?
Simple lookup from collection B to A and filter to keep only those documents where you don't find any matches.
db.collb.aggregate([
{
"$lookup": {
"from": "colla",
"localField": "_id",
"foreignField": "someIdsOfB",
"as": "a"
}
},
{
$match: {
$expr: {
$eq: [{$size: "$a"}, 0]
}
}
}
])
Demo
One option is:
db.collectionB.aggregate([
{$lookup: {
from: "collectionA",
let: {my_id: "$_id"},
pipeline: [
{$match: {$and: [
{_id: collADocId},
{$expr: {$in: ["$$my_id", "$some_ids_of_b"]}}
]}},
{$project: {_id: 1}}
],
as: "some_ids_of_b"
}},
{$match: {"some_ids_of_b.0": {$exists: false}}},
{$unset: "some_ids_of_b"}
])
See how it works on the playground example
You can do it with Aggregation Framework:
$group and $addToSet - To get all $some_ids_of_b from all the documents in A collection.
$set with $reduce - To create an array with all unique values of the IDs from the B collection.
$lookup - To fetch the documents from the B collection, where the _id of the document is not present in the $b_ids array.
$project - To project data as expected output.
db.A.aggregate([
{
"$group": {
"_id": null,
"b_ids": {
"$addToSet": "$some_ids_of_b"
}
}
},
{
"$set": {
b_ids: {
$reduce: {
input: "$b_ids",
initialValue: [],
in: {
$setUnion: [
"$$value",
"$$this"
]
}
}
}
}
},
{
"$lookup": {
from: "B",
let: {
b_ids: "$b_ids"
},
pipeline: [
{
"$match": {
"$expr": {
$ne: [
{
"$in": [
"$_id",
"$$b_ids"
]
},
true
]
}
}
}
],
as: "data"
}
},
{
"$project": {
data: 1,
_id: 0
}
}
])
Working Example

mongodb - find if value matches second element in array

Trying to return this document from a collection by checking if a variable (userId) matches the second element in users array.
I created a playground. Expected result is the user document xyz as thats the only user who liked rrr and the user rrr has not liked back - https://mongoplayground.net/p/WI7hqR7SIMh
Expected result:
[
{
"count": 1,
"users": [
{
"_id": "xyz",
"group": 1,
"name": "xyyy"
}
]
}
]
My query is below where variable userId is xw5vk1s and is the second element in above array. The two conditions I am checking are like: true and userId = second element is users array
const users = await db
.collection('users')
.aggregate([
{
$lookup: {
from: "likes",
let: {userId: "$_id"},
pipeline: [{$match: {$expr: {$and: [{like: true, "users.1": userId} ]}}}],
as: "valid"
}
},
{$match: {
"valid.0": {$exists: true},
}
},
{$unset: ["valid"]},
{$group: {_id: 0, users: {$push: "$$ROOT"}, count: {$sum: 1}}}
])
The query is not working.
One option is using two $lookups:
First find the pairs for partnerships that "likes" rrr
$match only the unpaired
Get the user data and format the response
db.partnership.aggregate([
{$match: {$expr: {$eq: [{$last: "$users"}, "rrr"]}}},
{$lookup: {
from: "partnership",
let: {likes: {$last: "$users"}, me: {$first: "$users"}},
pipeline: [
{$match: {
$expr: {$and: [
{$eq: [{$last: "$users"}, "$$me"]},
{$eq: [{$first: "$users"}, "$$likes"]}
]
}
}
}
],
as: "paired"
}
},
{$match: {"paired.0": {$exists: false}}},
{$project: {_id: 0, user: {$first: "$users"}}},
{$lookup: {
from: "users",
localField: "user",
foreignField: "_id",
as: "user"
}},
{$project: {user: {$first: "$user"}}},
{$replaceRoot: {newRoot: "$user"}}
])
See how it works on the playground example
** I hope this will solve your problem **
https://mongoplayground.net/p/8Ax_NaRhfHs
[
{
$addFields: {
secondItem: {
$arrayElemAt: [
"$users",
1
]
}
}
},
{
$match: {
$and: [
{
users: {
$in: [
"$users",
"xw5vk1s"
]
}
},
{
$expr: {
$eq: [
"$secondItem",
"xw5vk1s"
]
}
}
]
}
}
]

MongoDB aggregations group by and count with a join

I have MongoDB model called candidates
appliedJobs: [
{
job: { type: Schema.ObjectId, ref: "JobPost" },
date:Date
},
],
candidate may have multiple records in appliedJobs array. There I refer to the jobPost.
jobPost has the companyName, property.
companyName: String,
What I want is to get the company names with send job applications counts. For an example
|Company|Applications|
|--------|---------------|
|Facebook|10 applications|
|Google|5 applications|
I created this query
Candidate.aggregate([
{
$match: {
appliedJobs: { $exists: true },
},
},
{ $group: { _id: '$companyName', count: { $sum: 1 } } },
])
The problem here is I can't access the companyName like this. Because it's on another collection. How do I solve this?
In order to get data from another collection you can use $lookup (nore efficient) or populate (mongoose - considered more organized), so one option is:
db.candidate.aggregate([
{$match: {appliedJobs: {$exists: true}}},
{$unwind: "$appliedJobs"},
{$lookup: {
from: "JobPost",
localField: "appliedJobs.job",
foreignField: "_id",
as: "appliedJobs"
}
},
{$project: {companyName: {$first: "$appliedJobs.companyName"}}},
{$group: {_id: {candidate: "$_id", company: "$companyName"}, count: {$sum: 1}}},
{$group: {
_id: "$_id.candidate",
appliedJobs: {$push: {k: "$_id.company", v: "$count"}}
}},
{$project: {appliedJobs: {$arrayToObject: "$appliedJobs"}}}
])
See how it works on the playground example
Simply $unwind the appliedJobs array. Perform $lookup to get the companyName. Then, $group to get count of applications by company.
db.Candidate.aggregate([
{
$match: {
appliedJobs: {
$exists: true
}
}
},
{
$unwind: "$appliedJobs"
},
{
"$lookup": {
"from": "JobPost",
"localField": "appliedJobs._id",
"foreignField": "_id",
"as": "JobPostLookup"
}
},
{
$unwind: "$JobPostLookup"
},
{
"$group": {
"_id": "$JobPostLookup.companyName",
"Applications": {
"$sum": 1
}
}
}
])
Here is the Mongo Playground for your reference.

MongoDB: slow performance pipeline lookup compared to basic lookup

I have two collection:
matches:
[{
date: "2020-02-15T17:00:00Z",
players: [
{_id: "5efd9485aba4e3d01942a2ce"},
{_id: "5efd9485aba4e3d01942a2cf"}
]
},
{...}]
and players:
[{
_id: "5efd9485aba4e3d01942a2ce",
name: "Rafa Nadal"
},
{
_id: "5efd9485aba4e3d01942a2ce",
name: "Roger Federer"
},
{...}]
I need to use lookup pipeline because I'm building a graphql resolver with recursive functions and I need nested lookup. I've followed this example https://docs.mongodb.com/datalake/reference/pipeline/lookup-stage#nested-example
My problem is that with pipeline lookup I need 11 seconds but with basic lookup only 0.67 seconds. And my test database is very short! about 1300 players and 700 matches.
This is my pipeline lookup (11 seconds to resolve)
db.collection('matches').aggregate([{
$lookup: {
from: 'players',
let: { ids: '$players' },
pipeline: [{ $match: { $expr: { $in: ['$_id', '$$ids' ] } } }],
as: 'players'
}
}]);
And this my basic lookup (0.67 seconds to resolve)
db.collection('matches').aggregate([{
$lookup: {
from: "players",
localField: "players",
foreignField: "_id",
as: "players"
}
}]);
Why so much difference? In what way can I do faster pipeline lookup?
The thing is that when you do a lookup using pipeline with a match stage, then the index would be used only for the fields that are matched with $eq operator and for the rest index will not be used.
And the example you specified with pipeline will work like this ( again index will not be used here as it is not $eq )
db.matches.aggregate([
{
$lookup: {
from: "players",
let: {
ids: {
$map: {
input: "$players",
in: "$$this._id"
}
}
},
pipeline: [
{
$match: {
$expr: {
$in: [
"$_id",
"$$ids"
]
}
}
}
],
as: "players"
}
}
])
As players is an array of object so it need to be mapped to array of ids first
MongoDB Playground
As #namar sood comments there are several tickets that refer to this issue:
https://jira.mongodb.org/browse/SERVER-37470
https://jira.mongodb.org/browse/SERVER-32549
Meanwhile a solution could be (also works nested):
db.collection('matches').aggregate([
{ $unwind: '$players' },
{
$lookup: {
from: 'players',
let: { id: '$players' },
pipeline: [{ $match: { $expr: { $eq: ['$_id', '$$id' ] } } }],
as: 'players'
},
{ $unwind: '$players' },
{
$group: {
"_id": "$_id",
"data": { "$first": "$$ROOT" },
"players": {$push: "$players"}
}
},
{ $addFields: {"data.players": "$players"} },
{ $replaceRoot: { newRoot: "$data" }}
]);

let in lookup whit objectId

I want to use some ObjectId in my let but i have the same error every time :
Failed to execute script.
Error: invalid object id: length
Details:
#(shell):6:36
My query :
db.projects.aggregate([
{$lookup: {
from: 'pointValueChangements',
let: {id: '$_id'},
pipeline: [
{ $match: { projectId: new ObjectId('$$id'), date: {$lte: new Date()}}},
{ $sort: { date: -1}},
{ $limit: 1},
],
as: 'pointValue',
}},
{$project: {pointValue: 1}}
])
If i replace new ObjectId('$$id') by '$$id' the query work but the result is not good.
If i replace '$$id' by ObjectId(someMongoId) all is ok (but is not good because this id match only for one row...)
Try this query
db.projects.aggregate([
{$lookup: {
from: 'pointValueChangements',
let: {id: '$_id'},
pipeline: [
{ $match: {
$expr:
{
$and:
[
{ $eq: ['$projectId', '$$id'] },
{ date: {$lte: new Date() }},
],
},
}},
{ $sort: { date: -1}},
{ $limit: 1},
],
as: 'pointValue',
}},
{$project: {pointValue: 1}}
])