How to batch query by an array in mongodb? - mongodb

Question

I have a table Thread:
{
userId: String
messageId: String
}
Now I have an array of userIds, I need to query 20 messageIds for each of them, I can do it with a loop:
const messageIds = {}
for (const userId of userIds) {
const results = await Thread.find({ userId }).sort({ _id: -1 }).limit(20).exec()
messageIds[userId] = results.map(result => result.messageId)
}
But of course this doesn't perform well. Is there a better solution?

Answer 1

The problem with your approach is that you are issuing multiple separate queries to MongoDB.
The simplest workaround to this is using the $push and $slice approach. But this has the problem that the intermediate step would creating an array of huge size.
Another way could be to use $facet as part of aggregation query.
So you need a $facet step in the aggregation like -
[
{$facet: {
'userId1': [
{$match: {userId: 'userId1'} },
{$limit: 20},
{$group: {_id: '', msg: {$push: '$messageId'} } }
],
'userId2': [
{$match: {userId: 'userId2'} },
{$limit: 20},
{$group: {_id: '', msg: {$push: '$messageId'} } }
],
.... (for each userId in array)
}}
]
You can easily just generate this query by iterating over the list of users and adding keys for each user.
So you end up with an object where key is the userId and the value is the array of messages (obj[userId].msg)

Answer 2

You can use aggregation to group threads by userId, and return the top 20:
db.threads.aggregate([
{$match: {userId:{$in: userIds}}},
{$sort: {_id: -1}},
{$group: {_id: "$userId", threads: {$push: "$$ROOT"}}},
{$project: {_id:0, userId:"$_id", threads: {$slice:["$threads", 20]}}}
])