Group by, count and stream individual results from mongodb query - mongodb

Question

In mongodb, having a collection with sessionIds and labels, I would like to group by the sessionId where label equals 'view_item' and accomplish:
Get the count of sessionId groups.
Be able to stream each sessionId to the consumer (assuming I have limited memory resources and a large number individual sessionIds)
Assume following documents in a collection:
{ "label" : "view_item", "sessionId" : "01e5dnnpsczgfq58rmp0cjtjm0" }
{ "label" : "view_category", "sessionId" : "01e5dnnpsczgfq58rmp0cjtjm0" }
{ "label" : "view_item", "sessionId" : "01e5dnnpsczgfq58rmp0cjtjm0" }
{ "label" : "view_item", "sessionId" : "01e5g7vzx5dh0mv8m6g1zbdrnj" }
{ "label" : "view_item", "sessionId" : "01e5g7vzx5dh0mv8m6g1zbdrnj" }
{ "label" : "view_category", "sessionId" : "01e5g7vzx5dh0mv8m6g1zbdrnj" }
{ "label" : "view_item", "sessionId" : "01e5g7vzx5dh0mv8m6g1zbdrnj" }
The expected result would be something like this:
Get results somehow and...
result.count() // 2 (or some other way of getting the count)
await result.next() // { sessionId: '01e5dnnpsczgfq58rmp0cjtjm0' }
await result.next() // { sessionId: '01e5g7vzx5dh0mv8m6g1zbdrnj' }
await result.next() // null
I've been fiddling with the aggregation framework and manage to group and count. In theory I could do two queries to first get count and then the groups, but in a frequent write scenario I'm worried that doing two separate queries could lead to inconsistencies, especially since I haven't figured out how to include any start / end ids in the result from the count query, which could be used to confine the results from the groups query.
What I have so far is:
const result = collection.aggregate([
{ $match: { label: 'view_item' } },
{ $group : { _id: { sessionId: '$sessionId' } } },
]);
await result.next() // { _id: { sessionId: '01e5g7vzx5dh0mv8m6g1zbdrnj' } }
await result.next() // { _id: { sessionId: '01e5dnnpsczgfq58rmp0cjtjm0' } }
await result.next() // null
and
const result = collection.aggregate([
{ $match: { label: 'view_item' } },
{ $group : { _id: { sessionId: '$sessionId' } } },
{ $facet: { count: [{ $count: 'count' }] } }
]);
await result.next() // { count: [ { count: 2 } ] }
await result.next() // null
Question
How can the two queries above be combined to reliably get the count and a result with the grouped sessionId that can be streamed? (I assume any solution relying on result.toArray().length needs to load the whole result in memory, which is ruled out).
Is it possible to do in one single query or more likely to get the count and start / end ids in one query and then do a second query to get the groups confined by the start / end ids?
Thanks!

Answer 1

If I understand you requirements clearly, you need to gather all the sessions that have been assigned to each label in one array, and count that sessions
if so, we may use the $group to group the sessions assigned to each label,
and $size to calculate that array length
we may do something like that
db.collection.aggregate([
{
$match: {} // if you need the 'View_Item' labels only, than add it here
},
{
$group: {
_id: "$label", // make the _id of the results is the label
sessionsIds: { // array of sessions
$push: "$sessionId"
}
}
},
{
$project: { // use the $project as $size is available only in the $project stage
_id: 1,
sessionsIds: 1,
sessionsCount: {
$size: "$sessionsIds"
}
}
}
])
you could try that here in Mongo Playground
Update, If you need to get the number of unique sessions Ids, and no duplication in sessionsIds array, use $addToSet instead of $push
update 2: If we need to group by the sessionId and count how many documents have this sessionId, we can do something like
db.collection.aggregate([
{
$match: {} // if you need the 'View_Item' labels only, than add it here
},
{
$group: {
_id: "$sessionId",
count: {
$sum: 1
}
}
}
])
this will return a result
[
{
"_id": "01e5dnnpsczgfq58rmp0cjtjm0",
"count": 3
},
{
"_id": "01e5g7vzx5dh0mv8m6g1zbdrnj",
"count": 4
}
]
if you need to make the _id of the result be an object rather than ObjectId, we could do something like
db.collection.aggregate([
{
$match: {} // if you need the 'View_Item' labels only, than add it here
},
{
$group: {
_id: {
sessionId: "$sessionId"
},
count: {
$sum: 1
}
}
}
])
this will result in
[
{
"_id": {
"sessionId": "01e5dnnpsczgfq58rmp0cjtjm0"
},
"count": 3
},
{
"_id": {
"sessionId": "01e5g7vzx5dh0mv8m6g1zbdrnj"
},
"count": 4
}
]
you can try all of that here Mongo_Playground 2