Aggregate on populated value with mongoose - mongodb

Question

I will really appreicate your help with the following scenario .
I have this schema:
var Song = Schema({
author: { type: Schema.Types.ObjectId, ref: 'user' },
title: String,
photo: String,
date: Date,
duration: Number,
views: [{ type: Schema.Types.ObjectId, ref: 'user' }],
likes: [{ type: Schema.Types.ObjectId, ref: 'user' }]
})
var User = mongoose.Schema({
email:String,
name:String,
gender: String,
birthday: String,
city: String,
continent: String
});
I want to write a query that will present the user with the total likes and views from his songs group by each continent. For example:
{continent: 'Asia', views:4000, likes:5000},
{continent: 'Europe', views:3200, likes:4500}
Also I will be happy to know if its considered a "heavy query", and maybe if it is a smarter idea to save the like and view as a combination of the userId with the continent.

Answer 1

Rather than .populate() ( which is a "client" side operation ) you want the data to "join" on the server, where .aggregate() is a "server" side operation. This is what the $lookup operator is for:
It's probably most optimal to $map a "type" and $concatArrays first before doing the $lookup
Song.aggregate([
{ "$project": {
"author": "$author",
"data": {
"$concatArrays": [
{ "$map": {
"input": "$views",
"as": "el",
"in": { "type": "views", "_id": "$$el" }
}},
{ "$map": {
"input": "$likes",
"as": "el",
"in": { "type": "likes", "_id": "$$el" }
}}
]
}
}},
{ "$unwind": "$data" },
{ "$lookup": {
"from": "users",
"localField": "data._id",
"foreignField": "_id",
"as": "data._id"
}},
{ "$unwind": "$data._id" },
{ "$group": {
"_id": {
"author": "$author",
"continent": "$data._id.continent"
},
"views": {
"$sum": { "$cond": [ { "$eq": [ "$data.type", "views" ] }, 1, 0 ] }
},
"likes": {
"$sum": { "$cond": [ { "$eq": [ "$data.type", "likes" ] }, 1, 0 ] }
}
}}
], function(err, results) {
})
You do that "array joining" at the start because at some point you want the "likes" and "views" in a single array since if we tried to deal with them individually with later $unwind operations ( and you need to in order to "count" on the value of "continent" ) then you end up with a "cartesian product" since the contents of one array would be multiplied by contents present in the other.
So whist "joining" we mark with a "type" since they are no longer in separate fields, and we still need to distinquish between "likes" and "views" for counting.
The $lookup operation, is capable of working with a "flat" array in the most modern releases, but not with an "array of documents" as is constructed from the first stage. Simply $unwind to process first.
Once the $lookup is done the result in this form will be a single element array for every result contained at the "data._id" path. In order to continue processing we $unwind again.
Finally you $group, where a "compound key" is used for both the "author" and the "continent" values obtained from the joined data. In order to count, each of "likes" and "views" are applied to a $cond expression, which is a ternary ( if/then/else ) operator. Given a condition in the first argument, where that condition is true then the second argument value is returned or when false the third argument.
The results of those expressions are passed to $sum to accumulate, thus when the conditions are matching a positive count is returned and accumulates for the grouping key.
All aggregations are "heavy" operations, and performing "joins" is really considered even more "heavy".
In a large number of cases there is nothing wrong with your application performing this type of query at run time. It really comes down to if this runs at an effective speed on your data or not. If data is considerably large enough that such operations take excessive time, then you should be "pre-aggregating" by accumulating such summary data in separate records. In this case, incrementing a "like" or "view" count per author per continent or the like.