Multiple $lookup and sort nested arrays in MongoDB - mongodb

Question

I am learning mongodb, i searched for some tips at whole internet, but i still cant cant get the proper result.
The only thing i have to do is join 2 collections.
Let me introduce the problem.
COLLECTIONS
Artists
{
_id: 1,
Name: 'Artists one'
}
Albums
{
_id: 1,
title: "Album 01",
year
artists_id: 1
}
{
_id: 2,
title: "Album 02",
year: 2020,
artists_id: 1
}
Tracks
{
albums_id: 1,
track_number: 1,
title: 'Track 01',
time: 123
}
{
albums_id: 1,
track_number: 2,
title: 'Track 02',
time: 123
}
{
albums_id: 2,
track_number: 1,
title: 'Track 01',
time: 123
}
{
albums_id: 2,
track_number: 2,
title: 'Track 02',
time: 123
}
WHAT I WANT TO ACHIVE ?
a query should return result like below.
Albums should be sorted by year ascending.
Tracks should be sorted by track_number ascending (or descending whatever i wish)
{
Name: 'Artists one',
Albums: [
{
title: "Album 01",
tracks: [
{
title: 'Track 01'
},
{
title: 'Track 02'
}
]
},
{
title: "Album 02",
tracks: [
{
title: 'Track 01'
},
{
title: 'Track 02'
}
]
}
]
}
WHAT I END UP WITH ?
I can successfully print all data with sorted albums, but i don't know how to unwind tracks to sort them by track_number and group it again like in code up
db.artists.aggregate([
{
$lookup:
{
from: "albums",
localField: "_id",
foreignField: "artists_id",
as: "albums"
}
},
{
$unwind: "$albums"
},
{
$lookup:
{
from: "tracks",
localField: "albums._id",
foreignField: "albums_id",
as: "albums.tracks"
}
},
{
$sort:
{
"albums.year": 1
}
},
{
$group:
{
_id : "$_id",
"Name" : { $first: "$Name" },
albums: { $push: "$albums" }
}
},
{
$project:
{
"_id":0,
"Name":1,
"albums": {"title":1, "tracks": {"title":1}}
}
}
]).pretty()
WHAT I NEED
I know it can't be hard, i just still try to understund the aggregation framework. I will be really greatfull if someone can show me how to make this work - also if u can additionaly explain how to achive result consistent with the assumptions i mentioned before but with look:
{
Name: 'Artists one',
Albums: [
{
title: "Album 01",
tracks: ['Track 01' 'Track 02']
},
{
title: "Album 02",
tracks: ['Track 01' 'Track 02']
}
]
}
The code just would help me very much in understanding aggregation framework.

Answer 1

Because of sorting the query is a bit complex
db.artists.aggregate([
{
$lookup: {
from: "albums",
localField: "_id",
foreignField: "artists_id",
as: "albums"
}
},
{
$unwind: "$albums"
},
{
$lookup: {
from: "tracks",
localField: "albums._id",
foreignField: "albums_id",
as: "albums.tracks"
}
},
{
$unwind: "$albums.tracks"
},
{
$sort: {
"albums.tracks.track_number": 1
}
},
{
$group: {
_id: {
_id: "$_id",
Name: "$Name",
albumId: "$albums._id",
albumTitle: "$albums.title",
albumYear: "$albums.year"
},
albumsTracks: {
$push: "$albums.tracks"
}
}
},
{
$project: {
_id: "$_id._id",
Name: "$_id.Name",
albumId: "$_id.albumId",
albumTitle: "$_id.albumTitle",
albumYear: "$_id.albumYear",
tracks: "$albumsTracks"
}
},
{
$sort: {
albumYear: 1
}
},
{
$group: {
_id: {
_id: "$_id",
Name: "$Name"
},
Albums: {
$push: {
title: "$albumTitle",
tracks: "$tracks"
}
}
}
},
{
$project: {
_id: "$_id._id",
Name: "$_id.Name",
"Albums.tracks.title": 1
}
}
]).pretty()
In general, if you see such a overhead, it's a signal to think different structure to store your data. For example you may want to combine all data in one collection if you're sure single record won't exceed 16 mb at some point of time.

Answer 2

Since MongoDB 3.6, you can use conditional $lookup. For each albums, you fetch tracks.
db.artists.aggregate([
{
$lookup: {
from: "albums",
let: {
"artists_id": "$_id"
},
pipeline: [
{
$match: {
$expr: {
$eq: [
"$artists_id",
"$$artists_id"
]
}
}
},
{
$sort: {
year: 1
}
},
{
$lookup: {
from: "tracks",
let: {
"albums_id": "$_id"
},
pipeline: [
{
$match: {
$expr: {
$eq: [
"$albums_id",
"$$albums_id"
]
}
}
},
{
$sort: {
track_number: 1
}
}
],
as: "tracks"
}
},
{
$project: {
_id: 0,
title: 1,
tracks: {
$map: {
input: "$tracks",
in: "$$this.title"
}
}
}
}
],
as: "Albums"
}
},
{
$unset: "_id"
}
])
MongoPlayground