MongoDB data aggregation assistance - mongodb

Question

I have 'customers' collection with the following document:
{
id: 1,
name: 'Customer Name',
projects: [
{
id: 1000,
name: 'Project 1',
description: 'Project description',
instances: [10, 20],
},
{
id: 2000,
name: 'Project 2',
description: 'Project description',
instances: [30, 40, 10],
}
]
}
I have another collection 'instances' which looks like the following:
[
{
id: 10,
operatingSystem: 'Microsoft Windows 2012R2',
version: '3.1.5',
product: {
id: 100,
name: 'Product 1',
vendor: 'Vendor A',
},
},
{
id: 20,
operatingSystem: 'Microsoft Windows 2016',
version: '4.1.0',
product: {
id: 200,
name: 'Product 5',
vendor: 'Vendor B',
},
},
{
id: 30,
operatingSystem: 'Microsoft Windows 2019',
version: '3.0',
product: {
id: 300,
name: 'Product 2',
vendor: 'Vendor A',
},
},
{
id: 40,
operatingSystem: 'Linux',
version: '1.0',
product: {
id: 100,
name: 'Product 1',
vendor: 'Vendor A',
}
}
]
I'm trying to use the aggregation framework to make the results look like the following:
{
id: 1,
name: 'Customer Name',
projects: [
{
id: 1000,
name: 'Project 1',
description: 'Project description',
products: [
{
id: 100,
name: 'Product 1',
vendor: 'Vendor A',
instances: [
{
id: 10,
operatingSystem: 'Microsoft Windows 2012R2',
version: '3.1.5',
},
],
},
{
id: 200,
name: 'Product 5',
vendor: 'Vendor B',
instances: [
{
id: 20,
operatingSystem: 'Microsoft Windows 2016',
version: '4.1.0',
},
],
},
],
},
{
id: 2000,
name: 'Project 2',
description: 'Project description',
products: [
{
id: 300,
name: 'Product 2',
vendor: 'Vendor A',
instances: {
id: 30,
operatingSystem: 'Microsoft Windows 2019',
version: '3.0',
},
},
{
id: 100,
name: 'Product 1',
vendor: 'Vendor A',
instances: [
{
id: 40,
operatingSystem: 'Linux',
version: '1.0',
},
{
id: 10,
operatingSystem: 'Microsoft Windows 2012R2',
version: '3.1.5',
}
]
}
]
}
]
}
The current pipeline I managed to build is:
[{$match: {
_id: 1
}}, {$unwind: {
path: "$projects"
}
}, {$lookup: {
from: 'instances',
localField: 'projects.instances',
foreignField: '_id',
as: 'projects.instances'
}}, {$group: {
_id: "$projects.instances.product",
test: { "$push": "$$ROOT" }
}}, {$unwind: {
path: "$_id"
}}, {$unwind: {
path: "$test"
}}, {$project: {
_id: "$test._id",
name: "$test.name",
description: "$test.description",
projects: {
_id: "$test.projects._id",
name: "$test.projects.name",
description: "$test.projects.description",
products: {
_id: "$_id._id",
name: "$_id.name",
vendor: "$_id.vendor",
instances: "$test.projects.instances",
}
}
}}, {$group: {
_id: "$_id",
name: {"$first": "$name"},
projects: {
"$push": "$projects"
}
}}]
But I'm getting duplicates in the 'projects' array (if I'm having the same project with a different product, it will show twice instead of having a single project with 2 products in the products array
Will appreciate your help with finding the right pipeline stages to manipulate my results are expected

Answer 1

I didn't follow your pipeline as it was easier for me to just rewrite it from scratch but here's how I would do it, the obvious data structure manipulation concepts remain the same:
db.customers.aggregate([
{
$match: {
_id: 1
}
},
{
$unwind: "$projects"
},
{
$lookup: {
from: "instances",
let: {
instances: "$projects.instances"
},
pipeline: [
{
$match: {
$expr: {
$setIsSubset: [
[
"$id"
],
"$$instances"
]
}
}
}
],
as: "projects.instances"
}
},
{
$unwind: "$projects.instances"
},
{
$group: {
_id: {
id: "$_id",
project: "$projects.name",
product: "$projects.instances.product.id"
},
name: {
$first: "$name"
},
description: {
$first: "$projects.description"
},
product: {
$first: "$projects.instances.product"
},
instances: {
$push: {
id: "$projects.instances.id",
operatingSystem: "$projects.instances.operatingSystem",
version: "$projects.instances.version",
}
}
}
},
{
$group: {
_id: {
id: "$_id.id",
project: "$_id.project"
},
name: {
$first: "$name"
},
description: {
$first: "$description"
},
products: {
$push: {
$mergeObjects: [
"$product",
{
instances: "$instances"
}
]
}
}
}
},
{
$group: {
_id: "$_id.id",
name: {
$first: "$name"
},
projects: {
$push: {
name: "$project_name",
description: "$description",
products: "$products"
}
}
}
}
])
MongoPlayground