I am sure it was answered before but I can't find any proper explanation, maybe the aggregate methods I assumed are not relevant here.
I have a mongo collection containing an array which points to sub-docs
{
_id: '123',
name: 'my shop',
items: [
{
itemId: '234',
},
{
itemId: '345',
},
]
}
This is the collection of sub docs:
{
_id: '234',
name: 'apple',
amount: 13
},
{
_id: '345',
name: 'orange',
amount: 25
},
How can I replace in the upper document the id-reference to the actual document content.
Desired final result:
{
_id: '123',
name: 'my shop',
items: [
{
itemId: '234',
name: 'apple',
amount: 13
},
{
itemId: '345',
name: 'orange',
amount: 25
},
]
}
Let's say sample data for shop collection is:
{
_id: '123',
name: 'my shop',
items: [
{
itemId: '234',
},
{
itemId: '345',
},
]
}
And the sample data for item collection is:
{
_id: '234',
name: 'apple',
amount: 13
},
{
_id: '345',
name: 'orange',
amount: 25
}
Then the aggregation will be like this one:
db.shop.aggregate([{
$lookup: {
from: 'item',
localField: 'items.itemId',
foreignField: '_id',
as: 'items'
}
}])
Related
I need to create an aggregation using information from multiple collections and arrange them in a specific result format. Here we have three collections (parent, task, slots; the model of these cannot be changed anymore). Each parent can have many tasks, which consist of different slots (see the simplified sample data below)
Slot collection
[
{
_id: ObjectId('6015720974cff84db3499cad'),
name: 'Test 1',
start: ISO-Date('2021-07-13T09:00:00.000+00:00'),
end: ISO-Date('2021-07-13T09:15:00.000+00:00'),
parentId: 1,
taskId: 1
},
{
_id: ObjectId('6015720974cff84db3478cad'),
name: 'Test 2',
start: ISO-Date('2021-07-13T09:15:00.000+00:00'),
end: ISO-Date('2021-07-13T09::30.000+00:00'),
parentId: 1,
taskId: 1
},
{
_id: ObjectId('6015720974cff84db3499bcd'),
name: 'Test 3',
start: ISO-Date('2021-07-13T11:00:00.000+00:00'),
end: ISO-Date('2021-07-13T11:15:00.000+00:00'),
parentId: 1,
taskId: 2
},
{
_id: ObjectId('6015720974cff84db3499efg'),
name: 'Test 4',
start: ISO-Date('2021-07-13T13:00:00.000+00:00'),
end: ISO-Date('2021-07-13T13:15:00.000+00:00'),
parentId: 2,
taskId: 3
},
{
_id: ObjectId('6015720974cff84db349967e'),
name: 'Test 5',
start: ISO-Date('2021-07-13T13:15:00.000+00:00'),
end: ISO-Date('2021-07-13T13:30:00.000+00:00'),
parentId: 2,
taskId: 3
},
]
Task collection
[
{
_id: ObjectId('6015720974cff84db3499c87'),
name: 'Task 1',
taskId: 1,
},
{
_id: ObjectId('6015720974cff84db3499b6b'),
name: 'Task 2',
taskId: 2,
},
{
_id: ObjectId('6015720974cff84db3499b19'),
name: 'Task 3',
taskId: 3,
}
]
Parent collection
[
{
_id: ObjectId('6015720974cff84db34995a6'),
name: 'Parent 1',
parentId: 1
},
{
_id: ObjectId('6015720974cff84db349962f'),
name: 'Parent 2',
parentId: 2
}
]
The result should have the following format:
[
{
parentName: 'Parent 1',
tasks: [
{
taskName: 'Task 1',
slots: [
{
name: 'Test 1',
start: ISO-Date('2021-07-13T09:00:00.000+00:00'),
end: ISO-Date('2021-07-13T09:15:00.000+00:00'),
},
{
name: 'Test 2',
start: ISO-Date('2021-07-13T09:15:00.000+00:00'),
end: ISO-Date('2021-07-13T09::30.000+00:00'),
},
]
},
{
taskName: 'Task 2',
slots: [
{
name: 'Test 3',
start: ISO-Date('2021-07-13T11:00:00.000+00:00'),
end: ISO-Date('2021-07-13T11:15:00.000+00:00'),
}
]
}
]
},
{
parentName: 'Parent 2',
tasks: [
{
taskName: 'Task 3',
slots: [
{
name: 'Test 4',
start: ISO-Date('2021-07-13T13:00:00.000+00:00'),
end: ISO-Date('2021-07-13T13:15:00.000+00:00'),
},
{
name: 'Test 5',
start: ISO-Date('2021-07-13T13:15:00.000+00:00'),
end: ISO-Date('2021-07-13T13:30:00.000+00:00'),
}
]
}
]
}
]
How can I achieve this result structure by given data? The lookup part is not a big deal, but I currently struggle to achieve this structure, as I'm very new to MongoDB aggregation framework. Thank you very much for any hints.
You can use lookup to join collections
$lookup to join collections
$unwind to deconstruct the array
$group to reconstruct the array
Here is the code,
db.Task.aggregate([
{
"$lookup": {
"from": "Slot",
"localField": "taskId",
"foreignField": "taskId",
"as": "slots"
}
},
{ "$unwind": "$slots" },
{
"$lookup": {
"from": "Parent",
"localField": "slots.parentId",
"foreignField": "parentId",
"as": "parents"
}
},
{ "$unwind": "$parents" },
{
$group: {
_id: { pId: "$parents._id", tId: "$taskId" },
parentName: { $first: "$parents.name" },
taskName: { "$first": "$name" },
slots: { $push: "$slots" }
}
},
{
$group: {
_id: "$_id.pId",
parentName: { $first: "$parentName" },
tasks: {
$push: {
taskName: "$taskName",
slots: "$slots"
}
}
}
}
])
Working Mongo playground
My documents look like this:
{
{
mlsId: 'RTC749',
firstName: 'Tommy',
lastName: 'Davidson',
officeMlsId: 'RTC2421',
officeName: 'John Jones Real Estate LLC',
slug: 'tommy-davidson',
serviceAreas: [
{
name: 'Nashville',
slug: 'nashville',
type: 'city',
totalClosedSales: 3
},
{
name: 'Franklin',
slug: 'franklin',
type: 'city',
totalClosedSales: 7
}
},
{
id: 'RTC7280',
firstName: 'Jack',
lastName: 'Miller',
slug: 'jack-miller',
serviceAreas: [
{
name: 'Nashville',
slug: 'nashville',
type: 'city',
totalClosedSales: 4
},
{
name: 'Franklin',
slug: 'franklin',
type: 'city',
totalClosedSales: 10
}
]
},
}
The query to find documents based on slugs within the subdocuments looks like this:
const localAgents = await Agent.find(
{
'serviceAreas.slug': locationSlug,
},
'-_id -__v'
)
.sort({ 'serviceAreas.totalClosedSales': -1 })
Note that I'd like to find agents by location slug and sort the result using totalClosedSales however I'm unable to get it to work. So the desired result would look like this:
{
{
id: 'RTC7280',
firstName: 'Jack',
lastName: 'Miller',
slug: 'jack-miller',
serviceAreas: [
{
name: 'Franklin',
slug: 'franklin',
type: 'city',
totalClosedSales: 10
},
{
name: 'Nashville',
slug: 'nashville',
type: 'city',
totalClosedSales: 4
}
]
},
{
mlsId: 'RTC749',
firstName: 'Tommy',
lastName: 'Davidson',
officeMlsId: 'RTC2421',
officeName: 'John Jones Real Estate LLC',
slug: 'tommy-davidson',
serviceAreas: [
{
name: 'Nashville',
slug: 'nashville',
type: 'city',
totalClosedSales: 3
},
{
name: 'Franklin',
slug: 'franklin',
type: 'city',
totalClosedSales: 7
}
]
},
}
We can't sort array directly, But aggregation helps it
$unwind helps to de-structure the array
$sort helps to sort as you wish
$group helps to re-group the de-structured array
Mongo script is given below
db.collection.aggregate([
{
"$match": {
"serviceAreas.slug": "nashville"
}
},
{
$unwind: "$serviceAreas"
},
{
$sort: {
"serviceAreas.totalClosedSales": -1
}
},
{
$addFields: {
total: "$serviceAreas.totalClosedSales"
}
},
{
$sort: {
total: -1
}
},
{
$group: {
_id: "$_id",
mlsId: {
$first: "$mlsId"
},
firstName: {
$first: "$firstName"
},
lastName: {
$first: "$lastName"
},
slug: {
$first: "$slug"
},
serviceAreas: {
$push: "$serviceAreas"
}
}
}
])
Working Mongo playground
I have 'customers' collection with the following document:
{
id: 1,
name: 'Customer Name',
projects: [
{
id: 1000,
name: 'Project 1',
description: 'Project description',
instances: [10, 20],
},
{
id: 2000,
name: 'Project 2',
description: 'Project description',
instances: [30, 40, 10],
}
]
}
I have another collection 'instances' which looks like the following:
[
{
id: 10,
operatingSystem: 'Microsoft Windows 2012R2',
version: '3.1.5',
product: {
id: 100,
name: 'Product 1',
vendor: 'Vendor A',
},
},
{
id: 20,
operatingSystem: 'Microsoft Windows 2016',
version: '4.1.0',
product: {
id: 200,
name: 'Product 5',
vendor: 'Vendor B',
},
},
{
id: 30,
operatingSystem: 'Microsoft Windows 2019',
version: '3.0',
product: {
id: 300,
name: 'Product 2',
vendor: 'Vendor A',
},
},
{
id: 40,
operatingSystem: 'Linux',
version: '1.0',
product: {
id: 100,
name: 'Product 1',
vendor: 'Vendor A',
}
}
]
I'm trying to use the aggregation framework to make the results look like the following:
{
id: 1,
name: 'Customer Name',
projects: [
{
id: 1000,
name: 'Project 1',
description: 'Project description',
products: [
{
id: 100,
name: 'Product 1',
vendor: 'Vendor A',
instances: [
{
id: 10,
operatingSystem: 'Microsoft Windows 2012R2',
version: '3.1.5',
},
],
},
{
id: 200,
name: 'Product 5',
vendor: 'Vendor B',
instances: [
{
id: 20,
operatingSystem: 'Microsoft Windows 2016',
version: '4.1.0',
},
],
},
],
},
{
id: 2000,
name: 'Project 2',
description: 'Project description',
products: [
{
id: 300,
name: 'Product 2',
vendor: 'Vendor A',
instances: {
id: 30,
operatingSystem: 'Microsoft Windows 2019',
version: '3.0',
},
},
{
id: 100,
name: 'Product 1',
vendor: 'Vendor A',
instances: [
{
id: 40,
operatingSystem: 'Linux',
version: '1.0',
},
{
id: 10,
operatingSystem: 'Microsoft Windows 2012R2',
version: '3.1.5',
}
]
}
]
}
]
}
The current pipeline I managed to build is:
[{$match: {
_id: 1
}}, {$unwind: {
path: "$projects"
}
}, {$lookup: {
from: 'instances',
localField: 'projects.instances',
foreignField: '_id',
as: 'projects.instances'
}}, {$group: {
_id: "$projects.instances.product",
test: { "$push": "$$ROOT" }
}}, {$unwind: {
path: "$_id"
}}, {$unwind: {
path: "$test"
}}, {$project: {
_id: "$test._id",
name: "$test.name",
description: "$test.description",
projects: {
_id: "$test.projects._id",
name: "$test.projects.name",
description: "$test.projects.description",
products: {
_id: "$_id._id",
name: "$_id.name",
vendor: "$_id.vendor",
instances: "$test.projects.instances",
}
}
}}, {$group: {
_id: "$_id",
name: {"$first": "$name"},
projects: {
"$push": "$projects"
}
}}]
But I'm getting duplicates in the 'projects' array (if I'm having the same project with a different product, it will show twice instead of having a single project with 2 products in the products array
Will appreciate your help with finding the right pipeline stages to manipulate my results are expected
I didn't follow your pipeline as it was easier for me to just rewrite it from scratch but here's how I would do it, the obvious data structure manipulation concepts remain the same:
db.customers.aggregate([
{
$match: {
_id: 1
}
},
{
$unwind: "$projects"
},
{
$lookup: {
from: "instances",
let: {
instances: "$projects.instances"
},
pipeline: [
{
$match: {
$expr: {
$setIsSubset: [
[
"$id"
],
"$$instances"
]
}
}
}
],
as: "projects.instances"
}
},
{
$unwind: "$projects.instances"
},
{
$group: {
_id: {
id: "$_id",
project: "$projects.name",
product: "$projects.instances.product.id"
},
name: {
$first: "$name"
},
description: {
$first: "$projects.description"
},
product: {
$first: "$projects.instances.product"
},
instances: {
$push: {
id: "$projects.instances.id",
operatingSystem: "$projects.instances.operatingSystem",
version: "$projects.instances.version",
}
}
}
},
{
$group: {
_id: {
id: "$_id.id",
project: "$_id.project"
},
name: {
$first: "$name"
},
description: {
$first: "$description"
},
products: {
$push: {
$mergeObjects: [
"$product",
{
instances: "$instances"
}
]
}
}
}
},
{
$group: {
_id: "$_id.id",
name: {
$first: "$name"
},
projects: {
$push: {
name: "$project_name",
description: "$description",
products: "$products"
}
}
}
}
])
MongoPlayground
I need to aggregate "lastNames" and "occupations" for a "name" to get a result:
{
name: 'John',
occupations: ['software engineer', 'qa']
lastNames: ['Smith', 'Red', 'Doe']
}
input
name: 'John'
documents present in mongo:
{name: 'John', lastName: 'Smith', occupation: 'software engineer'}
{name: 'Steve', lastName: 'Smith', occupation: 'senior software engineer'}
{name: 'John', lastName: 'Doe', occupation: 'qa'}
{name: 'Steve', lastName: 'Doe', occupation: 'manager'}
{name: 'John', lastName: 'Red', occupation: 'software engineer'}
I started with this aggregation query:
Employees.aggregate([
{ $match: { name: name } },
{
$unwind: {
path: '$lastName',
},
},
{
$unwind: {
path: '$occupation',
},
},
{ $group: { _id: '$name' } },
]);
but this returns an empty array, so I kinda stuck as I never did aggregations before.
Is there a way to produce this required result?
Would be this one:
db.collection.aggregate([
{ $match: { name: "John" } },
{
$group: {
_id: "$name",
occupations: { $addToSet: "$occupation" },
lastNames: { $addToSet: "$lastName" },
}
},
{
$project: {
_id: 0,
name: "$_id",
occupations: 1,
lastNames: 2
}
}
])
Mongo playground
Let's say I have these two collections
book: {
_id: 'aaa'
name: 'Book 1',
chapters: [
0: {
_id: 'chapter0',
name: 'Chapter 1',
pages: [
0: {
_id: 'page0',
name: 'Page 1',
paragraphs: [
0: {
_id: 'paragraph0',
name: 'Paragraph 1',
bookmarks: [
0: {sentence: 3, reader: 'Foo'},
1: {sentence: 8, reader: 'Bar'},
2: {sentence: 14, reader: 'John'}
]
}
]
}
]
}
]
}
book: {
_id: 'bbb'
name: 'Book 2',
chapters: [
0: {
_id: 'chapter0',
name: 'Chapter 1',
pages: [
0: {
_id: 'page0',
name: 'Page 1',
paragraphs: [
0: {
_id: 'paragraph0',
name: 'Paragraph 1',
bookmarks: []
},
1: {
_id: 'paragraph1',
name: 'Paragraph 2',
bookmarks: [
0: {sentence: 2, reader: 'George'},
1: {sentence: 1, reader: 'Paul'},
2: {sentence: 76, reader: 'John'},
3: {sentence: 54, reader: 'Ringo'}
]
}
]
}
]
}
]
}
I want to be able to extract the array bookmarks and attach them to the book collection when getting the result. Something like this would be good:
{
id: 'aaa'
name: 'Book 1'
bookmarks: [{...}, {...}, {...}] //since the first book has 3 bookmarks
},
{
id: 'bbb'
name: 'Book 2'
bookmarks: [{...}, {...}, {...}, {...}] //since the second book has 4 bookmarks
},
And if there are no bookmarks, it should look like:
{
id: 'aaa'
name: 'Book 1'
bookmarks: [{...}, {...}, {...}] //since the first book has 3 bookmarks
},
{
id: 'bbb'
name: 'Book 2'
bookmarks: [{...}, {...}, {...}, {...}] //since the second book has 4 bookmarks
},
{
id: 'ccc'
name: 'Book 3'
bookmarks: [] //third book does not have bookmarks for example
},
I've tried aggregation with this code, but it just separates each bookmark per book and pushes it into the object.
return yield Books.aggregate()
.unwind('chapters')
.unwind('chapters.pages')
.unwind('chapters.pages.paragraphs')
.unwind('chapters.pages.paragraphs.bookmarks')
.group({
_id: '$_id',
books: {
$push: {
_id: '$_id',
name: '$name',
bookmarks: '$chapters.pages.paragraphs.bookmarks'
}
}
}).exec()
Can someone point me to the right direction? Thanks!
Try below aggregate pipeline:
Books.aggregate([
{
$unwind: "$book"
},
{
$unwind: "$book.chapters"
},
{
$unwind: "$book.chapters.pages"
},
{
$unwind: "$book.chapters.pages.paragraphs"
},
{
$unwind: {
path: "$book.chapters.pages.paragraphs.bookmarks",
preserveNullAndEmptyArrays: true
}
},
{
$group: {
_id: {
_id: "$_id",
book: "$book.name"
},
bookmarks: {
$push: "$book.chapters.pages.paragraphs.bookmarks"
}
}
}
])