How can I perform nested "joins" (joining 3 or more collections) in a MongoDB aggregation pipeline? - mongodb

Let's say we have 3 hypothetical collections in MongoDB: customers, orders, and orderItems.
Each customer has multiple orders, and each order has multiple order items.
Here's some sample data for these 3 collections:
customers
[
{
customer_id: 1,
name: "Jim Smith",
email: "jim.smith#example.com"
},
{
customer_id: 2,
name: "Bob Jones",
email: "bob.jones#example.com"
}
]
orders
[
{
order_id: 1,
customer_id: 1
},
{
order_id: 2,
customer_id: 1
}
]
orderItems
[
{
order_item_id: 1,
name: "Foo",
price: 4.99,
order_id: 1
},
{
order_item_id: 2,
name: "Bar",
price: 17.99,
order_id: 1
},
{
order_item_id: 3,
name: "baz",
price: 24.99,
order_id: 2
}
]
Desired Result
How can I write my aggregation pipeline so that the result returned looks something like this?
[
{
customer_id: 1,
name: "Jim Smith",
email: "jim.smith#example.com"
orders: [
{
order_id: 1,
items: [
{
name: "Foo",
price: 4.99
},
{
name: "Bar",
price: 17.99
}
]
},
{
order_id: 2,
items: [
{
name: "baz",
price: 24.99
}
]
}
]
},
{
customer_id: 2,
name: "Bob Jones",
email: "bob.jones#example.com"
orders: []
}
]

Do nested lookup using lookup with pipeline,
$lookup with orders collection,
let, define variable customer_id that is from main collection, to access this reference variable inside pipeline using $$ like $$customer_id,
pipeline can add pipeline stages same as we do in root level pipeline
$expr whenever we match internal fields it requires expression match condition, so $$customer_id is parent collection field that declared in let and $customer_id is child collection's/current collection's field
$lookup with orderitems collection
db.customers.aggregate([
{
$lookup: {
from: "orders",
let: { customer_id: "$customer_id" },
pipeline: [
{ $match: { $expr: { $eq: ["$$customer_id", "$customer_id"] } } },
{
$lookup: {
from: "orderitems",
localField: "order_id",
foreignField: "order_id",
as: "items"
}
}
],
as: "orders"
}
}
])
Playground
Tip:
Several joins considered as bad practice in NoSQL, I would suggest if you could add your order items in orders collection as array, you can save one join process for orderitems, see improved version in playground

Related

MongoDB - How to access a newly created array field created with $lookup and aggregate function

Assuming I have two collections:
courses:
[
{
_id: 1,
name: "Geometry",
teacher_id: 1
},
{
_id: 2,
name: "English",
teacher_id: 2
}
]
teachers:
[
{
_id: 1,
firstName: "John",
lastName: "Adams"
},
{
_id: 2,
firstName: "Mary",
lastName: "Jane"
}
]
Now I perform an aggregation on the two collections to create something similar to a join in SQL:
db.collection("courses").aggregate([
{
$lookup:{
from: "teachers",
localField: "teacher_id",
foreignField: "_id",
as: "teacher_info"
}
},
{
$match:{
//I want to perform a match or filter here on the teacher_info
}
}
]);
The $lookup and aggregation will return a list of documents that have a new teacher_info array field.
[
{
_id: 1,
name: "Geometry",
teacher_id: 1,
teacher_info: [
{
_id: 1,
firstName: "John",
lastName: "Adams"
},
]
},
{
_id: 2,
name: "English",
teacher_id: 1,
teacher_info: [
{
_id: 2,
firstName: "Mary",
lastName: "Jane"
},
]
}
]
I need to perform a match operation in the newly created teacher_info array field. For example, only keep the teacher that has the first name "John". How can I do so? Is that possible?
You can work with dot notation in your $match stage.
{
$match: {
"teacher_info.firstName": "John"
}
}
Demo # Mongo Playground

Getting the first created document and grouping by field name in MongoDB

I would like to get the very first department created for each company, however, I'm confused with the aggregate query.
Documents:
[
{
_id: "5b7579f2deea1c6e46fd9739",
name: "Sales",
companyId: "123",
},
{
_id: "5c5779f1dffe1c6e45df3973",
name: "Security",
companyId: "123",
},
{
_id: "5d9759f5ceda1c6e64df9772",
name: "Human Resource",
companyId: "789",
},
]
I'm expecting a result like this:
Expected Result:
[
{
_id: "5b7579f2deea1c6e46fd9739",
name: "Sales",
companyId: "123",
},
{
_id: "5d9759f5ceda1c6e64df9772",
name: "Human Resource",
companyId: "789",
},
]
But I'm getting only one result with my query.
Actual Result:
[
{
_id: "5b7579f2deea1c6e46fd9739",
name: "Sales",
companyId: "123",
},
]
Aggregate Query:
db.getCollection('departments').aggregate([
{
$sort:{ item: 1 }
},
{
$group: {
_id:'$item',
companyId: { $first:'$companyId'},
name: { $first:'$name'},
}
}
])
You need to group by companyId field like this:
db.departments.aggregate([
{
$group: {
_id: "$companyId",
doc: {
$first: "$$ROOT"
}
}
},
{
$replaceRoot: {
newRoot: "$doc"
}
}
])
Playground
If you have a natural sort field like a date field, it would be good to apply sort stage on that field before the group stage.

mongodb nested array element $lookup

I have two collections, orders and products. I like to join all the order.items[] to products collection to add more fields to the items[]
Sample Data:
orders
[{ _id: 1, items: [
{ product_id: 1, price: 1.99, qty: 2 },
{ product_id: 2, price: 3.99, qty: 5 } ]}]
products
[{ _id: 1, name: "Product 1" }, { _id: 2, name: "Product 2 }]
Expected output:
[{ _id: 1, items: [
{ product_id: 1, name: "Product 1", price: 1.99, qty: 2 },
{ product_id: 2, name: "Product 2",, price: 3.99, qty: 5 } ]}]
I have tried using $lookup and pipeline (mongodb 3.6) and not getting the name value or even the match is not working.
Thanks for a help!
This query will help you, sorry if I didn't use v3.6.
db.orders.aggregate([
{
$unwind: "$items"
},
{
$lookup:
{
from: "products",
localField: "items.product_id",
foreignField: "_id",
as: "tproduct"
}
},
{
$project:
{
"_id" : 1,
"items.product_id" : 1,
"items.name" : { $arrayElemAt: ["$tproduct.name", 0] },
"items.price" : 1,
"items.qty" : 1
}
},
{
$group :
{
_id : "$_id",
items: { $push: "$items" }
}
}
])
They are 4 stages that I will explain:
$unwind will create a single object for each element in the array.
$lookup will find the correct product, keep in mind that Product._id should be unique.
$project will format my documents and in items.name I'm taking the first element of the lookup sentence.
$group will use the _id to group and push each item into a new array.
I'm pretty sure there are cleaner and easier ways to write this, but this should work without problems.

MongoDB agregation and comparing

I need to write mongodb query that will agregate items by quantinty (orders collestion). After agregation, or with agregation, it should identify items that can't be supplied in stock collection, and return name of the person who made this order
code of orders insert
db.orders.insertMany([
{client: {firstName: "John",lastName: "Smith" },orderedItems: [{ name: "itemA", qty: 5},{ name: "itemS", qty: 3}]},
{client: {firstName: "Jan",lastName: "Nowak" },orderedItems: [{ name: "itemA", qty: 56},{ name: "itemS", qty: 53}]},
{client: {firstName: "Klara",lastName: "BolÄ…czka" },orderedItems: [{ name: "itemA", qty: 35},{ name: "itemS", qty: 23}]},
{client: {firstName: "Brajan",lastName: "Kowalski" },orderedItems: [{ name: "itemA", qty: 95},{ name: "itemS", qty: 13}]}
]);
code of stocks insert
db.stock.insertMany([
{ item: "itemA", qty: 40},
{ item: "itemS", qty: 113}
]);
Right now i got some mongoDB code, that just agregates orders
db.orders.aggregate([
{$unwind: "$orderedItems"},
{$group: { _id: "$orderedItems.name", total:{$sum: "$orderedItems.qty"}}}
]);
I tried to use lookup, but probably I'm doing something wrong.
Could I get some sugestions, please?
How would you like something like this:
db.orders.aggregate([
{
$unwind: "$orderedItems" // flatten the "orderedItems" array
},
{
$group: {
"_id": "$orderedItems.name", // group by orderItems.name
"total": {
$sum: "$orderedItems.qty" // calculate total quantity
},
"users": {
$addToSet: "$client" // create an array of clients who have ordered each item
}
}
}, {
$lookup: { // retrieve the stock information
from: "stock",
localField: "_id",
foreignField: "item",
as: "stock"
}
}, {
$unwind: "$stock" // flatten the result array - should always return only one document, given your data model
}, {
$addFields: { // add a new field...
"difference": { // ...called "difference"...
$subtract: [ "$stock.qty", "$total" ] // ...in which we store the difference between the stock qty and the ordered qty
}
}
}, {
$match: { // only return documents...
"difference": { // ...where the value for "difference"...
$lt: 0 // ...is less than zero
}
}
}])

Aggregation filter and lookup on Mongodb

My first collection employeecategory is like below;
[{
name: "GARDENING"
},
{
name: "SECURITY"
},
{
name: "PLUMBER"
}
]
My second collection complaints is like below;
[{
communityId: 1001,
category: "SECURITY",
//other fields
}, {
communityId: 1001,
category: "GARDENING",
//other fields
}]
I am trying to join above tables and get the below result;
[{
"count": 1,
"name": "GARDENING"
}, {
"count": 1,
"name": "SECURITY"
}, {
"count": 0,
"name": "PLUMBER"
}]
Even if there are no records in collection 2 I need count. I tried below aggregation but didn't worked. If I removed match condition it is working but I need to filter on community id. Could some please suggest best way to do achieve this. Mongo DB version is 3.4.0
db.employeecategory.aggregate(
[{
$match: {
"complaints.communityId": 1001
}
}, {
"$lookup": {
from: "complaints",
localField: "name",
foreignField: "category",
as: "embeddedData"
}
}]
)
It was not possible to achieve both filtering for communityId = 1001 and grouping without losing count = 0 category in a single aggregation. The way to do it is first start from complaints collection, and filter the communityId = 1001 objects, and create a temp collection with it. Then from employeecategory collection, $lookup to join with that temp collection, and $group with name, you will have your result at this point, then drop the temp table.
// will not modify complaints document, will create a filtered temp document
db.complaints.aggregate(
[{
$match: {
communityId: 1001
}
},
{
$out: "temp"
}
]
);
// will return the answer that is requested by OP
db.employeecategory.aggregate(
[{
$lookup: {
from: "temp",
localField: "name",
foreignField: "category",
as: "array"
}
}, {
$group: {
_id: "$name",
count: {
$sum: {
$size: "$array"
}
}
}
}]
).pretty();
db.temp.drop(); // to get rid of this temporary collection
will result;
{ _id: "PLUMBER", count: 0},
{ _id: "SECURITY", count: 2},
{ _id: "GARDENING", count: 1}
for the test data I've had;
db.employeecategory.insertMany([
{ name: "GARDENING" },
{ name: "SECURITY" },
{ name: "PLUMBER" }
]);
db.complaints.insertMany([
{ category: "GARDENING", communityId: 1001 },
{ category: "SECURITY", communityId: 1001 },
{ category: "SECURITY", communityId: 1001 },
{ category: "SECURITY", communityId: 1002 }
]);