mongodb nested array element $lookup - mongodb

I have two collections, orders and products. I like to join all the order.items[] to products collection to add more fields to the items[]
Sample Data:
orders
[{ _id: 1, items: [
{ product_id: 1, price: 1.99, qty: 2 },
{ product_id: 2, price: 3.99, qty: 5 } ]}]
products
[{ _id: 1, name: "Product 1" }, { _id: 2, name: "Product 2 }]
Expected output:
[{ _id: 1, items: [
{ product_id: 1, name: "Product 1", price: 1.99, qty: 2 },
{ product_id: 2, name: "Product 2",, price: 3.99, qty: 5 } ]}]
I have tried using $lookup and pipeline (mongodb 3.6) and not getting the name value or even the match is not working.
Thanks for a help!

This query will help you, sorry if I didn't use v3.6.
db.orders.aggregate([
{
$unwind: "$items"
},
{
$lookup:
{
from: "products",
localField: "items.product_id",
foreignField: "_id",
as: "tproduct"
}
},
{
$project:
{
"_id" : 1,
"items.product_id" : 1,
"items.name" : { $arrayElemAt: ["$tproduct.name", 0] },
"items.price" : 1,
"items.qty" : 1
}
},
{
$group :
{
_id : "$_id",
items: { $push: "$items" }
}
}
])
They are 4 stages that I will explain:
$unwind will create a single object for each element in the array.
$lookup will find the correct product, keep in mind that Product._id should be unique.
$project will format my documents and in items.name I'm taking the first element of the lookup sentence.
$group will use the _id to group and push each item into a new array.
I'm pretty sure there are cleaner and easier ways to write this, but this should work without problems.

Related

MongoDB - How to access a newly created array field created with $lookup and aggregate function

Assuming I have two collections:
courses:
[
{
_id: 1,
name: "Geometry",
teacher_id: 1
},
{
_id: 2,
name: "English",
teacher_id: 2
}
]
teachers:
[
{
_id: 1,
firstName: "John",
lastName: "Adams"
},
{
_id: 2,
firstName: "Mary",
lastName: "Jane"
}
]
Now I perform an aggregation on the two collections to create something similar to a join in SQL:
db.collection("courses").aggregate([
{
$lookup:{
from: "teachers",
localField: "teacher_id",
foreignField: "_id",
as: "teacher_info"
}
},
{
$match:{
//I want to perform a match or filter here on the teacher_info
}
}
]);
The $lookup and aggregation will return a list of documents that have a new teacher_info array field.
[
{
_id: 1,
name: "Geometry",
teacher_id: 1,
teacher_info: [
{
_id: 1,
firstName: "John",
lastName: "Adams"
},
]
},
{
_id: 2,
name: "English",
teacher_id: 1,
teacher_info: [
{
_id: 2,
firstName: "Mary",
lastName: "Jane"
},
]
}
]
I need to perform a match operation in the newly created teacher_info array field. For example, only keep the teacher that has the first name "John". How can I do so? Is that possible?
You can work with dot notation in your $match stage.
{
$match: {
"teacher_info.firstName": "John"
}
}
Demo # Mongo Playground

How can I perform nested "joins" (joining 3 or more collections) in a MongoDB aggregation pipeline?

Let's say we have 3 hypothetical collections in MongoDB: customers, orders, and orderItems.
Each customer has multiple orders, and each order has multiple order items.
Here's some sample data for these 3 collections:
customers
[
{
customer_id: 1,
name: "Jim Smith",
email: "jim.smith#example.com"
},
{
customer_id: 2,
name: "Bob Jones",
email: "bob.jones#example.com"
}
]
orders
[
{
order_id: 1,
customer_id: 1
},
{
order_id: 2,
customer_id: 1
}
]
orderItems
[
{
order_item_id: 1,
name: "Foo",
price: 4.99,
order_id: 1
},
{
order_item_id: 2,
name: "Bar",
price: 17.99,
order_id: 1
},
{
order_item_id: 3,
name: "baz",
price: 24.99,
order_id: 2
}
]
Desired Result
How can I write my aggregation pipeline so that the result returned looks something like this?
[
{
customer_id: 1,
name: "Jim Smith",
email: "jim.smith#example.com"
orders: [
{
order_id: 1,
items: [
{
name: "Foo",
price: 4.99
},
{
name: "Bar",
price: 17.99
}
]
},
{
order_id: 2,
items: [
{
name: "baz",
price: 24.99
}
]
}
]
},
{
customer_id: 2,
name: "Bob Jones",
email: "bob.jones#example.com"
orders: []
}
]
Do nested lookup using lookup with pipeline,
$lookup with orders collection,
let, define variable customer_id that is from main collection, to access this reference variable inside pipeline using $$ like $$customer_id,
pipeline can add pipeline stages same as we do in root level pipeline
$expr whenever we match internal fields it requires expression match condition, so $$customer_id is parent collection field that declared in let and $customer_id is child collection's/current collection's field
$lookup with orderitems collection
db.customers.aggregate([
{
$lookup: {
from: "orders",
let: { customer_id: "$customer_id" },
pipeline: [
{ $match: { $expr: { $eq: ["$$customer_id", "$customer_id"] } } },
{
$lookup: {
from: "orderitems",
localField: "order_id",
foreignField: "order_id",
as: "items"
}
}
],
as: "orders"
}
}
])
Playground
Tip:
Several joins considered as bad practice in NoSQL, I would suggest if you could add your order items in orders collection as array, you can save one join process for orderitems, see improved version in playground

How to make a lookup of the "Author" field of various arrays in mongodb?

My problem is that I want to do a Lookup of the field "Author" for the array of objects "Reviews", "Followers" and "Watching" but I don't know why it gives me this result in the others arrays, that value repeats the same number of times of the documents in the "Reviews" array.
.unwind({ path: '$reviews', preserveNullAndEmptyArrays: true })
.lookup({
from: 'users',
let: { userId: '$reviews.author' },
pipeline: [
{ $match: { $expr: { $eq: ['$_id', '$$userId'] } } },
{
$project: {
name: 1,
username: 1,
photo: 1,
rank: 1,
'premium.status': 1,
online: 1,
},
},
],
as: 'reviews.author',
})
.unwind({ path: '$followers', preserveNullAndEmptyArrays: true })
.lookup({
from: 'users',
let: { userId: '$followers.author' },
pipeline: [
{ $match: { $expr: { $eq: ['$_id', '$$userId'] } } },
{
$project: {
name: 1,
username: 1,
photo: 1,
rank: 1,
'premium.status': 1,
online: 1,
},
},
],
as: 'followers.author',
})
.unwind({ path: '$watching', preserveNullAndEmptyArrays: true })
.lookup({
from: 'users',
let: { userId: '$watching.author' },
pipeline: [
{ $match: { $expr: { $eq: ['$_id', '$$userId'] } } },
{
$project: {
name: 1,
username: 1,
photo: 1,
rank: 1,
'premium.status': 1,
online: 1,
},
},
],
as: 'watching.author',
})
.group({
_id: '$_id',
data: {
$first: '$$ROOT',
},
reviews: {
$push: '$reviews',
},
followers: {
$push: '$followers',
},
watching: {
$push: '$watching',
},
})
This is the result when "Reviews" has documents:
The "Followers / Watching" array has nothing in the database but it is shown here in this way, repeating that value the same number of documents that are in reviews, I don't know what happens.
And when all arrays are empty, this happens:
It keeps showing that, but I don't know how to repair it.
In summary, "Reviews", "Watching", and "Followers" have an "Author" field, and I want to do a lookup to the author field of watching, and also for followers and reviews but I have these problems. Please I need help.
Example: This is how it looks in the database:
Thank you very much in advance.
The $unwind stage creates a new document for every element in the array you are unwinding. Each new document will contain a copy of every other field in the document.
If the original document looks like
{
_id: "unique",
Array1:["A","B","C"],
Array2:["D","E","F"],
}
After unwinding "Array1", there will be 3 documents in the pipeline:
[
{
_id: "unique",
Array1:"A",
Array2:["D","E","F"]
},{
_id: "unique",
Array1:"B",
Array2:["D","E","F"]
},{
_id: "unique",
Array1:"C",
Array2:["D","E","F"]
}
]
Then unwinding "watchers" will expand each of the watchers arrays so that you have the cartesian product of the arrays. Playground
In your case, the original document has 2 reviews, but no followers and no watchers, so at the start of the pipeline contains one document, similar to:
[
{
_id: "ObjectId",
data: "other data"
reviews: [{content:"review1", author:"ObjectId"},
{content:"review2", author:"ObjectId"}]
}
]
After the first unwind, you have 2 documents:
[
{
_id: "ObjectId",
data: "other data"
reviews: {content:"review1", author:"ObjectId"}
},
{
_id: "ObjectId",
data: "other data"
reviews: {content:"review2", author:"ObjectId"}
}
]
The first lookup replaces the author ID with data for each document, then the second unwind is applied to each document.
Since that array is empty, the lookup returns an empty array, and the third unwind is applied.
Just before the $group stage, the pipeline contains 2 documents with the arrays:
[
{
_id: "ObjectId",
data: "other data"
reviews: {content:"review1", author:"ObjectId"},
followers: {author: []},
watchers: {author: []}
},
{
_id: "ObjectId",
data: "other data"
reviews: {content:"review2", author:"ObjectId"},
followers: {author:[]},
watchers: {author: []}
}
]
Since both documents have the same _id they are grouped together, with the final result containing the first document as "data".
For the arrays, as each document is encountered, the value of the corresponding field is pushed onto the array, resulting in each array having 2 values.

Mongo Aggregation Grouping and Mapping of Object Array

I'm need to group a mongo collection and aggregate.
Example Collection
[{id:"1", skill: "cooking"},{id:"1", skill: "fishing"}]
Lookup Collection
[{ name: "cooking", value: 3 }, { name: "fishing", value: 2 }]
Desired Result
[{id: "1", skills: [{ value: 3, "cooking" }, { value: 2, "fishing"}]}]
Here's how far I am.
db.talent.aggregate([
{
$group: '$id'
skills: { $addToSet: '$skill' }
},
])
Result:
[{id: "1", skills: ["cooking", "fishing"]}]
I'm wondering if this is even possible.
I miss SQL, need help!
We can do this using $lookup, $group and $project in the aggregation pipeline
Shown below is the mongodb shell query
db.example_collection.aggregate([
{
$lookup: {
from: "lookup_collection",
localField: "skill",
foreignField: "name",
as: "skills"
}
},
{
$group: {
_id: "$id",
skills: {
$push: "$skills"
}
}
},
{
$project: {
"id": "$_id",
"skills.name": 1,
"skills.value": 1,
"_id": 0
}
}
])

Aggregation filter and lookup on Mongodb

My first collection employeecategory is like below;
[{
name: "GARDENING"
},
{
name: "SECURITY"
},
{
name: "PLUMBER"
}
]
My second collection complaints is like below;
[{
communityId: 1001,
category: "SECURITY",
//other fields
}, {
communityId: 1001,
category: "GARDENING",
//other fields
}]
I am trying to join above tables and get the below result;
[{
"count": 1,
"name": "GARDENING"
}, {
"count": 1,
"name": "SECURITY"
}, {
"count": 0,
"name": "PLUMBER"
}]
Even if there are no records in collection 2 I need count. I tried below aggregation but didn't worked. If I removed match condition it is working but I need to filter on community id. Could some please suggest best way to do achieve this. Mongo DB version is 3.4.0
db.employeecategory.aggregate(
[{
$match: {
"complaints.communityId": 1001
}
}, {
"$lookup": {
from: "complaints",
localField: "name",
foreignField: "category",
as: "embeddedData"
}
}]
)
It was not possible to achieve both filtering for communityId = 1001 and grouping without losing count = 0 category in a single aggregation. The way to do it is first start from complaints collection, and filter the communityId = 1001 objects, and create a temp collection with it. Then from employeecategory collection, $lookup to join with that temp collection, and $group with name, you will have your result at this point, then drop the temp table.
// will not modify complaints document, will create a filtered temp document
db.complaints.aggregate(
[{
$match: {
communityId: 1001
}
},
{
$out: "temp"
}
]
);
// will return the answer that is requested by OP
db.employeecategory.aggregate(
[{
$lookup: {
from: "temp",
localField: "name",
foreignField: "category",
as: "array"
}
}, {
$group: {
_id: "$name",
count: {
$sum: {
$size: "$array"
}
}
}
}]
).pretty();
db.temp.drop(); // to get rid of this temporary collection
will result;
{ _id: "PLUMBER", count: 0},
{ _id: "SECURITY", count: 2},
{ _id: "GARDENING", count: 1}
for the test data I've had;
db.employeecategory.insertMany([
{ name: "GARDENING" },
{ name: "SECURITY" },
{ name: "PLUMBER" }
]);
db.complaints.insertMany([
{ category: "GARDENING", communityId: 1001 },
{ category: "SECURITY", communityId: 1001 },
{ category: "SECURITY", communityId: 1001 },
{ category: "SECURITY", communityId: 1002 }
]);