I am trying $lookup in Array but can't merge it in my object after $lookup.
Collection
_id: '5f7a1053477c8a1ae88e22cf',
name: 'Demo'
price: 423,
related: [
{
idProduct: '61140763ab806726a8ab7aea'
quantity: 2
},
{
idProduct: '61140763ab806726a8ab7aeb'
quantity: 6
},
]
Expected Output:
_id: '5f7a1053477c8a1ae88e22cf',
name: 'Demo',
price: 423,
related: [
{
idProduct: {
_id: '61140763ab806726a8ab7aea',
name: 'related1',
price: 22
},
quantity: 2
},
{
idProduct: {
_id: '61140763ab806726a8ab7aeb',
name: 'related2',
price: 53
},
quantity: 6
},
]
I need $lookup, idProduct and keep that data structure.
Any help? Thanks in advance
$unwind deconstruct related array
$lookup with collection 2, pass related.idProduct as local field and _id as foreign field and set in related.idProduct
$group by _id and reconstruct related array and get first required firlds
db.col1.aggregate([
{ $unwind: "$related" },
{
$lookup: {
from: "col2",
localField: "related.idProduct",
foreignField: "_id",
as: "related.idProduct"
}
},
{
$group: {
_id: "$_id",
name: { $first: "$name" },
price: { $first: "$price" },
related: { $push: "$related" }
}
}
])
Playground
The second approach without using $unwind stage,
$lookup with collection 2,
$map to iterate loop of related array
$filter to iterate loop of col2 array by idProduct field
$arrayElemAt to get first element from above filtered result
$mergeObjects to merge current object with updated idProduct field
db.col1.aggregate([
{
$lookup: {
from: "col2",
localField: "related.idProduct",
foreignField: "_id",
as: "col2"
}
},
{
$addFields: {
col2: "$$REMOVE",
related: {
$map: {
input: "$related",
as: "r",
in: {
$mergeObjects: [
"$$r",
{
idProduct: {
$arrayElemAt: [
{
$filter: {
input: "$col2",
cond: { $eq: ["$$r.idProduct", "$$this._id"] }
}
},
0
]
}
}
]
}
}
}
}
}
])
Playground
Related
In the lookup part in the aggregate method, how can I keep only documents that have a value in the foreign collection?
For instance, I have this collection users:
[
{ _id: 1, name: 'John', basketId: 4 },
{ _id: 2, name: 'mari', basketId: 9 },
{ _id: 3, name: 'tedd', basketId: 32 },
{ _id: 4, name: 'sara', basketId: 14 },
{ _id: 5, name: 'jane', basketId: 3 },
.
.
.
]
And another collection named baskets
[
{ _id: 1, items: 0 },
{ _id: 2, items: 2 },
{ _id: 3, items: 0 },
{ _id: 4, items: 0 },
{ _id: 5, items: 7 },
.
.
.
]
Now if I want to get users with basket items greater than 0, I use aggregate and lookup:
UserModel.aggregate([
{ $lookup:
{
from: 'baskets',
localField: 'basketId',
foreignField: '_id',
pipeline: [{ $match: { items: { $gt: 0 } } }],
as: 'basket'
}
}
])
It brings up ALL users with their basket data. For those users whose basket items are 0, it shows basket: [].
But I need to get ONLY users that have basket items greater than 0. How can it be done?
You shouldn't place the $match stage in the pipeline of $lookup. As what it did is filter the documents to be returned in the basket array.
Instead, you need a $match stage to filter the documents by comparing the first document's items value in the basket array.
UserModel.aggregate([
{
$lookup: {
from: "baskets",
localField: "basketId",
foreignField: "_id",
as: "basket"
}
},
{
$match: {
$expr: {
$gt: [
{
$first: "$basket.items"
},
0
]
}
}
}
])
Demo 1 # Mongo Playground
The question is ambiguous. You may look for the below query as well (but would return the same result as Demo 1):
UserModel.aggregate([
{
$lookup: {
from: "baskets",
localField: "basketId",
foreignField: "_id",
pipeline: [
{
$match: {
items: {
$gt: 0
}
}
}
],
as: "basket"
}
},
{
$match: {
$expr: {
$gt: [
{
$size: "$basket"
},
0
]
}
}
}
])
Or check is not an empty array
{
$ne: [
"$basket",
[]
]
}
Demo 2 # Mongo Playground
Here is a hypothetical case of orders and products.
'products' collection
[
{
"_id": "61c53eb76eb2dc65de621bd0",
"name": "Product 1",
"price": 80
},
{
"_id": "61c53efca0a306c3f1160754",
"name": "Product 2",
"price": 10
},
... // truncated
]
'orders' collection:
[
{
"_id": "61c53fb7dca0579de038cea8", // order id
"products": [
{
"_id": "61c53eb76eb2dc65de621bd0", // references products._id
"quantity": 1
},
{
"_id": "61c53efca0a306c3f1160754",
"quantity": 2
},
]
}
]
As you can see, an order owns a list of product ids. When I pull an order's details I also need the product details combined like so:
{
_id: ObjectId("61c53fb7dca0579de038cea8"),
products: [
{
_id: ObjectId("61c53eb76eb2dc65de621bd0"),
quantity: 1,
name: 'Product 1',
price: 80
},
{
_id: ObjectId("61c53efca0a306c3f1160754"),
quantity: 2,
name: 'Product 2',
price: 10
},
... // truncated
]
}
Here is the aggregation pipleline I came up with:
db.orders.aggregate([
{
$match: {_id: ObjectId('61c53fb7dca0579de038cea8')}
},
{
$unwind: {
path: "$products"
}
},
{
$lookup: {
from: 'products',
localField: 'products._id',
foreignField: '_id',
as: 'productDetail'
}
},
{
$unwind: {
path: "$productDetail"
}
},
{
$group: {
_id: "$_id",
products: {
$push: {$mergeObjects: ["$products", "$productDetail"]}
}
}
}
])
Given how the data is organized I'm doubting if the pipeline stages are optimal and could do better (possibility of reducing the number of stages, etc.). Any suggestions?
As already mentioned in comments the design is poor. You can avoid multiple $unwind and $group, usually the performance should be better with this:
db.orders.aggregate([
{ $match: { _id: "61c53fb7dca0579de038cea8" } },
{
$lookup: {
from: "products",
localField: "products._id",
foreignField: "_id",
as: "productDetail"
}
},
{
$project: {
products: {
$map: {
input: "$products",
as: "product",
in: {
$mergeObjects: [
"$$product",
{
$first: {
$filter: {
input: "$productDetail",
cond: { $eq: [ "$$this._id", "$$product._id" ] }
}
}
}
]
}
}
}
}
}
])
Mongo Playground
I have 3 collections to aggregate.
1st is colors collection
{
{
_id: 1, <- mongoose objectId
name: red
},
{
_id: 2, <- mongoose objectId
name: green
}
}
2nd is products
{
{
_id: Id777, <- mongoose objectId
productName: test prod 777
},
{
_id: Id888, <- mongoose objectId
productName: test prod 888
}
}
and 3rd it move collection
{
....other fields here
items: [
{
_id: an mongoose id,
itemId: Id777 <- in products collection,
itemColor: 1 <- id in colors collection,
coutn: 7,
....other fields
},
{
_id: an mongoose id,
itemId: Id888 <- in products collection,
itemColor: 2 <- id in colors collection
cout: 10
....other fields
}
]
}
I need to have an output like this:
{
////information from collection
items: [
{
itemId: test prod 777, itemColor: red, count: 7
},
{
itemId: test prod 888, itemColor: green, count: 10
}
]
}
My code is:
const moves = await ProductMoves.aggregate([
{ $match: query }, // this is my query
{
$lookup: {
from: 'products',
localField: 'items.itemId',
foreignField: '_id',
as: 'productName'
}
},
{
$unwind: { path: "$productName" , preserveNullAndEmptyArrays: true }
},
{
$lookup: {
from: 'colors',
localField: 'items.itemColor',
foreignField: '_id',
as: 'cName'
}
},
{
$unwind: { path: "$cName" , preserveNullAndEmptyArrays: true }
},
{
$addFields: {
mItems: {
prName: "$productName.productName",
prColor: "$cName.colorName"
},
productName: 0,
cName: 0
}
}
])
.sort({addedDate: -1})
.skip(+req.query.offset)
.limit(+req.query.limit)
but it returns only 1 element from the object array. probably I need something like a for loop, but i couldn't do it.
thank you for your responses, and have a good day!
$unwind deconstruct items array
$lookup with products collection
$lookup with colors collection
$addFields, $arrayElemAt to get first element from lookup result
$group by _id and reconstruct items array and pass other fields as well
there is no external methods in an aggregate function, you have to use stages for sort, skip and limit like below
$sort by addedDate in descending order
$skip and $limit result
const moves = await ProductMoves.aggregate([
{ $match: query }, // this is my query
{ $unwind: "$items" },
{
$lookup: {
from: "products",
localField: "items.itemId",
foreignField: "_id",
as: "itemId"
}
},
{
$lookup: {
from: "colors",
localField: "items.itemColor",
foreignField: "_id",
as: "itemColor"
}
},
{
$addFields: {
"items.itemId": { $arrayElemAt: ["$itemId.productName", 0] },
"items.itemColor": { $arrayElemAt: ["$itemColor.name", 0] }
}
},
{
$group: {
_id: "$_id",
items: { $push: "$items" },
addedDate: { $first: "$addedDate" }
// add other fields that you want in result like "addedDate"
}
},
{ $sort: { addedDate: -1 } },
{ $skip: +req.query.offset },
{ $limit: +req.query.limit }
])
Playground
I'm trying to do this for days, but can't find any success
I'm using MongoDB, and I tried to do it with many pipeline steps but I couldn't find a way.
I have a players collection, each player contains an items array
{
"_id": ObjectId("5fba17c1c4566e57fafdcd7e"),
"username": "moshe",
"items": [
{
"_id": ObjectId("5fbb5ac178045a985690b5fd"),
"equipped": false,
"itemId": "5fbb5ab778045a985690b5fc"
}
]
}
I have an items collection where there is more information about each item
in the player items array.
{
"_id": ObjectId("5fbb5ab778045a985690b5fc"),
"name": "Axe",
"damage": 4,
"defense": 6
}
My goal is to have a player document with all the information about the item inside his items array, so it will look like that:
{
"_id": ObjectId("5fba17c1c4566e57fafdcd7e"),
"username": "moshe",
"items": [
{
"_id": ObjectId("5fbb5ac178045a985690b5fd"),
"equipped": false,
"itemId": "5fbb5ab778045a985690b5fc",
"name": "Axe",
"damage": 4,
"defense": 6
}
]
}
$unwind deconstruct items array
$lookup to join items collection, pass itemsId into let after converting it to object id using $toObjectId and pass items object,
$match itemId condition
$mergeObject merge items object and $$ROOT object and replace to root using $replaceRoot
$group reconstruct items array again, group by _id and get first username and construct items array
db.players.aggregate([
{ $unwind: "$items" },
{
$lookup: {
from: "items",
let: {
itemId: { $toObjectId: "$items.itemId" },
items: "$items"
},
pipeline: [
{ $match: { $expr: { $eq: ["$_id", "$$itemId" ] } } },
{ $replaceRoot: { newRoot: { $mergeObjects: ["$$items", "$$ROOT"] } } }
],
as: "items"
}
},
{
$group: {
_id: "$_id",
username: { $first: "$username" },
items: { $push: { $first: "$items" } }
}
}
])
Playground
Second option using $map, and without $unwind,
$addFields for items convert itemId string to object type id using $toObjectId and $map
$lookup to join items collection
$project to show required fields, and merge items array and itemsCollection using $map to iterate loop of items array $filter to get matching itemId and $first to get first object from return result, $mergeObject to merge current object and returned object from $first
db.players.aggregate([
{
$addFields: {
items: {
$map: {
input: "$items",
in: {
$mergeObjects: ["$$this", { itemId: { $toObjectId: "$$this.itemId" } }]
}
}
}
}
},
{
$lookup: {
from: "items",
localField: "items.itemId",
foreignField: "_id",
as: "itemsCollection"
}
},
{
$project: {
username: 1,
items: {
$map: {
input: "$items",
as: "i",
in: {
$mergeObjects: [
"$$i",
{
$first: {
$filter: {
input: "$itemsCollection",
cond: { $eq: ["$$this._id", "$$i.itemId"] }
}
}
}
]
}
}
}
}
}
])
Playground
First I'd strongly suggest that you should store the items.itemId as ObjectId, not strings.
Then another simple solution can be:
db.players.aggregate([
{
$lookup: {
from: "items",
localField: "items.itemId",
foreignField: "_id",
as: "itemsDocuments",
},
},
{
$addFields: {
items: {
$map: {
input: { $zip: { inputs: ["$items", "$itemsDocuments"] } },
in: { $mergeObjects: "$$this" },
},
},
},
},
{ $unset: "itemsDocuments" },
])
I want to do a Full Outer Join in MongoDB by lookup mongoDB query. Is this possible? Is a Full Outer Join supported by MongoDB by any other alternative?
[Update:]
I want to achieve result from Collection1 & Collection2 as following attachment:
Example: Result Required
In above result column there may be different arithmetic operations and will be further used in calculations.
You can use $unionWith (starting 4.4)
Something like this:
db.c1.aggregate([
{$set: {
mark1: "$marks"
}},
{$unionWith: {
coll: 'c2',
pipeline: [{$set: {mark2: "$marks"}}]
}},
{$group: {
_id: "$name",
result: {
$sum: "$marks"
},
mark1: {$first: {$ifNull: ["$mark1", 0]}},
mark2: {$first: {$ifNull: ["$mark2", 0]}}
}}])
I have named the collections as coll1 and coll2 then just use this query it will give you the required output.
db.getCollection('coll1').aggregate([
{
$facet: {
commonRecords: [{
$lookup: {
from: "coll2",
localField: 'name',
foreignField: 'name',
as: "coll2"
}
},
{
$unwind: {
path: '$coll2',
preserveNullAndEmptyArrays: true
}
}
]
}
},
{
$lookup: {
from: "coll2",
let: {
names: {
$map: {
input: '$commonRecords',
as: 'commonRecord',
in: '$$commonRecord.name'
}
}
},
pipeline: [{
$match: {
$expr: {
$eq: [{
$indexOfArray: ['$$names', '$name']
}, -1]
}
}
}, ],
as: "coll2"
}
},
{
$addFields: {
coll2: {
$map: {
input: '$coll2',
as: 'doc',
in: {
coll2: '$$doc'
}
}
}
}
},
{
$project: {
records: {
$concatArrays: ['$commonRecords', '$coll2']
}
}
},
{
$unwind: '$records'
},
{
$replaceRoot: {
newRoot: '$records'
}
},
{
$project: {
_id: 0,
name: {
$ifNull: ['$name', '$coll2.name']
},
marks1: {
$ifNull: ['$marks', 0]
},
marks2: {
$ifNull: ['$coll2.marks', 0]
}
}
},
{
$addFields: {
result: {
$add: ['$marks1', '$marks2']
}
}
}
])
This is a sample:
{
$lookup:
{
from: [collection to join],
local_Field: [field from the input documents],
foreign_Field: [field from the documents of the "from" collection],
as: [output field]
}
}
show this link