I have two collections: Products and Stocks.
The relation between these two collections is one to one.
Products structure:
{
_id:
sku:
....
}
Stocks structure :
{
_id:
sku:
availability: []
....
}
My query:
Products.aggregate([
{
$match: cAux
}, {
$lookup: {
from: "Stocks",
localField: "sku",
foreignField: "sku",
as: "availability"
}
}, {
$sort: PRODUCT_SORT
}
]);
The result from this "join" is
{
_id:
sku:
availability: {_id:, sku:, **availabity**: []}
...
}
The join is okay, but I would like to only have the availability array field being joined and not the whole Stock document. Whats the best way to accomplish this? Any help would be appreciated.
Solution
Products.aggregate([
{
$match: cAux
}, {
$lookup: {
from: "Stocks",
localField: "sku",
foreignField: "sku",
as: "availability"
}
}, {
$project: {
...PRODUCT_FIELDS,
availability: {
$arrayElemAt: ['$availability.availability', 0]
}
}
}, {
$sort: PRODUCT_SORT
}
]);
Related
I've been trying to get get a specific field of $lookup result,
I start with this:
$lookup: {
from : "answers",
localField: "_id",
foreignField: "questionID",
as: "usersAnswered"
}
it returns something like this:
{
_id: "616974f1b4f67d0220fe2cf1",
questionText: "text abc ?",
userID: "614c7a75403a5636b4029f28",
usersAnswered: [{
_id: "6169635cb4f67d0220fe2aa4",
answerText: "xyz",
questionID: "616974f1b4f67d0220fe2cf1",
userID: "614c7a75403a5636b4029f21"
},{
_id: "6169635cb4f67d0220fe2ab8",
answerText: "lmo",
questionID: "616974f1b4f67d0220fe2cf1",
userID: "614c7a75403a5636b4029cc2"
}]
}
In this example what I'm trying to achieve is get only userID from usersAnswered array instead if whole object which I really don't need.
It should look like:
usersAnswered: [{
userID: "614c7a75403a5636b4029f21"
},{
userID: "614c7a75403a5636b4029cc2"
}]
}
I've actually tried to get it done but I wasn't able to accomplish it so had to get some help, could I perhaps do something like
$lookup: {
from : "answers",
localField: "_id",
foreignField: "questionID.userID",
as: "usersAnswered"
}
Or maybe the key is to $group after $unwind.
If you run already MongoDB 5.0 then try this one:
{
$lookup:
{
from: "answers",
localField: "_id",
foreignField: "questionID",
pipeline: [ {$project: {userID: 1} } ],
as: "usersAnswered"
}
}
Otherwise add this stage after $lookup:
{ $set:
{
usersAnswered: {
$map: { input: "$usersAnswered", in: { userID: "$$this.userID" } }
}
}
}
or
{ $set:
{
usersAnswered: {$first:
{
$map: { input: "$usersAnswered", in: { userID: "$$this.userID" } }
}
}
}
}
I am using mongoDB as NoSql Database. I have two collections One is CUSTOMER(CID,CNAME) and Other one is SHOP(Bill_No, Type, Amount,CID)
In SHOP Collection, CID is referenced with Customer(CID).Also Type can be either "SELL" or "PURCHASE".
db.CUSTOMER.insertOne({"CID":1,"CNAME":"Mark"});
db.CUSTOMER.insertOne({"CID":2,"CNAME":"Chris"});
db.CUSTOMER.insertOne({"CID":3,"CNAME":"James"});
db.SHOP.insertOne({"Bill_No":1,"TYPE":"SELL","Amount":1000,"CID":1});
db.SHOP.insertOne({"Bill_No":2,"TYPE":"SELL","Amount":350,"CID":2});
db.SHOP.insertOne({"Bill_No":3,"TYPE":"PURCHASE","Amount":450,"CID":1});
db.SHOP.insertOne({"Bill_No":4,"TYPE":"PURCHASE","Amount":360,"CID":3});
db.SHOP.insertOne({"Bill_No":5,"TYPE":"SELL","Amount":800,"CID":3});
What should be query to find the list of customers who has SELL and PURCHASE Both.
So according to given data the output should be like
"CID":1, "NAME":"MARK"
"CID":3, "NAME":"JAMES"
Thanks in advance..
Demo - https://mongoplayground.net/p/ZXylHb2g7Dm
Get all details linked to CUSTOMER from SHOP using $
lookup
Use $group combine all types into 1 document
To match both types "PURCHASE", "SELL" use $all
db.CUSTOMER.aggregate([
{
$lookup: {
from: "SHOP",
let: { c_id: "$CID" },
pipeline: [
{ $match: { $expr: {$eq: [ "$CID", "$$c_id" ] } } },
{ $group: { _id: null, types: { "$addToSet": "$TYPE" } } },
{ $match: { types: { $all: [ "PURCHASE", "SELL" ] } } }
],
as: "trades"
}
},
{ $match: { "trades.types.0": { $exists: true } } },
{ $project: { _id: 0, CID: 1, CNAME: 1 } }
])
You can use $lookup for aggregation, click here for reference.
Syntax
{
$lookup:
{
from: <collection to join>,
localField: <field from the input documents>,
foreignField: <field from the documents of the "from" collection>,
as: <output array field>
}
}
In your case the query would be
db.CUSTOMER.aggregate([
{
$lookup:
{
from: "SHOP",
localField: "CID",
foreignField: "CID",
as: "SELL_PURCHASE"
}
}
])
I have 3 collections to aggregate.
1st is colors collection
{
{
_id: 1, <- mongoose objectId
name: red
},
{
_id: 2, <- mongoose objectId
name: green
}
}
2nd is products
{
{
_id: Id777, <- mongoose objectId
productName: test prod 777
},
{
_id: Id888, <- mongoose objectId
productName: test prod 888
}
}
and 3rd it move collection
{
....other fields here
items: [
{
_id: an mongoose id,
itemId: Id777 <- in products collection,
itemColor: 1 <- id in colors collection,
coutn: 7,
....other fields
},
{
_id: an mongoose id,
itemId: Id888 <- in products collection,
itemColor: 2 <- id in colors collection
cout: 10
....other fields
}
]
}
I need to have an output like this:
{
////information from collection
items: [
{
itemId: test prod 777, itemColor: red, count: 7
},
{
itemId: test prod 888, itemColor: green, count: 10
}
]
}
My code is:
const moves = await ProductMoves.aggregate([
{ $match: query }, // this is my query
{
$lookup: {
from: 'products',
localField: 'items.itemId',
foreignField: '_id',
as: 'productName'
}
},
{
$unwind: { path: "$productName" , preserveNullAndEmptyArrays: true }
},
{
$lookup: {
from: 'colors',
localField: 'items.itemColor',
foreignField: '_id',
as: 'cName'
}
},
{
$unwind: { path: "$cName" , preserveNullAndEmptyArrays: true }
},
{
$addFields: {
mItems: {
prName: "$productName.productName",
prColor: "$cName.colorName"
},
productName: 0,
cName: 0
}
}
])
.sort({addedDate: -1})
.skip(+req.query.offset)
.limit(+req.query.limit)
but it returns only 1 element from the object array. probably I need something like a for loop, but i couldn't do it.
thank you for your responses, and have a good day!
$unwind deconstruct items array
$lookup with products collection
$lookup with colors collection
$addFields, $arrayElemAt to get first element from lookup result
$group by _id and reconstruct items array and pass other fields as well
there is no external methods in an aggregate function, you have to use stages for sort, skip and limit like below
$sort by addedDate in descending order
$skip and $limit result
const moves = await ProductMoves.aggregate([
{ $match: query }, // this is my query
{ $unwind: "$items" },
{
$lookup: {
from: "products",
localField: "items.itemId",
foreignField: "_id",
as: "itemId"
}
},
{
$lookup: {
from: "colors",
localField: "items.itemColor",
foreignField: "_id",
as: "itemColor"
}
},
{
$addFields: {
"items.itemId": { $arrayElemAt: ["$itemId.productName", 0] },
"items.itemColor": { $arrayElemAt: ["$itemColor.name", 0] }
}
},
{
$group: {
_id: "$_id",
items: { $push: "$items" },
addedDate: { $first: "$addedDate" }
// add other fields that you want in result like "addedDate"
}
},
{ $sort: { addedDate: -1 } },
{ $skip: +req.query.offset },
{ $limit: +req.query.limit }
])
Playground
I have a MongodDB database on MongoDB Atlas.
It has an "orders", "products", "itemTypes" and "brands".
"orders" only keep track of product id ordered.
"products" only keep track of brand id and itemType id
"itemTypes" keep track of item type name
"brands" keep track of brand name.
If I aggregate orders + products + itemTypes it is ok:
[{
$unwind: {
path: '$orders'
}
}, {
$lookup: {
from: 'products',
localField: 'orders.productId',
foreignField: 'productId',
as: 'products'
}
}, {
$lookup: {
from: 'itemTypes',
localField: 'products.typeId',
foreignField: 'typeId',
as: 'itemTypes'
}
}, {
$set: {
'orders.price': {
$arrayElemAt: ['$products.price', 0]
},
'orders.brandId': {
$arrayElemAt: ['$products.brandId', 0]
},
'orders.typeId': {
$arrayElemAt: ['$products.typeId', 0]
},
'orders.typeName': {
$arrayElemAt: ['$itemTypes.name', 0]
}
}
}, {
$group: {
_id: '$_id',
createdAt: {
$first: '$createdAt'
},
status: {
$first: '$status'
},
retailerId: {
$first: '$retailerId'
},
retailerName: {
$first: '$retailerName'
},
orderId: {
$first: '$orderId'
},
orders: {
$push: '$orders'
}
}
}]
If I aggregate orders + products + itemTypes + brands, either Mongo Compass or the web UI of Mongo Atlas aggregation builder will give operation timeout error.
[{
$unwind: {
path: '$orders'
}
}, {
$lookup: {
from: 'products',
localField: 'orders.productId',
foreignField: 'productId',
as: 'products'
}
}, {
$lookup: {
from: 'itemTypes',
localField: 'products.typeId',
foreignField: 'typeId',
as: 'itemTypes'
}
}, {
$lookup: {
from: 'brands',
localField: 'products.brandId',
foreignField: 'brandId',
as: 'brands'
}
}, {
$set: {
'orders.price': {
$arrayElemAt: ['$products.price', 0]
},
'orders.brandId': {
$arrayElemAt: ['$products.brandId', 0]
},
'orders.typeId': {
$arrayElemAt: ['$products.typeId', 0]
},
'orders.typeName': {
$arrayElemAt: ['$itemTypes.name', 0]
},
'orders.brandName': {
$arrayElemAt: ['$brands.name', 0]
}
}
}, {
$group: {
_id: '$_id',
createdAt: {
$first: '$createdAt'
},
status: {
$first: '$status'
},
retailerId: {
$first: '$retailerId'
},
retailerName: {
$first: '$retailerName'
},
orderId: {
$first: '$orderId'
},
orders: {
$push: '$orders'
}
}
}]
This is a demo of the aggregation that timed out:
https://mongoplayground.net/p/Jj6EhSl58MS
We have approximately 50k orders, 14k products, 200 brands, 89 item types.
Is there anyway to optimise this aggregation so that it won't timeout?
P/s: My ultimate goal is to visualise popular brands and item types ordered using beautiful chart in the Mongodb Charts function.
If you are on Mongo Atlas, you can use Triggers to run the aggregation query in the background - either when the database is updated or as a scheduled trigger (https://docs.mongodb.com/realm/triggers/).
When the trigger runs, you can save the result of the aggregation pipeline in a new collection using the "$merge" operation.
exports = function() {
const mongodb = context.services.get(CLUSTER_NAME);
const orders = mongodb.db(DATABASE_NAME).collection("orders");
const ordersSummary = mongodb.db(DATABASE_NAME).collection("orders.summary");
const pipeline = [
{
YOUR_PIPELINE
},
{ $merge: { into: "orders.summary", on: "_id", whenMatched: "replace", whenNotMatched: "insert" } }
];
orders.aggregate(pipeline);
};
This way, your charts will be very fast, since they only have to do a simple query from the new collection.
Do you have index on the collections you $lookup from:
products (productId) + itemTypes (typeId) + brands (brandId).
Otherwise, the lookups can take a long time to complete.
I have found a few questions that relate to this (here and here) but I have been unable to interpret the answers in a way that I can understand how to do what I need.
I have 3 collections: Organisations, Users, and Projects. Every project belongs to one user, and every user belongs to one organisation. From the user's id, I need to return all the projects that belong to the organisation that the logged-in user belongs to.
Returning the projects from the collection that belong to the user is easy, with this query:
const projects = await Project.find({ user: req.user.id }).sort({ createdAt: -1 })
Each user has an organisation id as a foreign key, and I think I need to do something with $lookup and perhaps $unwind mongo commands, but unlike with SQL queries I really struggle to understand what's going on so I can construct queries correctly.
EDIT: Using this query
const orgProjects = User.aggregate(
[
{
$match: { _id: req.user.id }
},
{
$project: { _id: 0, org_id: 1 }
},
{
$lookup: {
from: "users",
localField: "organisation",
foreignField: Organisation._id,
as: "users_of_org"
}
},
{
$lookup: {
from: "projects",
localField: "users_of_org._id",
foreignField: "user",
as: "projects"
}
},
{
$unset: ["organisation", "users_of_org"]
},
{
$unwind: "$projects"
},
{
$replaceWith: "$projects"
}
])
Seems to almost work, returning the following:
Aggregate {
_pipeline: [
{ '$match': [Object] },
{ '$project': [Object] },
{ '$lookup': [Object] },
{ '$lookup': [Object] },
{ '$unset': [Array] },
{ '$unwind': '$projects' },
{ '$replaceWith': '$projects' }
],
_model: Model { User },
options: {}
}
assuming your documents have a schema like this, you could do an aggregation pipeline like below with 2 $lookup stages.
db.users.aggregate(
[
{
$match: { _id: "user1" }
},
{
$project: { _id: 0, org_id: 1 }
},
{
$lookup: {
from: "users",
localField: "org_id",
foreignField: "org_id",
as: "users_of_org"
}
},
{
$lookup: {
from: "projects",
localField: "users_of_org._id",
foreignField: "user_id",
as: "projects"
}
},
{
$unset: ["org_id", "users_of_org"]
},
{
$unwind: "$projects"
},
{
$replaceWith: "$projects"
}
])