Couldn't find an answer here
I got a collection of segments, where an example segment looks like this:
{
_id: {
$oid: "62e5778e34362ad54db6b3f9"
},
name: "Good-Natured Developers"
}
And a collection of users, where an example user looks like this:
{
_id: {
$oid: "62e5225dfdc41ba4f7effefa"
},
age: {
$numberInt: "43"
},
segment_ids: [
{
$oid: "62e577a034362ad54db6b40a"
},
{
$oid: "62e5782f34362ad54db6b497"
},
]
}
How can I query how many users are there in every segment?
I tried this aggregation:
const usersInSegment = await userCollection
.aggregate([
{ $unwind: '$users'},
{ $project: { "_id": 1} },
{
$lookup: {
from: 'segments',
localField: 'segment_ids',
foreignField: '_id',
as: 'joined'
}
}
])
.limit(25)
.toArray()
But I get an empty array.
Related
I'm currently running a query that looks like this:
courses = await Enrollment.aggregate([
{
$match: {
userId: userId
}
},
{
$lookup: {
from: 'courses',
localField: 'course',
foreignField: '_id',
as: 'course'
}
},
{
$unwind: '$course'
},
{
$project: {
course: {
courseCode: true,
name: true,
officialCode: true,
}
}
}
]);
This produces results that look like this
[{
"_id": "61e0652ba5c2fe5bdcdbdc23",
"course": {
"courseCode": "code2",
"name": "Terst class 2",
"officialCode": "test 202"
}
}]
I'm wondering if there is a way for me to bring courseCode, name and officialCode to the "highest level" of the document?
Thank you in advance.
You can do it with $replaceRoot aggregation pipeline. Add this as the last step:
{
"$replaceRoot": {
"newRoot": "$course"
}
}
Working example
I have 3 collections to aggregate.
1st is colors collection with colors details, name value etc
{
{
_id: 1, <- mongoose objectId
name: red
},
{
_id: 2, <- mongoose objectId
name: green
}
}
2nd is products (products details)
{
{
_id: Id777, <- mongoose objectId
productName: test prod 777
},
{
_id: Id888, <- mongoose objectId
productName: test prod 888
}
}
and 3rd it move collection. here is the array of object to aggregation
{
....other fields here
items: [
{
_id: an mongoose id,
itemId: Id777 <- in products collection,
itemColor: 1 <- id in colors collection,
coutn: 7,
....other fields
},
{
_id: an mongoose id,
itemId: Id888 <- in products collection,
itemColor: 2 <- id in colors collection
cout: 10
....other fields
}
]
}
this code works good if I get many documents from collection.
but when I try to get only 1 document by _id, its didn't working
const query = {
$and: [
{appId: req.user.appId},
{_id: req.params._id} <- here is mongodb id
],
const moves = await ProductMoves.aggregate([
{ $match: query }, // this is my query
{ $unwind: "$items" },
{
$lookup: {
from: "products",
localField: "items.itemId",
foreignField: "_id",
as: "itemId"
}
},
{
$lookup: {
from: "colors",
localField: "items.itemColor",
foreignField: "_id",
as: "itemColor"
}
},
{
$addFields: {
"items.itemId": { $arrayElemAt: ["$itemId.productName", 0] },
"items.itemColor": { $arrayElemAt: ["$itemColor.name", 0] }
}
},
{
$group: {
_id: "$_id",
items: { $push: "$items" },
addedDate: { $first: "$addedDate" }
}
}
])
So, how I can get only 1 document by _id and aggregate it? Thank you!
I have 3 collections to aggregate.
1st is colors collection
{
{
_id: 1, <- mongoose objectId
name: red
},
{
_id: 2, <- mongoose objectId
name: green
}
}
2nd is products
{
{
_id: Id777, <- mongoose objectId
productName: test prod 777
},
{
_id: Id888, <- mongoose objectId
productName: test prod 888
}
}
and 3rd it move collection
{
....other fields here
items: [
{
_id: an mongoose id,
itemId: Id777 <- in products collection,
itemColor: 1 <- id in colors collection,
coutn: 7,
....other fields
},
{
_id: an mongoose id,
itemId: Id888 <- in products collection,
itemColor: 2 <- id in colors collection
cout: 10
....other fields
}
]
}
I need to have an output like this:
{
////information from collection
items: [
{
itemId: test prod 777, itemColor: red, count: 7
},
{
itemId: test prod 888, itemColor: green, count: 10
}
]
}
My code is:
const moves = await ProductMoves.aggregate([
{ $match: query }, // this is my query
{
$lookup: {
from: 'products',
localField: 'items.itemId',
foreignField: '_id',
as: 'productName'
}
},
{
$unwind: { path: "$productName" , preserveNullAndEmptyArrays: true }
},
{
$lookup: {
from: 'colors',
localField: 'items.itemColor',
foreignField: '_id',
as: 'cName'
}
},
{
$unwind: { path: "$cName" , preserveNullAndEmptyArrays: true }
},
{
$addFields: {
mItems: {
prName: "$productName.productName",
prColor: "$cName.colorName"
},
productName: 0,
cName: 0
}
}
])
.sort({addedDate: -1})
.skip(+req.query.offset)
.limit(+req.query.limit)
but it returns only 1 element from the object array. probably I need something like a for loop, but i couldn't do it.
thank you for your responses, and have a good day!
$unwind deconstruct items array
$lookup with products collection
$lookup with colors collection
$addFields, $arrayElemAt to get first element from lookup result
$group by _id and reconstruct items array and pass other fields as well
there is no external methods in an aggregate function, you have to use stages for sort, skip and limit like below
$sort by addedDate in descending order
$skip and $limit result
const moves = await ProductMoves.aggregate([
{ $match: query }, // this is my query
{ $unwind: "$items" },
{
$lookup: {
from: "products",
localField: "items.itemId",
foreignField: "_id",
as: "itemId"
}
},
{
$lookup: {
from: "colors",
localField: "items.itemColor",
foreignField: "_id",
as: "itemColor"
}
},
{
$addFields: {
"items.itemId": { $arrayElemAt: ["$itemId.productName", 0] },
"items.itemColor": { $arrayElemAt: ["$itemColor.name", 0] }
}
},
{
$group: {
_id: "$_id",
items: { $push: "$items" },
addedDate: { $first: "$addedDate" }
// add other fields that you want in result like "addedDate"
}
},
{ $sort: { addedDate: -1 } },
{ $skip: +req.query.offset },
{ $limit: +req.query.limit }
])
Playground
I have a MongodDB database on MongoDB Atlas.
It has an "orders", "products", "itemTypes" and "brands".
"orders" only keep track of product id ordered.
"products" only keep track of brand id and itemType id
"itemTypes" keep track of item type name
"brands" keep track of brand name.
If I aggregate orders + products + itemTypes it is ok:
[{
$unwind: {
path: '$orders'
}
}, {
$lookup: {
from: 'products',
localField: 'orders.productId',
foreignField: 'productId',
as: 'products'
}
}, {
$lookup: {
from: 'itemTypes',
localField: 'products.typeId',
foreignField: 'typeId',
as: 'itemTypes'
}
}, {
$set: {
'orders.price': {
$arrayElemAt: ['$products.price', 0]
},
'orders.brandId': {
$arrayElemAt: ['$products.brandId', 0]
},
'orders.typeId': {
$arrayElemAt: ['$products.typeId', 0]
},
'orders.typeName': {
$arrayElemAt: ['$itemTypes.name', 0]
}
}
}, {
$group: {
_id: '$_id',
createdAt: {
$first: '$createdAt'
},
status: {
$first: '$status'
},
retailerId: {
$first: '$retailerId'
},
retailerName: {
$first: '$retailerName'
},
orderId: {
$first: '$orderId'
},
orders: {
$push: '$orders'
}
}
}]
If I aggregate orders + products + itemTypes + brands, either Mongo Compass or the web UI of Mongo Atlas aggregation builder will give operation timeout error.
[{
$unwind: {
path: '$orders'
}
}, {
$lookup: {
from: 'products',
localField: 'orders.productId',
foreignField: 'productId',
as: 'products'
}
}, {
$lookup: {
from: 'itemTypes',
localField: 'products.typeId',
foreignField: 'typeId',
as: 'itemTypes'
}
}, {
$lookup: {
from: 'brands',
localField: 'products.brandId',
foreignField: 'brandId',
as: 'brands'
}
}, {
$set: {
'orders.price': {
$arrayElemAt: ['$products.price', 0]
},
'orders.brandId': {
$arrayElemAt: ['$products.brandId', 0]
},
'orders.typeId': {
$arrayElemAt: ['$products.typeId', 0]
},
'orders.typeName': {
$arrayElemAt: ['$itemTypes.name', 0]
},
'orders.brandName': {
$arrayElemAt: ['$brands.name', 0]
}
}
}, {
$group: {
_id: '$_id',
createdAt: {
$first: '$createdAt'
},
status: {
$first: '$status'
},
retailerId: {
$first: '$retailerId'
},
retailerName: {
$first: '$retailerName'
},
orderId: {
$first: '$orderId'
},
orders: {
$push: '$orders'
}
}
}]
This is a demo of the aggregation that timed out:
https://mongoplayground.net/p/Jj6EhSl58MS
We have approximately 50k orders, 14k products, 200 brands, 89 item types.
Is there anyway to optimise this aggregation so that it won't timeout?
P/s: My ultimate goal is to visualise popular brands and item types ordered using beautiful chart in the Mongodb Charts function.
If you are on Mongo Atlas, you can use Triggers to run the aggregation query in the background - either when the database is updated or as a scheduled trigger (https://docs.mongodb.com/realm/triggers/).
When the trigger runs, you can save the result of the aggregation pipeline in a new collection using the "$merge" operation.
exports = function() {
const mongodb = context.services.get(CLUSTER_NAME);
const orders = mongodb.db(DATABASE_NAME).collection("orders");
const ordersSummary = mongodb.db(DATABASE_NAME).collection("orders.summary");
const pipeline = [
{
YOUR_PIPELINE
},
{ $merge: { into: "orders.summary", on: "_id", whenMatched: "replace", whenNotMatched: "insert" } }
];
orders.aggregate(pipeline);
};
This way, your charts will be very fast, since they only have to do a simple query from the new collection.
Do you have index on the collections you $lookup from:
products (productId) + itemTypes (typeId) + brands (brandId).
Otherwise, the lookups can take a long time to complete.
I am doing an aggregation in Paper collection like below
const papers = await Paper.aggregate([
{
"$lookup": {
"from": "reviews",
"localField": "reviewId",
"foreignField": "_id",
"as": "review"
}
},
{ $unwind: '$review' }
]);
It returns the result that contains review object which has a reviews array like:
[
{
...
review: {
_id: 5f1638770f3a8d20f8c1beeb,
reviews: [Array],
},
...
}
]
If I make the review more clear, it is like below:
{
_id: 5f1638770f3a8d20f8c1beeb
reviews: [
{
_id: 5f164395857bcdd1d8674b69,
reviewerId: 5f15b28d534b5886c0d9eb8a
},
{
_id: 5f164395857bcdd1d8674b6a,
reviewerId: 5f1358c523dc2367c43a6311
}
]
}
In above, reviewerId inside reviews array refers to user id from "users" collection. I want to get users name, email, and address in reviews array like below:
{
reviews: [
{
_id: 5f164395857bcdd1d8674b69,
reviewerId: 5f15b28d534b5886c0d9eb8a
reviewer : {
name:"some_name",
email:"abc#example.com"
}
},
{
_id: 5f164395857bcdd1d8674b6a,
reviewerId: 5f1358c523dc2367c43a6311
reviewer : {
name:"some_name",
email:"efg#example.com"
}
}
]
}
How can I achieve it?
Hopefully, the structure of your collection almost similar as I mention below in the Mongo playground.
db.reviews.aggregate([
{
$unwind: {
path: "$reviews",
preserveNullAndEmptyArrays: false
}
},
{
$lookup: {
from: "user",
localField: "reviews.reviewerId",
foreignField: "_id",
as: "reviews.reviewer"
}
},
{
$group: {
_id: "$_id",
question: {
$first: "$question"
},
reviews: {
$addToSet: "$reviews"
}
}
}
])
Working Mongo playground