MongoDB query filter - mongodb

I have a collection Restaurant, Products, and Reviews
restaurants: [
{
_id: 1,
name: "Burger King",
location: {
type: "Point",
coordinates: [
11.111,
11.111
]
},
isOpen: true
},
{
_id: 2,
name: "McDonald's",
location: {
type: "Point",
coordinates: [
22.222,
22.222
]
},
isOpen: true
},
{
_id: 3,
name: "Chick-fil-A",
location: {
type: "Point",
coordinates: [
33.333,
33.333
]
},
isOpen: true
}
],
products: [
{
_id: 1,
name: "Breakfast Whopper Jr.",
price: "$1.29",
isAvailable: true,
isApproved: true,
quantitySold: 50,
restaurant: ObjectId("1")
},
{
_id: 2,
name: "Big Mac",
price: "$4.35",
isAvailable: true,
isApproved: true,
quantitySold: 59,
restaurant: ObjectId("2")
},
{
_id: 3,
name: "Spicy Chicken Sandwich",
price: "$3.29",
isAvailable: true,
isApproved: true,
quantitySold: 60,
restaurant: ObjectId("3")
},
{
_id: 4,
name: "Chicken Sandwich",
price: "$2.29",
isAvailable: true,
isApproved: true,
quantitySold: 58,
restaurant: ObjectId("3")
}
],
reviews: [
{
_id: 1,
message: "Big burger even if it's junior size.",
restaurant: ObjectId("1"),
product: ObjectId("1")
},
{
_id: 2,
message: "Big Mac is really the best burger in town.",
restaurant: ObjectId("2"),
product: ObjectId("2")
},
{
_id: 3,
message: "Spicy Chicken Sandwich rocks!",
restaurant: ObjectId("3"),
product: ObjectId("3")
},
{
_id: 4,
message: "Chicken Sandwich is the best sandwich of Chick-fil-A!",
restaurant: ObjectId("3"),
product: ObjectId("4")
},
{
_id: 5,
message: "Chicken Sandwich is the best!",
restaurant: ObjectId("3"),
product: ObjectId("4")
}
]
My implementation
db.products.aggregate([
{
$lookup: {
"from": "restaurant",
"localField": "restaurant",
"foreignField": "_id",
"as": "restaurant"
}
},
{
$match: {
"restaurant.isOpen": true,
"isApproved": true,
"isAvailable": true
}
},
{
$project: {
"restaurant.isOpen": 1,
"isApproved": 1,
"isAvailable": 1,
"restaurant.location": 1,
"quantitySold": 1
}
},
{
$match: {
"restaurant.location": {
$geoWithin: {
$centerSphere: [[222.22, 222.33], 10/6378.1] // sample coordinates
}
}
}
},
{
$sort: {
// best seller
"quantitySold": -1
}
}
In my implementation. I'm currently getting the 10km of restaurant that isOpen: true and its products that isAvailable: true and isApproved: true. I also sorted the best selling by the quantitySold field.
Now, I want to query also the most reviewed products and I want to show only 1 product per restaurant in my app based on best seller and most reviewed. Then, after that the remaining products will randomize below the sorted best seller and most reviewed.
Example. I'm in 10km of Burger King, MCDO, Chick-fil-A and I will see their products based on best seller and most reviewed (1 product). Then after, I will see all of their products randomize below.

Currently you have OPEN (restaurant), APPROVED and AVAILABLE list of items.
(You have to adjust the existing query's $project section to get few more fields accordingly).
To get the reviews you can use $lookup, where
{
$lookup:
{
from: "reviews",
localField: "_id",
foreignField: "product",
as: "product_reviews"
}
}
you will get product_reviews array, for each product. You can then perform $groupby and $count to get the total count per product in the next pipeline stage.
After getting the above list in the aggregation pipeline use GROUP BY as follows
{ $group : {
_id : "$restaurant_id",
items: { $push : "$$ROOT" }
}}
(Get restaurant id and name for products in $project stage in existing query). Now you have a list with restaurants id/names and their product arrays.
You can refer to below links to know more about $$ROOT
Mongo group and push: pushing all fields
As your list is already sorted you will have the best seller item at top.
Regarding the other food items do you really need to randomize every time? you can then try to work with $sample (refer :https://docs.mongodb.com/manual/reference/operator/aggregation/sample/)
or follow this mongo db aggregate randomize ( shuffle ) results as per your need.

Related

how to use $lookup to query a collection's id which refers to another collection's field(likes sql 'select id in [1,2,3...]')

I have a users collection as below:
[
{ _id:1, name:'tom', menus: [ 4, 5 ] },
{ _id:2, name:'jack', menus: [ 1, 2, 3 ] },
{ _id:3, name:'rose', menus: [ 4 ] },
]
and a menus collection:
[
{ _id:1, name:'/pets', route_url:'/pets', ... },
{ _id:2, name:'/pets/dog', route_url:'/pets/dog', ... },
{ _id:3, name:'/pets/cat', route_url:'/pests/cat', ... },
{ _id:4, name:'/shop', route_url:'/shop', ... },
{ _id:5, name:'/shop/dog', route_url:'/shop/dog', ... },
]
now I want to get a list includes tom's id, name and his menu info, i hope the result is:
[
{
user_id: 1,
user_name: 'tom',
_id: 4,
name: '/shop',
route_url: '/shop',
...
},
{
user_id: 1,
user_name: 'tom',
_id: 5,
name: '/shop/dog',
route_url: '/shop/dog',
...
}
]
how to design this query? below query can work, but it is too complicate and lacks of flexibility.
db.users.aggregate([{$match:{_id:1}},{$unwind:"$menus"},{$lookup:{from:"menus",localField:"menus",foreignField:"_id",as:"output"}},{$addFields:{"menus":{$arrayElemAt:["$output", 0]}}},{$project:{_id:0,user_id:"$_id", user_name:"$name",name:"$menus.name", ...}}])
so anyone has better ideas? thanks. And another question, can I use moogoose's populate method to obtain similar result?

Query to find all documents that don't include an item of specific `_id`

// Product
const ProductSchema = new Schema<TProductModel>(
{
name: String,
}
{ timestamps: true }
);
// OrderDetails
const OrderDetailsSchema = new Schema<TOrderDetailsModel>({
product: { type: Schema.Types.ObjectId, ref: 'Product' },
productsInOrder: { type: Number, default: 1 },
orderLimit: Number,
orderId: { type: Schema.Types.ObjectId, ref: 'Order' },
});
// Order
const OrderSchema = new Schema<TOrderModel>(
{
vendor: { type: Schema.Types.ObjectId, ref: 'Vendor' },
products: [{ type: Schema.Types.ObjectId, ref: 'OrderDetail'
}],
},
{ timestamps: true }
);
I need to find all Order documents that have vendor of vendorId and whose products don't contain an OrderDetails with a specific Product
Say I have an Order:
const myOrder = {
"_id": "62975f946a2047c4e9c67ac6",
"status": "NEW",
"vendor": {
// vendor details
},
"products": [
{
"_id": "629763b74ede0232a7e8e2ab",
"product": "62975f476a2047c4e9c67ab1", // this is reference to "Tape" product
"productsInOrder": 5,
"orderLimit": 5,
"orderId": "62975f946a2047c4e9c67ac6",
"__v": 0
},
],
"createdAt": "2022-06-01T12:46:12.735Z",
"updatedAt": "2022-06-01T13:04:03.025Z",
"orderNumber": 18,
"__v": 3
}
When adding a new order of the same vendor
const newOrder = {
"_id": "629763c24ede0232a7e8e2bc",
"product": "62975f476a2047c4e9c67ab1", // this is reference to "Ladder" product
"productsInOrder": 1,
"orderLimit": 5,
"orderId": "62975f946a2047c4e9c67ac6",
"__v": 0
}
I want to find an Order that doesn't have yet OrderDetails with product set to "62975f476a2047c4e9c67ab1".
I tried
const order = await OrderModel.findOne({
vendor,
$ne: ['products.product', product],
});
but it returns an order that already have product product, it looks like that:
{
_id: new ObjectId("62975f946a2047c4e9c67ac6"),
status: 'NEW',
vendor: new ObjectId("629581466e2469498cff053f"),
products: [
new ObjectId("62975f946a2047c4e9c67ac7"), // these 4 OrderDetails references all have the same `product`, so it shouldn't be found
new ObjectId("629763994ede0232a7e8e298"),
new ObjectId("629763b74ede0232a7e8e2ab"),
new ObjectId("629763c24ede0232a7e8e2bc")
],
createdAt: 2022-06-01T12:46:12.735Z,
updatedAt: 2022-06-01T13:04:03.025Z,
orderNumber: 18,
__v: 3
}
I also tried different aggregations but nothing worked.
I need a query that would find me an Order that don't include an OrderDetails with a product field equals to my provided productId.
I think I need to somehow populate Order's products field with actual OrderDetails objects before looking into its product but I don't know how to implement that.
you can try using MongoDB Aggregation Operations.
You need to see if IDs are stored as string or ObjectId in your schema.
products.aggregate([
{
{ $lookup: {
from: "product",
let: { "id": "$_id.product" },
pipeline: [
{ $match: { "$expr": { "$ne": ["<your ID>", "$$id"] }}},
],
as: "output"
}}
]);
Try it out and try to make more adjustments according to your needs. It's just a sudo code
Useful references MongoDB $lookup pipeline match by _id not working https://www.mongodb.com/docs/manual/reference/operator/aggregation/ne/
https://www.mongodb.com/docs/manual/reference/operator/aggregation/filter/
Try to add some schema and docs for a better understanding of users. Hope this will be helpful.
If you already have an order collection with data like:
{
"_id": "62975f946a2047c4e9c67ac6",
"status": "NEW",
"vendor": {id: 12},
"products": [
{
"_id": "629763b74ede0232a7e8e2ab",
"product": "62975f476a2047c4e9c67ab1", // this is reference to "Tape" product
"productsInOrder": 5,
"orderLimit": 5,
"orderId": "62975f946a2047c4e9c67ac6",
"__v": 0
},
],
"createdAt": "2022-06-01T12:46:12.735Z",
"updatedAt": "2022-06-01T13:04:03.025Z",
"orderNumber": 18,
}
For example, and you want to find a document with a vendor.id: 12 which its products do not contain an item with "product": "62975f476a2047c4e9c67ab1", you can use $elemMatch:
db.orders.find({
"vendor.id": 12,
products: {$not: {$elemMatch: {product: "62975f476a2047c4e9c67ab1"}}}
})
Playground example
If you want to do it in the middle of an aggregation pipeline, you can use a $filter:
db.orders.aggregate([
{$match: {"vendor.id": 12}},
{
$addFields: {
removeProducts: {
$size: {
$filter: {
input: "$products",
as: "item",
cond: {$eq: ["$$item.product", "62975f476a2047c4e9c67ab1"]}
}
}
}
}
},
{$match: {removeProducts: 0}},
{$unset: "removeProducts"}
])
Aggregation playground example

Mongoose GroupBy with Populate

i have User document and and Category document which is connected to shop document which look like
var ShopSchema = mongoose.Schema(
{
shopName: {
type: String,
required: [true, 'A Shop must have name'],
},
phone: {
type: String,
validate: [validator.isMobilePhone,"Phone number is not a valid number"],
required: [true, 'A Shop must have hepline number'],
},
shopStatus: {
type: Boolean,
default: false,
select: false
},
managerId:{
type: mongoose.Schema.ObjectId,
ref:'User',
require: [true,'Shop must have to a manager!']
},
createdAt:{
type:Date,
default: Date.now()
},
shopCategory:{
type: mongoose.Schema.ObjectId,
ref:'Category',
require: [true,'Shop must have a Category!']
},
},
{
toJSON: { virtuals: true },
toObject: { virtuals: true }
}
);
const Shop = mongoose.model("Shop", ShopSchema);
Now i am writing a query to get managers shop(groupBy managerId) with populate of Manager and Category data. but i am getting empty array everytime.
const doc = await Shop.aggregate([
{
$match: { shopStatus: {$ne: false} }
},
{
$group: {
_id:'$managerId',
numofShop: {$sum: 1},
shopCategory: {$first: "$shopCategory"}
}
},
{
$lookup: {
from: "User",
localField : "_id",
foreignField: "_id",
as: "Manager"
}
},
{
$lookup: {
from: "Category",
localField : "shopCategory",
foreignField: "_id",
as: "Category"
}
},
])
here is how my final result look like but this is an empty array.
{
"_id": "5f467660f630e804ec07fad8",
"numofShop": 2,
"Manager": [],
"Category": []
},
{
"_id": "5f44d2f4ff04993b40684bf9",
"numofShop": 1,
"Manager": [],
"Category": []
}
I want to find All shops groupBy managerId(Who own it). there is field of managerId referencing to User document who created this shop. i want data like this
{
"_id": "5f44d2f4ff04993b40684bf9",
"Manager": {},//this is populated through User Schema
"numofShop": 1,
"Shop": [
{
"shopName":"Grocery Shop",
"phone" : "111111",
"shopCategory" :{"_id": "","name":"Electronics"}
}
]
}
.....
.....
Find Example Data here
Shops- http://txt.do/1fwi0
3 shops created by two users
User - http://txt.do/1fwio
Categories- http://txt.do/1fwij
There are 2 fixes,
in second lookup you have no shopCategory field in localField because you have not defined shopCategory in $group,
{
$group: {
_id:'$managerId',
numofShop: {$sum: 1},
shopCategory: { $first: "$shopCategory" } // define here
}
},
in first lookup with user collection, there is no managerId field and you have assigned in _id field, then you can use _id in localField,
{
$lookup: {
from: "User",
localField : "_id", // change this from managerId
foreignField: "_id",
as: "Manager"
}
},
Updated things after updated question, Playground1, Playground2
for your second edit, you can't put condition for populate in find method, how to put condition in populate, you can follow the answer1 and answer2

MongoDB aggregate query $lookup

I'm trying to populate my user field with $lookup and here is my output:
[
{
...
user: [],
...
}
]
As you can see, The output is an array and user field is an empty array as well, also I want to make a query, that compare multiple fields. Lets say I have this object:
{
social: {
facebook: 'facebook.com/asd',
twitter: 'https://twitter.com/'
},
skills: [ 'asd', 'asd', 'html' ],
_id: 5eec9c9a2d6d1d46e8b05fb4,
user: {
_id: 5eec9c752d6d1d46e8b05fb3,
name: 'Simeon Lazarov',
avatar: 'uploads\\1592682240020 - background.jpg'
},
status: 'Junior Developer',
experience: [
{
current: true,
_id: 5eef06638af42217d8c5480a,
title: 'ASD',
company: 'SAD',
location: 'ASD',
from: 2020-06-08T00:00:00.000Z,
to: null
}
],
education: [
{
current: true,
_id: 5eef5f0dd06ee324a4a7b1d2,
school: 'asd',
degree: 'asd',
fieldofstudy: 'asd',
from: 2020-06-24T00:00:00.000Z,
to: null
}],
contacts: [ { _id: 5eee23bd862ab70ba0235a85, user: 5eedffa6062e6231a859433a } ],
date: 2020-06-19T11:08:10.883Z,
__v: 1,
website: 'asd',
company: 'asd'
}
And want to compare if company, skills(all skills of this object to be contained in others objects) or etc. are equal.
Here is my NOT working code:
await Profile.aggregate([{$lookup: {from:"user",localField: "user",
foreignField: "name", as: "user"}},{$match :{$or:[{"company":{$regex:profile.company}},{"skills":{ $eq: profile.skills} }]}}]).skip((req.params.page - 1) * 10).limit(11);
P.S Sorry for writing so bad, it's my first question here.
It seems you just have mismatches in your $lookup fields compare to what the schema holds, I rewrote it like this:
await Profile.aggregate([
{
$match: {
$or: [
{"company": {$regex: profile.company}},
{"skills": {$all: profile.skills}}
]
}
},
{
$skip: req.params.page ? req.params.page - 1 : 0 // make sure this can't be -1
},
{
$limit: 11
},
{
$lookup: {
from: "user",
localField: "user",
foreignField: "_id",
as: "user"
}
},
{
"$unwind": "$user"
}
])
Notice I moved the $lookup to be the last stage as it's the most expensive part of the pipeline and there's no need to do it on the entire collection.

Getting array of group _id while using MongoDB

I need to calculate some record using group in mongoDB but in my case array of _id are coming. I am explaining my query below.
let query = {
$group: {
_id: "$Products.ProductName",
dispatchcount: { $sum: 1 },
totalDispatchValue: {$sum:"$TotalAmount.TotalPayment"},
totalDiscountValue: {$sum: "$TotalDiscount.TotalDiscountAmount"}
}
}
pipeLine.push(query);
const orders = await dispatchOrdersCol.aggregate(pipeLine);
Actual Output:
[
{
"_id": [
"Unisex Navy Blue Belt"
],
"dispatchcount": 1,
"totalDispatchValue": 27922,
"totalDiscountValue": 4084
},
{
"_id": [
"Writing Ruled Note Book",
"Physics Record Book",
"Chemistry Record Book",
"Computer Science Record Book"
],
"dispatchcount": 1,
"totalDispatchValue": 2190,
"totalDiscountValue": 0
},
{
"_id": [
"PU2-Physics Part-1 Text Book",
"PU2-Physics Part-2 Text Book",
"PU2-Mathematics Part - 1 Text Book",
"PU2-Chemistry Part - 1 Text Book",
"PU2-English Text Book",
"PU2-Mathematics Part - 2 Text Book",
"PU2-Chemistry Part - 2 Text Book",
"PU2-English Work Book",
"PU2-TEXT BOOK",
"PU2-Sanskrit Text Book",
"Boys White & Blue Striped Half Shirt",
"Boys Navy Blue Regular Fit Trousers",
"Illume Calf-length Cotton Black Socks  "
],
"dispatchcount": 1,
"totalDispatchValue": 4131,
"totalDiscountValue": 150
},
{
"_id": [
"PU2-TEXT BOOK"
],
"dispatchcount": 1,
"totalDispatchValue": 1679,
"totalDiscountValue": 0
}
]
Here for _id key there are multiple values coming and some of the values also repeating in next record.
Mongoose Model:
const Model = new Schema({
DispatchId: { type: Number, required: true },
OrderNumber: { type: String, unique: true, required: true },
OrderStatus: { type: String },
OrderType: { type: String },
DispatchPriority: { type: String },
Comments: { type: Array },
Products: { type: Array },
Customer: { type: Object },
TotalDiscount: { type: Object },
TotalShipping: { type: Object },
TotalCoupon: { type: Object },
TotalAmount: { type: Object },
PaymentDetails: { type: Object },
ClientDetails: { type: Object },
DispatchCreatedAt: { type: String },
DispatchUpdatedAt: { type: String },
IsActive: { type: Boolean }
},
{
timestamps: {
createdAt: 'CreatedAt',
updatedAt: 'UpdatedAt'
},
collection: DISPATCH_ORDERS_COLLECTION
}
);
Here I need to calculate the total Payment and total Discount as per the product name.But in my case one product is also coming two times in two record.
In your documents Products is an array, (might be an array of Objects with a field ProductName) when you do "$Products.ProductName" it will give you an array of products names. So $group has to group on arrays with exactly same elements inside them(Even a mis-match of one element between two arrays would be treated as two values on _id in group stage). So you need to do $unwind on Products array prior to $group stage :
In-correct Query :
db.collection.aggregate([
/** You don't need this addFields stage, just given to test, you can remove `$group` stage & check the `products` field value */
{
$addFields: {
products: "$products.productName"
}
},
{
$group: {
_id: "$products" // arrays
}
}
])
Test : mongoplayground
Correct Query :
db.collection.aggregate([
{
$unwind: "$products"
},
{
$addFields: {
products: "$products.productName"
}
},
{
$group: {
_id: "$products" // Single value
}
}
])
Test : mongoplayground
Ref : $unwind