use lookup and group different collection mongodb - mongodb

Hello I have the following collections
const TransactionSchema = mongoose.Schema({
schedule: {
type: mongoose.Schema.ObjectId,
required: true,
ref: "Schedule"
},
uniqueCode: {
type: String,
required: true
},
created: {
type: Date,
default: Date.now
},
status: {
type: String,
required: false
},
})
const ScheduleSchema = mongoose.Schema({
start: {
type: Date,
required: true,
},
end: {
type: Date,
required: false,
},
location: {
type: mongoose.Schema.ObjectId,
required: true,
ref: "Location"
},
})
and I want to return how many times the schedule appear in transaction ( where the status is equal to 'Active') and group it based on its location Id and then lookup the location collection to show the name.
For example I have the following data.
transaction
[
{
"_id":"identifier",
"schedule":identifier1,
"uniqueCode":"312312312312",
"created":"Date",
"status": 'Active'
},
{
"_id":"identifier",
"schedule":identifier1,
"uniqueCode":"1213123123",
"created":"Date",
"status": "Deleted"
}
]
schedule
[
{
"_id":identifier1,
"start":"date",
"end":"date",
"location": id1
},
{
"_id":identifier2,
"start":"date",
"end":"date",
"location": id2
}
]
and I want to get the following result and limit the result by 10 and sort it based on its total value:
[
{
"locationName":id1 name,
"total":1
},
{
"locationName":id2 name,
"total":0
}
]
thank you. Sorry for my bad english.

A bit complex and long query.
$lookup - schedule collection joins with transaction collection by matching:
_id (schedule) with schedule (transaction)
status is Active
and return a transactions array.
$lookup - schedule collection joins with location collection to return location array.
$set - Take the first document in location array so this field would be a document field instead of an array. [This is needed to help further stage]
$group - Group by location._id. And need the fields such as location and total.
$sort - Sort by total DESC.
$limit - Limit to 10 documents to be returned.
$project - Decorate the output documents.
db.schedule.aggregate([
{
$lookup: {
from: "transaction",
let: {
scheduleId: "$_id"
},
pipeline: [
{
$match: {
$expr: {
$and: [
{
$eq: [
"$schedule",
"$$scheduleId"
]
},
{
$eq: [
"$status",
"Active"
]
}
]
}
}
}
],
as: "transactions"
}
},
{
$lookup: {
from: "location",
localField: "location",
foreignField: "_id",
as: "location"
}
},
{
$set: {
location: {
$first: "$location"
}
}
},
{
$group: {
_id: "$location._id",
location: {
$first: "$location"
},
total: {
$sum: {
$size: "$transactions"
}
}
}
},
{
$sort: {
"total": -1
}
},
{
$limit: 10
},
{
$project: {
_id: 0,
locationName: "$location.name",
total: 1
}
}
])
Sample Mongo Playground

Related

Boost search score from data in another collection

I use Atlas Search to return a list of documents (using Mongoose):
const searchResults = await Resource.aggregate()
.search({
text: {
query: searchQuery,
path: ["title", "tags", "link", "creatorName"],
},
}
)
.match({ approved: true })
.addFields({
score: { $meta: "searchScore" }
})
.exec();
These resources can be up and downvoted by users (like questions on Stackoverflow). I want to boost the search score depending on these votes.
I can use the boost operator for that.
Problem: The votes are not a property of the Resource document. Instead, they are stored in a separate collection:
const resourceVoteSchema = mongoose.Schema({
_id: { type: String },
userId: { type: mongoose.Types.ObjectId, required: true },
resourceId: { type: mongoose.Types.ObjectId, required: true },
upDown: { type: String, required: true },
After I get my search results above, I fetch the votes separately and add them to each search result:
for (const resource of searchResults) {
const resourceVotes = await ResourceVote.find({ resourceId: resource._id }).exec();
resource.votes = resourceVotes
}
I then subtract the downvotes from the upvotes on the client and show the final number in the UI.
How can I incorporate this vote points value into the score of the search results? Do I have to reorder them on the client?
Edit:
Here is my updated code. The only part that's missing is letting the resource votes boost the search score, while at the same time keeping all resource-votes documents in the votes field so that I can access them later. I'm using Mongoose syntax but an answer with normal MongoDB syntax will work for me:
const searchResults = await Resource.aggregate()
.search({
compound: {
should: [
{
wildcard: {
query: queryStringSegmented,
path: ["title", "link", "creatorName"],
allowAnalyzedField: true,
}
},
{
wildcard: {
query: queryStringSegmented,
path: ["topics"],
allowAnalyzedField: true,
score: { boost: { value: 2 } },
}
}
,
{
wildcard: {
query: queryStringSegmented,
path: ["description"],
allowAnalyzedField: true,
score: { boost: { value: .2 } },
}
}
]
}
}
)
.lookup({
from: "resourcevotes",
localField: "_id",
foreignField: "resourceId",
as: "votes",
})
.addFields({
searchScore: { $meta: "searchScore" },
})
.facet({
approved: [
{ $match: matchFilter },
{ $skip: (page - 1) * pageSize },
{ $limit: pageSize },
],
resultCount: [
{ $match: matchFilter },
{ $group: { _id: null, count: { $sum: 1 } } }
],
uniqueLanguages: [{ $group: { _id: null, all: { $addToSet: "$language" } } }],
})
.exec();
It could be done with one query only, looking similar to:
Resource.aggregate([
{
$search: {
text: {
query: "searchQuery",
path: ["title", "tags", "link", "creatorName"]
}
}
},
{$match: {approved: true}},
{$addFields: {score: {$meta: "searchScore"}}},
{
$lookup: {
from: "ResourceVote",
localField: "_id",
foreignField: "resourceId",
as: "votes"
}
}
])
Using the $lookup step to get the votes from the ResourceVote collection
If you want to use the votes to boost the score, you can replace the above $lookup step with something like:
{
$lookup: {
from: "resourceVote",
let: {resourceId: "$_id"},
pipeline: [
{
$match: {$expr: {$eq: ["$resourceId", "$$resourceId"]}}
},
{
$group: {
_id: 0,
sum: {$sum: {$cond: [{$eq: ["$upDown", "up"]}, 1, -1]}}
}
}
],
as: "votes"
}
},
{$addFields: { votes: {$arrayElemAt: ["$votes", 0]}}},
{
$project: {
"wScore": {
$ifNull: [
{$multiply: ["$score", "$votes.sum"]},
"$score"
]
},
createdAt: 1,
score: 1
}
}
As you can see on this playground example
EDIT: If you want to keep the votes on the results, you can do something like:
db.searchResults.aggregate([
{
$lookup: {
from: "ResourceVote",
localField: "_id",
foreignField: "resourceId",
as: "votes"
}
},
{
"$addFields": {
"votesCount": {
$reduce: {
input: "$votes",
initialValue: 0,
in: {$add: ["$$value", {$cond: [{$eq: ["$$this.upDown", "up"]}, 1, -1]}]}
}
}
}
},
{
$addFields: {
"wScore": {
$add: [{$multiply: ["$votesCount", 0.1]}, "$score"]
}
}
}
])
As can be seen here

Getting `[ [Object] ]` as nested array of object response on MongoDB Aggregation query

I'm trying to do an aggregation on two collections that has a linkage between them, and I need to access information in an array of objects in one of those collections.
Here are the schemas:
User Schema:
{
_id: ObjectId,
username: String,
password: String,
associatedEvents: [
{
event_id: ObjectId,
isCreator: boolean,
access_level: String,
}
]
}
Event Schema:
{
_id: ObjectId,
title: String,
associated_users: [
{
user_id: ObjectId
}
]
}
I'm attempting to get the users associated to an event for a specific user, and then get their access level information. Here's the aggregation I have:
const eventsJoined = await Event.aggregate([
{
$match: {
$expr: { $in: [id, "$associatedUserIds"] },
},
},
{
$lookup: {
from: "users",
localField: "associatedUserIds",
foreignField: "_id",
as: "user_info",
},
},
{ $unwind: "$user_info" },
{
$unwind: {
path: "$user_info.associatedEvents",
preserveNullAndEmptyArrays: true,
},
},
{
$group: {
_id: "$_id",
title: { $first: "$title" },
description: { $first: "$description" },
startDate: { $first: "$startdate" },
userInfo: { $first: "$user_info" },
usersAssociatedEvents: { $push: "$user_info.associatedEvents" },
},
},
{
$project: {
title: 1,
description: 1,
startDate: 1,
userInfo: 1,
usersAssociatedEvents: "$usersAssociatedEvents",
},
},
]);
And this is the result I'm getting:
[
{
_id: 609d5ad1ef4cdbeb32987739,
title: 'hello',
description: 'desc',
startDate: null,
usersAssociatedEvents: [ [Object] ]
}
]
As you can see, the query is already aggregating the correct data. But the last thing that's tripping me up is the fact that the aggregation is [ [Object] ] for usersAssociatedEvents instead of the actual contents of the object. Any idea on why that would be?

MongoDB aggregate returning only specific fields

I have the following code:
const profiles = await Profile.aggregate([
{
$lookup: {
from: "users",
localField: "user",
foreignField: "_id",
as: "user",
},
},
{
$unwind: "$user",
},
{
$match: {
"user.name": {
$regex: q.trim(),
$options: "i",
},
},
},
{
$skip: req.params.page ? (req.params.page - 1) * 10 : 0,
},
{
$limit: 11,
},
{
$group: {
_id: "$_id",
skills:{skills}
user: { name: "$name" },
user: { avatar: "$avatar" },
},
},
]);
I want to return only specific fields like skills _id and user.name and user.avatar, but this doesn't work. I'm pretty sure that the problem is in $group. I want to receive only these fields
[
{
_id: 5ef78d005d23020ca847aa76,
skills: [ 'asd' ],
user: {
_id: 5ef78c7c5d23020ca847aa75,
name: 'Simeon Lazarov',
avatar: 'uploads\\1593286096227 - background.jpg',
}
}
]
You can make use of $project to get specific fields.
After grouping add the below:
{
$project: {_id:1, skills:1, user:1}
}
Projection value of 0 means that the field needs to be excluded, Value 1 represents inclusion of the field.
Document reference: https://docs.mongodb.com/manual/reference/operator/aggregation/project/

Aggregate $lookup Array of Objects

I have collection schools with field groups and I am attempting to $lookup documents from the users collection. I am getting empty results however and an extra schools document.
Schools Schema
const SchoolSchema = new Schema({
groups: [
{
name: { type: String },
color: { type: String },
userDrivenName: { type: String },
},
]
});
module.exports = School = mongoose.model("School", SchoolSchema);
User Schema
const UserSchema = new Schema({
name: {
type: String,
required: true,
},
groups: [
{
groupId: { type: String },
name: { type: String },
color: { type: String },
userDrivenName: { type: String },
},
]
});
Query
db.schools.aggregate([
{
$match: {
_id: ObjectId("5d836e584a24e20e6090fd7b")
}
},
{
$project: {
groups: 1
}
},
{
$unwind: "$groups"
},
{
$lookup: {
from: "users",
let: {
groupId: "$groups._id"
},
pipeline: [
{
$match: {
"groups.groupId": "$$groupId"
}
}
],
as: "groups",
},
},
])
Results:
[
{
"_id": "5d836e584a24e20e6090fd7b",
"groups": []
},
{
"_id": "5d836e584a24e20e6090fd7b",
"groups": []
}
]
Expected Results:
[
{
"_id":"5d836e584a24e20e6090fd7b",
"groups":[
{
"_id":"5ec01fdc1dfb0a4f08316dfe",
"name":"GROUP 1",
"users":[
{
"name":"Luke Skywalker"
}
]
}
]
}
]
MongoPlayground
Two things:
There's a type mismatch between groupId and groups.groupId so you need to use $toString (based on your Mongo Playground example),
$lookup with custom pipelines allows only expression when you use $match so you need $in and $expr:
{
$lookup: {
from: "users",
let: { groupId: { $toString: "$groups._id" } },
pipeline: [
{
$match: {
$expr: {
$in: ["$$groupId","$groups.groupId"]
}
}
}
],
as: "groups"
}
}
Mongo Playground

MongoDB Aggregate Query Group By And Count

I have a subscribers collection:
{id: ObjectId ('52dskf45f10dsd4775'), agencyCode: '4579520',
listOfSubscriptions: [
{name: 'william', CreatedAt: ISODate ('2019-01-02'), subscriptionType: 'smartphones'},
{name: 'marianna', CreatedAt: ISODate ('2019-02-02'), subscriptionType: 'smartphones'},
{name: 'Freewind', CreatedAt: ISODate ('2019-04-02'), subscriptionType: 'smartphones'}
]
}
The agencys collection contains the following data:
{agencyCode: '4579520', BU: 'finances', company: ObjectId (' 445700500b24dsjdfm ')}
{agencyCode: '45007', BU: 'finances', company: ObjectId (' 445700500b24dsjdfm ')}
Collection subscriptionsType:
{id:ObjectId ('5lkf81gf45005drkj') ,nameSubscription: 'intense',SubscriptionType: 'smartphones' }
{id:ObjectId ('eb4512ezope780') ,nameSubscription: 'other',SubscriptionType: 'other' }
I want to do a cont on the subscribers by type of subscription.
Here is my request mongodb:
subscribers.aggregate(
[
{
$lookup: {
from: "agencys",
localField: "agencyCode",
foreignField: "agencyCode",
as: "data_agency"
}
},
{ $unwind: "$data_agency" },
{ $unwind: "$listOfSubscriptions" },
{
$match: {
$and: [
{ $agencyCode: '4579520' },
{ "data_agency.BU": 'finances' },
{ "data_agency.company": mongoose.Types.ObjectId('445700500b24dsjdfm') },
]
}
},
{
$group: {
"_id": "$listOfSubscriptions.subscriptionType",
countLines: { $sum: 1 },
}
},
]
result of my request:
{
"_id": "smartphones",
"countLines": 3
}
the result of my request does not display the rest of the subscription type.
I want to display all subscription types like this for the agency '4579520':
{
"_id": "smartphones",
"countLines": 3
},
{
"_id": "other",
"countLines": 0
}
Thanks,