I need to make a custom lookup that allow me to only get one item per each value.
example:
source record:
{
...
"names": [ "name 1", "name 2", "name 3" ],
...
}
records to populate:
{ "name": "name 1", "id": 1 }, // populate
{ "name": "name 1", "id": 2 }, // ignore
{ "name": "name 2", "id": 3 }, // populate
{ "name": "name 2", "id": 4 }, // ignore
{ "name": "name 3", "id": 5 }, // populate
so what I need here to only populate id 1, and id 3 and id 5, which means only one record per each name.
I used $group and it works fine but it was very slow for 500 million records DB.
so is there a way in the lookup pipeline to fetch only one record per value.
In the $lookup subpipeline, use $group to get only the $first record per group. $replaceRoot to wrangle to your desired form.
db.source.aggregate([
{
"$lookup": {
"from": "names",
"localField": "names",
"foreignField": "name",
"pipeline": [
{
$group: {
_id: "$name",
record: {
"$first": "$$ROOT"
}
}
},
{
"$replaceRoot": {
"newRoot": "$record"
}
}
],
"as": "names"
}
}
])
Mongo Playground
Related
The scenario is I have 2 collections - 1 with a list of users and 1 with a list of groups.
"users": [
{
"_id": 1,
"username": "Fred Smith",
"level": 12,
},
{
"_id": 2,
"username": "Bob Brown",
"level": 20,
},
{
"_id": 3,
"username": "Joe Blogs",
"level": 1
}
],
A group holds a list of users but crucially also potentially holds groups within the same list as well.
"groups": [
{
"_id": 1,
"groupname": "admin",
"members": ["Fred Smith"]
},
{
"_id": 2,
"groupname": "users",
"members": [
"Fred Smith",
"Bob Brown",
"contractors"
]
},
{
"_id": 3,
"groupname": "contractors",
"members": ["Joe Blogs"]
},
{
"_id": 4,
"groupname": "all",
"members": ["users"]
}]
Given a query of a specific username I need to return all groups that user is a member of.
So far I have the $graphLookup working in the following playground - but I am not sure of the next step to take into account the nested group name inside that list of users. There is also the possibility the groups may nested further groups.
db.users.aggregate([
{
$match: {
"username": "Joe Blogs"
}
},
{
"$graphLookup": {
"from": "groups",
"startWith": "$username",
"connectFromField": "username",
"connectToField": "members",
"as": "groups",
"maxDepth": 10
}
},
{
$project: {
"username": 1,
"groups": "$groups.groupname"
}
}
]
https://mongoplayground.net/p/GBRJp42bdjt
[
{
"_id": 3,
"groups": [
"contractors"
],
"username": "Joe Blogs"
}
]
In this example I have queried Fred who is a member of the contractors group but the contractors group is also a member of the users group and the users group is a member of the all group.
So I would need the query to return both contractors (which it does now) but also users and all.
It feels like given an initial array of groups in this query I then need to do a different query with the array of groups to find out what groups they are a member of.
I could modify the structure if required to make this easier but the nesting of groups is a requirement.
Any help appreciated.
As an update I have created another query which given an array of groups will return a normalised list of all the nested groups - https://mongoplayground.net/p/9NGG_wIJkGX
Now I just need to work out how to feed the results of the first query to the second query - if anyone has an idea I would appreciate it.
Your first playground is actually very close already. You just need to use groupname in connectFromField in the $graphLookup to traverse the collection. At the last $project stage, use $reduce with $setUnion to extract all the groups from the $graphLookup result.
db.users.aggregate([
{
$match: {
"username": "Joe Blogs"
}
},
{
"$graphLookup": {
"from": "groups",
"startWith": "$username",
"connectFromField": "groupname",
"connectToField": "members",
"as": "groups",
"maxDepth": 10
}
},
{
$project: {
"username": 1,
"groups": {
"$reduce": {
"input": "$groups",
"initialValue": [],
"in": {
"$setUnion": [
"$$value",
[
"$$this.groupname"
]
]
}
}
}
}
}
])
Mongo Playground
Let's say i have 2 collections
// Post collection:
{
"_id": "61f7a933b209737de4cc2657",
"author": "61d30188cf93e83e08d14112",
"title": "Some title",
"createts": 1643620659355
}
// User collection:
{
"_id": "61d30188cf93e83e08d14112",
"nickname": "Bob",
"link": "bobspage"
}
And i need to get this result
{
"_id": "61f7a933b209737de4cc2657",
"author": {
"_id": "61d30188cf93e83e08d14112",
"nickname": "Bob",
"link": "bobspage"
},
"title": "Some title",
"createts": 1643620659355
}
How can i make a request with aggregation, which will display this output ?
Simply use $lookup in an aggregation query:
First $lookup, which generate a field called author which is an array with all values joined.
To get author as an object get the first result for the array using $arrayElemAt.
db.post.aggregate([
{
"$lookup": {
"from": "user",
"localField": "author",
"foreignField": "_id",
"as": "author"
}
},
{
"$addFields": {
"author": {
"$arrayElemAt": [
"$author",
0
]
}
}
}
])
Example here
I have a question about how to aggregate mongoose deep collection, for example i have 3 collections:
Specialization job
[
{
"id": 122,
"name": "Administration",
},
{
"id": 133,
"name": "IT/Computer"
}
]
Job Position (relation with specialization collection, one specialization job have many job position)
[
{
"id": 1,
"name": "Manager",
"id_specialization": 122
},
{
"id": 2,
"name": "Front End Developer",
"id_specialization": 133
}
]
Job (relation with job position)
[
{
"id": 1,
"id_job_position": "1",
"location": "New York"
},
{
"id": 2,
"id_job_position": "2",
"location": "Dallas"
}
]
I want to make a filter by "specialization", if i choose "122" id of specialization, then i want to show job data which is job position is in that specialization.
[
{
"id": 1,
"id_job_position": "1",
"location": "New York"
}
]
Thanks before.
Demo - https://mongoplayground.net/p/1Bn1OUOODrT
Use $lookup
Performs a left outer join to an unsharded collection in the same database to filter in documents from the "joined" collection for processing. To each input document, the $lookup stage adds a new array field whose elements are the matching documents from the "joined" collection. The $lookup stage passes these reshaped documents to the next stage.
db.jobPosition.aggregate([
{
$match: {
id_specialization: 122
}
},
{
"$lookup": {
"from": "job",
"localField": "id",
"foreignField": "id_job_position",
"as": "jobs"
}
},
{
$project: {
jobs: 1
}
},
{
$unwind: "$jobs" // break into individual documents can skip if only 1 document will come from lookup
},
{
"$replaceRoot": {
"newRoot": "$jobs"
}
},
{
$project: { _id: 0 }
}
])
if only 1 document will come from the lookup
Demo - https://mongoplayground.net/p/CNevmFlEWWZ
db.jobPosition.aggregate([
{
$match: {
id_specialization: 122
}
},
{
"$lookup": {
"from": "job",
"localField": "id",
"foreignField": "id_job_position",
"as": "jobs"
}
},
{
"$replaceRoot": {
"newRoot": {
"$first": "$jobs"
}
}
},
{
"$project": {
_id: 0
}
}
])
I'm trying to build a voting system where you can have X num of options to vote for on an entry.
The query I'm building now is when retrieving an entry, I would like to get the numbers of votes per option on an entry.
I have a very clear understanding of how I would do this in SQL but grasping to understand the concepts of aggregation, lookup, and group in MongoDB
The model looks like this:
Entries
{
"_id": "5fc2765938401a2308e18ac5",
"options": [
{
"name": "First Option"
"_id": "5fc2765938401a2308e18ac6",
},
{
"name": "Second Option"
"_id": "5fc2765938401a2308e18are",
},
{
"name": "Third Option"
"_id": "5fc2765938401a2308e18aef",
},
],
},
{
"_id": "5fc2766438401a2308e18ac8",
"options": [
{
"name": "Some other option"
"_id": "5fc2766438401a2308e18ac9",
},
{
"_id": "5fc2766438401a2308e18aca",
"name": "This is also an option"
}
],
}
Votes
{
"_id": "5fc2765938401a2308e18ac5",
"entryId": "5fc2765938401a2308e18ac6"
},
{
"_id": "5fc2765938401a2308e18aer",
"entryId": "5fc2765938401a2308e18are"
},
{
"_id": "5fc2765938401a2308e18ek",
"entryId": "5fc2765938401a2308e18ac6"
}
...
And I want the results of Entry to look like this.
{
"_id": "5fc2765938401a2308e18ac5",
"options": [
{
"name": "First Option"
"_id": "5fc2765938401a2308e18ac6",
"votes": 1,
},
{
"name": "Second Option"
"_id": "5fc2765938401a2308e18are",
"votes": 0,
},
{
"name": "Third Option"
"_id": "5fc2765938401a2308e18aef",
"votes": 5,
},
],
},
{
"_id": "5fc2766438401a2308e18ac8",
"options": [
{
"name": "Some other option"
"_id": "5fc2766438401a2308e18ac9",
"votes": 3,
},
{
"_id": "5fc2766438401a2308e18aca",
"name": "This is also an option"
"votes": 10,
}
],
}
$lookup to join votes collection, pass local field optoins._id and foreign field entryId
$project get options votes, $map to iterate loop of options array, $filter to get matching entryId records and $size to get count of element in return array, merge votes field and current object using $mergeObjects
db.entries.aggregate([
{
$lookup: {
from: "votes",
localField: "options._id",
foreignField: "entryId",
as: "votes"
}
},
{
$project: {
options: {
$map: {
input: "$options",
as: "a",
in: {
$mergeObjects: [
"$$a",
{
votes: {
$size: {
$filter: {
input: "$votes",
cond: { $eq: ["$$this.entryId", "$$a._id"] }
}
}
}
}
]
}
}
}
}
}
])
Playground
Here is the problem, I am unable to get the following result. Please look into the piece of json and help me out.
This is my data:
[
{
"user_id": "65asdfksadjfk3u4",
"lat": 23.4343,
"long": 15.2382
}
]
Currently my result is:
[
{
"_id": "65asdfksadjfk3u4",
"name": "Srini",
"age": 26,
"some other key": "some other values"
}
]
I need to get the collection from the user_id and add it to the same array object. As you can notice both lat and long are being removed in my current result.
[
{
"_id": "65asdfksadjfk3u4",
"name": "Srini",
"age": 26,
"some other keys": "some other values",
"lat": 23.4343,
"long": 15.2382
}
]
You can append the $lookup stage to join the current pipeline results with the users collections by the user_id fields and then use $mergeObjects in the $replaceRoot to merge the joined documents from users and the current results:
db.collection.aggregate([
/* current pipeline here */
{ "$lookup": {
"from": "users",
"localField": "_id",
"foreignField": "user_id",
"as": "user"
} },
{ "$replaceRoot": {
"newRoot": {
"$mergeObjects": [
{ "$arrayElemAt": [ "$user", 0 ] },
"$$ROOT"
]
}
} },
{ "$project": { "user": 0, "user_id": 0 } }
]);