Data set with parent Child tree in MongoDB. and i need details of parent with each row. I am unable to stuff the data set as below in Mongo.
Collection Structure:
Expected Output:
You need $graphLookup to run recursive query in MongoDB. Once you get an object and all it's parents you can run $concatArrays to create one array and then use $unwind with $replaceRoot to get multiple documents in a result set:
db.collection.aggregate( [
{
$graphLookup: {
from: "collection",
startWith: "$parent_org",
connectFromField: "parent_org",
connectToField: "_id",
as: "hierarchy"
}
},
{
$match: {
_id: 4
}
},
{
$project: {
result: {
$concatArrays: [ "$hierarchy", [ { _id: "$_id", "org_name": "$org_name", parent_org: "$parent_org" } ] ]
}
}
},
{
$unwind: "$result"
},
{
$replaceRoot: {
newRoot: "$result"
}
},
{
$sort: {
_id: 1
}
}
])
Mongo Playground
Related
MongoDB Collection A contains documents with an array with some document ids of collection B:
Collection A:
{
some_ids_of_b: ["id1", ...]
}
Collection B:
{
_id: "id1"
},
{
_id: "id2"
},
...
How do I query all documents from B whose _ids are NOT in contained in the some_ids_of_b arrays of documents of A?
Simple lookup from collection B to A and filter to keep only those documents where you don't find any matches.
db.collb.aggregate([
{
"$lookup": {
"from": "colla",
"localField": "_id",
"foreignField": "someIdsOfB",
"as": "a"
}
},
{
$match: {
$expr: {
$eq: [{$size: "$a"}, 0]
}
}
}
])
Demo
One option is:
db.collectionB.aggregate([
{$lookup: {
from: "collectionA",
let: {my_id: "$_id"},
pipeline: [
{$match: {$and: [
{_id: collADocId},
{$expr: {$in: ["$$my_id", "$some_ids_of_b"]}}
]}},
{$project: {_id: 1}}
],
as: "some_ids_of_b"
}},
{$match: {"some_ids_of_b.0": {$exists: false}}},
{$unset: "some_ids_of_b"}
])
See how it works on the playground example
You can do it with Aggregation Framework:
$group and $addToSet - To get all $some_ids_of_b from all the documents in A collection.
$set with $reduce - To create an array with all unique values of the IDs from the B collection.
$lookup - To fetch the documents from the B collection, where the _id of the document is not present in the $b_ids array.
$project - To project data as expected output.
db.A.aggregate([
{
"$group": {
"_id": null,
"b_ids": {
"$addToSet": "$some_ids_of_b"
}
}
},
{
"$set": {
b_ids: {
$reduce: {
input: "$b_ids",
initialValue: [],
in: {
$setUnion: [
"$$value",
"$$this"
]
}
}
}
}
},
{
"$lookup": {
from: "B",
let: {
b_ids: "$b_ids"
},
pipeline: [
{
"$match": {
"$expr": {
$ne: [
{
"$in": [
"$_id",
"$$b_ids"
]
},
true
]
}
}
}
],
as: "data"
}
},
{
"$project": {
data: 1,
_id: 0
}
}
])
Working Example
I have collection like below named as "FormData",
{
"_id": ObjectId("5e3c27bf1ef77236945ef07b"),
"eed12747-0923-4290-b09c-5a05107f5609": "20200206",
"bd637691-782d-4cfd-8624-feeedfe11b3e": "20200206_1#mail.com"
}
I have another collection named as "Form" which will have Title of Fields,
{
"_id": ObjectId("5e3c27bf1ef77236945ef07b"),
"Fields":[
{
"FieldID": "eed12747-0923-4290-b09c-5a05107f5609",
"Title": "Phone"
},
{
"FieldID": "bd637691-782d-4cfd-8624-feeedfe11b3e",
"Title": "Email"
}]
}
Now I have to map element name with Form field title and I need result like below,
{
"_id": ObjectId("5e3c27bf1ef77236945ef07b"),
"Phone": "20200206",
"Email": "20200206_1#mail.com"
}
Please help me to solve this.
Thanks in advance!
You can:
$objectToArray to convert the $$ROOT document into an array of k-v pairs for future lookups
use a sub-pipeline in $lookup to find the value by the uuid
use $mergeObject to combine the original values(i.e. "20200206"...) with the new field name looked up (i.e. "Phone"...)
wrangle the result back into original form using $arrayToObject and $replaceRoot
db.FormData.aggregate([
{
$match: {
"_id": ObjectId("5e3c27bf1ef77236945ef07b")
}
},
{
$project: {
kv: {
"$objectToArray": "$$ROOT"
}
}
},
{
$unwind: "$kv"
},
{
"$lookup": {
"from": "Form",
"let": {
uuid: "$kv.k"
},
"pipeline": [
{
$match: {
"_id": ObjectId("5e3c27bf1ef77236945ef07b")
}
},
{
"$unwind": "$Fields"
},
{
$match: {
$expr: {
$eq: [
"$$uuid",
"$Fields.FieldID"
]
}
}
},
{
$project: {
_id: false,
k: "$Fields.Title"
}
}
],
"as": "formLookup"
}
},
{
$unwind: "$formLookup"
},
{
$project: {
kv: {
"$mergeObjects": [
"$kv",
"$formLookup"
]
}
}
},
{
$group: {
_id: "$_id",
kv: {
$push: "$kv"
}
}
},
{
"$project": {
newDoc: {
"$arrayToObject": "$kv"
}
}
},
{
"$replaceRoot": {
"newRoot": {
"$mergeObjects": [
{
"_id": "$_id"
},
"$newDoc"
]
}
}
}
])
Mongo Playground
Another option is to start from Form collection and avoid $unwind:
$match and $lookup to get all needed data into one document
$objectToArray to get known keys for FormData
Match the items using $indexOfArray and $arrayElemAt and merge them using $mergeObjects. Then use arrayToObject to format the response
db.Form.aggregate([
{$match: {_id: ObjectId("5e3c27bf1ef77236945ef07b")}},
{$lookup: {
from: "FormData",
localField: "_id",
foreignField: "_id",
as: "formLookup",
pipeline: [{$project: {_id: 0}}]
}},
{$set: {formLookup: {$objectToArray: {$first: "$formLookup"}}}},
{$replaceRoot: {
newRoot: {
$mergeObjects: [
{$arrayToObject: {
$map: {
input: "$formLookup",
in: {$mergeObjects: [
{v: "$$this.v"},
{k: {$getField: {
input: {$arrayElemAt: [
"$Fields",
{$indexOfArray: ["$Fields.FieldID", "$$this.k"]}
]},
field: "Title"
}}}
]}
}
}},
{_id: "$_id"}
]
}
}}
])
See how it works on the playground example
I'm trying to do this for days, but can't find any success
I'm using MongoDB, and I tried to do it with many pipeline steps but I couldn't find a way.
I have a players collection, each player contains an items array
{
"_id": ObjectId("5fba17c1c4566e57fafdcd7e"),
"username": "moshe",
"items": [
{
"_id": ObjectId("5fbb5ac178045a985690b5fd"),
"equipped": false,
"itemId": "5fbb5ab778045a985690b5fc"
}
]
}
I have an items collection where there is more information about each item
in the player items array.
{
"_id": ObjectId("5fbb5ab778045a985690b5fc"),
"name": "Axe",
"damage": 4,
"defense": 6
}
My goal is to have a player document with all the information about the item inside his items array, so it will look like that:
{
"_id": ObjectId("5fba17c1c4566e57fafdcd7e"),
"username": "moshe",
"items": [
{
"_id": ObjectId("5fbb5ac178045a985690b5fd"),
"equipped": false,
"itemId": "5fbb5ab778045a985690b5fc",
"name": "Axe",
"damage": 4,
"defense": 6
}
]
}
$unwind deconstruct items array
$lookup to join items collection, pass itemsId into let after converting it to object id using $toObjectId and pass items object,
$match itemId condition
$mergeObject merge items object and $$ROOT object and replace to root using $replaceRoot
$group reconstruct items array again, group by _id and get first username and construct items array
db.players.aggregate([
{ $unwind: "$items" },
{
$lookup: {
from: "items",
let: {
itemId: { $toObjectId: "$items.itemId" },
items: "$items"
},
pipeline: [
{ $match: { $expr: { $eq: ["$_id", "$$itemId" ] } } },
{ $replaceRoot: { newRoot: { $mergeObjects: ["$$items", "$$ROOT"] } } }
],
as: "items"
}
},
{
$group: {
_id: "$_id",
username: { $first: "$username" },
items: { $push: { $first: "$items" } }
}
}
])
Playground
Second option using $map, and without $unwind,
$addFields for items convert itemId string to object type id using $toObjectId and $map
$lookup to join items collection
$project to show required fields, and merge items array and itemsCollection using $map to iterate loop of items array $filter to get matching itemId and $first to get first object from return result, $mergeObject to merge current object and returned object from $first
db.players.aggregate([
{
$addFields: {
items: {
$map: {
input: "$items",
in: {
$mergeObjects: ["$$this", { itemId: { $toObjectId: "$$this.itemId" } }]
}
}
}
}
},
{
$lookup: {
from: "items",
localField: "items.itemId",
foreignField: "_id",
as: "itemsCollection"
}
},
{
$project: {
username: 1,
items: {
$map: {
input: "$items",
as: "i",
in: {
$mergeObjects: [
"$$i",
{
$first: {
$filter: {
input: "$itemsCollection",
cond: { $eq: ["$$this._id", "$$i.itemId"] }
}
}
}
]
}
}
}
}
}
])
Playground
First I'd strongly suggest that you should store the items.itemId as ObjectId, not strings.
Then another simple solution can be:
db.players.aggregate([
{
$lookup: {
from: "items",
localField: "items.itemId",
foreignField: "_id",
as: "itemsDocuments",
},
},
{
$addFields: {
items: {
$map: {
input: { $zip: { inputs: ["$items", "$itemsDocuments"] } },
in: { $mergeObjects: "$$this" },
},
},
},
},
{ $unset: "itemsDocuments" },
])
I have two collection:
matches:
[{
date: "2020-02-15T17:00:00Z",
players: [
{_id: "5efd9485aba4e3d01942a2ce"},
{_id: "5efd9485aba4e3d01942a2cf"}
]
},
{...}]
and players:
[{
_id: "5efd9485aba4e3d01942a2ce",
name: "Rafa Nadal"
},
{
_id: "5efd9485aba4e3d01942a2ce",
name: "Roger Federer"
},
{...}]
I need to use lookup pipeline because I'm building a graphql resolver with recursive functions and I need nested lookup. I've followed this example https://docs.mongodb.com/datalake/reference/pipeline/lookup-stage#nested-example
My problem is that with pipeline lookup I need 11 seconds but with basic lookup only 0.67 seconds. And my test database is very short! about 1300 players and 700 matches.
This is my pipeline lookup (11 seconds to resolve)
db.collection('matches').aggregate([{
$lookup: {
from: 'players',
let: { ids: '$players' },
pipeline: [{ $match: { $expr: { $in: ['$_id', '$$ids' ] } } }],
as: 'players'
}
}]);
And this my basic lookup (0.67 seconds to resolve)
db.collection('matches').aggregate([{
$lookup: {
from: "players",
localField: "players",
foreignField: "_id",
as: "players"
}
}]);
Why so much difference? In what way can I do faster pipeline lookup?
The thing is that when you do a lookup using pipeline with a match stage, then the index would be used only for the fields that are matched with $eq operator and for the rest index will not be used.
And the example you specified with pipeline will work like this ( again index will not be used here as it is not $eq )
db.matches.aggregate([
{
$lookup: {
from: "players",
let: {
ids: {
$map: {
input: "$players",
in: "$$this._id"
}
}
},
pipeline: [
{
$match: {
$expr: {
$in: [
"$_id",
"$$ids"
]
}
}
}
],
as: "players"
}
}
])
As players is an array of object so it need to be mapped to array of ids first
MongoDB Playground
As #namar sood comments there are several tickets that refer to this issue:
https://jira.mongodb.org/browse/SERVER-37470
https://jira.mongodb.org/browse/SERVER-32549
Meanwhile a solution could be (also works nested):
db.collection('matches').aggregate([
{ $unwind: '$players' },
{
$lookup: {
from: 'players',
let: { id: '$players' },
pipeline: [{ $match: { $expr: { $eq: ['$_id', '$$id' ] } } }],
as: 'players'
},
{ $unwind: '$players' },
{
$group: {
"_id": "$_id",
"data": { "$first": "$$ROOT" },
"players": {$push: "$players"}
}
},
{ $addFields: {"data.players": "$players"} },
{ $replaceRoot: { newRoot: "$data" }}
]);
Below is the sample MongoDB Data Model for a user collection:
{
"_id": ObjectId('58842568c706f50f5c1de662'),
"userId": "123455",
"user_name":"Bob"
"interestedTags": [
"music",
"cricket",
"hiking",
"F1",
"Mobile",
"racing"
],
"listFriends": [
"123456",
"123457",
"123458"
]
}
listFriends is an array of userId for other users
For a particular userId I need to extract the listFriends (userId's) and for those userId's I need to aggregate the interestedTags and their count.
I would be able to achieve this by splitting the query into two parts:
1.) Extract the listFriends for a particular userId,
2.) Use this list in an aggregate() function, something like this
db.user.aggregate([
{ $match: { userId: { $in: [ "123456","123457","123458" ] } } },
{ $unwind: '$interestedTags' },
{ $group: { _id: '$interestedTags', countTags: { $sum : 1 } } }
])
I am trying to solve the question: Is there a way to achieve the above functionality (both steps 1 and 2) in a single aggregate function?
You could use $lookup to look for friend documents. This stage is usually used to join two different collection, but it can also do join upon one single collection, in your case I think it should be fine:
db.user.aggregate([{
$match: {
_id: 'user1',
}
}, {
$unwind: '$listFriends',
}, {
$lookup: {
from: 'user',
localField: 'listFriends',
foreignField: '_id',
as: 'friend',
}
}, {
$project: {
friend: {
$arrayElemAt: ['$friend', 0]
}
}
}, {
$unwind: '$friend.interestedTags'
}, {
$group: {
_id: '$friend.interestedTags',
count: {
$sum: 1
}
}
}]);
Note: I use $lookup and $arrayElemAt which are only available in Mongo 3.2 or newer version, so check your Mongo version before using this pipeline.