MongoDB nested lookup with 3 child levels - mongodb

I need to retrieve the entire single object hierarchy from the database as a JSON.
I'm trying aggregate for hours and can't solve how to do it with my data. So I have three collections:
form
{ "_id" : "1", "name" : "My first form" }
{ "_id" : "2", "name" : "Second one" }
{ "_id" : "3", "name" : "Another" }
question
{ "_id" : "q1", "form" : "1", "title": "What's your country?"}
{ "_id" : "q2", "form" : "1", "title": "What your favorite color?"}
{ "_id" : "q3", "form" : "1", "title": "Where do you live?"}
{ "_id" : "q4", "form" : "2", "title": "Where to go?"}
option
{ "_id" : "o1", "question" : "q1", "text" : "Brazil" }
{ "_id" : "o2", "question" : "q1", "text" : "EUA" }
{ "_id" : "o3", "question" : "q1", "text" : "China" }
{ "_id" : "o4", "question" : "q2", "text" : "Red" }
{ "_id" : "o5", "question" : "q2", "text" : "Blue" }
{ "_id" : "o6", "question" : "q2", "text" : "Green" }
I need to retrieve each form with all corresponding questions, and in each question the options for it. Like this:
[
{
_id:"q1",
name: "My first form",
questions: [
{ "_id" : "q1",
"form" : "1",
"title": "What's your country?",
"options": [
{ "_id" : "o1", "question" : "q1", "text" : "Brazil" }
{ "_id" : "o2", "question" : "q1", "text" : "EUA" },
{ "_id" : "o3", "question" : "q1", "text" : "China" }
]
},
{ "_id" : "q2",
"form" : "1",
"title": "What your favorite color",
"options": [
{ "_id" : "o4", "question" : "q2", "text" : "Red" }
{ "_id" : "o5", "question" : "q2", "text" : "Blue" },
{ "_id" : "o6", "question" : "q2", "text" : "Green" }
]
},
{ "_id" : "q3",
"form" : "1",
"title": "Where do you live?",
"options": []
}
]
},
...
]
I've tried a lot of $lookup, $unwind, another $lookup and $project, but nothing give-me that result (forms with questions inside, questions with options inside).
Please, help me! :)

I think the around is querying the question collection, looking up their questions and grouping by form, and finally lookup form and project, in that order.
This should do it. Keep in mind that _id in the output of this aggregate is the form _id.
db.question.aggregate([
{$match: {}},
{$lookup: {
from: 'option',
localField: '_id',
foreignField: 'question',
as: 'options'
}},
{$group: {
_id: "$form",
questions: {$push: {
title: "$title",
options: "$options",
form: "$form"
}}
}},
{$lookup: {
from: 'form',
localField: "_id",
foreignField: "_id",
as: 'form'
}},
{$project: {
name: {$arrayElemAt: ["$form.name", 0]},
questions: true
}}
]);
Actually.. this seems like a better alternative. It will return forms with no questions too.
db.form.aggregate([
{$match: {}},
{$lookup: {
from: 'question',
localField: '_id',
foreignField: 'form',
as: 'questions'
}},
{$unwind: {
path: "$questions",
preserveNullAndEmptyArrays: true
}},
{$lookup: {
from: 'option',
localField: 'questions._id',
foreignField: 'question',
as: 'options'
}},
{$group: {
_id: "$_id",
name: {$first: "$name"},
question: {$push: {
title: "$questions.title",
form: "$questions.form",
options: "$options"
}}
}}
])

Related

Unable to aggregate two collections using lookup in MongoDB Atlas

I have an orders collection that looks like this:
{
"_id" : "wJNEiSYwBd5ozGtLX",
"orderId" : 52713,
"createdAt" : ISODate("2020-01-31T04:34:13.790Z"),
"status" : "closed",
"orders" : [
{
"_id" : "ziPzwLuZrz9MNkaRT",
"productId" : 10290,
"quantity" : 2
}
]
}
I have an products collection that looks like this
{
"_id" : "238cwwLkZa6gKNN86",
"productId" : 10290,
"title" : "Product Title",
"price" : 9.9
}
I am trying to merge the price information into the orders information.
Something like:
{
"_id" : "wJNEiSYwBd5ozGtLX",
"orderId" : 52713,
"createdAt" : ISODate("2020-01-31T04:34:13.790Z"),
"status" : "closed",
"orders" : [
{
"_id" : "ziPzwLuZrz9MNkaRT",
"productId" : 10290,
"quantity" : 2,
"price": 9.9
}
]
}
If I try a $lookup command on MongoDB Atlas Dashboard like this:
{
from: 'products',
localField: 'orders.productId',
foreignField: 'productId',
as: 'priceInfo'
}
The aggregated output is (not what I wanted):
{
"_id" : "wJNEiSYwBd5ozGtLX",
"orderId" : 52713,
"createdAt" : ISODate("2020-01-31T04:34:13.790Z"),
"status" : "closed",
"orders" : [
{
"_id" : "ziPzwLuZrz9MNkaRT",
"productId" : 10290,
}
],
"priceInfo": [
{
"_id" : "238cwwLkZa6gKNN86",
"productId" : 10290,
"title" : "Product Title",
"price" : 9.9
}
]
}
I do not need a separate priceInfo array. It will be best if I have the product details information merged into the "orders" array. What should be the aggregation lookup syntax to achieve the desired output?
Demo - https://mongoplayground.net/p/bLqcN7tauWU
Read - $lookup $unwind $first $set $push $group
db.orders.aggregate([
{ $unwind: "$orders" }, // break array of orders into individual documents
{
$lookup: { // join
"from": "products",
"localField": "orders.productId",
"foreignField": "productId",
"as": "products"
}
},
{
$set: {
"orders.price": { "$arrayElemAt": [ "$products.price", 0 ] } // set the price
}
},
{
$group: { // group records back
_id: "$_id",
createdAt: { $first: "$createdAt" },
status: { $first: "$status" },
orderId: { $first: "$orderId" },
orders: { $push: "$orders" }
}
}
])

Why "as" in $lookup is replacing the complete set?

Let me first introduce you to the 2 collections I am using :
Collection 1 : users
> db.users.find().pretty()
{
"_id" : ObjectId("5ee4e727d04e4b4ac1ef115b"),
"name" : "Ashutosh Tiwari",
"age" : 21,
"email" : "ashutosh#gmail.com"
}
{
"_id" : ObjectId("5ee4e727d04e4b4ac1ef115c"),
"name" : "Maximilian",
"age" : 32,
"email" : "max#yahoo.com"
}
Collection 2 : posts
> db.posts.find().pretty()
{
"_id" : ObjectId("5ee51b7ed9f661cad505fcc6"),
"title" : "First One",
"text" : "Hey this is the first Author",
"author" : ObjectId("5ee4e727d04e4b4ac1ef115c"),
"comments" : [
{
"user" : ObjectId("5ee4e727d04e4b4ac1ef115b"),
"comment" : "This is my comment"
}
]
}
{
"_id" : ObjectId("5ee5353cd9f661cad505fcc8"),
"title" : "First One",
"author" : ObjectId("5ee4e727d04e4b4ac1ef115c"),
"comments" : [
{
"user" : ObjectId("5ee4e727d04e4b4ac1ef115b"),
"comment" : "This is my comment"
}
]
}
I want to have the user inside comments array in 2nd Collection(posts) to be replaced by the user who has written that comment.
I have tried the query below but it is replacing the comments section !
> db.posts.aggregate([
{ $lookup:
{from: "users",
localField:"comments.user",
foreignField:"_id",
as:"comments.user"
}
} ]).pretty()
{
"_id" : ObjectId("5ee51b7ed9f661cad505fcc6"),
"title" : "First One",
"text" : "Hey this is the first Author",
"author" : ObjectId("5ee4e727d04e4b4ac1ef115c"),
"comments" : {
"user" : [
{
"_id" : ObjectId("5ee4e727d04e4b4ac1ef115b"),
"name" : "Ashutosh Tiwari",
"age" : 21,
"email" : "ashutosh#gmail.com"
}
]
}
}
{
"_id" : ObjectId("5ee5353cd9f661cad505fcc8"),
"title" : "First One",
"author" : ObjectId("5ee4e727d04e4b4ac1ef115c"),
"comments" : {
"user" : [
{
"_id" : ObjectId("5ee4e727d04e4b4ac1ef115b"),
"name" : "Ashutosh Tiwari",
"age" : 21,
"email" : "ashutosh#gmail.com"
}
]
}
}
So, here, whole comments section is now replaced whereas I wanted to have the details in comments.user section so I could see the comment and the user who has posted that comment.
you need to unwind the comments array first
your query may look something like this
db.posts.aggregate([
{
$unwind: "$comments" // unwind the comments array to get a stream of documents, each document has only one comment
},
{
$lookup: {
from: "users",
localField: "comments.user",
foreignField: "_id",
as: "comments.user"
}
},
{
$unwind: "$comments.user" // we know there is only one user inside a single comment, so we can unwind this user array to be an object too (as the lookup returns an array)
},
{
$group: { // then do a group by the document _id to get unique documents with comments array instead of the same document duplicated with different comments
_id: "$_id",
author: {
$first: "$author"
},
text: {
$first: "$text"
},
title: {
$first: "$title"
},
comments: {
$push: "$comments"
}
}
}
])
you can test it here
hope it helps
You can handle it in the projection.
db.posts.aggregate([
{ $lookup:
{from: "users",
localField:"comments.user",
foreignField:"_id",
as:"cu"
}
},
{$unwind:{path:"$cu"}},
{
$project:{
"title":1,
"text":1,
"author":1,
"comments":{
user: "$cu",
comment: { $arrayElemAt: [ "$comments.comment", 0 ] },
}
}
}
])

How to select collection with id in array embed from other collection mongodb

I have two collection: photos and users. I want select list user with user_id in likes array embed from photos collection.
photos:
{
"_id" : ObjectId(""),
"title": "Title",
"likes" : [
{
"user_id" : ObjectId("")
},
{
"user_id" : ObjectId("")
}
],
}
users:
{
{
"_id" : ObjectId(""),
"name": "Name1",
"avatar": "Path1",
},
{
"_id" : ObjectId(""),
"name": "Name2",
"avatar": "Path2",
},
}
Output with paging:
{
{
"_id" : ObjectId(""),
"name": "Name1",
"avatar": "Path1",
},
{
"_id" : ObjectId(""),
"name": "Name2",
"avatar": "Path2",
},
}
I think this will help you...
db.getCollection('photos').aggregate([{ $lookup:
{
from: "users",
localField: "likes.user_id",
foreignField: "_id",
as: "likes"
}
}
])
The output is:
{
"_id" : ObjectId(""),
"title" : "Title",
"likes" : [
{
"_id" : ObjectId(""),
"name" : "Name1",
"avatar" : "Path1"
},
{
"_id" : ObjectId(""),
"name" : "Name2",
"avatar" : "Path2"
}
]
}

Aggregate by array field with object ids

Is there a better way of retrieving values from a collection2 based on an array field with object ids in collection1? I've tried to use $project but failed to get all required fields
Collections to aggregate:
collection1:
{
"_id" : ObjectId("5a58910de202796cfef41c6a"),
"sortOrder" : 5,
"title" : "Question 1 ?",
"freeTextIncluded" : false,
"freeText" : false,
"resultChart" : "pieChart",
"answer" : [
ObjectId("5a579fefd5554706b446cc71"),
ObjectId("5a587f17e4b2de0d683f96a4"),
ObjectId("5a587f20e4b2de0d683f96a5"),
ObjectId("5a587f29e4b2de0d683f96a6")
],
"state" : "active",
"__v" : 1,
"description" : ""
}
collection2:
{
"_id" : ObjectId("5a579fefd5554706b446cc71"),
"slug" : "answer-1",
"title" : "Answer 1",
"state" : "active",
"__v" : 0,
"author" : ObjectId("5a2e6b56e593c8525ced34b8"),
"body" : "<p>Lipsum...</p>"
}
{
"_id" : ObjectId("5a587f17e4b2de0d683f96a4"),
"slug" : "answer-2",
"title" : "Answer 2",
"state" : "active",
"__v" : 0,
"body" : ""
}
{
"_id" : ObjectId("5a587f20e4b2de0d683f96a5"),
"slug" : "answer-3",
"title" : "Answer 3",
"state" : "active",
"__v" : 0,
"body" : "",
"isCorrect" : true,
"sortOrder" : 3
}
{
"_id" : ObjectId("5a587f29e4b2de0d683f96a6"),
"slug" : "answer-4",
"title" : "Answer 4",
"state" : "active",
"__v" : 0,
"body" : ""
}
This aggregation works ok but I'm just wondering if there's a better/shorter way of aggregating 2 collections...
db.getCollection('questions').aggregate([
{
$match: {'_id': ObjectId('5a58910de202796cfef41c6a') }
},
{
$unwind: "$answer"
},
{
$lookup:
{
from: "answers",
localField: "answer",
foreignField: "_id",
as: "answers"
}
},
{
$match: { "answers": { $ne: [] }}
},
{
$unwind: "$answers"
},
{
$group: {
_id : ObjectId('5a58910de202796cfef41c6a'),
answerList: {$push: "$answers"},
title: {$first: "$title"},
sortOrder: {$first: "$sortOrder"},
description: {$first: "$description"},
resultChart: {$first: "$resultChart"},
freeTextIncluded: {$first: "$freeTextIncluded"},
}
}
]);
You need to improve your query like this:
db.getCollection('test').aggregate([{
$match: {
'_id': ObjectId('5a58910de202796cfef41c6a')
}
},
{
$lookup: {
from: "answers",
localField: "answer",
foreignField: "_id",
as: "answers"
}
},
{
$unwind: {
path: "$answers",
preserveNullAndEmptyArrays: true
}
},
{
$group: {
_id: ObjectId('5a58910de202796cfef41c6a'),
answerList: {
$push: "$answers"
},
title: {
$first: "$title"
},
sortOrder: {
$first: "$sortOrder"
},
description: {
$first: "$description"
},
resultChart: {
$first: "$resultChart"
},
freeTextIncluded: {
$first: "$freeTextIncluded"
},
}
}])

Mongodb Aggregate calculate average and add it to the document

I have websites which contains 2 documents:
{
"_id" : ObjectId("58503934034b512b419a6eab"),
"website" : "https://www.stackoverflow.com",
"name" : "Stack Exchange",
"keywords" : [
"helping",
"C#",
"PYTHON"
]
}
{
"_id" : ObjectId("58503934034b512b419a6eab"),
"website" : "https://www.google.com.com",
"name" : "Stack Exchange",
"keywords" : [
"search",
"engine",
]
}
I also have another seo_tracking which contains:
{
"_id" : ObjectId("587373d6f6325811c8a0b3ad"),
"position" : "2",
"real_url" : "https://www.stackoverflow.com",
"created_at" : ISODate("2017-01-09T11:28:22.104Z"),
"keyword" : "helping"
},
{
"_id" : ObjectId("587373d6f6325811c8a0b3ad"),
"position" : "4",
"real_url" : "https://www.stackoverflow.com",
"created_at" : ISODate("2017-01-09T11:28:22.104Z"),
"keyword" : "C#"
}
etc.. This contains around 100+ documents
What I want to do is is aggregate the seo_tracking with website on the specific URL (www.stackexchange (in websites) would match www.stackoverflow.com in (seo_tracking)) which I can do fine. However, I would like to return for each of the websites the following:
{
"_id" : ObjectId("587373d6f6325811c8a0b3ad"),
"website":"https://www.stackoverflow.com",
"avg_position" : "2"
}
Then for Google etc.. Even if the avg_position is 0 .. I have tried the following:
db.seo_tracking.aggregate([
{
$lookup:
{
from: "websites",
localField: "real_url",
foreignField: "website",
as: "post_websites"
},
},
{
"$group": {
_id:null,
avg_position:{$avg:"$position"}
}
}
])
However, this just produces:
{
"_id" : null,
"avg_position" : 2.0
}
What I need to do is have website and ideally also need the ID
Any ideas to where I'm going wrong here?
You can try something like this. You'll need to $unwind to access the fields from joined collection and change your grouping key to use the _id from joined collection to get average for each website:
db.seo_tracking.aggregate([{
$lookup: {
from: "website",
localField: "real_url",
foreignField: "website",
as: "post_websites"
},
}, {
$unwind: "$post_websites"
}, {
"$group": {
_id: "$post_websites._id",
avg_position: {
$avg: "$position"
},
website: {
$first: "$real_url"
}
}
}])