How to $lookup in nested array of objects - mongodb

I have to two collections one is tours and other is destinations so in tours have i have an array of locations which has destination object with an id and that id is belongs to another destinations collection but the thing is i am not be able to lookup details of destination in the array of locations. every tried many query search here too. but not getting expected result.
Tours :
{
"_id" : ObjectId("5f3122f4d8d57e3b9650e5b4"),
"title" : "tour 1",
"locations" : [
{
"destination" : {
"id" : "5ec5ae9037ea99f20a79071a"
},
"services" : {
"hotel" : true
}
},
{
"destination" : {
"id" : "5ec5ae8e37ea99f20a78ef8c"
},
"services" : {
"hotel" : true
}
}
]
}
{
"_id" : ObjectId("5f2d65e68bc6e9155310d147"),
"title" : "tour 2",
"locations" : [
{
"destination" : {
"id" : "5ecf994435c3a6025d5bf126"
},
"services" : {
"hotel" : true
}
}
]
}
{
"_id" : ObjectId("5f2d66398bc6e9155310d161"),
"title" : "tour 3",
"locations" : [
{
"destination" : {
"id" : "5ec5ae8e37ea99f20a78ef8d"
},
"services" : {
"hotel" : true
}
}
]
}
Destinations :
{
"_id" : ObjectId("5ec5ae9037ea99f20a79071a"),
"name" : "dest 1",
"country" : "country name"
}
{
"_id" : ObjectId("5ec5ae8e37ea99f20a78ef8c"),
"name" : "dest 2",
"country" : "country name"
}
{
"_id" : ObjectId("5ec5ae8e37ea99f20a78ef8d"),
"name" : "dest 3",
"country" : "country name"
}
{
"_id" : ObjectId("5ecf994435c3a6025d5bf126"),
"name" : "dest 4",
"country" : "country name"
}
Expected result :
{
"_id" : ObjectId("5f3122f4d8d57e3b9650e5b4"),
"title" : "tour 1",
"locations" : [
{
"destination" : {
"id" : "5ec5ae9037ea99f20a79071a",
"name" : "dest 1",
"country" : "country name"
},
"services" : {
"hotel" : true
}
},
{
"destination" : {
"id" : "5ec5ae8e37ea99f20a78ef8c",
"name" : "dest 2",
"country" : "country name"
},
"services" : {
"hotel" : true
}
}
]
},
{
"_id" : ObjectId("5f2d65e68bc6e9155310d147"),
"title" : "tour 2",
"locations" : [
{
"destination" : {
"id" : "5ecf994435c3a6025d5bf126",
"name" : "dest 4",
"country" : "country name"
},
"services" : {
"hotel" : true
}
}
]
},
{
"_id" : ObjectId("5f2d66398bc6e9155310d161"),
"title" : "tour 3",
"locations" : [
{
"destination" : {
"id" : "5ec5ae8e37ea99f20a78ef8d",
"name" : "dest 3",
"country" : "country name"
},
"services" : {
"hotel" : true
}
}
]
}
Tried query :
db.tours.aggregate([
{
"$addFields": {
"locations": {
"$map": {
"input": "$locations",
"in": {
"$mergeObjects": [
"$$this",
{
"dest_oid": {
"$toObjectId": "$$this.destination.id"
}
}
]
}
}
}
}
},
{ "$unwind": "$locations" },
{ "$lookup": {
"from": "destinations",
"localField": "locations.dest_oid",
"foreignField": "_id",
"as": "locations.dest",
}},
{ "$unwind": "$locations.dest" },
{ "$group": {
"_id": "$_id",
"locations": { "$push": "$locations" }
}}
])
even i have tried this
MongoDB $lookup on nested document

Quick fixes,
$unwind locations array put first because need to convert id to object id
db.tours.aggregate([
{ $unwind: "$locations" },
you skip this part if you have already converted string id t object id
$addFields replace locations.destination.id to object id filtered your logic to short, here no need $map and $mergeObjects options
{
$addFields: {
"locations.destination.id": {
$toObjectId: "$locations.destination.id"
}
}
},
$lookup that you have already did, but change as locations.destination
{
$lookup: {
from: "destinations",
as: "locations.destination",
localField: "locations.destination.id",
foreignField: "_id"
}
},
$unwind locations.destination because its array and we need object
{
$unwind: {
path: "$locations.destination"
}
},
$group that you have already did, few changes, push first destination and services in locations and add first title
{
$group: {
_id: "$_id",
locations: { $push: "$locations" },
title: { $first: "$title" }
}
}
])
Playground: https://mongoplayground.net/p/yaTCij7NRUj

If you can convert/update the string destination.id in the Tours collection to ObjectId if they are not already. Following query should work.
Query:
db.Tours.aggregate([
{
$unwind: "$locations",
},
{
$lookup: {
from: "Destinations",
localField: "locations.destination.id",
foreignField: "_id",
as: "destination",
},
},
{
$project: {
title: "$title",
locations: {
destination: {
$arrayElemAt: ["$destination", 0],
},
services: "$locations.services",
},
},
},
{
$group: {
_id: "$_id",
title: {
$first: "$title",
},
locations: {
$push: "$locations",
},
},
},
]);
Playground Link

Related

Mongo merge results

Hello guys I have 3 collections that have dbref, where I wanted to mount a result similar to the one shown below but I couldn't get to something satifatorio using any aggregate can you indicate a better way?
Coolections:
data1 - Main data_reference1
referenced in data1 data_reference2
referenced in data_reference2
/* EXAMPLE RECORD data1 */
{
"_id" : ObjectId("604fafd443487df824aeca61"),
"name" : "doc1",
"reference_data1_list" : [
{
"data_reference" : ObjectId("604fb00643487df824aeca91"),
"user_add" : "USER01"
},
{
"data_reference" : ObjectId("604fb00d43487df824aeca9b"),
"user_add" : "USER01"
},
{
"data_reference" : ObjectId("604fb01743487df824aecaa7"),
"user_add" : "USER02"
}
]
}
/* EXAMPLE RECORDS data_reference1 */
/* 1 */
{
"_id" : ObjectId("604fb00643487df824aeca91"),
"reference_desc" : "Test Referenced Field 01",
"reference_data2" : ObjectId("604fb24743487df824aecd14")
}
/* 2 */
{
"_id" : ObjectId("604fb00d43487df824aeca9b"),
"reference_desc" : "Test Referenced Field 02",
"reference_data2" : ObjectId("604fb24743487df824aecd14")
}
/* 3 */
{
"_id" : ObjectId("604fb01743487df824aecaa7"),
"reference_desc" : "Test Referenced Field 03",
"reference_data2" : ObjectId("604fb25743487df824aecd27")
}
/* EXAMPLE RECORDS data_reference2 */
/* 1 */
{
"_id" : ObjectId("604fb24743487df824aecd14"),
"name" : "Student"
}
/* 2 */
{
"_id" : ObjectId("604fb25743487df824aecd27"),
"name" : "Theacher"
}
Desired result:
{
"_id": ObjectId("604fafd443487df824aeca61"),
"name": "doc1",
"reference_data1_list": [
{
"_id": ObjectId("604fb00643487df824aeca91"),
"reference_desc": "Test Referenced Field 01",
"reference_data2": {
"_id": ObjectId("604fb24743487df824aecd14"),
"name": "Student"
}
},
{
"_id": ObjectId("604fb00d43487df824aeca9b"),
"reference_desc": "Test Referenced Field 02",
"reference_data2": {
"_id": ObjectId("604fb24743487df824aecd14"),
"name": "Student"
}
},
{
"_id": "604fb01743487df824aecaa7",
"reference_desc": "Test Referenced Field 03",
"reference_data2": {
"_id": ObjectId("604fb25743487df824aecd27"),
"name": "Theacher"
}
}
]
}
I don't have much skill with the mongo and I didn't find any similar example to be able to use in the example.
Try this query:
db.data1.aggregate([
{ $unwind: "$reference_data1_list" },
{
$lookup: {
from: "data_reference1",
let: {
data_ref: "$reference_data1_list.data_reference",
user_add: "$reference_data1_list.user_add"
},
pipeline: [
{
$match: {
$expr: { $eq: ["$_id", "$$data_ref"] }
}
},
{
$addFields: { user_add: "$$user_add" }
}
],
as: "reference_data1_list"
}
},
{ $unwind: "$reference_data1_list" },
{
$lookup: {
from: "data_reference2",
localField: "reference_data1_list.reference_data2",
foreignField: "_id",
as: "reference_data1_list.reference_data2"
}
},
{ $unwind: "$reference_data1_list.reference_data2" },
{
$group: {
_id: "$_id",
name: { $first: "$name" },
reference_data1_list: { $push: "$reference_data1_list" }
}
}
]);
Output
{
"_id" : ObjectId("604fafd443487df824aeca61"),
"name" : "doc1",
"reference_data1_list" : [
{
"_id" : ObjectId("604fb00643487df824aeca91"),
"reference_desc" : "Test Referenced Field 01",
"reference_data2" : {
"_id" : ObjectId("604fb24743487df824aecd14"),
"name" : "Student"
},
"user_add" : "USER01"
},
{
"_id" : ObjectId("604fb00d43487df824aeca9b"),
"reference_desc" : "Test Referenced Field 02",
"reference_data2" : {
"_id" : ObjectId("604fb24743487df824aecd14"),
"name" : "Student"
},
"user_add" : "USER01"
},
{
"_id" : ObjectId("604fb01743487df824aecaa7"),
"reference_desc" : "Test Referenced Field 03",
"reference_data2" : {
"_id" : ObjectId("604fb25743487df824aecd27"),
"name" : "Theacher"
},
"user_add" : "USER02"
}
]
}

Mongo find query only returns one result

Hello I have the following data structure :
[
{
"name": "a name",
"project": [
{
companyName: "a name",
contactPerson: [
{
work_email: "test#test.com"
}
]
},
{
companyName: "a name1",
contactPerson: [
{
work_email: "test1#test.com"
}
]
},
{
companyName: "a name2",
contactPerson: [
{
work_email: "test2#test.com"
}
]
},
{
companyName: "a name3",
contactPerson: [
{
work_email: "test#test.com"
}
]
},
]
}
]
With this query i want to find all projects that have the email test#test.com :
db.collection.find({
"project.contactPerson.work_email": "test#test.com"
},
{
"project.$": 1
})
It only returns the first result it finds and then it just stops. but in my data i have two projects with that email and i want to find both. here's a playground you can use to further help me if you can. Thanks in advance and much appreciated : https://mongoplayground.net/p/4Mpp7kHi98u
The positional $ operator limits the contents of an to return either:
The first element that matches the query condition on the array.
The first element if no query condition is specified for the array
(Starting in MongoDB 4.4). Ref
You can do something like following,
[
{
"$unwind": "$project"
},
{
$addFields: {
"project.contactPerson": {
$filter: {
input: "$project.contactPerson",
cond: {
$eq: [
"$$this.work_email",
"test#test.com"
]
}
}
}
}
},
{
$match: {
$expr: {
$ne: [
"$project.contactPerson",
[]
]
}
}
},
{
$group: {
_id: "$_id",
name: {
$first: "$name"
},
project: {
"$addToSet": "$project"
}
}
}
]
Working Mongo playground
db.collection.aggregate([
{
$unwind: "$project"
},
{
$match: {
"project.contactPerson.work_email": "test#test.com"
}
},
{
"$group": {
"_id": "$_id",
"name": {
"$first": "$name"
},
"project": {
"$push": {
"companyName": "$project.companyName",
"contactPersion": "$project.contactPerson"
}
}
}
}
])
//step1(problem statement related find):, find shows all projects even one of the array element of contact is matched, use an aggregate function to display specific email ids
> db.test3.find({ "project.contact.email": "abc2#email.com" }).pretty();
{
"_id" : ObjectId("5f43fdc153e34ac6967fe8ce"),
"name" : "Pega Contractors",
"project" : [
{
"pname" : "pname1",
"contact" : [
{
"email" : "xyz1#email.com"
}
]
},
{
"pname" : "pname2",
"contact" : [
{
"email" : "abc2#email.com"
}
]
},
{
"pname" : "pname3",
"contact" : [
{
"email" : "xyz1#email.com"
}
]
}
]
}
--
//aggregate option:
//Step1: data preparation
> db.test3.find().pretty();
{
"_id" : ObjectId("5f43fdc153e34ac6967fe8ce"),
"name" : "Pega Contractors",
"project" : [
{
"pname" : "pname1",
"contact" : [
{
"email" : "xyz1#email.com"
}
]
},
{
"pname" : "pname2",
"contact" : [
{
"email" : "abc2#email.com"
}
]
},
{
"pname" : "pname3",
"contact" : [
{
"email" : "xyz1#email.com"
}
]
}
]
}
>
//step2: aggregate and unwind project for the next step pipeline input
> db.test3.aggregate([ {$unwind: "$project"}]);
{ "_id" : ObjectId("5f43fdc153e34ac6967fe8ce"), "name" : "Pega Contractors", "project" : { "pname" : "pname1", "contact" : [ { "email" : "xyz1#email.com" } ] } }
{ "_id" : ObjectId("5f43fdc153e34ac6967fe8ce"), "name" : "Pega Contractors", "project" : { "pname" : "pname2", "contact" : [ { "email" : "abc2#email.com" } ] } }
{ "_id" : ObjectId("5f43fdc153e34ac6967fe8ce"), "name" : "Pega Contractors", "project" : { "pname" : "pname3", "contact" : [ { "email" : "xyz1#email.com" } ] } }
//step3: Desired outcome, i.e display data specific to email
> db.test3.aggregate([
... {$unwind: "$project"},
... {$match: {"project.contact.email":"xyz1#email.com"}}
... ]);
{ "_id" : ObjectId("5f43fdc153e34ac6967fe8ce"), "name" : "Pega Contractors", "project" : { "pname" : "pname1", "contact" : [ { "email" : "xyz1#email.com" } ] } }
{ "_id" : ObjectId("5f43fdc153e34ac6967fe8ce"), "name" : "Pega Contractors", "project" : { "pname" : "pname3", "contact" : [ { "email" : "xyz1#email.com" } ] } }
> db.test3.aggregate([ {$unwind: "$project"}, {$match: {"project.contact.email":"acb2#email.com"}} ]);
> db.test3.aggregate([ {$unwind: "$project"}, {$match: {"project.contact.email":"abc2#email.com"}} ]);
{ "_id" : ObjectId("5f43fdc153e34ac6967fe8ce"), "name" : "Pega Contractors", "project" : { "pname" : "pname2", "contact" : [ { "email" : "abc2#email.com" } ] } }
>

How to create view to read from two collections in mongoDB?

Started with mongoDB syntax and use in project.
I am looking for a solution where I can combine more than two collections with couple of condition to create a view.
Here is my collection Range
/* 1 */
{
"_id" : ObjectId("1"),
"range" : {
"start" : "00"
},
"products" : [
{
"id" : "01",
"name" : "FirstProduct",
"type" : "First Type"
},
{
"id" : "02",
"name" : "Second Product",
"type" : "Second Type"
},
{
"id" : "03",
"name" : "Third Product",
"type" : "Third Type"
},
]
}
/* 2 */
{
"_id" : ObjectId("2"),
"range" : {
"start" : "100",
},
"products" : [
{
"id" : "01",
"name" : "First Product",
"type" : "First Type"
},
{
"id" : "02",
"name" : "Second Product",
"type" : "Second Type"
}
]
}
/* 3 */
{
"_id" : ObjectId("3"),
"range" : {
"start" : "500",
},
"products" : [
{
"id" : "01",
"name" : "First Product",
"type" : "First Type"
},
{
"id" : "02",
"name" : "Second Product",
"type" : "Second Type"
}
]
}
Second Collection. Stock
/* 1 */
{
"_id" : ObjectId("1"),
"range" : {
"start" : "00"
},
"products" : [
{
"id" : "01",
"expired" : false,
"returned" : false
},
{
"id" : "02",
"expired" : false,
"returned" : false
}
]
}
/* 2 */
{
"_id" : ObjectId("02"),
"range" : {
"start" : "100"
},
"products" : [
{
"id" : "01",
"expired" : true,
"returned" : true
},
{
"id" : "02",
"expired" : true,
"returned" : true
}
{
"id" : "03",
"expired" : true,
"returned" : true
}
]
}
Now want to have a view with combine result from above two collection above.
For each range document in Range collections
if Range.range.start = Stock.range.start
if Range.products.id = Stock.products.id
copy "expired" and "returned" field from Stock for that product and
add to Range.product
end if
end if
Return Range
So final result will something like below.
/* 1 */
{
"_id" : ObjectId("1"),
"range" : {
"start" : "00"
},
"products" : [
{
"id" : "01",
"name" : "FirstProduct",
"type" : "First Type"
"expired" : false,
"returned" : false
},
{
"id" : "02",
"name" : "Second Product",
"type" : "Second Type"
"expired" : false,
"returned" : false
}
]
}
/* 2 */
{
"_id" : ObjectId("2"),
"range" : {
"start" : "100",
},
"products" : [
{
"id" : "01",
"name" : "First Product",
"type" : "First Type",
"expired" : true,
"returned" : true
},
{
"id" : "02",
"name" : "Second Product",
"type" : "Second Type",
"expired" : true,
"returned" : true
}
]
}
/* 3 */
{
"_id" : ObjectId("3"),
"range" : {
"start" : "500",
},
"products" : [
{
"id" : "01",
"name" : "First Product",
"type" : "First Type"
},
{
"id" : "02",
"name" : "Second Product",
"type" : "Second Type"
}
]
}
I started with aggregate pipeline stages with fail to get right queries.
if anyone can help with right syntax and proper aggregate function.
Thanks in advance.
You need $lookup to merge the data from both collections but then you have to use $unwind to be able to match corresponding documents by product.id. In the last step you can use $group to get back an array:
db.Range.aggregate([
{
$lookup: {
from: "Stock",
localField: "range.start",
foreignField: "range.start",
as: "stock"
}
},
{
$unwind: "$stock"
},
{
$unwind: "$products"
},
{
$unwind: "$stock.products"
},
{
$match: { $expr: { $eq: [ "$products.id", "$stock.products.id" ] } }
},
{
$group: {
_id: "$_id",
"range": { $first: "$range" },
products: {
$push: {
id: "$products.id",
name: "$products.name",
type: "$products.type",
expired: "$stock.products.expired",
returned: "$stock.products.returned"
}
}
}
}
])
EDIT: Alternative solution which operates directly on arrays using $map and $filter below. The drawback is that the code is less readable but the good part is that it should return documents when there's no match and you should get better performance using this approach
db.Range.aggregate([
{
$lookup: {
from: "Stock",
localField: "range.start",
foreignField: "range.start",
as: "stock"
}
},
{
$unwind: "$stock"
},
{
$addFields: {
products: {
$map: {
input: "$products",
as: "p",
in: {
$let: {
vars: {
stockItem: {
$arrayElemAt: [
{ $filter: { input: "$stock.products", cond: { $eq: [ "$$p.id", "$$this.id" ] } } }, 0
]
}
},
in: {
$cond: [
{ $eq: [ "$$stockItem", undefined ] },
"$$p",
{
id: "$$p.id",
name: "$$p.name",
type: "$$p.type",
expired: "$$stockItem.expired",
returned: "$$stockItem.returned",
}
]
}
}
}
}
}
}
},
{
$project: {
stock: 0
}
}
])

Match documents with their inner array element variables in MongoDB

I can't understand how to compare a document variable to another document variable. My goal is to match all Authors who have at least one book written in their mothertongue (native language).
However, after unwinding the books array, My $match: { mothertongue: "$bookLang"}} doesn't return return anything, eventhough they're the same in the $project stage.
Can you help me without javascript?
This is my current query:
db.author.aggregate([
{
$unwind: "$books"
},
{
$project: {
books: true,
mothertongue: true,
bookLang: "$books.lang"
}
},
{
$match: { mothertongue: "$bookLang"}
}
])
And here is a sample of the dataset
{
"_id" : ObjectId("5aa7b34a338571a7470be0eb"),
"fname" : "Minna",
"lname" : "Canth",
"mothertongue" : "Finnish",
"birthdate" : ISODate("1844-03-19T00:00:00Z"),
"deathdate" : ISODate("1897-05-12T00:00:00Z"),
"books" : [
{
"title" : "Anna Liisa",
"lang" : "Finnish",
"language" : "finnish",
"edition" : 1,
"cover" : "Hard",
"year" : 1895,
"categorytags" : [
"Finland"
],
"publisher" : [
{
"name" : "Tammi",
"pubId" : ObjectId("5aa7b34a338571a7470be0e4")
}
]
},
{
"title" : "The Burglary and The House of Roinila",
"lang" : "English (UK)",
"translator" : ObjectId("5aa7b34a338571a7470be0ee"),
"cover" : "Soft",
"year" : 2010,
"categorytags" : [
"Finland"
],
"publisher" : [
{
"name" : "Jonathan Cape",
"pubId" : ObjectId("5aa7b34a338571a7470be0e7")
}
]
},
{
"title" : "Anna Liisa 2 ed.",
"lang" : "Finnish",
"language" : "finnish",
"edition" : 2,
"cover" : "hard",
"year" : 1958,
"categorytags" : [
"Finland"
],
"publisher" : [
{
"name" : "Otava",
"pubId" : ObjectId("5aa7b34a338571a7470be0e9")
}
]
}
]
}
End goal. note I'm not interested in formatting just yet, just the filtering
{
"Author" : "Charles Bukowski",
"BooksInMothertongue" : [
"Love Is a Dog from Hell"
]
}
{
"Author" : "Minna Canth",
"BooksInMothertongue" : [
"Anna Liisa",
"Anna Liisa 2 ed."
]
}
...
Try this
db.author.aggregate([{
$match: {
books: {
$ne: []
}
}
},
{
$project: {
books: {
$filter: {
input: "$books",
as: "book",
cond: {
$eq: ["$$book.lang", "$mothertongue"]
}
}
},
fname: 1
}
}, {
$unwind: "$books"
},
{
$group: {
_id: "$_id",
Author: {
$first: '$fname'
},
BooksInMothertongue: {
$push: "$books.title"
}
}
}
])

Issue retrieving subdocuments from MongoDB

I have the following dataset:
{
"_id" : ObjectId("59668a22734d1d48cf34de08"),
"name" : "Nobody Cares",
"menus" : [
{
"_id" : "menu_123",
"name" : "Weekend Menu",
"description" : "A menu for the weekend",
"groups" : [
{
"name" : "Spirits",
"has_mixers" : true,
"sizes" : [
"Single",
"Double"
],
"categories" : [
{
"name" : "Vodka",
"description" : "Maybe not necessary?",
"drinks" : [
{
"_id" : "drink_123",
"name" : "Absolut",
"description" : "Fancy ass vodka",
"sizes" : [
{
"_id" : "size_123",
"size" : "Single",
"price" : 300
}
]
}
]
}
]
}
],
"mixers" : [
{
"_id" : "mixer_1",
"name" : "Coca Cola",
"price" : 150
},
{
"_id" : "mixer_2",
"name" : "Lemonade",
"price" : 120
}
]
}
]
}
And I'm attempting to retrieve a single drink from that dataset, I'm using the following aggregate query:
db.getCollection('places').aggregate([
{ $match : {"menus.groups.categories.drinks._id" : "drink_123"} },
{ $unwind: "$menus" },
{ $project: { "_id": 1, "menus": { "groups": { "categories": { "drinks": { "name": 1 } } } } } }
])
However, it's returning the full structure of the dataset along with the correct data.
So instead of:
{
"_id": "drink_123",
"name": "Absolut"
}
I get:
{
"_id": ObjectId("59668a22734d1d48cf34de08"),
"menus": {
"groups": {
"categories": {
"drinks": { "name": "Absolut" }
}
}
}
}
For example. Any ideas how to just retrieve the subdocument?
If you need to retain the deeply nested model then this call will produce the desired output:
db.getCollection('places').aggregate([
{ $match : {"menus.groups.categories.drinks._id" : "drink_123"} },
{ $project: {"_id": '$menus.groups.categories.drinks._id', name: '$menus.groups.categories.drinks.name'}},
{ $unwind: "$name" },
{ $unwind: "$name" },
{ $unwind: "$name" },
{ $unwind: "$name" },
{ $unwind: "$_id" },
{ $unwind: "$_id" },
{ $unwind: "$_id" },
{ $unwind: "$_id" }
])
The numerous unwinds are the result of the deep nesting of the drinks subdocuments.
Though, FWIW, this sort of query does perhaps suggest that the model isn't 'read friendly'.