How to create view to read from two collections in mongoDB? - mongodb

Started with mongoDB syntax and use in project.
I am looking for a solution where I can combine more than two collections with couple of condition to create a view.
Here is my collection Range
/* 1 */
{
"_id" : ObjectId("1"),
"range" : {
"start" : "00"
},
"products" : [
{
"id" : "01",
"name" : "FirstProduct",
"type" : "First Type"
},
{
"id" : "02",
"name" : "Second Product",
"type" : "Second Type"
},
{
"id" : "03",
"name" : "Third Product",
"type" : "Third Type"
},
]
}
/* 2 */
{
"_id" : ObjectId("2"),
"range" : {
"start" : "100",
},
"products" : [
{
"id" : "01",
"name" : "First Product",
"type" : "First Type"
},
{
"id" : "02",
"name" : "Second Product",
"type" : "Second Type"
}
]
}
/* 3 */
{
"_id" : ObjectId("3"),
"range" : {
"start" : "500",
},
"products" : [
{
"id" : "01",
"name" : "First Product",
"type" : "First Type"
},
{
"id" : "02",
"name" : "Second Product",
"type" : "Second Type"
}
]
}
Second Collection. Stock
/* 1 */
{
"_id" : ObjectId("1"),
"range" : {
"start" : "00"
},
"products" : [
{
"id" : "01",
"expired" : false,
"returned" : false
},
{
"id" : "02",
"expired" : false,
"returned" : false
}
]
}
/* 2 */
{
"_id" : ObjectId("02"),
"range" : {
"start" : "100"
},
"products" : [
{
"id" : "01",
"expired" : true,
"returned" : true
},
{
"id" : "02",
"expired" : true,
"returned" : true
}
{
"id" : "03",
"expired" : true,
"returned" : true
}
]
}
Now want to have a view with combine result from above two collection above.
For each range document in Range collections
if Range.range.start = Stock.range.start
if Range.products.id = Stock.products.id
copy "expired" and "returned" field from Stock for that product and
add to Range.product
end if
end if
Return Range
So final result will something like below.
/* 1 */
{
"_id" : ObjectId("1"),
"range" : {
"start" : "00"
},
"products" : [
{
"id" : "01",
"name" : "FirstProduct",
"type" : "First Type"
"expired" : false,
"returned" : false
},
{
"id" : "02",
"name" : "Second Product",
"type" : "Second Type"
"expired" : false,
"returned" : false
}
]
}
/* 2 */
{
"_id" : ObjectId("2"),
"range" : {
"start" : "100",
},
"products" : [
{
"id" : "01",
"name" : "First Product",
"type" : "First Type",
"expired" : true,
"returned" : true
},
{
"id" : "02",
"name" : "Second Product",
"type" : "Second Type",
"expired" : true,
"returned" : true
}
]
}
/* 3 */
{
"_id" : ObjectId("3"),
"range" : {
"start" : "500",
},
"products" : [
{
"id" : "01",
"name" : "First Product",
"type" : "First Type"
},
{
"id" : "02",
"name" : "Second Product",
"type" : "Second Type"
}
]
}
I started with aggregate pipeline stages with fail to get right queries.
if anyone can help with right syntax and proper aggregate function.
Thanks in advance.

You need $lookup to merge the data from both collections but then you have to use $unwind to be able to match corresponding documents by product.id. In the last step you can use $group to get back an array:
db.Range.aggregate([
{
$lookup: {
from: "Stock",
localField: "range.start",
foreignField: "range.start",
as: "stock"
}
},
{
$unwind: "$stock"
},
{
$unwind: "$products"
},
{
$unwind: "$stock.products"
},
{
$match: { $expr: { $eq: [ "$products.id", "$stock.products.id" ] } }
},
{
$group: {
_id: "$_id",
"range": { $first: "$range" },
products: {
$push: {
id: "$products.id",
name: "$products.name",
type: "$products.type",
expired: "$stock.products.expired",
returned: "$stock.products.returned"
}
}
}
}
])
EDIT: Alternative solution which operates directly on arrays using $map and $filter below. The drawback is that the code is less readable but the good part is that it should return documents when there's no match and you should get better performance using this approach
db.Range.aggregate([
{
$lookup: {
from: "Stock",
localField: "range.start",
foreignField: "range.start",
as: "stock"
}
},
{
$unwind: "$stock"
},
{
$addFields: {
products: {
$map: {
input: "$products",
as: "p",
in: {
$let: {
vars: {
stockItem: {
$arrayElemAt: [
{ $filter: { input: "$stock.products", cond: { $eq: [ "$$p.id", "$$this.id" ] } } }, 0
]
}
},
in: {
$cond: [
{ $eq: [ "$$stockItem", undefined ] },
"$$p",
{
id: "$$p.id",
name: "$$p.name",
type: "$$p.type",
expired: "$$stockItem.expired",
returned: "$$stockItem.returned",
}
]
}
}
}
}
}
}
},
{
$project: {
stock: 0
}
}
])

Related

How to remove item in nested collection by id - mongoose

I want to remove a topic by id from the following collection. Here is the hierarchy of the collection - Area.
Area ---
|
---Subjects---
|
Courses---
|
Chapters----
|
Topics
The Area example is as following.
{
"_id" : ObjectId("607f7c67f64e993a5c9b9793"),
"name" : "Automatic Engineering",
"description" : "this is the description.",
"created" : ISODate("2021-04-21T01:14:15.736Z"),
"subjects" : [
{
"description" : "dddd",
"name" : "Transfer Function",
"_id" : ObjectId("607f7c7af64e993a5c9b9794"),
"created" : ISODate("2021-04-21T01:14:34.943Z"),
"courses" : [
{
"description" : "dddd",
"name" : "conception",
"_id" : ObjectId("607f7c9af64e993a5c9b9796"),
"created" : ISODate("2021-04-21T01:15:06.696Z"),
"chapters" : [
{
"description" : "chapter 1",
"name" : "Chapter 1",
"_id" : ObjectId("607f7cb4f64e993a5c9b9798"),
"created" : ISODate("2021-04-21T01:15:32.855Z"),
"topics" : [
{
"description" : "1",
"name" : "Topic 1",
"_id" : ObjectId("607f7ccef64e993a5c9b979b"),
"created" : ISODate("2021-04-21T01:15:58.064Z")
},
{
"description" : "2",
"name" : "Topic 2",
"_id" : ObjectId("607f7cd5f64e993a5c9b979c"),
"created" : ISODate("2021-04-21T01:16:05.663Z")
},
{
"description" : "3",
"name" : "Topic 3",
"_id" : ObjectId("607f7cdcf64e993a5c9b979d"),
"created" : ISODate("2021-04-21T01:16:12.336Z")
}
]
},
{
"description" : "chapter2",
"name" : "Chapter 2",
"_id" : ObjectId("607f7cbcf64e993a5c9b9799"),
"created" : ISODate("2021-04-21T01:15:40.231Z"),
"topics" : []
},
{
"description" : "chapter 3",
"name" : "Chapter 3",
"_id" : ObjectId("607f7cc5f64e993a5c9b979a"),
"created" : ISODate("2021-04-21T01:15:49.000Z"),
"topics" : []
}
]
},
{
"description" : "dddd",
"name" : "exercise",
"_id" : ObjectId("607f7ca6f64e993a5c9b9797"),
"created" : ISODate("2021-04-21T01:15:18.465Z"),
"chapters" : []
}
]
},
{
"description" : "this is the example",
"name" : "State Space Equation",
"_id" : ObjectId("607f7c8ef64e993a5c9b9795"),
"created" : ISODate("2021-04-21T01:14:54.056Z"),
"courses" : []
}
],
"__v" : 0
}
And I need to remove topic object in chapters array. So this is what I have done.
areaModel.findOneAndUpdate({
'_id': req.body.area_id,
'subjects._id': req.body.subject_id,
'courses._id': req.body.course_id,
'chapter._id': req.body.chapter_id
}, {
"$pull": {
"subjects.$.courses.$.chapters.$.topics": {
_id: req.body.topic_id
}
}
}, (err, result) =>{
if (err) {
return res.json({success: false, msg: err});
} else {
return res.json({success: true, msg:"successfully removed" });
}
});
By using $pull method of mongoose, I could remove the course from the area easily. But it was impossible for me to remove the more nested items like chapters and topics.
How can I solve this issue?
you can do so with the use of arrayFilters like so:
db.collection.updateOne(
{
_id: ObjectId("607f7c67f64e993a5c9b9793")
},
{
$pull: {
"subjects.$[s].courses.$[c].chapters.$[ch].topics": {
_id: ObjectId("607f7ccef64e993a5c9b979b")
}
}
},
{
arrayFilters: [
{
"s._id": { $eq: ObjectId("607f7c7af64e993a5c9b9794") }
},
{
"c._id": { $eq: ObjectId("607f7c9af64e993a5c9b9796") }
},
{
"ch._id": { $eq: ObjectId("607f7cb4f64e993a5c9b9798") }
}
]
})
https://mongoplayground.net/p/kVyV27nnkII
Try it:
areaModel.findOneAndUpdate(
{ _id: req.body.area_id },
{
$pull: {
"subjects.$[].courses.$[].chapters.$[].topics": {
_id: req.body.topic_id,
},
},
}
);

How to $lookup in nested array of objects

I have to two collections one is tours and other is destinations so in tours have i have an array of locations which has destination object with an id and that id is belongs to another destinations collection but the thing is i am not be able to lookup details of destination in the array of locations. every tried many query search here too. but not getting expected result.
Tours :
{
"_id" : ObjectId("5f3122f4d8d57e3b9650e5b4"),
"title" : "tour 1",
"locations" : [
{
"destination" : {
"id" : "5ec5ae9037ea99f20a79071a"
},
"services" : {
"hotel" : true
}
},
{
"destination" : {
"id" : "5ec5ae8e37ea99f20a78ef8c"
},
"services" : {
"hotel" : true
}
}
]
}
{
"_id" : ObjectId("5f2d65e68bc6e9155310d147"),
"title" : "tour 2",
"locations" : [
{
"destination" : {
"id" : "5ecf994435c3a6025d5bf126"
},
"services" : {
"hotel" : true
}
}
]
}
{
"_id" : ObjectId("5f2d66398bc6e9155310d161"),
"title" : "tour 3",
"locations" : [
{
"destination" : {
"id" : "5ec5ae8e37ea99f20a78ef8d"
},
"services" : {
"hotel" : true
}
}
]
}
Destinations :
{
"_id" : ObjectId("5ec5ae9037ea99f20a79071a"),
"name" : "dest 1",
"country" : "country name"
}
{
"_id" : ObjectId("5ec5ae8e37ea99f20a78ef8c"),
"name" : "dest 2",
"country" : "country name"
}
{
"_id" : ObjectId("5ec5ae8e37ea99f20a78ef8d"),
"name" : "dest 3",
"country" : "country name"
}
{
"_id" : ObjectId("5ecf994435c3a6025d5bf126"),
"name" : "dest 4",
"country" : "country name"
}
Expected result :
{
"_id" : ObjectId("5f3122f4d8d57e3b9650e5b4"),
"title" : "tour 1",
"locations" : [
{
"destination" : {
"id" : "5ec5ae9037ea99f20a79071a",
"name" : "dest 1",
"country" : "country name"
},
"services" : {
"hotel" : true
}
},
{
"destination" : {
"id" : "5ec5ae8e37ea99f20a78ef8c",
"name" : "dest 2",
"country" : "country name"
},
"services" : {
"hotel" : true
}
}
]
},
{
"_id" : ObjectId("5f2d65e68bc6e9155310d147"),
"title" : "tour 2",
"locations" : [
{
"destination" : {
"id" : "5ecf994435c3a6025d5bf126",
"name" : "dest 4",
"country" : "country name"
},
"services" : {
"hotel" : true
}
}
]
},
{
"_id" : ObjectId("5f2d66398bc6e9155310d161"),
"title" : "tour 3",
"locations" : [
{
"destination" : {
"id" : "5ec5ae8e37ea99f20a78ef8d",
"name" : "dest 3",
"country" : "country name"
},
"services" : {
"hotel" : true
}
}
]
}
Tried query :
db.tours.aggregate([
{
"$addFields": {
"locations": {
"$map": {
"input": "$locations",
"in": {
"$mergeObjects": [
"$$this",
{
"dest_oid": {
"$toObjectId": "$$this.destination.id"
}
}
]
}
}
}
}
},
{ "$unwind": "$locations" },
{ "$lookup": {
"from": "destinations",
"localField": "locations.dest_oid",
"foreignField": "_id",
"as": "locations.dest",
}},
{ "$unwind": "$locations.dest" },
{ "$group": {
"_id": "$_id",
"locations": { "$push": "$locations" }
}}
])
even i have tried this
MongoDB $lookup on nested document
Quick fixes,
$unwind locations array put first because need to convert id to object id
db.tours.aggregate([
{ $unwind: "$locations" },
you skip this part if you have already converted string id t object id
$addFields replace locations.destination.id to object id filtered your logic to short, here no need $map and $mergeObjects options
{
$addFields: {
"locations.destination.id": {
$toObjectId: "$locations.destination.id"
}
}
},
$lookup that you have already did, but change as locations.destination
{
$lookup: {
from: "destinations",
as: "locations.destination",
localField: "locations.destination.id",
foreignField: "_id"
}
},
$unwind locations.destination because its array and we need object
{
$unwind: {
path: "$locations.destination"
}
},
$group that you have already did, few changes, push first destination and services in locations and add first title
{
$group: {
_id: "$_id",
locations: { $push: "$locations" },
title: { $first: "$title" }
}
}
])
Playground: https://mongoplayground.net/p/yaTCij7NRUj
If you can convert/update the string destination.id in the Tours collection to ObjectId if they are not already. Following query should work.
Query:
db.Tours.aggregate([
{
$unwind: "$locations",
},
{
$lookup: {
from: "Destinations",
localField: "locations.destination.id",
foreignField: "_id",
as: "destination",
},
},
{
$project: {
title: "$title",
locations: {
destination: {
$arrayElemAt: ["$destination", 0],
},
services: "$locations.services",
},
},
},
{
$group: {
_id: "$_id",
title: {
$first: "$title",
},
locations: {
$push: "$locations",
},
},
},
]);
Playground Link

Problems aggregating MongoDB

I am having problems aggregating my Product Document in MongoDB.
My Product Document is:
{
"_id" : ObjectId("5d81171c2c69f45ef459e0af"),
"type" : "T-Shirt",
"name" : "Panda",
"description" : "Panda's are cool.",
"image" : ObjectId("5d81171c2c69f45ef459e0ad"),
"created_at" : ISODate("2019-09-17T18:25:48.026+01:00"),
"is_featured" : false,
"sizes" : [
"XS",
"S",
"M",
"L",
"XL"
],
"tags" : [ ],
"pricing" : {
"price" : 26,
"sale_price" : 8
},
"categories" : [
ObjectId("5d81171b2c69f45ef459e086"),
ObjectId("5d81171b2c69f45ef459e087")
],
"sku" : "5d81171c2c69f45ef459e0af"
},
And my Category Document is:
{
"_id" : ObjectId("5d81171b2c69f45ef459e087"),
"name" : "Art",
"description" : "These items are our artsy options.",
"created_at" : ISODate("2019-09-17T18:25:47.196+01:00")
},
My aim is to perform aggregation on the Product Document in order to count the number of items within each Category. So I have the Category "Art", I need to count the products are in the "Art" Category:
My current aggregate:
db.product.aggregate(
{ $unwind : "$categories" },
{
$group : {
"_id" : { "name" : "$name" },
"doc" : { $push : { "category" : "$categories" } },
}
},
{ $unwind : "$doc" },
{
$project : {
"_id" : 0,
"name" : "$name",
"category" : "$doc.category"
}
},
{
$group : {
"_id" : "$category",
"name": { "$first": "$name" },
"items_in_cat" : { $sum : 1 }
}
},
{ "$sort" : { "items_in_cat" : -1 } },
)
Which does actually work but not as I need:
{
"_id" : ObjectId("5d81171b2c69f45ef459e082"),
"name" : null, // Why is the name of the category no here?
"items_in_cat" : 4
},
As we can see the name is null. How can I aggregate the output to be:
{
"_id" : ObjectId("5d81171b2c69f45ef459e082"),
"name" : "Art",
"items_in_cat" : 4
},
We need to use $lookup to fetch the name from Category collection.
The following query can get us the expected output:
db.product.aggregate([
{
$unwind:"$categories"
},
{
$group:{
"_id":"$categories",
"items_in_cat":{
$sum:1
}
}
},
{
$lookup:{
"from":"category",
"let":{
"id":"$_id"
},
"pipeline":[
{
$match:{
$expr:{
$eq:["$_id","$$id"]
}
}
},
{
$project:{
"_id":0,
"name":1
}
}
],
"as":"categoryLookup"
}
},
{
$unwind:{
"path":"$categoryLookup",
"preserveNullAndEmptyArrays":true
}
},
{
$project:{
"_id":1,
"name":{
$ifNull:["$categoryLookup.name","NA"]
},
"items_in_cat":1
}
}
]).pretty()
Data set:
Collection: product
{
"_id" : ObjectId("5d81171c2c69f45ef459e0af"),
"type" : "T-Shirt",
"name" : "Panda",
"description" : "Panda's are cool.",
"image" : ObjectId("5d81171c2c69f45ef459e0ad"),
"created_at" : ISODate("2019-09-17T17:25:48.026Z"),
"is_featured" : false,
"sizes" : [
"XS",
"S",
"M",
"L",
"XL"
],
"tags" : [ ],
"pricing" : {
"price" : 26,
"sale_price" : 8
},
"categories" : [
ObjectId("5d81171b2c69f45ef459e086"),
ObjectId("5d81171b2c69f45ef459e087")
],
"sku" : "5d81171c2c69f45ef459e0af"
}
Collection: category
{
"_id" : ObjectId("5d81171b2c69f45ef459e086"),
"name" : "Art",
"description" : "These items are our artsy options.",
"created_at" : ISODate("2019-09-17T17:25:47.196Z")
}
{
"_id" : ObjectId("5d81171b2c69f45ef459e087"),
"name" : "Craft",
"description" : "These items are our artsy options.",
"created_at" : ISODate("2019-09-17T17:25:47.196Z")
}
Output:
{
"_id" : ObjectId("5d81171b2c69f45ef459e087"),
"items_in_cat" : 1,
"name" : "Craft"
}
{
"_id" : ObjectId("5d81171b2c69f45ef459e086"),
"items_in_cat" : 1,
"name" : "Art"
}

MongoDB creates array of arrays in $group $push instead of flat array

I am trying to group a set of documents after an $unwind operation. My documents look like this:
{
"_id" : ObjectId("5cdb5b5acadf5100019da2f4"),
"allowedLocations" : [
{
"type" : "country",
"value" : "world",
"label" : "World"
}
],
"disallowedLocations" : [
{
"type" : "country",
"value" : "CF",
"label" : "Central African Republic"
},
{
"type" : "country",
"value" : "CN",
"label" : "China"
}
],
}
{
"_id" : ObjectId("5cdb5b5acadf5100019da2f4"),
"allowedLocations" : [
{
"type" : "country",
"value" : "US",
"label" : "United States of America"
}
],
"disallowedLocations" : [
{
"type" : "country",
"value" : "CA",
"label" : "Canada"
},
{
"type" : "country",
"value" : "MX",
"label" : "Mexico"
}
],
}
I want to group them by _id and then concatenate the allowedLocations and disallowedLocations arrays into one. The group stage in my pipeline looks like this:
{
"$group" : {
"_id" : "$_id",
"allowedLocations" : {
"$push" : "$allowedLocations"
},
"disallowedLocations" : {
"$push" : "disallowedLocations"
}
}
}
The problem is, the result I get is not a document with both arrays concatenated, but an array of arrays, each element of the array being the array of each document:
{
"_id" : ObjectId("5cdb5b5acadf5100019da2f4"),
"allowedLocations" : [
[
{
"type" : "country",
"value" : "US",
"label" : "United States of America"
}
],
[
{
"type" : "country",
"value" : "world",
"label" : "World"
}
],
],
"disallowedLocations" : [
[
{
"type" : "country",
"value" : "CF",
"label" : "Central African Republic"
},
{
"type" : "country",
"value" : "CN",
"label" : "China"
}
],
[
{
"type" : "country",
"value" : "CA",
"label" : "Canada"
},
{
"type" : "country",
"value" : "MX",
"label" : "Mexico"
}
]
}
}
Is there a way to produce a flat array with only objects as elements? I also tried with $concatArrays before the push but that creates more arrays inside the arrays.
Two solutions here. You can either run $unwind on both arrays to get single allowed and disallowed location per document and then run your $group stage:
db.col.aggregate([
{
$unwind: "$allowedLocations"
},
{
$unwind: "$disallowedLocations"
},
{
"$group" : {
"_id" : "$_id",
"allowedLocations" : {
"$addToSet" : "$allowedLocations"
},
"disallowedLocations" : {
"$addToSet" : "$disallowedLocations"
}
}
}
])
or you can run your $group first and then use $reduce to flatten allowedLocations and disallowedLocations:
db.col.aggregate([
{
"$group" : {
"_id" : "$_id",
"allowedLocations" : {
"$push" : "$allowedLocations"
},
"disallowedLocations" : {
"$push" : "$disallowedLocations"
}
}
},
{
$project: {
_id: 1,
allowedLocations: {
$reduce: {
input: "$allowedLocations",
initialValue: [],
in: { $concatArrays: [ "$$value", "$$this" ] }
}
},
disallowedLocations: {
$reduce: {
input: "$disallowedLocations",
initialValue: [],
in: { $concatArrays: [ "$$value", "$$this" ] }
}
}
}
}
])

Match documents with their inner array element variables in MongoDB

I can't understand how to compare a document variable to another document variable. My goal is to match all Authors who have at least one book written in their mothertongue (native language).
However, after unwinding the books array, My $match: { mothertongue: "$bookLang"}} doesn't return return anything, eventhough they're the same in the $project stage.
Can you help me without javascript?
This is my current query:
db.author.aggregate([
{
$unwind: "$books"
},
{
$project: {
books: true,
mothertongue: true,
bookLang: "$books.lang"
}
},
{
$match: { mothertongue: "$bookLang"}
}
])
And here is a sample of the dataset
{
"_id" : ObjectId("5aa7b34a338571a7470be0eb"),
"fname" : "Minna",
"lname" : "Canth",
"mothertongue" : "Finnish",
"birthdate" : ISODate("1844-03-19T00:00:00Z"),
"deathdate" : ISODate("1897-05-12T00:00:00Z"),
"books" : [
{
"title" : "Anna Liisa",
"lang" : "Finnish",
"language" : "finnish",
"edition" : 1,
"cover" : "Hard",
"year" : 1895,
"categorytags" : [
"Finland"
],
"publisher" : [
{
"name" : "Tammi",
"pubId" : ObjectId("5aa7b34a338571a7470be0e4")
}
]
},
{
"title" : "The Burglary and The House of Roinila",
"lang" : "English (UK)",
"translator" : ObjectId("5aa7b34a338571a7470be0ee"),
"cover" : "Soft",
"year" : 2010,
"categorytags" : [
"Finland"
],
"publisher" : [
{
"name" : "Jonathan Cape",
"pubId" : ObjectId("5aa7b34a338571a7470be0e7")
}
]
},
{
"title" : "Anna Liisa 2 ed.",
"lang" : "Finnish",
"language" : "finnish",
"edition" : 2,
"cover" : "hard",
"year" : 1958,
"categorytags" : [
"Finland"
],
"publisher" : [
{
"name" : "Otava",
"pubId" : ObjectId("5aa7b34a338571a7470be0e9")
}
]
}
]
}
End goal. note I'm not interested in formatting just yet, just the filtering
{
"Author" : "Charles Bukowski",
"BooksInMothertongue" : [
"Love Is a Dog from Hell"
]
}
{
"Author" : "Minna Canth",
"BooksInMothertongue" : [
"Anna Liisa",
"Anna Liisa 2 ed."
]
}
...
Try this
db.author.aggregate([{
$match: {
books: {
$ne: []
}
}
},
{
$project: {
books: {
$filter: {
input: "$books",
as: "book",
cond: {
$eq: ["$$book.lang", "$mothertongue"]
}
}
},
fname: 1
}
}, {
$unwind: "$books"
},
{
$group: {
_id: "$_id",
Author: {
$first: '$fname'
},
BooksInMothertongue: {
$push: "$books.title"
}
}
}
])