Nested array query does not work with Mongoose - mongodb

I have a mongo documnet of
{
"locationId": "5f2084f756dc2428e96fcda4",
"information": [{
"category": ["5e5150c6c52a3904b74d6ff7"],
"date": {
"$date": {
"$numberLong": "1601407317343"
}
},
"private": true
}, {
"category": ["5e5150c6c52a3904b74d6ff7"],
"date": {
"$date": {
"$numberLong": "1601407317343"
}
},
"private": false
},
}],
}
So far, what I have is to query the nested array.
const f = await info.aggregate([
{
$match: {
$and: [
{'locationId': '5f2084f756dc2428e96fcda4'},
{'information.private': true}
]
},
},
]);
I am trying to query information.private = true. However, I receive both 'private: false' and 'private: true'. The outcome is
[
{
"_id": ObjectId("5a934e000102030405000000"),
"information": [
{
"category": [
"5e5150c6c52a3904b74d6ff7"
],
"date": ISODate("2020-09-29T19:21:57.343Z"),
"private": true
},
{
"category": [
"5e5150c6c52a3904b74d6ff7"
],
"date": ISODate("2020-09-29T19:21:57.343Z"),
"private": false
}
],
"locationId": "5f2084f756dc2428e96fcda4"
}
]
I also tried with elemMatch and returns the same results. I'd looked up multiple answers on Stackoverflow but the dot notation and elemMatch do not work in this case. I know that I have done something wrong but I can't figure it out.

You can try $filter,
$match your conditions
$addFields to add field information and $filter to iterate loop of array and get match documents as per condition
const f = await info.aggregate([
{
$match: {
locationId: "5f2084f756dc2428e96fcda4",
"information.private": true
}
},
{
$addFields: {
information: {
$filter: {
input: "$information",
cond: { $eq: ["$$this.private", true] }
}
}
}
}
])
Playground
Other option you can try $redact,
$match your condition
$redact Restricts the contents of the documents based on information stored in the documents themselves.
const f = await info.aggregate([
{ $match: { locationId: "5f2084f756dc2428e96fcda4" } },
{
$redact: {
$cond: [{ $eq: ["$private", false] }, "$$PRUNE", "$$DESCEND"]
}
}
])
Playground
The find projection can accept aggregation expressions and syntax starting from MongoDB 4.4.
const f = await info.find([
locationId: "5f2084f756dc2428e96fcda4",
"information.private": true
},
{
locationId: 1,
information: {
$filter: {
input: "$information",
cond: { $eq: ["$$this.private", true] }
}
}
})
Playground

Related

Add Aggregate field in MongoDB pipeline depending on all elements of an array

Given the following documents in a collection:
[{
"_id": {
"$oid": "63f06283b80a395adf27780d"
},
"suppliers": [
{
"name": "S1",
"duesPaid": true
},
{
"name": "S2",
"duesPaid": true
}
]
},{
"_id": {
"$oid": "63f06283b80a395adf27780e"
},
"suppliers": [
{
"name": "S1",
"duesPaid": true
},
{
"name": "S2",
"duesPaid": false
}
]
}]
I would like to create an aggregateField in each document that does the following: If the suppliers array has at least 1 element and every element in that has the duesPaid field == true, then add a field to the document suppliersPaid = true. Otherwise add suppliersPaid = false. The resulting documents from the pipeline should look like this:
[{
"_id": {
"$oid": "63f06283b80a395adf27780d"
},
"suppliers": [
{
"name": "S1",
"duesPaid": true
},
{
"name": "S2",
"duesPaid": true
}
],
"suppliersPaid": true,
},{
"_id": {
"$oid": "63f06283b80a395adf27780e"
},
"suppliers": [
{
"name": "S1",
"duesPaid": true
},
{
"name": "S2",
"duesPaid": false
}
],
"suppliersPaid": false,
}]
I have tried the following pipeline:
[{$addFields: {
suppliersPaid: {
$and: [
{ $gte: [{ $size: "$suppliers" }, 1] },
{
suppliers: {
$not: {
$elemMatch: { duesPaid: false },
},
},
},
],
},
}}]
and I get the following error: Invalid $addFields :: caused by :: Unrecognized expression '$elemMatch'
I've tried to eliminate the reliance on $elemMatch per the docs https://www.mongodb.com/docs/manual/reference/operator/query/elemMatch/#single-query-condition as such:
[{$addFields: {
suppliersPaid: {
$and: [
{ $gte: [{ $size: "$suppliers" }, 1] },
{
suppliers: {
$not: {
duesPaid: false
},
},
},
],
},
}}]
But this yields the incorrect result of setting suppliersPaid to true for both documents, which is incorrect.
Note: I would like to avoid using any sort of JS in this code i.e. no $where operators.
For the second condition:
$eq - Compare the result from 1.1 to return an empty array.
1.1. $filter - Filter the documents from suppliers containing { duesPaid: false }.
db.collection.aggregate([
{
$addFields: {
suppliersPaid: {
$and: [
{
$gte: [
{
$size: "$suppliers"
},
1
]
},
{
$eq: [
{
$filter: {
input: "$suppliers",
cond: {
$eq: [
"$$this.duesPaid",
false
]
}
}
},
[]
]
}
]
}
}
}
])
Demo # Mongo Playground

Filter result in mongo sub-sub array

I have some collections such us this:
[{
"_id": ObjectId("604f3ae3194f2135b0ade569"),
"parameters": [
{
"_id": ObjectId("602b7455f4b4bf5b41662ec1"),
"name": "Purpose",
"options": [
{
"id": ObjectId("602b764ff4b4bf5b41662ec2"),
"name": "debug",
"sel": false
},
{
"id": ObjectId("602b767df4b4bf5b41662ec3"),
"name": "performance",
"sel": false
},
{
"id": ObjectId("602b764ff4b4bf5b41662ec4"),
"name": "security",
"sel": false
},
{
"id": ObjectId("602b767df4b4bf5b41662ec5"),
"name": "Not Applicable",
"sel": false
}
],
"type": "multiple"
},
{
"_id": ObjectId("602b79d35d4a1333b8b6e5ba"),
"name": "Organization",
"options": [
{
"id": ObjectId("602b79d353c89933b8238325"),
"name": "SW",
"sel": false
},
{
"id": ObjectId("602b79d353c89933b8238326"),
"name": "HW",
"sel": false
}
],
"type": "multiple"
}
]
}]
The parameters are most 30.
I need to implements in mongo a "filter" collections.
If I filter one or more parameters._id, mongo return:
collection _id that have match options.sel of this parameters._id
collection _id that have all options.sel equal to false of this parameters._id
non return collection _id if parameters._id has set up options.name:"Not Applicable" at value options.sel:true
For example, if I match parameters._id:ObjectId("602b7455f4b4bf5b41662ec1") and this parameters.options.id:ObjectId("602b764ff4b4bf5b41662ec2"), I expect:
not collection _id that has, for parameters._id:ObjectId("602b7455f4b4bf5b41662ec1"), the specific parameters.options.id: ObjectId("602b767df4b4bf5b41662ec5") at value options.sel:true
all collection _id that match with query
all collection _id that has, for parameters._id:ObjectId("602b7455f4b4bf5b41662ec1"), all specific parameters.options.sel:false
Next I need to make this rule for more parameters.
I have think to implements three aggregation for every rule...
Do you have suggestion?
Try this query:
db.testCollection.aggregate([
{ $unwind: "$parameters" },
{
$match: {
"parameters._id": ObjectId("602b7455f4b4bf5b41662ec1"),
"parameters.options.id": ObjectId("602b767df4b4bf5b41662ec5")
}
},
{
$addFields: {
options: {
$filter: {
input: "$parameters.options",
as: "option",
cond: {
$and: [
{ $ne: ["$$option.sel", true] },
{ $ne: ["$$option.name", "Not Applicable"] }
]
}
}
}
}
}
])
With idea of #dheemanth-bath,
I have make this query:
db.testCollection.aggregate([
{ $unwind: "$parameters" },
{
$match: {
"parameters._id": ObjectId("602b7455f4b4bf5b41662ec1"),
}
},
{
$addFields: {
match: {
$filter: {
input: "$parameters.options",
as: "option",
cond: {
$and: [
{ $eq: ["$$option.id", ObjectId('602b767df4b4bf5b41662ec5')] },
{ $eq: ["$$option.sel", true] }
]
}
}
},
notDeclared: {
$filter: {
input: "$parameters.options",
as: "option",
cond: {
$and: [
{ $eq: ["$$option.name", "Not Applicable"]},
{ $eq: ["$$option.sel", true] }
]
}
}
}
}
}
])
The idea is that: after query count number of element of notDeclared. If > 0 waste the collection. Otherwise evaluate number of match, if is >0, there is at least one elements can match.
Good. But how I evaluate if all elements of options.sel are false?
And if I check another parameters, I need to make another aggregation (one for parameters)?

mongo $filter and multiple condition in sub-sub array

I have this portion of collections:
[{
"_id": ObjectId("604f3ae3194f2135b0ade569"),
"parameters": [
{
"_id": ObjectId("602b7455f4b4bf5b41662ec1"),
"name": "Purpose",
"options": [
{
"id": ObjectId("602b764ff4b4bf5b41662ec2"),
"name": "debug",
"sel": true
},
{
"id": ObjectId("602b767df4b4bf5b41662ec3"),
"name": "performance",
"sel": false
},
{
"id": ObjectId("602b764ff4b4bf5b41662ec4"),
"name": "security",
"sel": true
},
{
"id": ObjectId("602b767df4b4bf5b41662ec5"),
"name": "Not Applicable",
"sel": false
}
],
"type": "multiple"
}]
}]
And this query:
db.testCollection.aggregate([
{ $unwind: "$parameters" },
{
$match: {
"parameters._id": ObjectId("602b7455f4b4bf5b41662ec1"),
}
},
{
$addFields: {
match: {
$filter: {
input: "$parameters.options",
as: "option",
cond: {
$and: [
{ $eq: ["$$option.id", ObjectId('602b764ff4b4bf5b41662ec2')] },
{ $eq: ["$$option.sel", true] }
]
}
}
}
}
}
])
The result is a new array match with, in this case,
"match": [
{
"id": ObjectId("602b764ff4b4bf5b41662ec2"),
"name": "debug",
"sel": true
}
]
How I can match, for collections, match for example
"id": ObjectId("602b764ff4b4bf5b41662ec2") and
"id": ObjectId("602b764ff4b4bf5b41662ec4")
and have a result array with or two results or empty?
I am not sure it is possible in single condition inside $filter, but you can try with $let,
$let to declare a variable called filtered and store your filtered result
$in to check condition if filtered element size is 2 then return filtered result otherwise blank array
You have to put the number that you are matching number of ids in filter in $eq condition
{
$addFields: {
match: {
$let: {
vars: {
filtered: {
$filter: {
input: "$parameters.options",
as: "option",
cond: {
$and: [
{
$in: [
"$$option.id",
[ObjectId("602b764ff4b4bf5b41662ec2"), ObjectId("602b767df4b4bf5b41662ec3")]
]
},
{ $eq: ["$$option.sel", true] }
]
}
}
}
},
in: {
$cond: [
{ $eq: [{ $size: "$$filtered" }, 2] },
"$$filtered",
[]
]
}
}
}
}
}
Playground

Mongo db not in query by having two subset of documents from same collection

I am new to mongodb. Assume the following. There are 3 types of documents in one collection x, y and z.
docs = [{
"item_id": 1
"type": "x"
},
{
"item_id": 2
"type": "x"
},{
"item_id": 3
"type": "y",
"relavent_item_ids": [1, 2]
},
{
"item_id": 3
"type": "y",
"relavent_item_ids": [1, 2, 3]
},{
"item_id": 4
"type": "z",
}]
I want to get the following.
Ignore the documents with type z
Get all the documents of type x where it's item_id is not in relavent_item_ids of type y documents.
The result should have item_id field.
I tried doing match $in but this returns me all the records, I am unable to figure out how to have in condition with subset of documents of type y.
You can use below query
const item_ids = (await db.collection.find({ "type": "y" })).map(({ relavent_item_ids }) => relavent_item_ids)
const result = db.collection.find({
"item_id": { "$exists": true },
"type": { "$ne": "z", "$eq": "x" },
"relavent_item_ids": { "$nin": item_ids }
})
console.log({ result })
Ignore the documents with type z --> Use $ne not equal to query operator to filter out z types.
Get all the documents of type x where it's item_id is not in relavent_item_ids of type y documents --> Use $expr to match the same documents fields.
The result should have item_id field --> Use $exists query operator.
The solution:
db.test.aggregate( [
{
$facet: {
firstQuery: [
{
$match: { type: { $eq: "x", $ne: "z" } }
},
{
$project: {
item_id : 1, _id: 0
}
}
],
secondQuery: [
{
$match: { type: "y" }
},
{
$group: {
_id: null,
relavent: { $push: "$relavent_item_ids" }
}
},
{
$project: {
relavent: {
$reduce: {
input: "$relavent",
initialValue: [ ],
in: { $setUnion: [ "$$value", "$$this" ] }
}
}
}
}
]
}
},
{
$addFields: { secondQuery: { $arrayElemAt: [ "$secondQuery", 0 ] } }
},
{
$project: {
result: {
$filter: {
input: "$firstQuery" ,
as: "e",
cond: { $not: [ { $in: [ "$$e.item_id", "$secondQuery.relavent" ] } ] }
}
}
}
},
] )
Using the input documents in the question post and adding one more following document to the collection:
{
"item_id": 11,
"type": "x",
}
: only this document's item_id (value 11) will show in the output.
The aggregation uses a $facet to make two individual queries with a single pass. The first query gets all the "x" types (and ignores type "z") as an array. The second query gets an array of relavent_item_ids with unique values (from the documents of type "y"). The final, $project stage filters the first query result array with the condition:
Get all the documents of type x where it's item_id is not in
relavent_item_ids of type y documents
I am not sure if its an elegant solution.
db.getCollection('test').aggregate([
{
"$unwind": {
"path": "$relavent_item_ids",
"preserveNullAndEmptyArrays": true
}
},
{
"$group": {
"_id":null,
"relavent_item_ids": {"$addToSet":"$relavent_item_ids"},
"other_ids": {
"$addToSet":{
"$cond":[
{"$eq":["$type", "x"]},
"$item_id",
null
]
}
}
}
},
{
"$project":{
"includeIds": {"$setDifference":["$other_ids", "$relavent_item_ids"]}
}
},
{
"$unwind": "$includeIds"
},
{
"$match": {"includeIds":{"$ne":null}}
},
{
"$lookup":{
"from": "test",
"let": { "includeIds": "$includeIds"},
"pipeline": [
{ "$match":
{ "$expr":
{ "$and":
[
{ "$eq": [ "$item_id", "$$includeIds" ] },
{ "$eq": [ "$type", "x" ] }
]
}
}
}
],
"as": "result"
}
},
{
"$unwind": "$result"
},
])

MongoDB Aggregation - Lookup pipeline not returning any documents

I'm having hard time getting $lookup with a pipeline to work in MongoDB Compass.
I have the following collections:
Toys
Data
[
{
"_id": {
"$oid": "5d233c3bb173a546386c59bb"
},
"type": "multiple",
"tags": [
""
],
"searchFields": [
"Jungle Stampers - Two",
""
],
"items": [
{
"$oid": "5d233c3cb173a546386c59bd"
},
{
"$oid": "5d233c3cb173a546386c59be"
},
{
"$oid": "5d233c3cb173a546386c59bf"
},
{
"$oid": "5d233c3cb173a546386c59c0"
},
{
"$oid": "5d233c3cb173a546386c59c1"
},
{
"$oid": "5d233c3cb173a546386c59c2"
},
{
"$oid": "5d233c3cb173a546386c59c3"
},
{
"$oid": "5d233c3cb173a546386c59c4"
}
],
"name": "Jungle Stampers - Two",
"description": "",
"status": "active",
"category": {
"$oid": "5cfe727cac920000086b880e"
},
"subCategory": "Stamp Sets",
"make": "",
"defaultCharge": null,
"defaultOverdue": null,
"sizeCategory": {
"$oid": "5d0cfde57561e107c88fbde3"
},
"ageFrom": {
"$numberInt": "24"
},
"ageTo": {
"$numberInt": "120"
},
"images": [
{
"_id": {
"$oid": "5d233c3bb173a546386c59bc"
},
"id": {
"$oid": "5d233c39b173a546386c59ba"
},
"url": "/toyimages/5d233c39b173a546386c59ba.jpg",
"thumbUrl": "/toyimages/thumbs/tn_5d233c39b173a546386c59ba.jpg"
}
],
"__v": {
"$numberInt": "2"
}
}
]
Loans
Data
[
{
"_id": {
"$oid": "5e1f1661b712215978c746d9"
},
"tags": [],
"member": {
"$oid": "5e17495e4f81ab3f900dbb63"
},
"source": "admin portal - potter1#gmail.com",
"items": [
{
"id": {
"$oid": "5e1f160eb712215978c746d5"
},
"status": "new",
"_id": {
"$oid": "5e1f1661b712215978c746db"
},
"toy": {
"$oid": "5d233c3bb173a546386c59bb"
},
"cost": {
"$numberInt": "0"
}
},
{
"id": {
"$oid": "5e1f160eb712215978c746d5"
},
"status": "new",
"_id": {
"$oid": "5e1f1661b712215978c746da"
},
"toy": {
"$oid": "5d233b1ab173a546386c59b5"
},
"cost": {
"$numberInt": "0"
}
}
],
"dateEntered": {
"$date": {
"$numberLong": "1579095632870"
}
},
"dateDue": {
"$date": {
"$numberLong": "1579651200000"
}
},
"__v": {
"$numberInt": "0"
}
}
]
I am trying to return a list of toys and their associated loans that have a status of 'new' or 'out'.
I can use the following $lookup aggregate to fetch all loans:
{
from: 'loans',
localField: '_id',
foreignField: 'items.toy',
as: 'loansSimple'
}
However I am trying to use a pipeline to load loans that have the two statuses I am interested in, but it always only returns zero documents:
{
from: 'loans',
let: {
'toyid': '$_id'
},
pipeline: [
{
$match: {
$expr: {
$and: [
{$eq: ['$items.toy', '$$toyid']},
{$eq: ['$items.status', 'new']} // changed from $in to $eq for simplicity
]
}
}
}
],
as: 'loans'
}
This always seems to return 0 documents, however I arrange it:
Have I made a mistake somewhere?
I'm using MongoDB Atlas, v4.2.2, MongoDB Compass v 1.20.4
You are trying to search $$toyid inside inner array, but Operator Expression $eq cannot resolve it.
Best solution: $let (returns filtered loans by criteria) + $filter (applies filter for inner array) operator helps us to get desired result.
db.toys.aggregate([
{
$lookup: {
from: "loans",
let: {
"toyid": "$_id",
"toystatus": "new"
},
pipeline: [
{
$match: {
$expr: {
$gt: [
{
$size: {
$let: {
vars: {
item: {
$filter: {
input: "$items",
as: "tmp",
cond: {
$and: [
{
$eq: [
"$$tmp.toy",
"$$toyid"
]
},
{
$eq: [
"$$tmp.status",
"$$toystatus"
]
}
]
}
}
}
},
in: "$$item"
}
}
},
0
]
}
}
}
],
as: "loans"
}
}
])
MongoPlayground
Alternative solution 1. Use $unwind to flatten items attribute. (We create extra field named tmp which stores items value, flatten it with $unwind operator, match as you were doing and then exclude from result)
db.toys.aggregate([
{
$lookup: {
from: "loans",
let: {
"toyid": "$_id"
},
pipeline: [
{
$addFields: {
tmp: "$items"
}
},
{
$unwind: "$tmp"
},
{
$match: {
$expr: {
$and: [
{
$eq: [
"$tmp.toy",
"$$toyid"
]
},
{
$eq: [
"$tmp.status",
"new"
]
}
]
}
}
},
{
$project: {
tmp: 0
}
}
],
as: "loans"
}
}
])
MongoPlayground
Alternative solution 2. We use $reduce to create toy's array and with $in operator we check if toyid exists inside this array.
db.toys.aggregate([
{
$lookup: {
from: "loans",
let: {
"toyid": "$_id"
},
pipeline: [
{
$addFields: {
toys: {
$reduce: {
input: "$items",
initialValue: [],
in: {
$concatArrays: [
"$$value",
[
"$$this.toy"
]
]
}
}
}
}
},
{
$match: {
$expr: {
$in: [
"$$toyid",
"$toys"
]
}
}
},
{
$project: {
toys: 0
}
}
],
as: "loans"
}
}
])
$expr receives aggregation expressions, At that point $$items.toy is parsed for each element in an array as you would expect (however if it would it will still give you "bad" results as you'll get loans that have the required toy id and any other item with status new in their items array).
So you have two options to work around this:
If you don't care about the other items in the lookup'd document you can add an $unwind stage at the start of the lookup pipeline like so:
{
from: 'loans',
let: {
'toyid': '$_id'
},
pipeline: [
{
$unwind: "$items"
},
{
$match: {
$expr: {
$and: [
{$eq: ['$items.toy', '$$toyid']},
{$eq: ['$items.status', 'new']} // changed from $in to $eq for simplicity
]
}
}
}
],
as: 'loans'
}
If you do care about them just iterate the array in one of the possible ways to get a 'correct' match, here is an example using $filter
{
from: 'loads',
let: {
'toyid': '$_id'
},
pipeline: [
{
$addFields: {
temp: {
$filter: {
input: "$items",
as: "item",
cond: {
$and: [
{$eq: ["$$item.toy", "$$toyid"]},
{$eq: ["$$item.status", "new"]}
]
}
}
}
}
}, {$match: {"temp.0": {exists: true}}}
],
as: 'loans'
}