Filter result in mongo sub-sub array - mongodb

I have some collections such us this:
[{
"_id": ObjectId("604f3ae3194f2135b0ade569"),
"parameters": [
{
"_id": ObjectId("602b7455f4b4bf5b41662ec1"),
"name": "Purpose",
"options": [
{
"id": ObjectId("602b764ff4b4bf5b41662ec2"),
"name": "debug",
"sel": false
},
{
"id": ObjectId("602b767df4b4bf5b41662ec3"),
"name": "performance",
"sel": false
},
{
"id": ObjectId("602b764ff4b4bf5b41662ec4"),
"name": "security",
"sel": false
},
{
"id": ObjectId("602b767df4b4bf5b41662ec5"),
"name": "Not Applicable",
"sel": false
}
],
"type": "multiple"
},
{
"_id": ObjectId("602b79d35d4a1333b8b6e5ba"),
"name": "Organization",
"options": [
{
"id": ObjectId("602b79d353c89933b8238325"),
"name": "SW",
"sel": false
},
{
"id": ObjectId("602b79d353c89933b8238326"),
"name": "HW",
"sel": false
}
],
"type": "multiple"
}
]
}]
The parameters are most 30.
I need to implements in mongo a "filter" collections.
If I filter one or more parameters._id, mongo return:
collection _id that have match options.sel of this parameters._id
collection _id that have all options.sel equal to false of this parameters._id
non return collection _id if parameters._id has set up options.name:"Not Applicable" at value options.sel:true
For example, if I match parameters._id:ObjectId("602b7455f4b4bf5b41662ec1") and this parameters.options.id:ObjectId("602b764ff4b4bf5b41662ec2"), I expect:
not collection _id that has, for parameters._id:ObjectId("602b7455f4b4bf5b41662ec1"), the specific parameters.options.id: ObjectId("602b767df4b4bf5b41662ec5") at value options.sel:true
all collection _id that match with query
all collection _id that has, for parameters._id:ObjectId("602b7455f4b4bf5b41662ec1"), all specific parameters.options.sel:false
Next I need to make this rule for more parameters.
I have think to implements three aggregation for every rule...
Do you have suggestion?

Try this query:
db.testCollection.aggregate([
{ $unwind: "$parameters" },
{
$match: {
"parameters._id": ObjectId("602b7455f4b4bf5b41662ec1"),
"parameters.options.id": ObjectId("602b767df4b4bf5b41662ec5")
}
},
{
$addFields: {
options: {
$filter: {
input: "$parameters.options",
as: "option",
cond: {
$and: [
{ $ne: ["$$option.sel", true] },
{ $ne: ["$$option.name", "Not Applicable"] }
]
}
}
}
}
}
])

With idea of #dheemanth-bath,
I have make this query:
db.testCollection.aggregate([
{ $unwind: "$parameters" },
{
$match: {
"parameters._id": ObjectId("602b7455f4b4bf5b41662ec1"),
}
},
{
$addFields: {
match: {
$filter: {
input: "$parameters.options",
as: "option",
cond: {
$and: [
{ $eq: ["$$option.id", ObjectId('602b767df4b4bf5b41662ec5')] },
{ $eq: ["$$option.sel", true] }
]
}
}
},
notDeclared: {
$filter: {
input: "$parameters.options",
as: "option",
cond: {
$and: [
{ $eq: ["$$option.name", "Not Applicable"]},
{ $eq: ["$$option.sel", true] }
]
}
}
}
}
}
])
The idea is that: after query count number of element of notDeclared. If > 0 waste the collection. Otherwise evaluate number of match, if is >0, there is at least one elements can match.
Good. But how I evaluate if all elements of options.sel are false?
And if I check another parameters, I need to make another aggregation (one for parameters)?

Related

Add Aggregate field in MongoDB pipeline depending on all elements of an array

Given the following documents in a collection:
[{
"_id": {
"$oid": "63f06283b80a395adf27780d"
},
"suppliers": [
{
"name": "S1",
"duesPaid": true
},
{
"name": "S2",
"duesPaid": true
}
]
},{
"_id": {
"$oid": "63f06283b80a395adf27780e"
},
"suppliers": [
{
"name": "S1",
"duesPaid": true
},
{
"name": "S2",
"duesPaid": false
}
]
}]
I would like to create an aggregateField in each document that does the following: If the suppliers array has at least 1 element and every element in that has the duesPaid field == true, then add a field to the document suppliersPaid = true. Otherwise add suppliersPaid = false. The resulting documents from the pipeline should look like this:
[{
"_id": {
"$oid": "63f06283b80a395adf27780d"
},
"suppliers": [
{
"name": "S1",
"duesPaid": true
},
{
"name": "S2",
"duesPaid": true
}
],
"suppliersPaid": true,
},{
"_id": {
"$oid": "63f06283b80a395adf27780e"
},
"suppliers": [
{
"name": "S1",
"duesPaid": true
},
{
"name": "S2",
"duesPaid": false
}
],
"suppliersPaid": false,
}]
I have tried the following pipeline:
[{$addFields: {
suppliersPaid: {
$and: [
{ $gte: [{ $size: "$suppliers" }, 1] },
{
suppliers: {
$not: {
$elemMatch: { duesPaid: false },
},
},
},
],
},
}}]
and I get the following error: Invalid $addFields :: caused by :: Unrecognized expression '$elemMatch'
I've tried to eliminate the reliance on $elemMatch per the docs https://www.mongodb.com/docs/manual/reference/operator/query/elemMatch/#single-query-condition as such:
[{$addFields: {
suppliersPaid: {
$and: [
{ $gte: [{ $size: "$suppliers" }, 1] },
{
suppliers: {
$not: {
duesPaid: false
},
},
},
],
},
}}]
But this yields the incorrect result of setting suppliersPaid to true for both documents, which is incorrect.
Note: I would like to avoid using any sort of JS in this code i.e. no $where operators.
For the second condition:
$eq - Compare the result from 1.1 to return an empty array.
1.1. $filter - Filter the documents from suppliers containing { duesPaid: false }.
db.collection.aggregate([
{
$addFields: {
suppliersPaid: {
$and: [
{
$gte: [
{
$size: "$suppliers"
},
1
]
},
{
$eq: [
{
$filter: {
input: "$suppliers",
cond: {
$eq: [
"$$this.duesPaid",
false
]
}
}
},
[]
]
}
]
}
}
}
])
Demo # Mongo Playground

MongoDB - How to filter document objects within array?

I have a problem with filtering fields(objects) from my document using find function.
Here is my db.collection.find(..) document:
{
"_id": BinData(3, "Uz+QwtoVMt7hjpqMrLxVhQ=="),
"name": "jeorge",
"permissions": [
{
"key": "group.staff",
"value": true,
"context": [
{ "key": "server", "value": "test" }
]
},
{
"key": "group.tester",
"value": true,
"context": [
{ "key": "server", "value": "test" }
]
},
{
"key": "test.test",
"value": true
},
{
"key": "group.default",
"value": true
},
{
"key": "group.helper",
"value": true
}
]
}
How can I filter my document in two ways?
a) Display only fields with nested context object.
b) Display only fields, which hasn't got nested context objects.
Also I need to check if parent's object property key contains string "group" as value (If not - skip it).
For situation b, I have tried this function, but it prints result with only the first matched element (based on mongodb's documentation).
db.collection.find(
{
"_id": BinData(3, "Uz+QwtoVMt7hjpqMrLxVhQ==")
},
{
"permissions": {
$elemMatch: {
"key": {
$regex: "group",
},
"value": true
}
}
}
);
Is it possible by single query? Thanks!
db.collection.aggregate([
{
$match: { _id: BinData(3, "Uz+QwtoVMt7hjpqMrLxVhQ==") }
},
{
$set: {
permissions: {
$filter: {
input: "$permissions",
as: "p",
cond: {
$and: [
"$$p.context",
{ $regexMatch: { input: "$$p.key", regex: "group" } }
]
}
}
}
}
}
])
mongoplayground
db.collection.aggregate([
{
$match: {
_id: BinData(3, "Uz+QwtoVMt7hjpqMrLxVhQ==")
}
},
{
$set: {
permissions: {
$filter: {
input: "$permissions",
as: "p",
cond: {
$and: [
{
$not: [ "$$p.context" ]
},
{
$regexMatch: {
input: "$$p.key",
regex: "group"
}
}
]
}
}
}
}
}
])
mongoplayground

mongo $filter and multiple condition in sub-sub array

I have this portion of collections:
[{
"_id": ObjectId("604f3ae3194f2135b0ade569"),
"parameters": [
{
"_id": ObjectId("602b7455f4b4bf5b41662ec1"),
"name": "Purpose",
"options": [
{
"id": ObjectId("602b764ff4b4bf5b41662ec2"),
"name": "debug",
"sel": true
},
{
"id": ObjectId("602b767df4b4bf5b41662ec3"),
"name": "performance",
"sel": false
},
{
"id": ObjectId("602b764ff4b4bf5b41662ec4"),
"name": "security",
"sel": true
},
{
"id": ObjectId("602b767df4b4bf5b41662ec5"),
"name": "Not Applicable",
"sel": false
}
],
"type": "multiple"
}]
}]
And this query:
db.testCollection.aggregate([
{ $unwind: "$parameters" },
{
$match: {
"parameters._id": ObjectId("602b7455f4b4bf5b41662ec1"),
}
},
{
$addFields: {
match: {
$filter: {
input: "$parameters.options",
as: "option",
cond: {
$and: [
{ $eq: ["$$option.id", ObjectId('602b764ff4b4bf5b41662ec2')] },
{ $eq: ["$$option.sel", true] }
]
}
}
}
}
}
])
The result is a new array match with, in this case,
"match": [
{
"id": ObjectId("602b764ff4b4bf5b41662ec2"),
"name": "debug",
"sel": true
}
]
How I can match, for collections, match for example
"id": ObjectId("602b764ff4b4bf5b41662ec2") and
"id": ObjectId("602b764ff4b4bf5b41662ec4")
and have a result array with or two results or empty?
I am not sure it is possible in single condition inside $filter, but you can try with $let,
$let to declare a variable called filtered and store your filtered result
$in to check condition if filtered element size is 2 then return filtered result otherwise blank array
You have to put the number that you are matching number of ids in filter in $eq condition
{
$addFields: {
match: {
$let: {
vars: {
filtered: {
$filter: {
input: "$parameters.options",
as: "option",
cond: {
$and: [
{
$in: [
"$$option.id",
[ObjectId("602b764ff4b4bf5b41662ec2"), ObjectId("602b767df4b4bf5b41662ec3")]
]
},
{ $eq: ["$$option.sel", true] }
]
}
}
}
},
in: {
$cond: [
{ $eq: [{ $size: "$$filtered" }, 2] },
"$$filtered",
[]
]
}
}
}
}
}
Playground

Nested array query does not work with Mongoose

I have a mongo documnet of
{
"locationId": "5f2084f756dc2428e96fcda4",
"information": [{
"category": ["5e5150c6c52a3904b74d6ff7"],
"date": {
"$date": {
"$numberLong": "1601407317343"
}
},
"private": true
}, {
"category": ["5e5150c6c52a3904b74d6ff7"],
"date": {
"$date": {
"$numberLong": "1601407317343"
}
},
"private": false
},
}],
}
So far, what I have is to query the nested array.
const f = await info.aggregate([
{
$match: {
$and: [
{'locationId': '5f2084f756dc2428e96fcda4'},
{'information.private': true}
]
},
},
]);
I am trying to query information.private = true. However, I receive both 'private: false' and 'private: true'. The outcome is
[
{
"_id": ObjectId("5a934e000102030405000000"),
"information": [
{
"category": [
"5e5150c6c52a3904b74d6ff7"
],
"date": ISODate("2020-09-29T19:21:57.343Z"),
"private": true
},
{
"category": [
"5e5150c6c52a3904b74d6ff7"
],
"date": ISODate("2020-09-29T19:21:57.343Z"),
"private": false
}
],
"locationId": "5f2084f756dc2428e96fcda4"
}
]
I also tried with elemMatch and returns the same results. I'd looked up multiple answers on Stackoverflow but the dot notation and elemMatch do not work in this case. I know that I have done something wrong but I can't figure it out.
You can try $filter,
$match your conditions
$addFields to add field information and $filter to iterate loop of array and get match documents as per condition
const f = await info.aggregate([
{
$match: {
locationId: "5f2084f756dc2428e96fcda4",
"information.private": true
}
},
{
$addFields: {
information: {
$filter: {
input: "$information",
cond: { $eq: ["$$this.private", true] }
}
}
}
}
])
Playground
Other option you can try $redact,
$match your condition
$redact Restricts the contents of the documents based on information stored in the documents themselves.
const f = await info.aggregate([
{ $match: { locationId: "5f2084f756dc2428e96fcda4" } },
{
$redact: {
$cond: [{ $eq: ["$private", false] }, "$$PRUNE", "$$DESCEND"]
}
}
])
Playground
The find projection can accept aggregation expressions and syntax starting from MongoDB 4.4.
const f = await info.find([
locationId: "5f2084f756dc2428e96fcda4",
"information.private": true
},
{
locationId: 1,
information: {
$filter: {
input: "$information",
cond: { $eq: ["$$this.private", true] }
}
}
})
Playground

MongoDB Aggregation - Lookup pipeline not returning any documents

I'm having hard time getting $lookup with a pipeline to work in MongoDB Compass.
I have the following collections:
Toys
Data
[
{
"_id": {
"$oid": "5d233c3bb173a546386c59bb"
},
"type": "multiple",
"tags": [
""
],
"searchFields": [
"Jungle Stampers - Two",
""
],
"items": [
{
"$oid": "5d233c3cb173a546386c59bd"
},
{
"$oid": "5d233c3cb173a546386c59be"
},
{
"$oid": "5d233c3cb173a546386c59bf"
},
{
"$oid": "5d233c3cb173a546386c59c0"
},
{
"$oid": "5d233c3cb173a546386c59c1"
},
{
"$oid": "5d233c3cb173a546386c59c2"
},
{
"$oid": "5d233c3cb173a546386c59c3"
},
{
"$oid": "5d233c3cb173a546386c59c4"
}
],
"name": "Jungle Stampers - Two",
"description": "",
"status": "active",
"category": {
"$oid": "5cfe727cac920000086b880e"
},
"subCategory": "Stamp Sets",
"make": "",
"defaultCharge": null,
"defaultOverdue": null,
"sizeCategory": {
"$oid": "5d0cfde57561e107c88fbde3"
},
"ageFrom": {
"$numberInt": "24"
},
"ageTo": {
"$numberInt": "120"
},
"images": [
{
"_id": {
"$oid": "5d233c3bb173a546386c59bc"
},
"id": {
"$oid": "5d233c39b173a546386c59ba"
},
"url": "/toyimages/5d233c39b173a546386c59ba.jpg",
"thumbUrl": "/toyimages/thumbs/tn_5d233c39b173a546386c59ba.jpg"
}
],
"__v": {
"$numberInt": "2"
}
}
]
Loans
Data
[
{
"_id": {
"$oid": "5e1f1661b712215978c746d9"
},
"tags": [],
"member": {
"$oid": "5e17495e4f81ab3f900dbb63"
},
"source": "admin portal - potter1#gmail.com",
"items": [
{
"id": {
"$oid": "5e1f160eb712215978c746d5"
},
"status": "new",
"_id": {
"$oid": "5e1f1661b712215978c746db"
},
"toy": {
"$oid": "5d233c3bb173a546386c59bb"
},
"cost": {
"$numberInt": "0"
}
},
{
"id": {
"$oid": "5e1f160eb712215978c746d5"
},
"status": "new",
"_id": {
"$oid": "5e1f1661b712215978c746da"
},
"toy": {
"$oid": "5d233b1ab173a546386c59b5"
},
"cost": {
"$numberInt": "0"
}
}
],
"dateEntered": {
"$date": {
"$numberLong": "1579095632870"
}
},
"dateDue": {
"$date": {
"$numberLong": "1579651200000"
}
},
"__v": {
"$numberInt": "0"
}
}
]
I am trying to return a list of toys and their associated loans that have a status of 'new' or 'out'.
I can use the following $lookup aggregate to fetch all loans:
{
from: 'loans',
localField: '_id',
foreignField: 'items.toy',
as: 'loansSimple'
}
However I am trying to use a pipeline to load loans that have the two statuses I am interested in, but it always only returns zero documents:
{
from: 'loans',
let: {
'toyid': '$_id'
},
pipeline: [
{
$match: {
$expr: {
$and: [
{$eq: ['$items.toy', '$$toyid']},
{$eq: ['$items.status', 'new']} // changed from $in to $eq for simplicity
]
}
}
}
],
as: 'loans'
}
This always seems to return 0 documents, however I arrange it:
Have I made a mistake somewhere?
I'm using MongoDB Atlas, v4.2.2, MongoDB Compass v 1.20.4
You are trying to search $$toyid inside inner array, but Operator Expression $eq cannot resolve it.
Best solution: $let (returns filtered loans by criteria) + $filter (applies filter for inner array) operator helps us to get desired result.
db.toys.aggregate([
{
$lookup: {
from: "loans",
let: {
"toyid": "$_id",
"toystatus": "new"
},
pipeline: [
{
$match: {
$expr: {
$gt: [
{
$size: {
$let: {
vars: {
item: {
$filter: {
input: "$items",
as: "tmp",
cond: {
$and: [
{
$eq: [
"$$tmp.toy",
"$$toyid"
]
},
{
$eq: [
"$$tmp.status",
"$$toystatus"
]
}
]
}
}
}
},
in: "$$item"
}
}
},
0
]
}
}
}
],
as: "loans"
}
}
])
MongoPlayground
Alternative solution 1. Use $unwind to flatten items attribute. (We create extra field named tmp which stores items value, flatten it with $unwind operator, match as you were doing and then exclude from result)
db.toys.aggregate([
{
$lookup: {
from: "loans",
let: {
"toyid": "$_id"
},
pipeline: [
{
$addFields: {
tmp: "$items"
}
},
{
$unwind: "$tmp"
},
{
$match: {
$expr: {
$and: [
{
$eq: [
"$tmp.toy",
"$$toyid"
]
},
{
$eq: [
"$tmp.status",
"new"
]
}
]
}
}
},
{
$project: {
tmp: 0
}
}
],
as: "loans"
}
}
])
MongoPlayground
Alternative solution 2. We use $reduce to create toy's array and with $in operator we check if toyid exists inside this array.
db.toys.aggregate([
{
$lookup: {
from: "loans",
let: {
"toyid": "$_id"
},
pipeline: [
{
$addFields: {
toys: {
$reduce: {
input: "$items",
initialValue: [],
in: {
$concatArrays: [
"$$value",
[
"$$this.toy"
]
]
}
}
}
}
},
{
$match: {
$expr: {
$in: [
"$$toyid",
"$toys"
]
}
}
},
{
$project: {
toys: 0
}
}
],
as: "loans"
}
}
])
$expr receives aggregation expressions, At that point $$items.toy is parsed for each element in an array as you would expect (however if it would it will still give you "bad" results as you'll get loans that have the required toy id and any other item with status new in their items array).
So you have two options to work around this:
If you don't care about the other items in the lookup'd document you can add an $unwind stage at the start of the lookup pipeline like so:
{
from: 'loans',
let: {
'toyid': '$_id'
},
pipeline: [
{
$unwind: "$items"
},
{
$match: {
$expr: {
$and: [
{$eq: ['$items.toy', '$$toyid']},
{$eq: ['$items.status', 'new']} // changed from $in to $eq for simplicity
]
}
}
}
],
as: 'loans'
}
If you do care about them just iterate the array in one of the possible ways to get a 'correct' match, here is an example using $filter
{
from: 'loads',
let: {
'toyid': '$_id'
},
pipeline: [
{
$addFields: {
temp: {
$filter: {
input: "$items",
as: "item",
cond: {
$and: [
{$eq: ["$$item.toy", "$$toyid"]},
{$eq: ["$$item.status", "new"]}
]
}
}
}
}
}, {$match: {"temp.0": {exists: true}}}
],
as: 'loans'
}