mongodb $or syntax in aggregation pipeline - mongodb

Question

In mongodb I found a strange behavior of $or, think below collection:
{ "_id" : 1, "to" : [ { "_id" : 2 }, { "_id" : 4, "valid" : true } ] }
When aggregate with $match:
db.ooo.aggregate([{$match:{ $or: ['$to', '$valid'] }}])
Will throw error with aggregate failed: (sure, I know how to fix it...)
"ok" : 0,
"errmsg" : "$or/$and/$nor entries need to be full objects",
"code" : 2,
"codeName" : "BadValue"
But If the $or used in a $cond statement:
db.ooo.aggregate([{ "$redact": {
"$cond": {
"if": { $or: ["$to", "$valid"] },
"then": "$$DESCEND",
"else": "$$PRUNE"
}
}}])
The result will shown and no error thrown, see mongodb aggregate $redact to filter array elements
The question is what's going on with the $or syntax? why the same condition not work in $match but did work in $cond?
Also I'm looked up the docs:
$cond
If the evaluates to true, then $cond evaluates and returns the value of the expression. Otherwise, $cond evaluates and returns the value of the expression.
The arguments can be any valid expression. For more information on expressions, see Expressions.
$or
Evaluates one or more expressions and returns true if any of the expressions are true. Otherwise, $or returns false.
For more information on expressions, see Expressions.
PS: I'm using mongodb 3.4.5 but not tested on other version.
I don't have a clue...
UPDATE
Based on the answer of #Neil, I'm also tried the $filter usage with $or:
1.
db.ooo.aggregate([{ "$project": {
result:{
"$filter": {
"input": "$to",
"as": "el",
"cond": {$or: ["$$el.valid"]}
}
}
}}])
2.
db.ooo.aggregate([{ "$project": {
result:{
"$filter": {
"input": "$to",
"as": "el",
"cond": {$or: "$$el.valid"}
}
}
}}])
3.
db.ooo.aggregate([{ "$project": {
result:{
"$filter": {
"input": "$to",
"as": "el",
"cond": "$$el.valid"
}
}
}}])
All the above 3 $filter, the syntax are ok, the result is shown and no error thrown.
Seems $or will work with field names directly only in $cond or cond?
Or this is a hacking usage of $or?

Answer 1

The $redact pipeline stage is the is the wrong thing for this type of operation. Instead use $filter which actually "filters" things from arrays:
db.ooo.aggregate([
{ "$addFields": {
"to": {
"$filter": {
"input": "$to",
"as": "t",
"cond": { "$ifNull": [ "$$t.valid", false ] }
}
}
}}
])
Produces:
{
"_id" : ObjectId("594c5b0a212a102096cebf7e"),
"id" : 1,
"to" : [
{
"id" : 4,
"valid" : true
}
]
}
The problem with $redact in this case is the only way you can actually "redact" from an array is by using $$DESCEND on the false condition. This is recursive and evaluates the expression from the top level of the document downwards. At whatever level where the condition is not met, $redact will discard it. No "valid"field in the "top-level" means it would discard the whole document, unless we gave an alternate condition.
Since not all array elements have the "valid" field in "addition" to the top level of the document, we cannot even "hack it" to pretend something is there.
For example you appear to be trying to do this:
db.ooo.aggregate([
{ "$redact": {
"$cond": {
"if": {
"$or": [
{ "$ifNull": [ "$$ROOT.to", false ] },
"$valid"
]
},
"then": "$$DESCEND",
"else": "$$PRUNE"
}
}}
])
But when you look carefully, that kind of comparison will essentially "always" evaluate to true, no matter how hard you tried to hack a condition.
You could do:
db.ooo.aggregate([
{ "$redact": {
"$cond": {
"if": {
"$or": [
{ "$ifNull": [ "$to", false ] },
"$valid"
]
},
"then": "$$DESCEND",
"else": "$$PRUNE"
}
}}
])
Which does correctly remove the element from the array:
{
"_id" : ObjectId("594c5b0a212a102096cebf7e"),
"id" : 1,
"to" : [
{
"id" : 4,
"valid" : true
}
]
}
But it is overkill and adds unnecessary overhead to logic processing when the simple $filter will do in this case. You should only need this form when there are actually "nested" arrays that need to be recursively processed, and all conditions can actually be met at all levels.
The lesson here is to use the correct operators for their designed purpose.