I am trying to use the following query in MongoDB but it is not working.
db.test.aggregate(
$match: {
$and: [
type: { $in: ["TOYS"] },
type: { $nin: ["BARBIE"] },
time: { $lt: ISODate("2013-12-09T00:00:00Z") }
]
}
})
It says invalid character ":".
Is it possible to use $and with $match? I have seen an example on this forum of $or with $match so I presumed this is possible.
Thank you in advance for your help and guidance.
$and with $match works just fine.
You have syntax errors in your query. Try this.
db.test.aggregate([
{
$match: {
$and: [
{type: {$in: ["TOYS"]}},
{type: {$nin: ["BARBIE"]}},
{time: {$lt:ISODate("2013-12-09T00:00:00Z")}}
]
}
}
])
And for what you are trying to do, you do not need an $and.
{
$match: {
$or:[
{'sender':sender, 'recipient':recipient},
{'recipient':sender,'sender':recipient}
]
}
}
using $or
db.test.find( {$and: [ {"type": {$in: ["TOYS"]}},
{"type": {$nin: ["BARBIE"]}},
{"time": {$lt:ISODate("2013-12-09T00:00:00Z")}}
]
})
AND works with FIND, receives an array of matches (but it's not a match instruction)
Aggregation framework is for something completely different, it's like the word says, for aggregating (count, sum, avg, and so worth grouping or unwinding, etc)
example of $lookup then $match
db.orders.aggregate([
{
"$lookup": {
"from": "user",
"localField": "user",
"foreignField": "_id",
"as": "user"
}
},
{
"$unwind": "$user"
},
{
"$match": {
"$and": [
{
"privacy.mode": {
"$gt": 0
}
},
{
"user.privacy.mode": {
"$gt": 0
}
}
]
}
}
])
``
Related
Does anyone who knows how to filter data and only keep those history_doc.data.objectId equals _id?
I have tried so many methods but none of them works
{'history_doc.data.objectId': {$eq: '$_id'}}
{'history_doc.data.objectId': {$eq: {$toString: '$_id'}}}
You can use $expr and $eq with $toObjectId into aggregation query like this:
.aggregate({
$match: {
$expr: {
$eq: [
"$_id",
{
"$toObjectId": "$history_doc.data.objectId"
}
]
}
}
})
Example here.
You can do it with Aggregation Framework:
$match - to filter documents based on some custom criteria
$eq and $toString - To check it _id and history_doc.data.objectId are the same.
db.collection.aggregate([
{
"$match": {
"$expr": {
"$eq": [
{
"$toString": "$_id"
},
"$history_doc.data.objectId"
]
}
}
}
])
Working example
I have this aggregation:
db.getCollection("users").aggregate([
{
"$match": {
"_id": "5a708a38e6a4078bd49f01d5"
}
},
{
"$lookup": {
"from": "user-locations",
"localField": "locations",
"as": "locations",
"foreignField": "_id"
}
}
])
It works well, but there is one small thing that I don't understand and I can't fix.
In the query output, the locations array is reordered by ObjectId and I really need to keep the original order of data.
Here is how the locations array from the users collection looks like
'locations' : [
ObjectId("5b55e9820b720a1a7cd19633"),
ObjectId("5a708a38e6a4078bd49ef13f")
],
And here is the result after the aggregation:
'locations' : [
{
'_id' : ObjectId("5a708a38e6a4078bd49ef13f"),
'name': 'Location 2'
},
{
'_id' : ObjectId("5b55e9820b720a1a7cd19633"),
'name': 'Location 1'
}
],
What am I missing here? I really have no idea how to proceed with this issue.
Could you give me a push?
$lookup does not guarantee order of result documents, you can try a approach to manage natural order of document,
$unwind deconstruct locations array and add auto index number will start from 0,
$lookup with locations
$set to select first element from locations
$sort by index field in ascending order
$group by _id and reconstruct locations array
db.users.aggregate([
{ $match: { _id: "5a708a38e6a4078bd49f01d5" } },
{
$unwind: {
path: "$locations",
includeArrayIndex: "index"
}
},
{
$lookup: {
from: "user-locations",
localField: "locations",
foreignField: "_id",
as: "locations"
}
},
{ $set: { locations: { $arrayElemAt: ["$locations", 0] } } },
{ $sort: { index: 1 } },
{
$group: {
_id: "$_id",
locations: { $push: "$locations" }
}
}
])
Playground
From this closed bug report:
When using $lookup, the order of the documents returned is not guaranteed. The documents are returned in "natural order" - as they are encountered in the database. The only way to get a guaranteed consistent order is to add a $sort stage to the query.
Basically the way any Mongo query/pipeline works is that it returns documents in the order they were matched, meaning the "right" order is not guaranteed especially if there's indes usage involved.
What you should do is add a $sort stage as suggested, like so:
db.collection.aggregate([
{
"$match": {
"_id": "5a708a38e6a4078bd49f01d5"
}
},
{
"$lookup": {
"from": "user-locations",
"let": {
"locations": "$locations"
},
"pipeline": [
{
"$match": {
"$expr": {
"$setIsSubset": [
[
"$_id"
],
"$$locations"
]
}
}
},
{
$sort: {
_id: 1 // any other sort field you want.
}
}
],
"as": "locations",
}
}
])
You can also keep the original $lookup syntax you're using and just $unwind, $sort and then $group to restore the structure.
Why do I have to use this code: { $match: { $expr: { <aggregation expression> } } } to match a document using a document input variable as opposed to doing: { $match: { <query> } } ?
For example:
$lookup: {
from: "comments",
let: { myvar: '$myInputDocVariable'},
pipeline: [
{ $match:
{ $expr:
{ $and:
[
{ $eq: [ "$varFromCommentDocument", "$$myvar" ] },
]
}
}
},
],
as: "returnedValue"
}
The query above works fine but the query below does not work as expected. Why is this? Does this mean that if you are using input variables in a $lookup pipeline you have to use $expr? why is that?
$lookup: {
from: "comments",
let: { myvar: '$myInputDocVariable'},
pipeline: [
{ $match: { "$varFromCommentDocument", "$$myvar" } }
],
as: "returnedValue"
}
When you perform uncorrelated sub-queries for $lookup operator:
If you need to compare parent collection's field within pipeline, MongoDB cannot apply the standard query syntax (field:value) for variable / Aggregation expressions. In this case, you need to use $expr operator.
Example:
{ $match:
{ $expr:
{ $and:[
{ $eq: [ "$varFromCommentDocument", "$$myvar" ] },
]}
}
}
if it matches against "hard-coded" values, you don't need to use $expr operator.
Example:
$lookup: {
from: "comments",
pipeline: [
{ $match:{
"key": "value",
"key2": "value2"
}}
],
as: "returnedValue"
}
Does this mean that if you are using input variables in a $lookup
pipeline you have to use $expr
Yes correct, by default in filters i.e; in filter part of .find() or in $match aggregation stage you can't use an existing field in the document.
If at all if you need to use existing field's value in your query filter then you need to use aggregation pipeline, So in order to use aggregation pipeline in .find() or in $match you need to wrap your filter query with $expr. Same way to access local variables got created using let of $lookup filter in $match needs to be wrapped by $expr.
Let's consider below example :
Sample Docs :
[
{
"key": 1,
"value": 2
},
{
"key": 2,
"value": 4
},
{
"key": 5,
"value": 5
}
]
Query :
db.collection.find({ key: { $gt: 1 }, value: { $gt: 4 } })
Or
db.collection.aggregate([ { $match: { key: { $gt: 1 }, value: { $gt: 4 } } } ])
Test : mongoplayground
If you see the above query both input 1 & 4 are passed into query but it you check below query where you try to match key field == value field - it doesn't work :
db.collection.aggregate([ { $match: { key: { $eq: "$value" } } } ])
Test : mongoplayground
Above as you're comparing two existing fields then you can't do that as it mean you're checking for docs with key field value as string "$value". So to say it's not a string it's actually a reference to value field you need to use $eq aggregation operator rather than $eq query operator like below :
db.collection.aggregate([ { $match: { $expr: { $eq: [ "$key", "$value" ] } } } ])
Test : mongoplayground
I have collection with something similar datastructure
{
id: 1
limit: {
max: 10000,
used: 0
}
}
and I tried running the below query but it is giving 0 results
db.getCollection('promos').aggregate(
[
{ $match: { id: 1} },
{$match: { $expr: {$gt ["limit.max" , "limit.used"]}}}
])
I also used the below query
db.getCollection('promos').aggregate(
[
{ $match: { id: 1} },
{$match: { "$limit.max": {$gt: "limit.used"}}}
])
None of them is giving the result . Any help will be appreciated.
You need to prefix "field expressions" with the $. This also can be simply done in a .find()
db.getCollection('promos').find({
"id": 1,
"$expr": { "$gt": [ "$limit.max" , "$limit.used" ] }
})
Or a single $match stage if you really need to use aggregate instead:
db.getCollection('promos').aggregate([
{ "$match": {
"id": 1,
"$expr": { "$gt": [ "$limit.max" , "$limit.used" ] }
}}
])
That's how $expr works and you can "mix it" with other regular query operators in the same query or pipeline stage.
Also see $gt for general usage examples
Of course if you don't actually even have MongoDB 3.6, then you use $redact instead:
db.getCollection('promos').aggregate([
{ "$match": { "id": 1 } },
{ "$redact": {
"$cond": {
"if": { "$gt": [ "$limit.max" , "$limit.used" ] },
"then": "$$KEEP",
"else": "$$PRUNE"
}
}}
])
Or use $where. Works in all versions:
db.getCollection('promos').find({
"id": 1,
"$where": "this.limit.max > this.limit.used"
})
I have a collection that I'm performing an aggregation on and I've basically gotten it down to
{array:[1,2,3], value: 1},
{array:[1,2,3], value: 4}
How would I perform an aggregation match to check if the value is in the array? I tried using {$match: {"array: {$in: ["$value"]}}} but it doesn't find anything.
I would want the output (if using the above as an example) to be:
{array:[1,2,3], value:1}
You can use aggregation expression in regular query in 3.6 version.
db.collection_name.find({"$expr": {"$in": ["$value", "$array"]}})
Using Aggregation:
You can use $match + $expr in current 3.6 version.
db.collection_name.aggregate({"$match": {"$expr": {"$in": ["$value", "$array"]}}})
You can try $redact + $in expression in 3.4 version.
db.collection_name.aggregate({
"$redact": {
"$cond": [
{
"$in": [
"$value",
"$array"
]
},
"$$KEEP",
"$$PRUNE"
]
}
})
As stated, $where is a good option where you do not need to continue the logic in the aggregation pipeline.
But if you do then use $redact, with $map to transform the "value" into an array and use of $setIsSubSet to compare. It is the fastest way to do this since you do not need to duplicate documents using $unwind:
db.collection.aggregate([
{ "$redact": {
"$cond": {
"if": { "$setIsSubset": [
{ "$map": {
"input": { "$literal": ["A"] },
"as": "a",
"in": "$value"
}},
"$array"
]},
"then": "$$KEEP",
"else": "$$PRUNE"
}
}}
])
The $redact pipeline operator allows the proccessing of a logical condition within $cond and uses the special operations $$KEEP to "keep" the document where the logical condition is true or $$PRUNE to "remove" the document where the condition was false.
This allows it to work like $project with a subsequent $match, but in a single pipeline stage which is more efficient.
Considering these are native coded operators and not JavaScript then it is likely "the" fastest way to perform your match. So provided you are using a MongoDB 2.6 version or above, then this is the way you should be doing it to compare these elements in your document.
A slight variation based on #chridam's answer:
db.test.aggregate([
{ "$unwind": "$array" },
{ "$group": {
_id: { "_id": "$_id", "value": "$value" },
array: { $push: "$array" },
mcount: { $sum: {$cond: [{$eq: ["$value","$array"]},1,0]}}
}
},
{ $match: {mcount: {$gt: 0}}},
{ "$project": { "value": "$_id.value", "array": 1, "_id": 0 }}
])
The idea is to $unwind and $group back the array, counting in mcount the number of items matching the value. After that, a simple $match on mcount > 0 will filter out unwanted documents.
A more efficient approach would involve a single pipeline that uses the $redact operator as follows:
db.collection.aggregate([
{
"$redact": {
"$cond": [
{
"$setIsSubset": [
["$value"],
"$array"
]
},
"$$KEEP",
"$$PRUNE"
]
}
}
])
For earlier versions of MongoDB that do not support $redact (versions < 2.6) then consider this aggregation pipeline that uses the $unwind operator:
db.collection.aggregate([
{ "$unwind": "$array" },
{
"$project": {
"isInArray": {
"$cond": [
{ "$eq": [ "$array", "$value" ] },
1,
0
]
},
"value": 1,
"array": 1
}
},
{ "$sort": { "isInArray": -1 } },
{
"$group": {
"_id": {
"_id": "$_id",
"value": "$value"
},
"array": { "$push": "$array" },
"isInArray": { "$first": "$isInArray" }
}
},
{ "$match": { "isInArray": 1 } },
{ "$project": { "value": "$_id.value", "array": 1, "_id": 0 } }
])
A little late to answer but this presents another solution:
By using addFields and match separately, this gives more flexibility than the redact. You can expose several fields and then use other matching logic together based on the results.
db.applications.aggregate([
{$addFields: {"containsValueInArray": {$cond:[{$setIsSubset: [["valueToMatch"], "$arrayToMatchIn"]},true,false]}}},
{$match: {"containsValueInArray":true}}
]);
Try the combination of $eq and $setIntersection
{$group :{
_id: "$id",
yourName : { $sum:
{ $cond :[
{$and : [
{$eq:[{$setIntersection : ["$someArrayField", ["$value"]] },["$value"]]}
]
},1,0]
}
}
}
i prefer without grouping, there's an easy approach since v.3.2
...aggregate([
{
$addFields: {
arrayFilter: {
$filter: {
input: '$array',
as: 'item',
cond: ['$$item', '$value']
}
}
}
},
{
$unwind: '$arrayFilter'
},
{
$project: {
arrayFilter: 0
}
}
]);
Add a temporary filter field
$unwind on the resulting array (pipeline results with empty arrays get removed)
(optional) remove filter field from result via project
You can do it with simple $project & $match
db.test.aggregate([{
$project: {
arrayValue: 1,
value: 1,
"has_same_value" : { $in: ["$value", "$arrayValue"] }
}
},
{
$match: {has_same_value: true}
},
{
$project: {has_same_value: 0}
}])
"$match": { "name": { "$in":["Rio","Raja"] }} }])