What does $$this mean in MongoDB Aggregation? - mongodb

In a Mongo Aggregation example I've encountered the expression $$this, but cannot find a reference to it in the MongoDB documentation (not even in the documentation about aggregation variables)
Here is the sample data:
{ "_id" : 1, "actions" : [ 2, 6, 3, 8, 5, 3 ] }
{ "_id" : 2, "actions" : [ 6, 4, 2, 8, 4, 3 ] }
{ "_id" : 3, "actions" : [ 6, 4, 6, 4, 3 ] }
{ "_id" : 4, "actions" : [ 6, 8, 3 ] }
{ "_id" : 5, "actions" : [ 6, 8 ] }
{ "_id" : 6, "actions" : [ 6, 3, 11, 8, 3 ] }
{ "_id" : 7, "actions" : [ 6, 3, 8 ] }
Here is the code
Here is the code I'm looking at:
db.test.aggregate([
{$match:{actions:{$all:[6,3,8]}}},
{$project:{actions638:{$map:{
input:{$range:[0,{$subtract:[{$size:"$actions"},2]}]},
in:{$slice:["$actions","$$this",3]}
}}}}
])
and here is the output
{ "_id" : 1, "actions638" : [ [ 2, 6, 3 ], [ 6, 3, 8 ], [ 3, 8, 5 ], [ 8, 5, 3 ] ] }
{ "_id" : 2, "actions638" : [ [ 6, 4, 2 ], [ 4, 2, 8 ], [ 2, 8, 4 ], [ 8, 4, 3 ] ] }
{ "_id" : 4, "actions638" : [ [ 6, 8, 3 ] ] }
{ "_id" : 6, "actions638" : [ [ 6, 3, 11 ], [ 3, 11, 8 ], [ 11, 8, 3 ] ] }
{ "_id" : 7, "actions638" : [ [ 6, 3, 8 ] ] }

$$this refers to the current item inside the array that is being processed by the $map function.
An alternative is to use the as property so that instead of referring to $$this you refer to the name you provided in the as. For example (from the docs)
db.grades.aggregate(
[
{ $project:
{ adjustedGrades:
{
$map:
{
input: "$quizzes",
as: "grade",
in: { $add: [ "$$grade", 2 ] }
}
}
}
}
]
)

Related

Querying for arrays in MongoDB

Suppose in collection I have following documents:
[
{"title": "t1", "fingerprint":[1, 2, 3]},
{"title": "t2", "fingerprint":[4, 5, 6]}
]
I want to query documents in which at least one element in fingerprint at given position is equal to my querying array.
For example:
query([1, 7, 9]) should return [{"title": "t1", "fingerprint":[1, 2, 3]}]
query([1, 5, 9]) should return [{"title": "t1", "fingerprint":[1, 2, 3]}, {"title": "t2", "fingerprint":[4, 5, 6]}]
but query([5,1,9]) should return none records, because neither of records have same value at any of the positions in fingerprint array.
How to write given query?
When you are trying to match only documents with arrays where the sequence [ 1 2, 3 ] appears in values field and only in that exact order, you can do it this way:
db.testcol.find()
{ "_id" : "first", "value" : [ 1, 2, 3 ] }
{ "_id" : "second", "value" : [ 4, 5, 6 ] }
{ "_id" : "third", "value" : [ 1, 12, 13 ] }
{ "_id" : "fourth", "value" : [ 3, 2, 1 ] }
{ "_id" : "fifth", "value" : [ 1, 12, 13, 2, 3 ] }
{ "_id" : "sixth", "value" : [ 3, 2, 1, 2, 3 ] }
> db.testcol.aggregate([{$addFields:{
cmp: {$in:[
{$literal:[1,2,3]},
{$map: {
input:{$range:[0, {$subtract:[{$size:"$value"},2]}]},
as:"l",
in: {$slice: [ "$value", "$$l", 3] }
}}
]}
}}])
{ "_id" : "first", "value" : [ 1, 2, 3 ], "cmp" : true }
{ "_id" : "second", "value" : [ 4, 5, 6 ], "cmp" : false }
{ "_id" : "third", "value" : [ 1, 12, 13 ], "cmp" : false }
{ "_id" : "fourth", "value" : [ 3, 2, 1 ], "cmp" : false }
{ "_id" : "fifth", "value" : [ 1, 12, 13, 2, 3 ], "cmp" : false }
{ "_id" : "sixth", "value" : [ 3, 2, 1, 2, 3 ], "cmp" : true }
What the $addFields stage does is checks if [1,2,3] appears in a list of three element arrays starting at position 0 of value array and moving forward till two positions before the end.
As you can see, it's now trivial to add a $match stage to filter out documents where cmp is not true.
You can use the .$index notation to perform such a search.
Example for your query([1, 7, 9])
db.coll.find({$or: [{"fingerprint.0": 1}, {"fingerprint.1": 7 }, {"fingerprint.2": 9}]})
{ "_id" : ObjectId("59170da907e34e73c0c93a9b"), "title" : "t1", "fingerprint" : [ 1, 2, 3 ] }
And query([1, 5, 9])
db.coll.find({$or: [{"fingerprint.0": 1}, {"fingerprint.1": 5 }, {"fingerprint.2": 9}]})
{ "_id" : ObjectId("59170da907e34e73c0c93a9b"), "title" : "t1", "fingerprint" : [ 1, 2, 3 ] }
{ "_id" : ObjectId("59170da907e34e73c0c93a9c"), "title" : "t2", "fingerprint" : [ 4, 5, 6 ] }
$in operator is used to match a value against list of values.
According to above mentioned description please try executing following query in MongoDB shell
db.collection.find({fingerprint:{$in:[1,7,9]}})

Mongo db sorting on multikey index fields

I was going through mongo db indexes and found this when i create index on multi key field and try to sort the result the behavior is strange.
For example:
> db.testIndexes.find();
{ "_id" : ObjectId("584e6ca8d23d3b48f9cb819d"), "type" : "depart", "item" : "aaa", "ratings" : [ 5, 8, 9 ] }
{ "_id" : ObjectId("584e6cb2d23d3b48f9cb819e"), "type" : "depart", "item" : "aaa", "ratings" : [ 2, 3, 4 ] }
{ "_id" : ObjectId("584e6cbdd23d3b48f9cb819f"), "type" : "depart", "item" : "aaa", "ratings" : [ 10, 6, 1 ] }
db.testIndexes.createIndex({ratings:1});
Now if i sue these queries :
db.testIndexes.find().sort({ratings:1}).pretty();
Result is like this
{
"_id" : ObjectId("584e6cbdd23d3b48f9cb819f"),
"type" : "depart",
"item" : "aaa",
"ratings" : [
10,
6,
1
]
}
{
"_id" : ObjectId("584e6cb2d23d3b48f9cb819e"),
"type" : "depart",
"item" : "aaa",
"ratings" : [
2,
3,
4
]
}
{
"_id" : ObjectId("584e6ca8d23d3b48f9cb819d"),
"type" : "depart",
"item" : "aaa",
"ratings" : [
5,
8,
9
]
}
and for query
db.testIndexes.find().sort({ratings:-1}).pretty();
Results are:
{
"_id" : ObjectId("584e6cbdd23d3b48f9cb819f"),
"type" : "depart",
"item" : "aaa",
"ratings" : [
10,
6,
1
]
}
{
"_id" : ObjectId("584e6ca8d23d3b48f9cb819d"),
"type" : "depart",
"item" : "aaa",
"ratings" : [
5,
8,
9
]
}
{
"_id" : ObjectId("584e6cb2d23d3b48f9cb819e"),
"type" : "depart",
"item" : "aaa",
"ratings" : [
2,
3,
4
]
}
As results does not seems to follow and order so can anyone help how mongo is sorting these results.
Thanks
Virendra
Well it does seem like the results are not following any order but actually they are. In your first sort {ratings:1}, what's happening here is the results are ordered by the smallest element in ratings. Since these are your lists:
[ 10, 6, 1 ] [ 2, 3, 4 ] [ 5, 8, 9 ]
So the list [ 10, 6, 1 ] smallest element is 1, the list [ 2, 3, 4 ] smallest element is 2 and the list [ 5, 8, 9 ] smallest element is 5. So the results are ordered in that way.
When you sort by descending, the same order happens but by maximum element in ratings.
Hope this helps.

$add with some fields as Null returning sum value as Null

These are the documents in my collection
{ "_id" : 1, "quizzes" : [ 10, 6, 7 ], "labs" : [ 5, 8 ], "final" : 80, "midterm" : 75 ,"extraMarks":10}
{ "_id" : 2, "quizzes" : [ 9, 10 ], "labs" : [ 8, 8 ], "final" : 95, "midterm" : 80 }
{ "_id" : 3, "quizzes" : [ 4, 5, 5 ], "labs" : [ 6, 5 ], "final" : 78, "midterm" : 70 }
Only Document 1 has a field extra marks:
Now i have to make a projection as a sum of "final"+"midterm"+"extraMarks"
I have written a query for projection as follows:
db.students.aggregate([ { $project: { examTotal: { $add: [ "$final", "$midterm","$extraMarks" ] } } } ])
This query is giving me correct result for Document1 and for Doc2 and Doc3, as the field doesnt exist its giving sum as null.
Is it possible to check if the field is not null and add in the query itself..Any suggestions for this?
Is there any functionality similar to nvl (in SQL) in the queries Mongo DB?
You can do two projections, the first using $ifNull which is similar to nvl:
db.students.aggregate([
{ $project: { final: 1, midterm: 1, extraMarks: { $ifNull: [ "$extraMarks", 0 ] } } },
{ $project: { examTotal: { $add: [ "$final", "$midterm","$extraMarks" ] } } }
])
Use the $match stage, add it to the beginning of your pipeline
{$match: {"your_field":{$exists: true}}}

MongoDB - select document where all values in field array are present in a given array

I have documents like
{
foo : [1, 2]
}
{
foo : [2, 3]
}
Given an array like
[2, 3, 4]
How would I select only the second document? i.e. select only the documents where all the values in foo match values in a given array.
Basically there are some ways to match array. There is no exact solution for your need.
Considering you have documents like :
{ "_id" : ObjectId("51b05a712961f4704684d901"), "x" : [ 6, 7, 8, 9 ] }
{ "_id" : ObjectId("51b05a712961f4704684d902"), "x" : [ 7, 8, 9, 10 ] }
{ "_id" : ObjectId("51b05a712961f4704684d903"), "x" : [ 8, 9, 10, 11 ] }
You can use query1 like:
db.collection.find({x:[3,4,5,6]})
The result is exact match only for arrays like x
result1:
{ "_id" : ObjectId("51b05a712961f4704684d8fe"), "x" : [ 3, 4, 5, 6 ] }
query1 will not match :
{ "_id" : ObjectId("51b05a712961f4704684d8fe"), "x" : [ 3, 4, 5] }
{ "_id" : ObjectId("51b05a712961f4704684d8fe"), "x" : [ 3, 4, 5, 6, 7] }
You can use : query2 like:
db.t.find({x:{$all:[3,4]}})
result2 can be:
{ "_id" : ObjectId("51b05a722961f4704684daf1"), "x" : [ 3, 4, 5, 6 ] }
{ "_id" : ObjectId("51b05c332961f4704684dce4"), "x" : [ 3, 4, 5 ] }
{ "_id" : ObjectId("51b05c772961f4704684dce5"), "x" : [ 3, 4, 5, 6, 7 ] }
You can use : query3 like:
db.t.find({x:{$in:[3,4]}})
Result3 would look like:
{ "_id" : ObjectId("51b05a722961f4704684daf1"), "x" : [ 3, 4, 5, 6 ] }
{ "_id" : ObjectId("51b05a722961f4704684daf2"), "x" : [ 4, 5, 6, 7 ] }
See this question also : mongodb array matching
So there is an open/unresolved ticket for a $subset operator which does what you likely to.

Mongo Query question $gt,$lt

I have a query below. I want get items between 4 and 6 so only a:1 should match because it has the value 5 in b.
> db.test.find({ b : { $gt : 4 }, b: {$lt : 6}});
{ "_id" : ObjectId("4d54cff54364000000004331"), "a" : 1, "b" : [ 2, 3, 4, 5 ] }
{ "_id" : ObjectId("4d54d0074364000000004332"), "a" : 2, "b" : [ 2, 4, 6, 8 ] }
>
Can someone tell be why a:2 is matching this query? I can't really see why it is being returned.
I also tried what was specified in the tutorial but id did not seem to work:
> db.test.find({ b : { $gt : 4, $lt : 6}});
{ "_id" : ObjectId("4d54cff54364000000004331"), "a" : 1, "b" : [ 2, 3, 4, 5 ] }
{ "_id" : ObjectId("4d54d0074364000000004332"), "a" : 2, "b" : [ 2, 4, 6, 8 ] }
>
And this one to avoid any confusion regarding GT/GTE
> db.test.find({b: {$gt: 4.5, $lt: 5.5}});
{ "_id" : ObjectId("4d54cff54364000000004331"), "a" : 1, "b" : [ 2, 3, 4, 5 ] }
{ "_id" : ObjectId("4d54d0074364000000004332"), "a" : 2, "b" : [ 2, 4, 6, 8 ] }
>
only a:1 should be returned.
As suggested, I gave $elemMatch a try but it did not appear to work either (objectIds are different because I am on a different machine)
> db.test.find();
{ "_id" : ObjectId("4d5a24a5e82e00000000433f"), "a" : 1, "b" : [ 2, 3, 4, 5 ] }
{ "_id" : ObjectId("4d5a24bbe82e000000004340"), "a" : 2, "b" : [ 2, 4, 6, 8 ] }
> db.test.find({b: {$elemMatch: {$gt : 4, $lt: 6 }}});
>
No documents were returned.
This is a really confusing topic. I work at 10gen and I had to spend a while wrapping my head around it ;)
Let's walk through how the query engine processes this query.
Here's the query again:
> db.test.find({ b : { $gt : 4, $lt : 6}});
When it gets to the record that seems like it shouldn't match...
{ "_id" : ObjectId("4d54cff54364000000004331"), "a" : 1, "b" : [ 2, 4, 6, 8 ] }
The match is not performed against each element of the array, but rather against the array as a whole.
The comparison is performed in three steps:
Step 1: Find all documents where b has a value greater than 4
b: [2,4,6,8] matches because 6 & 8 are greater than 4
Step 2: Find all documents where b has a value less than 6
b: [2,4,6,8] matches because 2 & 4 are less than 6
Step 3: Find the set of documents that matched in both step 1 & 2.
The document with b: [2,4,6,8] matched both steps 1 & 2 so it is returned as a match. Note that results are also de-duplicated in this step, so the same document won't be returned twice.
If you want your query to apply to the individual elements of the array, rather than the array as a whole, you can use the $elemMatch operator. For example
> db.temp.find({b: {$elemMatch: {$gt: 4, $lt: 5}}})
> db.temp.find({b: {$elemMatch: {$gte: 4, $lt: 5}}})
{ "_id" : ObjectId("4d558b6f4f0b1e2141b66660"), "b" : [ 2, 3, 4, 5, 6 ] }
$gt
Syntax: {field: {$gt: value} }
eg:
db.inventory.find( { qty: { $gt: 20 } } )
$lt
Syntax: {field: {$lt: value} }
eg:
db.inventory.find( { qty: { $lt: 20 } } )
eg2:
db.inventory.find({ qty : { $gt : 20, $lt : 60}});
.find( {$and:[ {b:{$gt:4}}, {b:{$lt:6}} ]} )
Below is the detailed document for the understanding,
db.test.insertMany([
{"_id":1, "x":11, "a":1, "b":[1]},
{"_id":2, "x":15, "a":4, "b":[1,2,3]},
{"_id":3, "x":19, "a":5, "b":[1,2,3,4,5]},
{"_id":4, "x":13, "a":6, "b":[6,8,10]},
{"_id":5, "x":16, "a":13, "b":[11]},
{"_id":6, "x":18, "a":11, "b":[5]},
{"_id":7, "x":15, "a":15, "b":[3,5,7]},
{"_id":8, "x":12, "a":18, "b":[3,7,9]},
{"_id":9, "x":14, "a":21, "b":[4,6]}
]);
Below queries are included to make idea clear about comparision,
Query-1: db.test.find({b: {$lt: 6}}); //(any element of b) < 6
{ "_id" : 1, "x" : 11, "a" : 1, "b" : [ 1 ] }
{ "_id" : 2, "x" : 15, "a" : 4, "b" : [ 1, 2, 3 ] }
{ "_id" : 3, "x" : 19, "a" : 5, "b" : [ 1, 2, 3, 4, 5 ] }
{ "_id" : 6, "x" : 18, "a" : 11, "b" : [ 5 ] }
{ "_id" : 7, "x" : 15, "a" : 15, "b" : [ 3, 5, 7 ] }
{ "_id" : 8, "x" : 12, "a" : 18, "b" : [ 3, 7, 9 ] }
{ "_id" : 9, "x" : 14, "a" : 21, "b" : [ 4, 6 ] }
`
Query-2: db.test.find({b: {$gt: 4}, b:{$lt : 6}});// it is translated to db.test.find({b:{$lt : 6}}); hence the outcome of Query-1 and Query-2 is the same.
{ "_id" : 1, "x" : 11, "a" : 1, "b" : [ 1 ] }
{ "_id" : 2, "x" : 15, "a" : 4, "b" : [ 1, 2, 3 ] }
{ "_id" : 3, "x" : 19, "a" : 5, "b" : [ 1, 2, 3, 4, 5 ] }
{ "_id" : 6, "x" : 18, "a" : 11, "b" : [ 5 ] }
{ "_id" : 7, "x" : 15, "a" : 15, "b" : [ 3, 5, 7 ] }
{ "_id" : 8, "x" : 12, "a" : 18, "b" : [ 3, 7, 9 ] }
{ "_id" : 9, "x" : 14, "a" : 21, "b" : [ 4, 6 ] }
Query-3: db.test.find({b: {$gt: 4, $lt: 6}});
{ "_id" : 3, "a" : 5, "b" : [ 1, 2, 3, 4, 5 ] }//(element 5) > 4 and (element 5) < 6` => The matching element is same here element 5
{ "_id" : 6, "a" : 11, "b" : [ 5 ] }//(element 5) > 4 and (element 5) < 6 => The matching element is same here element 5
{ "_id" : 7, "a" : 15, "b" : [ 3, 5, 7 ] }//(element 5) > 4 and (element 5) < 6 => The matching element is same here element 5
{ "_id" : 8, "a" : 18, "b" : [ 3, 7, 9 ] }//(element 5) > 7 and (element 3) < 6 => The matching elements are different i.e. here element 5 and element 3
{ "_id" : 9, "a" : 21, "b" : [ 4, 6 ] }//(element 6) > 4 and (element 4) < 6 => The matching elements are different i.e. here element 4 and element 6
Query-4: db.test.find({b: {$elemMatch: {$gt : 4, $lt: 6 }}});
{ "_id" : 3, "a" : 5, "b" : [ 1, 2, 3, 4, 5 ] }//(element 5) > 4 and (element 5) <6
{ "_id" : 6, "a" : 11, "b" : [ 5 ] }//(element 5) > 4 and (element 5) <6
{ "_id" : 7, "a" : 15, "b" : [ 3, 5, 7 ] }//(element 5) > 4 and (element 5) <6
Query-3 and Query-4 are interesting to know about.
Query-3: List document having array b element x>4 and element y<6. The elements x and y may be the same or the different.
Query-4: List document having array b element x>4 and element y<6. The elements x and y must be the same.
Because you did not check the documentation.
See
http://www.mongodb.org/display/DOCS/Advanced+Queries
and check for "ranges" on the page.
Neither is your query syntax correct (compare against the example)
nor does your "why a:2" part of the question make any sense since 'a' is not involved in your query. If you want to search for a:1 then you have to include it in your query.
Keep in mind that all query clauses are AND combined by default unless you use the $or operator.