MongoDB sort vs aggregate $sort on array index - mongodb

Question

With a MongoDB collection test containing the following documents:
{ "_id" : 1, "color" : "blue", "items" : [ 1, 2, 0 ] }
{ "_id" : 2, "color" : "red", "items" : [ 0, 3, 4 ] }
if I sort them in reversed order based on the second element in the items array, using
db.test.find().sort({"items.1": -1})
they will be correctly sorted as:
{ "_id" : 2, "color" : "red", "items" : [ 0, 3, 4 ] }
{ "_id" : 1, "color" : "blue", "items" : [ 1, 2, 0 ] }
However, when I attempt to sort them using the aggregate function:
db.test.aggregate([{$sort: {"items.1": -1} }])
They will not sort correctly, even though the query is accepted as valid:
{
"result" : [
{
"_id" : 1,
"color" : "blue",
"items" : [
1,
2,
0
]
},
{
"_id" : 2,
"color" : "red",
"items" : [
0,
3,
4
]
}
],
"ok" : 1
}
Why is this?

Answer 1

The aggregation framework just does not "deal with" arrays in the same way as is applied to .find() queries in general. This is not only true of operations like .sort(), but also with other operators, and namely $slice, though that example is about to get a fix ( more later ).
So it pretty much is impossible to deal with anything using the "dot notation" form with an index of an array position as you have. But there is a way around this.
What you "can" do is basically work out what the "nth" array element actually is as a value, and then return that as a field that can be sorted:
db.test.aggregate([
{ "$unwind": "$items" },
{ "$group": {
"_id": "$_id",
"items": { "$push": "$items" },
"itemsCopy": { "$push": "$items" },
"first": { "$first": "$items" }
}},
{ "$unwind": "$itemsCopy" },
{ "$project": {
"items": 1,
"itemsCopy": 1,
"first": 1,
"seen": { "$eq": [ "$itemsCopy", "$first" ] }
}},
{ "$match": { "seen": false } },
{ "$group": {
"_id": "$_id",
"items": { "$first": "$items" },
"itemsCopy": { "$push": "$itemsCopy" },
"first": { "$first": "$first" },
"second": { "$first": "$itemsCopy" }
}},
{ "$sort": { "second": -1 } }
])
It's a horrible and "iterable" approach where you essentially "step through" each array element by getting the $first match per document from the array after processing with $unwind. Then after $unwind again, you test to see if that array elements are the same as the one(s) already "seen" from the identified array positions.
It's terrible, and worse for the more positions you want to move along, but it does get the result:
{ "_id" : 2, "items" : [ 0, 3, 4 ], "itemsCopy" : [ 3, 4 ], "first" : 0, "second" : 3 }
{ "_id" : 1, "items" : [ 1, 2, 0 ], "itemsCopy" : [ 2, 0 ], "first" : 1, "second" : 2 }
{ "_id" : 3, "items" : [ 2, 1, 5 ], "itemsCopy" : [ 1, 5 ], "first" : 2, "second" : 1 }
Fortunately, upcoming releases of MongoDB ( as currently available in develpment releases ) get a "fix" for this. It may not be the "perfect" fix that you desire, but it does solve the basic problem.
There is a new $slice operator available for the aggregation framework there, and it will return the required element(s) of the array from the indexed positions:
db.test.aggregate([
{ "$project": {
"items": 1,
"slice": { "$slice": [ "$items",1,1 ] }
}},
{ "$sort": { "slice": -1 } }
])
Which produces:
{ "_id" : 2, "items" : [ 0, 3, 4 ], "slice" : [ 3 ] }
{ "_id" : 1, "items" : [ 1, 2, 0 ], "slice" : [ 2 ] }
{ "_id" : 3, "items" : [ 2, 1, 5 ], "slice" : [ 1 ] }
So you can note that as a "slice", the result is still an "array", however the $sort in the aggregation framework has always used the "first position" of the array in order to sort the contents. That means that with a singular value extracted from the indexed position ( just as the long procedure above ) then the result will be sorted as you expect.
The end cases here are that is just how it works. Either live with the sort of operations you need from above to work with a indexed position of the array, or "wait" until a brand new shiny version comes to your rescue with better operators.