Database sort on numeric values that are actually strings - mongodb

Question

I have below query. In that field_a is String property and field_b is an array of type Number. I want an array having property field_a and field_b with unique combination. Here field_a contains numeric value but in string format. So I want to apply natural sort in aggregation pipeline. $natural can be used only with such query db.collection.find().sort( { $natural: 1 } )
So how can I use natural sort in MongoDB or I need to depend upon JS functions or on lodash/underscore.js ?
db.collection.aggregate([
{"$group": { "_id": { field_a: "$field_a", field_b: "$field_b" } } },
{ $project: { a: "$_id" } },
{"$group": {"_id": 'a', "res": {"$addToSet": "$_id" }}},
{"$unwind": "$res"},
{"$sort": { "res": 1}},
{"$group": { "_id": null, "res": {"$push": "$res" }}}
])

Answer 1

I short, this is what you want to do here:
db.collection.aggregate([
{ "$group": {
"_id": {
"field_a": {
"$concat": [
{ "$substrCP": [
"0000000000",
0,
{ "$subtract": [ 10, { "$strLenCP": "$field_a" } ] }
]},
"$field_a"
]
},
"field_b": "$field_b"
}
}},
{ "$sort": { "_id": 1 } }
])
Explanation
As a basic concept, the problem you have is that "strings" sort in in a way that does not translate to how numeric sorts work.
As a brief example, these documents use a string value:
{ "_id" : ObjectId("5928276f84c3559bc2fd458b"), "a" : "5" }
{ "_id" : ObjectId("5928277484c3559bc2fd458c"), "a" : "50" }
{ "_id" : ObjectId("5928277e84c3559bc2fd458d"), "a" : "60" }
{ "_id" : ObjectId("5928278284c3559bc2fd458e"), "a" : "6" }
If you try to sort these, then the lexical order applies:
> db.list.find().sort({ "a": 1 })
{ "_id" : ObjectId("5928276f84c3559bc2fd458b"), "a" : "5" }
{ "_id" : ObjectId("5928277484c3559bc2fd458c"), "a" : "50" }
{ "_id" : ObjectId("5928278284c3559bc2fd458e"), "a" : "6" }
{ "_id" : ObjectId("5928277e84c3559bc2fd458d"), "a" : "60" }
As strings, this makes sense. Since a "50" begins with "5" and is therefore less than the "6".
Since the aggregation framework cannot "cast" these as numeric values, then your only option is to present the "strings" in a way in which they will order lexically in the same way as they would with numeric values.
In brief terms we "zero pad" them, which is essentially making the values fixed length strings which are prefixed or "padded" with 0 which makes the "strings" appear to be ordered like they would be numerically:
db.list.aggregate([
{ "$project": {
"a": {
"$concat": [
{ "$substrCP": [
"0000000000",
0,
{ "$subtract": [ 10, { "$strLenCP": "$a" } ] }
]},
"$a"
]
}
}},
{ "$sort": { "a": 1 } }
])
And this will produce a list in order like:
{ "_id" : ObjectId("5928276f84c3559bc2fd458b"), "a" : "0000000005" }
{ "_id" : ObjectId("5928278284c3559bc2fd458e"), "a" : "0000000006" }
{ "_id" : ObjectId("5928277484c3559bc2fd458c"), "a" : "0000000050" }
{ "_id" : ObjectId("5928277e84c3559bc2fd458d"), "a" : "0000000060" }
The basic premise here is that you take a "template" string, which is in this example a string of 0 which is 10 characters long. Then we look at the length of the field data to transform using $strLenCP and $subtract that length from 10 which is the length of the template string used here.
The difference in length is fed to the $substrCP operator as the number of characters to take from the template. This output is then fed to $concat in order to make a "string" which like the template is 10 characters long, but starts with zeros and ends in the initial numeric string.
In your actual usage, this will end up as part of a composite key. Yet since the transformed key is first in the key order, then simply sorting by _id considers this and primarily sorts by values in the first key and then the second.