Turn _ids into Keys of New Object - mongodb

Question

I have a huge bunch of documents as such:
{
_id: '1abc',
colors: [
{ value: 'red', count: 2 },
{ value: 'blue', count: 3}
]
},
{
_id: '2abc',
colors: [
{ value: 'red', count: 7 },
{ value: 'blue', count: 34},
{ value: 'yellow', count: 12}
]
}
Is it possible to make use of aggregate() to get the following?
{
_id: 'null',
colors: {
"1abc": [
{ value: 'red', count: 2 },
{ value: 'blue', count: 3}
],
"2abc": [
{ value: 'red', count: 7 },
{ value: 'blue', count: 34},
{ value: 'yellow', count: 12}
]
}
}
Basically, is it possible to turn all of the original documents' _ids into keys of a new object in the singular new aggregated document?
So far, when trying to use$group, I had not been able to use a variable value, e.g. $_id, on the left hand side of an assignment. Am I missing something or is it simply impossible?
I can do this easily using Javascript but it is unbearably slow. Hence why I am looking to see if it is possible using mongo native aggregate(), which will probably be faster.
If impossible... I would appreciate any kind suggestions that could point towards a sufficient alternative (change structure, etc.?). Thank you!

Answer 1

Like a said in comments, whilst there are things you can do with the aggregation framework or even mapReduce to make the "server" reshape the response, it's kind of silly to do so.
Lets consider the cases:
Aggregate
db.collection.aggregate([
{ "$match": { "_id": { "$in": ["1abc","2abc"] } } },
{ "$group": {
"_id": null,
"result": { "$push": "$$ROOT" }
}},
{ "$project": {
"colors": {
"1abc": {
"$arrayElemAt": [
{ "$map": {
"input": {
"$filter": {
"input": "$result",
"as": "r",
"cond": { "$eq": [ "$$r._id", "1abc" ] },
}
},
"as": "r",
"in": "$$r.colors"
}},
0
]
},
"2abc": {
"$arrayElemAt": [
{ "$map": {
"input": {
"$filter": {
"input": "$result",
"as": "r",
"cond": { "$eq": [ "$$r._id", "2abc" ] },
}
},
"as": "r",
"in": "$$r.colors"
}},
0
]
}
}
}}
])
So the aggregation framework purely does not dynamically generate "keys" of a document. If you want to process this way, then you need to know all of the "values" that you are going to use to make the keys in the result.
After putting everything into one document with $group, you can then work with the result array to extact data for your "keys". The basic operators here are:
$filter to get the matched element of the array for the "value" that you want.
$map to return just the specific property from the filtered array
$arrayElemAt to just grab the single elment that was filtered out of the resulting mapped array
So it really isn't practical in a lot of cases, and the coding of the statement is fairly involved.
MapReduce
db.collection.mapReduce(
function() {
var obj = { "colors": {} };
obj.colors[this._id] = this.colors;
emit(null,obj);
},
function(key,values) {
var obj = { "colors": {} };
values.forEach(function(value) {
Object.keys(value.colors).forEach(function(key) {
obj.colors[key] = value.colors[key];
});
})
return obj;
},
{ "out": { "inline": 1 } }
)
Since it is actually written in a "language" then you have the ability to loop structures and "build things" in a more dynamic way.
However, close inspection should tell you that the "reducer" function here is not doing anything more than being the processor of "all the results" which have been "stuffed into it" but each emitted document.
That means that "iterating the values" fed to the reducer is really no different to "iterating the cursor", and that leads to the next conclusion.
Cursor Iteration
var result = { "colors": {} };
db.collection.find().forEach(function(doc) {
result.colors[doc._id] = doc.colors;
})
printjson(result)
The simplicity of this should really speak volumes. It is afterall doing exactly what you are trying to "shoehorn" into a server operation and nothing more, and just simply "rolls up it sleeves" and gets on with the task at hand.
The key point here is none of the process requires any "aggregation" in a real sense, that cannot be equally achieved by simply iterating the cursor and building up the response document.
This is really why you always need to look at what you are doing and choose the right method. "Server side" aggregation has a primary task of "reducing" a result so you would not need to iterate a cursor. But nothing here "reduces" anything. It's just all of the data, transformed into a different format.
Therefore the simple approach for this type of "transform" is to just iterate the cursor and build up your transformed version of "all the results" anyway.