MongoDB combining group aggregation and strLenBytes - mongodb

Question

I've a Mongo collection with documents like this:
{
"_id" : ObjectId("5a9d0d44c3a1ce5f14c6940a"),
"topic_id" : "5a7af30613b79405643e7da1",
"value" : "VMware Virtual Platform",
"timestamp" : "2018-03-05 09:26:25.136546",
"insert_ts" : "2018-03-05 09:26:25.136682",
"inserted_by" : 1
},
{
"_id" : ObjectId("5a9d0d44c3a1ce5f14c69409"),
"topic_id" : "5a7af30713b79479f82b4b84",
"value" : "VMware, Inc.",
"timestamp" : "2018-03-05 09:26:25.118931",
"insert_ts" : "2018-03-05 09:26:25.119081",
"inserted_by" : 1
},
{
"_id" : ObjectId("5a9d0d44c3a1ce5f14c69408"),
"topic_id" : "5a7af30713b7946d6d0a8772",
"value" : "Phoenix Technologies LTD 6.00 09/21/2015",
"timestamp" : "2018-03-05 09:26:25.101624",
"insert_ts" : "2018-03-05 09:26:25.101972",
"inserted_by" : 1
}
I would like to fetch some aggregated data from this collection. I want to know the oldest timestamp, the documents count and the total strlen of all values, but grouped by topic_id, where the document-id is greater than x.
In mysql, i would build a sql like this:
SELECT
MAX(_id) as max_id,
COUNT(*) as message_count,
MIN(timestamp) as min_timestamp,
LENGTH(GROUP_CONCAT(value)) as size
FROM `dev_topic_data_numeric`
WHERE _id > 22000
GROUP BY topic_id
How do i achieve this in MongoDB? I already tried to build it looking like this:
db.getCollection('topic_data_text').aggregate(
[
{
"$match":
{
"_id": {"$gte": ObjectId("5a9d0aefc3a1ce5f14c68c81") }
}
},
{
"$group":
{
"_id": "$topic_id",
"max_id": {"$max":"$_id"},
"min_timestamp": {"$min": "$timestamp"},
"message_count": {"$sum": 1},
/*"size": {"$strLenBytes": "$value" }*/
}
}
]
);
Then i uncomment $strLenBytes it crashes saying, that strLenBytes is not a group operator. The API of MongoDB does not help me here. How do i have to write it to get the strlen?
My expected result should look like this:
{
"_id" : "5a7af30613b79405643e7da1",
"max_id" : ObjectId("5a9d0d44c3a1ce5f14c6940a"),
"min_timestamp" : "2018-03-05 09:26:25.136546",
"message_count" : 1,
"size" : 23,
}
My MongoDB version is 3.4.4.

Answer 1

This is because $strLenBytes is not an accumulator, unlike $sum or $max. The $group stage accumulates values, so any operator that is valid in the $group stage are typically accumulators.
$strLenBytes converts one value to another in a 1-1 fashion. This is typically an operator for the $project stage.
Adding a $project stage in your aggregation should give you the result you require. Note that you would also need to modify the $group stage slightly to pass on the required values:
> db.test.aggregate([
{
"$match":
{
"_id": {"$gte": ObjectId("5a9d0aefc3a1ce5f14c68c81") }
}
},
{
"$group":
{
"_id": {"topic_id": "$topic_id", value: "$value"},
"max_id": {"$max":"$_id"},
"min_timestamp": {"$min": "$timestamp"},
"message_count": {"$sum": 1}
}
},
{
"$project":
{
"_id": "$_id.topic_id",
"max_id": "$max_id",
"min_timestamp": "$min_timestamp",
"message_count": "$message_count",
size: {"$strLenBytes": "$_id.value" }
}
}
])
Output using your example documents:
{
"_id": "5a7af30613b79405643e7da1",
"max_id": ObjectId("5a9d0d44c3a1ce5f14c6940a"),
"min_timestamp": "2018-03-05 09:26:25.136546",
"message_count": 1,
"size": 23
}
{
"_id": "5a7af30713b79479f82b4b84",
"max_id": ObjectId("5a9d0d44c3a1ce5f14c69409"),
"min_timestamp": "2018-03-05 09:26:25.118931",
"message_count": 1,
"size": 12
}
{
"_id": "5a7af30713b7946d6d0a8772",
"max_id": ObjectId("5a9d0d44c3a1ce5f14c69408"),
"min_timestamp": "2018-03-05 09:26:25.101624",
"message_count": 1,
"size": 40
}

Answer 2

After testing #kevin-adistambha's answer and some further experimenting, I found another way to achieve my wanted result - and maybe it has a better performance - but that needs more testing to be sure about this.
db.getCollection('topic_data_text').aggregate(
[
{
"$match":
{
"_id": {"$gt": ObjectId("5a9f9d8bd5de3ac75f8cc269") }
}
},
{
"$group":
{
"_id": "$topic_id",
"max_id": {"$max":"$_id"},
"min_timestamp": {"$min": "$timestamp"},
"message_count": {"$sum": 1},
"size": {"$sum": {"$strLenBytes": "$value"}}
}
}
]
);