How to make a interval average on each nested array with MongoDB - mongodb

Question

We are using the size bucket pattern for our datas and we are wondering how could we aggregate the nested array for getting a average every 1, 5 or 10 minutes on this nested array. The structure is as follow:
{
"_id" : ObjectId("62ad1548a7d67adf4943d2f7"),
"data" : ObjectId("6262a3048cbdacd24c38fbc4"),
"day" : ISODate("2022-06-17T00:00:00.000Z"),
"first" : ISODate("2022-06-17T23:59:04.659Z"),
"last" : ISODate("2022-06-17T23:59:59.915Z"),
"nSamples" : 97,
"samples" : [
{
"time" : ISODate("2022-06-17T23:59:04.659Z"),
"value" : 8
},
{
"time" : ISODate("2022-06-17T23:59:50.706Z"),
"value" : 6
},
///// MAX 200 Documents in nested array
{
"time" : ISODate("2022-06-17T23:59:59.342Z"),
"value" : 4
},
{
"time" : ISODate("2022-06-17T23:59:59.915Z"),
"value" : 12
}
],
"version" : 0
}
{
"_id" : ObjectId("62ad1548a7d67adf4943d2f7"),
"data" : ObjectId("62629d178cbdac1e9938fb9e"),
"day" : ISODate("2022-06-17T00:00:00.000Z"),
"first" : ISODate("2022-06-17T23:32:04.659Z"),
"last" : ISODate("2022-06-17T23:40:59.915Z"),
"nSamples" : 97,
"samples" : [
{
"time" : ISODate("2022-06-17T23:32:04.659Z"),
"value" : 2
},
{
"time" : ISODate("2022-06-17T23:33:50.706Z"),
"value" : 65
},
///// MAX 200 Documents in nested array
{
"time" : ISODate("2022-06-17T23:36:59.342Z"),
"value" : 45
},
{
"time" : ISODate("2022-06-17T23:40:59.915Z"),
"value" : 22
}
],
"version" : 0
}
Is it possible to get this structure ?
{
"_id" : ObjectId("62ad1548a7d67adf4943d2f7"),
"data" : ObjectId("6262a3048cbdacd24c38fbc4"),
"day" : ISODate("2022-06-17T00:00:00.000Z"),
"first" : ISODate("2022-06-17T23:59:04.659Z"),
"last" : ISODate("2022-06-17T23:59:59.915Z"),
"nSamples" : 97,
"samples" : [
// Less documents
{
"time" : ISODate("2022-06-17T23:56:00.000Z"),
"value" : 7
},
{
"time" : ISODate("2022-06-17T23:58:00.000Z"),
"value" : 6
},
{
"time" : ISODate("2022-06-17T23:59:00.000Z"),
"value" : 4
},
],
"version" : 0
}
{
"_id" : ObjectId("62ad1548a7d67adf4943d2f7"),
"data" : ObjectId("62629d178cbdac1e9938fb9e"),
"day" : ISODate("2022-06-17T00:00:00.000Z"),
"first" : ISODate("2022-06-17T23:59:04.659Z"),
"last" : ISODate("2022-06-17T23:59:59.915Z"),
"nSamples" : 97,
"samples" : [
// Less docs
{
"time" : ISODate("2022-06-17T23:32:00.000Z"),
"value" : 2
},
{
"time" : ISODate("2022-06-17T23:33:00.000Z"),
"value" : 65
},
{
"time" : ISODate("2022-06-17T23:36:00.000Z"),
"value" : 45
},
{
"time" : ISODate("2022-06-17T23:37:00.000Z"),
"value" : 22
}
],
"version" : 0
}
So far, I first start by a $match for selecting values and add a "fromDate" value for searching:
$match: {
$and: [
{
data: {"$in": [ObjectId("62629d178cbdac1e9938fb9e"), ObjectId("62629d178cbdac1e9938fb9f"), ObjectId("62629d178cbdac1e9938fb9d")]} ,
},
{
first: {
$gte: new Date('2022-07-08T08:10:22.525+00:00')
}
}
],
}
Then by using the $unwind aggregate on samples
$unwind: {
path: "$samples",
preserveNullAndEmptyArrays: true
}
And then grouping the values with an interval:
$group: {
_id: {
"$toDate": {
"$subtract": [
{ "$toLong": "$samples.time" },
{ "$mod": [ { "$toLong": "$samples.time" }, 1000 * 60 * 1 ] } // 1 for 1 minute could be replaced by 5, 10 etc...
]
}
},
values: {
$avg: "$samples.value"
}
}
The problem here is that it does not take in consideration that I have multiple "data"... How can I group them by data as well ? Is it possible to keep the same structure and having "less root documents" ?

Answer 1

One option to continue your query is:
db.collection.aggregate([
{$match: {
data: {"$in": [
ObjectId("62629d178cbdac1e9938fb9e"),
ObjectId("6262a3048cbdacd24c38fbc4"),
ObjectId("62629d178cbdac1e9938fb9d")
]
},
first: {$gte: ISODate("2021-07-08T08:10:22.525+00:00")}
}
},
{$unwind: {path: "$samples", preserveNullAndEmptyArrays: true}},
{$group: {
_id: {
time: {$toDate: {
$subtract: [
{$toLong: "$samples.time"},
{$mod: [{$toLong: "$samples.time"}, 60000]}
]
}
},
orig_id: "$_id"
},
values: {$avg: "$samples.value"},
root: {$first: "$$ROOT"}
}
},
{$sort: {"_id.time": 1}},
{$group: {_id: "$_id.orig_id", root: {$first: "$root"},
samples: {$push: {time: "$_id.time", value: "$values"}}}
},
{$set: {"root.samples": "$samples", "root._id": "$_id"}},
{$replaceRoot: {newRoot: "$root"}}
])
See how it works on the playground example