MongoDB with composite hashed sharding key - mongodb

Question

I'm trying to understand how sharding will behave in a new MongoDB (v 4.4).
Let's say my data is composed of groups (several) that can contain devices (more than groups).
All these devices will store some time-dependent values over a period of time (3-9 months).
My initial idea is to create a device document for each 24 hours of data (in order to avoid hitting the 16MB/document limit).
This way a device will look something like:
{
_id: ObjectId(...)
grp_id: ObjectId(...) // OR DBRef ?
time_bucket: ISODate(...)
data: {...} // data for the last 24 hours specified by time_bucket
}
Now on to the sharding keys.
In order to have something non-monotonic I'm thinking of using {grp_id: "hashed"} since these will be user assigned, hence will not have any monotonicity assigned. Also devices from the smae group will be in the same shard (within chunk limits).
But groups are fairly low cardinality and high frequency, so the sharding key should become: {grp_id: "hashed", _id: 1}.
But this can cause trouble if I insert too much data for a device over time, so refine the key again: {grp_id: "hashed", _id: 1, time_bucket: 1}. This also has the nice ability to localize data, it will target only the shard in a certain time period.
Now to the question(s):
Will this approach work (in theory)?
If I already included grp_id: "hashed" will having _id: 1 in the key have any negative impact since it is a monotonic key?
Will the data be split more evenly between shards if I hash the _id instead of the grp_id (since grp_id is very high frequency and low cardinality compared to devices)? The key will be {_id: "hashed", grp_id: 1, time_bucket: 1}. In this case, will grp_id help with data locality, so devices from the same group will be placed on the same shard?
More generally: I think I'm a bit confused on how the data will be distributed when a hashed key will be used versus another in combination with range based keys and the fact that for range-based keys you can also define zones (which have to be manually managed).
I just tested the two sharding methods, {grp_id: "hashed", _id: 1, time_bucket: 1} (initial) vs {_id: "hashed", grp_id: 1, time_bucket: 1} (question 3):
The first approach yields imbalanced shards:
db.adminCommand({listDatabases:1, filter: {name: "test1"}})
{
"databases" : [
{
"name" : "test1",
"sizeOnDisk" : 22700032,
"empty" : false,
"shards" : {
"sharded-0" : 6377472,
"sharded-1" : 1425408,
"sharded-2" : 4210688,
"sharded-3" : 10686464
}
}
],
"totalSize" : 22700032,
"totalSizeMb" : 21,
"ok" : 1,
"operationTime" : Timestamp(1602680982, 2),
"$clusterTime" : {
"clusterTime" : Timestamp(1602680982, 2),
"signature" : {
"hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="),
"keyId" : NumberLong(0)
}
}
}
While the second one yields balanced shards:
db.adminCommand({listDatabases:1, filter: {name: "test2"}})
{
"databases" : [
{
"name" : "test2",
"sizeOnDisk" : 21512192,
"empty" : false,
"shards" : {
"sharded-0" : 5152768,
"sharded-1" : 5185536,
"sharded-2" : 5218304,
"sharded-3" : 5955584
}
}
],
"totalSize" : 21512192,
"totalSizeMb" : 20,
"ok" : 1,
"operationTime" : Timestamp(1602682074, 1),
"$clusterTime" : {
"clusterTime" : Timestamp(1602682074, 1),
"signature" : {
"hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="),
"keyId" : NumberLong(0)
}
}
}
Both DBs contain 6 groups, each group 1000 devices, each device has 30 instances (data from 30 days).
I wonder what's the reason behind that, but I might ask a separate question.