How to take the most newest part of all reports and merge into one main report? - mongodb

Question

EDIT: I have found out that mongo does not allow to use special characters such as dots and the dollar sign as a key in the report so I had to rechange the structure of the JSON a bit. But, my question remains the same (I removed the old stuff so it will be more readable but you can still see it in the edit history section). The new structure looks as follows:
{
"name": "test1",
"main": [
{
"subs": [
{
"data": [
{
"group": "ABC",
"values": [
"tcsh"
]
},
{
"group": "AA",
"values": [
"6.13.00"
]
}
]
},
{
"data": [
{
"group": "xyz",
"values": [
"tcsh"
]
},
{
"group": "SADA",
"values": [
"6.13.00"
]
}
]
}
],
"main_name": "MAIN",
"main_path": "play_ground/MAIN"
},
{
"subs": [
{
"data": [
{
"group": "BAB",
"values": [
"tcsh"
]
},
{
"group": "GO",
"values": [
"6.13.00"
]
}
]
}
],
"main_name": "MAIN2",
"main_path": "play_ground/MAIN2"
}
],
"user": "easdasa",
"timestamp": "1564437533"
}
I want to get all reports that have a name test1 and a user easdasa. Then, I would like to take the latest block of data of each block of subs. This is done with the help of the timestamp.
For example in the following array I have two reports:
[{
"name": "test1",
"main": [
{
"subs": [
{
"data": [
{
"group": "xyz",
"values": [
"tcsh"
]
},
{
"group": "SADA",
"values": [
"6.13.00"
]
}
]
}
],
"main_name": "MAIN",
"main_path": "play_ground/MAIN"
}
],
"timestamp": "1564437533"
},
{
"name": "test1",
"main": [
{
"subs": [
{
"data": [
{
"group": "ABC",
"values": [
"tcsh"
]
},
{
"group": "AA",
"values": [
"6.13.00"
]
}
]
},
{
"data": [
{
"group": "xyz",
"values": [
"tcsh"
]
},
{
"group": "SADA",
"values": [
"5.0.1",
"12312"
]
}
]
}
],
"main_name": "MAIN",
"main_path": "play_ground/MAIN"
}
],
"timestamp": "1564437522"
}]
The first report is was created after the second report (due to the timestamp). I can see that there is a block that located in the second report but not in the first report:
{
"data": [
{
"group": "ABC",
"values": [
"tcsh"
]
},
{
"group": "AA",
"values": [
"6.13.00"
]
}
]
},
So I want the final report to heve it (besides all the blocks from the first report). Also, you can see that the values of the SADA group are diffrenet. So we want to take the first's report block. The final report should be:
{
"name": "test1",
"main": [
{
"subs": [
{
"data": [
{
"group": "ABC",
"values": [
"tcsh"
]
},
{
"group": "AA",
"values": [
"6.13.00"
]
}
]
},
{
"data": [
{
"group": "xyz",
"values": [
"tcsh"
]
},
{
"group": "SADA",
"values": [
"6.13.00"
]
}
]
}
],
"main_name": "MAIN",
"main_path": "play_ground/MAIN"
}
],
"timestamp": "1564437533"
}
In other words, in the (json) values of the data level I want to get the latest report and in the (json) values of the subs level I want to get all existing subs. So it will be more clear, in the (json) values of the data level I want to get all the groups and values of the latest report and for the (json) values of the subs level I want to have all the subs.
If I could specify steps:
Get all reports by user and name.
Theoritcly merge all report into one main report (the implmenetation could be diffrent). The merge will use be done by main_name.
Remove all old subs values by timestamp that already exists in the latest report so the final report will have in the subs level only the newest objects and object from the old reports that were not in the newer reports.
Which query I should be in order to get the wanted report?

Answer 1

Please use the below query and check on stats, I can really say performance can be improved by having proper indexing as per your requirements(querying), Please use $explain (enter link description here)
to check on query performance. I've considered your array exists in a field with key as values , Please let me know if this works or if it doesn't provide sample data, we can check on that:
db.getCollection('yourcollection').aggregate([{$unwind: '$values'},{$match : {'values.name': 'test1', 'values.user': 'galih'}},
{$sort: {'values.timestamp' : -1}},
{$limit: 1}
])