Update table with merge in mongodb - mongodb

Question

I have a collection with values associated to the sells of (almost a million) different products by day, and I have to create the collection with the aggregation per week. I do it with the following (working) query.
Brief explanation:
I filter the dates I want to use in the query.
I convert the weird date format to a real date.
I group by name of the object, year and week, getting the sum per week.
I group again by name to have all dates in the same document.
I save it to a table.
[
{
$match:
/**
* query: The query in MQL.
*/
{
$and: [
{
"_id.date": {
$gte: "20220103",
},
},
{
"_id.date": {
$lte: "20230122",
},
},
],
},
},
{
$project:
/**
* specifications: The fields to
* include or exclude.
*/
{
_id: 1,
realDate: {
$dateFromString: {
dateString: "$_id.date",
format: "%Y%m%d",
},
},
count: 1,
},
},
{
$group:
/**
* _id: The id of the group.
* fieldN: The first field name.
*/
{
_id: {
name: "$_id.name",
year: {
$isoWeekYear: "$realDate",
},
week: {
$isoWeek: "$realDate",
},
},
total: {
$sum: "$count",
},
},
},
{
$group:
/**
* _id: The id of the group.
* fieldN: The first field name.
*/
{
_id: "$_id.name",
dates: {
$addToSet: {
year: "$_id.year",
week: "$_id.week",
count: "$total",
},
},
},
},
{
$merge:
/**
* into: The target collection.
* on: Fields to identify.
* let: Defined variables.
* whenMatched: Action for matching docs.
* whenNotMatched: Action for non-matching docs.
*/
{
into: "dataPerWeek",
on: "_id",
},
},
]
That works, and generate documents like:
{
"_id": "myProduct",
"dates": [
{
"year": {
"$numberLong": "2022"
},
"week": 52,
"count": 10
},
{
"year": {
"$numberLong": "2022"
},
"week": 50,
"count": 6
},
{
"year": {
"$numberLong": "2022"
},
"week": 49,
"count": 2
},
{
"year": {
"$numberLong": "2022"
},
"week": 51,
"count": 5
},
{
"year": {
"$numberLong": "2023"
},
"week": 1,
"count": 5
},
{
"year": {
"$numberLong": "2023"
},
"week": 2,
"count": 2
},
{
"year": {
"$numberLong": "2023"
},
"week": 3,
"count": 4
}
]
}
Now, I would want now to update this list every week adding only the new elements to the array (or creating a new object if it does not exist. But, if I repeat the merge query above limiting the dates to the last week, it basically removes all other data points. Is is possible to do this "update" with a single query?

Answer 1

You should store date values as Date objects. Storing date values as string is a design flaw.
Your pipeline can be shorter and your $merge stage would be like this:
[
{
$match: {
"_id.date": {
$gte: ISODate("2022-01-03"),
$lte: ISODate("2023-01-22"),
}
}
},
{
$group: {
_id: {
name: "$_id.name",
week: { $dateTrunc: { date: "_id.date", unit: "week", startOfWeek: "monday" } }
},
total: { $sum: "$count" },
}
},
{
$group: {
_id: "$_id.name",
dates: {
$push: { // $push should be faster than $addToSet, result is the same
year: { $isoWeekYear: "$_id.week" },
week: { $isoWeek: "$_id.week" },
count: "$total",
}
}
}
},
{
$merge: {
into: "dataPerWeek",
on: "_id",
whenMatched: [
{ $set: { dates: { $concatArrays: ["$dates", "$$new.dates"] } } }
]
}
}
]
The dates elements are simply concatenated. If you like to update existing elements then you need to iterate over all elements with $map