MongoDB set of values with a limit size - mongodb

Question

I am updating a list of transactions by saving the transaction into the database list, I do not want to have duplicate entries in the list so I use $addtoset
this is because the request can be fired multiple times and we want to make sure that any changes are idempotent to the database. the only catch now is that we want to only store the latest 20 transactions
this could be done with a $push $sort $slice but I need to make sure duplicate entries are not available. there was a feature request to mongo back in 2015 for this to be added to the $addtoset feature, but they declined this due to 'sets' not being in an order...
which is what the $sort function would have been
I thought I could simply append an empty push update to the update object, but from what I understand, each update is potentially threaded and can lead to undesirable edits if the push/slice fires before the $addtoset
right now, the values are an aggregated string with the following formula
timestamp:value but I can easily change the structure to an object
{ts:timestamp, value:value}
Update:
current code, not sure if it will work as intended as each operation maybe independent
await historyDB
.updateOne(
{ trxnId: txid },
{
$addToSet: {
history: {
ts: time,
bid: bid.value,
txid: trxn.txid,
}
},
$push: {
history: {
$each: [{ts:-1}],
$sort: { ts: 1 },
$slice: -10,
},
},
},
{ upsert: true },
).exec();

Answer 1

Your query doesn't work, as you are trying to update history multiple times, which is not allowed in simple update document and raises error Updating the path 'history' would create a conflict at 'history'.
You can however subsequently update history field multiple times with aggregation pipeline.
await historyDB.updateOne(
{ trxnId: txid},
[{
$set: {
history: {
$let: {
vars: {
historyObj: {
ts: time,
bid: bid.value,
txid: trxn.txid,
},
historySafe: { $ifNull: ["$history", []] }
},
in: {
$cond: {
if: { $in: ["$$historyObj", "$$historySafe"] },
then: "$history",
else: { $concatArrays: [ "$$historySafe", ["$$historyObj"] ] }
}
}
}
}
},
},
{
$set: {
history: {
$function: {
body: function(entries) {
entries.sort((a, b) => a.ts - b.ts);
return entries;
},
args: [{ $ifNull: ["$history", []] }],
lang: "js"
}
}
},
},
{
$set: {
history: {
$slice: [ "$history", -10 ]
}
}
}],
{ upsert: true },
).exec()
As of MongoDB 6.0, the second $set stage, which provides sorting, can be replaced with $sortArray operator (see here).