Conditionally update/upsert embedded array with findOneAndUpdate in MongoDB - mongodb

I have a collection in the following format:
[
{
"postId": ObjectId("62dffd0acb17483cf015375f"),
"userId": ObjectId("62dff9584f5b702d61c81c3c"),
"state": [
{
"id": ObjectId("62dffc49cb17483cf0153220"),
"notes": "these are my custom notes!",
"lvl": 3,
},
{
"id": ObjectId("62dffc49cb17483cf0153221"),
"notes": "hello again",
"lvl": 0,
},
]
},
]
My goal is to be able to update and add an element in this array in the following situation:
If the ID of the new element is not in the state array, push the new element in the array
If the ID of the new element is in the state array and its lvl field is 0, update that element with the new information
If the ID of the new element exists in the array, and its lvl field is not 0, then nothing should happen. I will throw an error by seeing that no documents were matched.
Basically, to accomplish this I was thinking about using findOneAndUpdate with upsert, but I am not sure how to tell the query to update the state if lvl is 0 or don't do anything if it is bigger than 0 when the match is found.
For solving (1) this is what I was able to come up with:
db.collection.findOneAndUpdate(
{
"postId": ObjectId("62dffd0acb17483cf015375f"),
"userId": ObjectId("62dff9584f5b702d61c81c3c"),
"state.id": {
"$ne": ObjectId("62dffc49cb17483cf0153222"),
},
},
{
"$push": {"state": {"id": ObjectId("62dffc49cb17483cf0153222"), "lvl": 1}}
},
{
"new": true,
"upsert": true,
}
)
What is the correct way to approach this issue? Should I just split the query into multiple ones?
Edit: as of now I have done this in more than one query (one to fetch the document, then I iterate over its state array to check if the ID exists in it, and then I perform (1), (2) and (3) in a normal if-else clause)

If the ID of the new element exists in the array, and its lvl field is not 0, then nothing should happen. I will throw an error by seeing that no documents where matched.
First thing FYI,
upsert is not possible in the nested array
upsert will not add new elements to the array
upsert can add a new document with the new element
if you want to throw an error if the record does not present then you don't need upsert
Second thing, you can achieve this in one query by using an update with aggregation pipeline in MongoDB 4.2,
Note: Here i must inform you, this query will respond updated document but there will be no flag or any clue if this query fulfilled your first situation or second situation, or the third situation out of 3, you have to check in your client-side code through query response.
check conditions for postId and userId fields only
we are going to update state field under $set stage
check the condition if the provided id is present in state's id?
true, $map to iterate loop of state array
check conditions for id and lvl: 0?
true, $mergeObjects to merge current object with the new information
false, it will not do anything
false, then add that new element in state array, by $concatArrays operator
db.collection.findOneAndUpdate(
{
postId: ObjectId("62dffd0acb17483cf015375f"),
userId: ObjectId("62dff9584f5b702d61c81c3c")
},
[{
$set: {
state: {
$cond: [
{ $in: [ObjectId("62dffc49cb17483cf0153221"), "$state.id"] },
{
$map: {
input: "$state",
in: {
$cond: [
{
$and: [
{ $eq: ["$$this.id", ObjectId("62dffc49cb17483cf0153221")] },
{ $eq: ["$$this.lvl", 0] }
]
},
{
$mergeObjects: [
"$$this",
{
// update your new fields here
"notes": "new note"
}
]
},
"$$this"
]
}
}
},
{
$concatArrays: [
"$state",
[
// add new element
{
"id": ObjectId("62dffc49cb17483cf0153221"),
"lvl": 1
}
]
]
}
]
}
}
}],
{ returnNewDocument: true }
)
Playrgound
Third thing, you can execute 2 update queries,
The first query, for the case: element does not present and it will push a new element in state
let response = db.collection.findOneAndUpdate({
postId: ObjectId("62dffd0acb17483cf015375f"),
userId: ObjectId("62dff9584f5b702d61c81c3c"),
"state.id": { $ne: ObjectId("62dffc49cb17483cf0153221") }
},
{
$push: {
state: {
id: ObjectId("62dffc49cb17483cf0153221"),
lvl: 1
}
}
},
{
returnNewDocument: true
})
The second query on the base of if the response of the above query is null then this query will execute,
This will check state id and lvl: 0 conditions if conditions are fulfilled then execute the update fields operation, it will return null if the document is not found
You can throw if this will return null otherwise do stuff with response data and response success
if (response == null) {
response = db.collection.findOneAndUpdate({
postId: ObjectId("62dffd0acb17483cf015375f"),
userId: ObjectId("62dff9584f5b702d61c81c3c"),
state: {
$elemMatch: {
id: ObjectId("62dffc49cb17483cf0153221"),
lvl: 0
}
}
},
{
$set: {
// add your update fields
"state.$.notes": "new note"
}
},
{
returnNewDocument: true
});
// not found and throw an error
if (response == null) {
return {
// throw error;
};
}
}
// do stuff with "response" data and return result
return {
// success;
};
Note: As per the above options, I would recommend you that I explained in the Third thing that you can execute 2 update queries.

What you're trying became possible with the introduction pipelined updates, here is how I would do it by using $concatArrays to concat the exists state array with the new input and $ifNull in case of an upsert to init the empty value, like so:
const inputObj = {
"id": ObjectId("62dffc49cb17483cf0153222"),
"lvl": 1
};
db.collection.findOneAndUpdate({
"postId": ObjectId("62dffd0acb17483cf015375f"),
"userId": ObjectId("62dff9584f5b702d61c81c3c")
},
[
{
$set: {
state: {
$ifNull: [
"$state",
[]
]
},
}
},
{
$set: {
state: {
$concatArrays: [
{
$map: {
input: "$state",
in: {
$mergeObjects: [
{
$cond: [
{
$and: [
{
$in: [
inputObj.id,
"$state.id"
]
},
{
$eq: [
inputObj.lvl,
0
]
}
]
},
inputObj,
{},
]
},
"$$this"
]
}
}
},
{
$cond: [
{
$not: {
$in: [
inputObj.id,
"$state.id"
]
}
},
[
],
[]
]
}
]
}
}
}
],
{
"new": true,
"upsert": true
})
Mongo Playground
Prior to version 4.2 and the introduction of this feature what you're trying to do was not possible using the naive update syntax, If you are using an older version then you'd have to split this into 2 separate calls, first a findOne to see if the document exists, and only then an update based on that. obviously this can cause stability issue's if you have high update volume.

Related

MongoDB set of values with a limit size

I am updating a list of transactions by saving the transaction into the database list, I do not want to have duplicate entries in the list so I use $addtoset
this is because the request can be fired multiple times and we want to make sure that any changes are idempotent to the database. the only catch now is that we want to only store the latest 20 transactions
this could be done with a $push $sort $slice but I need to make sure duplicate entries are not available. there was a feature request to mongo back in 2015 for this to be added to the $addtoset feature, but they declined this due to 'sets' not being in an order...
which is what the $sort function would have been
I thought I could simply append an empty push update to the update object, but from what I understand, each update is potentially threaded and can lead to undesirable edits if the push/slice fires before the $addtoset
right now, the values are an aggregated string with the following formula
timestamp:value but I can easily change the structure to an object
{ts:timestamp, value:value}
Update:
current code, not sure if it will work as intended as each operation maybe independent
await historyDB
.updateOne(
{ trxnId: txid },
{
$addToSet: {
history: {
ts: time,
bid: bid.value,
txid: trxn.txid,
}
},
$push: {
history: {
$each: [{ts:-1}],
$sort: { ts: 1 },
$slice: -10,
},
},
},
{ upsert: true },
).exec();
Your query doesn't work, as you are trying to update history multiple times, which is not allowed in simple update document and raises error Updating the path 'history' would create a conflict at 'history'.
You can however subsequently update history field multiple times with aggregation pipeline.
await historyDB.updateOne(
{ trxnId: txid},
[{
$set: {
history: {
$let: {
vars: {
historyObj: {
ts: time,
bid: bid.value,
txid: trxn.txid,
},
historySafe: { $ifNull: ["$history", []] }
},
in: {
$cond: {
if: { $in: ["$$historyObj", "$$historySafe"] },
then: "$history",
else: { $concatArrays: [ "$$historySafe", ["$$historyObj"] ] }
}
}
}
}
},
},
{
$set: {
history: {
$function: {
body: function(entries) {
entries.sort((a, b) => a.ts - b.ts);
return entries;
},
args: [{ $ifNull: ["$history", []] }],
lang: "js"
}
}
},
},
{
$set: {
history: {
$slice: [ "$history", -10 ]
}
}
}],
{ upsert: true },
).exec()
As of MongoDB 6.0, the second $set stage, which provides sorting, can be replaced with $sortArray operator (see here).

MongoDB conditionial update

I am running a bit in circles here and would appreciate some help. What I am looking to do is either update or create a nested object contained in an array depending on whether this object exists.
I have a users collection and a user document has the following structure:
{
schema_version: 1,
display_name: 'xxxxxx',
email: 'xxxxxx',
email_verified:'xxxxxx',
...
custom_data: {
stripe_id: 'xxx',
subscriptions: [{
subscription_id: xxxx,
....
}],
...
},
}
In webhook calls from Stripe I am getting a subscription object with a subscription_id and a stripe_id.
What I want to do is check if subscription_id exists, if so, update the document, if not then create the document in the subscriptions array for the user document where stripe_id matches.
If I do something along the lines of:
db.collection.update(
{subscription_id: subscription.id},
{ $set: { 'custom_data.subscriptions': subscriptionData } },
{ upsert: true }
)
The problem is that I am creating subscription objects not bound to my user document where stripeID matches.
On the other hand, if I do something like this:
db.collection.update(
{'custom_data.stripe_id': stripe_id},
{ $set: { 'custom_data.subscriptions': subscriptionData } },
{ upsert: true }
)
I will potentially end up creating dupes in the subscriptions array when, in fact I would want to update the existing object where subscription_id matches.
Is there any way to do that in one query with Mongo, or will I have to resort to using 2 queries in an if statement?
Thanks in advance for any clarification on this.
You can do the followings with an aggregation pipeline:
$match with $or condition to search for custom_data.subscriptions.subscription_id or custom_data.stripe_id
$addFields with $map to conditional update your subscription object when matched
$addFields with $setUnion to insert an entry of incoming subscription object for the insert case
$merge to update the back into the original collection
db.collection.aggregate([
{
$match: {
$expr: {
$or: [
{
$eq: [
"$custom_data.subscriptions.subscription_id",
"xxxx"
]
},
{
$eq: [
"$custom_data.stripe_id",
"xxx"
]
}
]
}
}
},
{
"$addFields": {
"custom_data": {
subscriptions: {
"$map": {
"input": "$custom_data.subscriptions",
"as": "s",
"in": {
"$cond": {
// if subscription_id matched, replace with your incoming object
"if": {
$eq: [
"$$s.subscription_id",
"xxxx"
]
},
"then": {
subscription_id: "incoming_sub_id"
},
// if not matched, keep the original object
"else": "$$s"
}
}
}
}
}
}
},
{
"$addFields": {
"custom_data": {
subscriptions: {
// insert case; if the subscription array is empty, then union with your incoming object
$setUnion: [
"$custom_data.subscriptions",
[
{
subscription_id: "incoming_sub_id"
}
]
]
}
}
}
},
{
"$merge": {
"into": "collection",
"on": "_id",
"whenMatched": "replace"
}
}
])
Here is the Mongo playground for your reference.

How to avoid adding duplicate objects to an array in MongoDB

this is my schema:
new Schema({
code: { type: String },
toy_array: [
{
date:{
type:Date(),
default: new Date()
}
toy:{ type:String }
]
}
this is my db:
{
"code": "Toystore A",
"toy_array": [
{
_id:"xxxxx", // automatic
"toy": "buzz"
},
{
_id:"xxxxx", // automatic
"toy": "pope"
}
]
},
{
"code": "Toystore B",
"toy_array": [
{
_id:"xxxxx", // automatic
"toy": "jessie"
}
]
}
I am trying to update an object. In this case I want to update the document with code: 'ToystoreA' and add an array of subdocuments to the array named toy_array if the toys does not exists in the array.
for example if I try to do this:
db.mydb.findOneAndUpdate({
code: 'ToystoreA,
/*toy_array: {
$not: {
$elemMatch: {
toy: [{"toy":'woddy'},{"toy":"buzz"}],
},
},
},*/
},
{
$addToSet: {
toy_array: {
$each: [{"toy":'woddy'},{"toy":"buzz"}],
},
},
},
{
new: false,
}
})
they are added and is what I want to avoid.
how can I do it?
[
{
"code": "Toystore A",
"toy_array": [
{
"toy": "buzz"
},
{
"toy": "pope"
}
]
},
{
"code": "Toystore B",
"toy_array": [
{
"toy": "jessie"
}
]
}
]
In this example [{"toy":'woddy'},{"toy":"buzz"}] it should only be added 'woddy' because 'buzz' is already in the array.
Note:when I insert a new toy an insertion date is also inserted, in addition to an _id (it is normal for me).
As you're using $addToSet on an object it's failing for your use case for a reason :
Let's say if your document look like this :
{
_id: 123, // automatically generated
"toy": "buzz"
},
{
_id: 456, // automatically generated
"toy": "pope"
}
and input is :
[{_id: 789, "toy":'woddy'},{_id: 098, "toy":"buzz"}]
Here while comparing two objects {_id: 098, "toy":"buzz"} & {_id: 123, "toy":"buzz"} - $addToSet consider these are different and you can't use $addToSet on a field (toy) in an object. So try below query on MongoDB version >= 4.2.
Query :
db.collection.updateOne({"_id" : "Toystore A"},[{
$addFields: {
toy_array: {
$reduce: {
input: inputArrayOfObjects,
initialValue: "$toy_array", // taking existing `toy_array` as initial value
in: {
$cond: [
{ $in: [ "$$this.toy", "$toy_array.toy" ] }, // check if each new toy exists in existing arrays of toys
"$$value", // If yes, just return accumulator array
{ $concatArrays: [ [ "$$this" ], "$$value" ] } // If No, push new toy object into accumulator
]
}
}
}
}
}])
Test : aggregation pipeline test url : mongoplayground
Ref : $reduce
Note :
You don't need to mention { new: false } as .findOneAndUpdate() return old doc by default, if you need new one then you've to do { new: true }. Also if anyone can get rid of _id's from schema of array objects then you can just use $addToSet as OP was doing earlier (Assume if _id is only unique field), check this stop-mongoose-from-creating-id-property-for-sub-document-array-items.

delete element out of array with $pull and $cond operators

I want to pull elements out of the array only if some condition is met
This is my document structure:
{
_id: "userId",
posts: [{
_id: "postId",
comments:[{
_id: "commentId",
userid: "some id of an user" // USER
},{
_id: "commentId2",
userid: "some id of an user2"
}]
}]
}
I want to delete the element from the comments array with the given commentId. This should be done only if userid is USER. If that condition isn't met, that means that comment doesn't belongs to the user that wants to delete it so I decline it.
Tried Attempt :
Post.findOneAndUpdate(
{
_id: mongoose.Types.ObjectId(userId)
},
{
$pull: {
$cond: [
{
"posts.$[post].comments.$[comment].userid": {
$eq: USER
}
},
{
$pull: {
comments: {
_id: mongoose.Types.ObjectId(commentId)
}
}
}
]
}
},
{
arrayFilters: [
{
"comment._id": mongoose.Types.ObjectId(commentId)
},
{
"post._id": mongoose.Types.ObjectId(postId)
}
]
}
)
That code above doesn't work, I'm stuck there & I don't know how to continue. maybe somebody knows how to fix this.
You can try below query :
Post.findOneAndUpdate(
{
_id: mongoose.Types.ObjectId(userId) // Fetches actual document
},
// Any matching object that has these fields/values in comments array will be pulled out
{
$pull: {"posts.$[post].comments": { _id : mongoose.Types.ObjectId(commentId), "userid": USER }}},
{
arrayFilters: [
{
"post._id": mongoose.Types.ObjectId(postId) // Checks which object inside `posts` array needs to be updated
}
]
}
)
Note : Use an option { new : true } in mongoose to return updated document, or in shell use { returnNewDocument : true }

Aggregate and reduce a nested array based upon an ObjectId

I have an Event document structured like so and I'm trying to query against the employeeResponses array to gather all responses (which may or may not exist) for a single employee:
[
{
...
eventDate: 2019-10-08T03:30:15.000+00:00,
employeeResponses: [
{
_id:"5d978d372f263f41cc624727",
response: "Available to work.",
notes: ""
},
...etc
];
}
];
My current mongoose aggregation is:
const eventResponses = await Event.aggregate([
{
// find all events for a selected month
$match: {
eventDate: {
$gte: startOfMonth,
$lte: endOfMonth,
},
},
},
{
// unwind the employeeResponses array
$unwind: {
path: "$employeeResponses",
preserveNullAndEmptyArrays: true,
},
},
{
$group: {
_id: null,
responses: {
$push: {
// if a response id matches the employee's id, then
// include their response; otherwise, it's a "No response."
$cond: [
{ $eq: ["$employeeResponses._id", existingMember._id] },
"$employeeResponses.response",
"No response.",
],
},
},
},
},
{ $project: { _id: 0, responses: 1 } },
]);
As you'll no doubt notice, the query above won't work after more than 1 employee records a response because it treats each individual response as a T/F condition, instead of all of the responses within the employeeResponses array as a single T/F condition.
As a result, I had remove all subsequent queries after the initial $match and do a manual reduce:
const responses = eventResponses.reduce((acc, { employeeResponses }) => {
const foundResponse = employeeResponses.find(response => response._id.equals(existingMember._id));
return [...acc, foundResponse ? foundResponse.response : "No response."];
}, []);
I was wondering if it's possible to achieve the same reduce result above, but perhaps using mongo's $reduce function? Or refactor the aggregation query above to treat all responses within the employeeResponses as a single T/F condition?
The ultimate goal of this aggregation is extract any previously recorded employee's responses and/or lack of a response from each found Event within a current month and place their responses into a single array:
["I want to work.", "Available to work.", "Not available to work.", "No response.", "No response." ...etc]
You can use $filter with $map to reshape your data and filter by _id. Then you can keep using $push with $ifNull to provide default value if an array is empty:
db.collection.aggregate([
{
$addFields: {
employeeResponses: {
$map: {
input: {
$filter: {
input: "$employeeResponses",
cond: {
$eq: [ "$$this._id", "5d978d372f263f41cc624727"]
}
}
},
in: "$$this.response"
}
}
}
},
{
$group: {
_id: null,
responses: { $push: { $ifNull: [ { $arrayElemAt: [ "$employeeResponses", 0 ] }, "No response" ] } }
}
}
])
Mongo Playground