I have a collection I want to upsert with findOneAndUpdate. In addition to that I have two fields (isHandled, isNotADuplicate) that should be:
defaulted to 'false' upon insert
left untouched upon update (e.g. isHandled stays 'true')
I have however found that
isHandled, isNotADuplicate are always defaulted back to 'false'
_id is also regenerated upon every update (I use a compound key to query the doc, not _id)
My Model
export const QuickbrainFindingSchema = new Schema<QuickBrainFindingDocument>({
connectedApplicationType: { type: String, required: true, enum: ['jira'] },//e.g. jira
clientKey: { type: String, required: true },//e.g. 135eb702-156c-3b67-b9d0-a0c97548xxxx
//key
projectKey: { type: String, required: true },//e.g. AL
type: { type: String, required: true },
doc1key: { type: String, required: true },//e.g. AL-7
doc2key: { type: String, required: true },//e.g. AL-16
//data
calculationDate: { type: SchemaTypes.Date, default: Date.now },
direction: { type: String, required: true },
reasonAndMetric: { type: SchemaTypes.Mixed, reason: true },
scoreSummary: { type: String, reason: true },
isHandled: { type: SchemaTypes.Boolean, default: false },
isNotADuplicate: { type: SchemaTypes.Boolean, default: false },
similarityReference: { type: SchemaTypes.ObjectId, required: true, ref: "QuickbrainSimilarityMatrix" }
}, {
//options
});
QuickbrainFindingSchema.index(
{ connectedApplicationType: 1, clientKey: 1, project: 1, doc1key: 1, doc2key: 1, type: 1 },
{ unique: true, name: "compoundKey" }
);
export const QuickbrainFindingModel = model<QuickBrainFindingDocument>("QuickbrainFinding", QuickbrainFindingSchema);
My Code
public async addFinding(
projectKey: string,
doc1key: string,
doc2key: string,
type: ET_FindingType
, data: QuickbrainFindingData): Promise<QuickbrainFinding> {
let keyFull: QuickbrainFindingKey = {
connectedApplicationType: this.connectedApplicationType,
clientKey: this.clientKey,
projectKey: projectKey,
doc1key: doc1key,
doc2key: doc2key,
type: type
};
let insertObj: QuickbrainFinding = <QuickbrainFinding><unknown>{};
Object.assign(insert, keyFull);
Object.assign(insert, data);
delete (<any>insertObj).isHandled;
delete (<any>insertObj).isNotADuplicate;
return new Promise<QuickbrainFinding>(function (ok, nok) {
QuickbrainFindingModel.findOneAndUpdate(
keyFull, { $set: insertObj},
{
runValidators: true,
upsert: true,
setDefaultsOnInsert: true,
new: true,
omitUndefined: true,//I think only available for findAndReplace(..)
})
.lean().exec(function (err, result) {
if (err) {
nok(err);
}
else
ok(result)
});
});
}
Mongoose Debug Output
quickbrainfindings.findOneAndUpdate(
{
connectedApplicationType: 'jira',
clientKey: '135eb702-256c-3b67-b9d0-a0c975487af3',
projectKey: 'ITSMTEST',
doc1key: 'ITSMTEST-7',
doc2key: 'ITSMTEST-10',
type: 'Email'
},
{
'$setOnInsert':
{ __v: 0, isHandled: false, isNotADuplicate: false, _id: ObjectId("60789b02c094eb3ef07d2929") },
'$set': {
connectedApplicationType: 'jira',
clientKey: '135eb702-256c-3b67-b9d0-a0c975487af3', projectKey: 'ITSMTEST', doc1key: 'ITSMTEST-7', doc2key: 'ITSMTEST-10', type: 'Email',
calculationDate: new Date("Thu, 15 Apr 2021 19:58:58 GMT"),
direction: '2', scoreSummary: '100.0%',
similarityReference: ObjectId("60789b029df2079dfa8aa15a"),
reasonAndMetric: [{ reason: 'Title Substring', metricScore: '100%' },
{ reason: 'Title TokenSet', metricScore: '54%' }, { reason: 'Description TokenSet', metricScore: '100%' }]
}
},
{
runValidators: true, upsert: true, remove: false, projection: {},
returnOriginal: false
}
)
What happens
Existing documents are found, but when they are updated I'm confused that:
_id is regenerated
isHandled and isNotADuplicate are reset to 'false' (although insertObj does not contain them)
When looking at the debug output I can see that the new _id is the one fron $setOnInsert, which confuses the heck out of me, since the selector works
Notable
keyFull is used to query the existing document, it does not contain _id;
delete (<any>insertObj).isHandled <- the object used for $set does NOT contain isHandled
This is embarrasing to admit, but thanks to Joe I have found the problem.
Before every findOneAndUpdate / Upsert I had a delete statement removing the existing documents Pipeline:
Delete old documents
Calculate new documents
Upsert new documents -> always resulted in Insert
let matchAnyDoc = this.filterForDocKeyAny(projectKey, docKeyAny, findingType);
matchAnyDoc.forEach(async (condition) => {
QuickbrainFindingModel.deleteMany(condition).exec(function (err, res) {
if (err) {
nok(err);
} else {
ok();
}
});
}, this);
Related
I'm trying to access a database with the following difficult schema and want to update the file_name value if it exists, or create a new object with the file_name if it does not. As I have searched, the option { upsert: true, new: true, setDefaultsOnInsert: true }; will update do the second bit, but the problem is how do you access the file_name key nested deep within.
export type _ID = string;
export interface FileSchema {
[_id: _ID]: {
file_name: string;
creation_date: Date;
isPublished: boolean;
content: string;
};
}
export interface AccountSchema {
...
files: FileSchema;
}
const accountSchema = new Schema<AccountSchema>({
...
files: {
type: Map,
unique: true,
of: {
file_name: {
type: String,
required: true,
minlength: 4,
maxlength: 60,
unique: true,
},
creation_date: {
type: Date,
required: true,
},
isPublished: {
type: Boolean,
required: true,
},
content: {
type: String,
required: true,
},
},
},
});
Not familiar with Mongoose but you may be missing dot notation to access nested fields, look at
https://www.mongodb.com/docs/manual/tutorial/query-embedded-documents/#specify-equality-match-on-a-nested-field
I've searched a bit, and have come up with the solution below, although not pretty due to multiple queries. For nested objects with a dynamic key, you must use codes like [files.${file_id}.file_name] (with ``) to access the object within.
Reference 1
Reference 2
const accountSchema = new Schema<AccountSchema>({
...
files: {
type: Schema.Types.Map,
unique: true,
of: Object,
},
})
// checks if file exists, if exist then update file_name
const files: FileSchema | null = await DB_ACCOUNT.findOneAndUpdate(
{
_id,
username,
[`files.${file_id}`]: { $exists: true },
},
{
$set: {
[`files.${file_id}.file_name`]: file_name,
},
},
{ new: true, upsert: false, useFindAndModify: false }
);
const fn = genObjectId().toString();
// if file does not exist, then create a new file
const x = await DB_ACCOUNT.findOneAndUpdate(
{
_id,
username,
},
{
$set: {
[`files.${fn}`]: {
file_name,
creation_date: new Date(),
isPublished: false,
content: "",
},
},
},
{ new: true, upsert: true }
)
console.log("X: ", x);
I am pretty new to mongoDb and want to apply nested query.
I have a business schema like this:
const businessSchema = new mongoose.Schema(
{
name: {
type: String,
required: true,
},
businessType: {
type: Schema.Types.ObjectId,
ref: "businessCategory",
required: true,
},
email: {
type: String,
required: true,
},
password: {
type: String,
required: true,
select: false,
},
review: {
type: [reviewSchema],
},
isDeleted: {
type: Boolean,
default: false,
},
},
{ timestamps: true }
);
Business has a review where user can do the review and reviewSchema is
const reviewSchema = new mongoose.Schema(
{
user: {
type: Schema.Types.ObjectId,
ref: "users",
required: true,
},
rating: {
type: Number,
enum: [1, 2, 3, 4, 5],
},
reviewArray: {
type: [singleReviewSchema],
},
},
{ timestamps: true }
);
One user can do many reviews, and it has reviewArray.
ReviewArray schema is
const singleReviewSchema = new mongoose.Schema(
{
title: {
type: String,
},
description: {
type: String,
},
isDeleted: {
type: Boolean,
default: false,
},
},
{ timestamps: true }
);
How to fetch the business with a condition business: isDeleted:false and its reviews with singleReviewSchema: isDeleted:false
I dont know your model names, so please replace path with correct names
but it might look like:
businnesModel.find({isDeleted: false})
.populate({
path: 'reviewModelName',
model: 'review',
populate: {
path: 'reviewArray',
model: 'singleReviewModelName',
match: {
isDeleted : false
}
}
})
It should provide you array of businessModel documents - even when their singleReviews array will be empty (because all of reviews are deleted, or there was zero reviews). So you have to filter it out in JS.
To avoid filtering in JS, and to do it a bit more efficient way for mongodb, you can go with aggregate instead.
I am currently trying to setup a schema for custom Discord guild commands:
const GuildCommandsSchema = new mongoose.Schema({
_id: String,
commands: [
{
name: {
type: String,
unique: true,
required: true,
},
action: {
type: String,
required: true,
},
author: {
type: String,
required: true,
},
},
],
});
Is this ok, performancewise, or could I improve it?
I feel like Mongo would need to look through all commands, since it can't index any commands inside 'commands' even though 'name' is unique.
If that's fine, how can I access the values inside commands?
I would need to find the right command via 'name' if it exists, otherwise create it and add/update 'action' + 'author'.
I tried something like this:
const updatedCommand = await GuildCommands.findOneAndUpdate(
{ _id },
{
$set: {
[`commands.$[outer].name`]: name,
[`commands.$[outer].action`]: action,
[`commands.$[outer].author`]: author,
},
},
{
arrayFilters: [{ 'outer.name': name }],
}
);
Unfortunately that does not create commands if they don't exist.
Thanks for your help
aggregate
db.collection.update({},
{
$set: {
"commands.$[c].name": "1",
"commands.$[c].author": "1",
"commands.$[c].action": "1"
}
},
{
arrayFilters: [
{
"c.author": "34"
}
],
multi: true
})
mongoplayground
To answer my own question:
I changed my Schema to use Maps instead of Arrays for performance improvments and also better model management.
const GuildCommandsSchema = new mongoose.Schema(
{
_id: String,
commands: {
type: Map,
of: {
_id: false,
name: {
type: String,
required: true,
},
action: {
type: String,
required: true,
},
active: {
type: Boolean,
required: true,
default: true,
},
author: {
type: String,
required: true,
},
},
},
},
{ versionKey: false }
);
The new query to find and update/create a command is also better imo:
const findCommand = await GuildCommands.findOne({ _id });
if (!action) {
const getCommand = findCommand.commands.get(name);
if (getCommand) {
message.reply(getCommand.action);
} else {
message.reply(`Cannot find ${name}`);
}
} else {
findCommand.commands.set(name, {
name,
action,
author,
});
findCommand.save();
}
I want to push a Date object from my client into the nested array 'completed_dates' but I cannot figure out how to do so, or if I would need to change my schema in order for it to work.
{
_id: 606f1d67aa1d5734c494bf0a,
name: 'Courtney',
email: 'c#gmail.com',
password: '$2b$10$WQ22pIiwD8yDvRhdQ0olBe6JnnFqV2WOsC0cD/FkV4g7LPtUOpx1C',
__v: 35,
habits: [
{
_id: 6081d32580bfac579446eb81,
completed_dates: [],
name: 'first',
type: 'good',
days: 0,
checked: false
},
{
_id: 6081d32f80bfac579446eb82,
completed_dates: [],
name: 'seconds',
type: 'bad',
days: 0,
checked: false
},
]
}
and this is my schema
const habitSchema = new mongoose.Schema({
name: String,
category: String,
color: {
type: String,
},
date_added: {
type: String,
},
completed_dates: {
type: Array,
}
})
const userSchema = new mongoose.Schema({
name: {
type: String,
required: true,
min: 6,
max: 255,
},
email: {
type: String,
required: true,
max: 255
},
password: {
type: String,
required: true,
max: 1024,
min: 8,
},
habits: [habitSchema]
})
Here is what I have tried...
I've tried using findOneAndUpdate, using the document id of the logged in user, and trying to manipulate the update object to drill into the nested array. I can access the habits list of the correct user... using this code, but for this new problem, I want to go one level further and push to the 'completed_dates' array of a specific habit (based on name or _id).
//this only adds a habit object to the habits array.
User.findByIdAndUpdate(req.user._id,
{ $pull: { habits: { _id: itemsToDelete } } },
{ new: true , useFindAndModify: false},
function (err, data) {
if (err) {
res.send(err)
} else {
res.send(data.habits)
}
}
)
I have tried building on this existing code by trying to filter down one more level. (this doesn't work.)
const { date, name} = req.body.update
User.findByIdAndUpdate(req.user._id,
{ $push: { 'habits.$[req.body.name].completed_dates': req.body.date} },
{safe: true, upsert: true, new : true, useFindAndModify: false},
function (err, data) {
if (err) {
res.send(err)
} else {
//data.update
res.send(data.habits)
}
}
)
If anyone can link or help me out, I would appreciate it. Thanks
How can I overwrite the entire document, instead of just updating the fields?
Here is the method I use right now but doesn't work:
updateFilmTitle: function(req, res) {
var id = req.params.id;
console.log(id);
filmTitleModel.findByIdAndUpdate(id, req.body, {
overwrite: true
}, {
new: true
}, (error, response) => {
if (error) {
res.json(error);
console.error(error);
return;
}
console.log("filmTitle form has been updated!");
res.json(response);
console.log(response);
});
},
here how my model looks like,
var venueSchema = new Schema({
ticketServiceRequired: { type: Boolean, required: true },
filmSettings: {
type: {
filmContactName: { type: String, required: true },
filmSeatingAmount: { type: Number, required: true },
filmMediaDelivery: { type: Array, required: true },
filmRentalFee: {
price: { type: Number, required: true },
type: { type: String, required: true },
},
}
},
});
new and overwrite both are options, so it should be this:
filmTitleModel.findByIdAndUpdate(id, req.body, {
overwrite : true,
new : true
}, (error, response) => { ... });