Related
I'm trying to access a database with the following difficult schema and want to update the file_name value if it exists, or create a new object with the file_name if it does not. As I have searched, the option { upsert: true, new: true, setDefaultsOnInsert: true }; will update do the second bit, but the problem is how do you access the file_name key nested deep within.
export type _ID = string;
export interface FileSchema {
[_id: _ID]: {
file_name: string;
creation_date: Date;
isPublished: boolean;
content: string;
};
}
export interface AccountSchema {
...
files: FileSchema;
}
const accountSchema = new Schema<AccountSchema>({
...
files: {
type: Map,
unique: true,
of: {
file_name: {
type: String,
required: true,
minlength: 4,
maxlength: 60,
unique: true,
},
creation_date: {
type: Date,
required: true,
},
isPublished: {
type: Boolean,
required: true,
},
content: {
type: String,
required: true,
},
},
},
});
Not familiar with Mongoose but you may be missing dot notation to access nested fields, look at
https://www.mongodb.com/docs/manual/tutorial/query-embedded-documents/#specify-equality-match-on-a-nested-field
I've searched a bit, and have come up with the solution below, although not pretty due to multiple queries. For nested objects with a dynamic key, you must use codes like [files.${file_id}.file_name] (with ``) to access the object within.
Reference 1
Reference 2
const accountSchema = new Schema<AccountSchema>({
...
files: {
type: Schema.Types.Map,
unique: true,
of: Object,
},
})
// checks if file exists, if exist then update file_name
const files: FileSchema | null = await DB_ACCOUNT.findOneAndUpdate(
{
_id,
username,
[`files.${file_id}`]: { $exists: true },
},
{
$set: {
[`files.${file_id}.file_name`]: file_name,
},
},
{ new: true, upsert: false, useFindAndModify: false }
);
const fn = genObjectId().toString();
// if file does not exist, then create a new file
const x = await DB_ACCOUNT.findOneAndUpdate(
{
_id,
username,
},
{
$set: {
[`files.${fn}`]: {
file_name,
creation_date: new Date(),
isPublished: false,
content: "",
},
},
},
{ new: true, upsert: true }
)
console.log("X: ", x);
In my MongoDB model, there are 3 fields in total. Any document added to this collection must contain at least 1 of those 3 fields.
How can this be specified in the validation staged?
You can enum validation constraint which collection creation as below:
db.createCollection("jobs", {
validator: {
$jsonSchema: {
bsonType: "object",
required: [ "status" ],
properties: {
status: {
enum: [ "Done", "Failed", "Initial" ],
description: "can only be one of the enum values and is required"
},
}
}
}
})
From the docs
Mongoose has several inbuilt validators. Strings have enum as one of the validators. So enum creates a validator and checks if the value is given in an array. E.g:
var userSchema = new mongooseSchema({
status: {
type: String,
enum : ['Done','Failed', 'Initial'],
default: 'Initial'
},
})
You can use custom validator to check if we have one of 3 keys in the object
const testSchema = mongoose.Schema({
field1: {
type: String,
validate: {
validator: function(v) {
if (this.field2 == undefined && this.field3 == undefined) {
return true;
}
return false;
},
},
},
field2: {
type: String,
validate: {
validator: function(v) {
if (this.field1 == undefined && this.field3 == undefined) {
return true;
}
return false;
},
},
},
field3: {
type: String,
validate: {
validator: function(v) {
if (this.field2 == undefined && this.field1 == undefined) {
return true;
}
return false;
},
},
},
});
I have a collection I want to upsert with findOneAndUpdate. In addition to that I have two fields (isHandled, isNotADuplicate) that should be:
defaulted to 'false' upon insert
left untouched upon update (e.g. isHandled stays 'true')
I have however found that
isHandled, isNotADuplicate are always defaulted back to 'false'
_id is also regenerated upon every update (I use a compound key to query the doc, not _id)
My Model
export const QuickbrainFindingSchema = new Schema<QuickBrainFindingDocument>({
connectedApplicationType: { type: String, required: true, enum: ['jira'] },//e.g. jira
clientKey: { type: String, required: true },//e.g. 135eb702-156c-3b67-b9d0-a0c97548xxxx
//key
projectKey: { type: String, required: true },//e.g. AL
type: { type: String, required: true },
doc1key: { type: String, required: true },//e.g. AL-7
doc2key: { type: String, required: true },//e.g. AL-16
//data
calculationDate: { type: SchemaTypes.Date, default: Date.now },
direction: { type: String, required: true },
reasonAndMetric: { type: SchemaTypes.Mixed, reason: true },
scoreSummary: { type: String, reason: true },
isHandled: { type: SchemaTypes.Boolean, default: false },
isNotADuplicate: { type: SchemaTypes.Boolean, default: false },
similarityReference: { type: SchemaTypes.ObjectId, required: true, ref: "QuickbrainSimilarityMatrix" }
}, {
//options
});
QuickbrainFindingSchema.index(
{ connectedApplicationType: 1, clientKey: 1, project: 1, doc1key: 1, doc2key: 1, type: 1 },
{ unique: true, name: "compoundKey" }
);
export const QuickbrainFindingModel = model<QuickBrainFindingDocument>("QuickbrainFinding", QuickbrainFindingSchema);
My Code
public async addFinding(
projectKey: string,
doc1key: string,
doc2key: string,
type: ET_FindingType
, data: QuickbrainFindingData): Promise<QuickbrainFinding> {
let keyFull: QuickbrainFindingKey = {
connectedApplicationType: this.connectedApplicationType,
clientKey: this.clientKey,
projectKey: projectKey,
doc1key: doc1key,
doc2key: doc2key,
type: type
};
let insertObj: QuickbrainFinding = <QuickbrainFinding><unknown>{};
Object.assign(insert, keyFull);
Object.assign(insert, data);
delete (<any>insertObj).isHandled;
delete (<any>insertObj).isNotADuplicate;
return new Promise<QuickbrainFinding>(function (ok, nok) {
QuickbrainFindingModel.findOneAndUpdate(
keyFull, { $set: insertObj},
{
runValidators: true,
upsert: true,
setDefaultsOnInsert: true,
new: true,
omitUndefined: true,//I think only available for findAndReplace(..)
})
.lean().exec(function (err, result) {
if (err) {
nok(err);
}
else
ok(result)
});
});
}
Mongoose Debug Output
quickbrainfindings.findOneAndUpdate(
{
connectedApplicationType: 'jira',
clientKey: '135eb702-256c-3b67-b9d0-a0c975487af3',
projectKey: 'ITSMTEST',
doc1key: 'ITSMTEST-7',
doc2key: 'ITSMTEST-10',
type: 'Email'
},
{
'$setOnInsert':
{ __v: 0, isHandled: false, isNotADuplicate: false, _id: ObjectId("60789b02c094eb3ef07d2929") },
'$set': {
connectedApplicationType: 'jira',
clientKey: '135eb702-256c-3b67-b9d0-a0c975487af3', projectKey: 'ITSMTEST', doc1key: 'ITSMTEST-7', doc2key: 'ITSMTEST-10', type: 'Email',
calculationDate: new Date("Thu, 15 Apr 2021 19:58:58 GMT"),
direction: '2', scoreSummary: '100.0%',
similarityReference: ObjectId("60789b029df2079dfa8aa15a"),
reasonAndMetric: [{ reason: 'Title Substring', metricScore: '100%' },
{ reason: 'Title TokenSet', metricScore: '54%' }, { reason: 'Description TokenSet', metricScore: '100%' }]
}
},
{
runValidators: true, upsert: true, remove: false, projection: {},
returnOriginal: false
}
)
What happens
Existing documents are found, but when they are updated I'm confused that:
_id is regenerated
isHandled and isNotADuplicate are reset to 'false' (although insertObj does not contain them)
When looking at the debug output I can see that the new _id is the one fron $setOnInsert, which confuses the heck out of me, since the selector works
Notable
keyFull is used to query the existing document, it does not contain _id;
delete (<any>insertObj).isHandled <- the object used for $set does NOT contain isHandled
This is embarrasing to admit, but thanks to Joe I have found the problem.
Before every findOneAndUpdate / Upsert I had a delete statement removing the existing documents Pipeline:
Delete old documents
Calculate new documents
Upsert new documents -> always resulted in Insert
let matchAnyDoc = this.filterForDocKeyAny(projectKey, docKeyAny, findingType);
matchAnyDoc.forEach(async (condition) => {
QuickbrainFindingModel.deleteMany(condition).exec(function (err, res) {
if (err) {
nok(err);
} else {
ok();
}
});
}, this);
I'm working on a project that uses:
"#nestjs/core": "^7.0.0",
"#nestjs/mongoose": "^7.0.0",
"mongoose": "^5.9.12",
// ...
"typescript": "^3.7.4",
With mongoose/mongoDB config:
uri: MONGO_DB_URI,
useUnifiedTopology: true,
useNewUrlParser: true,
useFindAndModify: false,
useCreateIndex: true,
I'm trying to build a simple CRUD for this model:
export const ContactSchema = new mongoose.Schema(
{
source_id: { type: String, required: true },
firstName: { type: String, trim: true },
lastName: { type: String, trim: true },
phones: [
{
number: {
type: String,
required: true,
unique: true,
validate: {
validator: function(value) {
const phoneNumber = parsePhoneNumberFromString(value)
return phoneNumber && phoneNumber.isValid()
},
},
},
type: {
type: String,
default: function() {
return parsePhoneNumberFromString(this.number).getType() || "N/A"
},
},
code: {
type: Number,
default: function() {
return parsePhoneNumberFromString(this.number).countryCallingCode || undefined
},
},
national: {
type: Number,
default: function() {
return parsePhoneNumberFromString(this.number).nationalNumber || undefined
},
},
},
],
email: { type: String, unique: true, required: true, lowercase: true, trim: true },
},
{ timestamps: true },
)
ContactSchema.plugin(mongoosePaginate)
Like every CRUD app, I'm willing to have fildAll() & fildOne() routes that return the body of a given Contact with all his info including the list of its phone numbers. So I used:
// ...
async findAll(): Promise<Contact[]> {
return this.contactModel.find()
// then I add
.populate('phones')
}
async findBySourceId(id: string): Promise<Contact> {
return this.contactModel.findOne({ source_id: id })
// then I add
.populate('phones')
}
// ...
All info are well saved in the DB and there is no missing data (neither phones) and I'm sure that it works the beginning without even adding .poplate('x'), but that changed somewhere and it returns now unpopulated phone array.
Now It returns:
{
"_id": "5ebc22072e18637d84bcf6f0",
"firstName": "Maher",
"lastName": "Boubakri",
"phones": [],
"email": "mhb#test.im",
// ...
}
But, It should return:
{
"_id": "5ebc22072e18637d84bcf6f0",
"firstName": "Maher",
"lastName": "Boubakri",
"phones": [
{
"_id": "5ebc22072e18637d8fd948f9",
"number": "+21622123456",
"code": 216,
"type": "MOBILE",
"national": 22123456,
}
],
"email": "mhb#test.im",
// ...
}
Note: It is clear that MongoDB generates _id for every phone object, but, it is not a ref Object.
Any idea will be so helpful,
Thank you.
populate is used to join two (or more) collections using the references
here you don't have any references, so you don't need it
just use find() without populate
Based on #Mohammed 's comment and answer, adding .lean() after updating mongoose fixed the problem.
// ...
async findAll(): Promise<Contact[]> {
return this.contactModel.find().lean()
}
async findBySourceId(id: string): Promise<Contact> {
return this.contactModel.findOne({ source_id: id }).lean()
}
// ...
I have 2 Schemas : StepSchema, StepRelationshipSchema
var StepSchema = new Schema(
{
name: { type: String, required: true },
description: { type: String, default: '' },
isVisible: { type: Boolean, default: true }
}, options
);
var StepRelationshipSchema = new Schema(
{
workflowId: { type: String, required: true },
stepId: { type: String, required: true },
prevSteps: [ Schema.Types.Mixed ] ,
nextSteps: [ Schema.Types.Mixed ] ,
gotoStep: { type: String, default: '' }
}, options
);
In StepSchema, I want to create a static method to get nextSteps in StepRelationshipSchema.
Can I use this, thank you so much.
StepSchema.statics.getNextSteps = function(workflowId, currStepId) {
return StepRelationship.findOne({
workflowId: workflowId,
stepId: currStepId
}).nextSteps
};
As #JohnnyHK suggested in his comments, findOne() is async thus you need to use a callback function as follows:
// create a query for next stepswith a blogpost _id matching workflowId and currStepId
schema.statics.getNextSteps = function (workflowId, currStepId, callback) {
return this.model('StepRelationship').findOne({
workflowId: workflowId,
stepId: currStepId
}, callback);
}