Circular Reference Issue in Mongoose pre-hook - mongodb

In my MongoDB/Node backend environment I am using Mongoose pre and post hook middleware to check what's changed on the document, in order to create some system notes as a result.
One problem I'm running into is that when I try and lookup the record for the document in question I get a "Customer.findOne()" is not a function error. This is ONLY a problem when I'm looking up a record from the same collection from which the model just launched this pre and post hook triggers file. In other words, if my "Customer" model kicks off functions in a pre hook function in an external file, then I get an error if I then try and lookup a Customer with a standard findOne():
My customer model looks something like this:
module.exports = mongoose.model(
"Customer",
mongoose
.Schema(
{
__v: {
type: Number,
select: false
},
deleted: {
type: Boolean,
default: false
},
// Other props
searchResults: [
{
matchKey: String,
matchValue: String
}
]
},
{
timestamps: true
}
)
.pre("save", function(next) {
const doc = this;
trigger.preSave(doc);
next();
})
.post("save", function(doc) {
trigger.postSave(doc);
})
.post("update", function(doc) {
trigger.postSave(doc);
})
.post("findOneAndUpdate", function(doc) {
trigger.postSave(doc);
})
);
... the problematic findOne() function in the triggers file being called from the model looks like this:
const Customer = require("../../models/customer");
exports.preSave = async function(doc) {
this.preSaveDoc = await Customer.findOne({
_id: doc._id
}).exec();
};
To clarify, this is NOT a problem if I'm using a findOne() to lookup a record from a different collection in this same triggers file. Then it works fine. See below when finding a Contact -- no problem here:
const Contact = require("../../models/contact");
exports.preSave = async function(doc) {
this.preSaveDoc = await Contact.findOne({
_id: doc._id
}).exec();
};
The workaround I've found is to use Mongo instead of Mongoose, like so:
exports.preSave = async function(doc) {
let MongoClient = await require("../../config/database")();
let db = MongoClient.connection.db;
db.collection("customers")
.findOne({ _id: doc._id })
.then(doc => {
this.preSaveDoc = doc;
});
}
... but I'd prefer to use Mongoose syntax here. How can I use a findOne() in a pre-hook function being called from the same model/collection as the lookup type?

I have ran similar issue few days ago.
Effectively it is a circular dependency problem. When you call .findOne() on your customer model it doesn't exist as it is not exported yet.
You should probably try something like that :
const customerSchema = mongoose.Schema(...);
customerSchema.pre("save", async function(next) {
const customer = await Customer.findOne({
_id: this._id
}).exec();
trigger.setPreSaveDoc(customer);
next();
})
const Customer = mongoose.model("Customer", customerSchema)
module.export Customer;
Here customer will be defined because it is not called (the pre hook) before its creation.
As an easier way (I am not sure about it) but you could try to move the Contact import in your Trigger file under the save function export. That way I think the decencies may works.
Did it helps ?

Related

Mongoose: deleteOne middleware for cascading delete not working

With remove being deprecated in Mongoose 5.7.13, I want to use deleteOne instead. I need to get the id of the deleted document so that I can then delete further related documents in other collections in a cascade. I thought that "this" within the context of the pre middleware hook was meant to refer to the removed document, but instead it's just an empty object. Is there a canonical working example of this? I'm still currently using 5.7.12 at this point - will that make a difference here?
Here is the code I'm currently working with. The issue is that I can't get the projectId at the start because the reference is completely empty. Doing this on post rather than pre, or switching the option to run on query rather than document all yield the same result.
ProjectSchema.pre("deleteOne", {document:true}, (next) => {
const projectId = this._id;
ListModel.find({parentProject:projectId}, (err, lists) => {
if(err){
console.log("error cascading project delete to lists", {err});
}
lists.map(list => {
ListModel.deleteOne({_id:list._id}, (err, result) => {
if(err) {
console.log("error on project delete cascade", {err});
}
});
});
});
});
It depends whether you call deleteOne on document or on model. The later just have no document to bind it to.
The former gives you the document as you expect:
const project = await ProjectModel.findOne();
project.deleteOne();
The later gives you the Query. There is no _id in the query, but it has this.op for example, which in this middleware will be "deleteOne":
await ProjectModel.deleteOne();
The only way to get the document id in this case is to ensure it is provided in the query:
await ProjectModel.deleteOne({_id: "alex"});
Then you can get it in the middleware from the filter:
const projectId = this.getFilter()["_id"]
You can specify query: false in second parameter of the middleware to ensure the it is not invoked when you call deleteOne on model. So the best you can do:
ProjectSchema.pre("deleteOne", {document:true, query: false}, (next) => {
const projectId = this._id;
....
});
ProjectSchema.pre("deleteOne", {document:false, query: true}, (next) => {
const projectId = this.getFilter()["_id"];
if (typeof projectId === "undefined") {
// no way to make cascade deletion since there is no _id
// in the delete query
// I would throw an exception, but it's up to you how to deal with it
// to ensure data integrity
}
});
Please take a look at corresponding tests on v5.7.12: https://github.com/Automattic/mongoose/blob/5.7.12/test/model.middleware.test.js#L436
In the mongoose docs it says "Model.deleteOne() does not trigger pre('remove') or post('remove') hooks."
There is solution if you can refactor your delete operations with findByIdAndDelete, it triggers the findOneAndDelete middleware,
So we can add this middleware to Project Schema.
Project model:
const mongoose = require("mongoose");
const ProjectChild = require("./projectChild");
const ProjectSchema = new mongoose.Schema({
name: String
});
ProjectSchema.post("findOneAndDelete", async function(doc) {
console.log(doc);
if (doc) {
const deleteResult = await ProjectChild.deleteMany({
parentProject: doc._id
});
console.log("Child delete result: ", deleteResult);
}
});
module.exports = mongoose.model("Project", ProjectSchema);
ProjectChild model:
const mongoose = require("mongoose");
const projectChildSchema = new mongoose.Schema({
name: String,
parentProject: {
type: mongoose.Schema.Types.ObjectId,
ref: "Project"
}
});
module.exports = mongoose.model("ProjectChild", projectChildSchema);
I created a project like this:
{
"_id": "5dea699cb10c442260245abf",
"name": "Project 1",
"__v": 0
}
And created 2 project child for this project:
Child 1
{
"_id": "5dea69c7b10c442260245ac0",
"name": "Child 1 (project 1)",
"parentProject": "5dea699cb10c442260245abf",
"__v": 0
}
Child 2
{
"_id": "5dea69e8b10c442260245ac1",
"name": "Child 2 (project 1)",
"parentProject": "5dea699cb10c442260245abf",
"__v": 0
}
I created a sample route to delete a project by its id like this:
router.delete("/project/:id", async (req, res) => {
const result = await Project.findByIdAndDelete(req.params.id);
res.send(result);
});
When I send a DELETE request to this route, we see the following info in the console:
console.log(doc);
{ _id: 5dea699cb10c442260245abf, name: 'Project 1', __v: 0 }
console.log("Child delete result: ", deleteResult);
Child delete result: { n: 2, ok: 1, deletedCount: 2 }
So we could deleted the 2 children of the project, when we deleted the project.
As an alternative you can also use findOneAndRemove, it triggers findOneAndRemove post middleware.
So in the ProjectSchema we replace the post middleware like this:
ProjectSchema.post("findOneAndRemove", async function(doc) {
console.log(doc);
if (doc) {
const deleteResult = await ProjectChild.deleteMany({
parentProject: doc._id
});
console.log("Child delete result: ", deleteResult);
}
});
When we use a findOneAndRemove operation, the result will be the same as the first alternative:
const result = await Project.findOneAndRemove({ _id: req.params.id });

Mongo `pre` hook not firing as expected on `save()` operation

I am using pre and post hooks in my MongoDB/Node backend in order to compare a pre-save and post-save version of a document so I can generate notes via model triggers based on what's changed. In one of my models/collections this is working, but in another, it's not working as expected, and I'm not sure why.
In the problem case, some research has determined that even though I am calling a pre hook trigger on an operation that uses a save(), when I console out the doc state passed in that pre hook, it's already had the change applied. In other words, the hook is not firing before the save() operation, but after, from what I can tell.
Here is my relevant model code:
let Schema = mongoose
.Schema(CustomerSchema, {
timestamps: true
})
.pre("save", function(next) {
const doc = this;
console.log("doc in .pre: ", doc); // this should be the pre-save version of the doc, but it is the post-save version
console.log("doc.history.length in model doc: ", doc.history.length);
trigger.preSave(doc);
next();
})
.post("save", function(doc) {
trigger.postSave(doc);
})
.post("update", function(doc) {
trigger.postSave(doc);
});
module.exports = mongoose.model("Customer", Schema);
The relevant part of the save() operation that I'm doing looks like this (all I'm doing is pushing a new element to an array on the doc called "history"):
exports.updateHistory = async function(req, res) {
let request = new CentralReqController(
req,
res,
{
// Allowed Parameters
id: {
type: String
},
stageId: {
type: String
},
startedBy: {
type: String
}
},
[
// Required Parameters
"id",
"stageId",
"startedBy"
]
);
let newHistoryObj = {
stageId: request.parameters.stageId,
startDate: new Date(),
startedBy: request.parameters.startedBy,
completed: false
};
let customerToUpdate = await Customer.findOne({
_id: request.parameters.id
}).exec();
let historyArray = await customerToUpdate.history;
console.log("historyArray.length before push in update func: ", historyArray.length);
historyArray.push(newHistoryObj);
await customerToUpdate.save((err, doc) => {
if (doc) console.log("history update saved...");
if (err) return request.sendError("Customer history update failed.", err);
});
};
So, my question is, if a pre hook on a save() operation is supposed to fire BEFORE the save() happens, why does the document I look at via my console.log show a document that's already had the save() operation done on it?
You are a bit mistaken on what the pre/post 'save' hooks are doing. In pre/post hook terms, save is the actual save operation to the database. That being said, the this you have in the pre('save') hook, is the object you called .save() on, not the updated object from the database. For example:
let myCustomer = req.body.customer; // some customer object
// Update the customer object
myCustomer.name = 'Updated Name';
// Save the customer
myCustomer.save();
We just updated the customers name. When the .save() is called, it triggers the hooks, like you stated above. Only the difference is, the this in the pre('save') hook is the same object as myCustomer, not the updated object from the database. On the contrary, the doc object in the `post('save') hook IS the updated object from the database.
Schema.pre('save', function(next) {
console.log(this); // Modified object (myCustomer), not from DB
)};
Schema.post('save', function(doc) {
console.log(doc); // Modified object DIRECTLY from DB
});

Model.find is not a function in mongoose

I am new in node and mongodb. I am trying to query a different model(Event) from another model(Company).
Basically in Event model there is a field called company. I would like to get the company where id is an Event ID.
I have all the event IDs in an array.
let eventIds = [ 5b76a8139dc71a4a12564cd2,
5b9a1685c239342d4635466c,
5b8e753bdbccf803e906aaeb ]
Event Schema --
var EventSchema = new Schema({
title:{type:String,require:true,index:true},
description:{type:String,require:false},
companies:[
{type:Schema.Types.ObjectId,ref:"Company",require:true,index:true}
]
});
In company model --
var mongoose = require('mongoose'),
Schema = mongoose.Schema,
Event = require('./event.js');
var CompanySchema = new Schema({
name:{type:String,require:true,index:true},
description:{type:String,require:false}},{
//no auto indexing at the beginning
autoIndex:true,
//no strict to save changes in the valuesBeforeChange field.
strict:false}
);
CompanySchema.static("searchCompanies",function(callback,criteria){
"use strict";
var That = this;
var query = That.find();
async.waterfall([
function(callback){
let eventIds = [5b76a8139dc71a4a12564cd2,5b9a1685c239342d4635466c,5b8e753bdbccf803e906aaeb ];
Event.find({ $in: eventIds}, function(err, docs){
console.log(docs);
});
}
],function(err,companyResultObj){
callback(err,companyResultObj);
});
});
I am getting Event.find is not a function error message. How can I query a different model(event) from another model(company)
Any help is highly appreciated.
Not sure why but I had to do this in the following way.
Event.find({ $in: eventIds}, function(err, docs){
To
mongoose.model('Event').find({_id:eventIds}, function(err, docs){
which returned 3 documents which are correct.
Use alias when you require file
EventModel = require('./event.js');
then
EventModel.find({ $in: eventIds}, function(err, docs){
console.log(docs);
});
How are you exporting your EventModel?
Assuming you are exporting it like as a module (module.exports = { EventModel }),
you want to go "const Event = require('./event.js').EventModel;"
Then simply go with "Event.find(..."
This worked for me:
In my project I pass a Model into some middleware and started seeing this issue so I used mongoose.model('Name of Model')
example: .get(advancedResults(mongoose.model('Store'), 'stores'), getStores)
If you're importing your model from an index.js or index.ts file, you had
better import the file as:
import YourModel from "./<file>/index"

Mongoose: Populate a populated field

I'm using MongoDB as a log keeper for my app to then sync mobile clients. I have this models set up in NodeJS:
var UserArticle = new Schema({
date: { type: Number, default: Math.round((new Date()).getTime() / 1000) }, //Timestamp!
user: [{type: Schema.ObjectId, ref: "User"}],
article: [{type: Schema.ObjectId, ref: "Article"}],
place: Number,
read: Number,
starred: Number,
source: String
});
mongoose.model("UserArticle",UserArticle);
var Log = new Schema({
user: [{type: Schema.ObjectId, ref: "User"}],
action: Number, // O => Insert, 1 => Update, 2 => Delete
uarticle: [{type: Schema.ObjectId, ref: "UserArticle"}],
timestamp: { type: Number, default: Math.round((new Date()).getTime() / 1000) }
});
mongoose.model("Log",Log);
When I want to retrive the log I use the follwing code:
var log = mongoose.model('Log');
log
.where("user", req.session.user)
.desc("timestamp")
.populate("uarticle")
.populate("uarticle.article")
.run(function (err, articles) {
if (err) {
console.log(err);
res.send(500);
return;
}
res.json(articles);
As you can see, I want mongoose to populate the "uarticle" field from the Log collection and, then, I want to populate the "article" field of the UserArticle ("uarticle").
But, using this code, Mongoose only populates "uarticle" using the UserArticle Model, but not the article field inside of uarticle.
Is it possible to accomplish it using Mongoose and populate() or I should do something else?
Thank you,
From what I've checked in the documentation and from what I hear from you, this cannot be achieved, but you can populate the "uarticle.article" documents yourself in the callback function.
However I want to point out another aspect which I consider more important. You have documents in collection A which reference collection B, and in collection B's documents you have another reference to documents in collection C.
You are either doing this wrong (I'm referring to the database structure), or you should be using a relational database such as MySQL here. MongoDB's power relies in the fact you can embed more information in documents, thus having to make lesser queries (having your data in a single collection). While referencing something is ok, having a reference and then another reference doesn't seem like you're taking the full advantage of MongoDB here.
Perhaps you would like to share your situation and the database structure so we could help you out more.
You can use the mongoose-deep-populate plugin to do this. Usage:
User.find({}, function (err, users) {
User.deepPopulate(users, 'uarticle.article', function (err, users) {
// now each user document includes uarticle and each uarticle includes article
})
})
Disclaimer: I'm the author of the plugin.
I faced the same problem,but after hours of efforts i find the solution.It can be without using any external plugin:)
applicantListToExport: function (query, callback) {
this
.find(query).select({'advtId': 0})
.populate({
path: 'influId',
model: 'influencer',
select: { '_id': 1,'user':1},
populate: {
path: 'userid',
model: 'User'
}
})
.populate('campaignId',{'campaignTitle':1})
.exec(callback);
}
Mongoose v5.5.5 seems to allow populate on a populated document.
You can even provide an array of multiple fields to populate on the populated document
var batch = await mstsBatchModel.findOne({_id: req.body.batchId})
.populate({path: 'loggedInUser', select: 'fname lname', model: 'userModel'})
.populate({path: 'invoiceIdArray', model: 'invoiceModel',
populate: [
{path: 'updatedBy', select: 'fname lname', model: 'userModel'},
{path: 'createdBy', select: 'fname lname', model: 'userModel'},
{path: 'aircraftId', select: 'tailNum', model: 'aircraftModel'}
]});
how about something like:
populate_deep = function(type, instance, complete, seen)
{
if (!seen)
seen = {};
if (seen[instance._id])
{
complete();
return;
}
seen[instance._id] = true;
// use meta util to get all "references" from the schema
var refs = meta.get_references(meta.schema(type));
if (!refs)
{
complete();
return;
}
var opts = [];
for (var i=0; i<refs.length; i++)
opts.push({path: refs[i].name, model: refs[i].ref});
mongoose.model(type).populate(instance, opts, function(err,o){
utils.forEach(refs, function (ref, next) {
if (ref.is_array)
utils.forEach(o[ref.name], function (v, lnext) {
populate_deep(ref.ref_type, v, lnext, seen);
}, next);
else
populate_deep(ref.ref_type, o[ref.name], next, seen);
}, complete);
});
}
meta utils is rough... want the src?
or you can simply pass an obj to the populate as:
const myFilterObj = {};
const populateObj = {
path: "parentFileds",
populate: {
path: "childFileds",
select: "childFiledsToSelect"
},
select: "parentFiledsToSelect"
};
Model.find(myFilterObj)
.populate(populateObj).exec((err, data) => console.log(data) );

MongoDB: output 'id' instead of '_id'

I am using mongoose (node), what is the best way to output id instead of _id?
Given you're using Mongoose, you can use 'virtuals', which are essentially fake fields that Mongoose creates. They're not stored in the DB, they just get populated at run time:
// Duplicate the ID field.
Schema.virtual('id').get(function(){
return this._id.toHexString();
});
// Ensure virtual fields are serialised.
Schema.set('toJSON', {
virtuals: true
});
Any time toJSON is called on the Model you create from this Schema, it will include an 'id' field that matches the _id field Mongo generates. Likewise you can set the behaviour for toObject in the same way.
See:
http://mongoosejs.com/docs/api.html
http://mongoosejs.com/docs/guide.html#toJSON
http://mongoosejs.com/docs/guide.html#toObject
You can abstract this into a BaseSchema all your models then extend/invoke to keep the logic in one place. I wrote the above while creating an Ember/Node/Mongoose app, since Ember really prefers to have an 'id' field to work with.
As of Mongoose v4.0 part of this functionality is supported out of the box. It's no longer required to manually add a virtual id field as explained by #Pascal Zajac.
Mongoose assigns each of your schemas an id virtual getter by default
which returns the documents _id field cast to a string, or in the case
of ObjectIds, its hexString. If you don't want an id getter added to
your schema, you may disable it passing this option at schema
construction time. Source
However, to export this field to JSON, it's still required to enable serialization of virtual fields:
Schema.set('toJSON', {
virtuals: true
});
I used this :
schema.set('toJSON', {
virtuals: true,
versionKey:false,
transform: function (doc, ret) { delete ret._id }
});
I think it would be great if they automatically suppress _id when virtuals is true.
I create a toClient() method on my models where I do this. It's also a good place to rename/remove other attributes you don't want to send to the client:
Schema.method('toClient', function() {
var obj = this.toObject();
//Rename fields
obj.id = obj._id;
delete obj._id;
return obj;
});
Here is an alternative version of the answer provided by #user3087827. If you find that schema.options.toJSON is undefined then you can use:
schema.set('toJSON', {
transform: function (doc, ret, options) {
ret.id = ret._id;
delete ret._id;
delete ret.__v;
}
});
//Transform
Schema.options.toJSON.transform = function (doc, ret, options) {
// remove the _id of every document before returning the result
ret.id = ret._id;
delete ret._id;
delete ret.__v;
}
there is a "Schema.options.toObject.transform" property to do the reverse or you could just setup as a virtual id.
If you want to use id instead of _id globally then you can set toJSON config on mongoose object(starting from v5.3):
mongoose.set('toJSON', {
virtuals: true,
transform: (doc, converted) => {
delete converted._id;
}
});
Overwrite default method toJSON by new one:
schema.method('toJSON', function () {
const { __v, _id, ...object } = this.toObject();
object.id = _id;
return object;
});
There is also normalize-mongoose a simple package that removes _id and __v for you.
From something like this:
import mongoose from 'mongoose';
import normalize from 'normalize-mongoose';
const personSchema = mongoose.Schema({ name: String });
personSchema.plugin(normalize);
const Person = mongoose.model('Person', personSchema);
const someone = new Person({ name: 'Abraham' });
const result = someone.toJSON();
console.log(result);
So let's say you have something like this:
{
"_id": "5dff03d3218b91425b9d6fab",
"name": "Abraham",
"__v": 0
}
You will get this output:
{
"id": "5dff03d3218b91425b9d6fab",
"name": "Abraham"
}
I created an easy to use plugin for this purpose that I apply for all my projects and to all schema's globally. It converts _id to id and strips the __v parameter as well.
So it converts:
{
"_id": "400e8324a71d4410b9dc3980b5f8cdea",
"__v": 2,
"name": "Item A"
}
To a simpler and cleaner:
{
"id": "400e8324a71d4410b9dc3980b5f8cdea",
"name": "Item A"
}
Usage as a global plugin:
const mongoose = require('mongoose');
mongoose.plugin(require('meanie-mongoose-to-json'));
Or for a specific schema:
const mongoose = require('mongoose');
const Schema = mongoose.Schema;
const MySchema = new Schema({});
MySchema.plugin(require('meanie-mongoose-to-json'));
Hope this helps someone.
You can also use the aggregate function when searching for items to return. $project will allow you to create fields, which you can do and assign it to _id.
<model>.aggregate([{$project: {_id: 0, id: '$_id'}], (err, res) => {
//
})
If you are using lodash to pick the elements you want, this will work for you.
UserSchema.virtual('id').get(function(){
return this._id.toHexString();
});
UserSchema.set('toObject', { virtuals: true })
UserSchema.methods.toJSON = function() {
return _.pick(
this.toObject(),
['id','email','firstName','lastName','username']
);
Override toJSONmethod for specific model schema.
https://mongoosejs.com/docs/api.html#schema_Schema-method
YourSchema.methods.toJSON = function () {
return {
id: this._id,
some_field: this.some_field,
created_at: this.createdAt
}
}
Create a base schema
import { Schema } from "mongoose";
export class BaseSchema extends Schema {
constructor(sche: any) {
super(sche);
this.set('toJSON', {
virtuals: true,
transform: (doc, converted) => {
delete converted._id;
}
});
}
}
Now in your mongoose model, use BaseSchema instead of Schema
import mongoose, { Document} from 'mongoose';
import { BaseSchema } from '../../helpers/mongoose';
const UserSchema = new BaseSchema({
name: String,
age: Number,
});
export interface IUser {
name: String,
age: Number,
}
interface IPlanModel extends IUser, Document { }
export const PlanDoc = mongoose.model<IPlanModel>('User', UserSchema);
Typescript implementation of #Pascal Zajac answer
There's another driver that does that http://alexeypetrushin.github.com/mongo-lite set convertId option to true. See "Defaults & Setting" section for more details.
Mongoose assigns each of your schemas an id virtual getter by default which returns the document's _id field cast to a string, or in the case of ObjectIds, its hexString.
https://mongoosejs.com/docs/guide.html
You can also use pre 'save' hook:
TouSchema.pre('save', function () {
if (this.isNew) {
this._doc.id = this._id;
}
}
JSON.parse(JSON.stringify(doc.toJSON()))