Mongoose: deleteOne middleware for cascading delete not working - mongodb

With remove being deprecated in Mongoose 5.7.13, I want to use deleteOne instead. I need to get the id of the deleted document so that I can then delete further related documents in other collections in a cascade. I thought that "this" within the context of the pre middleware hook was meant to refer to the removed document, but instead it's just an empty object. Is there a canonical working example of this? I'm still currently using 5.7.12 at this point - will that make a difference here?
Here is the code I'm currently working with. The issue is that I can't get the projectId at the start because the reference is completely empty. Doing this on post rather than pre, or switching the option to run on query rather than document all yield the same result.
ProjectSchema.pre("deleteOne", {document:true}, (next) => {
const projectId = this._id;
ListModel.find({parentProject:projectId}, (err, lists) => {
if(err){
console.log("error cascading project delete to lists", {err});
}
lists.map(list => {
ListModel.deleteOne({_id:list._id}, (err, result) => {
if(err) {
console.log("error on project delete cascade", {err});
}
});
});
});
});

It depends whether you call deleteOne on document or on model. The later just have no document to bind it to.
The former gives you the document as you expect:
const project = await ProjectModel.findOne();
project.deleteOne();
The later gives you the Query. There is no _id in the query, but it has this.op for example, which in this middleware will be "deleteOne":
await ProjectModel.deleteOne();
The only way to get the document id in this case is to ensure it is provided in the query:
await ProjectModel.deleteOne({_id: "alex"});
Then you can get it in the middleware from the filter:
const projectId = this.getFilter()["_id"]
You can specify query: false in second parameter of the middleware to ensure the it is not invoked when you call deleteOne on model. So the best you can do:
ProjectSchema.pre("deleteOne", {document:true, query: false}, (next) => {
const projectId = this._id;
....
});
ProjectSchema.pre("deleteOne", {document:false, query: true}, (next) => {
const projectId = this.getFilter()["_id"];
if (typeof projectId === "undefined") {
// no way to make cascade deletion since there is no _id
// in the delete query
// I would throw an exception, but it's up to you how to deal with it
// to ensure data integrity
}
});
Please take a look at corresponding tests on v5.7.12: https://github.com/Automattic/mongoose/blob/5.7.12/test/model.middleware.test.js#L436

In the mongoose docs it says "Model.deleteOne() does not trigger pre('remove') or post('remove') hooks."
There is solution if you can refactor your delete operations with findByIdAndDelete, it triggers the findOneAndDelete middleware,
So we can add this middleware to Project Schema.
Project model:
const mongoose = require("mongoose");
const ProjectChild = require("./projectChild");
const ProjectSchema = new mongoose.Schema({
name: String
});
ProjectSchema.post("findOneAndDelete", async function(doc) {
console.log(doc);
if (doc) {
const deleteResult = await ProjectChild.deleteMany({
parentProject: doc._id
});
console.log("Child delete result: ", deleteResult);
}
});
module.exports = mongoose.model("Project", ProjectSchema);
ProjectChild model:
const mongoose = require("mongoose");
const projectChildSchema = new mongoose.Schema({
name: String,
parentProject: {
type: mongoose.Schema.Types.ObjectId,
ref: "Project"
}
});
module.exports = mongoose.model("ProjectChild", projectChildSchema);
I created a project like this:
{
"_id": "5dea699cb10c442260245abf",
"name": "Project 1",
"__v": 0
}
And created 2 project child for this project:
Child 1
{
"_id": "5dea69c7b10c442260245ac0",
"name": "Child 1 (project 1)",
"parentProject": "5dea699cb10c442260245abf",
"__v": 0
}
Child 2
{
"_id": "5dea69e8b10c442260245ac1",
"name": "Child 2 (project 1)",
"parentProject": "5dea699cb10c442260245abf",
"__v": 0
}
I created a sample route to delete a project by its id like this:
router.delete("/project/:id", async (req, res) => {
const result = await Project.findByIdAndDelete(req.params.id);
res.send(result);
});
When I send a DELETE request to this route, we see the following info in the console:
console.log(doc);
{ _id: 5dea699cb10c442260245abf, name: 'Project 1', __v: 0 }
console.log("Child delete result: ", deleteResult);
Child delete result: { n: 2, ok: 1, deletedCount: 2 }
So we could deleted the 2 children of the project, when we deleted the project.
As an alternative you can also use findOneAndRemove, it triggers findOneAndRemove post middleware.
So in the ProjectSchema we replace the post middleware like this:
ProjectSchema.post("findOneAndRemove", async function(doc) {
console.log(doc);
if (doc) {
const deleteResult = await ProjectChild.deleteMany({
parentProject: doc._id
});
console.log("Child delete result: ", deleteResult);
}
});
When we use a findOneAndRemove operation, the result will be the same as the first alternative:
const result = await Project.findOneAndRemove({ _id: req.params.id });

Related

Why does express/mongodb updateOne post allow $set of some values, but will not update others?

I am baffled by this post method, it will update fields 'x and y', but any attempt to set an array of widgets fails.
It is finding the correct item to update, passing all the required information through, but it will not allow insertion of, or update to 'widgets' fields.
Even if I remove the data intended for widgets and arbitrarily send through 'foo' it will not update with a field 'widgets'.
What am I doing wrong here???
API Call to Update Widgets. The Arbitrary X and Y values will update on the database, but any attempt to update widget makes no change
const saveUpdatedWidgets = async (update, _id) => {
console.log("called to update widgets ",update.widgets," in pagecard saveUpdatedWidgets")
let widgetObject = []
for(let u=0;u<update.widgets.length;u++){
widgetObject.push({
id: update.widgets[u].id,
text: update.widgets[u].text
})
}
Api.withToken().post('/pagewidget/'+_id,
{widgets: widgetObject, x:250, y:250}
).then(function (response) {
console.log("?worked ",response.data)
}).catch(function (error) {
console.log("page save failed for some reason on pagecard: ",error.response);
});
};
This will return the following in the console:
Code for post method is:
//THIS ROUTER WILL NOT UPDATE ANY WIDGETS FOR SOME REASON
router.post('/pagewidget/:_id',auth, async(req,res)=>{
console.log("request to update ",req.body," for id ",req.params," in pagewidgetsave post")
const query = { "_id": req.params };
const addedWidgets = req.body;
const newValues = { $set: addedWidgets }
try {
const thePage = await Pages.updateOne( query, newValues);
res.status(201).send(thePage)
console.log("updated Page: ",thePage);
}
catch(e){
console.log(e);
res.status(400).send(e)
}
})
Results from the console running node shows that values are going through, but only x and y actually update in database..
Here is the axios api.js file if there are any issues here:
import axios from 'axios';
const baseURL = process.env.REACT_APP_BASE_URL || "http://localhost:3001"
export default {
noToken() {
return axios.create({
baseURL: baseURL
});
},
withToken() {
const tokenStr = window.sessionStorage.getItem("token")
return axios.create({
baseURL: baseURL,
headers: {"Authorization" : `Bearer ${tokenStr}`}
});
}
}
What is going on!!?? It finds the page OK, and updates x and y values, but can't update widgets, even if the values for widget are just a string or number...
I found the issue. the MongoDB documentation doesn't mention this too well, and in its examples for updateOne() it passes an object for the update argument.
BUT, if you are setting a new field, this argument must be wrapped inside an array to use $set, this is because it can accept both methods to $set and to $unset. (see mongoDB docs)
(i.e. updateOne({query} , [{$set: {field:"value"}, {$unset: {otherfield:"othervalue"}])
In the end the post method just had to change to const thePage = await Pages.updateOne( query, [newValues]); (with newValues stored as an object inside an array, to allow addition of $unset if it was needed.
This is why it would update existing values OK, but it would not set new values into the database.
What a journey....
Full code for post method here
router.post('/pagewidget/:_id',auth, async(req,res)=>{
const query = {"_id": req.params._id};
const addedWidgets = req.body;
const newValues = { $set: addedWidgets }
try {
const thePage = await Pages.updateOne( query, [newValues]);
res.status(201).send(thePage)
console.log("updated Page: ",thePage);
}
catch(e){
console.log(e);
res.status(400).send(e)
}
})

Circular Reference Issue in Mongoose pre-hook

In my MongoDB/Node backend environment I am using Mongoose pre and post hook middleware to check what's changed on the document, in order to create some system notes as a result.
One problem I'm running into is that when I try and lookup the record for the document in question I get a "Customer.findOne()" is not a function error. This is ONLY a problem when I'm looking up a record from the same collection from which the model just launched this pre and post hook triggers file. In other words, if my "Customer" model kicks off functions in a pre hook function in an external file, then I get an error if I then try and lookup a Customer with a standard findOne():
My customer model looks something like this:
module.exports = mongoose.model(
"Customer",
mongoose
.Schema(
{
__v: {
type: Number,
select: false
},
deleted: {
type: Boolean,
default: false
},
// Other props
searchResults: [
{
matchKey: String,
matchValue: String
}
]
},
{
timestamps: true
}
)
.pre("save", function(next) {
const doc = this;
trigger.preSave(doc);
next();
})
.post("save", function(doc) {
trigger.postSave(doc);
})
.post("update", function(doc) {
trigger.postSave(doc);
})
.post("findOneAndUpdate", function(doc) {
trigger.postSave(doc);
})
);
... the problematic findOne() function in the triggers file being called from the model looks like this:
const Customer = require("../../models/customer");
exports.preSave = async function(doc) {
this.preSaveDoc = await Customer.findOne({
_id: doc._id
}).exec();
};
To clarify, this is NOT a problem if I'm using a findOne() to lookup a record from a different collection in this same triggers file. Then it works fine. See below when finding a Contact -- no problem here:
const Contact = require("../../models/contact");
exports.preSave = async function(doc) {
this.preSaveDoc = await Contact.findOne({
_id: doc._id
}).exec();
};
The workaround I've found is to use Mongo instead of Mongoose, like so:
exports.preSave = async function(doc) {
let MongoClient = await require("../../config/database")();
let db = MongoClient.connection.db;
db.collection("customers")
.findOne({ _id: doc._id })
.then(doc => {
this.preSaveDoc = doc;
});
}
... but I'd prefer to use Mongoose syntax here. How can I use a findOne() in a pre-hook function being called from the same model/collection as the lookup type?
I have ran similar issue few days ago.
Effectively it is a circular dependency problem. When you call .findOne() on your customer model it doesn't exist as it is not exported yet.
You should probably try something like that :
const customerSchema = mongoose.Schema(...);
customerSchema.pre("save", async function(next) {
const customer = await Customer.findOne({
_id: this._id
}).exec();
trigger.setPreSaveDoc(customer);
next();
})
const Customer = mongoose.model("Customer", customerSchema)
module.export Customer;
Here customer will be defined because it is not called (the pre hook) before its creation.
As an easier way (I am not sure about it) but you could try to move the Contact import in your Trigger file under the save function export. That way I think the decencies may works.
Did it helps ?

Remove multiple documents from mongo in a single query

I have a list of mongo '_id' which I want to delete. Currently I am doing this
# inactive_users --> list of inactive users
for item in inactive_users:
db.users.remove({'_id' : item})
but my problem is the list is too huge... (it might go 100,000 +). So querying for every item in list will only increase the load on server. Is their a way to pass the entire list in mongo query so that I dont have to fire query again and again.
Thank you
db.users.deleteMany({'_id':{'$in':inactive_users}})
List them all and use $in operator:
db.users.remove({_id:{$in:[id1, id2, id3, ... ]}})
You need to pass the ids in a specific format using ObjectId():
db.users.remove({_id: {$in: [ObjectId('Item1'), ObjectId('Item2'), ObjectId('Item2')]}});
Remove doesn't accept integer - you have to use ObjectId instance with _id format as a string.
var collection = db.users;
var usersDelete = [];
var ObjectID = req.mongo.ObjectID; //req is request from express
req.body.forEach(function(item){ //req.body => [{'_id' : ".." , "name" : "john"}]
usersDelete.push(new ObjectID(item._id));
});
collection.remove({'_id':{'$in': usersDelete}},function(){
//res.json(contatos);
});
I had the same question and ran across these answers but it seems the MongoDB manual is recommending deleteMany instead of remove. deleteMany returns the delete count as well as an acknowledgement of the write concern (if the operation succeeded).
const ids = [id1, id2, id3...];
const query = { _id: { $in: ids} };
dbo.collection("users").deleteMany(query, function (err, obj) {
if (err) throw err;
});
Or with an arrow function:
const ids = [id1, id2, id3...];
const query = { _id: { $in: ids} };
dbo.collection("users").deleteMany(query, (err, obj) => {
if (err) throw err;
});
Or better yet, with a promise:
const ids = [id1, id2, id3...];
const query = { _id: { $in: ids} };
dbo.collection("users").deleteMany(query)
.then(result => {
console.log("Records Deleted");
console.log(JSON.stringify(result));
//for number removed...
console.log("Removed: " + result["n"]);
})
.catch(err => {
console.log("Error");
console.log(err);
});

Cascade style delete in Mongoose

Is there a way to delete all children of an parent in Mongoose, similar to using MySQLs foreign keys?
For example, in MySQL I'd assign a foreign key and set it to cascade on delete. Thus, if I were to delete a client, all applications and associated users would be removed as well.
From a top level:
Delete Client
Delete Sweepstakes
Delete Submissions
Sweepstakes and submissions both have a field for client_id. Submissions has a field for both sweepstakes_id, and client_id.
Right now, I'm using the following code and I feel that there has to be a better way.
Client.findById(req.params.client_id, function(err, client) {
if (err)
return next(new restify.InternalError(err));
else if (!client)
return next(new restify.ResourceNotFoundError('The resource you requested could not be found.'));
// find and remove all associated sweepstakes
Sweepstakes.find({client_id: client._id}).remove();
// find and remove all submissions
Submission.find({client_id: client._id}).remove();
client.remove();
res.send({id: req.params.client_id});
});
This is one of the primary use cases of Mongoose's 'remove' middleware.
clientSchema.pre('remove', function(next) {
// 'this' is the client being removed. Provide callbacks here if you want
// to be notified of the calls' result.
Sweepstakes.remove({client_id: this._id}).exec();
Submission.remove({client_id: this._id}).exec();
next();
});
This way, when you call client.remove() this middleware is automatically invoked to clean up dependencies.
In case your references are stored other way around, say, client has an array of submission_ids, then in a similar way as accepted answer you can define the following on submissionSchema:
submissionSchema.pre('remove', function(next) {
Client.update(
{ submission_ids : this._id},
{ $pull: { submission_ids: this._id } },
{ multi: true }) //if reference exists in multiple documents
.exec();
next();
});
which will remove the submission's id from the clients' reference arrays on submission.remove().
Here's an other way I found
submissionSchema.pre('remove', function(next) {
this.model('Client').remove({ submission_ids: this._id }, next);
next();
});
I noticed that all of answers here have a pre assigned to the schema and not post.
my solution would be this: (using mongoose 6+)
ClientSchema.post("remove", async function(res, next) {
await Sweepstakes.deleteMany({ client_id: this._id });
await Submission.deleteMany({ client_id: this._id });
next();
});
By definition post gets executed after the process ends pre => process => post.
Now, you're probably wondering how is this different than the other solutions provided here.
What if a server error or the id of that client was not found?
On pre, it would delete all sweeptakes and submissions before the deleting process start for client. Thus, in case of an error, it would be better to cascade delete the other documents once client or the main document gets deleted.
async and await are optional here. However, it matters on large data. so that the user wouldn't get those "going to be deleted" cascade documents data if the delete progress is still on.
At the end, I could be wrong, hopefully this helps someone in their code.
Model
const orderSchema = new mongoose.Schema({
// Множество экземпляров --> []
orderItems: [{
type: mongoose.Schema.Types.ObjectId,
ref: 'OrderItem',
required: true
}],
...
...
});
asyncHandler (optional)
const asyncHandler = fn => (req, res, next) =>
Promise
.resolve(fn(req, res, next))
.catch(next)
module.exports = asyncHandler;
controller
const asyncHandler = require("../middleware/asyncErrHandler.middleware");
// **Models**
const Order = require('../models/order.mongo');
const OrderItem = require('../models/order-item.mongo');
// #desc Delete order
// #route DELETE /api/v1/orders/:id
// #access Private
exports.deleteOrder = asyncHandler(async (req, res, next) => {
let order = await Order.findById(req.params.id)
if (!order) return next(
res.status(404).json({ success: false, data: null })
)
await order.remove().then( items => {
// Cascade delete -OrderItem-
items.orderItems.forEach( el => OrderItem.findById(el).remove().exec())
}).catch(e => { res.status(400).json({ success: false, data: e }) });
res.status(201).json({ success: true, data: null });
});
https://mongoosejs.com/docs/api/model.html#model_Model-remove

MongoDB: output 'id' instead of '_id'

I am using mongoose (node), what is the best way to output id instead of _id?
Given you're using Mongoose, you can use 'virtuals', which are essentially fake fields that Mongoose creates. They're not stored in the DB, they just get populated at run time:
// Duplicate the ID field.
Schema.virtual('id').get(function(){
return this._id.toHexString();
});
// Ensure virtual fields are serialised.
Schema.set('toJSON', {
virtuals: true
});
Any time toJSON is called on the Model you create from this Schema, it will include an 'id' field that matches the _id field Mongo generates. Likewise you can set the behaviour for toObject in the same way.
See:
http://mongoosejs.com/docs/api.html
http://mongoosejs.com/docs/guide.html#toJSON
http://mongoosejs.com/docs/guide.html#toObject
You can abstract this into a BaseSchema all your models then extend/invoke to keep the logic in one place. I wrote the above while creating an Ember/Node/Mongoose app, since Ember really prefers to have an 'id' field to work with.
As of Mongoose v4.0 part of this functionality is supported out of the box. It's no longer required to manually add a virtual id field as explained by #Pascal Zajac.
Mongoose assigns each of your schemas an id virtual getter by default
which returns the documents _id field cast to a string, or in the case
of ObjectIds, its hexString. If you don't want an id getter added to
your schema, you may disable it passing this option at schema
construction time. Source
However, to export this field to JSON, it's still required to enable serialization of virtual fields:
Schema.set('toJSON', {
virtuals: true
});
I used this :
schema.set('toJSON', {
virtuals: true,
versionKey:false,
transform: function (doc, ret) { delete ret._id }
});
I think it would be great if they automatically suppress _id when virtuals is true.
I create a toClient() method on my models where I do this. It's also a good place to rename/remove other attributes you don't want to send to the client:
Schema.method('toClient', function() {
var obj = this.toObject();
//Rename fields
obj.id = obj._id;
delete obj._id;
return obj;
});
Here is an alternative version of the answer provided by #user3087827. If you find that schema.options.toJSON is undefined then you can use:
schema.set('toJSON', {
transform: function (doc, ret, options) {
ret.id = ret._id;
delete ret._id;
delete ret.__v;
}
});
//Transform
Schema.options.toJSON.transform = function (doc, ret, options) {
// remove the _id of every document before returning the result
ret.id = ret._id;
delete ret._id;
delete ret.__v;
}
there is a "Schema.options.toObject.transform" property to do the reverse or you could just setup as a virtual id.
If you want to use id instead of _id globally then you can set toJSON config on mongoose object(starting from v5.3):
mongoose.set('toJSON', {
virtuals: true,
transform: (doc, converted) => {
delete converted._id;
}
});
Overwrite default method toJSON by new one:
schema.method('toJSON', function () {
const { __v, _id, ...object } = this.toObject();
object.id = _id;
return object;
});
There is also normalize-mongoose a simple package that removes _id and __v for you.
From something like this:
import mongoose from 'mongoose';
import normalize from 'normalize-mongoose';
const personSchema = mongoose.Schema({ name: String });
personSchema.plugin(normalize);
const Person = mongoose.model('Person', personSchema);
const someone = new Person({ name: 'Abraham' });
const result = someone.toJSON();
console.log(result);
So let's say you have something like this:
{
"_id": "5dff03d3218b91425b9d6fab",
"name": "Abraham",
"__v": 0
}
You will get this output:
{
"id": "5dff03d3218b91425b9d6fab",
"name": "Abraham"
}
I created an easy to use plugin for this purpose that I apply for all my projects and to all schema's globally. It converts _id to id and strips the __v parameter as well.
So it converts:
{
"_id": "400e8324a71d4410b9dc3980b5f8cdea",
"__v": 2,
"name": "Item A"
}
To a simpler and cleaner:
{
"id": "400e8324a71d4410b9dc3980b5f8cdea",
"name": "Item A"
}
Usage as a global plugin:
const mongoose = require('mongoose');
mongoose.plugin(require('meanie-mongoose-to-json'));
Or for a specific schema:
const mongoose = require('mongoose');
const Schema = mongoose.Schema;
const MySchema = new Schema({});
MySchema.plugin(require('meanie-mongoose-to-json'));
Hope this helps someone.
You can also use the aggregate function when searching for items to return. $project will allow you to create fields, which you can do and assign it to _id.
<model>.aggregate([{$project: {_id: 0, id: '$_id'}], (err, res) => {
//
})
If you are using lodash to pick the elements you want, this will work for you.
UserSchema.virtual('id').get(function(){
return this._id.toHexString();
});
UserSchema.set('toObject', { virtuals: true })
UserSchema.methods.toJSON = function() {
return _.pick(
this.toObject(),
['id','email','firstName','lastName','username']
);
Override toJSONmethod for specific model schema.
https://mongoosejs.com/docs/api.html#schema_Schema-method
YourSchema.methods.toJSON = function () {
return {
id: this._id,
some_field: this.some_field,
created_at: this.createdAt
}
}
Create a base schema
import { Schema } from "mongoose";
export class BaseSchema extends Schema {
constructor(sche: any) {
super(sche);
this.set('toJSON', {
virtuals: true,
transform: (doc, converted) => {
delete converted._id;
}
});
}
}
Now in your mongoose model, use BaseSchema instead of Schema
import mongoose, { Document} from 'mongoose';
import { BaseSchema } from '../../helpers/mongoose';
const UserSchema = new BaseSchema({
name: String,
age: Number,
});
export interface IUser {
name: String,
age: Number,
}
interface IPlanModel extends IUser, Document { }
export const PlanDoc = mongoose.model<IPlanModel>('User', UserSchema);
Typescript implementation of #Pascal Zajac answer
There's another driver that does that http://alexeypetrushin.github.com/mongo-lite set convertId option to true. See "Defaults & Setting" section for more details.
Mongoose assigns each of your schemas an id virtual getter by default which returns the document's _id field cast to a string, or in the case of ObjectIds, its hexString.
https://mongoosejs.com/docs/guide.html
You can also use pre 'save' hook:
TouSchema.pre('save', function () {
if (this.isNew) {
this._doc.id = this._id;
}
}
JSON.parse(JSON.stringify(doc.toJSON()))