Related
In my MongoDB/Node backend environment I am using Mongoose pre and post hook middleware to check what's changed on the document, in order to create some system notes as a result.
One problem I'm running into is that when I try and lookup the record for the document in question I get a "Customer.findOne()" is not a function error. This is ONLY a problem when I'm looking up a record from the same collection from which the model just launched this pre and post hook triggers file. In other words, if my "Customer" model kicks off functions in a pre hook function in an external file, then I get an error if I then try and lookup a Customer with a standard findOne():
My customer model looks something like this:
module.exports = mongoose.model(
"Customer",
mongoose
.Schema(
{
__v: {
type: Number,
select: false
},
deleted: {
type: Boolean,
default: false
},
// Other props
searchResults: [
{
matchKey: String,
matchValue: String
}
]
},
{
timestamps: true
}
)
.pre("save", function(next) {
const doc = this;
trigger.preSave(doc);
next();
})
.post("save", function(doc) {
trigger.postSave(doc);
})
.post("update", function(doc) {
trigger.postSave(doc);
})
.post("findOneAndUpdate", function(doc) {
trigger.postSave(doc);
})
);
... the problematic findOne() function in the triggers file being called from the model looks like this:
const Customer = require("../../models/customer");
exports.preSave = async function(doc) {
this.preSaveDoc = await Customer.findOne({
_id: doc._id
}).exec();
};
To clarify, this is NOT a problem if I'm using a findOne() to lookup a record from a different collection in this same triggers file. Then it works fine. See below when finding a Contact -- no problem here:
const Contact = require("../../models/contact");
exports.preSave = async function(doc) {
this.preSaveDoc = await Contact.findOne({
_id: doc._id
}).exec();
};
The workaround I've found is to use Mongo instead of Mongoose, like so:
exports.preSave = async function(doc) {
let MongoClient = await require("../../config/database")();
let db = MongoClient.connection.db;
db.collection("customers")
.findOne({ _id: doc._id })
.then(doc => {
this.preSaveDoc = doc;
});
}
... but I'd prefer to use Mongoose syntax here. How can I use a findOne() in a pre-hook function being called from the same model/collection as the lookup type?
I have ran similar issue few days ago.
Effectively it is a circular dependency problem. When you call .findOne() on your customer model it doesn't exist as it is not exported yet.
You should probably try something like that :
const customerSchema = mongoose.Schema(...);
customerSchema.pre("save", async function(next) {
const customer = await Customer.findOne({
_id: this._id
}).exec();
trigger.setPreSaveDoc(customer);
next();
})
const Customer = mongoose.model("Customer", customerSchema)
module.export Customer;
Here customer will be defined because it is not called (the pre hook) before its creation.
As an easier way (I am not sure about it) but you could try to move the Contact import in your Trigger file under the save function export. That way I think the decencies may works.
Did it helps ?
I am using pre and post hooks in my MongoDB/Node backend in order to compare a pre-save and post-save version of a document so I can generate notes via model triggers based on what's changed. In one of my models/collections this is working, but in another, it's not working as expected, and I'm not sure why.
In the problem case, some research has determined that even though I am calling a pre hook trigger on an operation that uses a save(), when I console out the doc state passed in that pre hook, it's already had the change applied. In other words, the hook is not firing before the save() operation, but after, from what I can tell.
Here is my relevant model code:
let Schema = mongoose
.Schema(CustomerSchema, {
timestamps: true
})
.pre("save", function(next) {
const doc = this;
console.log("doc in .pre: ", doc); // this should be the pre-save version of the doc, but it is the post-save version
console.log("doc.history.length in model doc: ", doc.history.length);
trigger.preSave(doc);
next();
})
.post("save", function(doc) {
trigger.postSave(doc);
})
.post("update", function(doc) {
trigger.postSave(doc);
});
module.exports = mongoose.model("Customer", Schema);
The relevant part of the save() operation that I'm doing looks like this (all I'm doing is pushing a new element to an array on the doc called "history"):
exports.updateHistory = async function(req, res) {
let request = new CentralReqController(
req,
res,
{
// Allowed Parameters
id: {
type: String
},
stageId: {
type: String
},
startedBy: {
type: String
}
},
[
// Required Parameters
"id",
"stageId",
"startedBy"
]
);
let newHistoryObj = {
stageId: request.parameters.stageId,
startDate: new Date(),
startedBy: request.parameters.startedBy,
completed: false
};
let customerToUpdate = await Customer.findOne({
_id: request.parameters.id
}).exec();
let historyArray = await customerToUpdate.history;
console.log("historyArray.length before push in update func: ", historyArray.length);
historyArray.push(newHistoryObj);
await customerToUpdate.save((err, doc) => {
if (doc) console.log("history update saved...");
if (err) return request.sendError("Customer history update failed.", err);
});
};
So, my question is, if a pre hook on a save() operation is supposed to fire BEFORE the save() happens, why does the document I look at via my console.log show a document that's already had the save() operation done on it?
You are a bit mistaken on what the pre/post 'save' hooks are doing. In pre/post hook terms, save is the actual save operation to the database. That being said, the this you have in the pre('save') hook, is the object you called .save() on, not the updated object from the database. For example:
let myCustomer = req.body.customer; // some customer object
// Update the customer object
myCustomer.name = 'Updated Name';
// Save the customer
myCustomer.save();
We just updated the customers name. When the .save() is called, it triggers the hooks, like you stated above. Only the difference is, the this in the pre('save') hook is the same object as myCustomer, not the updated object from the database. On the contrary, the doc object in the `post('save') hook IS the updated object from the database.
Schema.pre('save', function(next) {
console.log(this); // Modified object (myCustomer), not from DB
)};
Schema.post('save', function(doc) {
console.log(doc); // Modified object DIRECTLY from DB
});
If a user has an array called "tags":
var User = new Schema({
email: {
type: String,
unique: true,
required: true
},
tags: [{
type: mongoose.Schema.Types.ObjectId,
ref:'Tag',
required: true
}],
created: {
type: Date,
default: Date.now
}
});
and I do a populate('tags') on a query:
User.findById(req.params.id)
.populate("tags")
.exec(function(err, user) { ... });
If one of the tags in the list has actually been deleted, is there a way to remove this dead reference in "tags"?
Currently, the returned user object IS returning the desired result -- ie. only tags that actually exist are in the tags array... however, if I look at the underlying document in mongodb, it still contains the dead tag id in the array.
Ideally, I would like to clean these references up lazily. Does anyone know of a good strategy to do this?
I've tried to find some built-in way to do that but seems that mongoose doesn't provide such functionality.
So I did something like this
User.findById(userId)
.populate('tags')
.exec((err, user) => {
user.tags = user.tags.filter(tag => tag != null);
res.send(user); // Return result as soon as you can
user.save(); // Save user without dead refs to database
})
This way every time you fetch user you also delete dead refs from the document. Also, you can create isUpdated boolean variable to not call user.save if there was no deleted refs.
const lengthBeforeFilter = user.tags.length;
let isUpdated = user.tags.length;
user.tags = user.tags.filter(tag => tag != null);
isUpdated = lengthBeforeFilter > user.tags.length;
res.send(user);
if (isUpdated) {
user.save();
}
Assuming you delete these tags via mongoose, you can use the post middleware.
This will be executed after you've deleted a tag.
tagSchema.post('remove', function(doc) {
//find all users with referenced tag
//remove doc._id from array
});
its sample retainNullValues: true
Example:
User.findById(req.params.id)
.populate({
path: "tag",
options: {
retainNullValues: true
}
})
I am using mongoose (node), what is the best way to output id instead of _id?
Given you're using Mongoose, you can use 'virtuals', which are essentially fake fields that Mongoose creates. They're not stored in the DB, they just get populated at run time:
// Duplicate the ID field.
Schema.virtual('id').get(function(){
return this._id.toHexString();
});
// Ensure virtual fields are serialised.
Schema.set('toJSON', {
virtuals: true
});
Any time toJSON is called on the Model you create from this Schema, it will include an 'id' field that matches the _id field Mongo generates. Likewise you can set the behaviour for toObject in the same way.
See:
http://mongoosejs.com/docs/api.html
http://mongoosejs.com/docs/guide.html#toJSON
http://mongoosejs.com/docs/guide.html#toObject
You can abstract this into a BaseSchema all your models then extend/invoke to keep the logic in one place. I wrote the above while creating an Ember/Node/Mongoose app, since Ember really prefers to have an 'id' field to work with.
As of Mongoose v4.0 part of this functionality is supported out of the box. It's no longer required to manually add a virtual id field as explained by #Pascal Zajac.
Mongoose assigns each of your schemas an id virtual getter by default
which returns the documents _id field cast to a string, or in the case
of ObjectIds, its hexString. If you don't want an id getter added to
your schema, you may disable it passing this option at schema
construction time. Source
However, to export this field to JSON, it's still required to enable serialization of virtual fields:
Schema.set('toJSON', {
virtuals: true
});
I used this :
schema.set('toJSON', {
virtuals: true,
versionKey:false,
transform: function (doc, ret) { delete ret._id }
});
I think it would be great if they automatically suppress _id when virtuals is true.
I create a toClient() method on my models where I do this. It's also a good place to rename/remove other attributes you don't want to send to the client:
Schema.method('toClient', function() {
var obj = this.toObject();
//Rename fields
obj.id = obj._id;
delete obj._id;
return obj;
});
Here is an alternative version of the answer provided by #user3087827. If you find that schema.options.toJSON is undefined then you can use:
schema.set('toJSON', {
transform: function (doc, ret, options) {
ret.id = ret._id;
delete ret._id;
delete ret.__v;
}
});
//Transform
Schema.options.toJSON.transform = function (doc, ret, options) {
// remove the _id of every document before returning the result
ret.id = ret._id;
delete ret._id;
delete ret.__v;
}
there is a "Schema.options.toObject.transform" property to do the reverse or you could just setup as a virtual id.
If you want to use id instead of _id globally then you can set toJSON config on mongoose object(starting from v5.3):
mongoose.set('toJSON', {
virtuals: true,
transform: (doc, converted) => {
delete converted._id;
}
});
Overwrite default method toJSON by new one:
schema.method('toJSON', function () {
const { __v, _id, ...object } = this.toObject();
object.id = _id;
return object;
});
There is also normalize-mongoose a simple package that removes _id and __v for you.
From something like this:
import mongoose from 'mongoose';
import normalize from 'normalize-mongoose';
const personSchema = mongoose.Schema({ name: String });
personSchema.plugin(normalize);
const Person = mongoose.model('Person', personSchema);
const someone = new Person({ name: 'Abraham' });
const result = someone.toJSON();
console.log(result);
So let's say you have something like this:
{
"_id": "5dff03d3218b91425b9d6fab",
"name": "Abraham",
"__v": 0
}
You will get this output:
{
"id": "5dff03d3218b91425b9d6fab",
"name": "Abraham"
}
I created an easy to use plugin for this purpose that I apply for all my projects and to all schema's globally. It converts _id to id and strips the __v parameter as well.
So it converts:
{
"_id": "400e8324a71d4410b9dc3980b5f8cdea",
"__v": 2,
"name": "Item A"
}
To a simpler and cleaner:
{
"id": "400e8324a71d4410b9dc3980b5f8cdea",
"name": "Item A"
}
Usage as a global plugin:
const mongoose = require('mongoose');
mongoose.plugin(require('meanie-mongoose-to-json'));
Or for a specific schema:
const mongoose = require('mongoose');
const Schema = mongoose.Schema;
const MySchema = new Schema({});
MySchema.plugin(require('meanie-mongoose-to-json'));
Hope this helps someone.
You can also use the aggregate function when searching for items to return. $project will allow you to create fields, which you can do and assign it to _id.
<model>.aggregate([{$project: {_id: 0, id: '$_id'}], (err, res) => {
//
})
If you are using lodash to pick the elements you want, this will work for you.
UserSchema.virtual('id').get(function(){
return this._id.toHexString();
});
UserSchema.set('toObject', { virtuals: true })
UserSchema.methods.toJSON = function() {
return _.pick(
this.toObject(),
['id','email','firstName','lastName','username']
);
Override toJSONmethod for specific model schema.
https://mongoosejs.com/docs/api.html#schema_Schema-method
YourSchema.methods.toJSON = function () {
return {
id: this._id,
some_field: this.some_field,
created_at: this.createdAt
}
}
Create a base schema
import { Schema } from "mongoose";
export class BaseSchema extends Schema {
constructor(sche: any) {
super(sche);
this.set('toJSON', {
virtuals: true,
transform: (doc, converted) => {
delete converted._id;
}
});
}
}
Now in your mongoose model, use BaseSchema instead of Schema
import mongoose, { Document} from 'mongoose';
import { BaseSchema } from '../../helpers/mongoose';
const UserSchema = new BaseSchema({
name: String,
age: Number,
});
export interface IUser {
name: String,
age: Number,
}
interface IPlanModel extends IUser, Document { }
export const PlanDoc = mongoose.model<IPlanModel>('User', UserSchema);
Typescript implementation of #Pascal Zajac answer
There's another driver that does that http://alexeypetrushin.github.com/mongo-lite set convertId option to true. See "Defaults & Setting" section for more details.
Mongoose assigns each of your schemas an id virtual getter by default which returns the document's _id field cast to a string, or in the case of ObjectIds, its hexString.
https://mongoosejs.com/docs/guide.html
You can also use pre 'save' hook:
TouSchema.pre('save', function () {
if (this.isNew) {
this._doc.id = this._id;
}
}
JSON.parse(JSON.stringify(doc.toJSON()))
I'm using the Mongoose Library for accessing MongoDB with node.js
Is there a way to remove a key from a document? i.e. not just set the value to null, but remove it?
User.findOne({}, function(err, user){
//correctly sets the key to null... but it's still present in the document
user.key_to_delete = null;
// doesn't seem to have any effect
delete user.key_to_delete;
user.save();
});
In early versions, you would have needed to drop down the node-mongodb-native driver. Each model has a collection object that contains all the methods that node-mongodb-native offers. So you can do the action in question by this:
User.collection.update({_id: user._id}, {$unset: {field: 1 }});
Since version 2.0 you can do:
User.update({_id: user._id}, {$unset: {field: 1 }}, callback);
And since version 2.4, if you have an instance of a model already you can do:
doc.field = undefined;
doc.save(callback);
You'll want to do this:
User.findOne({}, function(err, user){
user.key_to_delete = undefined;
user.save();
});
I use mongoose and using any of the above functions did me the requirement. The function compiles error free but the field would still remain.
user.set('key_to_delete', undefined, {strict: false} );
did the trick for me.
At mongo syntax to delete some key you need do following:
{ $unset : { field : 1} }
Seems at Mongoose the same.
Edit
Check this example.
Try:
User.findOne({}, function(err, user){
// user.key_to_delete = null; X
`user.key_to_delete = undefined;`
delete user.key_to_delete;
user.save();
});
if you want to remove a key from collection try this method.
db.getCollection('myDatabaseTestCollectionName').update({"FieldToDelete": {$exists: true}}, {$unset:{"FieldToDelete":1}}, false, true);
Could this be a side problem like using
function (user)
instead of
function(err, user)
for the find's callback ? Just trying to help with this as I already had the case.
Mongoose document is NOT a plain javascript object and that's why you can't use delete operator.(Or unset from 'lodash' library).
Your options are to set doc.path = null || undefined or to use Document.toObject() method to turn mongoose doc to plain object and from there use it as usual.
Read more in mongoose api-ref:
http://mongoosejs.com/docs/api.html#document_Document-toObject
Example would look something like this:
User.findById(id, function(err, user) {
if (err) return next(err);
let userObject = user.toObject();
// userObject is plain object
});
the problem with all of these answers is that they work for one field. for example let's say i want delete all fields from my Document if they were an empty string "".
First you should check if field is empty string put it to $unset :
function unsetEmptyFields(updateData) {
const $unset = {};
Object.keys(updatedData).forEach((key) => {
if (!updatedData[key]) {
$unset[key] = 1;
delete updatedData[key];
}
});
updatedData.$unset = $unset;
if (isEmpty(updatedData.$unset)) { delete updatedData.$unset; }
return updatedData;
}
function updateUserModel(data){
const updatedData = UnsetEmptyFiled(data);
const Id = "";
User.findOneAndUpdate(
{ _id: Id },
updatedData, { new: true },
);
}
I believe that, if you desire remove a specific field into a collection, you should do this:
User.remove ({ key_to_delete: req.params.user.key_to_delete});
you can use
delete user._doc.key