Populate none ref objects of a nested array - mongodb

I'm working on a project that uses:
"#nestjs/core": "^7.0.0",
"#nestjs/mongoose": "^7.0.0",
"mongoose": "^5.9.12",
// ...
"typescript": "^3.7.4",
With mongoose/mongoDB config:
uri: MONGO_DB_URI,
useUnifiedTopology: true,
useNewUrlParser: true,
useFindAndModify: false,
useCreateIndex: true,
I'm trying to build a simple CRUD for this model:
export const ContactSchema = new mongoose.Schema(
{
source_id: { type: String, required: true },
firstName: { type: String, trim: true },
lastName: { type: String, trim: true },
phones: [
{
number: {
type: String,
required: true,
unique: true,
validate: {
validator: function(value) {
const phoneNumber = parsePhoneNumberFromString(value)
return phoneNumber && phoneNumber.isValid()
},
},
},
type: {
type: String,
default: function() {
return parsePhoneNumberFromString(this.number).getType() || "N/A"
},
},
code: {
type: Number,
default: function() {
return parsePhoneNumberFromString(this.number).countryCallingCode || undefined
},
},
national: {
type: Number,
default: function() {
return parsePhoneNumberFromString(this.number).nationalNumber || undefined
},
},
},
],
email: { type: String, unique: true, required: true, lowercase: true, trim: true },
},
{ timestamps: true },
)
ContactSchema.plugin(mongoosePaginate)
Like every CRUD app, I'm willing to have fildAll() & fildOne() routes that return the body of a given Contact with all his info including the list of its phone numbers. So I used:
// ...
async findAll(): Promise<Contact[]> {
return this.contactModel.find()
// then I add
.populate('phones')
}
async findBySourceId(id: string): Promise<Contact> {
return this.contactModel.findOne({ source_id: id })
// then I add
.populate('phones')
}
// ...
All info are well saved in the DB and there is no missing data (neither phones) and I'm sure that it works the beginning without even adding .poplate('x'), but that changed somewhere and it returns now unpopulated phone array.
Now It returns:
{
"_id": "5ebc22072e18637d84bcf6f0",
"firstName": "Maher",
"lastName": "Boubakri",
"phones": [],
"email": "mhb#test.im",
// ...
}
But, It should return:
{
"_id": "5ebc22072e18637d84bcf6f0",
"firstName": "Maher",
"lastName": "Boubakri",
"phones": [
{
"_id": "5ebc22072e18637d8fd948f9",
"number": "+21622123456",
"code": 216,
"type": "MOBILE",
"national": 22123456,
}
],
"email": "mhb#test.im",
// ...
}
Note: It is clear that MongoDB generates _id for every phone object, but, it is not a ref Object.
Any idea will be so helpful,
Thank you.

populate is used to join two (or more) collections using the references
here you don't have any references, so you don't need it
just use find() without populate

Based on #Mohammed 's comment and answer, adding .lean() after updating mongoose fixed the problem.
// ...
async findAll(): Promise<Contact[]> {
return this.contactModel.find().lean()
}
async findBySourceId(id: string): Promise<Contact> {
return this.contactModel.findOne({ source_id: id }).lean()
}
// ...

Related

trying to populate a single field in each object inside an array of objects by dynamically picking the model to use with mongoose

I'm trying to use refPath to reference which collection to pull the population data from inside my schema, and even though it looks identical to the examples I've seen, its just not working.
Here is my schema for statesPersons, not super important, but it contains the activeWork array of objects.
import mongoose, {model, Schema} from "mongoose";
const statesPersonsSchema = new Schema(
{
profileId: {
type: String,
required: true,
unique: true,
},
department: {
type: String,
required: true,
index: true,
},
firstName: String,
lastName: String,
location: String,
org: String,
title: String,
jobDescription: String,
email: {
type: String,
lowercase: true,
},
phoneNumber: String,
activeWork: ["activeWork"],
emailList: [String],
jobAssignments: [String],
affiantInfo: {
affiantInfoTitle: String,
affiantInfoExperience: String,
},
assessments: [
{
assessdBy: {
type: Schema.Types.ObjectId,
ref: "statesPerson",
},
dueDate: Date,
questions: {},
},
],
},
{ strictPopulate: false }
);
export default mongoose.model("statesPersons", statesPersonsSchema);
Here is my schema for activeWork, the array of objects. This has the referenceId that I need to populate as well as the collectionType which I pull what collection it is from.
import mongoose, {model, Schema} from "mongoose";
const activeWorkSchema = new Schema(
{
active: Boolean,
collectionType: {
type: String,
enum: ["messages", "cases"],
},
referenceId: {
type: Schema.Types.ObjectId,
refPath: "collectionType",
},
sentBy: {
type: Schema.Types.String,
ref: "statesPersons",
},
sentTo: {
type: Schema.Types.String,
ref: "statesPersons",
},
timeRecived: Date,
dueDate: Date,
subject: String,
viewed: Boolean,
content: {},
},
{ strictPopulate: false }
);
export default mongoose.model("activeWork", activeWorkSchema);
And here is my query.
export async function getStatesPersonsActiveWorkByProfileId(req, res){
mongoose.set('debug', true);
try{
const { profileId } = req.params
const data = await statesPersons
.find({ profileId })
.populate('statesPersons.activeWork.referenceId')
.exec()
return res.send({
message: "success",
data: data,
status: 200 })
}catch(e) {
console.error(e.message)
return res.send({
message: "couldn't fetch active work",
data: null,
status: 500 })
}
}
its returning with the statesPersons object and the activeWork contains the objectId I need to populate, but its not populating. it looks like this.
"activeWork": [
{
"active": true,
"collectionType": "messages",
"referenceId": "63a49e3052658ce60c1dafcb",
"sentBy": "108416469928574003772",
"dueDate": "2018-02-21T11:16:50.362Z",
"subject": "testing",
"viewed": false,
"_id": "63a49e3052658ce60c1dafce"
I can force it to work by changing the query to be explicit.
const data = await statesPersons
.find({ profileId })
.populate({path: 'activeWork.referenceId', model: 'messages'})
.exec()
which looks like this.
activeWork": [
{
"active": true,
"collectionType": "messages",
"referenceId": {
"_id": "63a49e3052658ce60c1dafcb",
"involvedParties": [
"108416469928574003772",
"100335565301468600000"
],
"comments": [
{
"sender": [
"108416469928574003772"
],
"dateSent": "2022-12-22T18:13:04.604Z",
"content": "There is no way this is going to work.",
"_id": "63a49e3052658ce60c1dafcc"
}
],
But this wont work because I need it to be able to pull what model to use from the collectionType field
sorry for the late response , it seems like you are trying to populate the multilevel documents multilevel population.
here is an example.
db.statesPersonsSchema.find({ profileId }). populate({
path: 'activeWorkSchema',
populate: { path: 'referenceId' }
});

How to create or update within object of objects

I'm trying to access a database with the following difficult schema and want to update the file_name value if it exists, or create a new object with the file_name if it does not. As I have searched, the option { upsert: true, new: true, setDefaultsOnInsert: true }; will update do the second bit, but the problem is how do you access the file_name key nested deep within.
export type _ID = string;
export interface FileSchema {
[_id: _ID]: {
file_name: string;
creation_date: Date;
isPublished: boolean;
content: string;
};
}
export interface AccountSchema {
...
files: FileSchema;
}
const accountSchema = new Schema<AccountSchema>({
...
files: {
type: Map,
unique: true,
of: {
file_name: {
type: String,
required: true,
minlength: 4,
maxlength: 60,
unique: true,
},
creation_date: {
type: Date,
required: true,
},
isPublished: {
type: Boolean,
required: true,
},
content: {
type: String,
required: true,
},
},
},
});
Not familiar with Mongoose but you may be missing dot notation to access nested fields, look at
https://www.mongodb.com/docs/manual/tutorial/query-embedded-documents/#specify-equality-match-on-a-nested-field
I've searched a bit, and have come up with the solution below, although not pretty due to multiple queries. For nested objects with a dynamic key, you must use codes like [files.${file_id}.file_name] (with ``) to access the object within.
Reference 1
Reference 2
const accountSchema = new Schema<AccountSchema>({
...
files: {
type: Schema.Types.Map,
unique: true,
of: Object,
},
})
// checks if file exists, if exist then update file_name
const files: FileSchema | null = await DB_ACCOUNT.findOneAndUpdate(
{
_id,
username,
[`files.${file_id}`]: { $exists: true },
},
{
$set: {
[`files.${file_id}.file_name`]: file_name,
},
},
{ new: true, upsert: false, useFindAndModify: false }
);
const fn = genObjectId().toString();
// if file does not exist, then create a new file
const x = await DB_ACCOUNT.findOneAndUpdate(
{
_id,
username,
},
{
$set: {
[`files.${fn}`]: {
file_name,
creation_date: new Date(),
isPublished: false,
content: "",
},
},
},
{ new: true, upsert: true }
)
console.log("X: ", x);

Look up and create or update object inside array

I am currently trying to setup a schema for custom Discord guild commands:
const GuildCommandsSchema = new mongoose.Schema({
_id: String,
commands: [
{
name: {
type: String,
unique: true,
required: true,
},
action: {
type: String,
required: true,
},
author: {
type: String,
required: true,
},
},
],
});
Is this ok, performancewise, or could I improve it?
I feel like Mongo would need to look through all commands, since it can't index any commands inside 'commands' even though 'name' is unique.
If that's fine, how can I access the values inside commands?
I would need to find the right command via 'name' if it exists, otherwise create it and add/update 'action' + 'author'.
I tried something like this:
const updatedCommand = await GuildCommands.findOneAndUpdate(
{ _id },
{
$set: {
[`commands.$[outer].name`]: name,
[`commands.$[outer].action`]: action,
[`commands.$[outer].author`]: author,
},
},
{
arrayFilters: [{ 'outer.name': name }],
}
);
Unfortunately that does not create commands if they don't exist.
Thanks for your help
aggregate
db.collection.update({},
{
$set: {
"commands.$[c].name": "1",
"commands.$[c].author": "1",
"commands.$[c].action": "1"
}
},
{
arrayFilters: [
{
"c.author": "34"
}
],
multi: true
})
mongoplayground
To answer my own question:
I changed my Schema to use Maps instead of Arrays for performance improvments and also better model management.
const GuildCommandsSchema = new mongoose.Schema(
{
_id: String,
commands: {
type: Map,
of: {
_id: false,
name: {
type: String,
required: true,
},
action: {
type: String,
required: true,
},
active: {
type: Boolean,
required: true,
default: true,
},
author: {
type: String,
required: true,
},
},
},
},
{ versionKey: false }
);
The new query to find and update/create a command is also better imo:
const findCommand = await GuildCommands.findOne({ _id });
if (!action) {
const getCommand = findCommand.commands.get(name);
if (getCommand) {
message.reply(getCommand.action);
} else {
message.reply(`Cannot find ${name}`);
}
} else {
findCommand.commands.set(name, {
name,
action,
author,
});
findCommand.save();
}

How do I remove a specific item from model's array?

This is my user model with watched and watchLater arrays:
const UserSchema = new mongoose.Schema(
{
email: { type: String, required: true, unique: true },
watched: [{ type: Number }],
watchLater: [{ type: Number }],
},
{ timestamps: true },
)
I have this function where I want to remove id from watchLater and add to watched:
async addToWatched(id) {
const _id = this.getUserId()
return await this.store.User.findByIdAndUpdate(
{ _id },
// remove id from watchLater and add to watched,
{ new: true },
)
}
How do I do that?
Been reading docs a bit. This seems to work:
return await this.store.User.findByIdAndUpdate(
{ _id },
// remove id from watchLater and add to watched,
{ $pull: { watchLater: id }, $addToSet: { watched: id } },
{ new: true },
)

Eager Loading : How to disable specific fields of included table

I am trying to do Eager Loading in Sequelize with PostgreSQL where I need to find the Users which have a given specific Mail id or basically, i am performing find operation on Mail model while using include to include User model
UserModel :
module.exports = function (sequelize, Sequelize) {
var User = sequelize.define('User', {
userId: {
type: Sequelize.INTEGER,
primaryKey: true,
autoIncrement: true
},
firstname: {
type: Sequelize.STRING,
require: true
},
lastname: {
type: Sequelize.STRING,
require: true
},
age: {
type: Sequelize.INTEGER,
require: true
},
phone: {
type: Sequelize.STRING,
require: true
},
location: {
type: Sequelize.STRING,
require: true
},
createdAt: {
type: Sequelize.DATE,
defaultValue: Sequelize.NOW
},
updatedAt: {
type: Sequelize.DATE,
defaultValue: Sequelize.NOW
}
});
return User;
};
MailModel :
module.exports = function (sequelize, Sequelize) {
var User = require('./User.js')(sequelize, Sequelize)
var Mail = sequelize.define('Mail', {
name: {
type: Sequelize.STRING,
require: true
}
});
Mail.belongsTo(User);
return Mail;
};
MailController :
var db = require('../services/db.js');
module.exports = {
create: function (req, res, next) {
var Mailm = db.MailModel;
var name = req.body;
try {
db.sequelize.sync().then(function () {
Mailm.create(name).then(function (found) {
return res.json({
success: true,
message: found.get({
plain: true
})
});
})
});
} catch (ex) {
res.json({
success: false,
exception: ex
});
return;
}
},
query: function (req, res, next) {
var Mailm = db.MailModel;
var Userm = db.UserModel;
var name = req.body;
var option = {};
option.where = name;
option.include = [{
model: Userm
}];
try {
Mailm.findAll(option).then(function (found) {
console.log(found);
return res.json({
success: true,
message: found
});
});
} catch (ex) {
res.json({
success: false,
exception: ex
});
return;
}
}
};
It is returning me the records of both User and Mail table in exactly the right way .
Output :
{
"success": true,
"message":[
{
"id": 2,
"name": "Mailb2",
"createdAt": "2015-07-30T07:32:51.807Z",
"updatedAt": "2015-07-30T07:32:51.807Z",
"UserUserId": 2,
"User":{
"userId": 2,
"firstname": "Prerna",
"lastname": "Jain",
"age": 20,
"phone": "9812123456",
"location": "Sirsa",
"createdAt": "2015-07-30T07:30:48.000Z",
"updatedAt": "2015-07-30T07:30:48.000Z"
}
}
]
}
But I want to disable createdAt and updatedAt fields of User table so that it does not give me these two fields in the output for User.
I have tried a lot as of how to do this but still in vain.Can anyone please help me out.
I bet this is coming late, add attribute/properties to your models called timestamps, it accepts a boolean as a value. For example:
module.exports = function (sequelize, Sequelize) {
var User = require('./User.js')(sequelize, Sequelize)
var Mail = sequelize.define('Mail', {
name: {
type: Sequelize.STRING,
require: true
}
},
{
// This does the magic
timestamps: false,
});
Mail.belongsTo(User);
return Mail;
};
Also, add it to the User model:
var User = sequelize.define('User', {
userId: {
type: Sequelize.INTEGER,
primaryKey: true,
autoIncrement: true
},
firstname: {
type: Sequelize.STRING,
require: true
},
lastname: {
type: Sequelize.STRING,
require: true
},
age: {
type: Sequelize.INTEGER,
require: true
},
phone: {
type: Sequelize.STRING,
require: true
},
location: {
type: Sequelize.STRING,
require: true
},
createdAt: {
type: Sequelize.DATE,
defaultValue: Sequelize.NOW
},
updatedAt: {
type: Sequelize.DATE,
defaultValue: Sequelize.NOW
}
},
{
timestamps: false
});
return User;
};
You can use
Model.findAll({
attributes: { exclude: ['baz'] }
});
more examples with attributes - http://docs.sequelizejs.com/en/latest/docs/querying/#attributes