Im trying to get a Full Text Search working through Fastify and Mongoose. But it returns an empty array.
There is no info online about this, so I just played with it for a while.
in db.ts I set the:
useCreateIndex: true
The Model (Benefit.ts):
export class Benefit extends BaseModel {
_id: ObjectID;
#prop({required: true, unique: true, index: true, text: true}) <-- set index & text to true
name: string;
#prop({index: true, text: true})
details: string;
const BenefitModel = getModelForClass(Benefit);
BenefitModel.createIndexes(err => console.log('ERROR', err)); <-- this is what I tried // err=null
export default BenefitModel;
The actual search (Benefit.controller.ts):
const searchResults = await benefitService.find(
{$text: {$search: textSearch}}, <-- textSearch is valid and comes as a string
{score: {$meta: 'textScore'}},
);
the return is with status 200:
{
"status": 1,
"results": []
}
I think I dont set the index correctly?
Any help will be greatly appriciated.
And also will live long, because there is no info about it anywhere.
Thanks.
EDIT:
current route setup
export class BenefitController {
protected routes: fastify.RouteOptions[];
constructor() {
this.routes = [];
this.setupRoutes();
}
public getRoutes(): fastify.RouteOptions[] {
return this.routes;
}
protected setupRoutes(): void {
this.routes.push({
method: 'POST',
url: '/searchBenefit',
handler: this.searchBenefit,
});
}
new BenefitController().getRoutes().forEach((route: fastify.RouteOptions) => {
fastify.route(route);
});
BTW I did set the indexes properly cause when I do listIndexes()
I get:
{ v: 2, key: { _id: 1 }, name: '_id_', ns: 'some-address.db.benefits' },
{
v: 2,
unique: true,
key: { name: 1 },
name: 'name_1',
ns: 'some-address.db.benefits',
background: true
},
{
v: 2,
key: { details: 1 },
name: 'details_1',
ns: 'some-address.db.benefits',
background: true
},
{
v: 2,
key: { _fts: 'text', _ftsx: 1 },
name: 'details_text',
ns: 'some-address.db.benefits',
background: true,
weights: { details: 1 },
default_language: 'english',
language_override: 'language',
textIndexVersion: 3
}
Related
I am using findOneAndUpdate, where I want
to return updated document
i dont want to return the entire document but only the following:
one object out of an array + a virtual property in the document.
const notifications = {
to:
messages: [
{_id: "23452", title:"hello"}, {_id: "23452", title:"bye"}
]
...
}
so for example I would want to only return the object {_id: "23452", title:"bye"} AND unreadCount virtual field prop.
my code works so far as I am returning updated document and only the message I want, but I dont know how to return also the unreadCount prop.
schema:
const notificationSchema = new mongoose.Schema({
to: {
type: mongoose.Schema.Types.ObjectId,
ref: 'User'
},
messages: [{
title: {
type: String,
required: true
},
isRead: {
type: Boolean,
default: false
},
createdAt: {
type: Date,
default: new Date()
}
}, ]
},
{timestamps: true, toObject: {virtuals: true}
});
notificationSchema.virtual('unreadCount').get(function() {
... return count;...
})
updateRead: async (userId, id) => {
const notification = await Notification.findOneAndUpdate({to: userId, 'messages._id': id}, {
$set: { "messages.$.isRead": true} },
{ select: {
messages: {
$elemMatch: {_id: id}
}
}, new: true});
}
I am currently trying to setup a schema for custom Discord guild commands:
const GuildCommandsSchema = new mongoose.Schema({
_id: String,
commands: [
{
name: {
type: String,
unique: true,
required: true,
},
action: {
type: String,
required: true,
},
author: {
type: String,
required: true,
},
},
],
});
Is this ok, performancewise, or could I improve it?
I feel like Mongo would need to look through all commands, since it can't index any commands inside 'commands' even though 'name' is unique.
If that's fine, how can I access the values inside commands?
I would need to find the right command via 'name' if it exists, otherwise create it and add/update 'action' + 'author'.
I tried something like this:
const updatedCommand = await GuildCommands.findOneAndUpdate(
{ _id },
{
$set: {
[`commands.$[outer].name`]: name,
[`commands.$[outer].action`]: action,
[`commands.$[outer].author`]: author,
},
},
{
arrayFilters: [{ 'outer.name': name }],
}
);
Unfortunately that does not create commands if they don't exist.
Thanks for your help
aggregate
db.collection.update({},
{
$set: {
"commands.$[c].name": "1",
"commands.$[c].author": "1",
"commands.$[c].action": "1"
}
},
{
arrayFilters: [
{
"c.author": "34"
}
],
multi: true
})
mongoplayground
To answer my own question:
I changed my Schema to use Maps instead of Arrays for performance improvments and also better model management.
const GuildCommandsSchema = new mongoose.Schema(
{
_id: String,
commands: {
type: Map,
of: {
_id: false,
name: {
type: String,
required: true,
},
action: {
type: String,
required: true,
},
active: {
type: Boolean,
required: true,
default: true,
},
author: {
type: String,
required: true,
},
},
},
},
{ versionKey: false }
);
The new query to find and update/create a command is also better imo:
const findCommand = await GuildCommands.findOne({ _id });
if (!action) {
const getCommand = findCommand.commands.get(name);
if (getCommand) {
message.reply(getCommand.action);
} else {
message.reply(`Cannot find ${name}`);
}
} else {
findCommand.commands.set(name, {
name,
action,
author,
});
findCommand.save();
}
I try to find refs I've stored in my db by sending an array of object to my API.
In the array I send there are 4 objects. 2 of them are already stored, 2 of them are new.
Here is what is stored in my DB:
[
{
ref: 'xxx14-010-S',
colorWay: 'Black',
},
{
ref: 'xxx18-050-S',
colorWay: 'Black',
},
...
]
My req.body look like this :
[
{
ref: 'xxx14-010-S',
colorWay: 'Black',
},
{
ref: 'xxx18-050-S',
colorWay: 'Black',
},
{
ref: 'xxx20-010-S',
colorWay: 'Black',
},
{
ref: 'xx324-010-S',
colorWay: 'Black',
}
]
And here is the code in my express router:
router.post("/api/v1/lineList", async (req, res, next) => {
try {
const existingRefs = await LineList.find({ref: {$in: req.body.ref}});
res.send(existingRefs);
} catch (err) {
next(err);
}
});
This returns an empty array where I expect to find the 2 stored objects.
How shall I proceed ?
Is there a way to get the 2 found objects and also be notified that the 2 other object where not in the db ?
Thanks a lot !
EDIT:
Here is the model:
const lineListSchema = new mongoose.Schema({
ref: {
type: String,
required: true,
unique: true,
trim: true,
uppercase: true,
},
styleName: {
type: String,
required: true,
trim: true,
uppercase: true,
},
colorWay: {
type: String,
required: true,
trim: true,
uppercase: true,
},
...
createdAt: {
type: Date,
default: Date.now,
expires: 3600,
},
});
$in syntax is:
{ field: { $in: [<value1>, <value2>, ... <valueN> ] } }
So you need to pass an array like this:
await LineList.find({ ref: { $in: req.body.map(val => val.ref) } });
I want to push a Date object from my client into the nested array 'completed_dates' but I cannot figure out how to do so, or if I would need to change my schema in order for it to work.
{
_id: 606f1d67aa1d5734c494bf0a,
name: 'Courtney',
email: 'c#gmail.com',
password: '$2b$10$WQ22pIiwD8yDvRhdQ0olBe6JnnFqV2WOsC0cD/FkV4g7LPtUOpx1C',
__v: 35,
habits: [
{
_id: 6081d32580bfac579446eb81,
completed_dates: [],
name: 'first',
type: 'good',
days: 0,
checked: false
},
{
_id: 6081d32f80bfac579446eb82,
completed_dates: [],
name: 'seconds',
type: 'bad',
days: 0,
checked: false
},
]
}
and this is my schema
const habitSchema = new mongoose.Schema({
name: String,
category: String,
color: {
type: String,
},
date_added: {
type: String,
},
completed_dates: {
type: Array,
}
})
const userSchema = new mongoose.Schema({
name: {
type: String,
required: true,
min: 6,
max: 255,
},
email: {
type: String,
required: true,
max: 255
},
password: {
type: String,
required: true,
max: 1024,
min: 8,
},
habits: [habitSchema]
})
Here is what I have tried...
I've tried using findOneAndUpdate, using the document id of the logged in user, and trying to manipulate the update object to drill into the nested array. I can access the habits list of the correct user... using this code, but for this new problem, I want to go one level further and push to the 'completed_dates' array of a specific habit (based on name or _id).
//this only adds a habit object to the habits array.
User.findByIdAndUpdate(req.user._id,
{ $pull: { habits: { _id: itemsToDelete } } },
{ new: true , useFindAndModify: false},
function (err, data) {
if (err) {
res.send(err)
} else {
res.send(data.habits)
}
}
)
I have tried building on this existing code by trying to filter down one more level. (this doesn't work.)
const { date, name} = req.body.update
User.findByIdAndUpdate(req.user._id,
{ $push: { 'habits.$[req.body.name].completed_dates': req.body.date} },
{safe: true, upsert: true, new : true, useFindAndModify: false},
function (err, data) {
if (err) {
res.send(err)
} else {
//data.update
res.send(data.habits)
}
}
)
If anyone can link or help me out, I would appreciate it. Thanks
I have a collection I want to upsert with findOneAndUpdate. In addition to that I have two fields (isHandled, isNotADuplicate) that should be:
defaulted to 'false' upon insert
left untouched upon update (e.g. isHandled stays 'true')
I have however found that
isHandled, isNotADuplicate are always defaulted back to 'false'
_id is also regenerated upon every update (I use a compound key to query the doc, not _id)
My Model
export const QuickbrainFindingSchema = new Schema<QuickBrainFindingDocument>({
connectedApplicationType: { type: String, required: true, enum: ['jira'] },//e.g. jira
clientKey: { type: String, required: true },//e.g. 135eb702-156c-3b67-b9d0-a0c97548xxxx
//key
projectKey: { type: String, required: true },//e.g. AL
type: { type: String, required: true },
doc1key: { type: String, required: true },//e.g. AL-7
doc2key: { type: String, required: true },//e.g. AL-16
//data
calculationDate: { type: SchemaTypes.Date, default: Date.now },
direction: { type: String, required: true },
reasonAndMetric: { type: SchemaTypes.Mixed, reason: true },
scoreSummary: { type: String, reason: true },
isHandled: { type: SchemaTypes.Boolean, default: false },
isNotADuplicate: { type: SchemaTypes.Boolean, default: false },
similarityReference: { type: SchemaTypes.ObjectId, required: true, ref: "QuickbrainSimilarityMatrix" }
}, {
//options
});
QuickbrainFindingSchema.index(
{ connectedApplicationType: 1, clientKey: 1, project: 1, doc1key: 1, doc2key: 1, type: 1 },
{ unique: true, name: "compoundKey" }
);
export const QuickbrainFindingModel = model<QuickBrainFindingDocument>("QuickbrainFinding", QuickbrainFindingSchema);
My Code
public async addFinding(
projectKey: string,
doc1key: string,
doc2key: string,
type: ET_FindingType
, data: QuickbrainFindingData): Promise<QuickbrainFinding> {
let keyFull: QuickbrainFindingKey = {
connectedApplicationType: this.connectedApplicationType,
clientKey: this.clientKey,
projectKey: projectKey,
doc1key: doc1key,
doc2key: doc2key,
type: type
};
let insertObj: QuickbrainFinding = <QuickbrainFinding><unknown>{};
Object.assign(insert, keyFull);
Object.assign(insert, data);
delete (<any>insertObj).isHandled;
delete (<any>insertObj).isNotADuplicate;
return new Promise<QuickbrainFinding>(function (ok, nok) {
QuickbrainFindingModel.findOneAndUpdate(
keyFull, { $set: insertObj},
{
runValidators: true,
upsert: true,
setDefaultsOnInsert: true,
new: true,
omitUndefined: true,//I think only available for findAndReplace(..)
})
.lean().exec(function (err, result) {
if (err) {
nok(err);
}
else
ok(result)
});
});
}
Mongoose Debug Output
quickbrainfindings.findOneAndUpdate(
{
connectedApplicationType: 'jira',
clientKey: '135eb702-256c-3b67-b9d0-a0c975487af3',
projectKey: 'ITSMTEST',
doc1key: 'ITSMTEST-7',
doc2key: 'ITSMTEST-10',
type: 'Email'
},
{
'$setOnInsert':
{ __v: 0, isHandled: false, isNotADuplicate: false, _id: ObjectId("60789b02c094eb3ef07d2929") },
'$set': {
connectedApplicationType: 'jira',
clientKey: '135eb702-256c-3b67-b9d0-a0c975487af3', projectKey: 'ITSMTEST', doc1key: 'ITSMTEST-7', doc2key: 'ITSMTEST-10', type: 'Email',
calculationDate: new Date("Thu, 15 Apr 2021 19:58:58 GMT"),
direction: '2', scoreSummary: '100.0%',
similarityReference: ObjectId("60789b029df2079dfa8aa15a"),
reasonAndMetric: [{ reason: 'Title Substring', metricScore: '100%' },
{ reason: 'Title TokenSet', metricScore: '54%' }, { reason: 'Description TokenSet', metricScore: '100%' }]
}
},
{
runValidators: true, upsert: true, remove: false, projection: {},
returnOriginal: false
}
)
What happens
Existing documents are found, but when they are updated I'm confused that:
_id is regenerated
isHandled and isNotADuplicate are reset to 'false' (although insertObj does not contain them)
When looking at the debug output I can see that the new _id is the one fron $setOnInsert, which confuses the heck out of me, since the selector works
Notable
keyFull is used to query the existing document, it does not contain _id;
delete (<any>insertObj).isHandled <- the object used for $set does NOT contain isHandled
This is embarrasing to admit, but thanks to Joe I have found the problem.
Before every findOneAndUpdate / Upsert I had a delete statement removing the existing documents Pipeline:
Delete old documents
Calculate new documents
Upsert new documents -> always resulted in Insert
let matchAnyDoc = this.filterForDocKeyAny(projectKey, docKeyAny, findingType);
matchAnyDoc.forEach(async (condition) => {
QuickbrainFindingModel.deleteMany(condition).exec(function (err, res) {
if (err) {
nok(err);
} else {
ok();
}
});
}, this);