My goal is to add a comment to my CommentFeed and while doing that I want to push that comment into my topComments field and also update the 'numOfComments' . I want to limit the topComments to only 3 comments (How would I even set that up?). And how do I take the previous value of numOfComments and add one to it?
CommentFeed.findOneAndUpdate(
{ _id: commentId },
{
$push: {
comments: {
text: req.body.text
},
$push: topComments:{text: req.body.text}, <--- Limit this somehow to only allow an array length of 3?
$set: numOfComments: ? , <---What kind of logic is used here?
}
},
{ new: true }
)
CommentFeed Schema
const CommentFeedSchema = new Schema({
topComments:[{text:{type:String}}],
numOfComments:{type:Number},
comments: [
text: { type: String, required: true }
]});
For the first issue (limiting the topComments array size) you can use the $slice operator. This has already been answered in other questions. But you might consider computing topComments from comments using the$slice operator in the projection argument:
CommentFeed.find( {}, { comments: { $slice: -3 } } )
For the second issue (updating a document using existing fields from that document), it is not something you can do in a simple findOneAndUpdate call. This was also discussed in other questions.
But you might consider computing numOfComments instead of updating it every time. You can do that with the $size operator of the aggregation framework:
CommentFeed.aggregate({$project: { numOfComments: { $size:"$comments" }}})
In my Movie schema, I have a field "release_date" who can contain nested subdocuments.
These subdocuments contains three fields :
country_code
date
details
I need to guarantee the first two fields are unique (primary key).
I first tried to set a unique index. But I finally realized that MongoDB does not support unique indexes on subdocuments.
Index is created, but validation does not trigger, and I can still add duplicates.
Then, I tried to modify my update function to prevent duplicates, as explained in this article (see Workarounds) : http://joegornick.com/2012/10/25/mongodb-unique-indexes-on-single-embedded-documents/
$ne works well but in my case, I have a combination of two fields, and it's a way more complicated...
$addToSet is nice, but not exactly what I am searching for, because "details" field can be not unique.
I also tried plugin like mongoose-unique-validator, but it does not work with subdocuments ...
I finally ended up with two queries. One for searching existing subdocument, another to add a subdocument if the previous query returns no document.
insertReleaseDate: async(root, args) => {
const { movieId, fields } = args
// Searching for an existing primary key
const document = await Movie.find(
{
_id: movieId,
release_date: {
$elemMatch: {
country_code: fields.country_code,
date: fields.date
}
}
}
)
if (document.length > 0) {
throw new Error('Duplicate error')
}
// Updating the document
const response = await Movie.updateOne(
{ _id: movieId },
{ $push: { release_date: fields } }
)
return response
}
This code works fine, but I would have preferred to use only one query.
Any idea ? I don't understand why it's so complicated as it should be a common usage.
Thanks RichieK for your answer ! It's working great.
Just take care to put the field name before "$not" like this :
insertReleaseDate: async(root, args) => {
const { movieId, fields } = args
const response = await Movie.updateOne(
{
_id: movieId,
release_date: {
$not: {
$elemMatch: {
country_code: fields.country_code,
date: fields.date
}
}
}
},
{ $push: { release_date: fields } }
)
return formatResponse(response, movieId)
}
Thanks a lot !
Env:
MongoDB (3.2.0) with Mongoose
Collection:
users
Text Index creation:
BasicDBObject keys = new BasicDBObject();
keys.put("name","text");
BasicDBObject options = new BasicDBObject();
options.put("name", "userTextSearch");
options.put("unique", Boolean.FALSE);
options.put("background", Boolean.TRUE);
userCollection.createIndex(keys, options); // using MongoTemplate
Document:
{"name":"LEONEL"}
Queries:
db.users.find( { "$text" : { "$search" : "LEONEL" } } ) => FOUND
db.users.find( { "$text" : { "$search" : "leonel" } } ) => FOUND (search caseSensitive is false)
db.users.find( { "$text" : { "$search" : "LEONÉL" } } ) => FOUND (search with diacriticSensitive is false)
db.users.find( { "$text" : { "$search" : "LEONE" } } ) => FOUND (Partial search)
db.users.find( { "$text" : { "$search" : "LEO" } } ) => NOT FOUND (Partial search)
db.users.find( { "$text" : { "$search" : "L" } } ) => NOT FOUND (Partial search)
Any idea why I get 0 results using as query "LEO" or "L"?
Regex with Text Index Search is not allowed.
db.getCollection('users')
.find( { "$text" : { "$search" : "/LEO/i",
"$caseSensitive": false,
"$diacriticSensitive": false }} )
.count() // 0 results
db.getCollection('users')
.find( { "$text" : { "$search" : "LEO",
"$caseSensitive": false,
"$diacriticSensitive": false }} )
.count() // 0 results
MongoDB Documentation:
Text Search
$text
Text Indexes
Improve Text Indexes to support partial word match
As at MongoDB 3.4, the text search feature is designed to support case-insensitive searches on text content with language-specific rules for stopwords and stemming. Stemming rules for supported languages are based on standard algorithms which generally handle common verbs and nouns but are unaware of proper nouns.
There is no explicit support for partial or fuzzy matches, but terms that stem to a similar result may appear to be working as such. For example: "taste", "tastes", and tasteful" all stem to "tast". Try the Snowball Stemming Demo page to experiment with more words and stemming algorithms.
Your results that match are all variations on the same word "LEONEL", and vary only by case and diacritic. Unless "LEONEL" can be stemmed to something shorter by the rules of your selected language, these are the only type of variations that will match.
If you want to do efficient partial matches you'll need to take a different approach. For some helpful ideas see:
Efficient Techniques for Fuzzy and Partial matching in MongoDB by John Page
Efficient Partial Keyword Searches by James Tan
There is a relevant improvement request you can watch/upvote in the MongoDB issue tracker: SERVER-15090: Improve Text Indexes to support partial word match.
As Mongo currently does not supports partial search by default...
I created a simple static method.
import mongoose from 'mongoose'
const PostSchema = new mongoose.Schema({
title: { type: String, default: '', trim: true },
body: { type: String, default: '', trim: true },
});
PostSchema.index({ title: "text", body: "text",},
{ weights: { title: 5, body: 3, } })
PostSchema.statics = {
searchPartial: function(q, callback) {
return this.find({
$or: [
{ "title": new RegExp(q, "gi") },
{ "body": new RegExp(q, "gi") },
]
}, callback);
},
searchFull: function (q, callback) {
return this.find({
$text: { $search: q, $caseSensitive: false }
}, callback)
},
search: function(q, callback) {
this.searchFull(q, (err, data) => {
if (err) return callback(err, data);
if (!err && data.length) return callback(err, data);
if (!err && data.length === 0) return this.searchPartial(q, callback);
});
},
}
export default mongoose.models.Post || mongoose.model('Post', PostSchema)
How to use:
import Post from '../models/post'
Post.search('Firs', function(err, data) {
console.log(data);
})
Without creating index, we could simply use:
db.users.find({ name: /<full_or_partial_text>/i}) (case insensitive)
If you want to use all the benefits of MongoDB's full-text search AND want partial matches (maybe for auto-complete), the n-gram based approach mentioned by Shrikant Prabhu was the right solution for me. Obviously your mileage may vary, and this might not be practical when indexing huge documents.
In my case I mainly needed the partial matches to work for just the title field (and a few other short fields) of my documents.
I used an edge n-gram approach. What does that mean? In short, you turn a string like "Mississippi River" into a string like "Mis Miss Missi Missis Mississ Mississi Mississip Mississipp Mississippi Riv Rive River".
Inspired by this code by Liu Gen, I came up with this method:
function createEdgeNGrams(str) {
if (str && str.length > 3) {
const minGram = 3
const maxGram = str.length
return str.split(" ").reduce((ngrams, token) => {
if (token.length > minGram) {
for (let i = minGram; i <= maxGram && i <= token.length; ++i) {
ngrams = [...ngrams, token.substr(0, i)]
}
} else {
ngrams = [...ngrams, token]
}
return ngrams
}, []).join(" ")
}
return str
}
let res = createEdgeNGrams("Mississippi River")
console.log(res)
Now to make use of this in Mongo, I add a searchTitle field to my documents and set its value by converting the actual title field into edge n-grams with the above function. I also create a "text" index for the searchTitle field.
I then exclude the searchTitle field from my search results by using a projection:
db.collection('my-collection')
.find({ $text: { $search: mySearchTerm } }, { projection: { searchTitle: 0 } })
I wrapped #Ricardo Canelas' answer in a mongoose plugin here on npm
Two changes made:
- Uses promises
- Search on any field with type String
Here's the important source code:
// mongoose-partial-full-search
module.exports = exports = function addPartialFullSearch(schema, options) {
schema.statics = {
...schema.statics,
makePartialSearchQueries: function (q) {
if (!q) return {};
const $or = Object.entries(this.schema.paths).reduce((queries, [path, val]) => {
val.instance == "String" &&
queries.push({
[path]: new RegExp(q, "gi")
});
return queries;
}, []);
return { $or }
},
searchPartial: function (q, opts) {
return this.find(this.makePartialSearchQueries(q), opts);
},
searchFull: function (q, opts) {
return this.find({
$text: {
$search: q
}
}, opts);
},
search: function (q, opts) {
return this.searchFull(q, opts).then(data => {
return data.length ? data : this.searchPartial(q, opts);
});
}
}
}
exports.version = require('../package').version;
Usage
// PostSchema.js
import addPartialFullSearch from 'mongoose-partial-full-search';
PostSchema.plugin(addPartialFullSearch);
// some other file.js
import Post from '../wherever/models/post'
Post.search('Firs').then(data => console.log(data);)
If you are using a variable to store the string or value to be searched:
It will work with the Regex, as:
{ collection.find({ name of Mongodb field: new RegExp(variable_name, 'i') }
Here, the I is for the ignore-case option
The quick and dirty solution, that worked for me: use text search first, if nothing is found, then make another query with a regexp. In case you don't want to make two queries - $or works too, but requires all fields in query to be indexed.
Also, you'd better not to use case-insensitive rx, because it can't rely on indexes. In my case I've made lowercase copies of used fields.
Good n-gram based approach for fuzzy matching is explained here
(Also explains how to score higher for Results using prefix Matching)
https://medium.com/xeneta/fuzzy-search-with-mongodb-and-python-57103928ee5d
Note : n-gram based approaches can be storage extensive and mongodb collection size will increase.
I create an additional field which combines all the fields within a document that I want to search. Then I just use regex:
user = {
firstName: 'Bob',
lastName: 'Smith',
address: {
street: 'First Ave',
city: 'New York City',
}
notes: 'Bob knows Mary'
}
// add combined search field with '+' separator to preserve spaces
user.searchString = `${user.firstName}+${user.lastName}+${user.address.street}+${user.address.city}+${user.notes}`
db.users.find({searchString: {$regex: 'mar', $options: 'i'}})
// returns Bob because 'mar' matches his notes field
// TODO write a client-side function to highlight the matching fragments
full/partial search in MongodB for a "pure" Meteor-project
I adpated flash's code to use it with Meteor-Collections and simpleSchema but without mongoose (means: remove the use of .plugin()-method and schema.path (altough that looks to be a simpleSchema-attribute in flash's code, it did not resolve for me)) and returing the result array instead of a cursor.
Thought that this might help someone, so I share it.
export function partialFullTextSearch(meteorCollection, searchString) {
// builds an "or"-mongoDB-query for all fields with type "String" with a regEx as search parameter
const makePartialSearchQueries = () => {
if (!searchString) return {};
const $or = Object.entries(meteorCollection.simpleSchema().schema())
.reduce((queries, [name, def]) => {
def.type.definitions.some(t => t.type === String) &&
queries.push({[name]: new RegExp(searchString, "gi")});
return queries
}, []);
return {$or}
};
// returns a promise with result as array
const searchPartial = () => meteorCollection.rawCollection()
.find(makePartialSearchQueries(searchString)).toArray();
// returns a promise with result as array
const searchFull = () => meteorCollection.rawCollection()
.find({$text: {$search: searchString}}).toArray();
return searchFull().then(result => {
if (result.length === 0) throw null
else return result
}).catch(() => searchPartial());
}
This returns a Promise, so call it like this (i.e. as a return of a async Meteor-Method searchContact on serverside).
It implies that you attached a simpleSchema to your collection before calling this method.
return partialFullTextSearch(Contacts, searchString).then(result => result);
import re
db.collection.find({"$or": [{"your field name": re.compile(text, re.IGNORECASE)},{"your field name": re.compile(text, re.IGNORECASE)}]})
At Sails.js (which is using Waterline ORM), how can I query a model that return records only when the criteria is right for the associations. Followings are the codes:
Order.find()
.populate('books', {title: {startsWith: 'Star Trek'}})
.exec(function (err, foundOrders) {
....
});
The models are as below:
Order:
module.exports = {
attributes: {
...
books: {
collection: book,
via: order
through: orderbook
}
}
Book:
module.exports = {
attributes: {
....
order: {
collection: order,
via: book
through: orderbook
}
}
Orderbook:
module.exports = {
attributes: {
order: {
model: order
},
book: {
model: book
}
}
}
I found that the resulting recordset is all orders. Only that the populated result of each order contains only books which title starts with 'Star Trek'.
That's not what I want. I want to return orders that only have books which title starts with 'Star Trek'.
Please anyone can suggest how to make right to my query statement?
When you will check: http://sailsjs.com/documentation/reference/waterline-orm/queries/populate , you will find:
Something.find()
.populate(association, subcriteria)
.exec(function afterwards(err, populatedRecords){
});
So you can use it e.g. like in docs:
User.find({
name:'Finn'
}).populate('currentSwords', {
where: {
color: 'purple'
},
limit: 3,
sort: 'hipness DESC'
}).exec(function (err, usersNamedFinn){
Nice and easy. Good luck!
After googling for this question several days, I still couldn't find a good solution. But I have solved the question in a less perfect way, but at least it works.
Book.find({title: {startsWith: 'Star Tek'}}).populate('orders')
.exec(function (err, foundBooks) {
if (err) return res.negotiate(err);
var orderIds = [];
for (var i=0; i<foundBooks.length; i++) {
for (var j=0; j<foundBooks[i].orders.length; j++) {
orderIds.push(foundBooks[i].orders[j].id;
}
}
Order.find({id: orderIds}).populate('books').populate( .. some other associations .. )
.exec(function (err, foundOrders) {
...
});
});
Hope that somebody would suggest a more elegant way.
I ran into a similar problem trying to return users with a given role. I did a find() on my User model and populated the roles association with the criteria for a "student" role. I had the same results: all of my users were returned, but some just had an empty roles association. I solved it with an array filter.
let students = await User.find({})
.populate('school')
.populate('roles', { where: { name: 'student' }});
sails.log("Full result set:", students);
filteredStudents = students.filter( element => {
sails.log(element.roles);
return element.roles.length != 0;
});
console.log("Filtered result set:", filteredStudents);
The key piece is checking the length of the association array to see if it's non-empty. For this particular question (which I realize is now over four years old) here is what I would suggest (starting after the ....):
Order.find()
.populate('books', {title: {startsWith: 'Star Trek'}})
.exec(function (err, foundOrders) {
....
var filteredOrders = foundOrders.filter( element => element.books.length != 0 );
// move on with your day using filteredOrders
});
I'm trying to update a sub document on an existing collection. I'm getting a MongoDB error message.
"MongoError: The positional operator did not find the match needed from the query. Unexpanded update: articleWords.$ [409]"
From my Articles Simple Schema
"articleWords.$": {
type: Object
},
"articleWords.$.wordId": {
type: String,
label: 'Word ID'
},
"articleWords.$.word": {
type: String,
label: 'Word'
},
Update Function
function updateArticle(_id,wordArr) {
_.each(wordArr,function(elem) {
var ret = Articles.update(
{'_id': _id},
{ $set: { 'articleWords.$': { 'wordId': elem.wordId, 'word': elem.word } }
});
});
return true;
}
As you can see I am passing an array of objects. Is there a better way to do this than _.each ?
CLARIFICATION
Thank you to #corvid for the answer. I think I didn't make my question clear enough. There does exist an article record, but there is no data added to the articleWords attribute. Essentially we are updating a record but insert into the articleWords array.
A second attempt, is also not working
_.each(wordArr,function(elem) {
var ret = Articles.update(
{'_id': _id},
{ $set: { 'articleWords.$.wordId': elem.wordId, 'articleWords.$.word': elem.word } }
);
});
Yes, you need your selector to match something within the subdocument. For example,
Articles.update({
'_id': <someid>,
'words.wordId': <somewordid>
}, {
$set: {
'words.$.word': elem.word,
'words.$.wordId': elem.wordId
}
});
If the array doesn't exist yet then you're going about this in the hardest way possible. You can just set the entire array at one go:
var ret = Articles.update(
{'_id': _id},
{ $set: { articleWords: wordArr }}
);
I can see that wordArr already has the id and string. This will work as long as it doesn't have more content. If it does then you can just make a second version with the parts you want to keep.