case-insensitive query on mongodb - mongodb

is there a way to query for a case-insensitive value on mongo without using map/reduce?

Suppose you have document that contains tag field and you want search on it
Tags
{
tag,
...
}
First option is use regex(but it work slow as #RestRisiko said):
db.tags.find( { "tag" : { "$regex" : "C#", "$options" : "-i" } })
Second option is create another, lower case field( and in mongodb it best way):
Tags
{
tag,
tagLower,
..
}
And use find as usual:
db.tags.find( { "tagLower" : "c#"})
It will work faster, because above code can use index for search.

You have to normalize the data to be queried. Using a regular expression for case-insensitive search might work as well it won't use indexes. So your only option is to normalize. If you need to preserve the original state then you need to denormalize the data and store the normalized values in a dedicated column of the document.

When using it with Node.js, it's best to build a RegEx object in the query.
Room.findOne({'name': new RegExp(roomName, 'i')}, {}, function(err, room) {
...

Use regular expressions matching as below. The 'i' shows case insensitivity.
var collections = mongoDatabase.GetCollection("Abcd");
var queryA = Query.And(
Query.Matches("strName", new BsonRegularExpression("MSID", "i")),
Query.Matches("strVal", new BsonRegularExpression("154800", "i")));
var queryB = Query.And(
Query.Matches("strName", new BsonRegularExpression("Operation","i")),
Query.Matches("strVal", new BsonRegularExpression("8221", "i")));
var getA = collections.Find(queryA);
var getB = collections.Find(queryB);

If there are some special characters in the query, regex simple will not work. You will need to escape those special characters.
The following helper function can help without installing any third-party library:
const escapeSpecialChars = (str) => {
return str.replace(/[-[\]{}()*+?.,\\^$|#\s]/g, "\\$&");
}
And your query will be like this:
db.collection.find({ field: { $regex: escapeSpecialChars(query), $options: "i" }})
OR
db.collection.find({ field: { $regex: `.*${escapeSpecialChars(query)}.*`, $options: "i" }})
Hope it will help!

With MongoDB v3.6+, you can chain up $toLower/$toUpper with $expr to perform case-insensitive search.
db.collection.find({
$expr: {
$eq: [
{
$toLower: "$value"
},
{
$toLower: "apple"// your search string here
}
]
}
})
Here is the Mongo Playground for your reference.

Related

How can I find average age of users searched by word in field MongoDB? [duplicate]

I want to query something with SQL's like query:
SELECT * FROM users WHERE name LIKE '%m%'
How can I achieve the same in MongoDB? I can't find an operator for like in the documentation.
That would have to be:
db.users.find({"name": /.*m.*/})
Or, similar:
db.users.find({"name": /m/})
You're looking for something that contains "m" somewhere (SQL's '%' operator is equivalent to regular expressions' '.*'), not something that has "m" anchored to the beginning of the string.
Note: MongoDB uses regular expressions which are more powerful than "LIKE" in SQL. With regular expressions you can create any pattern that you imagine.
For more information on regular expressions, refer to Regular expressions (MDN).
db.users.insert({name: 'patrick'})
db.users.insert({name: 'petra'})
db.users.insert({name: 'pedro'})
Therefore:
For:
db.users.find({name: /a/}) // Like '%a%'
Output: patrick, petra
For:
db.users.find({name: /^pa/}) // Like 'pa%'
Output: patrick
For:
db.users.find({name: /ro$/}) // Like '%ro'
Output: pedro
In
PyMongo using Python
Mongoose using Node.js
Jongo, using Java
mgo, using Go
you can do:
db.users.find({'name': {'$regex': 'sometext'}})
In PHP, you could use the following code:
$collection->find(array('name'=> array('$regex' => 'm'));
Here are different types of requirements and solutions for string search with regular expressions.
You can do with a regular expression which contains a word, i.e., like. Also you can use $options => i for a case insensitive search.
Contains string
db.collection.find({name:{'$regex' : 'string', '$options' : 'i'}})
Doesn't contain string, only with a regular expression
db.collection.find({name:{'$regex' : '^((?!string).)*$', '$options' : 'i'}})
Exact case insensitive string
db.collection.find({name:{'$regex' : '^string$', '$options' : 'i'}})
Start with string
db.collection.find({name:{'$regex' : '^string', '$options' : 'i'}})
End with string
db.collection.find({name:{'$regex' : 'string$', '$options' : 'i'}})
Keep Regular Expressions Cheat Sheet as a bookmark, and a reference for any other alterations you may need.
You would use a regular expression for that in MongoDB.
For example,
db.users.find({"name": /^m/})
You have two choices:
db.users.find({"name": /string/})
or
db.users.find({"name": {"$regex": "string", "$options": "i"}})
For the second one, you have more options, like "i" in options to find using case insensitive.
And about the "string", you can use like ".string." (%string%), or "string.*" (string%) and ".*string) (%string) for example. You can use a regular expression as you want.
If using Node.js, it says that you can write this:
db.collection.find( { field: /acme.*corp/i } );
// Or
db.collection.find( { field: { $regex: 'acme.*corp', $options: 'i' } } );
Also, you can write this:
db.collection.find( { field: new RegExp('acme.*corp', 'i') } );
Already you got the answers, but to match with a regular expression with case insensitivity, you could use the following query:
db.users.find ({ "name" : /m/i } ).pretty()
The i in the /m/i indicates case insensitivity and .pretty() provides a prettier output.
For Mongoose in Node.js:
db.users.find({'name': {'$regex': '.*sometext.*'}})
With MongoDB Compass, you need to use the strict mode syntax, as such:
{ "text": { "$regex": "^Foo.*", "$options": "i" } }
(In MongoDB Compass, it's important that you use " instead of ')
You can use the new feature of MongoDB 2.6:
db.foo.insert({desc: "This is a string with text"});
db.foo.insert({desc:"This is a another string with Text"});
db.foo.ensureIndex({"desc":"text"});
db.foo.find({
$text:{
$search:"text"
}
});
In a Node.js project and using Mongoose, use a like query:
var User = mongoose.model('User');
var searchQuery = {};
searchQuery.email = req.query.email;
searchQuery.name = {$regex: req.query.name, $options: 'i'};
User.find(searchQuery, function(error, user) {
if(error || user === null) {
return res.status(500).send(error);
}
return res.status(200).send(user);
});
You can use a where statement to build any JavaScript script:
db.myCollection.find( { $where: "this.name.toLowerCase().indexOf('m') >= 0" } );
Reference: $where
In MongoDb, can use like using MongoDb reference operator regular expression(regex).
For Same Ex.
MySQL - SELECT * FROM users WHERE name LIKE '%m%'
MongoDb
1) db.users.find({ "name": { "$regex": "m", "$options": "i" } })
2) db.users.find({ "name": { $regex: new RegExp("m", 'i') } })
3) db.users.find({ "name": { $regex:/m/i } })
4) db.users.find({ "name": /mail/ })
5) db.users.find({ "name": /.*m.*/ })
MySQL - SELECT * FROM users WHERE name LIKE 'm%'
MongoDb Any of Above with /^String/
6) db.users.find({ "name": /^m/ })
MySQL - SELECT * FROM users WHERE name LIKE '%m'
MongoDb Any of Above with /String$/
7) db.users.find({ "name": /m$/ })
In Go and the mgo driver:
Collection.Find(bson.M{"name": bson.RegEx{"m", ""}}).All(&result)
where the result is the struct instance of the sought-after type.
In SQL, the ‘like’ query looks like this:
select * from users where name like '%m%'
In the MongoDB console, it looks like this:
db.users.find({"name": /m/}) // Not JSON formatted
db.users.find({"name": /m/}).pretty() // JSON formatted
In addition, the pretty() method will produce a formatted JSON structure in all the places which is more readable.
For PHP mongo Like.
I had several issues with PHP mongo like. I found that concatenating the regular expression parameters helps in some situations - PHP mongo find field starts with.
For example,
db()->users->insert(['name' => 'john']);
db()->users->insert(['name' => 'joe']);
db()->users->insert(['name' => 'jason']);
// starts with
$like_var = 'jo';
$prefix = '/^';
$suffix = '/';
$name = $prefix . $like_var . $suffix;
db()->users->find(['name' => array('$regex'=>new MongoRegex($name))]);
output: (joe, john)
// contains
$like_var = 'j';
$prefix = '/';
$suffix = '/';
$name = $prefix . $like_var . $suffix;
db()->users->find(['name' => array('$regex'=>new MongoRegex($name))]);
output: (joe, john, jason)
String yourdb={deepakparmar, dipak, parmar}
db.getCollection('yourdb').find({"name":/^dee/})
ans deepakparmar
db.getCollection('yourdb').find({"name":/d/})
ans deepakparmar, dipak
db.getCollection('yourdb').find({"name":/mar$/})
ans deepakparmar, parmar
Using template literals with variables also works:
{"firstname": {$regex : `^${req.body.firstname}.*` , $options: 'si' }}
Regular expressions are expensive to process.
Another way is to create an index of text and then search it using $search.
Create a text index of fields you want to make searchable:
db.collection.createIndex({name: 'text', otherField: 'text'});
Search for a string in the text index:
db.collection.find({
'$text'=>{'$search': "The string"}
})
Use regular expressions matching as below. The 'i' shows case insensitivity.
var collections = mongoDatabase.GetCollection("Abcd");
var queryA = Query.And(
Query.Matches("strName", new BsonRegularExpression("ABCD", "i")),
Query.Matches("strVal", new BsonRegularExpression("4121", "i")));
var queryB = Query.Or(
Query.Matches("strName", new BsonRegularExpression("ABCD","i")),
Query.Matches("strVal", new BsonRegularExpression("33156", "i")));
var getA = collections.Find(queryA);
var getB = collections.Find(queryB);
It seems that there are reasons for using both the JavaScript /regex_pattern/ pattern as well as the MongoDB {'$regex': 'regex_pattern'} pattern. See: MongoDB RegEx Syntax Restrictions
This is not a complete regular expression tutorial, but I was inspired to run these tests after seeing a highly voted ambiguous post above.
> ['abbbb','bbabb','bbbba'].forEach(function(v){db.test_collection.insert({val: v})})
> db.test_collection.find({val: /a/})
{ "val" : "abbbb" }
{ "val" : "bbabb" }
{ "val" : "bbbba" }
> db.test_collection.find({val: /.*a.*/})
{ "val" : "abbbb" }
{ "val" : "bbabb" }
{ "val" : "bbbba" }
> db.test_collection.find({val: /.+a.+/})
{ "val" : "bbabb" }
> db.test_collection.find({val: /^a/})
{ "val" : "abbbb" }
> db.test_collection.find({val: /a$/})
{ "val" : "bbbba" }
> db.test_collection.find({val: {'$regex': 'a$'}})
{ "val" : "bbbba" }
A like query would be as shown below:
db.movies.find({title: /.*Twelve Monkeys.*/}).sort({regularizedCorRelation : 1}).limit(10);
For the Scala ReactiveMongo API,
val query = BSONDocument("title" -> BSONRegex(".*" + name + ".*", "")) // like
val sortQ = BSONDocument("regularizedCorRelation" -> BSONInteger(1))
val cursor = collection.find(query).sort(sortQ).options(QueryOpts().batchSize(10)).cursor[BSONDocument]
If you are using Spring-Data MongoDB, you can do it in this way:
String tagName = "m";
Query query = new Query();
query.limit(10);
query.addCriteria(Criteria.where("tagName").regex(tagName));
If you have a string variable, you must convert it to a regex, so MongoDB will use a like statement on it.
const name = req.query.title; //John
db.users.find({ "name": new Regex(name) });
Is the same result as:
db.users.find({"name": /John/})
Use aggregation substring search (with index!!!):
db.collection.aggregate([{
$project : {
fieldExists : {
$indexOfBytes : ['$field', 'string']
}
}
}, {
$match : {
fieldExists : {
$gt : -1
}
}
}, {
$limit : 5
}
]);
You can query with a regular expression:
db.users.find({"name": /m/});
If the string is coming from the user, maybe you want to escape the string before using it. This will prevent literal chars from the user to be interpreted as regex tokens.
For example, searching the string "A." will also match "AB" if not escaped.
You can use a simple replace to escape your string before using it. I made it a function for reusing:
function textLike(str) {
var escaped = str.replace(/[\-\[\]\/\{\}\(\)\*\+\?\.\\\^\$\|]/g, '\\$&');
return new RegExp(escaped, 'i');
}
So now, the string becomes a case-insensitive pattern matching also the literal dot. Example:
> textLike('A.');
< /A\./i
Now we are ready to generate the regular expression on the go:
db.users.find({ "name": textLike("m") });
If you want a 'like' search in MongoDB then you should go with $regex. By using it, the query will be:
db.product.find({name:{$regex:/m/i}})
For more, you can read the documentation as well - $regex
One way to find the result as with equivalent to a like query:
db.collection.find({name:{'$regex' : 'string', '$options' : 'i'}})
Where i is used for a case-insensitive fetch data.
Another way by which we can also get the result:
db.collection.find({"name":/aus/})
The above will provide the result which has the aus in the name containing aus.

Mongo $in query with case-insensitivity

I'm using Mongoose.js to perform an $in query, like so:
userModel.find({
'twitter_username': {
$in: friends
}
})
friends is just an array of strings. However, I'm having some case issues, and wondering if I can use Mongo's $regex functionality to make this $in query case-insensitive?
From the docs:
To include a regular expression in an $in query expression, you can
only use JavaScript regular expression objects (i.e. /pattern/ ). For
example:
{ name: { $in: [ /^acme/i, /^ack/ ] } }
One way is to create regular Expression for each Match and form the friends array.
var friends = [/^name1$/i,/^name2$/i];
or,
var friends = [/^(name1|name2)$/i]
userModel.find({"twitter_username":{$in:friends }})
Its a little tricky to do something like that
at first you should convert friends to new regex array list with:
var insesitiveFriends = [];
friends.forEach(function(item)
{
var re = new RegExp(item, "i");
insesitiveFriends.push(re);
})
then run the query
db.test.find(
{
'twitter_username':
{
$in: insesitiveFriends
}
})
I have the sample documents in test collection
/* 0 */
{
"_id" : ObjectId("5485e2111bb8a63952bc933d"),
"twitter_username" : "David"
}
/* 1 */
{
"_id" : ObjectId("5485e2111bb8a63952bc933e"),
"twitter_username" : "david"
}
and with var friends = ['DAvid','bob']; I got both documents
Sadly, mongodb tends to be pretty case-sensitive at the core. You have a couple of options:
1) Create a separate field that is a lowercase'd version of the twitter_username, and index that one instead. Your object would have twitter_username and twitter_username_lc. The non-lowerecase one you can use for display etc, but the lowercase one you index and use in your where clause etc.
This is the route I chose to go for my application.
2) Create a really ugly regex from your string of usernames in a loop prior to your find, then pass it in:
db.users.find({handle:{$regex: /^benhowdle89|^will shaver|^superman/i } })
Note that using the 'starts with' ^ carrot performs better if the field is indexed.

MongoDB: Search in array

I have a field in a MongoDB Collection like:
{
place = ['London','Paris','New York']
}
I need a query that will return only that particular entry of the array, where a specific character occurs. For Example, I want to search for the terms having the letter 'o'(case-insensitive) in them. It should just return 'London' and 'New York'.
I tried db.cities.find({"place":/o/i}), but it returns the whole array.
You'll need to $unwind using an aggregate query, then match.
db.cities.aggregate([ { $unwind:'$place' }, { $match: { place : {$regex: /o/i } } } ])
In simple you find with $regex below query will work without aggregation
db.collectionName.find({ place : {$regex: /o/i } })

Checking if a field contains a string

I'm looking for an operator, which allows me to check, if the value of a field contains a certain string.
Something like:
db.users.findOne({$contains:{"username":"son"}})
Is that possible?
You can do it with the following code.
db.users.findOne({"username" : {$regex : "son"}});
As Mongo shell support regex, that's completely possible.
db.users.findOne({"username" : /.*son.*/});
If we want the query to be case-insensitive, we can use "i" option, like shown below:
db.users.findOne({"username" : /.*son.*/i});
See: http://www.mongodb.org/display/DOCS/Advanced+Queries#AdvancedQueries-RegularExpressions
https://docs.mongodb.com/manual/reference/sql-comparison/
http://php.net/manual/en/mongo.sqltomongo.php
MySQL
SELECT * FROM users WHERE username LIKE "%Son%"
MongoDB
db.users.find({username:/Son/})
As of version 2.4, you can create a text index on the field(s) to search and use the $text operator for querying.
First, create the index:
db.users.createIndex( { "username": "text" } )
Then, to search:
db.users.find( { $text: { $search: "son" } } )
Benchmarks (~150K documents):
Regex (other answers) => 5.6-6.9 seconds
Text Search => .164-.201 seconds
Notes:
A collection can have only one text index. You can use a wildcard text index if you want to search any string field, like this: db.collection.createIndex( { "$**": "text" } ).
A text index can be large. It contains one index entry for each unique post-stemmed word in each indexed field for each document inserted.
A text index will take longer to build than a normal index.
A text index does not store phrases or information about the proximity of words in the documents. As a result, phrase queries will run much more effectively when the entire collection fits in RAM.
As this is one of the first hits in the search engines, and none of the above seems to work for MongoDB 3.x, here is one regex search that does work:
db.users.find( { 'name' : { '$regex' : yourvalue, '$options' : 'i' } } )
No need to create and extra index or alike.
Here's what you have to do if you are connecting MongoDB through Python
db.users.find({"username": {'$regex' : '.*' + 'Son' + '.*'}})
you may also use a variable name instead of 'Son' and therefore the string concatenation.
Simplest way to accomplish this task
If you want the query to be case-sensitive
db.getCollection("users").find({'username':/Son/})
If you want the query to be case-insensitive
db.getCollection("users").find({'username':/Son/i})
ideal answer its use index
i option for case-insensitive
db.users.findOne({"username" : new RegExp(search_value, 'i') });
This should do the work
db.users.find({ username: { $in: [ /son/i ] } });
The i is just there to prevent restrictions of matching single cases of letters.
You can check the $regex documentation on MongoDB documentation.
Here's a link: https://docs.mongodb.com/manual/reference/operator/query/regex/
I use this code and it work for search substring
db.users.find({key: { $regex: new RegExp(value, 'i')}})
If you need to do the search for more than one attribute you can use the $or. For example
Symbol.find(
{
$or: [
{ 'symbol': { '$regex': input, '$options': 'i' } },
{ 'name': { '$regex': input, '$options': 'i' } }
]
}
).then((data) => {
console.log(data)
}).catch((err) => {
console.log(err)
})
Here you are basing your search on if the input is contained in the symbol attribute or the name attribute.
If the regex is not working in your Aggregate solution and you have nested object. Try this aggregation pipeline: (If your object structure is simple then, just remove the other conditions from below query):
db.user.aggregate({$match:
{$and:[
{"UserObject.Personal.Status":"ACTV"},
{"UserObject.Personal.Address.Home.Type":"HME"},
{"UserObject.Personal.Address.Home.Value": /.*son.*/ }
]}}
)
One other way would be to directly query like this:
db.user.findOne({"UserObject.Personal.Address.Home.Value": /.*son.*/ });
If your regex includes a variable, make sure to escape it.
function escapeRegExp(string) {
return string.replace(/[.*+?^${}()|[\]\\]/g, '\\$&'); // $& means the whole matched string
}
This can be used like this
new RegExp(escapeRegExp(searchString), 'i')
Or in a mongoDb query like this
{ '$regex': escapeRegExp(searchString) }
Posted same comment here
For aggregation framework
Field search
('$options': 'i' for case insensitive search)
db.users.aggregate([
{
$match: {
'email': { '$regex': '#gmail.com', '$options': 'i' }
}
}
]);
Full document search
(only works on fields indexed with a text index
db.articles.aggregate([
{
$match: { $text: { $search: 'brave new world' } }
}
])
How to ignore HTML tags in a RegExp match:
var text = '<p>The <b>tiger</b> (<i>Panthera tigris</i>) is the largest cat species, most recognizable for its pattern of dark vertical stripes on reddish-orange fur with a lighter underside. The species is classified in the genus <i>Panthera</i> with the lion, leopard, jaguar, and snow leopard. It is an apex predator, primarily preying on ungulates such as deer and bovids.</p>';
var searchString = 'largest cat species';
var rx = '';
searchString.split(' ').forEach(e => {
rx += '('+e+')((?:\\s*(?:<\/?\\w[^<>]*>)?\\s*)*)';
});
rx = new RegExp(rx, 'igm');
console.log(text.match(rx));
This is probably very easy to turn into a MongoDB aggregation filter.

mongoose node.js, query with $lt and $gt not working

I want to get all the pupils whose last mark is between 15 and 20. To do so, I perform the following query in my mongoDB using mongoose:
The models are working fine (all the other queries are ok).
Pupils.find({"marks[-1].value": {'$lt' : 20 }, "marks[-1].value" : { '$gt' : 15 }}, function(err, things){
This is not working, is there something I missed ?
* UPDATE *
I found something like:
Pupils.find({ "marks[-1].value": {$gt : 15, $lt : 20}});
But this does not work either. Is there a way to get the last mark of the marks array in this case ?
Lets consider your Pupils collection:
Pupils
{
_id,
Marks(integer),
LatestMark(int)
}
I suggest to add latest mark into Pupil document(as you can see at the document above), and update it each time when you adding new mark into nested collection.
Then you will able to query on it like this:
db.Pupils.find({ "LatestMark": {$gt : 15, $lt : 20}});
Also you can query latest mark using $where, but be care because:
Javascript executes more slowly than
the native operators, but is very flexible
I believe it's not working because the embedded collections in mongo are accessed like this:
"marks.0.value", although I haven't used mongoose.
Unfortunately for your scenario, I do not think there is a way to use negative indexing. (Mongo doesn't guarantee a preserved natural order unless you use a capped collection anyway though)
You may be able to accomplish this using Map/Reduce or a group command
http://www.mongodb.org/display/DOCS/MapReduce
This is tip than an answer.
Use double quotes for special key words like $elemMatch, $get, $lt etc while using mangoose.
In following code,$gt will not work properly.
var elmatch = { companyname: mycompany };
var condition = { company_info: { $elemMatch: elmatch } };
if(isValid(last_id)){
condition._id = { $gt: new ObjectId(last_id) };
}
console.log(condition);
User.find(condition).limit(limit).sort({_id: 1}).exec(function (err, lists) {
if (!err) {
res.send(lists);
res.end();
}else{
res.send(err);
res.end();
}
});
But this issue is solved when i'm using double quotes for special keywords
var elmatch = { companyname: mycompany };
var condition = { company_info: { "$elemMatch": elmatch } };
if(isValid(last_id)){
condition._id = { "$gt": new ObjectId(last_id) };
}
console.log(condition);
User.find(condition).limit(limit).sort({_id: 1}).exec(function (err, lists) {
if (!err) {
res.send(lists);
res.end();
}else{
res.send(err);
res.end();
}
});
I hope it will be helpful.