I'm trying to filter algolia results and want to use the name of a product to find similar products in our database. I would prefer to ignore the current product so, in the case no results are found, the removeWordsIfNoResults option will be used.
I'm using the Ruby api.
This is basically what I'm trying but it is not working. Is it possible to filter out a specific objectID?
Product.raw_search("Men's Bridge & Burn Camden Steel Blue",
{ :hitsPerPage => 10,
:page => 0,
:attributesToRetrieve => "objectID",
:filters => "objectID != '65411'",
:removeWordsIfNoResults => 'lastWords'
}
)
objectID is a specific attribute in Algolia that is always a string value and cannot be added to the attributesForFaceting .
However, if you're filling it with the ids you have in your database, you can index another attribute (let's say id) with the same value.
Numerical value
As an numerical value, you can directly filter on it using Algolia. However, since you'll only be using the != operator, I recommend you to declare it in the numericAttributesToIndex list with an equalOnly modifier:
index.set_settings({
numericAttributesToIndex: %w(equalOnly(id))
})
At query time, you can pass it as you did in your example if you remove the quotes around the value:
index.raw_search('Product title', {
filters: 'id!=65411'
# Your other parameters
})
String value
With a string value, you want to add it to your attributesForFaceting :
index.set_settings({
attributesForFaceting: %w(id)
})
Then query it like so:
index.raw_search('Product title', {
filters: 'id:-65411'
# Your other parameters
})
(The - for exclusion is described in the facetFilters documentation)
Related
I am working on a function that helps me find similar documents, sorted by score, using the full-text search feature of MongoDB Atlas.
I set my collection index as "dynamic".
I am looking for similarities in text fields, such as "name" or "description", but I also want to look in another field, "thematic", that stores integer values (ids) of thematics.
Example:
Let say that I have a reference document as follows:
{
name: "test",
description: "It's a glorious day!",
thematic: [9, 3, 2, 33]
}
I want my search to match these int in the thematic field and include their weight in the score calculation.
For instance, if I compare my reference document with :
{
name: "test2",
description: "It's a glorious night!",
thematic: [9, 3, 6, 22]
}
I want to increase the score since the thematic field shares the 9 and 3 values with the reference document.
Question:
What search operator should I use to achieve this? I can input array of strings as queries with a text operator but I don't know how to proceed with integers.
Should I go for another approach? Like splitting the array to compare into several compound.should.term queries?
Edit:
After a fair amount of search, I found this here and here:
Atlas Search cannot index numeric or date values if they are part of an array.
Before I consider to change the whole data structure of my objects, I wanted to make sure that there is no workaround.
For instance, could it be done with custom analyzers?
I solved it by adding a trigger to my collection. Each time a document is inserted or updated, I update the thematic and other similar fields counterparts, e.g. _thematic, where I store the string value of the integers. I then use this _thematic field for search.
Here is a sample code demonstrating it:
exports = function (changeEvent) {
const fullDocument = changeEvent.fullDocument;
const format = (itemSet) => {
let rst = [];
Object.keys(itemSet).forEach(item => rst.push(itemSet[item].toString()));
return rst;
};
let setter = {
_thematic: fullDocument.thematic ? format(fullDocument.thematic) : [],
};
const docId = changeEvent.documentKey._id;
const collection = context.services.get("my-cluster").db("dev").collection("projects");
const doc = collection.findOneAndUpdate({ _id: docId },
{ $set: setter });
return;
};
I'm pretty sure it can be done in a cleaner way, so if someone post it, I'll switch the selected answer to her/his.
Another way to solve this is to make a custom analyser with character mapping that will replace each digit with its string counterpart. I haven’t tried this one tho. See https://docs.atlas.mongodb.com/reference/atlas-search/analyzers/custom/#mapping
Alternatives welcome!
Assume, I have a User collection with many fields. I know a user mail for example test#test.com. I am looking on a way to print the Key of test#test.com which is "email" in mongo print output.
{"email":"test#test.com"}
You can find the document you are interested in, project such that only the email attribute is returned and then grab the field name from the returned document. For example:
db.collection.find(
// find all documents (you'll probably have your own criteria to add here)
{},
// project on the email attribute and - to reduce noise - exclude
// the default _id projection
{email: 1, '_id': 0}
)
// it would make sense to limit here since you are only interested in an attribute name
// and, presumably, all relevant documents would have the same attribute name
.limit(1)
// since you only projected on email the returned document is
// expected to have a single attribute
.forEach(function(myDoc) {
for (var key in myDoc) {
// print the attribute name
print("key: " + key);
}
})
FWIW, this is ~unusual not least since if you know you are interested in the email attribute then why would you need to write code to get your hands on it :) Perhaps if you provide more detail on why you want to do this another solution might become apparent.
If you are looking for a generic solution to 'find me the names of all keys' in my docs then something like this might be better:
db.collection.find(
// find all documents (you'll probably have your own criteria to add here)
{},
// to reduce noise exclude the default _id projection
{'_id': 0}
)
// it would make sense to limit here since you are only interested in an attribute name
// and, presumably, all relevant documents would have the same attribute name
.limit(1)
.forEach(function(myDoc) {
for (var key in myDoc) {
// print the attribute name
print("key: " + key);
}
})
I want to add the unique index to a field ignoring null values in the unique indexed field and ignoring the documents that are filtered based on partialFilterExpression.
The problem is Sparse indexes can't be used with the Partial index.
Also, adding unique indexes, adds the null value to the index key field and hence the documents can't be ignored based on $exist criteria in the PartialFilterExpression.
Is it possible in MongoDB 3.2 to get around this situation?
I am adding this answer as I was looking for a solution and didn't find one. This may not answer exactly this question or may be, but will help lot of others out there like me.
Example. If the field with null is houseName and it is of type string, the solution can be like this
db.collectionName.createIndex(
{name: 1, houseName: 1},
{unique: true, partialFilterExpression: {houseName: {$type: "string"}}}
);
This will ignore the null values in the field houseName and still be unique.
Yes, you can create partial index in MongoDB 3.2
Please see https://docs.mongodb.org/manual/core/index-partial/#index-type-partial
MongoDB recommend usage of partial index over sparse index. I'll suggest you to drop your sparse index in favor of partial index.
You can create partial index in mongo:3.2.
Example, if ipaddress can be "", but "127.0.0.1" should be unique. The solution can be like this:
db.collectionName.createIndex(
{"ipaddress":1},
{"unique":true, "partialIndexExpression":{"ipaddress":{"$gt":""}}})
This will ignore "" values in ipaddress filed and still be unique
{
"YourField" : {
"$exists" : true,
"$gt" : "0",
"$type" : "string"
}
}
To create at mongodbCompass you must write it as JSON:
for find other types wich supports see this link.
Yes, that can be a kind of a problem that the partial filter expression cannot contain any 'not' filters.
For those who can be interested in a C# solution for an index like this, here is an example.
We have a 'User' entity, which has one-to-one 'relation' to a 'Doctor' entity.
This relation is represented by the not required, nullable field 'DoctorId' in the 'User' entity. In other words, there is a requirement that a given 'Doctor' can be linked to only single 'User' at a time.
So we need an unique index which can fire an exception when something attempts to set DoctorId to the same Guid which already set for any other 'User' entity. At the same time multiple 'null' entries must be allowed for the 'DoctorId' field, since many users do not have any doctor attached to them.
The solution to build this kind of an index looks like:
var uniqueDoctorIdIndexDefinition = new IndexKeysDefinitionBuilder<User>()
.Ascending(o => o.DoctorId);
var existsFilter = Builders<User>.Filter.Exists(o => o.DoctorId);
var notNullFilter = Builders<User>.Filter.Type(o => o.DoctorId, BsonType.String);
var andFilter = Builders<User>.Filter.And(existsFilter, notNullFilter);
var createIndexOptions = new CreateIndexOptions<User>
{
Unique = true,
Name = UniqueDoctorIdIndexName,
PartialFilterExpression = andFilter,
};
var uniqueDoctorIdIndex = new CreateIndexModel<User>(
uniqueDoctorIdIndexDefinition,
createIndexOptions);
users.Indexes.CreateOne(uniqueDoctorIdIndex);
Probably in your description of a 'User' entity you must directly specify the BsonType of the 'DoctorId' field, by using an attribute, for example in our case it was:
[BsonRepresentation(BsonType.String)]
public Guid? DoctorId { get; set; }
I am more than sure that there is a more proficient and compact solution for this problem, so would be happy if somebody suggests it here.
Here is an example that I modified from the mongoDB partial index documentation:
db.contacts.createIndex(
{ email: 1 },
{ unique: true, partialFilterExpression: { email: { $exists: true } } }
)
IMPORTANT
To use the partial index, a query must contain the filter expression (or a modified filter expression that specifies a subset of the filter expression) as part of its query condition.
You can see that queries such as:
db.contacts.find({'email':'name#email.com'}).explain()
will indicate that they doing an index scan, even if you don't specify {$exists: true} because you're implicitly specifying a subset of the partialFilterExpression by specifying an email in your filter.
On the other hand, the following query will do a collection scan:
db.contacts.find({email: {$exists: false}})
WARNING
mythicalcoder's answer (currently the highest voted answer) is very misleading because it successfully creates a unique index, but the query planner will not generally be able to use the index you've created unless you add houseName: {$type: "string"} into your filter expression. This can have performance costs which you might not be aware of and can cause problems down the road.
being a newbie to mongo, stuck in a conditional query:
I want to perform a search on the basis of 3 criteria, first name, last name and email id:
below query works perfect when all the fields exist:
db.students.find( { $and: [ { first_name: /^Abc$/i }, { last_name: 'Xyz'},{email_id:'gd#he'} ]})
the problem is when I don't give an email id , the query dosen't returns any result as it considers the email id to be null and searches for the combination 'Abc Firtis null',
where as I want the below scenario to be fulfilled:
I have a collection of students:
- FirstName: 1. ABC 2. ABC 3.ABC
- LastName: 1.XYZ 2. XY 3. XZ
- EmailID: 1.abc#xyz 2.Ab#xy 3.Ab#xz
if one enters only the first name in the search it should return all the 3 results
if user enters first name and last name it should return first two results and if the user enters all three details it should return only 1 result.
Any leads would be highly appreciated.
You seem to be talking about "input" data being different for the queries you want to issue and how to contruct the query to ignore fields as criteria for which you have no input.
This is all really about how the input is being collected as to how you handle it, but it all boils down to that you "conditionally build" the query ( which is just a data structure anyway ) rather than statically define a query and somehow ignore null or empty data.
So if you have seperate variables, then you test each value and build the query:
var firstName = "abc",
lastName = "xy,
email = null,
query = {};
if (firstName) {
query.firstName = new RegExp("^"+firstName,"i")
}
if (lastName) {
query.lastName = new RegExp("^"+lastName,"i")
}
if (email) {
query.email = new RegExp("^"+email,"i")
}
db.students.find(query)
This would build a query object that would end up like this based on the inputs:
{ "firstName": /^abc/i, "lastName": /^xy/i }
So since there is no value in email then the condition is not included. The end result is the condition not provided is not even queried for and then you get the relevant matches.
The same approach is basically simplified if you have some structured input to begin with:
var params = {
firstName = "abc",
lastName = "xy"
};
var query = {};
Object.keys(params).forEach(function(key) {
if (params[key])
query[key] = new RegExp("^"+key,"i");
});
db.students.find(query);
And it's the same thing, but since you have all parameter keys in one place then you can iterate them to build the query rather than test individually.
This is generally the case where you have input from something like a web request with parameters that come into req.params or even req.body depending on your method of input. So if you structure your code to accept input into a similar object ( or already have it ) then you use it to build your query.
Also note that all MongoDB query arguments are implicitly an "AND" condition by definition, so there is rarely any need to use $and unless you explicitly have multiple conditions to meet for the same document property. Even then there are generally better syntax alternates.
No Need to give and you can simply try this
find( { first_name: { $regex: '/^Abc$/i' }, last_name:'Xyz',email_id:'gd#he'}
I currently have the following dataset (simplified):
{
'component_id':1,
'_locales':[
{
'url': 'dutch',
'locale': 'nl_NL'
},
{
'url': 'english',
'locale': 'en_US'
}
]
} (etc more rows similar to this but unique urls)
When I query for a specific url and locale I use the following query
db.find({'_locales': { '$elemMatch': { 'locale': 'nl_NL', 'url': 'dutch' } }});
I get the proper row back however '_locales' return the entire array including en_US which I don't need, is there anyway it only returns the matched array row, in this case:
'_locales':[
{
'url': 'dutch',
'locale': 'nl_NL'
}]
I have a feeling I have to iterate through the locales and match the row to the locale. It doesn't feel right, is there a better solution to do this (not iterating over the result set)? For example, changing the table structure? I was hoping to do it this way without making a second table for the locales only..
You always query for top level documents. Just because your query criteria involve matching a specific array element doesn't tell MongoDB that it should only return that element. Currently there's no way to return specific array elements other than using the $slice operator which is not what you need here.
There are feature requests in MongoDB JIRA that will allow for what you want but currently it is not possible.