Columns Priority while searching with Lucene.NET - lucene.net

Team,
I have 6 indexed columns to search as below.
Name
Description
SKU
Category
Price
SearchCriteria
Now, While searching I have need to perform search on "SearchCritera" column first then rest of the columns.
In short - The products with matched "SearchCritera" shold display on the top of search results.
var parser = new MultiFieldQueryParser(Version.LUCENE_30,
new[] { "SearchCriteria",
"Name",
"Description",
"SKU",
"Category",
"Price"
}, analyzer);
var query = parseQuery(searchQuery, parser);
var finalQuery = new BooleanQuery();
finalQuery.Add(parser.Parse(searchQuery), Occur.SHOULD);
var hits = searcher.Search(finalQuery, null, hits_limit, Sort.RELEVANCE);

There are 2 ways to do it.
The first method is using field boosting:
During indexing set a boost to the fields by their priority:
Field name = new Field("Name", strName, Field.Store.NO, Field.Index.ANALYZED);
name.Boost = 1;
Field searchCriteria = new Field("SearchCriteria", strSearchCriteria, Field.Store.NO, Field.Index.ANALYZED);
searchCriteria.Boost = 2;
doc.Add(name);
doc.Add(searchCriteria);
This way the scoring of the terms in SearchCriteria field will be doubled then the scoring of the terms in the Name field.
This method is better if you always wants SearchCriteria to be more important than Name.
The second method is to using MultiFieldQueryParser boosting during search:
Dictionary<string,float> boosts = new Dictionary<string,float>();
boosts.Add("SearchCriteria",2);
boosts.Add("Name",1);
MultiFieldQueryParser parser = new MultiFieldQueryParser(Lucene.Net.Util.Version.LUCENE_30new[], new[] { "SearchCriteria", "Name"}, analyzer, boosts);
This method is better if you want the boosting to work only in some scenarios of your application.
You should try and see if the boosting number fits your needs (the sensitivity of the priority you are looking for) and change them according to your needs.
to make the example short and readable I used only 2 of your fields but you should use all of them of curseā€¦

Related

SapUi5 Table Multiple Filter

I'm developing a sap ui5 application using sap.ui.table.Table.
I need to apply a filter based on multiple strings. For example, if the user input is an array like:
["Stack", "Overflow"]
I need:
Filter all table fields by "Stack";
Filter the result of point 1 by "Overflow";
the result will be all rows that have "Stack" and "Overflow", no matter the field.
Does anyone have a solution?
As per the sap.ui.model.Filter documentation, you can create a filter either based on a filter info object, or from an array of previously created filters. This allows us to do the following:
Create a filter for the first value (eg "Stack")
Create a filter for the second value (eg "Overflow")
Create a filter which contains both of these values, and use it to filter the table.
Let's have a look at some code.
// We will only display rows where ProductName contains
// "Stack" AND CustomerName equals "Overflow"
var oFilterForProductName,
oFilterForCustomerName,
aArrayWhichContainsBothPreviousFilters = [],
oFilterToSetOnTheTable;
var sValueToFilterTheProductNameOn = "Stack",
sValueToFilterTheCustomerNameOn = "Overflow";
var sKeyForProductNameInTheTableModel = "ProductName",
sKeyForCustomerNameInTheTableModel = "CustomerName";
var oTableToFilter = this.byId("myTableId");
// Step 1: create two filters
oFilterForProductName = new sap.ui.model.Filter(
sKeyForProductNameInTheTableModel,
sap.ui.model.FilterOperator.Contains,
sValueToFilterTheProductNameOn);
oFilterForCustomerName = new sap.ui.model.Filter(
sKeyForCustomerNameInTheTableModel,
sap.ui.model.FilterOperator.EQ,
sValueToFilterTheCustomerNameOn);
// Step 2: add these two filters to an array
aArrayWhichContainsBothPreviousFilters.push(oFilterForProductName);
aArrayWhichContainsBothPreviousFilters.push(oFilterForCustomerName);
// Step 3: create a filter based on the array of filters
oFilterToSetOnTheTable = new sap.ui.model.Filter({
filters: aArrayWhichContainsBothPreviousFilters,
and: true
});
oTableToFilter.getBinding("items").filter(oFilterToSetOnTheTable , sap.ui.model.FilterType.Application);
Hope this helps. Let me know if you have any questions.
Chris
Please pass that array in for loop and pass filters like,
var tableId = this.byId("oTable");
for(var i=0;i < array.length ; i++)
{
oTable.getBinding().filter(new sap.ui.model.Filter("", sap.ui.model.FilterOperator.Contains, array[0]));
}
it may be helpful for you.

Exclude field from full-text search

I need to do the full text search in the MongoDB (version 2.4). I use the following fragment of code.
DBObject textSearchCommand = new BasicDBObject();
textSearchCommand.put("text", "profile");
textSearchCommand.put("search", pattern);
textSearchCommand.put("limit", searchLimit);
textSearchCommand.put("filter",new BasicDBObject("personInfo", new BasicDBObject("$ne",null)));
CommandResult commandResult = mongoTemplate.executeCommand(textSearchCommand);
BasicDBList results = (BasicDBList) commandResult.get("results");
It works well but I want to exclude one field (person picture data) from the text search.
Note: I don't want to exclude this field from the result. I want that MongoDB does not search in this field.
Which fields to search in is determined when you create the text index. When you only want the text index to apply to selected fields, you need to provide these fields at creation like this for example:
db.articles.createIndex(
{
title: "text",
synopsis: "text",
content: "text",
tags: "text"
}
)
When this is not an option for some reason (like when you don't know all possible field names which might be relevant for text search), an (admittedly dirty) workaround could be to store the non-searchable content in a different data-type than a string, for example as binary data.

How to implement search with multiple filters using lucene.net

I'm new to lucene.net. I want to implement search functionality on a client database. I have the following scenario:
Users will search for clients based on the currently selected city.
If the user wants to search for clients in another city, then he has to change the city and perform the search again.
To refine the search results we need to provide filters on Areas (multiple), Pincode, etc. In other words, I need the equivalent lucene queries to the following sql queries:
SELECT * FROM CLIENTS
WHERE CITY = N'City1'
AND (Area like N'%area1%' OR Area like N'%area2%')
SELECT * FROM CILENTS
WHERE CITY IN ('MUMBAI', 'DELHI')
AND CLIENTTYPE IN ('GOLD', 'SILVER')
Below is the code I've implemented to provide search with city as a filter:
private static IEnumerable<ClientSearchIndexItemDto> _search(string searchQuery, string city, string searchField = "")
{
// validation
if (string.IsNullOrEmpty(searchQuery.Replace("*", "").Replace("?", "")))
return new List<ClientSearchIndexItemDto>();
// set up Lucene searcher
using (var searcher = new IndexSearcher(_directory, false))
{
var hits_limit = 1000;
var analyzer = new StandardAnalyzer(Lucene.Net.Util.Version.LUCENE_30);
// search by single field
if (!string.IsNullOrEmpty(searchField))
{
var parser = new QueryParser(Lucene.Net.Util.Version.LUCENE_30, searchField, analyzer);
var query = parseQuery(searchQuery, parser);
var hits = searcher.Search(query, hits_limit).ScoreDocs;
var results = _mapLuceneToDataList(hits, searcher);
analyzer.Close();
searcher.Dispose();
return results;
}
else // search by multiple fields (ordered by RELEVANCE)
{
var parser = new MultiFieldQueryParser(Lucene.Net.Util.Version.LUCENE_30, new[]
{
"ClientId",
"ClientName",
"ClientTypeNames",
"CountryName",
"StateName",
"DistrictName",
"City",
"Area",
"Street",
"Pincode",
"ContactNumber",
"DateModified"
}, analyzer);
var query = parseQuery(searchQuery, parser);
var f = new FieldCacheTermsFilter("City",new[] { city });
var hits = searcher.Search(query, f, hits_limit, Sort.RELEVANCE).ScoreDocs;
var results = _mapLuceneToDataList(hits, searcher);
analyzer.Close();
searcher.Dispose();
return results;
}
}
}
Now I have to provide more filters on Area, Pincode, etc. in which Area is multiple. I tried BooleanQuery like below:
var cityFilter = new TermQuery(new Term("City", city));
var areasFilter = new FieldCacheTermsFilter("Area",areas); -- where type of areas is string[]
BooleanQuery filterQuery = new BooleanQuery();
filterQuery.Add(cityFilter, Occur.MUST);
filterQuery.Add(areasFilter, Occur.MUST); -- here filterQuery.Add not have an overloaded method which accepts string[]
If we perform the same operation with single area then it's working fine.
I've tried with ChainedFilter like below, which doesn't seems to satisfy the requirement. The below code performs or operation on city and areas. But the requirement is to perform OR operation between the areas provided in the given city.
var f = new ChainedFilter(new Filter[] { cityFilter, areasFilter });
Can anybody suggest to me how to achieve this in lucene.net? Your help will be appreciated.
You're looking for the BooleanFilter. Almost any query object has a matching filter object.
Look into TermsFilter (from Lucene.Net.Contrib.Queries) if your indexing doesn't match the requirements of FieldCacheTermsFilter. From the documentation of the later; "this filter requires that the field contains only a single term for all documents".
var cityFilter = new FieldCacheTermsFilter("CITY", new[] {"MUMBAI", "DELHI"});
var clientTypeFilter = new FieldCacheTermsFilter("CLIENTTYPE", new [] { "GOLD", "SILVER" });
var areaFilter = new TermsFilter();
areaFilter.AddTerm(new Term("Area", "area1"));
areaFilter.AddTerm(new Term("Area", "area2"));
var filter = new BooleanFilter();
filter.Add(new FilterClause(cityFilter, Occur.MUST));
filter.Add(new FilterClause(clientTypeFilter, Occur.MUST));
filter.Add(new FilterClause(areaFilter, Occur.MUST));
IndexSearcher searcher = null; // TODO.
Query query = null; // TODO.
Int32 hits_limit = 0; // TODO.
var hits = searcher.Search(query, filter, hits_limit, Sort.RELEVANCE).ScoreDocs;
What you are looking for is nested boolean queries so that you have an or (on your cities) but that whole group (matching the or) is itself matched as an and
filter1 AND filter2 AND filter3 AND (filtercity1 OR filtercity2 OR filtercity3)
There is already a good description of how to do this here:
How to create nested boolean query with lucene API (a AND (b OR c))?

Google like autosuggest with Lucene.net

I have a Lucene index that stores customers that basically includes a view model (documents fields that are stored and not indexed), an ID (field stored and indexed to permit find and update of document), and a list of terms covered by the google-like search (multiple field instances of name Term). Terms may be field in the view model or not.
This works fine for the actual searching of documents by term. The question is how I can implement auto-suggest, basically get a list of Term (the field, not Lucene Term) values that might be the continuation of the entered value (i.e. "Co" might result in "Colorado", "Coloring Book", etc because those are actual values in at least one Document's Term field.
Theres a lot of way to do this, but if you need a quick and simple way to do it, use a TermEnum.
Just paste this little code sample in a new C# console application and check if it works for you to start from.
RAMDirectory dir = new RAMDirectory();
IndexWriter iw = new IndexWriter(dir, new KeywordAnalyzer(), IndexWriter.MaxFieldLength.UNLIMITED);
Document d = new Document();
Field f = new Field("text", "", Field.Store.YES, Field.Index.ANALYZED);
d.Add(f);
f.SetValue("abc");
iw.AddDocument(d);
f.SetValue("colorado");
iw.AddDocument(d);
f.SetValue("coloring book");
iw.AddDocument(d);
iw.Commit();
IndexReader reader = iw.GetReader();
TermEnum terms = reader.Terms(new Term("text", "co"));
int maxSuggestsCpt = 0;
// will print:
// colorado
// coloring book
do
{
Console.WriteLine(terms.Term.Text);
maxSuggestsCpt++;
if (maxSuggestsCpt >= 5)
break;
}
while (terms.Next() && terms.Term.Text.StartsWith("co"));
reader.Dispose();
iw.Dispose();

NumericRangeQuery in NHibernate.Search

I am creating a search, where the user can both choose an interval and search on a term in the same go.
This is however giving me trouble, since I have up until have only used the usual text query.
I am wondering how I am to go about using both a NumericRangeQuery and a regular term query. Usually I would use a query below:
var parser = new MultiFieldQueryParser(
new[] { "FromPrice", "ToPrice", "Description"}, new SimpleAnalyzer());
Query query = parser.Parse(searchQuery.ToString());
IFullTextSession session = Search.CreateFullTextSession(this.Session);
IQuery fullTextQuery = session.CreateFullTextQuery(query, new[] { typeof(MyObject) });
IList<MyObject> results = fullTextQuery.List<MyObject>();
But if I was to e.g. search the range FromPrice <-> ToPrice and also the description, how should I do this, since session.CreateFullTextQuery only takes one Query object?
you can create a single query that is a BooleanQuery combining all the conditions you want to be met.
For the ranges, heres a link to the synthax using the QueryParser:
http://lucene.apache.org/core/old_versioned_docs/versions/2_9_2/queryparsersyntax.html#Range Searches