Accent insensitive in filter backend to datatable in ASP.NET MVC 5 - entity-framework

I made a method on the back-end side to handle the filter of my datatable.
On the other hand, this one does not manage the accents of the French language, so if I have "école" and I write "ecole" it cannot find it.
I found this method on another question on stackoverflow
public static String RemoveDiacritics(this String s)
{
String normalizedString = s.Normalize(NormalizationForm.FormD);
StringBuilder stringBuilder = new StringBuilder();
for (int i = 0; i < normalizedString.Length; i++)
{
Char c = normalizedString[i];
if (CharUnicodeInfo.GetUnicodeCategory(c) != UnicodeCategory.NonSpacingMark)
{
stringBuilder.Append(c);
}
}
return stringBuilder.ToString().Normalize(NormalizationForm.FormC);
}
and it works, but only for part of my problem. It works on the letter or the word that is written in the search, but I am not able to apply it in my linq query, so with the .RemoveDiacritics() method my "école" becomes "ecole", but I don't am not able to apply it in the column of my table and it always looks for "école".
Here the code for the search:
if (search != null)
{
int n;
search = search.Trim();
var isNumeric = int.TryParse(search, out n);
if (isNumeric)
{
IdFilter = n;
query = query.Where(x => x.UsagerId == IdFilter || x.Niveau == IdFilter);
}
else if (search != "")
{
// this line work
textFilter = search.ToLower().RemoveDiacritics();
// This is the full line, but absolutely takes the accents out to get the right information out
// query = query.Where(x => x.Nom.ToLower().Contains(textFilter) || x.Prenom.ToLower().Contains(textFilter) || x.Username.ToLower().Contains(textFilter) || x.Email.ToLower().Contains(textFilter) || x.EtabNom.ToLower().Contains(textFilter) || x.ActifStatut.ToLower().Contains(textFilter));
// This is the line that will replace the line above, which I try and it doesn't work ( this part: x.Prenom.ToLower().RemoveDiacritics())
query = query.Where(x => x.Prenom.ToLower().RemoveDiacritics().Contains(textFilter));
}
}
This is the basic query:
IQueryable<ListeUsagers> query = (from u in db.USAGERs
join e in db.ETABLISSEMENTs on u.USAGER_INST equals e.ETAB_CODE
where u.USAGER_INST == instId && u.USAGER_NIVEAU > 3 && u.USAGER_NIVEAU < 5 //&& u.USAGER_ACTIF == 1
select new ListeUsagers()
{
UsagerId = u.USAGER_id,
Nom = u.USAGER_NOM,
Prenom = u.USAGER_PRENOM,
EtabCode = e.ETAB_CODE,
EtabNom = e.ETAB_NOM_COURT,
EtabType = e.ETAB_TYPE,
Niveau = u.USAGER_NIVEAU,
Username = u.USAGER_USERNAME,
UserPassword = u.USAGER_MP,
DateCreation = u.USAGER_DATE_INSC,
Sexe = u.USAGER_SEXE,
Lang = u.USAGER_LANGUE,
Telephone = u.USAGER_TELEPHONE,
Email = u.USAGER_EMAIL,
FonctionTravail = u.USAGER_FONCTION,
LieuTravail = u.USAGER_LIEUTRAVAIL,
Note = u.USAGER_NOTE,
Actif = u.USAGER_ACTIF,
ActifStatut = u.USAGER_ACTIF == 0 ? "Inactif" : "Actif"
});
This is the error:
LINQ to Entities does not recognize the method 'System.String RemoveDiacritics(System.String)' method, and this method cannot be translated into a store expression.

There's built-in functionality to do this in entityframework: https://learn.microsoft.com/en-us/ef/core/miscellaneous/collations-and-case-sensitivity if you're using EF 5+
You'll want an accent insensitive collation ("AI", not "AS" in the examples on that page.)

Related

Combining Linq expressions using EFCore 3.0

I have a function doing a complicated Where query on my db context and then applies another transformation passed to it:
static IQueryable<T> Query<T>(Func<IQueryable<ServicesData>, IQueryable<T>> f, string path1 = null, string path2 = null, string path3 = null, string path4 = null, string path5 = null) {
try {
using (var dbc = new MyDbContext() ) {
var res = dbc.ServicesData
.Where(sd =>
(path1 == null || (path1.Contains("%") || path1.Contains("_") ? EF.Functions.Like(sd.Path1, path1) : sd.Path1 == path1))
&& (path2 == null || (path2.Contains("%") || path2.Contains("_") ? EF.Functions.Like(sd.Path2, path2) : sd.Path2 == path2))
&& (path3 == null || (path3.Contains("%") || path3.Contains("_") ? EF.Functions.Like(sd.Path3, path3) : sd.Path3 == path3))
&& (path4 == null || (path4.Contains("%") || path4.Contains("_") ? EF.Functions.Like(sd.Path4, path4) : sd.Path4 == path4))
&& (path5 == null || (path5.Contains("%") || path5.Contains("_") ? EF.Functions.Like(sd.Path5, path5) : sd.Path5 == path5)));
return f(res.ToList().AsQueryable());
//return f(res).ToList().AsQueryable();
}
} catch (Exception ex_) {
return VList<T>.Empty.AsQueryable();
}
}
This is used ie like this:
IQueryable<int> Int1InLastHour(IQueryable<ServicesData> input) {
var lastHour = DateTimeOffset.Now.AddHours(-1).ToUnixTimeMilliseconds();
return input
.Where(v => (v.Time <= lastHour) && (v.Int1 is object))
.Select(v => v.Int1.Value);
}
var lastHourProcessTime = Query(Int1InLastHour, "Publisher", "%", "ItemProcessTime").Sum();
This works, however since I call res.ToList() before calling f the linq in f is done in memory and not on the DB SQL
If I try to replace f(res.ToList().AsQueryable()) with f(res).ToList().AsQueryable() I get an exception:
{"Processing of the LINQ expression '[EntityShaperExpression][ServicesData]' by 'RelationalProjectionBindingExpressionVisitor' failed. This may indicate either a bug or a limitation in EF Core. See https://go.microsoft.com/fwlink/?linkid=2101433 for more detailed information."}
Is there any way for me to solve this ? can I somehow pass the query (Func<IQueryable<ServicesData>, IQueryable<T>>) and then combine it to the query in Query before excecuting it on the dbc ?
A few issues. You can split the querying to break down your results, but the scope of your DbContext needs to be at the outermost point of the chain, not inside the inner-most:
This here:
static IQueryable<T> Query<T>(Func<IQueryable<ServicesData>, IQueryable<T>> f, string path1 = null, string path2 = null, string path3 = null, string path4 = null, string path5 = null) {
try {
using (var dbc = new MyDbContext() ) { // DbContext should not be scoped here...
var res = dbc.ServicesData
As the simplest re-factor:
static IQueryable<T> Query<T>(MyDbContext dbc, Func<IQueryable<ServicesData>, IQueryable<T>> f, string path1 = null, string path2 = null, string path3 = null, string path4 = null, string path5 = null) {
try
{
var res = dbc.ServicesData.AsQueryable();
if(path1 != null)
if(path1.Contains("%") || path1.Contains("_"))
res = res.Where(EF.Functions.Like(sd.Path1, path1));
else
res = res.Where(sd.Path1 == path1);
// Repeat for Path 2 - 5 ....
return f(res);
}
catch (Exception ex_)
{
return VList<T>.Empty.AsQueryable();
}
}
Firstly, we pass in the DbContext. If the context is scoped here, the list must be materialized before being returned. The goal is to allow callers to further reduce the expression before executing the list. This means the DbContext needs to be scoped outside of this initial generation and passed in. With IoC containers managing lifetime scope you can bypass this if the DbContext is injected and scoped to a Request or common lifetime scope.
The next improvement suggestion is to move the conditional checks for the parameters out of the Linq and into regular conditions so that the Like / Equals check will only be added if the condition was provided. This will result in simpler, faster SQL being run on the server.
So the end result would look something like:
using (var dbContext = new MyDbContext())
{
var lastHourProcessTime = Query(dbContext, Int1InLastHour, "Publisher", "%", "ItemProcessTime").Sum();
}
I sort of get where you're trying to go here, but abstracting expressions from EF is bound to lead to confusing code and still prone to limitations and bugs. IMO keeping it simpler generally leads to less issues, but give this a go and see if it gets you closer.

mvc entity framework select with dynamic where clause

I am doing a asp.net-mvc with entity-framework App.
I have a select instruction, where I need to have a dymanic where condition.
It is a common case where you have a filter composed by a string like "aaaa bbbb cccc". I need to bring all data that contains all the filter string or part of it.
I doing part ot it witt Split function, but it is difficult to get all posible combinations.
I Would rather use a Store Procedure. But the porpose of it, is to use Entity Framework.
As far I did this.
public IEnumerable<UploadSearch> GetUploadsBySearch(string search)
{
IEnumerable<UploadSearch> viewModel = (from uploads in _db.Uploads
.Where(p => p.ProcessState_id == Security.APPROVED && p.Finder.Contains(search))
.OrderByDescending(p => p.UploadDate)
select new UploadSearch
{
User_id = uploads.User_id,
UserName = uploads.Users.Name,
UserLastName = uploads.Users.LastName,
});
And I Add a for instruction to loop throu the string,
string[] param = search.Replace(" "," ").Split(' ');
string _param = "";
int large = param.Length;
for (int i=0;i<large-1;i++)
{
_param +=param[i] ' ' + param[i + 1];
IEnumerable<UploadSearch> _viewModel = (from uploads in _db.Uploads
.Where(p => p.ProcessState_id == Security.APPROVED && p.Finder.Contains(_param))
.OrderByDescending(p => p.UploadDate)
select new UploadSearch
{
User_id = uploads.User_id,
UserName = uploads.Users.Name,
UserLastName = uploads.Users.LastName,
});
viewModel = viewModel.Union(_viewModel);
}
And I use a Union clause.
But There has to be another way to do it.
Any Ideas?
You should use LinqKit, it has a PredicateBuilder class that allows you to dynamically build queries
https://github.com/scottksmith95/LINQKit#predicatebuilder
IQueryable<Product> SearchProducts (params string[] keywords)
{
var predicate = PredicateBuilder.New<Product>();
foreach (string keyword in keywords)
{
string temp = keyword;
predicate = predicate.Or (p => p.Description.Contains (temp));
}
return dataContext.Products.Where (predicate);
}

How to write this Query in LINQ

I have one LINQ query with foreach loop. Everything is fine. But it takes more time to get the value. So anybody suggest me how can i do this in LINQ query itself.
Code
NormValue = "";
c = 0;
var NormValuelist = db.BCont.Where(x => x.BId == BId && x.TNo == Tag).ToList();
foreach (var item in NormValuelist)
{
if (c == 0)
NormValue = item.NormValue;
else
NormValue += " " + item.NormValue;
c = 1;
}
Thanks
You can rewrite this query with string.Join to avoid creating multiple string objects in a loop, like this:
string NormValue = string.Join(" ", db.BCont.Where(x => x.BId == BId && x.TNo == Tag));
The number of round-trips to DB will remain the same, but the creation of List<string> and the partially concatenated string objects will be optimized out.
In addition to using String.Join, you could also use Enumerable.Aggregate:
var NormValueList =
db.BCont.Where(x => x.Bid == BId && x.TNo == Tag)
.Select(x => x.NormValue)
.Aggregate((s, x) => s + " " + x);
If you are having large items in "NormValuelist" then it would be better to use StringBuilder instead of string(NormValue)

Entity Framework + conditionally appended Where() clauses

This is driving me nuts. What am I missing here. I'm using EF and if I have code like the following:
using (LexiconEntities ctx = new LexiconEntities())
{
var query = from w in ctx.Words
select new WordEntryDataModel
{
Word = w.Anagram,
NumOfAnagrams = w.NumAnagrams.Value,
Length = w.Length.Value,
...
};
SearchCriterion c1 = new SearchCriterion();
SearchCriterion c2 = new SearchCriterion();
c1.MinValue = 3;
c1.MaxValue = 3;
c2.MinValue = 4;
c2.MaxValue = 4;
query = query.Where(w => w.Length >= c1.MinValue && w.Length <= c1.MaxValue);
query = query.Where(w => w.NumOfAnagrams >= c2.MinValue && w.NumOfAnagrams <= c2.MaxValue);
...
}
And when I debug the query, I get the proper results (8 records). This also works as expected in Linqpad (which frickin' rocks).
But if I construct the search criteria as a List of criterion objects, and I dynamically add on Where() clauses by iterating over the search criteria as follows:
foreach (SearchCriterion c in criteria.SearchCriteria)
{
switch (c.Type)
{
case SearchCriterionType.WordLength:
query = query.Where(w => w.Length >= c.MinValue && w.Length <= c.MaxValue);
break;
case SearchCriterionType.NumberOfAnagrams:
query = query.Where(w => w.NumOfAnagrams >= c.MinValue && w.NumOfAnagrams <= c.MaxValue);
break;
...
case SearchCriterionType.NumberOfVowels:
query = query.Where(w => w.NumOfVowels >= c.MinValue && w.NumOfVowels <= c.MaxValue);
break;
}
}
...
I get the totally different (and incorrect) results. I've debugged the switch statement and my search criteria has two properly constructed criterion objects set to correct values. There's something about the conditionally added where clauses that my query doesn't like.
What am I doing wrong?
Closure. Assign c to a local variable within the loop. Also see my SO answer here

Using an existing IQueryable to create a new dynamic IQueryable

I have a query as follows:
var query = from x in context.Employees
where (x.Salary > 0 && x.DeptId == 5) || x.DeptId == 2
order by x.Surname
select x;
The above is the original query and returns let's say 1000 employee entities.
I would now like to use the first query to deconstruct it and recreate a new query that would look like this:
var query = from x in context.Employees
where ((x.Salary > 0 && x.DeptId == 5) || x.DeptId == 2) && (x,i) i % 10 == 0
order by x.Surname
select x.Surname;
This query would return 100 surnames.
The syntax is probably incorrect, but what I need to do is attach an additional where clause and modify the select to a single field.
I've been looking into the ExpressionVisitor but I'm not entirely sure how to create a new query based on an existing query.
Any guidance would be appreciated. Thanks you.
In an expression visitor you would override the method call. Check if the method is Queryable.Where, and if so, the methods second parameter is a quoted expression of type lambda expression. Fish it out and you can screw with it.
static void Main()
{
IQueryable<int> queryable = new List<int>(Enumerable.Range(0, 10)).AsQueryable();
IQueryable<string> queryable2 = queryable
.Where(integer => integer % 2 == 0)
.OrderBy(x => x)
.Select(x => x.ToString());
var expression = Rewrite(queryable2.Expression);
}
private static Expression Rewrite(Expression expression)
{
var visitor = new AddToWhere();
return visitor.Visit(expression);
}
class AddToWhere : ExpressionVisitor
{
protected override Expression VisitMethodCall(MethodCallExpression node)
{
ParameterExpression parameter;
LambdaExpression lambdaExpression;
if (node.Method.DeclaringType != typeof(Queryable) ||
node.Method.Name != "Where" ||
(lambdaExpression = ((UnaryExpression)node.Arguments[1]).Operand as LambdaExpression).Parameters.Count != 1 ||
(parameter = lambdaExpression.Parameters[0]).Type != typeof(int))
{
return base.VisitMethodCall(node);
}
return Expression.Call(
node.Object,
node.Method,
this.Visit(node.Arguments[0]),
Expression.Quote(
Expression.Lambda(
lambdaExpression.Type,
Expression.AndAlso(
lambdaExpression.Body,
Expression.Equal(
Expression.Modulo(
parameter,
Expression.Constant(
4
)
),
Expression.Constant(
0
)
)
),
lambdaExpression.Parameters
)
)
);
}
}
}