How to configure query by example to include null values? - spring-data-jpa

I have the following matcher:
Example.of(
user,
ExampleMatcher.matching()
.withMatcher("name", contains().ignoreCase())
.withMatcher("phoneNumber", contains())
)
It works fine except null values. For example, it doesn't return the users whose phone number is NULL.
I have tried to following to include the NULL values but it didn't work:
Example.of(
user,
ExampleMatcher.matching()
.withMatcher("name", contains().ignoreCase())
.withIncludeNullValues()
.withMatcher("phoneNumber", contains())
.withIncludeNullValues()
)
The generated SQL is the following:
select
user0_.id as id1_9_,
user0_.document_expiry_date as document2_9_,
user0_.document_type as document3_9_,
user0_.document_url as document4_9_,
user0_.email as email5_9_,
user0_.name as name6_9_,
user0_.phone_number as phone_nu7_9_
from
user user0_
where
(lower(user0_.name) like ?)
and (user0_.id is null)
and (user0_.document_type is null)
and (user0_.document_url is null)
and (user0_.email is null)
and (user0_.phone_number like ?)
and (user0_.document_expiry_date is null)
How can I configure it to include rows with NULL columns as well?

Confronted the same problem, but I think there is no way to accomplish this goal. Here is the source code of method withMatcher:
public ExampleMatcher withMatcher(String propertyPath, ExampleMatcher.GenericPropertyMatcher genericPropertyMatcher) {
Assert.hasText(propertyPath, "PropertyPath must not be empty!");
Assert.notNull(genericPropertyMatcher, "GenericPropertyMatcher must not be empty!");
ExampleMatcher.PropertySpecifiers propertySpecifiers = new ExampleMatcher.PropertySpecifiers(this.propertySpecifiers);
ExampleMatcher.PropertySpecifier propertySpecifier = new ExampleMatcher.PropertySpecifier(propertyPath);
if (genericPropertyMatcher.ignoreCase != null) {
propertySpecifier = propertySpecifier.withIgnoreCase(genericPropertyMatcher.ignoreCase);
}
if (genericPropertyMatcher.stringMatcher != null) {
propertySpecifier = propertySpecifier.withStringMatcher(genericPropertyMatcher.stringMatcher);
}
if (genericPropertyMatcher.valueTransformer != null) {
propertySpecifier = propertySpecifier.withValueTransformer(genericPropertyMatcher.valueTransformer);
}
propertySpecifiers.add(propertySpecifier);
return new ExampleMatcher(this.nullHandler, this.defaultStringMatcher, propertySpecifiers, this.ignoredPaths, this.defaultIgnoreCase, this.mode);
}
this method returns a ExampleMatcher with the 'global' nullHandler which applied to all the fields. That means, you can't specify a different nullHander for some special fields.

You can implements JpaSpecificationExecutor<T> in your repository and use findAll(Specification specification), and set like param:
//My solution, create a specification with predicate and add example.
public Specification<MaestroPlasticoTebca> specificationAttributeNull(boolean isNull, Example<MaestroPlasticoTebca> example, String attributeName) {
return (root, query, builder) -> {
final List<Predicate> predicates = new ArrayList<>();
if (isNull) {
predicates.add(builder.isNull(root.get(attributeName)));
} else {
predicates.add(builder.isNotNull(root.get(attributeName)));
}
predicates.add(QueryByExamplePredicateBuilder.getPredicate(root, builder, example));
return builder.and(predicates.toArray(new Predicate[0]));
};
}
And use it:
Example example = Example.of(
user,
ExampleMatcher.matching()
.withMatcher("name", contains().ignoreCase())
);
repository.findAll(specificationAttributeNull(true, example, "phoneNumber"));

Related

Combining Linq expressions using EFCore 3.0

I have a function doing a complicated Where query on my db context and then applies another transformation passed to it:
static IQueryable<T> Query<T>(Func<IQueryable<ServicesData>, IQueryable<T>> f, string path1 = null, string path2 = null, string path3 = null, string path4 = null, string path5 = null) {
try {
using (var dbc = new MyDbContext() ) {
var res = dbc.ServicesData
.Where(sd =>
(path1 == null || (path1.Contains("%") || path1.Contains("_") ? EF.Functions.Like(sd.Path1, path1) : sd.Path1 == path1))
&& (path2 == null || (path2.Contains("%") || path2.Contains("_") ? EF.Functions.Like(sd.Path2, path2) : sd.Path2 == path2))
&& (path3 == null || (path3.Contains("%") || path3.Contains("_") ? EF.Functions.Like(sd.Path3, path3) : sd.Path3 == path3))
&& (path4 == null || (path4.Contains("%") || path4.Contains("_") ? EF.Functions.Like(sd.Path4, path4) : sd.Path4 == path4))
&& (path5 == null || (path5.Contains("%") || path5.Contains("_") ? EF.Functions.Like(sd.Path5, path5) : sd.Path5 == path5)));
return f(res.ToList().AsQueryable());
//return f(res).ToList().AsQueryable();
}
} catch (Exception ex_) {
return VList<T>.Empty.AsQueryable();
}
}
This is used ie like this:
IQueryable<int> Int1InLastHour(IQueryable<ServicesData> input) {
var lastHour = DateTimeOffset.Now.AddHours(-1).ToUnixTimeMilliseconds();
return input
.Where(v => (v.Time <= lastHour) && (v.Int1 is object))
.Select(v => v.Int1.Value);
}
var lastHourProcessTime = Query(Int1InLastHour, "Publisher", "%", "ItemProcessTime").Sum();
This works, however since I call res.ToList() before calling f the linq in f is done in memory and not on the DB SQL
If I try to replace f(res.ToList().AsQueryable()) with f(res).ToList().AsQueryable() I get an exception:
{"Processing of the LINQ expression '[EntityShaperExpression][ServicesData]' by 'RelationalProjectionBindingExpressionVisitor' failed. This may indicate either a bug or a limitation in EF Core. See https://go.microsoft.com/fwlink/?linkid=2101433 for more detailed information."}
Is there any way for me to solve this ? can I somehow pass the query (Func<IQueryable<ServicesData>, IQueryable<T>>) and then combine it to the query in Query before excecuting it on the dbc ?
A few issues. You can split the querying to break down your results, but the scope of your DbContext needs to be at the outermost point of the chain, not inside the inner-most:
This here:
static IQueryable<T> Query<T>(Func<IQueryable<ServicesData>, IQueryable<T>> f, string path1 = null, string path2 = null, string path3 = null, string path4 = null, string path5 = null) {
try {
using (var dbc = new MyDbContext() ) { // DbContext should not be scoped here...
var res = dbc.ServicesData
As the simplest re-factor:
static IQueryable<T> Query<T>(MyDbContext dbc, Func<IQueryable<ServicesData>, IQueryable<T>> f, string path1 = null, string path2 = null, string path3 = null, string path4 = null, string path5 = null) {
try
{
var res = dbc.ServicesData.AsQueryable();
if(path1 != null)
if(path1.Contains("%") || path1.Contains("_"))
res = res.Where(EF.Functions.Like(sd.Path1, path1));
else
res = res.Where(sd.Path1 == path1);
// Repeat for Path 2 - 5 ....
return f(res);
}
catch (Exception ex_)
{
return VList<T>.Empty.AsQueryable();
}
}
Firstly, we pass in the DbContext. If the context is scoped here, the list must be materialized before being returned. The goal is to allow callers to further reduce the expression before executing the list. This means the DbContext needs to be scoped outside of this initial generation and passed in. With IoC containers managing lifetime scope you can bypass this if the DbContext is injected and scoped to a Request or common lifetime scope.
The next improvement suggestion is to move the conditional checks for the parameters out of the Linq and into regular conditions so that the Like / Equals check will only be added if the condition was provided. This will result in simpler, faster SQL being run on the server.
So the end result would look something like:
using (var dbContext = new MyDbContext())
{
var lastHourProcessTime = Query(dbContext, Int1InLastHour, "Publisher", "%", "ItemProcessTime").Sum();
}
I sort of get where you're trying to go here, but abstracting expressions from EF is bound to lead to confusing code and still prone to limitations and bugs. IMO keeping it simpler generally leads to less issues, but give this a go and see if it gets you closer.

Why does this query always return ALL records?

I'm using WCF RIA in a Lightswitch project to create some query results. This query brings back all results regardless. I cannot make it filter the records based on the parameter passed (string Town).
public IQueryable<Enquiries> TestQuery(string Town)
{
List<Enquiries> riaenqs = new List<Enquiries>();
var enqs = this.Context.ClientEnquiries
.Include("Client")
.Include("Client.Town")
.OrderBy(enq => enq.Id);
if (Town != null)
{
enqs.Where(enq => enq.Client.Town.TownName == Town);
}
foreach (ClientEnquiry item in enqs.ToList())
{
Enquiries enq = new Enquiries();
enq.Id = item.Id;
enq.ClientName = item.Client.FirstName + " " + item.Client.Surname;
enq.Town = item.Client.Town != null ? item.Client.Town.TownName : null;
riaenqs.Add(enq);
}
return riaenqs.AsQueryable();
}
During debugging I can see that the Town is correctly populated and I can see that the query is built accordingly if Town is not null. However, when I hit the foreach statement where the linq to ef query is executed I always get all the results. I just cannot figure out where I'm slipping up.
The LINQ methods like the Where do not modify the collection/expression but always returning a new one.
So you need to reassign the result of the Where to your original variable enqs:
if (Town != null)
{
enqs = enqs.Where(enq => enq.Client.Town.TownName == Town);
}

declare variable to store linq entity for conditional statements

I am trying to look up record using if I have the key then use Find if not use Where
private ApplicationDbContext db = new ApplicationDbContext();
public bool DeactivatePrice(int priceId = 0, string sponsorUserName = "")
{
var prices = db.BeveragePrices;
// if we have an id then find
if (priceId != 0)
{
prices = prices.Find(priceId);
}
else
{
prices = prices.Where(b => b.UserCreated == sponsorUserName);
}
if (prices != null)
{
// do something
}
return true;
I get the following error for
prices = prices.Find(priceId);
Cannot convert app.Model.BeveragePrices from system.data.entity.dbset
I am copying the pattern from this answer but something must be different.
Seems you forgot to put a predicate inside the Find function call. Also you need to do ToList on the collection. The second option is a lot more efficient. The first one gets the whole collection before selection.
Another note commented by #Alla is that the find returns a single element. So I assume another declaration had been made for 'price' in the first option I state down here.
price = prices.ToList.Find(b => b.PriceId == priceId);
Or
prices = prices.Select(b => b.PriceId == priceId);
I assume the field name is PriceId.

Generic repository: how to filter eager-loaded navigation properties

this is my generic respoitory that im using, as the title states I want to know how to filter the navigation properties.
public IEnumerable<T> Query(
Expression<Func<T, bool>> filter = null,
Func<IQueryable<T>, IOrderedQueryable<T>> orderBy = null,
string includeProperties = "")
{
IQueryable<T> query = _objectSet.Where(e => !e.IsDeleted);
if (filter != null)
{
query = query.Where(filter);
}
foreach (var includeProperty in includeProperties.Split
(new char[] { ',' }, StringSplitOptions.RemoveEmptyEntries))
{
query = query.Include(includeProperty);
}
if (orderBy != null)
{
return orderBy(query).ToList();
}
else
{
return query.ToList();
}
}
Controller:
var viewModel = new StudentViewModel();
viewModel.Students= _unitOfWork.Students.Query(
includeProperties: "Subjects, Instructors");
Now my problem is I want to add a .Where(e => !e.IsDeleted)
to [Subjects] and [Instructors] using the repository.
Thanks
EDIT:
according to Ladislav, currently it is not possible (also mentioned here in msdn: http://blogs.msdn.com/b/adonet/archive/2011/01/31/using-dbcontext-in-ef-feature-ctp5-part-6-loading-related-entities.aspx)
Can i just use this instead?
viewModel.Subjects = viewModel.Students.Where(i => i.StudentID ==Id.Value)
.Single().Subjects.Where(e => !e.IsDeleted);
My only worry is that the query may return lots of records with isDeleted==true. Sure the code I posted as an alternative works, I just don't want to pull the data that I don't need even though I can filter it using the above code
LINQ to SQL supports this scenario using the LoadWith DataLoadOption. The example at http://msdn.microsoft.com/en-us/library/system.data.linq.dataloadoptions.loadwith.aspx shows the simple case that EF does support with the Include statement.
Northwnd db = new Northwnd(#"c:\northwnd.mdf");
DataLoadOptions dlo = new DataLoadOptions();
dlo.LoadWith<Customer>(c => c.Orders);
db.LoadOptions = dlo;
However, unlike EF, LINQ to SQL also supports the following:
dlo.LoadWith<Customer>(c => c.Orders.Where(o => o.ShippedDate is Null);
If you feel this is an important enhancement scenario for EF, consider voting for it at http://data.uservoice.com/forums/72025-entity-framework-feature-suggestions/suggestions/1015345-allow-filtering-for-include-extension-method.
For now, your best option is to project your filter in the Select clause, but that gets tricky with a generic repository.

The 'ArrayIndex' LINQ expression node type is not supported by LINQ to Entities - using interface & ReportViewer

a quick question really.
I'm struggling to implement Linq2Entities statement that could take more than one value for a particular "field". I'm passing a number of strings to the getClientsProjected() I can easily compare single value. But I've got on my page multiple dropdown and out of that I get string separated with coma I then later use to split it to string[] e.g. __ACCOUNT_SITE = "1234,5678" (see the code below) I've tried for/foreach/contains none of which worked...
public IQueryable<ClientViewModel> getClientsProjected(string __ACCOUNT_SITE, string __ACCOUNT)
{
var projectedClients = from c in getClosedSRs()
select new ClientViewModel
{
_ACCOUNT_ID_CSR = c.ACCOUNT_ID_CSR,
_ACCOUNT = c.ACCOUNT,
_ACCOUNT_FAMILY = c.ACCOUNT_FAMILY,
...
...
_ACCOUNT_SITE = c.ACCOUNT_SITE
};
if (String.IsNullOrEmpty(__ACCOUNT) != true && __ACCOUNT != "ALL")
{
//this works fine as an __ACCOUNT is of a single value
projectedClients = projectedClients.Where(c => c._ACCOUNT == __ACCOUNT);
}
if (String.IsNullOrEmpty(__ACCOUNT_SITE) != true && __ACCOUNT_SITE != "ALL")
{
String[] splitSites = __ACCOUNT_SITE.Split(',');
//????????????????????????????????????????????????
}
return projectedClients;
}
Now, to most of you this will make complete sense. I've read many articles but did not find a proper answer. I however can't use Linq2SQL as already built my entire site using L2E, interface and ReportViewer.
Any workaround?
If you are trying to filter projectedClients based on the values in splitSites, then use:
if (String.IsNullOrEmpty(__ACCOUNT_SITE) != true && __ACCOUNT_SITE != "ALL")
{
String[] splitSites = __ACCOUNT_SITE.Split(',');
projectedClients = projectedClients.Where(x => splitSites.Contains(x._ACCOUNT);
}