I am working on getting back up to speed on Spring, Spring Data and JPA. I was interested in implementing the Specifications piece to enable a more dynamic query.
Say I have the following methods:
protected String containsLowerCase(String searchField) {
return WILDCARD + searchField.toLowerCase() + WILDCARD;
}
protected Specification<T> attributeContains(String attribute, String value) {
return (root, query, cb) -> {
return value == null ? null : cb.like(
cb.lower(root.get(attribute)),
containsLowerCase(value)
);
};
}
#Override
public Specification<Widget> getFilter(WidgetListRequest request) {
return (root, query, cb) -> Specifications.where(
Specifications.where(nameContains(request.getSearch()))
.or(descriptionContains(request.getSearch()))
)
.and(testValBetween(request.getMinTestVal(), request.getMaxTestVal()))
.toPredicate(root, query, cb);
}
private Specification<Widget> nameContains(String name) {
return attributeContains("name", name);
}
And in my service I call like:
WidgetListRequest request = new WidgetListRequest();
request.setSearch(search);
request.setMinTestVal(minTestVal.doubleValue());
request.setMaxTestVal(maxTestVal.doubleValue());
return widgetRepo.findAll(widgetSpec.getFilter(request));
If this was straight Repo call I know it, like save() is safe against SQL injection, and I know that JPQL is safe when you put parameters in. But, as I have tested, the above is NOT safe from SQL injection since it's using a concatenated string for the wildcards. I've seen some examples using the CriteraBuilder and CriteriaQuery to add parameters (ex: this), but I am not sure how that will work when you are using the Specifications on the service-side.
What I would need to do, I think, is somehow return the Specification with ParameterExpressions, but I don't know how to set them once the Specification is returned to the service class, nor how to add the wildcards such that they don't get escaped. How do I escape just the passed in value of containsLowerCase() above?
Related
I have a service that passes in parameters for how much I want to include for navigation properties. Based upon the boolean args it concatenates an entity list to include each required foreign entity.
At runtime I want to include either no navigation entities or many.
What I can't do is daisy chain with .Include().Include as I don't know which and how many to include based around passed in args.
I want to achieve this, but I don't seem to be able to pass in a comma separated entity list. Any ideas?
var res = db.Entity.Include(entityListCommaSeparated).Where(_=>_.ID == ID).FirstOrDefault();
This looks like a repository pattern, and generally gets messy if you want to try and "hide" EF / the DbContext from calling code.
A couple options you can consider:
Down the complexity rabit hole: use a params Expression<Func<TEntity, object>>[] includes in your applicable repository methods, and then be prepared to also pass OrderBy expressions, as well as pagination values when you want to return multiple entities.
THrough the simplicity mirror: Embrace IQueryable as a return type and let the consumers handle Includes, OrderBy's, Counts/Any/Skip/Take/First/ToList, and .Select() as they need.
Option 1:
public Order GetById(int id, params Expression<Func<Order, object>>[] includes)
{
var query = db.Orders.Where(x => x.ID == id);
// This part can be moved into an extension method or a base repository method.
if(includes.Any)
includes.Aggregate(query, (current, include) =>
{
current.Include(include);
}
// Don't use .FirstOrDefault() If you intend for 1 record to be returned, use .Single(). If it really is optional to find, .SingleOrDefault()
return query.Single();
}
//ToDo
public IEnumerable<Order> GetOrders(/* criteria?, includes?, order by?, (ascending/descending) pagination? */)
{ }
// or
public IEnumerable<Order> GetOrdersByCustomer(/* includes?, order by?, (ascending/descending) pagination? */)
{ }
// plus..
public IEnumerable<Order> GetOrdersByDate(/* includes?, order by?, (ascending/descending) pagination? */)
{ }
public bool CustomerHasOrders(int customerId)
{ }
public bool OrderExists(int id)
{ }
public int OrdersOnDate(DateTime date)
{ }
// etc. etc. etc.
Keep in mind this doesn't handle custom order by clauses, and the same will be needed for methods that are returning lists of entities. Your repository is also going to need to expose methods for .Any() (DoesExist) because everyone loves checking for #null on every return. :) Also .Count().
Option 2:
public IQueryable<Order> GetById(int id)
{
return db.Orders.Where(x => x.ID == id);
}
public IQueryable<Order> GetOrders()
{
return db.Orders.AsQueryable();
}
Callers can grok Linq and .Include() what they want before calling .Single(), or do a .Any().. They may not need the entire entity graph so they can .Select() from the entity and related entities without .Include() to compose and execute a more efficient query to populate a ViewModel / DTO. GetById might be used in a number of places so we can reduce duplication and support it in the repository. We don't need all of the filter scenarios etc, callers can call GetOrders and then filter as they see fit.
Why bother with a repository if it just returns DBSets?
Centralize low-level data filtering. For instance if you use Soft Deletes (IsActive) or are running multi-tenant, or explicit authorization. These common rules can be centralized at the repository level rather than having to remembered everywhere a DbSet is touched.
Testing is simpler. While you can mock a DbContext, or point it at an in-memory database, mocking a repository returning IQueryable is simpler. (Just populate a List<TEntity> and return .AsQueryable().
Repositories handle Create and Delete. Create to serve as a factory to ensure that all required data and relationships are established for a viable entity. Delete to handle soft-delete scenarios, cascades/audits etc. beyond what the DB handles behind the scenes.
I'm using .net web Api with Entity Framework. Its really nice that you can just do
[EnableQuery]
public IQueryable<Dtos.MyDto> Get()
{
return dbContext.MyEntity.Select(m => new MyDto
{
Name = m.Name
});
}
And you get odata applying to the Iqueryable, note also returning a projected dto.
But that select is a expression and so its being turned to into sql. Now in the above case that's fine. But what if I need to do some "complex" formatting on the return dto, its going to start having issues as SQL wont be able to do it.
Is it possible to create an IQueryable Wrapper?
QWrapper<TEntity,TDo>(dbcontext.MyEntity, Func<TEntity,TDo> dtoCreator)
it implements IQueryable so we still return it allowing webapi to apply any odata but the Func gets called once EF completes thus allowing 'any' .net code to be called as its not converting to SQL.
I don't want to do dbContext.MyEntity.ToList().Select(...).ToQueryable() or whatever as that will always return the entire table from the db.
Thoughts?
since you query already returns the data you expected, how about adding .Select(s=>new MyEntity(){ Name=s.Name }) for returning them as OData response? like:
return dbContext.MyEntity.Select(m => new MyDto
{
Name = m.Name
}).Select(s=>new MyEntity(){ Name=s.Name });
I used "Code-First from Database" to create a model on my project. Standard queries (without "expand" and "select") work just fine, but they do not work when including "expand" and "select". What could be the problem?
EDIT:
Here is a code sample:
/**
* Find charges by location and category
* #method
* #param {int, int} locationId, categoryId
* #return {object array}
*/
this.filter = function (params) {
var p1 = breeze.Predicate.create('locationId', '==', params.locationId),
p2 = breeze.Predicate.create('categoryId', '==', params.categoryId),
query = breeze.EntityQuery
.from(resourceName).expand('ChargeItem').select('id,chargeItemId,chargeItem.name');
if (params.locationId && params.categoryId) return executeQuery(query.where(p1.and(p2)));
if (!params.locationId && params.categoryId) return executeQuery(query.where(p2));
if (params.locationId && !params.categoryId) return executeQuery(query.where(p1));
return executeCacheQuery(query);
};
function executeCacheQuery(query) {
return manager().executeQueryLocally(query);
}
function executeQuery(query) {
return manager().executeQuery(query.using(breeze.FetchStrategy.FromLocalCache))
.then(function (cacheData) {
if (cacheData.results.length == 0) {
return manager().executeQuery(query.using(breeze.FetchStrategy.FromServer))
.then(function (serverData) { return serverData.results; });
} else return cacheData.results;
});
}
function manager() {
return entityManagerProvider.manager();
}
};
resourceName refers to the Charges function in the Breeze ApiController (for this instance, breeze/charges). The Charge table (in the database) has three foreign keys: LocationId, CategoryId and ChargeItemId. I would like to find charges by locationId and/or categoryId and select charge Id, chargeItemId and chargeItem.Name (Name field in the ChargeItem table). It is not working at all when using expand() and select(). It does not even hit the server when I check the network traffic.
Any questions would be welcome. Thanks in advance for the help.
My bet is that the query is failing and you're just not seeing the error.
Notice that you are not catching an error if the query fails. You have then() clauses but no catch() clauses.
Lesson #1 - always check for errors in a promise.
And why is your query failing? I suspect that you have misspelled the navigation property path in the expand() clause.
Lesson #2 - always use the client-side spelling for property names when composing queries.
You're evidently using the NamingConvention.camelCase to map PascalCase property names on the server ("ChargeItemId") to camelCase property names on the client ("chargeItemId").
You should use the camelCase spelling consistently everywhere on the client ... including in the expand clause. I'll be it works if you change the relevant part of your query to .expand("chargeItem")
But do first go back and put in the catch callback so you can see the error. Then fix the error.
Suggestion
I don't know if this is your real query. If it is, you can simplify it substantially by building the query incrementally, perhaps like this:
var query = breeze.EntityQuery.from(resourceName)
.expand('chargeItem')
.select('id, chargeItemId, chargeItem.name');
if (params.locationId) {
query = query.where('locationId', '==', params.locationId);
}
if (params.categoryId) {
query = query.where('locationId', '==', params.categoryId);
}
return executeCacheQuery(query);
This works because Breeze ANDs where clauses together.
I generally use a generic repository to boilerplate my EF queries so I have to write limited code and also use caching. The source code for the repository can be found here.
The backbone query within the code is this one below. FromCache<T>() is an IEnumerable<T> extension method that utilizes the HttpContext.Cache to store the query using a stringified representation of the lambda expression as a key.
public IQueryable<T> Any<T>(Expression<Func<T, bool>> expression = null)
where T : class, new()
{
// Check for a filtering expression and pull all if not.
if (expression == null)
{
return this.context.Set<T>()
.AsNoTracking()
.FromCache<T>(null)
.AsQueryable();
}
return this.context.Set<T>()
.AsNoTracking<T>()
.Where<T>(expression)
.FromCache<T>(expression)
.AsQueryable<T>();
}
Whilst this all works it is subject to the N+1 problem for related tables since If I were to write a query like so:
var posts = this.ReadOnlySession.Any<Post>(p => p.IsDeleted == false)
.Include(p => p.Author);
The Include() will have no effect on my query since it has already been run in order to be cached.
Now I know that I can force Entity Framework to use eager loading within my model by removing the virtual prefix on my navigation properties but that to me feels like the wrong place to do it as you cannot predict the types of queries you will be making. To me it feels like something I would be doing in a controller class. What I am wondering is whether I can pass a list of includes into my Any<T>() method that I could then iterate though when I make the call?
ofDid you mean something like...
IQueryable<T> AnyWithInclude<T,I>(Expression<Func<T,bool>> predicate,
Expression<Func<T,I>> includeInfo)
{
return DbSet<T>.where(predicate).include(includeInfo);
}
the call
Context.YourDbSetReference.AnyWithInclude(t => t.Id==someId, i => i.someNavProp);
In response to extra question on as collection.
I realised late, there was an overload on Property. You can just pass a string
This might work but call is not easy. Well I find it hard.
IQueryable<T> GetListWithInclude<I>(Expression<Func<T, bool>> predicate,
params Expression<Func<T, I>>[] IncludeCollection);
so i tried
public virtual IQueryable<T> GetListWithInclude(Expression<Func<T, bool>> predicate,
List<string> includeCollection)
{ var result = EntityDbSet.Where(predicate);
foreach (var incl in includeCollection)
{
result = result.Include(incl);
}
return result;
}
and called with
var ic = new List<string>();
ic.Add("Membership");
var res = context.DbSte<T>.GetListWithInclude( t=>t.UserName =="fred", ic);
worked as before.
In the interest of clarity I'm adding the solution I came up with based upon #soadyp's answer.
public IQueryable<T> Any<T>(Expression<Func<T, bool>> expression = null,
params Expression<Func<T, object>>[] includeCollection)
where T : class, new()
{
IQueryable<T> query = this.context.Set<T>().AsNoTracking().AsQueryable<T>();
if (includeCollection.Any())
{
query = includeCollection.Aggregate(query,
(current, include) => current.Include(include));
}
// Check for a filtering expression and pull all if not.
if (expression != null)
{
query = query.Where<T>(expression);
}
return query.FromCache<T>(expression, includeCollection)
.AsQueryable<T>();
}
Usage:
// The second, third, fourth etc parameters are the strongly typed includes.
var posts = this.ReadOnlySession.Any<Post>(p => p.IsDeleted == false,
p => p.Author);
I use DTO's to map between my Business and Entity Framework layer via the Repository Pattern.
A Standard call would look like
public IClassDTO Fetch(Guid id)
{
var query = from s in _db.Base.OfType<Class>()
where s.ID == id
select s;
return query.First();
}
Now I wish to pass in filtering criteria from the business layer so I tried
public IEnumerable<IClassDTO> FetchAll(ISpecification<IClassDTO> whereclause)
{
var query = _db.Base.OfType<Class>()
.AsExpandable()
.Where(whereclause.EvalPredicate);
return query.ToList().Cast<IClassDTO>();
}
The Call from the business layer would be something like
Specification<IClassDTO> school =
new Specification<IClassDTO>(s => s.School.ID == _schoolGuid);
IEnumerable<IClassDTO> testclasses = _db.FetchAll(school);
The problem I am having is that the .Where clause on the EF query cannot be inferred from the usage. If I use concrete types in the Expression then it works find but I do not want to expose my business layer to EF directly.
Try making FetchAll into a generic on a class instead, like this:-
public IEnumerable<T> FetchAll<T> (Expression<Func<T,bool>> wherePredicate)
where T:IClassDTO //not actually needed
{
var query = _db.Base.OfType<T>()
.AsExpandable()
.Where(wherePredicate);
return query;
}
pass in school.Evalpredicate instead. FetchAll doesn't appear to need to know about the whole specification, it just needs the predicate, right? If you need to cast it to IClassDTO, do that after you have the results in a List.