I am using Spring Data JPA and QueryDsl (v.4.2.2), Java 8. I can explicitly construct search predicates and pass them to the repository methods. However, I like the idea of using the #QuerydslPredicate annotation on a web/REST controller's method argument when the queried entities have more than a few properties, and I want the flexibility of filtering the search by any of them. So, something like this, generally, works very well:
#GetMapping("/accounts/summaries")
public PageDto<AccountSummaryDto> getAccountSummaries(#QuerydslPredicate(root = AccountSummary.class) Predicate accountSearchPredicate,
#RequestParam(name = "pageIndex", defaultValue = "0") int pageIndex,
#RequestParam(name = "pageSize", defaultValue = "25") int pageSize,
#RequestParam(name = "sortBy", defaultValue = "id") String sortBy,
#RequestParam(name = "sortOrder", defaultValue = "desc") String sortOrder) {
// delegating to web-agnostic service that:
// - creates Pageable pageRequest,
// - calls accountSummaryRepository.findAll(predicate, pageRequest),
// - constructs custom PageDto wrapper, etc.
return accountService.retrieveAccountSummaries(accountSearchPredicate, pageIndex, pageSize, sortBy, sortOrder);
}
My Spring Data JPA repository interface looks similar to this:
public interface AccountSummarySearchRepository
extends JpaRepository<AccountSummary, Integer>, QuerydslPredicateExecutor<AccountSummary>, QuerydslBinderCustomizer<QAccountSummary > {
#Override
default void customize(QuerydslBindings bindings, QAccountSummary acctSummary) {
bindings.bind(acctSummary.customer.firstName).first((path, value) -> path.isNull().or(path.startsWithIgnoreCase(value))) ;
bindings.bind(acctSummary.customer.lastName).first((path, value) -> path.isNull().or(path.startsWithIgnoreCase(value))) ;
// etc.
// default binding for String properties to be case insensitive "contains" match
bindings.bind(String.class).first(
(StringPath path, String value) -> path.isNull().or(path.containsIgnoreCase(value)));
}
My question:
The bindings in the customize method are set using the entity field
paths and the values of the request parameters that match those
paths. If the parameter is not specified, is there a way to bind the
path to some constant value or a value obtained dynamically?
For example, I want to always ONLY retrieve the entities where property deleted is set to false - without forcing the client to pass that as a query parameter? Similarly, I may want to set other default lookup values dynamically for each query. For example, I may want to "retrieve only those accounts where assignedTo == [current user ID available on a ThreadLocal]...
The following will not work
bindings.bind(acctSummary.deleted).first((path, value) -> path.eq(false));
because it, obviously, expects the first occurrence of the path/value pair for deleted=... in the Predicate (mapped from the incoming request params via the #QuerydslPredicate annotation. I don't want to pass that as a parameter because the requester does not even need to know about the existence of such field.
Is there a simple way to infuse the Predicate instance that is auto-populated via the #QuerydslPredicate annotation with any additional implicit/default criteria that are not explicitly passed in the web request? Could this be done in the customize method? I suppose, one (very ugly) way would be to intercept the HTTP request in a filter - before it is processed by the Spring-QueryDsl framework - and replace it with a new request with added parameters? That would be a horrible solution, and I feel there has to be a better way to do it via some hook/capability provided by the framework itself.
Unfortunately, there seem to be no comprehensive documentation for Spring QueryDsl support - other than some very simplistic examples.
Thanks for your help!
Answering my own question... I was hoping to find a hook in the framework where I could add the code to enhance the auto-generated predicate with criteria common for all my queries - before it arrives in the controller method, but wasn’t able to figure that out. Overriding QuerydslPredicateArgumentResolver doesn't seem a good or necessary option. And, quite frankly, I've come to the conclusion that this wasn't such a great idea to begin with. It seems that any modifications to the search criteria should be done in a more obvious way - in the business tier. So I decided to simply update the predicate in the service method:
public PageDto<AccountSummaryDto> retrieveByPredicate(Predicate predicate, int pageIndex, int pageSize, String sortBy, String sortOrder) {
Pageable pageRequest = PageRequest.of(pageIndex, pageSize, Sort.Direction.fromString(sortOrder), sortBy);
QAccountSummary accountSummary = QAccountSummary.accountSummary; //QueryDsl auto-generated query type for AccountSummary (path root)
// construct new enhanced search predicate w/added criteria common for all queries
// using original predicate generated by framework from request params as base
BooleanBuilder updatedPredicate = new BooleanBuilder(predicate)
.and(accountSummary.somethingNested.id.eq(SomeThreadContext.getSomethingId()))
.and(accountSummary.deleted.eq(false))
.and(accountSummary.someProperty.eq("xyz"));
Page<accountSummary> page = summarySearchRepository.findAll(updatedPredicate, pageRequest);
return toAccountSummaryPageDto(page); // custom method that converts results to page DTO w/entity dots and page stats
}
The construction of the updated predicate may be extracted into a separate private method on the service should it be desirable to use it in multiple search methods and/or if more logic is required to dynamically generate additional search criteria.
Related
I have a service that passes in parameters for how much I want to include for navigation properties. Based upon the boolean args it concatenates an entity list to include each required foreign entity.
At runtime I want to include either no navigation entities or many.
What I can't do is daisy chain with .Include().Include as I don't know which and how many to include based around passed in args.
I want to achieve this, but I don't seem to be able to pass in a comma separated entity list. Any ideas?
var res = db.Entity.Include(entityListCommaSeparated).Where(_=>_.ID == ID).FirstOrDefault();
This looks like a repository pattern, and generally gets messy if you want to try and "hide" EF / the DbContext from calling code.
A couple options you can consider:
Down the complexity rabit hole: use a params Expression<Func<TEntity, object>>[] includes in your applicable repository methods, and then be prepared to also pass OrderBy expressions, as well as pagination values when you want to return multiple entities.
THrough the simplicity mirror: Embrace IQueryable as a return type and let the consumers handle Includes, OrderBy's, Counts/Any/Skip/Take/First/ToList, and .Select() as they need.
Option 1:
public Order GetById(int id, params Expression<Func<Order, object>>[] includes)
{
var query = db.Orders.Where(x => x.ID == id);
// This part can be moved into an extension method or a base repository method.
if(includes.Any)
includes.Aggregate(query, (current, include) =>
{
current.Include(include);
}
// Don't use .FirstOrDefault() If you intend for 1 record to be returned, use .Single(). If it really is optional to find, .SingleOrDefault()
return query.Single();
}
//ToDo
public IEnumerable<Order> GetOrders(/* criteria?, includes?, order by?, (ascending/descending) pagination? */)
{ }
// or
public IEnumerable<Order> GetOrdersByCustomer(/* includes?, order by?, (ascending/descending) pagination? */)
{ }
// plus..
public IEnumerable<Order> GetOrdersByDate(/* includes?, order by?, (ascending/descending) pagination? */)
{ }
public bool CustomerHasOrders(int customerId)
{ }
public bool OrderExists(int id)
{ }
public int OrdersOnDate(DateTime date)
{ }
// etc. etc. etc.
Keep in mind this doesn't handle custom order by clauses, and the same will be needed for methods that are returning lists of entities. Your repository is also going to need to expose methods for .Any() (DoesExist) because everyone loves checking for #null on every return. :) Also .Count().
Option 2:
public IQueryable<Order> GetById(int id)
{
return db.Orders.Where(x => x.ID == id);
}
public IQueryable<Order> GetOrders()
{
return db.Orders.AsQueryable();
}
Callers can grok Linq and .Include() what they want before calling .Single(), or do a .Any().. They may not need the entire entity graph so they can .Select() from the entity and related entities without .Include() to compose and execute a more efficient query to populate a ViewModel / DTO. GetById might be used in a number of places so we can reduce duplication and support it in the repository. We don't need all of the filter scenarios etc, callers can call GetOrders and then filter as they see fit.
Why bother with a repository if it just returns DBSets?
Centralize low-level data filtering. For instance if you use Soft Deletes (IsActive) or are running multi-tenant, or explicit authorization. These common rules can be centralized at the repository level rather than having to remembered everywhere a DbSet is touched.
Testing is simpler. While you can mock a DbContext, or point it at an in-memory database, mocking a repository returning IQueryable is simpler. (Just populate a List<TEntity> and return .AsQueryable().
Repositories handle Create and Delete. Create to serve as a factory to ensure that all required data and relationships are established for a viable entity. Delete to handle soft-delete scenarios, cascades/audits etc. beyond what the DB handles behind the scenes.
I am trying to get Envelope's back from a query. Envelope is defined as follows.
[<CLIMutable>]
type Envelope<'T> = {
Id : Guid
StreamId: Guid
Created : DateTimeOffset
Item : 'T }
MyLibAAS.DataStore.MyLibAASDbContext is a EF DbContext written in c#. When I extend it in f# as follows, I get the error: Only parameterless constructors and initializers are supported in LINQ to Entities.
type MyLibAAS.DataStore.MyLibAASDbContext with
member this.GetEvents streamId =
query {
for event in this.Events do
where (event.StreamId = streamId)
select {
Id = event.Id;
StreamId = streamId;
Created = event.Timestamp;
Item = (JsonConvert.DeserializeObject<QuestionnaireEvent> event.Payload)
}
}
If I return the event and map it to Envelope after the fact, it works fine.
type MyLibAAS.DataStore.MyLibAASDbContext with
member this.GetEvents streamId =
query {
for event in this.Events do
where (event.StreamId = streamId)
select event
} |> Seq.map (fun event ->
{
Id = event.Id
StreamId = streamId
Created = event.Timestamp
Item = (JsonConvert.DeserializeObject<QuestionnaireEvent> event.Payload)
}
)
Why does this make a difference? The Envelope type is not even a EF type.
How F# records work
F# records get compiled into .NET classes with read-only properties and a constructor that takes values of all fields as parameters (plus a few interfaces).
For example, your record would be expressed in C# roughly as follows:
public class Envelope<T> : IComparable<Envelope<T>>, IEquatable<Envelope<T>>, ...
{
public Guid Id { get; private set; }
public Guid StreamId { get; private set; }
public DateTimeOffset Created { get; private set; }
public T Item { get; private set; }
public Envelope( Guid id, Guid streamId, DateTimeOffset created, T item ) {
this.Id = id;
this.StreamId = streamId;
this.Created = created;
this.Item = item;
}
// Plus implementations of IComparable, IEquatable, etc.
}
When you want to create an F# record, the F# compiler emits a call to this constructor, supplying values for all fields.
For example, the select part of your query would look in C# like this:
select new Envelope<QuestionnaireEvent>(
event.Id, streamId, event.Timestamp,
JsonConvert.DeserializeObject<QuestionnaireEvent>(event.Payload) )
Entity Framework limitations
It so happens that Entity Framework does not allow calling non-default constructors in queries. There is a good reason: if it did allow it, you could, in principle, construct a query like this:
from e in ...
let env = new Envelope<E>( e.Id, ... )
where env.Id > 0
select env
Entity Framework wouldn't know how to compile this query, because it doesn't know that the value of e.Id passed to the constructor becomes the value of the property env.Id. This is always true for F# records, but not for other .NET classes.
Entity Framework could, in principle, recognize that Envelope is an F# record and apply the knowledge of the connection between constructor arguments and record properties. But it doesn't. Unfortunately, the designers of Entity Framework did not think of F# as a valid use case.
(fun fact: C# anonymous types work the same way, and EF does make an exception for them)
How to fix this
In order to make this work, you need to declare Envelope as a type with default constructor. The only way to do this is to make it a class, not a record:
type Envelope<'T>() =
member val Id : Guid = Guid.Empty with get, set
member val StreamId : Guid = Guid.Empty with get, set
member val Created : DateTimeOffset = DateTimeOffset.MinValue with get, set
member val Item : 'T = Unchecked.defaultof<'T> with get, set
And then create it using property initialization syntax:
select Envelope<_>( Id = event.Id, StreamId = streamId, ... )
Why does moving the select to a Seq.map work
The Seq.map call is not part of the query expression. It does not end up as part of the IQueryable, so it does not end up compiled to SQL by Entity Framework. Instead, EF compiles just what's inside query and returns you the resulting sequence, after fetching it from SQL Server. And only after that you apply Seq.map to that sequence.
The code inside Seq.map is executed on CLR, not compiled to SQL, so it can call anything it wants, including non-default constructors.
This "fix" comes with a cost though: instead of just the fields you need, the whole Event entity gets fetched from DB and materialized. If this entity is heavy, this may have a performance impact.
Another thing to watch out for
Even if you fix the problem by making Envelope a type with default constructor (as suggested above), you'll still hit the next problem: the method JsonConvert.DeserializeObject can't be compiled to SQL, so Entity Framework will complain about it, too. The way you should do it is fetch all fields to the CLR side, then apply whatever non-SQL-compilable transformations you need.
Using LINQ to Entities, everything that happens in the query computational expression is actually executed within the database engine, which may reside on a remote server. Everything outside of it is executed in the running application on the client.
So, in your first snippet, Entity Framework refuses to execute Envelope<'T>'s constructor because, in order to do so, it would need to translate that into SQL commands for the server. This is plainly not something it can guarantee, because the constructor could potentially contain any sort of side effects and .NET framework code - it could request user input, read files from the client's hard disk, whatever.
What EF can do, in your second snippet, is sending its own POCO event objects back to the client, which is then tasked with Seq.mapping them to your fancy Envelopes, which it can do because it's running on your client machine with access to the full .NET framework.
Addendum: So why are parameterless constructors ok? What if I were to call MsgBox() in a parameterless constructor? I think that parameterless constructors work by having the client construct the objects (without knowing the query results), sending them to the server in serialised form, and having the server just fill the object's properties with the query results.
I haven't actually tested that. But F# record types have no parameterless constructors anyway, so the point is moot in your case.
Exposing my EF models to an API always seemed wrong. I'd like my API to return a custom entity model to the caller but use EF on the back.
So I may have PersonRestEntity and a controller for CRUD ops against that and a Person EF code-first entity behind in and map values.
When I do this, I can no longer use the following to allow ~/people?$top=10 etc. in the URL
[EnableQuery]
public IQueryable<Person> Get(ODataQueryOptions<Person> query) { ... }
Because that exposes Person which is private DB implementation.
How can I have my cake and eat it?
I found a way. The trick is not to just return the IQueryable from the controller, because you need to materialise the query first. This doesn't mean materialising the whole set into RAM, the query is still run at the database, but by explicitly applying the query and materialising the results you can return mapped entities thereafter.
Define this action, specifying the DbSet entity type:
public async Task<HttpResponseMessage> Get(ODataQueryOptions<Person> oDataQuery)
And then apply the query manually to the DbSet<Person> like so:
var queryable = oDataQuery.ApplyTo(queryableDbSet);
Then use the following to run the query and turn the results into the collection of entities you publicly expose:
var list = await queryable.ToListAsync(cancellationToken);
return list
.OfType<Person>()
.Select(p => MyEntityMapper.MapToRestEntity(p));
Then you can return the list in an HttpResponseMessage as normal.
That's it, though obviously where the property names between the entities don't match or are absent on either class, there's going to be some issues, so its probably best to ensure the properties you want to include in query options are named the same in both entities.
Else, I guess you could choose to not support filters and just allow $top and $skip and impose a default order yourself. This can be achieved like so, making sure to order the queryable first, then skip, then top. Something like:
IQueryable queryable = people
.GetQueryable(operationContext)
.OrderBy(r => r.Name);
if (oDataQuery.Skip != null)
queryable = oDataQuery.Skip.ApplyTo(queryable, new System.Web.OData.Query.ODataQuerySettings());
if (oDataQuery.Top != null)
queryable = oDataQuery.Top.ApplyTo(queryable, new System.Web.OData.Query.ODataQuerySettings());
var list = await queryable.ToListAsync(operationContext.CreateToken());
return list
.OfType<Person>()
.Select(i => this.BuildPersonEntity(i));
More information:
If you simply use the non-generic ODataQueryOptions you get
Cannot create an EDM model as the action 'Get' on controller 'People'
has a return type 'System.Net.Http.HttpResponseMessage' that does not
implement IEnumerable
And other errors occur under different circumstances.
I want to create a list of complex DTO objects with data from several Entities and one non-Entity-parameter. Let's say my DTO class has constructor:
public MyDto(String entityField, String someString) {...}
and I would like to use the CriteriaBuilder.construct method to create my list by doing like this:
CriteriaBuilder builder = em.getCriteriaBuilder();
CriteriaQuery<MyDto> query = builder.createQuery(MyDto.class);
Root<MyEntity> root = query.from(MyEntity.class);
builder.construct(MyDto.class, root.get("entityField"), someString);
...
but I am not allowed to do it, because the construct method wants from me only javax.persistence.criteria.Selection arguments.
The question: is there is a way to do it similar to this (at one blow) with Criteria API? Or I need to load MyEntity objects first and go through them and create a list of DTOs (not so pretty)?
I use this approach each time I've a projection that collect fields from different entities or for privacy reason i must not return some data (for example passwords)
query.select(
builder.construct(
MyDto.class,
root.get("myfield"), // for field
cb.literal(1), // for number
cb.literal("blah blah") // for string
));
According to EclipseLink/Examples/JPA/MappingSelectionCriteria I can make some filtering on OneToOne or OneToMany relationships. To do that I have to implement DescriptorCustomizer.
My question is: Can I do some conditional filtering with this technique and how? I mean, in the example of mentioned link we can write something like this
public class ConfigureBsFilter implements DescriptorCustomizer {
public void customize(ClassDescriptor descriptor) throws Exception {
OneToManyMapping mapping = (OneToManyMapping) descriptor
.getMappingForAttributeName("bs");
ExpressionBuilder eb = new ExpressionBuilder(mapping
.getReferenceClass());
Expression fkExp = eb.getField("A_ID").equal(eb.getParameter("A_ID"));
Expression activeExp = eb.get("active").equal(true);
mapping.setSelectionCriteria(fkExp.and(activeExp));
}
}
But what if in the expression
Expression activeExp = eb.get("active").equal(true);
the "active" is not always true but have to be set at runtime by some parameter. Can I do that and how?
Looking at wiki.eclipse.org/Using_Advanced_Query_API_(ELUG) you could use a query redirector on the ForeignReferenceMapping#getSelectionQuery() so that your query redirector can dynamically clone the query and add filters as required. Passing parameters to the redirector will need to be creative though, such as storing them on the thread context or in the session's properties map.