Exposing my EF models to an API always seemed wrong. I'd like my API to return a custom entity model to the caller but use EF on the back.
So I may have PersonRestEntity and a controller for CRUD ops against that and a Person EF code-first entity behind in and map values.
When I do this, I can no longer use the following to allow ~/people?$top=10 etc. in the URL
[EnableQuery]
public IQueryable<Person> Get(ODataQueryOptions<Person> query) { ... }
Because that exposes Person which is private DB implementation.
How can I have my cake and eat it?
I found a way. The trick is not to just return the IQueryable from the controller, because you need to materialise the query first. This doesn't mean materialising the whole set into RAM, the query is still run at the database, but by explicitly applying the query and materialising the results you can return mapped entities thereafter.
Define this action, specifying the DbSet entity type:
public async Task<HttpResponseMessage> Get(ODataQueryOptions<Person> oDataQuery)
And then apply the query manually to the DbSet<Person> like so:
var queryable = oDataQuery.ApplyTo(queryableDbSet);
Then use the following to run the query and turn the results into the collection of entities you publicly expose:
var list = await queryable.ToListAsync(cancellationToken);
return list
.OfType<Person>()
.Select(p => MyEntityMapper.MapToRestEntity(p));
Then you can return the list in an HttpResponseMessage as normal.
That's it, though obviously where the property names between the entities don't match or are absent on either class, there's going to be some issues, so its probably best to ensure the properties you want to include in query options are named the same in both entities.
Else, I guess you could choose to not support filters and just allow $top and $skip and impose a default order yourself. This can be achieved like so, making sure to order the queryable first, then skip, then top. Something like:
IQueryable queryable = people
.GetQueryable(operationContext)
.OrderBy(r => r.Name);
if (oDataQuery.Skip != null)
queryable = oDataQuery.Skip.ApplyTo(queryable, new System.Web.OData.Query.ODataQuerySettings());
if (oDataQuery.Top != null)
queryable = oDataQuery.Top.ApplyTo(queryable, new System.Web.OData.Query.ODataQuerySettings());
var list = await queryable.ToListAsync(operationContext.CreateToken());
return list
.OfType<Person>()
.Select(i => this.BuildPersonEntity(i));
More information:
If you simply use the non-generic ODataQueryOptions you get
Cannot create an EDM model as the action 'Get' on controller 'People'
has a return type 'System.Net.Http.HttpResponseMessage' that does not
implement IEnumerable
And other errors occur under different circumstances.
Related
i have problem with translated query, ToList(), AsEnumerable etc.
I need construct or create query which is shared.
Branches -> Customer -> some collection -> some collection
Customer -> some collection -> some collection.
Do you help me how is the best thingh how to do it and share the query.
i access to repository via graphql use projection etc.
public IQueryable<CustomerTableGraphQL> BranchTableReportTest(DateTime actualTime, long userId)
{
var r =
(
from b in _dbContext.Branches
let t = Customers(b.Id).ToList()
select new CustomerTableGraphQL
{
Id = b.Id,
Name = b.Name,
Children =
(
from c in t
select new CustomerTableGraphQL
{
Id = c.Id,
Name = c.Name
}
)
.AsEnumerable()
}
);
return r;
}
public IQueryable<Customer> Customers(long branchId) =>
_dbContext.Customers.Where(x => x.BranchId.Value == branchId).ToList().AsQueryable();
Some example how to doit and share iquearable between query
Using ToList / AsEnumerable etc. entirely defeats the potential benefits of using IQueryable. If your code needs to do this rather than return an IQueryable<TEntity> then you should be returning IEnumerable<TResult> where TResult is whatever entity or DTO/ViewModel you want to return.
An example of an IQueryable<TEntity> repository pattern would be something like this:
public IQueryable<Customer> GetCustomersByBranch(long branchId) =>
_dbContext.Customers.Where(x => x.BranchId.Value == branchId);
Normally I wouldn't really even have a repository method for that, I'd just use:
public IQueryable<Customer> GetCustomers() =>
_dbContext.Customers.AsQueryable();
... as the "per branch" is simple enough for the consumer to request without adding methods for every possible filter criteria. The AsQueryable in this case is only needed because I want to ensure the result matches the IQueryable type casting. When your expression has a Where clause then this is automatically interpreted as being an IQueryable result.
So a caller calling the Repository's "GetCustomers()" method would look like:
// get customer details for our branch.
var customers = _Repository.GetCustomers()
.Where(x => x.BranchId == branchId)
.OrderBy(x => x.LastName)
.ThenBy(x => x.FirstName)
.Select(x => new CustomerSummaryViewModel
{
CustomerId = x.Id,
FirstName = x.FirstName,
LastName = x.LastName,
// ...
}).Skip(pageNumber * pageSize)
.Take(pageSize)
.ToList();
In this example the repository exposes a base query to fetch data, but without executing/materializing anything. The consumer of that call is then free to:
Filter the data by branch,
Sort the data,
Project the data down to a desired view model
Paginate the results
... before the query is actually run. This pulls just that page of data needed to populate the VM after filters and sorts as part of the query. That Repository method can serve many different calls without needing parameters, code, or dedicated methods to do all of that.
Repositories returning IQueryable that just expose DbSets aren't really that useful. The only purpose they might provide is making unit testing a bit easier as Mocking the repository is simpler than mocking a DbContext & DbSets. Where the Repository pattern does start to help is in enforcing standardized rules/filters on data. Examples like soft delete flags or multi-tenant systems where rows might belong to different clients so a user should only ever search/pull across one tenant's data. This also extends to details like authorization checks before data is returned. Some of this can be managed by things like global query filters but wherever there are common rules to enforce about what data is able to be retrieved, the Repository can serve as a boundary to ensure those rules are applied consistently. For example with a soft-delete check:
public IQueryable<Customer> GetCustomers(bool includeInactive = false)
{
var query = _context.Customers.AsQueryable();
if (!includeInactive)
query = query.Where(x => x.IsActive);
return query;
}
A repository can be given a dependency for locating the current logged in user and retrieving their roles, tenant information, etc. then use that to ensure that:
a user is logged in.
The only data retrieved is available to that user.
An appropriate exception is raised if specific data is requested that this user should never be able to access.
An IQueryable repository does require a Unit of Work scope pattern to work efficiently within an application. IQueryable queries do no execute until something like a ToList or Single, Any, Count, etc. are called. This means that the caller of the repository ultimately needs to be managing the scope of the DbContext that the repository is using, and this sometimes rubs developers the wrong way because they feel the Repository should be a layer of abstraction between the callers (Services, Controllers, etc.) and the data access "layer". (EF) To have that abstraction means adding a lot of complexity that ultimately has to conform to the rules of EF (or even more complexity to avoid that) or significantly hamper performance. In cases where there is a clear need or benefit to tightly standardizing a common API-like approach for a Repository that all systems will conform to, then an IQueryable pattern is not recommended over a general IEnumerable typed result. The benefit of IQueryable is flexibility and performance. Consumers decide and optimize for how the data coming from the Repository is consumed. This flexibility extends to cover both synchronous and asynchronous use cases.
EF Core will translate only inlined query code. This query will work:
public IQueryable<CustomerTableGraphQL> BranchTableReportTest(DateTime actualTime, long userId)
{
var r =
(
from b in _dbContext.Branches
select new CustomerTableGraphQL
{
Id = b.Id,
Name = b.Name,
Children =
(
from c in _dbContext.Customers
where c.BranchId == b.Id
select new CustomerTableGraphQL
{
Id = c.Id,
Name = c.Name
}
)
.AsEnumerable()
}
);
return r;
}
If you plan to reuse query parts, you have to deal with LINQKit and its ExpandableAttribute (will show sample on request)
I have a service that passes in parameters for how much I want to include for navigation properties. Based upon the boolean args it concatenates an entity list to include each required foreign entity.
At runtime I want to include either no navigation entities or many.
What I can't do is daisy chain with .Include().Include as I don't know which and how many to include based around passed in args.
I want to achieve this, but I don't seem to be able to pass in a comma separated entity list. Any ideas?
var res = db.Entity.Include(entityListCommaSeparated).Where(_=>_.ID == ID).FirstOrDefault();
This looks like a repository pattern, and generally gets messy if you want to try and "hide" EF / the DbContext from calling code.
A couple options you can consider:
Down the complexity rabit hole: use a params Expression<Func<TEntity, object>>[] includes in your applicable repository methods, and then be prepared to also pass OrderBy expressions, as well as pagination values when you want to return multiple entities.
THrough the simplicity mirror: Embrace IQueryable as a return type and let the consumers handle Includes, OrderBy's, Counts/Any/Skip/Take/First/ToList, and .Select() as they need.
Option 1:
public Order GetById(int id, params Expression<Func<Order, object>>[] includes)
{
var query = db.Orders.Where(x => x.ID == id);
// This part can be moved into an extension method or a base repository method.
if(includes.Any)
includes.Aggregate(query, (current, include) =>
{
current.Include(include);
}
// Don't use .FirstOrDefault() If you intend for 1 record to be returned, use .Single(). If it really is optional to find, .SingleOrDefault()
return query.Single();
}
//ToDo
public IEnumerable<Order> GetOrders(/* criteria?, includes?, order by?, (ascending/descending) pagination? */)
{ }
// or
public IEnumerable<Order> GetOrdersByCustomer(/* includes?, order by?, (ascending/descending) pagination? */)
{ }
// plus..
public IEnumerable<Order> GetOrdersByDate(/* includes?, order by?, (ascending/descending) pagination? */)
{ }
public bool CustomerHasOrders(int customerId)
{ }
public bool OrderExists(int id)
{ }
public int OrdersOnDate(DateTime date)
{ }
// etc. etc. etc.
Keep in mind this doesn't handle custom order by clauses, and the same will be needed for methods that are returning lists of entities. Your repository is also going to need to expose methods for .Any() (DoesExist) because everyone loves checking for #null on every return. :) Also .Count().
Option 2:
public IQueryable<Order> GetById(int id)
{
return db.Orders.Where(x => x.ID == id);
}
public IQueryable<Order> GetOrders()
{
return db.Orders.AsQueryable();
}
Callers can grok Linq and .Include() what they want before calling .Single(), or do a .Any().. They may not need the entire entity graph so they can .Select() from the entity and related entities without .Include() to compose and execute a more efficient query to populate a ViewModel / DTO. GetById might be used in a number of places so we can reduce duplication and support it in the repository. We don't need all of the filter scenarios etc, callers can call GetOrders and then filter as they see fit.
Why bother with a repository if it just returns DBSets?
Centralize low-level data filtering. For instance if you use Soft Deletes (IsActive) or are running multi-tenant, or explicit authorization. These common rules can be centralized at the repository level rather than having to remembered everywhere a DbSet is touched.
Testing is simpler. While you can mock a DbContext, or point it at an in-memory database, mocking a repository returning IQueryable is simpler. (Just populate a List<TEntity> and return .AsQueryable().
Repositories handle Create and Delete. Create to serve as a factory to ensure that all required data and relationships are established for a viable entity. Delete to handle soft-delete scenarios, cascades/audits etc. beyond what the DB handles behind the scenes.
I'm using .net web Api with Entity Framework. Its really nice that you can just do
[EnableQuery]
public IQueryable<Dtos.MyDto> Get()
{
return dbContext.MyEntity.Select(m => new MyDto
{
Name = m.Name
});
}
And you get odata applying to the Iqueryable, note also returning a projected dto.
But that select is a expression and so its being turned to into sql. Now in the above case that's fine. But what if I need to do some "complex" formatting on the return dto, its going to start having issues as SQL wont be able to do it.
Is it possible to create an IQueryable Wrapper?
QWrapper<TEntity,TDo>(dbcontext.MyEntity, Func<TEntity,TDo> dtoCreator)
it implements IQueryable so we still return it allowing webapi to apply any odata but the Func gets called once EF completes thus allowing 'any' .net code to be called as its not converting to SQL.
I don't want to do dbContext.MyEntity.ToList().Select(...).ToQueryable() or whatever as that will always return the entire table from the db.
Thoughts?
since you query already returns the data you expected, how about adding .Select(s=>new MyEntity(){ Name=s.Name }) for returning them as OData response? like:
return dbContext.MyEntity.Select(m => new MyDto
{
Name = m.Name
}).Select(s=>new MyEntity(){ Name=s.Name });
Does anyone have any idea how to limit result set of EntityFramework permanently? I'm speaking about something like this Conditional Mapping. This is exactly what I want to achieve with one exception: I want to do this programmatically. That's because condition value will be passed to EF only on context creation. Beside I don't want this column to disappear from mapping.
I know how to achieve this with EF2.0 and reflection. I was using CreateQuery() method to generate my own ObjectQuery. CreateQuery() allows to inject my own ESQL query with additional condition e.g. WHERE TABLE.ClientID == value.
Problem with EF40 is that there is no more ObjectQuery but only ObjectSet and CreateQuery() is not used. I have no idea how to inject my own ESQL query.
The reason why I want to limit result sets is that I want to separate clients data from each other. This separation should be done automatically inside context so that programmers will not have to add condition .Where(x => x.ClientID == 5) to each individual query.
Maybe my approach is completely bad — but I don't know any alternative.
You don't need reflection for this. You can simply use class inherited from ObjectContext or create custom implementation of UnitOfWork and Repositories which will wrap this functionality in better way (upper layer has access only to UnitOfWork and Repositories which do not expose EF context).
Simple example of object context:
public class CustomContext : ObjectContext
{
private ObjectSet<MyObject> _myObjectsSet;
private int _clientId;
public CustomContext(string connectionString, int clientId)
: base(connectionString)
{
_myObjectSet = CreateObjectSet<MyObject>();
_clientId = clientId;
}
public IQueryable<MyObject> MyObjectQuery
{
get
{
return _myObjectsSet.Where(o => o.ClientId == _clientId);
}
}
}
I use DTO's to map between my Business and Entity Framework layer via the Repository Pattern.
A Standard call would look like
public IClassDTO Fetch(Guid id)
{
var query = from s in _db.Base.OfType<Class>()
where s.ID == id
select s;
return query.First();
}
Now I wish to pass in filtering criteria from the business layer so I tried
public IEnumerable<IClassDTO> FetchAll(ISpecification<IClassDTO> whereclause)
{
var query = _db.Base.OfType<Class>()
.AsExpandable()
.Where(whereclause.EvalPredicate);
return query.ToList().Cast<IClassDTO>();
}
The Call from the business layer would be something like
Specification<IClassDTO> school =
new Specification<IClassDTO>(s => s.School.ID == _schoolGuid);
IEnumerable<IClassDTO> testclasses = _db.FetchAll(school);
The problem I am having is that the .Where clause on the EF query cannot be inferred from the usage. If I use concrete types in the Expression then it works find but I do not want to expose my business layer to EF directly.
Try making FetchAll into a generic on a class instead, like this:-
public IEnumerable<T> FetchAll<T> (Expression<Func<T,bool>> wherePredicate)
where T:IClassDTO //not actually needed
{
var query = _db.Base.OfType<T>()
.AsExpandable()
.Where(wherePredicate);
return query;
}
pass in school.Evalpredicate instead. FetchAll doesn't appear to need to know about the whole specification, it just needs the predicate, right? If you need to cast it to IClassDTO, do that after you have the results in a List.