Using Entity Framework Core I'm trying to map an entity containing a child collection.
Without projectTo the generated SQL includes a where clause and only fetches the entities that belong in the child collection. When I add a projectTo, the result is correct but looking at the SQL I see no where clause. It appears to load all the entities of the child type and then do a where in memory.
I've tried all sorts of mappings on the collection but nothing seems to change this behavior.
Is there a way to improve this query?
The code in question is:
var parent = _context
.Parents
.Where(a => a.FamilyId == familyId && a.Id == id)
.Include(r => r.Children)
//.ProjectTo<ParentDetailViewModel>()
.AsNoTracking() // In case projection contains sub entities
.SingleOrDefault();
Without the project to I see the following SQL in the profiler
exec sp_executesql N'SELECT [r].[Id], [r].[FieldOne], [r].[Cid], [r].[FieldTwo], [r].[ParentId]
FROM [Child] AS [r]
WHERE EXISTS (
SELECT TOP(2) 1
FROM [Parent] AS [a]
WHERE (([a].[FamilyId] = #__familyId_0) AND ([a].[Id] = #__id_1)) AND ([r].[ParentId] = [a].[Id]))
ORDER BY [r].[ParentId]',N'#__familyId_0 int,#__id_1 int',#__familyId_0=1,#__id_1=1
With project I see this. No where clause included.
SELECT [r].[ParentId], [r].[Id]
FROM [Child] AS [r]
As a general rule of thumb, I put the ProjectTo as the last thing before your query materialization. Put ProjectTo after AsNoTracking - HOWEVER, the AsNoTracking is pointless. With ProjectTo, you're skipping entities entirely, going straight from SQL to DTO, no entities involved at all.
Also, the Include is pointless. With ProjectTo/Select, EF knows exactly what entities it needs to join, because they're in the Select projection!
Apparently caused by an Entity Framework Core bug. Fixed in 1.1.0 preview
Related
I am working with Entity Framework and pretty new with it.
I have a table named: Order and table named: Products.
Each order have a lot of products.
When generating the entities I get Order object with ICollection to products.
The problem is I have a lot of products to each order (20K) and when I do
order.Products.where(......)
The EF runs a select statement only with orderId= 123 and does the rest of the where in the code.
Because I have a lot of results - the select takes a lot of time. How can I change the code - that the select in the DB will be with the where conditions?
This statement:
var prods = order.Products.Where(...);
is equivalent to:
var temps = order.Products;
var prods = temps.Where(...);
Unlike Where(...), which returns an IQueryable, order.Products triggers a lazy loading, which produces an ICollection and will be executed immediately, not delayed. So it's this order.Products part that generates the select statement you see. It fetches all the products belonging to that order into memory. Then the Where(...) part is executed in memory, hence the bad performance.
To avoid this, you should use order.Products only if you really want all the products on an order. If you want only a subset of them, do something like the following:
ctx.Products.Where(prod => prod.Order.Id == order.Id && ...)
Note that ctx is the database context, not the order object.
If you think that the prod.Order.Id == order.Id clause above looks a little dirty, here's a purer but longer alternative:
ctx.Entry(order).Collection(ord => ord.Products).Query().Where(...)
which produces exactly the same SQL query.
We use Entity Framework and this leads to big performance problems whenever we use Count() on child collections of database entities.
As workaround I've used joins of root collections of data context. Then the resulting sql query uses the desired COUNT(*). But this solution is really ugly.
The slow query is:
var booked = erf.Sessions.All(s => s.Exams.All(e => e.Candidates.Count() >= e.CandidatesExpected))
If by "child collections" you mean navigation properties of type ICollection<T> defined in your entities, then it's Linq-to-Entities (not Linq-to-Sql, as you specified). Thus your Count() method is just an extension method defined in Enumerable class, which is executed on entities already materialized into memory. To get results you are expecting, you need to use Count() on DbSet queries.
I am using JPA2 with it's Criteria API to select my entities from the database. The implementation is OpenJPA on WebSphere Application Server. All my entities are modeled with Fetchtype=Lazy.
I select an entity with some criteria from the database and want to load all nested data from sub-tables at once.
If I have a datamodel where table A is joined oneToMany to table B, I can use a Fetch-clause in my criteria query:
CriteriaBuilder cb = entityManager.getCriteriaBuilder();
CriteriaQuery<A> cq = cb.createQuery(A.class);
Root<A> root = cq.from(A.class);
Fetch<A,B> fetch = root.fetch(A_.elementsOfB, JoinType.LEFT);
This works fine. I get an element A and all of its elements of B are filled correctly.
Now table B has a oneToMany-relationship to table C and I want to load them too. So I add the following statement to my query:
Fetch<B,C> fetch2 = fetch.fetch(B_.elementsOfC, JoinType.LEFT);
But this wont do anything.
Does anybody know how to fetch multi level entities in one query?
It does not work with JPQL and there is no way to make it work in CriteriaQueries either. Specification limits fetched entities to the ones in that are referenced directly from the returned entity:
About fetch join with CriteriaQuery:
An association or attribute referenced by the fetch method must be
referenced from an entity or embeddable that is returned as the result
of the query.
About fetch join in JPQL:
The association referenced by the right side of the FETCH JOIN clause
must be an association or ele ment collection that is referenced from
an entity or embeddable that is returned as a result of the query.
Same limitation is also told in OpenJPA documentation.
For what is worth. I do this all the time and it works just fine.
Several points:
I'm using jpa 2.1, but I'm almost sure it used to work in jpa 2.0 as well.
I'm using the criteria api, and I know some things work diferent in jpql. So don't think it works some way or doesn't work because that's what happens in jpql. Most often they do behave in the same way, but not always.
(Also i'm using plain criteria api, no querydsl or anything. Sometimes it makes a difference)
My associations tend to be SINGULAR_ATTRIBUTE. So maybe that's the problem here. Try a test with the joins in reverse "c.fetch(b).fetch(a)" and see if that works. I know it's not the same, but just to see if it gives you any hint. I'm almost sure I have done it with onetomany left fetch joins too, though.
Yep. I just checked and found it: root.fetch("targets", LEFT).fetch("destinations", LEFT).fetch("internal", LEFT)
This has been working without problems for months, maybe more than a year.
I just run a test and it generates this query:
select -- all fields from all tables
...
from agreement a
left outer join target t on a.id = t.agreement_id
left outer join destination d on t.id = d.target_id
left outer join internal i on d.id = i.destination_id
And returns all rows with all associations with all fields.
Maybe the problem is a different thing. You just say "it wont do anyhting". I don't know if it throws an exception or what, but maybe it executes the query properly but doesn't return the rows you expect because of some conditions or something like that.
You could design a view in the DB joining tables b and c, create the entity and fetchit insted of the original entity.
One of the few valid complaints I hear about EF4 vis-a-vis NHibernate is that EF4 is poor at handling lazily loaded collections. For example, on a lazily-loaded collection, if I say:
if (MyAccount.Orders.Count() > 0) ;
EF will pull the whole collection down (if it's not already), while NH will be smart enough to issue a select count(*)
NH also has some nice batch fetching to help with the select n + 1 problem. As I understand it, the closest EF4 can come to this is with the Include method.
Has the EF team let slip any indication that this will be fixed in their next iteration? I know they're hard at work on POCO, but this seems like it would be a popular fix.
What you describe is not N+1 problem. The example of N+1 problem is here. N+1 means that you execute N+1 selects instead of one (or two). In your example it would most probably mean:
// Lazy loads all N Orders in single select
foreach(var order in MyAccount.Orders)
{
// Lazy loads all Items for single order => executed N times
foreach(var orderItem in order.Items)
{
...
}
}
This is easily solved by:
// Eager load all Orders and their items in single query
foreach(var order in context.Accounts.Include("Orders.Items").Where(...))
{
...
}
Your example looks valid to me. You have collection which exposes IEnumerable and you execute Count operation on it. Collection is lazy loaded and count is executed in memory. The ability for translation Linq query to SQL is available only on IQueryable with expression trees representing the query. But IQueryable represents query = each access means new execution in DB so for example checking Count in loop will execute a DB query in each iteration.
So it is more about implementation of dynamic proxy.
Counting related entities without loading them will is already possible in Code-first CTP5 (final release will be called EF 4.1) when using DbContext instead of ObjectContext but not by direct interaction with collection. You will have to use something like:
int count = context.Entry(myAccount).Collection(a => a.Orders).Query().Count();
Query method returns prepared IQueryable which is probably what EF runs if you use lazy loading but you can further modify query - here I used Count.
I've created a function import that returns the results of a stored proceedure as one of my entities. however I can't seem to traverse my through navigation properties to access the data in other entities. I know that you can use include() for objectQueries but can't find anything that will force the EF to load my relations for entity results of function imports.
Any ideas??
Thanks in advance.
This is not possible in EF 1.0
The reason is that EF will consider stored procedure values to be just values and not navigation properites.
For example, Employee entity has multiple Order entities. In Order you have a property called EmployeeID. When the database fills your query using include statements, it creates 1 projection query in SQL to populate all of the Order data that a particular Employee could have.
So if I said
var employee = context.Employees.Include("Orders").Where(e => e.ID == 1).First();
var orders = employee.Orders;
The SQL for the first query will create a projection query which will contain orders where the EmployeeID = 1.
Now when your stored procedure runs, this can do any code behind the scenes (in otherwords it can return any set of data). So when SQL runs the stored procedure, it just runs the code in that stored procedure and does not have any knowledge that EmployeeID on Order is an FK to that property. Additionally, if your stored procedure returns an Employee entity, then you are looking at another scenario where you will not even have an OrderID to pursue.
To work around this though, you can setup your query in EF using Include statements that can mirror any stored procedure. If you use the proper mix of .Select and .Include statements you should be able to do the same thing.