I am using JPA2 with it's Criteria API to select my entities from the database. The implementation is OpenJPA on WebSphere Application Server. All my entities are modeled with Fetchtype=Lazy.
I select an entity with some criteria from the database and want to load all nested data from sub-tables at once.
If I have a datamodel where table A is joined oneToMany to table B, I can use a Fetch-clause in my criteria query:
CriteriaBuilder cb = entityManager.getCriteriaBuilder();
CriteriaQuery<A> cq = cb.createQuery(A.class);
Root<A> root = cq.from(A.class);
Fetch<A,B> fetch = root.fetch(A_.elementsOfB, JoinType.LEFT);
This works fine. I get an element A and all of its elements of B are filled correctly.
Now table B has a oneToMany-relationship to table C and I want to load them too. So I add the following statement to my query:
Fetch<B,C> fetch2 = fetch.fetch(B_.elementsOfC, JoinType.LEFT);
But this wont do anything.
Does anybody know how to fetch multi level entities in one query?
It does not work with JPQL and there is no way to make it work in CriteriaQueries either. Specification limits fetched entities to the ones in that are referenced directly from the returned entity:
About fetch join with CriteriaQuery:
An association or attribute referenced by the fetch method must be
referenced from an entity or embeddable that is returned as the result
of the query.
About fetch join in JPQL:
The association referenced by the right side of the FETCH JOIN clause
must be an association or ele ment collection that is referenced from
an entity or embeddable that is returned as a result of the query.
Same limitation is also told in OpenJPA documentation.
For what is worth. I do this all the time and it works just fine.
Several points:
I'm using jpa 2.1, but I'm almost sure it used to work in jpa 2.0 as well.
I'm using the criteria api, and I know some things work diferent in jpql. So don't think it works some way or doesn't work because that's what happens in jpql. Most often they do behave in the same way, but not always.
(Also i'm using plain criteria api, no querydsl or anything. Sometimes it makes a difference)
My associations tend to be SINGULAR_ATTRIBUTE. So maybe that's the problem here. Try a test with the joins in reverse "c.fetch(b).fetch(a)" and see if that works. I know it's not the same, but just to see if it gives you any hint. I'm almost sure I have done it with onetomany left fetch joins too, though.
Yep. I just checked and found it: root.fetch("targets", LEFT).fetch("destinations", LEFT).fetch("internal", LEFT)
This has been working without problems for months, maybe more than a year.
I just run a test and it generates this query:
select -- all fields from all tables
...
from agreement a
left outer join target t on a.id = t.agreement_id
left outer join destination d on t.id = d.target_id
left outer join internal i on d.id = i.destination_id
And returns all rows with all associations with all fields.
Maybe the problem is a different thing. You just say "it wont do anyhting". I don't know if it throws an exception or what, but maybe it executes the query properly but doesn't return the rows you expect because of some conditions or something like that.
You could design a view in the DB joining tables b and c, create the entity and fetchit insted of the original entity.
Related
I have an entity model where I need to dig down two collections to get the data required on the screen - in Hibernate that triggered first an exception when I created a JPA join fetch query until I changed the collections to sets. It is still a cartesian product though. I am trying to write a JPARepository function to execute the following query now, but only with specific fields, because the resulting query returns more than 100 fields of which I need less than 10.
SELECT p FROM Person p
JOIN FETCH p.responsibilities r
JOIN FETCH r.configurations c
JOIN FETCH c.type
WHERE p.id = :id
My JPA Repository method is already using projection interfaces but I think because I am using the #Query annotation it does not optimize the query. When I tried it as a plain, typed findById() method with projection interfaces it did execute a load of subqueries which costs much more time.
Any recommendations
With lazy loading used by default, I know that you should call .Include() on your Entity Framework entities to pull in associated entities you want in your queries to reduce the number of calls to the db if you're calling LINQ methods on your entities. If you don't, you run the risk of repeated database calls for each row (the N+1 problem)
Can someone confirm that if I write a canonical LINQ query, with the joins defined explicitly, that we guard against N+1?
from x in _context.tblOrder
join y in _context.tblCustomer equals y.id = x.customerId
select x
Is there any way N+1 could creep in when we're loading in all the required entities with joins?
EDIT
As background, someone asked how junior developers could guard against N+1. I mentioned the simplest way would be to write out your queries and define your joins, I want confirmation that was I indicated was 100% accurate.
If what you are really asking is
Will this query hit the database once?
Then the answer is yes. LINQ to EF translates your expression to raw SQL and only when you evaluate the query will it send anything to the database e.g. ToList()/foreach/for etc. and once that query is sent nothing else is unless you explicitly tell it otherwise.
Your LINQ statement could be simplified using a Lambda expression e.g.
_context.tblOrder.Include("Customer").ToList();
This would give you all the order details, including all related customer details, in a single database trip.
Just because you specify tables in a join doesn't mean that you can't run into a n+1 issue when you iterate over the values. Consider the following extension to your query:
var query = from o in Orders
join c in Customers on o.CustomerID equals c.CustomerID
select o;
foreach (var o in query)
{
Console.WriteLine(String.Format("{0}: {1}", o.OrderDate, o.Employee.FirstName));
}
In this case, each time you navigate through the order's Employee object, the employee is fetched from the database for that order. If you wanted to avoid the issue, you could project the values you want in the select clause:
var query = from o in Orders
join c in Customers on o.CustomerID equals c.CustomerID
select new {o.OrderDate, o.Employee.FirstName};
foreach (var o in query)
{
Console.WriteLine(String.Format("{0}: {1}", o.OrderDate, o.FirstName));
}
Note, in this case, you don't even need the join as you can just use the navigation properties instead. Of course, if you don't allow navigation properties in your entities and rely only on joins, you can avoid the n+1 situation, but that is not a very OOP way of solving the problem.
I think you would be safe guaranteeing against n+1 if you only return anonymous types from your queries, but that would be rather restrictive as well.
The best option is to make sure to profile your application's generated SQL and know precisely when and why you are hitting the database. I discuss some of the profilers available at http://www.thinqlinq.com/Post.aspx/Title/LINQ-to-Database-Performance-hints.
I don't think I will ever fully understand fetch joins.
I have a query where I'm attempting to eagerly "inflate" references down two levels.
That is, my A has an optional Collection of Bs, and each B has either 0 or 1 C. The size of the B collection is known to be small (10-20 tops). I'd like to prefetch this graph.
A's B relationship is marked as FetchType.LAZY and is optional. B's relationship to C is also optional and FetchType.LAZY.
I was hoping I could do:
SELECT a
FROM A a
LEFT JOIN FETCH a.bs // look, no alias; JPQL forbids it
LEFT JOIN a.bs b // "repeated" join necessary since you can't alias fetch joins
LEFT JOIN FETCH b.c // this doesn't seem to do anything
WHERE a.id = :id
When I run this, I see that As B collection is indeed fetched (I see a LEFT JOIN in the SQL referencing the table to which B is mapped).
However, I see no such evidence that C's table is fetched.
How can I prefetch all Cs and all Bs and all Cs that are "reachable" from a given A? I can't see any way to do this.
The JPA spec does not allow aliasing a fetch join, but some JPA providers do.
EclipseLink does as of 2.4. EclipseLink also allow nested join fetch using the dot notation (i.e. "JOIN FETCH a.bs.c"), and supports a query hint "eclipselink.join-fetch" that allows nested joins (you can specify multiple hints of the same hint name).
In general you need to be careful when using an alias on a fetch join, as you can affect the data that is returned.
See,
http://java-persistence-performance.blogspot.com/2012/04/objects-vs-data-and-filtering-join.html
I'm using Hibernate (and this may be specific to it) and I've had success with this:
SELECT DISTINCT a, b
FROM A a
LEFT JOIN a.bs b
LEFT JOIN FETCH a.bs
LEFT JOIN FETCH b.c
WHERE a.id = :id
(Note the b in the select list).
This was the only way I found this would work for me, note that this returns Object[] for me and I then filter it in code like so:
(List<A>) q.getResultList().stream().map(pair -> (A) (((Object[])pair)[0])).distinct().collect(Collectors.toList());
Not exactly JPQL, but you can achieve that in pure JPA with Criteria queries:
CriteriaBuilder cb = entityManager.getCriteriaBuilder();
CriteriaQuery<MyEntity> q = cb.createQuery(MyEntity.class);
Root<MyEntity> root = q.from(MyEntity.class);
q.select(root);
Fetch bsFetch = root.fetch("b", JoinType.LEFT); //fetch b, property of MyEntity and hold fetch object
bsFetch.fetch("c", JoinType.LEFT); //fetch c, property of b
Support for this kind of nested fetch is vendor-specific (as JPA doesn't require them to do so), but both eclipselink and hibernate do support it, and this way your code remains vendor independant.
JPA does not allow nested join fetches, nor allow an alias on a join
fetch, so this is probably JPA provider specific.
In EclipseLink, you can specify a query hint to perform nested join
fetches.
You can't make it recursive in JPQL though, you could go only at best
n levels. In EclipseLink you could use #JoinFetch or #BatchFetch on
the mapping to make the querying recursive.
See,
http://java-persistence-performance.blogspot.com/2010/08/batch-fetching-optimizing-object-graph.html
Source: http://www.coderanch.com/t/570828/ORM/databases/Recursive-fetch-join-recursively-fetching
Ok, lets say you have two tables: Order and OrderLine and for some reason they do not have a foreign key relationship in the database (it's an example, live with it). Now, you want to join these two tables using Entity Framework and you cook up something like this:
using (var model = new Model())
{
var orders = from order in model.Order
join orderline in model.OrderLine on order.Id equals orderline.OrderId into orderlines
from ol in orderlines.DefaultIfEmpty()
select new {order = order, orderlines = orderlines};
}
Now, the above will produce orders and orderlines, left-joined and all, but it has numerous issues:
It's plain ugly
It returns an anonymous type
It returns multiple instances of the same order and I you have to do Distinct() on the client side because orders.Distinct() fails.
What I am looking for is a solution which is:
Pretty
Returns a statically well-known type instead of the anonymous type (I tried to project the query result, but I got into problems with the OrderLines)
Runs Distinct on the server side
Anyone?
Even if the database tables do not have a foreign key relationship setup, you can configure Entity Framework as if they do.
Add an OrderDetails navigation property to your Order class and then just query Orders.
Greetings,
Considering the Northwind sample tables Customers, Orders, and OrderDetails I would like to eager load the related entities corresponding to the tables mentioned above and yet I need ot order the child entities on the database before fetching entities.
Basic case:
var someQueryable = from customer in northwindContext.Customers.Include("Orders.OrderDetails")
select customer;
but I also need to sort Orders and OrderDetails on the database side (before fetching those entities into memory) with respect to some random column on those tables. Is it possible without some projection, like it is in T-SQL? It doesn't matter whether the solution uses e-SQL or LINQ to Entities. I searched the web but I wasn't satisfied with the answers I found since they mainly involve projecting data to some anonymous type and then re-query that anonymous type to get the child entities in the order you like. Also using CreateSourceQuery() doesn't seem to be an option for me since I need to get the data as it is on the database side, with eager loading but just by ordering child entities. That is I want to do the "ORDER BY" before executing any query and then fetch the entities in the order I'd like. Thanks in advance for any guidance. As a personal note, please excuse the direct language since I am kinda pissed at Microsoft for releasing the EF in such an immature shape even compared to Linq to SQL (which they seem to be getting away slowly). I hope this EF thingie will get much better and without significant bugs in the release version of .NET FX 4.0.
Actually I have Tip that addresses exactly this issue.
Sorting of related entities is not 'supported', but using the projection approach Craig shows AND relying on something called 'Relationship Fixup' you can get something very similar working:
If you do this:
var projection = from c in ctx.Customers
select new {
Customer = c,
Orders = c.Orders.OrderByDescending(
o => o.OrderDate
)
};
foreach(var anon in projection )
{
anon.Orders //is sorted (because of the projection)
anon.Customer.Orders // is sorted too! because of relationship fixup
}
Which means if you do this:
var customers = projection.AsEnumerable().Select(x => x.Customer);
you will have customers that have sorted orders!
See the tip for more info.
Hope this helps
Alex
You are confusing two different problems. The first is how to materialize entities in the database, the second is how to retrieve an ordered list. The EntityCollection type is not an ordered list. In your example, customer.Orders is an EntityCollection.
On the other hand, if you want to get a list in a particular order, you can certainly do that; it just can't be in a property of type EntityCollection. For example:
from c in northwindContext.Customers
orderby c.SomeField
select new {
Name = c.Name,
Orders = from o in c.Orders
orderby c.SomeField
select new {
SomeField = c.SomeField
}
}
Note that there is no call to Include. Because I am projecting, it is unnecessary.
The Entity Framework may not work in the way you expect, coming from a LINQ to SQL background, but it does work. Be careful about condemning it before you understand it; deciding that it doesn't work will prevent you from learning how it does work.
Thank you both. I understand that I can use projection to achieve what I wanted but I thought there might be an easy way to do it since in T-SQL world it's perfectly possible with a few nested queries (or joins) and order bys. On the other hand seperation of concerns sounds reasonable and we are in the entity domain now so I will use the way you two both recommended though I have to admit this is easier and cleaner to achieve in LINQ to SQL by using AssociateWith.
Kind regards.