I'm trying to use EntityGraphs or JPQL to create 1 select instead of many small (sub) selects. However, there sub entities are loaded in extra selects.
Example:
#NamedEntityGraph( name = "All",
attributeNodes = {
#NamedAttributeNode( value = "bars", subgraph = "subgraph.foobars") },
subgraphs = {
#NamedSubgraph( name = "subgraph.foobars",
attributeNodes = {
#NamedAttributeNode(value = "fooBars", subgraph = "subgraph.foobar"),
#NamedAttributeNode( "mqttEndpoints" ) } ),
#NamedSubgraph( name = "subgraph.foobar",
attributeNodes = {
#NamedAttributeNode( "name" ) } ) })
public class Foo {
#OneToMany( fetch = FetchType.LAZY )
private Set<Bar> bars = Sets.newHashSet();
}
public class Bar {
#OneToMany( fetch = FetchType.LAZY )
private Set<FooBar> fooBars = Sets.newHashSet();
}
public class FooBar {
String name;
}
#EntityGraph( value = "All", type = EntityGraph.EntityGraphType.FETCH )
Optional<Foo> findById( String id);
#Query( "SELECT DISTINCT f FROM foo f"
+ " LEFT JOIN FETCH f.bar bars "
+ " LEFT JOIN FETCH bar.fooBars foobars "
+ " WHERE t.id =:id " )
Optional<Foo> readById( String id);
Log
SELECT ...
FROM foo t1
LEFT OUTER JOIN bars t0 ON (t0.bar_id = t1.id)
SELECT name
FROM foobar
WHERE ...
If i use queryhints then it works with subselect so what is wrong?
https://www.eclipse.org/eclipselink/documentation/2.5/jpa/extensions/q_left-join-fetch.htm
With hibernate this works well.
EclipseLink you'll notice always does an extra query for any relationship mapping unless you specify a fetchJoin annotation (or query hint, or modify the mapping with customizers) that tells it what to do, and this is independent of the relationship being eager or lazily fetched. Hibernate on the other hand interprets all eager access to use join. No point debating which is better - they are situational and there are good reasons to go with either solution as a generic one.
This means that if you want a join with EclipseLink on the fly, you'll have to do more than just indicate the relationship needs to be eagerly fetched, and include in query hints. If you are going with fetch graphs to optimize things, it might be a helpful to look into the other fetch types, such as batch fetching. An extra query/statement isn't always a bad thing for performance, especially as object graphs grow. The database is going to be forced to return N*M duplicate rows of Foo data, one for each Bar and FooBar combination in the results. Depending on the data size, there will be a point where it is more efficient to get the children in separate queries.
Related
Given the following code:
var persons = context.PERSONs.Select(
x =>
new
{
personId = x.PERSON_ID,
personName = x.PERSON_NAME,
items = x.ITEMs.Select(
y =>
new
{
itemID = y.ITEM_ID,
itemName = y.ITEM_NAME,
properties = y.PROPERTies.Select(
z =>
new
{
z.PROPERTY_ID,
z.PROPERTY_NAME
}
)
}
)
}
).ToList();
How can I avoid select n + 1 problems with it? Tried .Include("ITEMs").Include("ITEMs.PROPERTies") but it didn't help. Would expect a single query with 2 left outer joins.
Note - would like a generic answer because I'm working on an OData background where it's hard to craft queries for each entity by hand
-edit-
Database: MS SQL Server
Entity Framework version: 6
Can confirm that all properties are simple mapped properties (ints and strings actullay, no functions nor computed values)
Entity relation: Transaction(#ManyToOne - eager by default) -> Account
String sql = "SELECT new com.test.Pojo(t.account, SUM(t.value)) FROM Transaction t GROUP BY t.account";
List list = entityManager.createQuery(sql).getResultList();
By default JPA using Hibernate implementation will generate 1 + n queries. The n queries are for lazy loading of the account entities.
How can I make this query eager and load everything with a single query? The sql equivalent would be something like
SELECT account.*, SUM(t.value) FROM transactions JOIN accounts on transactions.account_id = accounts.id GROUP BY account.id
, a syntax that works well on PostgreSQL. From my findings Hibernate is generating a query that justifies the lazy loading.
SELECT account.id, SUM(t.value) FROM transactions JOIN accounts on transactions.account_id = accounts.id GROUP BY account.id
Try marking the #ManyToOne field as lazy:
#ManyToOne(fetch = FetchType.LAZY)
private Account account;
And change your query using a JOIN FETCH of the account field to generate only one query with all you need, like this:
String sql = "SELECT new com.test.Pojo(acc, SUM(t.value)) "
+ "FROM Transaction t JOIN FETCH t.account acc GROUP BY acc";
UPDATE:
Sorry, you're right, the fetch attribute of #ManyToOne is not required because in Hibernate that is the default value. The JOIN FETCH isn't working, it's causing a QueryException: "Query specified join fetching, but the owner of the fetched association was not present".
I have tried with some other approaches, the most simple one that avoids doing n + 1 queries is to remove the creation of the Pojo object from your query and process the result list, manually creating the objects:
String hql = "SELECT acc, SUM(t.value)"
+ " FROM " + Transaction.class.getName() + " t"
+ " JOIN t.account acc"
+ " GROUP BY acc";
Query query = getEntityManager().createQuery(hql);
List<Pojo> pojoList = new ArrayList<>();
List<Object[]> list = query.getResultList();
for (Object[] result : list)
pojoList.add(new Pojo((Account)result[0], (BigDecimal)result[1]));
Well PostgreSQL (And any other SQL database too) will block you from using mentioned query: you have to group by all columns of account table, not by id. That is why Hibernate generates the query, grouping by ID of the account - That is what is intended to be, and then fetching the other parts. Because it cannot predict in general way, what else will be needed to be joined and grouped(!!!), and in general this could produce situation, when multiple entities with the same ID are fetched (just create a proper query and take a look at execution plan, this will be especially significant when you have OneToMany fields in your Account entity, or any other ManyToOne part of the Account entity) that is why Hibernate behaves this way.
Also, having accounts with mentioned IDs in First level cache, will force Hibernate to pick them up from that. Or IF they are rarely modified entities, you can put them in Second level cache, and hibernate will not make query to database, but rather pick them from Second level cache.
If you need to get those from database in single hint, but not use all the goodness of Hibernate, just go to pure JPA Approach based on Native queries, like this:
#NamedNativeQuery(
name = "Pojo.groupedInfo",
query = "SELECT account.*, SUM(t.value) as sum FROM transactions JOIN accounts on transactions.account_id = accounts.id GROUP BY account.id, account.etc ...",
resultClass = Pojo.class,
resultSetMapping = "Pojo.groupedInfo")
#SqlResultSetMapping(
name = "Pojo.groupedInfo",
classes = {
#ConstructorResult(
targetClass = Pojo.class,
columns = {
#ColumnResult(name = "sum", type = BigDecimal.class),
/*
* Mappings for Account part of entity.
*/
}
)
}
)
public class Pojo implements Serializable {
private BigDecimal sum;
/* .... */
public Pojo(BigDecimal sum, ...) {}
/* .... */
}
For sure this will work for you well, unless you will use the Account, fetched by this query in other entities. This will make Hibernate "mad" - the "entity", but not fetched by Hibernate...
Interesting, the described behaviour is as if t instances are returned from the actual query and t.account association in the first argument of Pojo constructor is actually navigated on t instances when marshalling results of the query (when creating Pojo instances from the result rows of the query). I am not sure if this is a bug or intended feature for constructor expressions.
But the following form of the query should work (no t.account navigation in the constructor expression, and no join fetch without the owner of the fetched association because it does not make sense to eagerly initialize something that is not actually returned from the query):
SELECT new com.test.Pojo(acc, SUM(t.value))
FROM Transaction t JOIN t.account acc
GROUP BY acc
EDIT
Very good observation by Ilya Dyoshin about the group by clause; I completely oversaw it here. To stay in the HQL world, you could simply preload all accounts with transactions before executing the query with grouping:
SELECT acc FROM Account acc
WHERE acc.id in (SELECT t.account.id FROM Transaction t)
I have the following QueryDSL query:
QCustomer customer = QCustomer.customer;
BooleanBuilder builder = new BooleanBuilder();
builder.or(customer.person.name.containsIgnoreCase(query));
builder.or(customer.company.name.containsIgnoreCase(query));
return builder;
And I expect to get results from Persons that contains the name = query and/or Companies that contains the query parameter. But I get nothing.
This is my Customer class mapping:
#OneToOne(orphanRemoval = false, optional = true, cascade = CascadeType.ALL)
private Company company;
#OneToOne(orphanRemoval = false, optional = true, cascade = CascadeType.ALL)
private Person person;
Did someone knows what I'm missing here?
I expect to get a query like this:
select o
from Customer
where o.person.name like '%:name%' or o.company.name like '%:name%'
This is the generated query:
select
count(customer0_.uid) as col_0_0_
from
Customer customer0_
cross join
Person person1_
cross join
Company company2_
where
customer0_.person_uid=person1_.uid
and customer0_.company_uid = company2_.uid
and (lower(person1_.name) like ? escape '!' or lower(company2_.name) like ? escape '!') limit ?
It uses a count because it's the first query that Spring Data use to paginate the result.
The query looks ok. Most probably you get wrong results because the implicit property based joins make the joins inner joins.
Using left joins you might get the results you need.
QPerson person = QPerson.person;
QCompany company = QCompany.company;
BooleanBuilder builder = new BooleanBuilder();
builder.or(person.name.containsIgnoreCase(str));
builder.or(company.name.containsIgnoreCase(str));
query.from(customer)
.leftJoin(customer.person, person)
.leftJoin(customer.company, company)
.where(builder);
I am in the process of converting an application that uses LINQ to SQL over to LINQ to Entities. I use a repository pattern and I have run in a problem that works in LINQ to SQL but not Entities.
In my data layer, I use LINQ statements to fill my object graph so that none of my database entities are exposed anywhere else. In this example, I have a Lookup Respository that returns a list of Categories. It looks like this:
public IQueryable<Entities.DomainModels.Category> getCategories()
{
return (from c in Categories
where !c.inactive
orderby c.categoryName
select new Entities.DomainModels.Category
{
id = c.categoryID,
category = c.categoryName,
inactive = c.inactive
});
}
Later, I want to put the categories into a sub query and it looks like this:
var d = from p in Programs
let categories = (from pc in p.Categories
join c in getCategories() on pc.categoryID equals c.id
select c)
select new
{
id = p.id,
title = p.title
categories = categories.ToList()
};
When I run this, I get the following error:
LINQ to Entities does not recognize the method 'System.Linq.IQueryable`1[Entities.DomainModels.Category] getCategories()' method, and this method cannot be translated into a store expression.
For reference, the following works though it doesn't return the data I need (it's basically a join):
var q = from p in Programs
from pc in p.Categories
join c in getCategories() on pc.categoryID equals c.id
select new
{
id = p.id,
category = c
};
I understand what the error means in concept however LINQ to SQL would make it work. I have this pattern throughout my data layer and I really want to keep it. Should this be working? If not, how can I modify it without mixing my layers.
You cant pass getCategories() to EF.
The query must be destructible to expression tree.
Calculate getCategories() first.
eg
var simpleList = getCategories().Select(id).Tolist;
then use a contains
where(t=> simpleList.Contains(t.CatId) // or the query syntax equivalent
Hy all,
I have three tables Child, Pet and Toy. Pet has a reference key the child id, and a toy has a reference key to a dog id.
I want to load all data about a Child and his pets, but i don't want to load the toy data
#OneToMany(mappedBy = "petEntity", fetch = FetchType.LAZY)
public Set<PetEntity> getPetEntitySet() {
return petEntitySet;
}
#ManyToOne(fetch = FetchType.LAZY)
#JoinColumn(name = "ChildId", insertable = false, updatable = false)
public ChildEntity getChildEntity() {
return childEntity;
}
same for set of toys.
to load data i have
List<ChildEntity> list = entityManager.createQuery(
"SELECT c FROM ChildEntity c "+
"LEFT JOIN FETCH c.petEntitySet ",
ChildEntity.class).getResultList();
return list;
but this thing loads me all data, not just the information about child and his pets.
How can i suppress the entity manager to load only the data in the table, and not to make joins when i don't want that
Thanks for your advices
i forgot to mention that in this chase it doesn't only load all data, but it returns more than just one copy of a child elemnt
The reason why it is loading more data is because you are using the keyword fetch. This will EAGERLY load whatever the child's pets.
Just remove the fetch and lazy loading will occur.
Update
I see that this is actually what you want, so just ensure equals() and hashCode() is correctly overriden in all relevant classes, as I assume you are using Set.
In this way, jpa knows how to look for duplicates
Second update
Yes you can easily use DISTINCT in your queries.
Just add DISTINCT in your select
List<ChildEntity> list = entityManager.createQuery(
"SELECT DISTINCT c FROM ChildEntity c "+
"LEFT JOIN FETCH c.petEntitySet ",
ChildEntity.class).getResultList();
return list;