The following code gives an error
List<List<Consumption>> children = new ArrayList<List<Consumption>>();
try {
for(DrinkType drinkType : groups){
QueryBuilder<Consumption, ?> qb = getHelper()
.getConsumptionDao().queryBuilder();
qb.groupBy(Consumption.COLUMN_DRINK_ID).having("MIN("+Consumption.COLUMN_PRICE+")").orderBy(Consumption.COLUMN_PRICE, true);
QueryBuilder<Drink, Integer> qb2 = getHelper().getDrinkDao().queryBuilder();
qb2.where().eq(Drink.COLUMN_DRINKTYPE, drinkType.getFlag());
qb.join(qb2);
Log.d("Query", qb.prepareStatementString());
children.add(qb.query());
}
} catch (SQLException e) {
e.printStackTrace();
}
It seems that the QueryBuilder puts the HAVING clause behind the ORDER BY clause instead of in front.
java.sql.SQLException: Problems executing Android query:
SELECT `consumption`.* FROM `consumption` INNER JOIN `drink` ON
`consumption`.`drink_id` = `drink`.`_id` WHERE `drink`.`drink_type` = 'P'
GROUP BY `consumption`.`drink_id` ORDER BY `consumption`.`price` HAVING MIN(price)
I tried to work around this by doing a raw query instead, but if i use the following query
SELECT co.* FROM consumption co INNER JOIN drink dr ON dr._id = co.drink_id
WHERE dr.drink_type = 'P' GROUP BY co.drink_id HAVING MIN(price) ORDER BY price
This gives back a different result than if i perform the same query directly on my sqlite database using SQLite Manager. I think ORMlite might have a problem when both HAVING and ORDER BY are used in the same query.
Is there a possible workaround so that I can get this query to work?
Related
Entity relation: Transaction(#ManyToOne - eager by default) -> Account
String sql = "SELECT new com.test.Pojo(t.account, SUM(t.value)) FROM Transaction t GROUP BY t.account";
List list = entityManager.createQuery(sql).getResultList();
By default JPA using Hibernate implementation will generate 1 + n queries. The n queries are for lazy loading of the account entities.
How can I make this query eager and load everything with a single query? The sql equivalent would be something like
SELECT account.*, SUM(t.value) FROM transactions JOIN accounts on transactions.account_id = accounts.id GROUP BY account.id
, a syntax that works well on PostgreSQL. From my findings Hibernate is generating a query that justifies the lazy loading.
SELECT account.id, SUM(t.value) FROM transactions JOIN accounts on transactions.account_id = accounts.id GROUP BY account.id
Try marking the #ManyToOne field as lazy:
#ManyToOne(fetch = FetchType.LAZY)
private Account account;
And change your query using a JOIN FETCH of the account field to generate only one query with all you need, like this:
String sql = "SELECT new com.test.Pojo(acc, SUM(t.value)) "
+ "FROM Transaction t JOIN FETCH t.account acc GROUP BY acc";
UPDATE:
Sorry, you're right, the fetch attribute of #ManyToOne is not required because in Hibernate that is the default value. The JOIN FETCH isn't working, it's causing a QueryException: "Query specified join fetching, but the owner of the fetched association was not present".
I have tried with some other approaches, the most simple one that avoids doing n + 1 queries is to remove the creation of the Pojo object from your query and process the result list, manually creating the objects:
String hql = "SELECT acc, SUM(t.value)"
+ " FROM " + Transaction.class.getName() + " t"
+ " JOIN t.account acc"
+ " GROUP BY acc";
Query query = getEntityManager().createQuery(hql);
List<Pojo> pojoList = new ArrayList<>();
List<Object[]> list = query.getResultList();
for (Object[] result : list)
pojoList.add(new Pojo((Account)result[0], (BigDecimal)result[1]));
Well PostgreSQL (And any other SQL database too) will block you from using mentioned query: you have to group by all columns of account table, not by id. That is why Hibernate generates the query, grouping by ID of the account - That is what is intended to be, and then fetching the other parts. Because it cannot predict in general way, what else will be needed to be joined and grouped(!!!), and in general this could produce situation, when multiple entities with the same ID are fetched (just create a proper query and take a look at execution plan, this will be especially significant when you have OneToMany fields in your Account entity, or any other ManyToOne part of the Account entity) that is why Hibernate behaves this way.
Also, having accounts with mentioned IDs in First level cache, will force Hibernate to pick them up from that. Or IF they are rarely modified entities, you can put them in Second level cache, and hibernate will not make query to database, but rather pick them from Second level cache.
If you need to get those from database in single hint, but not use all the goodness of Hibernate, just go to pure JPA Approach based on Native queries, like this:
#NamedNativeQuery(
name = "Pojo.groupedInfo",
query = "SELECT account.*, SUM(t.value) as sum FROM transactions JOIN accounts on transactions.account_id = accounts.id GROUP BY account.id, account.etc ...",
resultClass = Pojo.class,
resultSetMapping = "Pojo.groupedInfo")
#SqlResultSetMapping(
name = "Pojo.groupedInfo",
classes = {
#ConstructorResult(
targetClass = Pojo.class,
columns = {
#ColumnResult(name = "sum", type = BigDecimal.class),
/*
* Mappings for Account part of entity.
*/
}
)
}
)
public class Pojo implements Serializable {
private BigDecimal sum;
/* .... */
public Pojo(BigDecimal sum, ...) {}
/* .... */
}
For sure this will work for you well, unless you will use the Account, fetched by this query in other entities. This will make Hibernate "mad" - the "entity", but not fetched by Hibernate...
Interesting, the described behaviour is as if t instances are returned from the actual query and t.account association in the first argument of Pojo constructor is actually navigated on t instances when marshalling results of the query (when creating Pojo instances from the result rows of the query). I am not sure if this is a bug or intended feature for constructor expressions.
But the following form of the query should work (no t.account navigation in the constructor expression, and no join fetch without the owner of the fetched association because it does not make sense to eagerly initialize something that is not actually returned from the query):
SELECT new com.test.Pojo(acc, SUM(t.value))
FROM Transaction t JOIN t.account acc
GROUP BY acc
EDIT
Very good observation by Ilya Dyoshin about the group by clause; I completely oversaw it here. To stay in the HQL world, you could simply preload all accounts with transactions before executing the query with grouping:
SELECT acc FROM Account acc
WHERE acc.id in (SELECT t.account.id FROM Transaction t)
Following is the code which is blowing up if the list which is being passed in to "IN" clause has several values. In my case the count is 1400 values. Also the customer table has several thousands (arround 100,000) of records in it. The query is executing against DERBY database.
public List<Customer> getCustomersNotIn(String custType, List<Int> customersIDs) {
TypedQuery<Customer> query = em.createQuery("from Customer where type=:custType and customerId not in (:customersIDs)", Customer.class);
query.setParameter("custType", custType);
query.setParameter("customersIDs", customersIDs);
List<Customer> customerList = query.getResultList();
return customerList;
}
The above mentioned method perfectly executes if the list has less values ( probably less than 1000 ), if the list customersIDs has more values since the in clause executes based on it, it throws an error saying "Statement too complex"
Since i am new to JPA can any one please tell me how to write the above mention function in the way described below.. * PLEASE READ COMMENTS IN CODE *
public List<Customer> getCustomersNotIn(String custType, List<Int> customersIDs) {
// CREATE A IN-MEMORY TEMP TABLE HERE...
// INSERT ALL VALUES FROM customerIDs collection into temp table
// Change following query to get all customers EXCEPT THOSE IN TEMP TABLE
TypedQuery<Customer> query = em.createQuery("from Customer where type=:custType and customerId not in (:customersIDs)", Customer.class);
query.setParameter("custType", custType);
query.setParameter("customersIDs", customersIDs);
List<Customer> customerList = query.getResultList();
// REMOVE THE TEMP TABLE FROM MEMORY
return customerList;
}
The Derby IN clause support does have a limit on the number of values that can be supplied in the IN clause.
The limit is related to an underlying limitation in the size of a single function in the Java bytecode format; Derby currently implements IN clause execution by generating Java bytecode to evaluate the IN clause, and if the generated bytecode would exceed the JVM's basic limitations, Derby throws the "statement too complex" error.
There have been discussions about ways to fix this, for example see:
DERBY-6784
DERBY-6301, or
DERBY-216
But for now, your best approach is probably to find a way to express your query without generating such a large and complex IN clause.
Ok here is my solution that worked for me. I could not change the part generating the customerList since it is not possible for me, so the solution has to be from within this method. Bryan your explination was the best one, i am still confuse how "in" clause worked perfectly with table. Please see below solution.
public List<Customer> getCustomersNotIn(String custType, List<Int> customersIDs) {
// INSERT customerIds INTO TEMP TABLE
storeCustomerIdsIntoTempTable(customersIDs)
// I AM NOT SURE HOW BUT, "not in" CLAUSE WORKED INCASE OF TABLE BUT DID'T WORK WHILE PASSING LIST VALUES.
TypedQuery<Customer> query = em.createQuery("select c from Customer c where c.customerType=:custType and c.customerId not in (select customerId from TempCustomer)");
query.setParameter("custType", custType);
List<Customer> customerList = query.getResultList();
// REMOVE THE DATA FROM TEMP TABLE
deleteCustomerIdsFromTempTable()
return customerList;
}
private void storeCustomerIdsIntoTempTable(List<Int> customersIDs){
// I ENDED UP CREATING TEMP PHYSICAL TABLE, INSTEAD OF JUST IN MEMORY TABLE
TempCustomer tempCustomer = null;
try{
tempCustomerDao.deleteAll();
for (Int customerId : customersIDs) {
tempCustomer = new TempCustomer();
tempCustomer.customerId=customerId;
tempCustomerDao.save(tempCustomer);
}
}catch(Exception e){
// Do logging here
}
}
private void deleteCustomerIdsFromTempTable(){
try{
// Delete all data from TempCustomer table to start fresh
int deletedCount= tempCustomerDao.deleteAll();
LOGGER.debug("{} customers deleted from temp table", deletedCount);
}catch(Exception e){
// Do logging here
}
}
JPA and the underlying Hibernate simply translate it into a normal JDBC-understood query. You wouldn't write a query with 1400 elements in the IN clause manually, would you? There is no magic. Whatever doesn't work in normal SQL, wouldn't in JPQL.
I am not sure how you get that list (most likely from another query) before you call that method. Your best option would be joining those tables on the criteria used to get those IDs. Generally you want to execute correlated filters like that in one query/transaction which means one method instead of passing long lists around.
I also noticed your customerId is double - a poor choice for a PK. Typically people use long (autoincremented/sequenced, etc.) And I don't get the "temp table" logic.
I like the idea of Named Queries in JPA for static queries I'm going to do, but I often want to get the count result for the query as well as a result list from some subset of the query. I'd rather not write two nearly identical NamedQueries. Ideally, what I'd like to have is something like:
#NamedQuery(name = "getAccounts", query = "SELECT a FROM Account")
.
.
Query q = em.createNamedQuery("getAccounts");
List r = q.setFirstResult(s).setMaxResults(m).getResultList();
int count = q.getCount();
So let's say m is 10, s is 0 and there are 400 rows in Account. I would expect r to have a list of 10 items in it, but I'd want to know there are 400 rows total. I could write a second #NamedQuery:
#NamedQuery(name = "getAccountCount", query = "SELECT COUNT(a) FROM Account")
but it seems a DRY violation to do that if I'm always just going to want the count. In this simple case it is easy to keep the two in sync, but if the query changes, it seems less than ideal that I have to update both #NamedQueries to keep the values in line.
A common use case here would be fetching some subset of the items, but needing some way of indicating total count ("Displaying 1-10 of 400").
So the solution I ended up using was to create two #NamedQuerys, one for the result set and one for the count, but capturing the base query in a static string to maintain DRY and ensure that both queries remain consistent. So for the above, I'd have something like:
#NamedQuery(name = "getAccounts", query = "SELECT a" + accountQuery)
#NamedQuery(name = "getAccounts.count", query = "SELECT COUNT(a)" + accountQuery)
.
static final String accountQuery = " FROM Account";
.
Query q = em.createNamedQuery("getAccounts");
List r = q.setFirstResult(s).setMaxResults(m).getResultList();
int count = ((Long)em.createNamedQuery("getAccounts.count").getSingleResult()).intValue();
Obviously, with this example, the query body is trivial and this is overkill. But with much more complex queries, you end up with a single definition of the query body and can ensure you have the two queries in sync. You also get the advantage that the queries are precompiled and at least with Eclipselink, you get validation at startup time instead of when you call the query.
By doing consistent naming between the two queries, it is possible to wrap the body of the code to run both sets just by basing the base name of the query.
Using setFirstResult/setMaxResults do not return a subset of a result set, the query hasn't even been run when you call these methods, they affect the generated SELECT query that will be executed when calling getResultList. If you want to get the total records count, you'll have to SELECT COUNT your entities in a separate query (typically before to paginate).
For a complete example, check out Pagination of Data Sets in a Sample Application using JSF, Catalog Facade Stateless Session, and Java Persistence APIs.
oh well you can use introspection to get named queries annotations like:
String getNamedQueryCode(Class<? extends Object> clazz, String namedQueryKey) {
NamedQueries namedQueriesAnnotation = clazz.getAnnotation(NamedQueries.class);
NamedQuery[] namedQueryAnnotations = namedQueriesAnnotation.value();
String code = null;
for (NamedQuery namedQuery : namedQueryAnnotations) {
if (namedQuery.name().equals(namedQueryKey)) {
code = namedQuery.query();
break;
}
}
if (code == null) {
if (clazz.getSuperclass().getAnnotation(MappedSuperclass.class) != null) {
code = getNamedQueryCode(clazz.getSuperclass(), namedQueryKey);
}
}
//if not found
return code;
}
List<Thing> results = em.createQuery(
"select t from Thing t where t.status in ('A', 'E')")
.getResultList();
This is using create query and without parameter.And it works fine
List<String> statusList= new ArrayList<String>();
statusList.add("A");
statusList.add("E");
List<Thing> results = em.createQuery(
"select t from Thing t where t.status in ('"+statusList.toSting()+"')")
.getResultList();
This is not working .I don't want IN(:parameter) to set .Only I need in the above descried one
You can not do it like above You have to use in as below
Query query=em.createQuery(
"select t from Thing t where t.status in (:list)")
query.setParameterList("list", statusList)
query.getResultList();
Your statusList.toSting() will not produce a string like 'A','E' becuase you have added the statusList.toSting() in () so if your ultimate result will looks like ('A','E') then only your query will give correct results
I am in the process of converting an application that uses LINQ to SQL over to LINQ to Entities. I use a repository pattern and I have run in a problem that works in LINQ to SQL but not Entities.
In my data layer, I use LINQ statements to fill my object graph so that none of my database entities are exposed anywhere else. In this example, I have a Lookup Respository that returns a list of Categories. It looks like this:
public IQueryable<Entities.DomainModels.Category> getCategories()
{
return (from c in Categories
where !c.inactive
orderby c.categoryName
select new Entities.DomainModels.Category
{
id = c.categoryID,
category = c.categoryName,
inactive = c.inactive
});
}
Later, I want to put the categories into a sub query and it looks like this:
var d = from p in Programs
let categories = (from pc in p.Categories
join c in getCategories() on pc.categoryID equals c.id
select c)
select new
{
id = p.id,
title = p.title
categories = categories.ToList()
};
When I run this, I get the following error:
LINQ to Entities does not recognize the method 'System.Linq.IQueryable`1[Entities.DomainModels.Category] getCategories()' method, and this method cannot be translated into a store expression.
For reference, the following works though it doesn't return the data I need (it's basically a join):
var q = from p in Programs
from pc in p.Categories
join c in getCategories() on pc.categoryID equals c.id
select new
{
id = p.id,
category = c
};
I understand what the error means in concept however LINQ to SQL would make it work. I have this pattern throughout my data layer and I really want to keep it. Should this be working? If not, how can I modify it without mixing my layers.
You cant pass getCategories() to EF.
The query must be destructible to expression tree.
Calculate getCategories() first.
eg
var simpleList = getCategories().Select(id).Tolist;
then use a contains
where(t=> simpleList.Contains(t.CatId) // or the query syntax equivalent