I'm working with Spring Data Repositories manipulating data from a PostreSQL database.
In one of my repositories, I have a very simple query:
#Query(value = "FROM MyEntity entity WHERE entity.entityId IN (:entityIds)")
Collection<MyEntity> getEntitiesByIds(#Param("entityIds") Collection<Long> entityIds);
The problem is, when I pass in a collection that has more than 32k~ ids this query throws the following error:
java.io.IOException: Tried to send an out-of-range integer as a 2-byte value: 70324
I did some research and found out that this error happens because postgresql jdbc driver has a limit of 32k~ parameters you can pass in to a query. So I have two options here:
Do the query in batches. At this point I think this is my best shot but I was hoping to find a solution so I can pull my rows in one database call.
Build a temporary table with the ids, then join it against MyEntity table, but I'm running again into the same original problem. For this approach I was thinking to pass a String with all the concatenated ids and then use a native query to pull the information but it its not working as I expected:
#Query(value = ";WITH cteAllEntityIds(entity_id) AS( " +
"VALUES (?1)) " +
"SELECT p.* FROM my_entity e " +
"JOIN cteAllEntityIds cte ON e.entity_id = cte.entity_id", nativeQuery = true)
Collection<ProfileViewEntity> getProfilesByIds(String profileIds);
Of course, the parameter I'm passing in is a String whereas the entity_id is a number in the data base.
Do you know a way to accomplish approach #2? How to build queries dynamically and make jpa to interpret it that way?
Do you know a better solution to this problem?
Any suggestions are welcome.
Thanks.
Related
Hello experts of the world. Need some help concerning executing a query with SpringData.
The expectation is to execute the Query below in the Spring Data annotation by combining with the repository method name (Automated Query Construction) to get a unique result. Apparently it fails from time to time by saying the result is not Unique.
The question here is if the method name is still considered in Query Construction while also executing the query in the annotation.
#Query("SELECT r from Revision r WHERE r.revisionBid = ?1 AND r.revisionStatusId = ?2 ORDER BY r.lastModifiedDate DESC")
Optional<Revision> findFirst(Integer revisionBid, Integer revisionStatusId);
Thanks in advance!
The query creation for limiting to 1 result is defined here with FIRST & TOP included in the method name.
https://docs.spring.io/spring-data/jpa/docs/current/reference/html/#repositories.query-methods.query-creation
I don't think "findFirst" will work when you're using an #Query, as the query will be constructed from HQL expression in the #Query rather than the fluent API passing over the method name. Because of this, when the query returns multiple results, it will throw the exception as the Optional is told to wrap a single returned object, not a collection. Add a LIMIT clause to the HQL query and you should be good.
I want to use spring data rest to update rows of certain user , but at run time this query has strange "cross join" added to the query .
spring data rest method
#Modifying
#Transactional
#Query("Update Notification n SET n.noticed = true Where n.notificationPost.owner.userId = 1 ")
public void postNoticed();
run time created query
Hibernate: update notification cross join set noticed=true where owner_id=?
My only concern is why "cross join" added as it gives sql error
org.postgresql.util.PSQLException: ERROR: syntax error at or near "cross"
I call this method directly by rest invoke , and also from mvc controller, both ways produce the same error
Thanks in advance.
Found solution as stated in http://forum.spring.io/forum/spring-projects/data/114271-spring-data-jpa-modifying-query-failure
"No joins, either implicit or explicit, can be specified in a bulk HQL query. Sub-queries can be used in the where-clause, where the subqueries themselves may contain joins. "(Hibernate doc reference: http://docs.jboss.org/hibernate/core.../#batch-direct)."
So I edited my code to use sub query
#Modifying
#Transactional
#Query("Update Notification n SET n.noticed = true Where n.notificationPost.postId in (SELECT n2.notificationPost.postId FROM Notification n2 where n2.notificationPost.owner.userId =:#{#security.principal.user.userId}) ")
public int postNoticed();
I have a custom query along these lines. I get the list of orderIds from outside. I have the entire order object list with me, so I can change the query in any way, if needed.
#Query("SELECT p FROM Person p INNER JOIN p.orders o WHERE o.orderId in :orderIds)")
public List<Person> findByOrderIds(#Param("orderIds") List<String> orderIds);
This query works fine, but sometimes it may have anywhere between 50-1000 entries in the orderIds list sent from outside function. So it becomes very slow, taking as much as 5-6 seconds which is not fast enough. My question is, is there a better, faster way to do this? When I googled, and on this site, I see we can use ANY, EXISTS: Postgresql: alternative to WHERE IN respective WHERE NOT IN or create a temporary table: https://dba.stackexchange.com/questions/12607/ways-to-speed-up-in-queries-under-postgresql or join this to VALUES clause: Alternative when IN clause is inputed A LOT of values (postgreSQL). All these answers are tailored towards direct SQL calls, nothing based on JPA. ANY keyword is not supported by spring-data. Not sure about creating temporary tables in custom queries. I think I can do it with native queries, but have not tried it. I am using spring-data + OpenJPA + PostgresSQL.
Can you please suggest a solution or give pointers? I apologize if I missed anything.
thanks,
Alice
You can use WHERE EXISTS instead of IN Clause in a native SQL Query as well as in HQL in JPA which results in a lot of performance benefits. Please see sample below
Sample JPA Query:
SELECT emp FROM Employee emp JOIN emp.projects p where NOT EXISTS (SELECT project from Project project where p = project AND project.status <> 'Active')
I implemented a method that allows me to generate a test ( entire of question ) automatically but the problem is :
the method will obviously take the number of random questions but also the category of questions generated (I have a entié category and therefore a table too) I don't know where i will put the category in query.
and secondly RANDOM() is not taked by JPQL what can i do ?
schema of the DataBase
public List<Question> prepareRandomTest(int number_of_questions, Categorie categorie){
String jpql = "SELECT q FROM Question q ORDER BY RANDOM() LIMIT "+number_of_questions ;
Query query = entityManager.createQuery(jpql);
return query.getResultList();
}
You are trying to use Java Persistence Query Language here, Hence your solution is not taking RANDOM into account. Use Native query and build Query from native sql string, Native query is just a plain sql statement without Entity object reference (like Question). This way a normal sql keywords like RANDOM etc are easily read.
Native Query Tutorial
Instead of Using
String jpql = "SELECT q FROM Question q ORDER BY RANDOM() LIMIT "+number_of_questions ;
use:
"SELECT * FROM question where category="+category+" ORDER BY RANDOM() LIMIT "+number_of_questions;
Additional Advice:
Get the category string from an "Enum" in your code to match the string value in the database category column.
OrientDB is throwing a java.lang.ClassCastException when a sever function (a query on indexed fields) is executed with formatted params.
Following messages are seen with the exception.
"Error on using index", "Probably you need to rebuild indexes. Now executing query using cluster scan
e.g.
db.query("SELECT FROM Employee WHERE department = ?", departmentRid);
where, Employee.department is indexed with NOT_UNIQUE_HASH_INDEX
When I removed the formatted params and injected them manually index worked out fine.
e.g.
db.query("SELECT FROM Employee WHERE department = " + departmentRid);
Any reason why the first approach didn't work? I'd like to refrain from injecting the params manually.
Note that the both approaches gives me the correct result. The problem is that the index is not applied for the first approach.
I am using orientdb-community-2.1.0
Appreciate your help.
I tried it with version 2.1.0, I have the NOTUNIQUE_HASH_INDEX on department field of Employee and it works