Group by date intervals using JPA's Criteria API - postgresql

I'm trying to group entities by date intervals using JPA's Criteria API. I use this way of querying for entities as this is a part of the service that serves API requests which may ask for any field of any entity, including sorting, filtering, grouping and aggregations. Everything works fine except for grouping by date fields. My underlying DBMS i PostgreSQL.
To give a minimal example, here's my entity class:
#Entity
#Table(name = "receipts")
public class DbReceipt {
#Id
#GeneratedValue(strategy = GenerationType.AUTO)
private Long id;
private Date sellDate;
// Many other fields
}
This example discusses grouping my "month" interval (therefore grouping by year+month), but in the end I'm looking for a solution that would let me group by any interval, such as "year", "day" or "minutes".
What I'm trying to achieve is the following query, but using Criteria API:
SELECT TO_CHAR(sell_date, 'YYYY-MM') AS alias1 FROM receipts GROUP BY alias1;
My attempt to do so is this:
#Service
public class ReceiptServiceImpl extends ReceiptService {
#Autowired
private EntityManager em;
#Override
public void test() {
CriteriaBuilder cb = em.getCriteriaBuilder();
CriteriaQuery<Object[]> query = cb.createQuery(Object[].class);
Root<?> root = query.from(DbReceipt.class);
Expression<?> expr = cb.function("to_char", String.class, root.get("sellDate"), cb.literal("YYYY-MM"));
query.groupBy(expr);
query.multiselect(expr);
TypedQuery<Object[]> typedQuery = em.createQuery(query);
List<Object[]> resultList = typedQuery.getResultList();
}
}
The reason I use to_char function and not MONTH and similar is that I need entities like 2019-05 and 2020-05 to not be grouped together. I also narrow this example down to only year and month to keep things short, but the goal is to group by any date interval.
The code above creates the following query (SQL logging enabled) which results in an error:
Hibernate: select to_char(dbreceipt0_.sell_date, ?) as col_0_0_ from receipts dbreceipt0_ group by to_char(dbreceipt0_.sell_date, ?)
24-05-2020 12:16:30.071 [http-nio-1234-exec-5] WARN o.h.e.jdbc.spi.SqlExceptionHelper.logExceptions - SQL Error: 0, SQLState: 42803
24-05-2020 12:16:30.071 [http-nio-1234-exec-5] ERROR o.h.e.jdbc.spi.SqlExceptionHelper.logExceptions - ERROR: column "dbreceipt0_.sell_date" must appear in the GROUP BY clause or be used in an aggregate function
Position: 16
which to me is caused by the fact that the whole expression is put into the 'group by' part of the query, rather than just an alias. Now, I've tried to assign an alias to the expression (which returns Selection<T> and groupBy accepts expressions, therefore I can only really use that in the multiselect), but that didn't affect how the query is performed - nothing changed.
How do I achieve grouping by year and month as described above using Criteria API? Maybe there's a different way other than using to_char? Maybe there's a way to give an alias to the groupBy method that would cause it to group by an alias instead of the whole expression?

I think it's a bug in PostgreSQL (the error comes from there, not from Hibernate). I have tried a slightly modified version of your code with EclipseLink + Derby and works perfectly.
Note that I had to use numbers instead of strings because Derby DB doesn't have an equivalent of TO_CHAR function.
Expression<Integer> year = cb.function("YEAR", Integer.class, root.get("sellDate"));
Expression<Integer> month = cb.function("MONTH", Integer.class, root.get("sellDate"));
Expression<Integer> expr = cb.sum(month, cb.prod(12, year));
query.groupBy(expr);
query.multiselect(expr);
This returns the following SQL:
SELECT (MONTH(MY_DATE) + (12 * YEAR(MY_DATE)))
FROM MY_DATE_TABLE
GROUP BY (MONTH(MY_DATE) + (12 * YEAR(MY_DATE)))
Note that there are no portable solutions for manipulating dates in JPA criteria queries. If the number of groups to be queried simultaneously is not too high I'd go with a more practical approach where you find the dates in Java and pass them as literals to the query builder.
Another workaround is to query with a groupBy(root.get("sellDate")) and then aggregate the results in Java according to the desired time period.
Post Scriptum: I don't think it's relevant, however I modified the query's return type from Object[] to Object.

Related

Using Hibernate to get the latest row with a LocalDateTime

I have a Java8 application using SpringBoot, which is pulling in Hibernate core 5.3.10 (connected to PostgreSQL 11). A simplified explanation: the application is maintaining a history of changes to a series of bespoke UserData records, containing two LocalDateTime columns for startDate and endDate, and a userId column (plus other columns).
The semantics of the application are that a history of changes to the UserData are maintained by the start_date and end_date contain no duplicates (respectively), and that any row where the endDate is null is the currently active record for a user. (Accordingly, a user may have no currently active record.)
It is a simple matter to retrieve all of the active rows, and this is working well.
However, I have a requirement to retrieve the latest row for all users irrespective of whether the row is active or not.
One way to achieve this outcome using SQL is with a query something like the following:
select ud1.* from user_data ud1
where ud1.start_date = (select max(ud2.start_date) from user_data ud2
where ud2.user_id = ud1.user_id);
I have been attempting to write a CriteriaQuery to replicate the SQL, but I have run into a data type problem with the CriteriaBuilder.max method. The max method is complaining that it will only accept a type Number column. The code looks like this:
final CriteriaBuilder builder = entityManager.getCriteriaBuilder();
final CriteriaQuery<UserDate> criteriaQuery = builder.createQuery(UserData.class);
final Root<UserData> ud1 = criteriaQuery.from(UserData.class);
final Subquery<LocalDateTime> maxUserDataStartDate = criteriaQuery.subquery(LocalDateTime.class);
final Root<UserData> ud2 = maxUserDataStartDate.from(UserData.class);
maxUserDataStartDate.select(builder.max(ud2.get("startDate"));
// ...
The problem is with the last line, where it complains that ud2.get("startDate") is not an extension of type Number - which is true, of course.
Does anybody know how to fix this situation? Especially, does anybody have an example they can share that does what I'm after?
You can do order by start_date desc and get top 1
CriteriaBuilder builder = entityManager.getCriteriaBuilder();
CriteriaQuery<UserData> criteriaQuery = builder.createQuery(UserData.class);
Root<UserData> ud = criteriaQuery.from(UserData.class);
criteriaQuery.orderBy(builder.desc(ud.get("startDate")));
entityManager.createQuery(criteriaQuery).getFirstResult();

How to use '::' in spring boot?

how to use a query from the Postgre to Spring boot?
I used this query to get data in postgre
"SELECT * FROM mytable WHERE delivery_date::date = '2019-10-28' limit 10;"
I'm trying to make a rest fire with an entity like this
#Entity
#Table(name ="mytable")
public class DeliveryDataEntity implements Serializable {
#Column(name="delivery_date")
private Date delivery_date;
#Column(name="delivery_name")
private String delivery_name;
#Id
#Column(name="delivery_id")
private int delivery_id;
}
and my repo code like this
#Repository
public interface DeliveryDataRepository extends PagingAndSortingRepository<DeliveryDataEntity, Date>{
#Query(value="SELECT * FROM mytable WHERE delivery_date = ?1", nativeQuery = true)
Page<DeliveryDataEntity> findByUserAndStatusOrderByCreatedAtDesc(Date delivery_date, PageRequest pageRequest
);
}
If I run there are no errors, but the data isn't readable.
I tried changing the query to be like this
#Query(value="SELECT * FROM mytable WHERE delivery_date >= ?1", nativeQuery = true)
and
Query(value="SELECT * FROM accounts_data WHERE updated \\= ?1", nativeQuery = true)
but, if I run it will get null results.
how do i solve this problem?
thanks
Assuming that the likely problem be the double colon (::) cast operator, which Spring Boot doesn't like, you could try refactoring your query to use a range instead:
SELECT *
FROM mytable
WHERE delivery_date >= '2019-10-28' AND delivery_date < '2019-10-29'
ORDER BY <some column>
LIMIT 10;
The logic in the WHERE clause simply states to find all records whose delivery dates occur on or after midnight of 28-OCT-2019, and strictly before midnight of the following day.
JPA/Hibernate might also support using the CAST function, since Postgres does implement CAST, so this version might also work:
SELECT *
FROM mytable
WHERE (CAST delivery_date AS date) = '2019-10-28'
ORDER BY <some column>
LIMIT 10;
I think you do not need the (::) operator. If you want to consider only dates (without times), you can use the #Temportal annotation, as explained in this Baeldung tutorial.
Also, I think you don't need to use native queries.

JPQL get difference in days between a column and now

I'm using hibernate and jpa in a spring boot application.
I have a hibernate entity with 2 columns, type (string) and date (LocalDate).
I want to select all the columns and the difference of days between my date column and now.
I succeeded in PostgreSQL: select date, date - current date from table;
I'm using an interface which extends JpaRepository. Until now i was successfully to insert new queries through #Query annotations.
I can't figure it out how to select my columns and a column containing the difference of days in jpql.
Kind regards,
Indeed you can use query creator. You have to create a new entity which has the same columns as the original table and in addition the calculated difference between the date column and now. You have to be careful which date type is your column.
In my experience only Date from java.util can be used for differences with CURRENT_DATE (i may be wrong).
public List<NewEntity> getTest() {
String queryStr = "select new com.entity.NewEntity(n.type, n.date - CURRENT_DATE as
time) from Table as n"
TypedQuery<PunteInspectieTimpRamas> query = entityManager.createQuery(queryStr,
NewEntity.class);
return query.getResults()
}
Not sure you can do this with jpql.
But you can use your query as a native query, directly in you repository:
#Query(value = "SELECT ... ", nativeQuery = true)
ClassToReturn methodName();
see https://www.baeldung.com/spring-data-jpa-query

JPA date literal

How represent a date in a JPA query, without using (typed) parameters?
If the date is really fixed (for example, 1 mar 1980), the code:
TypedQuery<MyEntity> q = em.createQuery("select myent from db.MyEntity myent where myent.theDate=?1", db.MyEntity.class).setParameter(1, d);
having set:
Date d = new Date(80, Calendar.MARCH, 1);
is quite verbose, isn't it? I would like to embed 1980/1/3 into my query.
UPDATE:
I modified the sample date to 1980/1/3, because 1980/1/1 as it was, was ambiguous.
IIRC you can use date literals in JPQL queries just like you do it in JDBC, so something like:
// d at the beginning means 'date'
{d 'yyyy-mm-dd'} i.e. {d '2009-11-05'}
// t at the beginning means 'time'
{t 'hh-mm-ss'} i.e. {t '12-45-52'}
// ts at the beginning means 'timestamp'; the part after dot is optional
{ts 'yyyy-mm-dd hh-mm-ss.f'} i.e. {ts '2009-11-05 12-45-52.325'}
should do the work (the curly braces and apostrophes are required).
I spent a couple days digging around on this. Seems the root of the problem is that the Hibernate generated grammar does not include support for temporal literals.
The JPA specification does include support for temporal literals but does not require persistence providers to translate from from the JPA syntax to the native syntax of the JDBC driver. From the JPA2 Spec 4.6.1:
"The JDBC escape syntax may be used for the specification of date, time, and timestamp literals. For example:
SELECT o
FROM Customer c JOIN c.orders o
WHERE c.name = 'Smith'
AND o.submissionDate < {d '2008-12-31'}
The portability of this syntax for date, time, and timestamp literals is dependent upon the JDBC driver in use. Persistence providers are not required to translate from this syntax into the native syntax of the database or driver."
It would be nice if Hibernate did provide support for date literals, but it seems the implementation for this is a little more involved that I'd suspected.
The functionality lacking here as far as my needs are concerned is that you cannot do a select coalesce(somePath, someDateLiteral) query. You can still do a where somePath=someDate. As long as they're mapped entities you can throw whatever you want in a where clause.
I used criteriaQuery for my querys, its just great. It looks like:
#Override
public List<Member> findAllByDimensionAtTime(Dimension selectedDimension,
Date selectedDate) {
CriteriaBuilder criteriaBuilder = em.getCriteriaBuilder();
CriteriaQuery<Member> criteriaQuery = criteriaBuilder
.createQuery(Member.class);
Root<Member> member = criteriaQuery.from(Member.class);
criteriaQuery
.select(member)
.where(criteriaBuilder.lessThanOrEqualTo(
member.get(Member_.validFrom), selectedDate),
criteriaBuilder.greaterThanOrEqualTo(
member.get(Member_.validTo), selectedDate),
criteriaBuilder.equal(
member.get(Member_.myDimensionId),
selectedDimension.getId())).distinct(true);
return em.createQuery(criteriaQuery).getResultList();
validFrom and validTo are date fields!
Edit: shorter Example (according to yours):
CriteriaBuilder criteriaBuilder = em.getCriteriaBuilder();
CriteriaQuery<MyEntity> criteriaQuery = criteriaBuilder
.createQuery(MyEntity.class);
Root<MyEntity> entity= criteriaQuery.from(MyEntity.class);
criteriaQuery
.select(member)
.where(criteriaBuilder.equal(
entity.get(MyEntity_.theDate),
new Date(80, Calendar.MARCH, 1);)).distinct(true);
return em.createQuery(criteriaQuery).getResultList();

Lambda expression for Entity Framework query

I'm stuck trying to map this sql:
select dt.Id, dateadd(dd,datediff(dd,0,dt.CreatedAt),0) as Ct, DId, Amount
from dt, ad
where dt.ADId = ad.ADId and ad.Id = '13B29A01-8BF0-4EC9-80CA-089BA341E93D'
order by dateadd(dd,datediff(dd,0,dt.CreatedAt),0) desc, DId asc
Into an Entity Framework-compatible lambda query expression. I'd appreciate any help.
I think something like the code below should work.
Guid dtId = new Guid("13B29A01-8BF0-4EC9-80CA-089BA341E93D");
DateTime compareDt = new DateTime(...);
var q = from dt in dts
where dt.id == dtId
orderby dt.Ad.CreatedAt, Did
select new
{
dt.Ad,
(dt.CreatedAt - compareDt).Days,
DId,
Amount
};
dtId has to be outside of the query, because the entity framework mapper doesn't understand constructors that take parameters, similar for the DateTime. I couldn't completely figure out the datediff/dateadd part, looks like you're determining the total amount of days since a given datetime, which is what I assumed.
It could be that the datetime subtraction followed by TimeSpan.Days doesn't work in the query. If that's the case, pull it outside like so:
var q = from dt in dts
where dt.id == dtId
orderby dt.Ad.CreatedAt, Did
select new
{
dt.Ad,
dt.CreatedAt,
DId,
Amount
};
This creates an IEnumerable of objects that have a DateTime CreatedAt property. If you call .ToList() now, the query is executed, and you can calculate stuff that Entity Framework doesn't support. Again, if the first attempt worked, that's the best solution if the number of days since X is what you need.
from item in q.ToList()
select new
{
dt.Ad
(dt.CreatedAt - compareDt).Days,
DId,
Amount
};