Micronaut: JPA batch update operations is not working for postgresql - postgresql

I'm trying to run batch operations in my Micronaut app.
Currently, I can run insert operations in batch (with reWriteBatchedInserts option activated) but the configuration is not working for updates.
Something special about my app is that I want to set the id manually.
This is my entity:
#Data
#Entity
#SuperBuilder
#NoArgsConstructor
#Table(name = "PRODUCT")
public class ArticleItem implements Cloneable, Serializable {
#Id
#Column
private Long id;
The service:
#Singleton
#Slf4j
public class ArticleServiceImpl implements ArticleService {
#Transactional
public List<GenericSingleResponse> internalSaveArticles(
List<ArticleItem> articleItemList,
Integer erp, String traceId, String json
) {
articlePort.saveAll(insertList);
The port:
#Singleton
public class ArticlePortImpl implements ArticlePort {
private final EntityManager entityManager;
#Transactional
public void updateAll(List<ArticleItem> articleItemList) {
articleItemList.forEach(f -> entityManager.merge(f));
}
The configuration:
datasources:
default:
url: jdbc:postgresql://localhost:5432/postgres?reWriteBatchedInserts=true
jpa:
default:
properties:
hibernate:
hbm2ddl:
auto: none
jdbc:
batch_size: 100
batch_versioned_data: true
order_inserts: true
order_updates: true
batch_versioned_data: true
show_sql: false
id:
optimizer:
pooled:
preferred: pooled-lo

Related

Micronaut: configure JPA batch operations

I'm trying to configure my Micronaut project to insert/update using batch operations (create one single insert with 50 items included instead of 50 different inserts).
I cannot find a more detailed manual so the configuration that I'm using is this one:
datasources:
default:
url: jdbc:postgresql:///postgres?cloudSqlInstance=uri&socketFactory=com.google.cloud.sql.postgres.SocketFactory&ipTypes=PRIVATE
username: ****
password: ****
driverClassName: org.postgresql.Driver
jpa:
default:
properties:
hibernate:
hbm2ddl:
auto: none
jdbc:
batch_size: 50
order_inserts: true
order_updates: true
batch_versioned_data: true
show_sql: false
This is the repo:
import io.micronaut.data.annotation.Repository;
import io.micronaut.data.repository.GenericRepository;
#Repository
public interface ArticleExportPort extends GenericRepository<ArticleExport, Long> {
void insertMany(Iterable<ArticleExport> articleExports);
List<ArticleExport> findAll();
}
And this is the Entity:
#Data
#Entity
#Table(name = "PRODUCT_EXPORT")
public class ArticleExport {
#Id
private Long Id;
#NotNull
#Column(name = "PRODUCT_ID",nullable = false)
private Long productId;
}
The database is a PostgreSQL located in Google Cloud but also testing in local it doesn't work.
Any suggestion?
Thanks!

Why JPQL ignore parent's fields?

App's stack: Hibernate, Spring Data, JPA.
There are some entities in the app. I try make JPQL-query in repository of my class OpenParagraph.
OpenParagraph:
#Entity
#Table(name = "open_paragraphs")
#NoArgsConstructor
#AllArgsConstructor
#Getter
#Setter
#ToString
public class OpenParagraph extends ProgramEntry {
#NotNull
#Column(name = "sort_num")
private Integer sortNum;
}
OpenParagraph has a parent: abstract class ProgramEntry.
ProgramEntry:
#MappedSuperclass
#NoArgsConstructor
#AllArgsConstructor
#Getter
#Setter
#ToString
public abstract class ProgramEntry extends AbstractBaseEntity {
#NotNull
#ManyToOne
#JoinColumn(name = "paragraph_id")
private Paragraph paragraph;
#NotNull
#ManyToOne
#JoinColumn(name = "program_id")
private Program program;
}
So, i tring to appeal to OpenParagraph's field "Paragraph", but IDEA tells me it's mistake:
It doesn't offer me the "program" field:
IDEA offer fields only from OpenParagraph, not from parent.
My question: this is IDEA's fail? If this is'nt IDEA's fail, then how i can call "program" in this query?
This is/was a bug of Intellij IDEA (maybe related to this?). But:
It is possible to query by fields of the super class (or MappedSuperclass). Here is an example:
#MappedSuperclass
#Getter
#Setter
public class Foo extends AbstractPersistable<Long> {
#Column
private String fooValue;
}
#Entity
#Getter
#Setter
public class Bar extends Foo {
#Column
private String barValue;
}
public interface BarRepository extends JpaRepository<Bar, Long> {
#Query("SELECT b FROM Bar b WHERE b.fooValue = ?1")
List<Bar> findByFooValue(String fooValue);
}
Given this, calling the repository method, something like this will be logged (with enabled sql logging):
Hibernate: select bar0_.id as id1_0_, bar0_.foo_value as foo_valu2_0_, bar0_.bar_value as bar_valu3_0_ from bar bar0_ where bar0_.foo_value=?
Hint:
If you are using Spring Boot (with the test dependency/dependencies and an embedded test db like h2), it is quite easy to execute such methods without to run the whole application. Here just a small snipped that would execute the method (even though this is no test, but that's enough to call methods somehow):
#SpringBootTest
public class BarRepositoryTest {
#Autowired
BarRepository barRepository;
#Test
public void testFindByFooValue() {
barRepository.findByFooValue("foo");
}
}

Spring JPA Repository is unable to create mapping tables of my entities

I'm using spring data with H2 database and it's not creating database tables for my entity objects.
application.properties
spring.datasource.driverClassName=org.h2.Driver
spring.datasource.url=jdbc:h2:file:~/app_db
spring.datasource.username=sa
spring.datasource.password=sa
spring.jpa.database-platform=org.hibernate.dialect.H2Dialect
spring.jpa.hibernate.ddl-auto=create
DomainApplication.java
#SpringBootApplication
#Import(DefaultPersistenceConfig.class)
#EntityScan(basePackages = {"com.sample.Domain.modal"})
public class DomainApplication {
public static void main(String[] args) {
SpringApplication.run(DomainApplication.class, args);
}
}
DefaultPersistanceConfig:
#Configuration
#EnableJpaRepositories(basePackages = {"com.sample.Domain.model" }, entityManagerFactoryRef = "defaultEntityManagerFactory", transactionManagerRef = "defaultTransactionManager")
#EnableTransactionManagement
public class DefaultPersistenceConfig {
#Autowired
public DataSource defaultDataSource;
#Bean
#Primary
#Qualifier("defaultEntityManagerFactory")
public LocalContainerEntityManagerFactoryBean defaultEntityManagerFactory(EntityManagerFactoryBuilder builder) {
return builder.dataSource(defaultDataSource).persistenceUnit("hib-unit")
.packages("com.sample.Domain.model").build();
}
#Bean
#Primary
#Qualifier("defaultTransactionManager")
public PlatformTransactionManager defaultTransactionManager(EntityManagerFactory emf) {
return new JpaTransactionManager(emf);
}
}
Entity class:
#Entity
#Table(name = "GENERAL_LEDGER_TRANSACTION")
public class GeneralLedgerTransaction implements Serializable {
private static final long serialVersionUID = 1L;
#Id
#GeneratedValue(strategy = GenerationType.IDENTITY)
#Column(name = "ID")
private Long id = 0L;
....
JPA Repository interface:
#Repository
public interface GeneralLedgerTransactionRepository extends JpaRepository<GeneralLedgerTransaction, Long> {
}
When I run the application and make a rest call to the controller to fetch data from the repository, it should return an empty array but I keep getting an error below.
org.h2.jdbc.JdbcSQLException: Table "GENERAL_LEDGER_TRANSACTION" not found; SQL statement:
select ... from GENERAL_LEDGER_TRANSACTION generalled0_ [42102-197]
at org.h2.message.DbException.getJdbcSQLException(DbException.java:357) ~[h2-1.4.197.jar:1.4.197] ...
add spring.datasource.initialize=true in your application.properties

Hibernate Postgres Slow Insert

I try to insert data into Postgres using JPA/Hibernate/Postgres.
The data is parsed from a CSV File and then should be saved into a postgres database. The code that persists the data looks as follows:
#Autowired
KundeRepository repo;
#Transactional
public void safe(Kunde kd) {
repo.save(kd);
}
public void safeAll(Iterable<Kunde> kt) {
repo.save(kt);
repo.flush();
}
The entity look as follows
public class account implements Serializable {
private static final long serialVersionUID = 1L;
#Id
#GeneratedValue(strategy = GenerationType.AUTO)
private Long id;
}
#ManyToOne
Kunde kunde;
#OneToMany
List<TransaktionsGruppe> gruppen;
#Entity
#Table(name = "kunde")
#NoArgsConstructor
public class Kunde implements Serializable {
public static final String kundennummerKey = "KUNDENNUMMER";
private static final long serialVersionUID = 1L;
#Id
#Getter
#Setter
private String id;
#OneToMany
List<Account> accounts;
#Entity
#Table(name = "transaktionsgruppe")
public class Transaktionsgruppe implements Serializable {
#Id
#GeneratedValue(strategy = GenerationType.AUTO)
private Long id;
#ManyToOne
Account acc;
private String bezeichnung;
When I now pass a Collection to the safeAll method the inserts are really slow. Especially it seems to call the hibernate sequence for every insert. Is there a way to speed things up?
My configuration looks as follows:
datasource:
type: com.zaxxer.hikari.HikariDataSource
url: dburl
username: user
password: pw
hikari:
validation-timeout: 10000
health-check-properties: {"connectivityCheckTimeoutMs","1000"}
jpa:
show-sql: true
properties:
hibernate.cache.use_second_level_cache: false
hibernate.cache.use_query_cache: false
hibernate.generate_statistics: false
hibernate.jdbc.batch.size: 100
hibernate.order_inserts: true+
Currently show-sql is enabled. It is about 60000 Entities overall and it takes more than 20 Minutes. the entitties are rather small in size
Your safeAll() method is not annotated with #Transactional so Spring opens and closes a transaction for each item of your list. By annotating it, Spring will open and close one single transaction for the whole list.

JPA Criteria to query hierarchy of child objects

I have two tables that are represented by following entity object hierarchies:
#Entity
#Table(name = Transport.TABLE_NAME)
#DiscriminatorColumn(name="transport_type", discriminatorType = DiscriminatorType.INTEGER)
#Inheritance(strategy=InheritanceType.SINGLE_TABLE)
public abstract class Transport {
...
private Date departure;
}
#Entity
#DiscriminatorValue("1")
public class Bicycle extends Transport {
...
#OneToOne(mappedBy = "transport", fetch = FetchType.LAZY)
private BikePassenger passenger;
}
#Entity
#DiscriminatorValue("2")
public class Car extends Transport {
...
#OneToMany(mappedBy = "transport", fetch = FetchType.EAGER)
private List<CarPassanger> passengers;
}
#Entity
#Table(name = Passenger.TABLE_NAME)
#DiscriminatorColumn(name="passenger_type", discriminatorType = DiscriminatorType.INTEGER)
#Inheritance(strategy=InheritanceType.SINGLE_TABLE)
public abstract class Passenger {
...
private int passengerGUID;
}
#Entity
#DiscriminatorValue("1")
public class BicyclePassenger extends Passenger {
...
#OneToOne(fetch = FetchType.EAGER)
#JoinColumn(name = "transportid")
private Bicycle transport;
}
#Entity
#DiscriminatorValue("2")
public class CarPassenger extends Passenger {
...
#ManyToOne(fetch = FetchType.EAGER)
#JoinColumn(name = "transportid")
private Car transport;
}
Now using JPA Criteria API (or at least JPA) how do I:
Get all transports that have passengers with specific passengerGUID?
Group transports (along with passengers) by departure date?
As I see it #1 should have nice solution but I was able to get out only with 2 subselects for each subtype. Which looks ugly to me.
And finally third question - is it good model at all? From OOP point of view to me it looks ok, but from ORM point of view and easiness of queries it looks not so good...
p.s. I'm using hibernate JPA 2.1