wrong sequence of SELECT's and UPDATES in spring-boot-service accessing repository - jpa

i have all simple classes, but i have a problem with the spring-boot-service and the repository.
It's like I have a test class with the following test and the necessary method execut():
#Test
public void deposit() throws Exception {
long balance = accountService.getBalance(accountNr, pin);
execute(() -> accountService.deposit(accountNr, amount), INVOCATIONS);
long newBalance = accountService.getBalance(accountNr, pin);
assertEquals(balance + INVOCATIONS * amount, newBalance);
}
public static void execute(Task task, int times) throws InterruptedException
{
ExecutorService executorService = Executors.newCachedThreadPool();
for (int i = 0; i < times; i++) {
executorService.submit(() -> {
try {
task.run();
} catch (Exception ex) {
throw new RuntimeException(ex);
}
});
}
executorService.shutdown();
executorService.awaitTermination(1, TimeUnit.HOURS);
}
Then there is a very simple entity with four attributes:
#Entity
public class Account {
#Id
#GeneratedValue
private Integer nr;
#Version
private Integer version;
private String pin;
private long balance;
...
The service has a method that first searches for an account, modifies a value, and attempts to store it in the database:
public void deposit(int accountNr, long amount) throws InvalidCredentials, InvalidTransaction {
Account account = accountRepository.getAccountByNr(accountNr);
account.deposit(amount);
accountRepository.saveAndFlush(account);
}
When I now perform the test, the SELECT's and UPDATES are confused, so that the correct value is not in the database at the end.
I then provided the service method with #Transactional(isolation = Isolation.READ_COMMITTED, propagation = Propagation.REQUIRES_NEW), which didn't help either.
Does anyone have an idea?
The log output after spring - banner is:
INFO 20320 --- [ main] o.e.b.a.AccountServiceConcurrentIT : Started AccountServiceConcurrentIT in 9.063 seconds (JVM running for 11.324)
DEBUG 20320 --- [ main] org.hibernate.SQL : select nextval ('hibernate_sequence')
DEBUG 20320 --- [ main] org.hibernate.SQL : insert into account (balance, pin, version, nr) values (?, ?, ?, ?)
DEBUG 20320 --- [ main] org.hibernate.SQL : select account0_.balance as col_0_0_ from account account0_ where account0_.nr=? and account0_.pin=?
DEBUG 20320 --- [pool-1-thread-8] org.hibernate.SQL : select account0_.nr as nr1_0_, account0_.balance as balance2_0_, account0_.pin as pin3_0_, account0_.version as version4_0_ from account account0_ where account0_.nr=?
DEBUG 20320 --- [pool-1-thread-2] org.hibernate.SQL : select account0_.nr as nr1_0_, account0_.balance as balance2_0_, account0_.pin as pin3_0_, account0_.version as version4_0_ from account account0_ where account0_.nr=?
DEBUG 20320 --- [pool-1-thread-6] org.hibernate.SQL : select account0_.nr as nr1_0_, account0_.balance as balance2_0_, account0_.pin as pin3_0_, account0_.version as version4_0_ from account account0_ where account0_.nr=?
DEBUG 20320 --- [pool-1-thread-9] org.hibernate.SQL : select account0_.nr as nr1_0_, account0_.balance as balance2_0_, account0_.pin as pin3_0_, account0_.version as version4_0_ from account account0_ where account0_.nr=?
DEBUG 20320 --- [pool-1-thread-2] org.hibernate.SQL : update account set balance=?, pin=?, version=? where nr=?
DEBUG 20320 --- [pool-1-thread-6] org.hibernate.SQL : update account set balance=?, pin=?, version=? where nr=?
DEBUG 20320 --- [pool-1-thread-8] org.hibernate.SQL : update account set balance=?, pin=?, version=? where nr=?
DEBUG 20320 --- [pool-1-thread-4] org.hibernate.SQL : select account0_.nr as nr1_0_, account0_.balance as balance2_0_, account0_.pin as pin3_0_, account0_.version as version4_0_ from account account0_ where account0_.nr=?
DEBUG 20320 --- [pool-1-thread-4] org.hibernate.SQL : update account set balance=?, pin=?, version=? where nr=?
DEBUG 20320 --- [pool-1-thread-9] org.hibernate.SQL : update account set balance=?, pin=?, version=? where nr=?
DEBUG 20320 --- [pool-1-thread-3] org.hibernate.SQL : select account0_.nr as nr1_0_, account0_.balance as balance2_0_, account0_.pin as pin3_0_, account0_.version as version4_0_ from account account0_ where account0_.nr=?
DEBUG 20320 --- [pool-1-thread-5] org.hibernate.SQL : select account0_.nr as nr1_0_, account0_.balance as balance2_0_, account0_.pin as pin3_0_, account0_.version as version4_0_ from account account0_ where account0_.nr=?
DEBUG 20320 --- [ool-1-thread-10] org.hibernate.SQL : select account0_.nr as nr1_0_, account0_.balance as balance2_0_, account0_.pin as pin3_0_, account0_.version as version4_0_ from account account0_ where account0_.nr=?
DEBUG 20320 --- [pool-1-thread-1] org.hibernate.SQL : select account0_.nr as nr1_0_, account0_.balance as balance2_0_, account0_.pin as pin3_0_, account0_.version as version4_0_ from account account0_ where account0_.nr=?
DEBUG 20320 --- [pool-1-thread-7] org.hibernate.SQL : select account0_.nr as nr1_0_, account0_.balance as balance2_0_, account0_.pin as pin3_0_, account0_.version as version4_0_ from account account0_ where account0_.nr=?
DEBUG 20320 --- [pool-1-thread-7] org.hibernate.SQL : update account set balance=?, pin=?, version=? where nr=?
DEBUG 20320 --- [ool-1-thread-10] org.hibernate.SQL : update account set balance=?, pin=?, version=? where nr=?
DEBUG 20320 --- [pool-1-thread-5] org.hibernate.SQL : update account set balance=?, pin=?, version=? where nr=?
DEBUG 20320 --- [pool-1-thread-3] org.hibernate.SQL : update account set balance=?, pin=?, version=? where nr=?
DEBUG 20320 --- [pool-1-thread-1] org.hibernate.SQL : update account set balance=?, pin=?, version=? where nr=?
DEBUG 20320 --- [ main] org.hibernate.SQL : select account0_.balance as col_0_0_ from account account0_ where account0_.nr=? and account0_.pin=?
java.lang.AssertionError:
Expected :10000
Actual :1000

You are using the ExecutorService to start 10 threads.
These 10 threads are running independently. That means there is no predictable order which thread runs first. As you can see in the log output:
[pool-1-thread-8]
[pool-1-thread-2]
[pool-1-thread-6]
[pool-1-thread-9]
[pool-1-thread-2]
[pool-1-thread-6]
[pool-1-thread-8]
[pool-1-thread-4]
[pool-1-thread-4]
[pool-1-thread-9]
[pool-1-thread-3]
[pool-1-thread-5]
[ool-1-thread-10]
[pool-1-thread-1]
[pool-1-thread-7]
[pool-1-thread-7]
[ool-1-thread-10]
[pool-1-thread-5]
[pool-1-thread-3]
[pool-1-thread-1]
To avoid overwriting of your account balance you must use pessimistic locking.
That can be achieved on the repository with a #Lock annotation:
#Lock(LockModeType.PESSIMISTIC_WRITE)
Account account = accountRepository.getAccountByNr(accountNr);
An you have to make sure that the deposit method is transactional to run all the code in the same transaction:
#Transactional
public void deposit(int accountNr, long amount) throws InvalidCredentials, InvalidTransaction {
Account account = accountRepository.getAccountByNr(accountNr);
account.deposit(amount);
accountRepository.saveAndFlush(account);
}
So every call to getAccountByNr will lock the record and the end of the transaction will release the lock.

Related

Hibernate not updating schema automatically

I am using SpringBoot with Hibernate. I have 2 Postgres datasources. I have an existing database (pims) and a new database (powwow), and some entities. When I start up SpringBoot, I would like it to automatically create the tables in the new powwow database, however it is not doing so.
application.properties
# pims datasource
spring.datasource1.driver-class-name=org.postgresql.Driver
spring.datasource1.jdbc-url=jdbc:postgresql://localhost:5432/pims
spring.datasource1.username=postgres
spring.datasource1.password=postgres
# powwow datasource
spring.datasource2.driver-class-name=org.postgresql.Driver
spring.datasource2.jdbc-url=jdbc:postgresql://localhost:5432/powwow
spring.datasource2.username=postgres
spring.datasource2.password=postgres
spring.jpa.properties.hibernate.dialect=org.hibernate.dialect.PostgreSQLDialect
spring.jpa.database-platform=postgres
spring.jpa.hibernate.ddl-auto=update
#spring.jpa.properties.hibernate.hbm2ddl.auto=update
#spring.jpa.generate-ddl=true
#spring.jpa.properties.hibernate.dialect=org.hibernate.dialect.PostgreSQLDialect
spring.datasource2.dbcp2.test-while-idle=true
spring.datasource2.dbcp2.validation-query=select 1
spring.jpa.hibernate.naming.implicit-strategy=org.hibernate.boot.model.naming.ImplicitNamingStrategyLegacyHbmImpl
spring.jpa.hibernate.naming.physical-strategy=org.springframework.boot.orm.jpa.hibernate.SpringPhysicalNamingStrategy
# logging
logging.level.org.hibernate.SQL=DEBUG
logging.level.org.hibernate.type.descriptor.sql.BasicBinder=TRACE
Example entity:
PowWowActivityEntity.java
#Entity
#Table(name = "powwowactivity")
public class PowWowActivityEntity {
#Id
#Column(name = "activity_id")
#GeneratedValue(strategy = GenerationType.IDENTITY)
private Long activity_id;
#Column(name = "status")
private String status;
Output
2022-08-16 09:19:13.227 INFO 65945 --- [ main] com.clubtravel.powwow.PowWowApplication : Started PowWowApplication in 11.544 seconds (JVM running for 12.617)
2022-08-16 09:19:13.255 DEBUG 65945 --- [ scheduling-1] org.hibernate.SQL : select count(*) as col_0_0_ from powwowglaccountmapping powwowglac0_
2022-08-16 09:19:13.269 DEBUG 65945 --- [ Async-1] org.hibernate.SQL : insert into powwowactivity (comment, create_date, status, user_name) values (?, ?, ?, ?)
2022-08-16 09:19:13.272 TRACE 65945 --- [ Async-1] o.h.type.descriptor.sql.BasicBinder : binding parameter [1] as [VARCHAR] - [Server started on HOST: Richards-MacBook-Pro.local.]
2022-08-16 09:19:13.273 TRACE 65945 --- [ Async-1] o.h.type.descriptor.sql.BasicBinder : binding parameter [2] as [TIMESTAMP] - [Tue Aug 16 09:19:13 SAST 2022]
2022-08-16 09:19:13.274 TRACE 65945 --- [ Async-1] o.h.type.descriptor.sql.BasicBinder : binding parameter [3] as [VARCHAR] - [Info]
2022-08-16 09:19:13.274 TRACE 65945 --- [ Async-1] o.h.type.descriptor.sql.BasicBinder : binding parameter [4] as [VARCHAR] - [system]
2022-08-16 09:19:13.281 WARN 65945 --- [ Async-1] o.h.engine.jdbc.spi.SqlExceptionHelper : SQL Error: 0, SQLState: 42P01
2022-08-16 09:19:13.281 ERROR 65945 --- [ Async-1] o.h.engine.jdbc.spi.SqlExceptionHelper : ERROR: relation "powwowactivity" does not exist
Question
I thought by adding the following line, the new tables would automatically be created by updating the schema. Any ideas? Is this not working because I have more than one datasource?
spring.jpa.hibernate.ddl-auto=update
The reason it was not working is because i have 2 datasources defined. If i change the config to update, it works (properties.put("hibernate.hbm2ddl.auto", "update");):
#Bean(name = "powwowEntityManager")
#Primary
public LocalContainerEntityManagerFactoryBean powwowEntityManager() {
LocalContainerEntityManagerFactoryBean em = new LocalContainerEntityManagerFactoryBean();
em.setDataSource(powwowDataSource());
em.setPackagesToScan(new String[] { "com.clubtravel.powwow.entities.powwow" });
HibernateJpaVendorAdapter vendorAdapter = new HibernateJpaVendorAdapter();
em.setJpaVendorAdapter(vendorAdapter);
HashMap<String, Object> properties = new HashMap<>();
// properties.put("hibernate.hbm2ddl.auto",env.getProperty("hibernate.hbm2ddl.auto"));
// properties.put("hibernate.dialect",env.getProperty("hibernate.dialect"));
properties.put("hibernate.hbm2ddl.auto", "update");
properties.put("hibernate.dialect", "org.hibernate.dialect.PostgreSQLDialect");
em.setJpaPropertyMap(properties);
logger.info("Setting spring.datasource2 (powwowEntityManager) hibernate.hbm2ddl.auto = "+env.getProperty("hibernate.hbm2ddl.auto")+" and hibernate.dialect = "+env.getProperty("hibernate.dialect"));
return em;
}

Spring Batch Multiple Threads, about the usage of throttleLimit

I see two threads running a job in my console when i set the param throttleLimit=1 and start spring batch project. But I only allow one thread to execute this job, why do have two threads to run this job?
codes for below
1、taskExecutor config
#Configuration
public class BatchTaskExecutor {
#Bean
public TaskExecutor taskExecutor() {
// 要加个名称,不然识别不了
SimpleAsyncTaskExecutor taskExecutor = new SimpleAsyncTaskExecutor("spring_batch");
// 最大线程数
taskExecutor.setConcurrencyLimit(Runtime.getRuntime().availableProcessors());
return taskExecutor;
}
}
2、step config
#Bean
public Step readMultiFormattedDatFileStep(StepBuilderFactory stepBuilderFactory,
ItemReader<CommonEntity> readMultiFormattedDatFileReader,
ItemWriter<CommonEntity> readMultiFormattedDatFileWriter,
TaskExecutor taskExecutor) {
return stepBuilderFactory.get("readMultiFormattedDatFileStep")
.<CommonEntity, CommonEntity>chunk(10)
.reader(readMultiFormattedDatFileReader)
.writer(readMultiFormattedDatFileWriter)
.taskExecutor(taskExecutor)
.throttleLimit(1)
.build();
}
3、console
2022-07-08 18:22:47.954 INFO 40286 --- [ main] o.s.batch.core.job.SimpleStepHandler : Executing step: [readMultiFormattedDatFileStep]
2022-07-08 18:22:47.984 INFO 40286 --- [ spring_batch1] p.x.batch.job.ReadMultiFormattedDatFile : skip header record, first date: 2022-07-06, second date: 2022-07-07
2022-07-08 18:22:48.000 INFO 40286 --- [ spring_batch2] p.x.batch.job.ReadMultiFormattedDatFile : u/i common entity record, id: 206, content: 神韵学Spring Batch
2022-07-08 18:22:48.002 INFO 40286 --- [ spring_batch2] p.x.batch.job.ReadMultiFormattedDatFile : u/i common entity record, id: 205, content: 神韵学Spring Batch
2022-07-08 18:22:48.003 INFO 40286 --- [ spring_batch2] p.x.batch.job.ReadMultiFormattedDatFile : u/i common entity record, id: 204, content: 神韵学Spring Batch
2022-07-08 18:22:48.003 INFO 40286 --- [ spring_batch2] p.x.batch.job.ReadMultiFormattedDatFile : u/i common entity record, id: 203, content: 神韵学Spring Batch
2022-07-08 18:22:48.004 INFO 40286 --- [ spring_batch2] p.x.batch.job.ReadMultiFormattedDatFile : u/i common entity record, id: 202, content: 神韵学Spring Batch
2022-07-08 18:22:48.004 INFO 40286 --- [ spring_batch2] p.x.batch.job.ReadMultiFormattedDatFile : u/i common entity record, id: 201, content: 神韵学Spring Batch
2022-07-08 18:22:48.005 INFO 40286 --- [ spring_batch2] p.x.batch.job.ReadMultiFormattedDatFile : u/i common entity record, id: 200, content: 神韵学Spring Batch
2022-07-08 18:22:48.005 INFO 40286 --- [ spring_batch2] p.x.batch.job.ReadMultiFormattedDatFile : u/i common entity record, id: 2, content: 神韵学Spring Batch2
2022-07-08 18:22:48.006 INFO 40286 --- [ spring_batch2] p.x.batch.job.ReadMultiFormattedDatFile : u/i common entity record, id: 1, content: 神韵学Spring Batch
2022-07-08 18:22:48.006 INFO 40286 --- [ spring_batch2] p.x.batch.job.ReadMultiFormattedDatFile : skip tail record, total count: 9
Hibernate: select commonenti0_.id as id1_0_0_, commonenti0_.content as content2_0_0_ from common commonenti0_ where commonenti0_.id=?
Hibernate: select commonenti0_.id as id1_0_0_, commonenti0_.content as content2_0_0_ from common commonenti0_ where commonenti0_.id=?
Hibernate: select commonenti0_.id as id1_0_0_, commonenti0_.content as content2_0_0_ from common commonenti0_ where commonenti0_.id=?
Hibernate: select commonenti0_.id as id1_0_0_, commonenti0_.content as content2_0_0_ from common commonenti0_ where commonenti0_.id=?
Hibernate: select commonenti0_.id as id1_0_0_, commonenti0_.content as content2_0_0_ from common commonenti0_ where commonenti0_.id=?
Hibernate: select commonenti0_.id as id1_0_0_, commonenti0_.content as content2_0_0_ from common commonenti0_ where commonenti0_.id=?
Hibernate: select commonenti0_.id as id1_0_0_, commonenti0_.content as content2_0_0_ from common commonenti0_ where commonenti0_.id=?
Hibernate: select commonenti0_.id as id1_0_0_, commonenti0_.content as content2_0_0_ from common commonenti0_ where commonenti0_.id=?
Hibernate: select commonenti0_.id as id1_0_0_, commonenti0_.content as content2_0_0_ from common commonenti0_ where commonenti0_.id=?
2022-07-08 18:22:48.153 INFO 40286 --- [ main] o.s.batch.core.step.AbstractStep : Step: [readMultiFormattedDatFileStep] executed in 198ms
2022-07-08 18:22:48.162 INFO 40286 --- [ main] o.s.b.c.l.support.SimpleJobLauncher : Job: [SimpleJob: [name=readMultiFormattedDatFileJob]] completed with the following parameters: [{test=1657275767774}] and the following status: [COMPLETED] in 236ms
2022-07-08 18:22:48.162 INFO 40286 --- [ main] pers.xue.batch.SpringBatchApplication : Job readMultiFormattedDatFileJob run end, exitCode: exitCode=COMPLETED;exitDescription=

Show log message about Rollback process

I am testing some rollback capabilities with Spring Data JDBC, but reviewing the logs from one test execution, I don´t see that log with something like "Rollback for xxx"
Log configuration:
logging.level.org.springframework.data=INFO
logging.level.org.springframework.jdbc.core.JdbcTemplate=TRACE
logging.level.org.springframework.transaction=TRACE
logging.level.org.springframework.transaction.interceptor=TRACE
Log output:
The logs represent a Service execution plus the execution of 2 EventListener executed in Serial. When the last Event Listener throw an Exception, I don´t see any log message about the Rollback process, but it is executed the rollback process under the hood in an effective way.
2022-01-04 09:36:47.685 INFO 11820 --- [o-auto-2-exec-1] c.d.b.application.BalanceServiceImpl : Witdhdraw 10.00 for customer 1
2022-01-04 09:36:47.689 TRACE 11820 --- [o-auto-2-exec-1] o.s.t.i.TransactionInterceptor : Getting transaction for [org.springframework.data.jdbc.repository.support.SimpleJdbcRepository.findById]
2022-01-04 09:36:47.776 DEBUG 11820 --- [o-auto-2-exec-1] o.s.jdbc.core.JdbcTemplate : Executing prepared SQL query
2022-01-04 09:36:47.777 DEBUG 11820 --- [o-auto-2-exec-1] o.s.jdbc.core.JdbcTemplate : Executing prepared SQL statement [SELECT "BALANCE"."BALANCE" AS "BALANCE", "BALANCE"."ID_BALANCE" AS "ID_BALANCE", "BALANCE"."ID_CUSTOMER" AS "ID_CUSTOMER", "BALANCE"."LAST_UPDATE" AS "LAST_UPDATE", "BALANCE"."WITHDRAW_LIMIT" AS "WITHDRAW_LIMIT" FROM "BALANCE" WHERE "BALANCE"."ID_BALANCE" = ?]
2022-01-04 09:36:47.850 TRACE 11820 --- [o-auto-2-exec-1] o.s.t.i.TransactionInterceptor : Completing transaction for [org.springframework.data.jdbc.repository.support.SimpleJdbcRepository.findById]
2022-01-04 09:36:47.853 TRACE 11820 --- [o-auto-2-exec-1] o.s.t.i.TransactionInterceptor : Getting transaction for [org.springframework.data.jdbc.repository.support.SimpleJdbcRepository.save]
2022-01-04 09:36:47.899 DEBUG 11820 --- [o-auto-2-exec-1] o.s.jdbc.core.JdbcTemplate : Executing prepared SQL update
2022-01-04 09:36:47.900 DEBUG 11820 --- [o-auto-2-exec-1] o.s.jdbc.core.JdbcTemplate : Executing prepared SQL statement [UPDATE "BALANCE" SET "BALANCE" = ?, "ID_CUSTOMER" = ?, "LAST_UPDATE" = ?, "WITHDRAW_LIMIT" = ? WHERE "BALANCE"."ID_BALANCE" = ?]
2022-01-04 09:36:47.913 TRACE 11820 --- [o-auto-2-exec-1] o.s.jdbc.core.JdbcTemplate : SQL update affected 1 rows
2022-01-04 09:36:47.918 TRACE 11820 --- [o-auto-2-exec-1] o.s.t.i.TransactionInterceptor : Completing transaction for [org.springframework.data.jdbc.repository.support.SimpleJdbcRepository.save]
2022-01-04 09:36:47.919 INFO 11820 --- [o-auto-2-exec-1] c.d.s.application.BalanceEventListener : New Event Received: MoneyWithDrewEvent Balance[balanceId=1, balance=90.00, customerId=null, lastUpdate=2022-01-04 09:36:47.852322144, withdrawLimit=null]
2022-01-04 09:36:47.933 INFO 11820 --- [o-auto-2-exec-1] c.d.s.application.BalanceEventListener2 : New Event Received: MoneyWithdrewEventStep2 Balance[balanceId=1, balance=90.00, customerId=null, lastUpdate=2022-01-04 09:36:47.852322144, withdrawLimit=null]
2022-01-04 09:36:47.934 TRACE 11820 --- [o-auto-2-exec-1] o.s.t.i.TransactionInterceptor : Completing transaction for [com.ddd.balance.application.BalanceServiceImpl.witdhdraw] after exception: java.lang.RuntimeException: Katakroker
Test:
//Initially the customer has a balance with 100
#Test
public void given_balanceController_when_withdraw_happy_path_Ok() {
//Given
String address2 = "http://localhost:" + port + "/api/withdraw";
BigDecimal amount = new BigDecimal("10.00");
WithdrawRequest widthDrawRequest = new WithdrawRequest(1L, amount);
HttpHeaders headers = new HttpHeaders();
HttpEntity<WithdrawRequest> request = new HttpEntity<>(widthDrawRequest, headers);
//When
ResponseEntity<String> result2 = this.restTemplate.postForEntity(address2, request, String.class);
//Then
WithdrawResponse expectedResponse = new WithdrawResponse(false);
then(result2.getStatusCode()).isEqualTo(HttpStatus.OK);
//then(result2.getBody()).contains(asJsonString(expectedResponse));
//Internally raise an Exception
//Step Verify
//Given
String address3 = "http://localhost:" + port + "/api/balance/1";
//When
ResponseEntity<BalanceResponse> result3 = this.restTemplate.getForEntity(address3, BalanceResponse.class);
//Then
then(result3.getStatusCode()).isEqualTo(HttpStatus.OK);
then(result3.getBody().amount()).isEqualTo(new BigDecimal("100.00"));
}
Repository example:
https://github.com/jabrena/101-ddd/blob/feature/balance-iteration2/card-operations-example/src/test/java/com/ddd/balance/infrastructure/rest/BalanceControllerE2ETest.java
Spring Data JDBC execute the rollback internally but I don´t see the traces.
How to enable it?
I was able to see the log with the following configuration:
logging.level.org.springframework.jdbc.support.JdbcTransactionManager=TRACE
Log message:
2022-01-04 10:59:08.417 DEBUG 2951 --- [o-auto-2-exec-1] o.s.jdbc.support.JdbcTransactionManager : Creating new transaction with name [com.ddd.balance.application.BalanceServiceImpl.witdhdraw]: PROPAGATION_REQUIRED,ISOLATION_DEFAULT
2022-01-04 10:59:08.418 DEBUG 2951 --- [o-auto-2-exec-1] o.s.jdbc.support.JdbcTransactionManager : Acquired Connection [HikariProxyConnection#1551466954 wrapping conn10: url=jdbc:h2:mem:cf10c02a-8c7f-4667-b14a-9de91f7651a9 user=SA] for JDBC transaction
2022-01-04 10:59:08.420 DEBUG 2951 --- [o-auto-2-exec-1] o.s.jdbc.support.JdbcTransactionManager : Switching JDBC Connection [HikariProxyConnection#1551466954 wrapping conn10: url=jdbc:h2:mem:cf10c02a-8c7f-4667-b14a-9de91f7651a9 user=SA] to manual commit
2022-01-04 10:59:08.421 TRACE 2951 --- [o-auto-2-exec-1] o.s.t.i.TransactionInterceptor : Getting transaction for [com.ddd.balance.application.BalanceServiceImpl.witdhdraw]
2022-01-04 10:59:08.447 INFO 2951 --- [o-auto-2-exec-1] c.d.b.application.BalanceServiceImpl : Witdhdraw 10.00 for customer 1
2022-01-04 10:59:08.449 DEBUG 2951 --- [o-auto-2-exec-1] o.s.jdbc.support.JdbcTransactionManager : Participating in existing transaction
2022-01-04 10:59:08.450 TRACE 2951 --- [o-auto-2-exec-1] o.s.t.i.TransactionInterceptor : Getting transaction for [org.springframework.data.jdbc.repository.support.SimpleJdbcRepository.findById]
2022-01-04 10:59:08.521 DEBUG 2951 --- [o-auto-2-exec-1] o.s.jdbc.core.JdbcTemplate : Executing prepared SQL query
2022-01-04 10:59:08.522 DEBUG 2951 --- [o-auto-2-exec-1] o.s.jdbc.core.JdbcTemplate : Executing prepared SQL statement [SELECT "BALANCE"."BALANCE" AS "BALANCE", "BALANCE"."ID_BALANCE" AS "ID_BALANCE", "BALANCE"."ID_CUSTOMER" AS "ID_CUSTOMER", "BALANCE"."LAST_UPDATE" AS "LAST_UPDATE", "BALANCE"."WITHDRAW_LIMIT" AS "WITHDRAW_LIMIT" FROM "BALANCE" WHERE "BALANCE"."ID_BALANCE" = ?]
2022-01-04 10:59:08.585 TRACE 2951 --- [o-auto-2-exec-1] o.s.t.i.TransactionInterceptor : Completing transaction for [org.springframework.data.jdbc.repository.support.SimpleJdbcRepository.findById]
2022-01-04 10:59:08.587 DEBUG 2951 --- [o-auto-2-exec-1] o.s.jdbc.support.JdbcTransactionManager : Participating in existing transaction
2022-01-04 10:59:08.588 TRACE 2951 --- [o-auto-2-exec-1] o.s.t.i.TransactionInterceptor : Getting transaction for [org.springframework.data.jdbc.repository.support.SimpleJdbcRepository.save]
2022-01-04 10:59:08.628 DEBUG 2951 --- [o-auto-2-exec-1] o.s.jdbc.core.JdbcTemplate : Executing prepared SQL update
2022-01-04 10:59:08.629 DEBUG 2951 --- [o-auto-2-exec-1] o.s.jdbc.core.JdbcTemplate : Executing prepared SQL statement [UPDATE "BALANCE" SET "BALANCE" = ?, "ID_CUSTOMER" = ?, "LAST_UPDATE" = ?, "WITHDRAW_LIMIT" = ? WHERE "BALANCE"."ID_BALANCE" = ?]
2022-01-04 10:59:08.648 TRACE 2951 --- [o-auto-2-exec-1] o.s.jdbc.core.JdbcTemplate : SQL update affected 1 rows
2022-01-04 10:59:08.653 TRACE 2951 --- [o-auto-2-exec-1] o.s.t.i.TransactionInterceptor : Completing transaction for [org.springframework.data.jdbc.repository.support.SimpleJdbcRepository.save]
2022-01-04 10:59:08.655 INFO 2951 --- [o-auto-2-exec-1] c.d.s.application.BalanceEventListener : New Event Received: MoneyWithDrewEvent Balance[balanceId=1, balance=90.00, customerId=null, lastUpdate=2022-01-04 10:59:08.586936363, withdrawLimit=null]
2022-01-04 10:59:08.675 INFO 2951 --- [o-auto-2-exec-1] c.d.s.application.BalanceEventListener2 : New Event Received: MoneyWithdrewEventStep2 Balance[balanceId=1, balance=90.00, customerId=null, lastUpdate=2022-01-04 10:59:08.586936363, withdrawLimit=null]
2022-01-04 10:59:08.676 TRACE 2951 --- [o-auto-2-exec-1] o.s.t.i.TransactionInterceptor : Completing transaction for [com.ddd.balance.application.BalanceServiceImpl.witdhdraw] after exception: java.lang.RuntimeException: Katakroker
2022-01-04 10:59:08.676 DEBUG 2951 --- [o-auto-2-exec-1] o.s.jdbc.support.JdbcTransactionManager : Initiating transaction rollback
2022-01-04 10:59:08.676 DEBUG 2951 --- [o-auto-2-exec-1] o.s.jdbc.support.JdbcTransactionManager : Rolling back JDBC transaction on Connection [HikariProxyConnection#1551466954 wrapping conn10: url=jdbc:h2:mem:cf10c02a-8c7f-4667-b14a-9de91f7651a9 user=SA]
2022-01-04 10:59:08.678 DEBUG 2951 --- [o-auto-2-exec-1] o.s.jdbc.support.JdbcTransactionManager : Releasing JDBC Connection [HikariProxyConnection#1551466954 wrapping conn10: url=jdbc:h2:mem:cf10c02a-8c7f-4667-b14a-9de91f7651a9 user=SA] after transaction
you can use
logging.level.org.springframework.orm.jpa=TRACE
logging.level.org.springframework.transaction=TRACE

Exposing JPA transaction takes too much time

I worked with SpringBoot 1.5.10.RELEASE, Postgre Server 9.6, Driver version is 9.4.1212.jre7, tomcat-jdbc (version of spring bom).
Here is configuration JPA :
spring.datasource.driver-class-name=org.postgresql.Driver
spring.jpa.hibernate.ddl-auto=none
spring.jpa.database-platform=org.hibernate.dialect.PostgreSQLDialect
The Java code is :
AssureMaladieController.java
#BasePathAwareController
public class AssureMaladieController {
#Autowired
IAssureMaladieService assureMaladieService;
#Autowired
RepositoryEntityLinks repositoryEntityLinks;
#RequestMapping(value = "/assures/{id}/reglements/find", method = RequestMethod.GET)
#ResponseBody
public ResponseEntity<AbstractSupport> getReglements(#PathVariable Long id, #RequestParam EnumTypeReglement typeReglement, #RequestParam #DateTimeFormat(pattern = "yyyy-MM-dd") Date date ) {
log.trace("entrée dans getReglements")
List<Reglement> reglements =assureMaladieService.getReglementsAfter(id,typeReglement, date);
}
AssureMaladieService.java
#Service
public class AssureMaladieService implements IAssureMaladieService{
#Autowired
private ReglementRepository reglementRepository;
#Transactional
public List<Reglement> getReglementsAfter(Long idAssure, EnumTypeReglement typeReglement, Date dateDebut){
log.trace("entrée dans getReglementsAfter")
if (personnePhysiqueRepository.findOne(idAssure)== null){
throw new RessourceNonTrouveeException();
}
List<Reglement> reglements= reglementRepository.findByAssureMaladie_IdAndTypeReglement_CodeAndDateGreaterThanEqualOrderByDateDesc(idAssure,typeReglement,dateDebut);
return reglements;
}
ReglementRepository.java
import org.springframework.data.repository.CrudRepository;
#RepositoryRestResource(path="reglements")
public interface ReglementRepository extends CrudRepository<Reglement,Long> {
#Override
#PostAuthorize("(returnObject!=null?hasPermission(returnObject.assureMaladie.matricule, 'maladie','R'):true)")
Reglement findOne(#Param("id")Long var1);
}
Sometimes, JPA takes several minutes to pass from first class to second
On an anormal case, logs are :
2018-09-13 14:36:54.826 DEBUG 21979 --- [http-nio-8001-exec-9] n.c.g.c.i.c.AssureMaladieController : entrée dans getReglements avec les id : 484608 type règlement : ASSURE date : Mon Aug 13 00:00:00 NCT 2018
2018-09-13 14:36:54.826 DEBUG 21979 --- [http-nio-8001-exec-9] o.s.orm.jpa.JpaTransactionManager : Creating new transaction with name [nc.cafat.gen.cli.impl.service.AssureMaladieService.getReglementsAfter]: PROPAGATION_REQUIRED,ISOLATION_DEFAULT; ''
2018-09-13 14:36:54.826 DEBUG 21979 --- [http-nio-8001-exec-9] o.s.orm.jpa.JpaTransactionManager : Opened new EntityManager [org.hibernate.jpa.internal.EntityManagerImpl#1cf1ad2a] for JPA transaction
2018-09-13 14:52:32.272 DEBUG 21979 --- [http-nio-8001-exec-9] o.s.orm.jpa.JpaTransactionManager : Exposing JPA transaction as JDBC transaction [org.springframework.orm.jpa.vendor.HibernateJpaDialect$HibernateConnectionHandle#665fa46c]
2018-09-13 14:52:32.273 TRACE 21979 --- [http-nio-8001-exec-9] .s.t.s.TransactionSynchronizationManager : Bound value [org.springframework.jdbc.datasource.ConnectionHolder#7329c08c] for key [org.apache.tomcat.jdbc.pool.DataSource#4165b803{ConnectionPool[defaultAutoCommit=null; defaultReadOnly=null; defaultTransactionIsolation=-1; defaultCatalog=null; driverClassName=org.postgresql.Driver; maxActive=100; maxIdle=100; minIdle=10; initialSize=10; maxWait=30000; testOnBorrow=true; testOnReturn=false; timeBetweenEvictionRunsMillis=5000; numTestsPerEvictionRun=0; minEvictableIdleTimeMillis=60000; testWhileIdle=false; testOnConnect=false; password=********; url=jdbc:postgresql://DBPG-PRD-80:5432/prd_cafat_01; username=prdcafatuser; validationQuery=SELECT 1; validationQueryTimeout=-1; validatorClassName=null; validationInterval=3000; accessToUnderlyingConnectionAllowed=true; removeAbandoned=false; removeAbandonedTimeout=60; logAbandoned=false; connectionProperties=null; initSQL=null; jdbcInterceptors=null; jmxEnabled=true; fairQueue=true; useEquals=true; abandonWhenPercentageFull=0; maxAge=0; useLock=false; dataSource=null; dataSourceJNDI=null; suspectTimeout=0; alternateUsernameAllowed=false; commitOnReturn=false; rollbackOnReturn=false; useDisposableConnectionFacade=true; logValidationErrors=false; propagateInterruptState=false; ignoreExceptionOnPreLoad=false; useStatementFacade=true; }] to thread [http-nio-8001-exec-9]
2018-09-13 14:52:32.273 TRACE 21979 --- [http-nio-8001-exec-9] .s.t.s.TransactionSynchronizationManager : Bound value [org.springframework.orm.jpa.EntityManagerHolder#189284e4] for key [org.springframework.orm.jpa.LocalContainerEntityManagerFactoryBean#30d8f1e5] to thread [http-nio-8001-exec-9]
2018-09-13 14:52:32.273 TRACE 21979 --- [http-nio-8001-exec-9] .s.t.s.TransactionSynchronizationManager : Initializing transaction synchronization
2018-09-13 14:52:32.273 TRACE 21979 --- [http-nio-8001-exec-9] o.s.t.i.TransactionInterceptor : Getting transaction for [nc.cafat.gen.cli.impl.service.AssureMaladieService.getReglementsAfter]
2018-09-13 14:52:32.273 DEBUG 21979 --- [http-nio-8001-exec-9] n.c.g.c.i.service.AssureMaladieService : entrée dans getReglementsAfter avec les idAssure : 484608 type règlement : ASSURE dateDebut : Mon Aug 13 00:00:00 NCT 2018
More than 15 minutes !
On a normal case, logs are :
2018-09-13 14:56:47.796 DEBUG 21979 --- [http-nio-8001-exec-6] n.c.g.c.i.c.AssureMaladieController : entrée dans getReglements avec les id : 484608 type règlement : ASSURE date : Mon Aug 13 00:00:00 NCT 2018
2018-09-13 14:56:47.796 DEBUG 21979 --- [http-nio-8001-exec-6] o.s.orm.jpa.JpaTransactionManager : Creating new transaction with name [nc.cafat.gen.cli.impl.service.AssureMaladieService.getReglementsAfter]: PROPAGATION_REQUIRED,ISOLATION_DEFAULT; ''
2018-09-13 14:56:47.797 DEBUG 21979 --- [http-nio-8001-exec-6] o.s.orm.jpa.JpaTransactionManager : Opened new EntityManager [org.hibernate.jpa.internal.EntityManagerImpl#36b59887] for JPA transaction
2018-09-13 14:56:47.797 DEBUG 21979 --- [http-nio-8001-exec-6] o.s.orm.jpa.JpaTransactionManager : Exposing JPA transaction as JDBC transaction
2018-09-13 14:56:47.798 TRACE 21979 --- [http-nio-8001-exec-6] .s.t.s.TransactionSynchronizationManager : Bound value [org.springframework.jdbc.datasource.ConnectionHolder#62337496] for key [org.apache.tomcat.jdbc.pool.DataSource#4165b803{ConnectionPool[defaultAutoCommit=null; defaultReadOnly=null; defaultTransactionIsolation=-1; defaultCatalog=null; driverClassName=org.postgresql.Driver; maxActive=100; maxIdle=100; minIdle=10; initialSize=10; maxWait=30000; testOnBorrow=true; testOnReturn=false; timeBetweenEvictionRunsMillis=5000; numTestsPerEvictionRun=0; minEvictableIdleTimeMillis=60000; testWhileIdle=false; testOnConnect=false; ; username=prdcafatuser; validationQuery=SELECT 1; validationQueryTimeout=-1; validatorClassName=null; validationInterval=3000; accessToUnderlyingConnectionAllowed=true; removeAbandoned=false; removeAbandonedTimeout=60; logAbandoned=false; connectionProperties=null; initSQL=null; jdbcInterceptors=null; jmxEnabled=true; fairQueue=true; useEquals=true; abandonWhenPercentageFull=0; maxAge=0; useLock=false; dataSource=null; dataSourceJNDI=null; suspectTimeout=0; alternateUsernameAllowed=false; commitOnReturn=false; rollbackOnReturn=false; useDisposableConnectionFacade=true; logValidationErrors=false; propagateInterruptState=false; ignoreExceptionOnPreLoad=false; useStatementFacade=true; }] to thread [http-nio-8001-exec-6]
2018-09-13 14:56:47.798 TRACE 21979 --- [http-nio-8001-exec-6] .s.t.s.TransactionSynchronizationManager : Bound value [org.springframework.orm.jpa.EntityManagerHolder#4961b514] for key [org.springframework.orm.jpa.LocalContainerEntityManagerFactoryBean#30d8f1e5] to thread [http-nio-8001-exec-6]
2018-09-13 14:56:47.798 TRACE 21979 --- [http-nio-8001-exec-6] .s.t.s.TransactionSynchronizationManager : Initializing transaction synchronization
2018-09-13 14:56:47.798 TRACE 21979 --- [http-nio-8001-exec-6] o.s.t.i.TransactionInterceptor : Getting transaction for [nc.cafat.gen.cli.impl.service.AssureMaladieService.getReglementsAfter]
2018-09-13 14:56:47.798 DEBUG 21979 --- [http-nio-8001-exec-6] n.c.g.c.i.service.AssureMaladieService : entrée dans getReglementsAfter avec les idAssure : 484608 type règlement : ASSURE dateDebut : Mon Aug 13 00:00:00 NCT 2018
Visual VM confirms that thread is taking 15 minutes in CPU (945160 ~ 15m45s)
CPU time
The line taking more part is by the driver on socket read :
Socket read
This posts seems to relate the same issue : https://github.com/brettwooldridge/HikariCP/issues/603
We are trying this :
switch from tomcat-jdbc to hikari 3.2.0 (seems to have a fix on this
since 2.6.2)
updatejdbc PG driver from 9.4 to 42.2.5
(It would have been easier to upgrade to spring 2.0 but we're facing an other problem to migrate)

With in-mem database for testing, entity manager not releasing locks 'unable to obtain lock'

I have a Spring Boot app with some integration tests inspecting the results from front-end operations on data in the database.
The app uses DataNucleus JPA underneath spring-data-jpa and spring-data-rest, with an in-memory database, e.g. Derby, set up automatically via Spring Boot testing.
I used to use Hibernate, but I swopped it for DataNucleus. The tests all passed with Hibernate, but now my test JdbcTemplate queries are hanging, as though JPA isn't releasing its locks.
I've tried it with H2 (fails silently), Derby (hangs until time-out) and HSQLDB (hangs forever).
I have tried various work-arounds, e.g. without transactions #Transactional(Transactional.TxType.NEVER) or with/without commits #Rollback(true/false)
Spring Boot instantiates the datasource automatically and injects it into the EntityManagerFactory and the JdbcTemplate.
This is the test:
#ExtendWith(SpringExtension.class)
#SpringBootTest(webEnvironment = SpringBootTest.WebEnvironment.MOCK,
classes = { TestDataSourceConfig.class })
#EnableAutoConfiguration
#AutoConfigureMockMvc
#AutoConfigureTestDatabase(connection = EmbeddedDatabaseConnection.DERBY)
#Transactional
public class SymbolRestTests {
#Autowired
private MockMvc mockMvc;
#Autowired
private JdbcTemplate jdbcTemplate;
#Autowired
private SymbolRepository symbolRepository;
#PersistenceContext
private EntityManager entityManager;
#Before
public void setUp() throws Exception {
symbolRepository.deleteAll();
entityManager.flush();
entityManager.clear();
}
#Test
public void shouldCreateEntity() throws Exception {
String testTitle = "TEST.CODE.1";
String testExtra = "Test for SymbolRestTests.java";
String json = createJsonExample(testTitle, testExtra, true);
MockHttpServletRequestBuilder requestBuilder =
post("/symbols").content(json);
mockMvc.perform(requestBuilder)
.andExpect(status().isCreated())
.andExpect(header().string("Location",
containsString("symbols/")));
entityManager.flush();
entityManager.close(); // this didn't help
String sql = "SELECT count(*) FROM symbol WHERE title = ?";
// exception thrown on this next line
int count = jdbcTemplate.queryForObject(
sql, new Object[] { testTitle }, Integer.class);
Assert.assertThat(count, is(1));
}
}
and this is the error from HSQLDB (seems to be the most informative):
org.springframework.dao.CannotAcquireLockException: PreparedStatementCallback;
SQL [SELECT count(*) FROM symbol WHERE title = ?];
A lock could not be obtained within the time requested;
nested exception is java.sql.SQLTransactionRollbackException:
A lock could not be obtained within the time requested
at org.springframework.jdbc.support.SQLErrorCodeSQLExceptionTranslator.doTranslate(SQLErrorCodeSQLExceptionTranslator.java:259)
at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:73)
at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:649)
at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:684)
at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:716)
at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:726)
at org.springframework.jdbc.core.JdbcTemplate.queryForObject(JdbcTemplate.java:794)
at org.springframework.jdbc.core.JdbcTemplate.queryForObject(JdbcTemplate.java:813)
at com.gis.integration.SymbolRestTests.shouldCreateEntity(SymbolRestTests.java:127)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.junit.platform.commons.util.ReflectionUtils.invokeMethod(ReflectionUtils.java:316)
at org.junit.jupiter.engine.execution.ExecutableInvoker.invoke(ExecutableInvoker.java:114)
at org.junit.jupiter.engine.descriptor.MethodTestDescriptor.lambda$invokeTestMethod$6(MethodTestDescriptor.java:171)
at org.junit.jupiter.engine.execution.ThrowableCollector.execute(ThrowableCollector.java:40)
at org.junit.jupiter.engine.descriptor.MethodTestDescriptor.invokeTestMethod(MethodTestDescriptor.java:168)
at org.junit.jupiter.engine.descriptor.MethodTestDescriptor.execute(MethodTestDescriptor.java:115)
at org.junit.jupiter.engine.descriptor.MethodTestDescriptor.execute(MethodTestDescriptor.java:57)
at org.junit.platform.engine.support.hierarchical.HierarchicalTestExecutor.lambda$execute$1(HierarchicalTestExecutor.java:81)
at org.junit.platform.engine.support.hierarchical.SingleTestExecutor.executeSafely(SingleTestExecutor.java:66)
at org.junit.platform.engine.support.hierarchical.HierarchicalTestExecutor.execute(HierarchicalTestExecutor.java:76)
at org.junit.platform.engine.support.hierarchical.HierarchicalTestExecutor.lambda$execute$1(HierarchicalTestExecutor.java:91)
at org.junit.platform.engine.support.hierarchical.SingleTestExecutor.executeSafely(SingleTestExecutor.java:66)
at org.junit.platform.engine.support.hierarchical.HierarchicalTestExecutor.execute(HierarchicalTestExecutor.java:76)
at org.junit.platform.engine.support.hierarchical.HierarchicalTestExecutor.lambda$execute$1(HierarchicalTestExecutor.java:91)
at org.junit.platform.engine.support.hierarchical.SingleTestExecutor.executeSafely(SingleTestExecutor.java:66)
at org.junit.platform.engine.support.hierarchical.HierarchicalTestExecutor.execute(HierarchicalTestExecutor.java:76)
at org.junit.platform.engine.support.hierarchical.HierarchicalTestExecutor.execute(HierarchicalTestExecutor.java:51)
at org.junit.platform.engine.support.hierarchical.HierarchicalTestEngine.execute(HierarchicalTestEngine.java:43)
at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:137)
at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:87)
at org.junit.platform.launcher.Launcher.execute(Launcher.java:93)
at com.intellij.junit5.JUnit5IdeaTestRunner.startRunnerWithArgs(JUnit5IdeaTestRunner.java:61)
at com.intellij.rt.execution.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:51)
at com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:242)
at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:70)
Caused by: java.sql.SQLTransactionRollbackException: A lock could not be obtained within the time requested
at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
at org.apache.derby.impl.jdbc.Util.generateCsSQLException(Unknown Source)
at org.apache.derby.impl.jdbc.TransactionResourceImpl.wrapInSQLException(Unknown Source)
at org.apache.derby.impl.jdbc.TransactionResourceImpl.handleException(Unknown Source)
at org.apache.derby.impl.jdbc.EmbedConnection.handleException(Unknown Source)
at org.apache.derby.impl.jdbc.ConnectionChild.handleException(Unknown Source)
at org.apache.derby.impl.jdbc.EmbedResultSet.closeOnTransactionError(Unknown Source)
at org.apache.derby.impl.jdbc.EmbedResultSet.movePosition(Unknown Source)
at org.apache.derby.impl.jdbc.EmbedResultSet.next(Unknown Source)
at org.springframework.jdbc.core.RowMapperResultSetExtractor.extractData(RowMapperResultSetExtractor.java:92)
at org.springframework.jdbc.core.RowMapperResultSetExtractor.extractData(RowMapperResultSetExtractor.java:60)
at org.springframework.jdbc.core.JdbcTemplate$1.doInPreparedStatement(JdbcTemplate.java:697)
at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:633)
... 35 more
Caused by: ERROR 40XL1: A lock could not be obtained within the time requested
at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
at org.apache.derby.impl.services.locks.ConcurrentLockSet.lockObject(Unknown Source)
at org.apache.derby.impl.services.locks.ConcurrentLockSet.zeroDurationLockObject(Unknown Source)
at org.apache.derby.impl.services.locks.AbstractPool.zeroDurationlockObject(Unknown Source)
at org.apache.derby.impl.services.locks.ConcurrentPool.zeroDurationlockObject(Unknown Source)
at org.apache.derby.impl.store.raw.xact.RowLocking2nohold.lockRecordForRead(Unknown Source)
at org.apache.derby.impl.store.access.heap.HeapController.lockRow(Unknown Source)
at org.apache.derby.impl.store.access.heap.HeapController.lockRow(Unknown Source)
at org.apache.derby.impl.store.access.btree.index.B2IRowLocking3.lockRowOnPage(Unknown Source)
at org.apache.derby.impl.store.access.btree.index.B2IRowLocking3._lockScanRow(Unknown Source)
at org.apache.derby.impl.store.access.btree.index.B2IRowLockingRR.lockScanRow(Unknown Source)
at org.apache.derby.impl.store.access.btree.BTreeForwardScan.fetchRows(Unknown Source)
at org.apache.derby.impl.store.access.btree.BTreeScan.fetchNextGroup(Unknown Source)
at org.apache.derby.impl.sql.execute.BulkTableScanResultSet.reloadArray(Unknown Source)
at org.apache.derby.impl.sql.execute.BulkTableScanResultSet.getNextRowCore(Unknown Source)
at org.apache.derby.impl.sql.execute.ProjectRestrictResultSet.getNextRowCore(Unknown Source)
at org.apache.derby.impl.sql.execute.ScalarAggregateResultSet.getRowFromResultSet(Unknown Source)
at org.apache.derby.impl.sql.execute.ScalarAggregateResultSet.getNextRowCore(Unknown Source)
at org.apache.derby.impl.sql.execute.ProjectRestrictResultSet.getNextRowCore(Unknown Source)
at org.apache.derby.impl.sql.execute.BasicNoPutResultSetImpl.getNextRow(Unknown Source)
... 41 more
Update using DataNucleus transaction documentation I have added some DataNucleus-specific properties to the persistence.xml (without result):
<?xml version="1.0" encoding="UTF-8" ?>
<persistence version="2.1"
xmlns="http://xmlns.jcp.org/xml/ns/persistence"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://xmlns.jcp.org/xml/ns/persistence
persistence_2_1.xsd">
<persistence-unit name="test">
<provider>org.datanucleus.api.jpa.PersistenceProviderImpl</provider>
<exclude-unlisted-classes>false</exclude-unlisted-classes>
<properties>
<property name="datanucleus.schema.autoCreateAll" value="true"/>
<property name="datanucleus.transactionIsolation" value="read-uncommitted"/>
<property name="datanucleus.NontransactionalRead" value="true"/>
<property name="datanucleus.NontransactionalWrite" value="true"/>
</properties>
</persistence-unit>
</persistence>
UPDATE #2
The log output at DEBUG level showing the DataNucleus-JPA/JdbcTemplate log statements (first the JPA INSERT, then the JdbcTemplate SELECT COUNT(*)):
2017-06-09 14:56:18.055 DEBUG 9492 --- [ main] DataNucleus.Connection : ManagedConnection(non-enlisted) "org.datanucleus.store.rdbms.ConnectionFactoryImpl$ManagedConnectionImpl#5621a671 [conn=org.apache.derby.impl.jdbc.EmbedConnection#2006fdaa, commitOnRelease=true, closeOnRelease=true, closeOnTxnEnd=true]" is being committed.
2017-06-09 14:56:18.055 DEBUG 9492 --- [ main] DataNucleus.Connection : ManagedConnection(non-enlisted) "org.datanucleus.store.rdbms.ConnectionFactoryImpl$ManagedConnectionImpl#5621a671 [conn=org.apache.derby.impl.jdbc.EmbedConnection#2006fdaa, commitOnRelease=true, closeOnRelease=true, closeOnTxnEnd=true]" closed
2017-06-09 14:56:18.068 DEBUG 9492 --- [ main] DataNucleus.Persistence : Object "com.bp.gis.tardis.entity.SymbolEntity#59c08cf1" being inserted into table "SYMBOL"
2017-06-09 14:56:18.071 DEBUG 9492 --- [ main] DataNucleus.Connection : ManagedConnection(non-enlisted) "org.datanucleus.store.rdbms.ConnectionFactoryImpl$ManagedConnectionImpl#63a7af06 [conn=null, commitOnRelease=false, closeOnRelease=false, closeOnTxnEnd=true]" opened with isolation level "read-uncommitted" and auto-commit=false
2017-06-09 14:56:18.074 DEBUG 9492 --- [ main] DataNucleus.Transaction : Running enlist operation on resource: org.datanucleus.store.rdbms.ConnectionFactoryImpl$EmulatedXAResource#3d4b45b, error code TMNOFLAGS and transaction: [DataNucleus Transaction, ID=Xid= , enlisted resources=[]]
2017-06-09 14:56:18.076 DEBUG 9492 --- [ main] DataNucleus.Connection : ManagedConnection(enlisted) "org.datanucleus.store.rdbms.ConnectionFactoryImpl$ManagedConnectionImpl#63a7af06 [conn=org.apache.derby.impl.jdbc.EmbedConnection#19d76106, commitOnRelease=false, closeOnRelease=false, closeOnTxnEnd=true]" starting for transaction "Xid= " with flags "0"
2017-06-09 14:56:18.078 DEBUG 9492 --- [ main] DataNucleus.Connection : ManagedConnection added to the pool : "org.datanucleus.store.rdbms.ConnectionFactoryImpl$ManagedConnectionImpl#63a7af06 [conn=org.apache.derby.impl.jdbc.EmbedConnection#19d76106, commitOnRelease=false, closeOnRelease=false, closeOnTxnEnd=true]" for key="org.datanucleus.ExecutionContextImpl#2c47a053" in factory="ConnectionFactory:tx[org.datanucleus.store.rdbms.ConnectionFactoryImpl#204c5ddf]"
2017-06-09 14:56:18.125 DEBUG 9492 --- [ main] DataNucleus.Datastore : Using PreparedStatement "org.datanucleus.store.rdbms.ParamLoggingPreparedStatement#37753b69" for connection "org.apache.derby.impl.jdbc.EmbedConnection#19d76106"
2017-06-09 14:56:18.134 DEBUG 9492 --- [ main] DataNucleus.Datastore.Native : INSERT INTO SYMBOL (ACTIVE,CREATED,EXTRA,GLOBAL_READ,GLOBAL_WRITE,LAST_MODIFIED,TITLE) VALUES (<'Y'>,<2017-06-09>,<null>,<'N'>,<'N'>,<2017-06-09>,<'TEST.CODE.1'>)
2017-06-09 14:56:18.169 DEBUG 9492 --- [ main] DataNucleus.Datastore.Persist : Execution Time = 36 ms (number of rows = 1) on PreparedStatement "org.datanucleus.store.rdbms.ParamLoggingPreparedStatement#37753b69"
2017-06-09 14:56:18.172 DEBUG 9492 --- [ main] DataNucleus.Datastore.Persist : Object "com.bp.gis.tardis.entity.SymbolEntity#59c08cf1" was inserted in the datastore and was given strategy value of "1"
2017-06-09 14:56:18.179 DEBUG 9492 --- [ main] DataNucleus.Cache : Object "com.bp.gis.tardis.entity.SymbolEntity#59c08cf1" (id="org.datanucleus.identity.IdentityReference#93fb44") being changed to be referenced by id="com.bp.gis.tardis.entity.SymbolEntity:1" in Level 1 cache
2017-06-09 14:56:18.180 DEBUG 9492 --- [ main] DataNucleus.Cache : Object "com.bp.gis.tardis.entity.SymbolEntity#59c08cf1" (id="com.bp.gis.tardis.entity.SymbolEntity:1") added to Level 1 cache (loadedFlags="[YYYYYYYYY]")
2017-06-09 14:56:18.180 DEBUG 9492 --- [ main] DataNucleus.Transaction : Object "com.bp.gis.tardis.entity.SymbolEntity#59c08cf1" (id="org.datanucleus.identity.IdentityReference#93fb44") enlisted in transactional cache is now enlisted using id="com.bp.gis.tardis.entity.SymbolEntity:1"
2017-06-09 14:56:18.181 DEBUG 9492 --- [ main] DataNucleus.Persistence : Insert of object "com.bp.gis.tardis.entity.SymbolEntity#59c08cf1" is calling insertPostProcessing for field "com.bp.gis.tardis.entity.SymbolEntity.timeSeriesList"
2017-06-09 14:56:18.181 DEBUG 9492 --- [ main] DataNucleus.Datastore : Closing PreparedStatement "org.datanucleus.store.rdbms.ParamLoggingPreparedStatement#37753b69"
2017-06-09 14:56:18.181 DEBUG 9492 --- [ main] DataNucleus.Persistence : Insert of object "com.bp.gis.tardis.entity.SymbolEntity#59c08cf1" is calling postInsert for field "com.bp.gis.tardis.entity.SymbolEntity.timeSeriesList"
2017-06-09 14:56:18.205 DEBUG 9492 --- [ main] DataNucleus.Persistence : Object "com.bp.gis.tardis.entity.SymbolEntity#59c08cf1" field "timeSeriesList" is replaced by a SCO wrapper of type "org.datanucleus.store.types.wrappers.backed.List" [cache-values=true, lazy-loading=true, allow-nulls=true]
2017-06-09 14:56:18.207 DEBUG 9492 --- [ main] DataNucleus.Persistence : ExecutionContext.internalFlush() process finished
2017-06-09 14:56:18.218 DEBUG 9492 --- [ main] o.s.jdbc.core.JdbcTemplate : Executing prepared SQL query
2017-06-09 14:56:18.220 DEBUG 9492 --- [ main] o.s.jdbc.core.JdbcTemplate : Executing prepared SQL statement [SELECT count(*) FROM symbol WHERE title = ?]
2017-06-09 14:56:18.250 TRACE 9492 --- [ main] o.s.jdbc.core.StatementCreatorUtils : Setting SQL statement parameter value: column index 1, parameter value [TEST.CODE.1], value class [java.lang.String], SQL type unknown
2017-06-09 14:57:18.298 INFO 9492 --- [ main] o.s.b.f.xml.XmlBeanDefinitionReader : Loading XML bean definitions from class path resource [org/springframework/jdbc/support/sql-error-codes.xml]
2017-06-09 14:57:18.390 INFO 9492 --- [ main] o.s.jdbc.support.SQLErrorCodesFactory : SQLErrorCodes loaded: [DB2, Derby, H2, HSQL, Informix, MS-SQL, MySQL, Oracle, PostgreSQL, Sybase, Hana]
MockHttpServletRequest:
HTTP Method = POST
Request URI = /symbols
Parameters = {}
Headers = {}
Handler:
Type = org.springframework.data.rest.webmvc.RepositoryEntityController
Method = public org.springframework.http.ResponseEntity<org.springframework.hateoas.ResourceSupport> org.springframework.data.rest.webmvc.RepositoryEntityController.postCollectionResource(org.springframework.data.rest.webmvc.RootResourceInformation,org.springframework.data.rest.webmvc.PersistentEntityResource,org.springframework.data.rest.webmvc.PersistentEntityResourceAssembler,java.lang.String) throws org.springframework.web.HttpRequestMethodNotSupportedException
Async:
Async started = false
Async result = null
Resolved Exception:
Type = null
ModelAndView:
View name = null
View = null
Model = null
FlashMap:
Attributes = null
MockHttpServletResponse:
Status = 201
Error message = null
Headers = {X-Application-Context=[application:-1], Location=[http://localhost/symbols/0]}
Content type = null
Body =
Forwarded URL = null
Redirected URL = http://localhost/symbols/0
Cookies = []
2017-06-09 14:57:18.424 DEBUG 9492 --- [ main] DataNucleus.Transaction : Transaction rolling back for ExecutionContext org.datanucleus.ExecutionContextImpl#2c47a053
2017-06-09 14:57:18.429 DEBUG 9492 --- [ main] DataNucleus.Lifecycle : Object "com.bp.gis.tardis.entity.SymbolEntity#59c08cf1" (id="com.bp.gis.tardis.entity.SymbolEntity:1") has a lifecycle change : "P_NEW"->""
2017-06-09 14:57:18.447 DEBUG 9492 --- [ main] DataNucleus.Transaction : Object "com.bp.gis.tardis.entity.SymbolEntity#59c08cf1" (id="com.bp.gis.tardis.entity.SymbolEntity:1") was evicted from transactional cache
2017-06-09 14:57:18.448 DEBUG 9492 --- [ main] DataNucleus.Persistence : Disconnecting com.bp.gis.tardis.entity.SymbolEntity#59c08cf1 from StateManager[pc=com.bp.gis.tardis.entity.SymbolEntity#59c08cf1, lifecycle=P_NEW]
2017-06-09 14:57:18.451 DEBUG 9492 --- [ main] DataNucleus.Cache : Object with id="com.bp.gis.tardis.entity.SymbolEntity:1" being removed from Level 1 cache [current cache size = 1]
2017-06-09 14:57:18.451 DEBUG 9492 --- [ main] DataNucleus.Transaction : Rolling back [DataNucleus Transaction, ID=Xid= , enlisted resources=[org.datanucleus.store.rdbms.ConnectionFactoryImpl$EmulatedXAResource#3d4b45b]]
2017-06-09 14:57:18.452 DEBUG 9492 --- [ main] DataNucleus.Connection : ManagedConnection(enlisted) "org.datanucleus.store.rdbms.ConnectionFactoryImpl$ManagedConnectionImpl#63a7af06 [conn=org.apache.derby.impl.jdbc.EmbedConnection#19d76106, commitOnRelease=false, closeOnRelease=false, closeOnTxnEnd=true]" rolling back for transaction "Xid= "
2017-06-09 14:57:18.461 DEBUG 9492 --- [ main] DataNucleus.Connection : ManagedConnection(non-enlisted) "org.datanucleus.store.rdbms.ConnectionFactoryImpl$ManagedConnectionImpl#63a7af06 [conn=org.apache.derby.impl.jdbc.EmbedConnection#19d76106, commitOnRelease=false, closeOnRelease=false, closeOnTxnEnd=true]" closed
2017-06-09 14:57:18.461 DEBUG 9492 --- [ main] DataNucleus.Connection : ManagedConnection removed from the pool : "org.datanucleus.store.rdbms.ConnectionFactoryImpl$ManagedConnectionImpl#63a7af06 [conn=org.apache.derby.impl.jdbc.EmbedConnection#19d76106, commitOnRelease=false, closeOnRelease=false, closeOnTxnEnd=true]" for key="org.datanucleus.ExecutionContextImpl#2c47a053" in factory="ConnectionFactory:tx[org.datanucleus.store.rdbms.ConnectionFactoryImpl#204c5ddf]"
2017-06-09 14:57:18.462 DEBUG 9492 --- [ main] DataNucleus.Transaction : Transaction rolled back in 38 ms
2017-06-09 14:57:18.462 DEBUG 9492 --- [ main] DataNucleus.Persistence : ExecutionContext "org.datanucleus.ExecutionContextImpl#2c47a053" closed
2017-06-09 14:57:18.463 INFO 9492 --- [ main] o.s.t.c.transaction.TransactionContext : Rolled back transaction for test context [DefaultTestContext#1f6f0fe2 testClass = SymbolRestTests, testInstance = com.bp.gis.tardis.integration.SymbolRestTests#22604c7e, testMethod = shouldCreateEntity#SymbolRestTests, testException = org.springframework.dao.CannotAcquireLockException: PreparedStatementCallback; SQL [SELECT count(*) FROM symbol WHERE title = ?]; A lock could not be obtained within the time requested; nested exception is java.sql.SQLTransactionRollbackException: A lock could not be obtained within the time requested, mergedContextConfiguration = [WebMergedContextConfiguration#3a48c398 testClass = SymbolRestTests, locations = '{}', classes = '{class com.bp.gis.tardis.config.TestDataSourceConfig, class com.bp.gis.tardis.config.TestDataSourceConfig}', contextInitializerClasses = '[]', activeProfiles = '{}', propertySourceLocations = '{}', propertySourceProperties = '{logging.level.DataNucleus=DEBUG, logging.level.com.bp.gis.tardis=TRACE, logging.level.org.springframework.jdbc.core=TRACE, security.basic.enabled=false, org.springframework.boot.test.context.SpringBootTestContextBootstrapper=true}', contextCustomizers = set[[ImportsContextCustomizer#1b4ba615 key = [#org.springframework.boot.autoconfigure.AutoConfigurationPackage(), #org.junit.FixMethodOrder(value=NAME_ASCENDING), #org.springframework.context.annotation.Import(value=[class org.springframework.boot.autoconfigure.AutoConfigurationPackages$Registrar]), #org.springframework.boot.test.autoconfigure.jdbc.AutoConfigureTestDatabase(replace=ANY, connection=DERBY), #org.springframework.boot.autoconfigure.EnableAutoConfiguration(exclude=[], excludeName=[]), #org.springframework.test.context.BootstrapWith(value=class org.springframework.boot.test.context.SpringBootTestContextBootstrapper), #org.junit.jupiter.api.extension.ExtendWith(value=[class org.springframework.test.context.junit.jupiter.SpringExtension]), #org.springframework.transaction.annotation.Transactional(propagation=REQUIRED, rollbackForClassName=[], readOnly=false, isolation=DEFAULT, transactionManager=, noRollbackFor=[], noRollbackForClassName=[], value=, timeout=-1, rollbackFor=[]), #org.springframework.boot.test.autoconfigure.web.servlet.AutoConfigureMockMvc(webDriverEnabled=true, print=DEFAULT, webClientEnabled=true, secure=true, addFilters=true, printOnlyOnFailure=true), #org.springframework.boot.autoconfigure.ImportAutoConfiguration(value=[], exclude=[], classes=[]), #org.junit.platform.commons.meta.API(value=Experimental), #org.springframework.context.annotation.Import(value=[class org.springframework.boot.autoconfigure.ImportAutoConfigurationImportSelector]), #org.springframework.boot.test.autoconfigure.properties.PropertyMapping(value=spring.test.mockmvc, skip=NO), #org.springframework.boot.test.context.SpringBootTest(webEnvironment=MOCK, value=[], properties=[logging.level.DataNucleus=DEBUG, logging.level.com.bp.gis.tardis=TRACE, logging.level.org.springframework.jdbc.core=TRACE, security.basic.enabled=false], classes=[class com.bp.gis.tardis.config.TestDataSourceConfig]), #org.springframework.boot.test.autoconfigure.properties.PropertyMapping(value=spring.test.database, skip=NO), #org.springframework.context.annotation.Import(value=[class org.springframework.boot.autoconfigure.EnableAutoConfigurationImportSelector])]], org.springframework.boot.test.context.SpringBootTestContextCustomizer#702657cc, org.springframework.boot.test.context.filter.ExcludeFilterContextCustomizer#6025e1b6, org.springframework.boot.test.mock.mockito.MockitoContextCustomizer#0, org.springframework.boot.test.autoconfigure.properties.PropertyMappingContextCustomizer#e30f6a3a, org.springframework.boot.test.autoconfigure.web.servlet.WebDriverContextCustomizerFactory$Customizer#1ff4931d], resourceBasePath = 'src/main/webapp', contextLoader = 'org.springframework.boot.test.context.SpringBootContextLoader', parent = [null]]].
UPDATE #3
The logging shows the INSERT by DataNucleus, followed several log statements that all look kosher about entity objects, and then comes
ExecutionContext.internalFlush() process finished
which sounds like the DataNucleus EntityManager flushing. Then comes the logging statement from Spring's JdbcTemplate which wants to read what JPA insert, and it all goes wrong.
Spring does in fact totally wrap the EntityManager for its testing framework and the wrapper just swallows the call to entityManager.close() so that call won't cause transactions to complete.
Spring also throws an error on calls to entityManager.getTransaction().
Typically in my experience of Spring, what just worked with the conventional Spring approach, e.g. Spring Data JPA with fully integrated Hibernate, does not work with DataNucleus.
Thanks go to this SO Q: How to manually force a commit in a #Transactional method?
I took out the #Transactional annotation from the class declaration, and put nothing on the test method, and called a seperate method with REQUIRES_NEW to do the REST-to-database functionality I want to test. Then in the original test method after that call, it was all committed and not locked so I could use Spring's JdbcTemplate to examine the result.
#Test
public void shouldCreateEntity() throws Exception {
String testTitle = "TEST.CODE.1";
String testExtra = "Test for SymbolRestTests.java";
String json = createJsonExample(testTitle, testExtra, true);
log.debug(String.format("JSON==%s", json));
doInNewTransaction(json);
String sql = "SELECT count(*) FROM symbol WHERE title = ?";
int count = jdbcTemplate.queryForObject(
sql, new Object[] { testTitle }, Integer.class);
Assert.assertThat(count, is(1));
}
#Transactional(propagation = Propagation.REQUIRES_NEW)
private void doInNewTransaction(String json) throws Exception {
MockHttpServletRequestBuilder requestBuilder =
post("/symbols").content(json);
mockMvc.perform(requestBuilder)
.andExpect(status().isCreated())
.andExpect(header().string("Location",
containsString("symbols/")));
}