Insert test record for Spring Batch integration test with #Transactional(propagation = NOT_SUPPORTED) does not roll back - spring-batch

I would like to insert a test record while testing my Spring Batch job.
Normally I'd annotate the test class with #Transactional, but that does not work with a test class that is annotated with #SpringBatchTest.
#SpringBatchTest
#SpringBootTest
#Transactional
public class JobTest {
#Autowired
private JobLauncherTestUtils jobLauncherTestUtils;
#Autowired
private JdbcTemplate jdbcTemplate;
#Test
void someTest() throws Exception {
jdbcTemplate.update("insert into some_table (some_col1, some_col2) values ('foo', 'bar')");
jobLauncherTestUtils.launchJob();
}
}
Existing transaction detected in JobRepository. Please fix this and try again (e.g. remove #Transactional annotations from client).
java.lang.IllegalStateException: Existing transaction detected in JobRepository. Please fix this and try again (e.g. remove #Transactional annotations from client).
at org.springframework.batch.core.repository.support.AbstractJobRepositoryFactoryBean$1.invoke(AbstractJobRepositoryFactoryBean.java:177)
I have also tried #Transactional(propagation = NOT_SUPPORTED), but the record does not rollback. This was the suggestion in https://stackoverflow.com/a/46546171/10243546
I just want test records for the test, so I wasn't sure if this answer applies (https://stackoverflow.com/a/55001568/10243546) since this is just for a test.

You need to move test setup outside the test method, for example in a method annotated with #Before (JUnit 4) or #BeforeEach (JUnit 5). Something like:
#Before
public void initData() {
jdbcTemplate.update("insert into some_table (some_col1, some_col2) values ('foo', 'bar')");
}
#Test
void someTest() throws Exception {
jobLauncherTestUtils.launchJob();
}

I was able to get such a test running using TransactionTemplate in the test method:
#SpringBatchTest
class MyJobTest {
// [...]
#Autowired
PlatformTransactionManager txManager;
#Test
void some_test() {
// setup must be done in a custom transaction due too #SpringBatchTest
new TransactionTemplate(txManager).executeWithoutResult(t -> {
// use jdbcTemplate or entityManager to insert data
jdbcTemplate.update("insert into some_table (some_col1, some_col2) values ('foo', 'bar')");
});
jobLauncherTestUtils.launchJob();
}
}

Related

JPA. Update dml doesn't work with #Transactional annotation

I tryng to perform an an unpdate within a UnitTest method with org.springframework.transaction.annotation.Transactional annotation, but it seems that the update doesn't work.
If I remove #Transactional annotation on the method, the update works succesfully but it is visible for all others tests too.
Could you please indicate to me where I'm wrong?
I need the update effective only within the method with the #Transactional annotation and not visible for all others methods.
I'm using
Srping boot v 2.6.6. to start the application. I use JPA and Oracle Data Base.
This is my repository class where I use native query.
#Repository
#Transactional
public interface EsercentiRepository extends JpaRepository<EsercentiEntity, Long> {
// Update
#Modifying
#Query(value="update esercenti set sslfl=:sslfl where id_conv=:idConv", nativeQuery=true)
public void updateSslFlagByIdViaQuery(#Param("idConv") long idConv, #Param("sslfl") String sslfl);
// select
#Query(value="select id_conv,c_code,vendor_id,pos_id,abi_code,funzioni,ds,pos_id_sia,rifmer3d,sslfl "
+ " from esercenti where id_conv=:idConv", nativeQuery=true)
public EsercentiEntity getEsercentiByIdConvViaQuery(#Param("idConv") long idConv);
}
This is my Service class.
#Service
#Transactional
public class EsercentiServices implements IEsercenti {
#Autowired
private EsercentiRepository esercentiRepository;
#Override
public void updateSslFlagByIdViaQuery(long idConv, String sslfl) throws Exception {
esercentiRepository.updateSslFlagByIdViaQuery(idConv, sslfl);
}
#Override
public EsercentiEntity getEsercentiByIdConvViaQuery(long idConv) throws Exception {
return esercentiRepository.getEsercentiByIdConvViaQuery(idConv);
}
}
And this is my SpringBootTest class located in the 'test' directory where I use Junit 5 to perform Functional tests.
#EnableTransactionManagement
#TestInstance(TestInstance.Lifecycle.PER_CLASS)
#TestMethodOrder(MethodOrderer.OrderAnnotation.class)
#ExtendWith(SpringExtension.class)
#SpringBootTest
class TestGpay extends BaseServiceTester {
#Autowired
private IEsercenti esercentiServices;
[..]
#Test
#Transactional
public void TestDirectAuthGpayWithPayload(TestInfo testInfo) {
try {
// [..... Some codes]
EsercentiEntity ese = esercentiServices.getEsercentiByIdConvViaQuery(7826L);
esercentiServices.updateSslFlagByIdViaQuery(7826L, "Y");
EsercentiEntity ese2 = esercentiServices.getEsercentiByIdConvViaQuery(7826L);
// Send the http request. I expect to find the data changed on DB as per above update, but it is not.
WebUtils.HttpResponse response = netsJsonClient(endPoint, "POST", jsonObjectRequest.toString(), merId, merIdKsig);
// If I cancel the #Transactoinal annotation on the method level, the update is ok,
// but it is a global update and not only related to this database session.
} catch (Exception e) {
e.printStackTrace();
closeDriver(driver);
fail("Exception on test case " + testInfo.getDisplayName() + " Full Error:" + e);
}
}
Thanks in advance.

Spring Batch, JpaRepository and Rollback

I have a Spring Batch application(Spring Boot 2.3.5.RELEASE) that uses a JpaRepository to insert some custom log messages into a database as Spring Batch is processing. This is separate from the out of the box Spring Batch tables. Seems that when I throw an exception from my ItemProcessorAdapter, it is caught by the ItemProcessListener onProcessError() method. In this method I am performing a JpaRepository save() and flush(). No errors are logged, but once I leave this method the JpaRepository does a rollback.
Is this normal behavior? How can I get around it?
When using JpaRepository, is there a way to set a #Transactional(noRollbackFor = {xxxException.class})? I tried this and it seemed to have no effect.
Sample code snippet is below.
#Configuration
public class BatchJobConfiguration {
//Omitted for clarity....
#Bean
#StepScope
public CompositeItemProcessor<Decision,Decision> itemProcessor() {
CompositeItemProcessor<Decision,Decision> itemProcessor = new CompositeItemProcessor<>();
itemProcessor.setDelegates(Arrays.asList(
decisionValidatingItemProcessor(),
myItemProcessor(null)
));
return itemProcessor;
} // end itemProcessor()
#Bean
public BeanValidatingItemProcessor<Decision> decisionValidatingItemProcessor() {
BeanValidatingItemProcessor<Decision> beanValidatingItemProcessor = new BeanValidatingItemProcessor<>();
beanValidatingItemProcessor.setFilter(true);
return beanValidatingItemProcessor;
} // end decisionValidatingItemProcessor()
#Bean
public ItemProcessorAdapter<Decision,Decision> myItemProcessor(DecisionProcessingService service) {
ItemProcessorAdapter<Decision,Decision> adapter = new ItemProcessorAdapter<>();
adapter.setTargetObject(service);
adapter.setTargetMethod("processDecision");
return adapter;
}
#Bean
#StepScope
public DecisionItemProcessListener decisionItemProcessListener() {
return new DecisionItemProcessListener(mpJpaRepository);
}
}
#Service
public class DecisionProcessingService {
public Decision processDecision(Decision decision) throws BatchException {
....
throw new BatchException("An error occurred");
}
}
public class DecisionItemProcessListener implements ItemProcessListener<Decision,Decision> {
private MyJpaRepository mpJpaRepository;
public DecisionItemProcessListener(MyJpaRepository mpJpaRepository) {
this.mpJpaRepository = mpJpaRepository;
}
....
#Override
public void onProcessError(Decision decision, Exception e) {
MyEntityObject obj = MyEntityObject.builder()
.msg(e.getMessage())
.build();
mpJpaRepository.save(obj);
mpJpaRepository.flush();
// after this, the insert above is rolled back.
} // end onProcessError()
}
The callback you are using here ItemProcessListener#onProcessError is called with-in a transaction (driven by Spring Batch) that is going to be rolled-back due to the exception thrown by the item processor.
If you want to save data in that method, you need to use a new transaction (use the REQUIRES_NEW propagation).
EDIT: I shared a minimal complete example here: https://github.com/benas/spring-batch-lab/tree/master/issues/so64913980.

How not to rollback #DataJpaTest?

In the following code
#DataJpaTest
#Transactional(propagation = Propagation.NOT_SUPPORTED)
#AutoConfigureTestDatabase(replace= AutoConfigureTestDatabase.Replace.NONE)
public class GenreDaoJpaTest{
#Autowired
private TestEntityManager entityManager;
#Autowired
private GenreRepository dao;
....
}
when I'm adding #Transactional(propagation = Propagation.NOT_SUPPORTED) with the purpose to cancel a roolback after each test I'm getting an exception:
ava.lang.IllegalStateException: No transactional EntityManager found
at org.springframework.util.Assert.state(Assert.java:73)
at org.springframework.boot.test.autoconfigure.orm.jpa.TestEntityManager.getEntityManager(TestEntityManager.java:237)
at org.springframework.boot.test.autoconfigure.orm.jpa.TestEntityManager.persist(TestEntityManager.java:92)
at ru.otus.ea.dao.GenreDaoJpaTest.init(GenreDaoJpaTest.java:38)
It there a way to autowire TestEntityManager and not to roolback transactions in tests?
Your TestEntityManager is autowired but you are executing the persist call outside of a transaction.
You can autowire TransactionTemplate:
#Autowired
private TransactionTemplate transactionTemplate;
And execute your DB interactions using its execute method:
User savedUser = transactionTemplate.execute((conn) -> {
return testEntityManager.persist(new User("foo"));
});
Also you should be aware that now you are responsible for cleanup of test DB after tests execute (which might be hard to maintain as logic grows):
#BeforeEach // just to be sure
#AfterEach
public void cleanup() {
userRepository.deleteAll();
}

Spring Boot Hibernate Postgresql #Transactional does not rollback [duplicate]

I want to read text data fixtures (CSV files) at the start on my application and put it in my database.
For that, I have created a PopulationService with an initialization method (#PostConstruct annotation).
I also want them to be executed in a single transaction, and hence I added #Transactional on the same method.
However, the #Transactional seems to be ignored :
The transaction is started / stopped at my low level DAO methods.
Do I need to manage the transaction manually then ?
Quote from legacy (closed) Spring forum:
In the #PostConstruct (as with the afterPropertiesSet from the InitializingBean interface) there is no way to ensure that all the post processing is already done, so (indeed) there can be no Transactions. The only way to ensure that that is working is by using a TransactionTemplate.
So if you would like something in your #PostConstruct to be executed within transaction you have to do something like this:
#Service("something")
public class Something {
#Autowired
#Qualifier("transactionManager")
protected PlatformTransactionManager txManager;
#PostConstruct
private void init(){
TransactionTemplate tmpl = new TransactionTemplate(txManager);
tmpl.execute(new TransactionCallbackWithoutResult() {
#Override
protected void doInTransactionWithoutResult(TransactionStatus status) {
//PUT YOUR CALL TO SERVICE HERE
}
});
}
}
I think #PostConstruct only ensures the preprocessing/injection of your current class is finished. It does not mean that the initialization of the whole application context is finished.
However you can use the spring event system to receive an event when the initialization of the application context is finished:
public class MyApplicationListener implements ApplicationListener<ContextRefreshedEvent> {
public void onApplicationEvent(ContextRefreshedEvent event) {
// do startup code ..
}
}
See the documentation section Standard and Custom Events for more details.
As an update, from Spring 4.2 the #EventListener annotation allows a cleaner implementation:
#Service
public class InitService {
#Autowired
MyDAO myDAO;
#EventListener(ContextRefreshedEvent.class)
public void onApplicationEvent(ContextRefreshedEvent event) {
event.getApplicationContext().getBean(InitService.class).initialize();
}
#Transactional
public void initialize() {
// use the DAO
}
}
Inject self and call through it the #Transactional method
public class AccountService {
#Autowired
private AccountService self;
#Transactional
public void resetAllAccounts(){
//...
}
#PostConstruct
private void init(){
self.resetAllAccounts();
}
}
For older Spring versions which do not support self-injection, inject BeanFactory and get self as beanFactory.getBean(AccountService.class)
EDIT
It looks like that since this solution has been posted 1.5 years ago developers are still under impression that if a method,
annotated with #Transactional, is called from a #PostContruct-annotated method invoked upon the Bean initialization, it won't be actually executed inside of Spring Transaction, and awkward (obsolete?) solutions get discussed and accepted instead of this very simple and straightforward one and the latter even gets downvoted.
The Doubting Thomases :) are welcome to check out an example Spring Boot application at GitHub which implements the described above solution.
What actually causes, IMHO, the confusion: the call to #Transactional method should be done through a proxied version of a Bean where such method is defined.
When a #Transactional method is called from another Bean, that another Bean usually injects this one and invokes its proxied (e.g. through #Autowired) version of it, and everything is fine.
When a #Transactional method is called from the same Bean directly, through usual Java call, the Spring AOP/Proxy machinery is not involved and the method is not executed inside of Transaction.
When, as in the suggested solution, a #Transactional method is called from the same Bean through self-injected proxy (self field), the situation is basically equivalent to a case 1.
#Platon Serbin's answer didn't work for me. So I kept searching and found the following answer that saved my life. :D
The answer is here No Session Hibernate in #PostConstruct, which I took the liberty to transcribe:
#Service("myService")
#Transactional(readOnly = true)
public class MyServiceImpl implements MyService {
#Autowired
private MyDao myDao;
private CacheList cacheList;
#Autowired
public void MyServiceImpl(PlatformTransactionManager transactionManager) {
this.cacheList = (CacheList) new TransactionTemplate(transactionManager).execute(new TransactionCallback(){
#Override
public Object doInTransaction(TransactionStatus transactionStatus) {
CacheList cacheList = new CacheList();
cacheList.reloadCache(MyServiceImpl.this.myDao.getAllFromServer());
return cacheList;
}
});
}
The transaction part of spring might not be initialized completely at #PostConstruct.
Use a listener to the ContextRefreshedEvent event to ensure, that transactions are available:
#Component
public class YourService
implements ApplicationListener<ContextRefreshedEvent> // <= ensure correct timing!
{
private final YourRepo repo;
public YourService (YourRepo repo) {this.repo = repo;}
#Transactional // <= ensure transaction!
#Override
public void onApplicationEvent(ContextRefreshedEvent event) {
repo.doSomethingWithinTransaction();
}
}
Using transactionOperations.execute() in #PostConstruct or in #NoTransaction method both works
#Service
public class ConfigurationService implements ApplicationContextAware {
private static final Logger LOG = LoggerFactory.getLogger(ConfigurationService.class);
private ConfigDAO dao;
private TransactionOperations transactionOperations;
#Autowired
public void setTransactionOperations(TransactionOperations transactionOperations) {
this.transactionOperations = transactionOperations;
}
#Autowired
public void setConfigurationDAO(ConfigDAO dao) {
this.dao = dao;
}
#PostConstruct
public void postConstruct() {
try { transactionOperations.execute(new TransactionCallbackWithoutResult() {
#Override
protected void doInTransactionWithoutResult(final TransactionStatus status) {
ResultSet<Config> configs = dao.queryAll();
}
});
}
catch (Exception ex)
{
LOG.trace(ex.getMessage(), ex);
}
}
#NoTransaction
public void saveConfiguration(final Configuration configuration, final boolean applicationSpecific) {
String name = configuration.getName();
Configuration original = transactionOperations.execute((TransactionCallback<Configuration>) status ->
getConfiguration(configuration.getName(), applicationSpecific, null));
}
#Override
public void setApplicationContext(ApplicationContext applicationContext) throws BeansException {
}
}

How to save test data to be consumed by a spring batch integration test

I am trying to use my JPA repositories in order to save test data into h2 to be then used by a spring batch integration test.
Here is my integration test:
#SpringBootTest(webEnvironment = SpringBootTest.WebEnvironment.NONE, classes = Batch.class)
public class MessageDigestMailingStepIT extends AbstractBatchIntegrationTest {
#Autowired
#Qualifier("messagesDigestMailingJob")
private Job messagesDigestMailingJob;
#Autowired
private JobLauncher jobLauncher;
#Autowired
private JobRepository jobRepository;
#Autowired
private UserAccountRepository userAccountRepository;
#Autowired
private MessageRepository messageRepository;
private JobLauncherTestUtils jobLauncherTestUtils;
#Before
public void setUp() {
this.jobLauncherTestUtils = new JobLauncherTestUtils();
this.jobLauncherTestUtils.setJobLauncher(jobLauncher);
this.jobLauncherTestUtils.setJobRepository(jobRepository);
this.jobLauncherTestUtils.setJob(messagesDigestMailingJob);
}
#Test
#Transactional
public void shouldSendMessageDigestAndUpdateNotificationSent() {
UserAccount userAccount = DomainFactory.createUserAccount("me#example.com");
userAccountRepository.save(userAccount);
JobParameters jobParameters = new JobParametersBuilder().addDate("execution_date", new Date()).toJobParameters();
jobLauncherTestUtils.launchStep("messagesDigestMailingStep", jobParameters);
//Assertions
}
}
Notice the #Transactional on the test method. Unfortunately Spring batch uses its own transactions and my use of #Transactional clashes with spring batch transactions.
Here is the error message I get:
java.lang.IllegalStateException: Existing transaction detected in JobRepository. Please fix this and try again (e.g. remove #Transactional annotations from client).
Can someone please advise how to insert test data to be available for a spring batch integration test?
edit: For good measure, here is the definition of the AbstractBatchIntegrationTest class:
#AutoConfigureTestEntityManager
#AutoConfigureJson
#AutoConfigureJsonTesters
#RunWith(SpringJUnit4ClassRunner.class)
#ActiveProfiles(Profiles.TEST)
#ComponentScan(basePackages = {"com.bignibou.it.configuration", "com.bignibou.configuration"})
public abstract class AbstractBatchIntegrationTest {
}
edit: I have decided to rely only on the #Sql annotation as follows:
#Sql(scripts = "insert_message.sql", executionPhase = Sql.ExecutionPhase.BEFORE_TEST_METHOD)
#Sql(scripts = "clean_database.sql", executionPhase = Sql.ExecutionPhase.AFTER_TEST_METHOD)
#Test
public void shouldSendMessageDigestAndUpdateNotificationSent() {
...
Remove #Transactional from the test so that the UserAccount gets immediately persisted to the database. Then use #Sql with ExecutionPhase.AFTER_TEST_METHOD to execute a clean-up script (or inlined statement) to manually undo the changes performed during the test.