I am trying to use my JPA repositories in order to save test data into h2 to be then used by a spring batch integration test.
Here is my integration test:
#SpringBootTest(webEnvironment = SpringBootTest.WebEnvironment.NONE, classes = Batch.class)
public class MessageDigestMailingStepIT extends AbstractBatchIntegrationTest {
#Autowired
#Qualifier("messagesDigestMailingJob")
private Job messagesDigestMailingJob;
#Autowired
private JobLauncher jobLauncher;
#Autowired
private JobRepository jobRepository;
#Autowired
private UserAccountRepository userAccountRepository;
#Autowired
private MessageRepository messageRepository;
private JobLauncherTestUtils jobLauncherTestUtils;
#Before
public void setUp() {
this.jobLauncherTestUtils = new JobLauncherTestUtils();
this.jobLauncherTestUtils.setJobLauncher(jobLauncher);
this.jobLauncherTestUtils.setJobRepository(jobRepository);
this.jobLauncherTestUtils.setJob(messagesDigestMailingJob);
}
#Test
#Transactional
public void shouldSendMessageDigestAndUpdateNotificationSent() {
UserAccount userAccount = DomainFactory.createUserAccount("me#example.com");
userAccountRepository.save(userAccount);
JobParameters jobParameters = new JobParametersBuilder().addDate("execution_date", new Date()).toJobParameters();
jobLauncherTestUtils.launchStep("messagesDigestMailingStep", jobParameters);
//Assertions
}
}
Notice the #Transactional on the test method. Unfortunately Spring batch uses its own transactions and my use of #Transactional clashes with spring batch transactions.
Here is the error message I get:
java.lang.IllegalStateException: Existing transaction detected in JobRepository. Please fix this and try again (e.g. remove #Transactional annotations from client).
Can someone please advise how to insert test data to be available for a spring batch integration test?
edit: For good measure, here is the definition of the AbstractBatchIntegrationTest class:
#AutoConfigureTestEntityManager
#AutoConfigureJson
#AutoConfigureJsonTesters
#RunWith(SpringJUnit4ClassRunner.class)
#ActiveProfiles(Profiles.TEST)
#ComponentScan(basePackages = {"com.bignibou.it.configuration", "com.bignibou.configuration"})
public abstract class AbstractBatchIntegrationTest {
}
edit: I have decided to rely only on the #Sql annotation as follows:
#Sql(scripts = "insert_message.sql", executionPhase = Sql.ExecutionPhase.BEFORE_TEST_METHOD)
#Sql(scripts = "clean_database.sql", executionPhase = Sql.ExecutionPhase.AFTER_TEST_METHOD)
#Test
public void shouldSendMessageDigestAndUpdateNotificationSent() {
...
Remove #Transactional from the test so that the UserAccount gets immediately persisted to the database. Then use #Sql with ExecutionPhase.AFTER_TEST_METHOD to execute a clean-up script (or inlined statement) to manually undo the changes performed during the test.
Related
We have a spring boot rest api (spring boot 2.3.0.RELEASE) that uses spring cloud sleuth (version 2.2.3.RELEASE).
At some point, we use the trace id from spring sleuth as data. The trace id is fetched by autowiring the Tracing bean and then accessing the current span. Lets say we defined a bean SimpleCorrelationBean with:
#Autowired
private Tracer tracer;
public String getCorrelationId() {
return tracer.currentSpan().context().traceIdString();
}
This seem to work perfectly when running the spring boot application, but when we try to access the tracer.currentSpan() in the unit tests, this is null. It looks like spring cloud sleuth is not creating any span while running tests..
I think it has something to do with the application context that is set up during the unit test, but I don't know how to enable spring cloud sleuth for the test application context.
Below is a simple test class where the error occurs in simpleTest1. In simpleTest2, no error occurs.
simpleTest1 errors because tracer.currentSpan() is null
#ExtendWith({ RestDocumentationExtension.class, SpringExtension.class })
#SpringBootTest(classes = MusicService.class)
#WebAppConfiguration
#ActiveProfiles("unit-test")
#ComponentScan(basePackageClasses = datacast2.data.JpaConfig.class)
public class SimpleTest {
private static final Logger logger = LoggerFactory.getLogger(SimpleTest.class);
#Autowired
private WebApplicationContext context;
private MockMvc mockMvc;
#Autowired
private FilterChainProxy springSecurityFilterChain;
#Autowired
private SimpleCorrelationBean simpleCorrelationBean;
#Autowired
private Tracer tracer;
#BeforeEach
public void setup(RestDocumentationContextProvider restDocumentation) throws Exception {
this.mockMvc = MockMvcBuilders.webAppContextSetup(this.context)
.apply(documentationConfiguration(restDocumentation))
.addFilter(springSecurityFilterChain).build();
}
#Test
public void simpleTest1() throws Exception {
try {
String correlationId = simpleCorrelationBean.getCorrelationId();
}catch(Exception e) {
logger.error("This seem to fail.", e);
}
}
#Test
public void simpleTest2() throws Exception {
//It looks like spring cloud sleuth is not creating a span, so we create one ourselfs
Span newSpan = this.tracer.nextSpan().name("simpleTest2");
try (Tracer.SpanInScope ws = this.tracer.withSpanInScope(newSpan.start())) {
String correlationId = simpleCorrelationBean.getCorrelationId();
}
finally {
newSpan.finish();
}
}
}
The question is: how to enable spring cloud sleuth for a mockMvc during unit tests?
The issue here is that MockMvc is created manually instead of relying on autoconfiguration. In this particular case custom configuration of MockMvc could be necessary. However, at least for my version of Spring Boot (2.7.6), there is no need to manually configure MockMvc, even though I use Spring Security and Spring Security Test. I couldn't figure out how to enable tracing when manually configuring MockMvc though.
I would like to insert a test record while testing my Spring Batch job.
Normally I'd annotate the test class with #Transactional, but that does not work with a test class that is annotated with #SpringBatchTest.
#SpringBatchTest
#SpringBootTest
#Transactional
public class JobTest {
#Autowired
private JobLauncherTestUtils jobLauncherTestUtils;
#Autowired
private JdbcTemplate jdbcTemplate;
#Test
void someTest() throws Exception {
jdbcTemplate.update("insert into some_table (some_col1, some_col2) values ('foo', 'bar')");
jobLauncherTestUtils.launchJob();
}
}
Existing transaction detected in JobRepository. Please fix this and try again (e.g. remove #Transactional annotations from client).
java.lang.IllegalStateException: Existing transaction detected in JobRepository. Please fix this and try again (e.g. remove #Transactional annotations from client).
at org.springframework.batch.core.repository.support.AbstractJobRepositoryFactoryBean$1.invoke(AbstractJobRepositoryFactoryBean.java:177)
I have also tried #Transactional(propagation = NOT_SUPPORTED), but the record does not rollback. This was the suggestion in https://stackoverflow.com/a/46546171/10243546
I just want test records for the test, so I wasn't sure if this answer applies (https://stackoverflow.com/a/55001568/10243546) since this is just for a test.
You need to move test setup outside the test method, for example in a method annotated with #Before (JUnit 4) or #BeforeEach (JUnit 5). Something like:
#Before
public void initData() {
jdbcTemplate.update("insert into some_table (some_col1, some_col2) values ('foo', 'bar')");
}
#Test
void someTest() throws Exception {
jobLauncherTestUtils.launchJob();
}
I was able to get such a test running using TransactionTemplate in the test method:
#SpringBatchTest
class MyJobTest {
// [...]
#Autowired
PlatformTransactionManager txManager;
#Test
void some_test() {
// setup must be done in a custom transaction due too #SpringBatchTest
new TransactionTemplate(txManager).executeWithoutResult(t -> {
// use jdbcTemplate or entityManager to insert data
jdbcTemplate.update("insert into some_table (some_col1, some_col2) values ('foo', 'bar')");
});
jobLauncherTestUtils.launchJob();
}
}
I am trying to commit jms and database transaction in spring batch job. I was under assumption that spring batch transaction are xa transactions. But in my item writer even when jms transaction errored out database transaction is committing. Can any one pls help me if I am missing something. Do I need to third party libraries for XA in spring batch?
I am actually throwing exception intentionally to test transaction roll back. Now even without any jms transaction just a database transaction is committing even with exception thrown from the item writer.Below is the method in writer which is saving into DB. compEvent object is jpa repository injected into this class
private void writeCEs(Map<TrueEvent, List<Event>> agentMap)
throws FailedCompensationException, Exception {
for (Entry<TrueEvent, List<Event>> agent : agentMap.entrySet()) {
agent.getValue().stream().forEach((ce) -> {
compEvent.save(ce);
});
updateRecordFileStatus(agent.getKey());
//postToAccounting(agent.getKey(), agent.getValue());
}
throw new Exception("Testing XA roolback.... ");
}
Below is my batch configuration
#EnableBatchProcessing
#EnableTransactionManagement
#Configuration
#ComponentScan({ "com.pm.*" })
public class TrueBatchConfig extends DefaultBatchConfigurer {
#Autowired
private JobBuilderFactory jobs;
#Autowired
private StepBuilderFactory steps;
#Autowired
EventReader reader;
#Autowired
private EventProcessor processor;
#Autowired
private EventWriter writer;
#Bean
protected Step step1(ThreadPoolTaskExecutor executor) {
DefaultTransactionAttribute attribute = new DefaultTransactionAttribute();
attribute.setPropagationBehavior(Propagation.REQUIRED.value());
attribute.setIsolationLevel(Isolation.DEFAULT.value());
attribute.setTimeout(30);
return steps.get("step1").<List<TrueEvent>, Map<TrueUpEvent, List<Event>>>chunk(10).reader(reader)
.processor(processor).writer(writer).transactionAttribute(attribute).build();
}
#Bean(name = "firstBatchJob")
public Job job(#Qualifier("step1") Step step1) {
return jobs.get("firstBatchJob").start(step1).build();
}
}
In the following code
#DataJpaTest
#Transactional(propagation = Propagation.NOT_SUPPORTED)
#AutoConfigureTestDatabase(replace= AutoConfigureTestDatabase.Replace.NONE)
public class GenreDaoJpaTest{
#Autowired
private TestEntityManager entityManager;
#Autowired
private GenreRepository dao;
....
}
when I'm adding #Transactional(propagation = Propagation.NOT_SUPPORTED) with the purpose to cancel a roolback after each test I'm getting an exception:
ava.lang.IllegalStateException: No transactional EntityManager found
at org.springframework.util.Assert.state(Assert.java:73)
at org.springframework.boot.test.autoconfigure.orm.jpa.TestEntityManager.getEntityManager(TestEntityManager.java:237)
at org.springframework.boot.test.autoconfigure.orm.jpa.TestEntityManager.persist(TestEntityManager.java:92)
at ru.otus.ea.dao.GenreDaoJpaTest.init(GenreDaoJpaTest.java:38)
It there a way to autowire TestEntityManager and not to roolback transactions in tests?
Your TestEntityManager is autowired but you are executing the persist call outside of a transaction.
You can autowire TransactionTemplate:
#Autowired
private TransactionTemplate transactionTemplate;
And execute your DB interactions using its execute method:
User savedUser = transactionTemplate.execute((conn) -> {
return testEntityManager.persist(new User("foo"));
});
Also you should be aware that now you are responsible for cleanup of test DB after tests execute (which might be hard to maintain as logic grows):
#BeforeEach // just to be sure
#AfterEach
public void cleanup() {
userRepository.deleteAll();
}
What is the best way to continually run a Spring Batch job? Do we need to write a shell file which loops and starts the job at predefined intervals? Or is there a way within Spring Batch itself to configure a job so that it repeats at either
1) pre-defined intervals
2) after the completion of each run
Thanks
If you want to launch your jobs periodically, you can combine Spring Scheduler and Spring Batch. Here is a concrete example : Spring Scheduler + Batch Example.
If you want to re-launch your job continually (Are you sure !), You can configure a Job Listener on your job. Then, through the method jobListener.afterJob(JobExecution jobExecution) you can relaunch your job.
Id did something like this for importing emails, so i have to check it periodically
#SpringBootApplication
#EnableScheduling
public class ImportBillingFromEmailBatchRunner
{
private static final Logger LOG = LoggerFactory.getLogger(ImportBillingFromEmailBatchRunner.class);
public static void main(String[] args)
{
SpringApplication app = new SpringApplication(ImportBillingFromEmailBatchRunner.class);
app.run(args);
}
#Bean
BillingEmailCronService billingEmailCronService()
{
return new BillingEmailCronService();
}
}
So the BillingEmailCronService takes care of the continuation:
public class BillingEmailCronService
{
private static final Logger LOG = LoggerFactory.getLogger(BillingEmailCronService.class);
#Autowired
private JobLauncher jobLauncher;
#Autowired
private JobExplorer jobExplorer;
#Autowired
private JobRepository jobRepository;
#Autowired
private JobBuilderFactory jobBuilderFactory;
#Autowired
private #Qualifier(BillingBatchConfig.QUALIFIER)
Step fetchBillingFromEmailsStep;
#Scheduled(fixedDelay = 5000)
public void run()
{
LOG.info("Procesando correos con facturas...");
try
{
Job job = createNewJob();
JobParameters jobParameters = new JobParameters();
jobLauncher.run(job, jobParameters);
}catch(...)
{
//Handle each exception
}
}
}
Implement your createNewJob logic and try it out.
one easy way would be configure cron job from Unix which will run application at specified interval