Spring batch use JpaPagingItemReader JpaItemWriter no transaction is in progress - jpa

My project uses java config, the DataSource entityManagerFactory PlatformTransactionManager created by other java config file.
When ItemReader read data, ItemProcessor data transformed ItemWriter has some error
How can I solve this problem?
Error message
2015-01-17 16:26:30.390 ERROR AbstractStep:225 - Encountered an error executing step testStep in job validJob
javax.persistence.TransactionRequiredException: no transaction is in progress
at org.hibernate.jpa.spi.AbstractEntityManagerImpl.checkTransactionNeeded(AbstractEntityManagerImpl.java:1171)
at org.hibernate.jpa.spi.AbstractEntityManagerImpl.flush(AbstractEntityManagerImpl.java:1332)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:483)
How can I enable transactions on the following Code:
#Configuration
#EnableBatchProcessing
public class JobConfiguration {
#Inject
private PlatformTransactionManager transactionManager;
#Inject
private EntityManagerFactory emf;
#Bean
public Job validJob(JobBuilderFactory jobBuilderFactory,Step step) {
return jobBuilderFactory.get("validJob").flow(step).end().build();
}
#Bean
public Step testStep(StepBuilderFactory stepBuilderFactory, ItemReader<PurchaseOrder> reader,
ItemWriter<PurchaseOrder> writer, ItemProcessor<PurchaseOrder, PurchaseOrder> processor) {
return stepBuilderFactory.get("testStep")//
.transactionManager(transactionManager)//
.<PurchaseOrder, PurchaseOrder>chunk(100)//
.reader(reader)//
.processor(processor)//
.writer(writer)//
.build();
}
#Bean
public ItemReader<PurchaseOrder> reader() {
JpaPagingItemReader<PurchaseOrder> reader = new JpaPagingItemReader<PurchaseOrder>();
reader.setEntityManagerFactory(emf);
reader.setQueryString(" select p from PurchaseOrder p ");
reader.setTransacted(false);
return reader;
}
#Bean
public ItemProcessor<PurchaseOrder, PurchaseOrder> processor() {
return new PurchaseOrderItemProcessor();
}
#Bean
public ItemWriter<PurchaseOrder> writer() {
JpaItemWriter<PurchaseOrder> writer = new JpaItemWriter<PurchaseOrder>();
writer.setEntityManagerFactory(emf);
return writer;
}

Related

Can multi job Spring Batch app load minimum set of beans? [duplicate]

I have a Spring Batch project with multiple jobs (job A, job B, job C,...). When I run a particular job A, I got the log of the job A shows that all of the beans of job B, C,... are created too. Is there any way to avoid the creation of the other beans when job A is launched.
I have tried to use #Lazy annotation but it 's seem not working.
#Configuration
#EnableBatchProcessing
public class BatchConfiguration {
#Autowired
public JobBuilderFactory jobBuilderFactory;
#Autowired
public StepBuilderFactory stepBuilderFactory;
#Autowired
#Qualifier("springDataSource")
public DataSource springDataSource;
#Autowired
#Qualifier("batchJobDataSource")
public DataSource batchJobDataSource;
}
#Configuration
#PropertySource("classpath:partner.properties")
public class B extends BatchConfiguration {
#Value("${partnerId}")
private String partnerId;
#Lazy
#Bean
public Job ProcessB(JobCompletionNotificationListener listener) {
return jobBuilderFactory
.get("ProcessB")
.incrementer(new RunIdIncrementer())
.listener(listener)
.start(ProcessStepB())
.build();
}
#Lazy
#Bean
public Step (ProcessStepB() {
return stepBuilderFactory
.get("(ProcessStepB")
.<PartnerDTO, PartnerDTO> chunk(1)
.reader(getPartner())
.processor(process())
.writer(save())
.build();
}
#Lazy
#Bean(destroyMethod = "")
public Reader getPartner() {
return new Reader(batchJobDataSource,partnerId);
}
#Lazy
#Bean
public Processor process() {
return new Processor();
}
#Lazy
#Bean
HistoryWriter historyWriter() {
return new HistoryWriter(batchJobDataSource);
}
#Lazy
#Bean
UpdateWriter updateWriter() {
return new UpdateWriter(batchJobDataSource);
}
#Lazy
#Bean
public CompositeItemWriter<PartnerDTO> saveTransaction() {
List<ItemWriter<? super PartnerDTO>> delegates = new ArrayList<>();
delegates.add(updateWriter());
delegates.add(historyWriter());
CompositeItemWriter<PartnerDTO> itemWriter = new CompositeItemWriter<>();
itemWriter.setDelegates(delegates);
return itemWriter;
}
}
I have also put the #Lazy over the #Configuration but it does work too.
That should not be an issue. But here are a few ideas to try:
Use Spring profiles to isolate job beans
If you use Spring Boot 2.2+, try to activate the lazy bean initialization mode
Package each job in its own jar. This is the best option IMO.

Spring batch with DB2 and MongoDB using Spring boot

I need to read the DB2 database and copy data to mongoDB using Spring Batch. As I am going to write the data to mongoDB, so I dont need the transaction. I would like to keep the metadata tables scripts in mongoDB only not in DB2 but I couldn't find the metadata tables scripts for mongoDB. In the server startup time, spring boot expect the batch_job_instance table in Db2 instead of mongoDB. I annotated mongoDB as primary but still it is throwing an error.
Someone can help me with this. Thanks in advance.
MongoConfig.java:
#Configuration
#EnableMongoRepositories("com.test.mongodb")
public class MongoConfig extends AbstractMongoConfiguration {
private final Logger log = LoggerFactory.getLogger(MongoConfig.class);
#Value("${spring.data.mongodb.host}")
private String host;
#Value("${spring.data.mongodb.port}")
private Integer port;
#Value("${spring.data.mongodb.username}")
private String username;
#Value("${spring.data.mongodb.database}")
private String database;
#Value("${spring.data.mongodb.password}")
private String password;
#Bean
public ValidatingMongoEventListener validatingMongoEventListener() {
return new ValidatingMongoEventListener(validator());
}
#Bean
public LocalValidatorFactoryBean validator() {
return new LocalValidatorFactoryBean();
}
#Override
public String getDatabaseName() {
return database;
}
#Override
#Bean
public Mongo mongo() throws Exception {
return new MongoClient(singletonList(new ServerAddress(host, port)),
singletonList(MongoCredential.createCredential(username, database, password.toCharArray())));
}
#Override
#Bean
#Primary
#ConfigurationProperties(prefix = "spring.data.mongodb")
public MongoTemplate mongoTemplate() throws Exception {
return new MongoTemplate(mongo(), database);
}
}
application.properties:
# DB2
spring.datasource.jndi-name=java:jboss/datasources/Db2XaDsn
# Mongo DB
spring.data.mongodb.host=localhost
spring.data.mongodb.port=27017
spring.data.mongodb.username=admin
spring.data.mongodb.password=admin
spring.data.mongodb.database=test
Batch class:
#Configuration
#EnableBatchProcessing
public class ItemBatch {
#Autowired
private JobBuilderFactory jobBuilderFactory;
#Autowired
private StepBuilderFactory stepBuilderFactory;
#Autowired
EntityManagerFactory entityManagerFactory;
#Autowired
MongoTemplate mongoTemplate;
#Bean
public Job readDB2() {
return jobBuilderFactory.get("readDB2").start(step1()).build();
}
#Bean
public Step step1() {
return stepBuilderFactory.get("step1").<com.model.db2.Item, Item>chunk(200).reader(reader())
.writer(writer()).build();
}
#Bean
public ItemReader<com.model.db2.Item> reader() {
JpaPagingItemReader<com.model.db2.Item> reader = new JpaPagingItemReader<>();
reader.setQueryString("select i from Item i");
reader.setEntityManagerFactory(entityManagerFactory);
return reader;
}
#Bean
public MongoItemWriter<Item> writer() {
MongoItemWriter<Item> writer = new MongoItemWriter<>();
try {
writer.setTemplate(mongoTemplate);
} catch (Exception e) {
e.printStackTrace();
}
writer.setCollection("item");
return writer;
}
}
Error:
00:44:39,484 ERROR [org.jboss.msc.service.fail] (ServerService Thread Pool -- 72) MSC000001: Failed to start service jboss.undertow.deployment.default-server.default-host./itemapi: org.jboss.msc.service.StartException in service jboss.undertow.deployment.default-server.default-host./itemapi: java.lang.RuntimeException: java.lang.IllegalStateException: Failed to execute CommandLineRunner
at org.wildfly.extension.undertow.deployment.UndertowDeploymentService$1.run(UndertowDeploymentService.java:85)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
at org.jboss.threads.JBossThread.run(JBossThread.java:320)
Caused by: java.lang.RuntimeException: java.lang.IllegalStateException: Failed to execute CommandLineRunner
at io.undertow.servlet.core.DeploymentManagerImpl.deploy(DeploymentManagerImpl.java:231)
at org.wildfly.extension.undertow.deployment.UndertowDeploymentService.startContext(UndertowDeploymentService.java:100)
at org.wildfly.extension.undertow.deployment.UndertowDeploymentService$1.run(UndertowDeploymentService.java:82)
... 6 more
Caused by: java.lang.IllegalStateException: Failed to execute CommandLineRunner
at org.springframework.boot.SpringApplication.callRunner(SpringApplication.java:735)
at org.springframework.boot.SpringApplication.callRunners(SpringApplication.java:716)
at org.springframework.boot.SpringApplication.afterRefresh(SpringApplication.java:703)
at org.springframework.boot.SpringApplication.run(SpringApplication.java:304)
at org.springframework.boot.web.support.SpringBootServletInitializer.run(SpringBootServletInitializer.java:154)
at org.springframework.boot.web.support.SpringBootServletInitializer.createRootApplicationContext(SpringBootServletInitializer.java:134)
at org.springframework.boot.web.support.SpringBootServletInitializer.onStartup(SpringBootServletInitializer.java:87)
at org.springframework.web.SpringServletContainerInitializer.onStartup(SpringServletContainerInitializer.java:169)
at io.undertow.servlet.core.DeploymentManagerImpl.deploy(DeploymentManagerImpl.java:184)
... 8 more
Caused by: org.springframework.jdbc.BadSqlGrammarException: PreparedStatementCallback; bad SQL grammar [SELECT JOB_INSTANCE_ID, JOB_NAME from BATCH_JOB_INSTANCE where JOB_NAME = ? order by JOB_INSTANCE_ID desc]; nested exception is com.ibm.db2.jcc.am.SqlSyntaxErrorException: DB2 SQL Error: SQLCODE=-204, SQLSTATE=42704, SQLERRMC=TEST.BATCH_JOB_INSTANCE, DRIVER=4.18.60
at org.springframework.jdbc.support.SQLErrorCodeSQLExceptionTranslator.doTranslate(SQLErrorCodeSQLExceptionTranslator.java:231)
at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:73)
at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:649)
at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:684)
at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:716)
at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:726)
at org.springframework.batch.core.repository.dao.JdbcJobInstanceDao.getJobInstances(JdbcJobInstanceDao.java:230)
at org.springframework.batch.core.explore.support.SimpleJobExplorer.getJobInstances(SimpleJobExplorer.java:173)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:333)
at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:190)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)
at org.springframework.batch.core.configuration.annotation.SimpleBatchConfiguration$PassthruAdvice.invoke(SimpleBatchConfiguration.java:127)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:213)
at com.sun.proxy.$Proxy302.getJobInstances(Unknown Source)
at org.springframework.boot.autoconfigure.batch.JobLauncherCommandLineRunner.getNextJobParameters(JobLauncherCommandLineRunner.java:131)
at org.springframework.boot.autoconfigure.batch.JobLauncherCommandLineRunner.execute(JobLauncherCommandLineRunner.java:212)
at org.springframework.boot.autoconfigure.batch.JobLauncherCommandLineRunner.executeLocalJobs(JobLauncherCommandLineRunner.java:231)
at org.springframework.boot.autoconfigure.batch.JobLauncherCommandLineRunner.launchJobFromProperties(JobLauncherCommandLineRunner.java:123)
at org.springframework.boot.autoconfigure.batch.JobLauncherCommandLineRunner.run(JobLauncherCommandLineRunner.java:117)
at org.springframework.boot.SpringApplication.callRunner(SpringApplication.java:732)
... 16 more
Caused by: com.ibm.db2.jcc.am.SqlSyntaxErrorException: DB2 SQL Error: SQLCODE=-204, SQLSTATE=42704, SQLERRMC=TEST.BATCH_JOB_INSTANCE, DRIVER=4.18.60
at com.ibm.db2.jcc.am.kd.a(kd.java:747)
at com.ibm.db2.jcc.am.kd.a(kd.java:66)
at com.ibm.db2.jcc.am.kd.a(kd.java:135)
at com.ibm.db2.jcc.am.bp.c(bp.java:2788)
at com.ibm.db2.jcc.am.bp.d(bp.java:2776)
at com.ibm.db2.jcc.am.bp.a(bp.java:2209)
at com.ibm.db2.jcc.am.cp.a(cp.java:7886)
at com.ibm.db2.jcc.t4.bb.h(bb.java:141)
at com.ibm.db2.jcc.t4.bb.b(bb.java:41)
at com.ibm.db2.jcc.t4.p.a(p.java:32)
at com.ibm.db2.jcc.t4.vb.i(vb.java:145)
at com.ibm.db2.jcc.am.bp.kb(bp.java:2178)
at com.ibm.db2.jcc.am.cp.xc(cp.java:3686)
at com.ibm.db2.jcc.am.cp.b(cp.java:4493)
at com.ibm.db2.jcc.am.cp.kc(cp.java:767)
at com.ibm.db2.jcc.am.cp.executeQuery(cp.java:732)
at org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.executeQuery(WrappedPreparedStatement.java:504)
at org.springframework.jdbc.core.JdbcTemplate$1.doInPreparedStatement(JdbcTemplate.java:692)
at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:633)
... 38 more
Answering my own question. I am able use spring batch without metadata tables using the below configuration:
#Configuration
public class SpringBatchInMemoryConfiguration {
#Bean
public ResourcelessTransactionManager transactionManager() {
return new ResourcelessTransactionManager();
}
#Bean
public MapJobRepositoryFactoryBean mapJobRepositoryFactory(ResourcelessTransactionManager txManager)
throws Exception {
MapJobRepositoryFactoryBean factory = new MapJobRepositoryFactoryBean(txManager);
factory.afterPropertiesSet();
return factory;
}
#Bean
public JobRepository jobRepository(MapJobRepositoryFactoryBean factory) throws Exception {
return factory.getObject();
}
#Bean
public JobExplorer jobExplorer(MapJobRepositoryFactoryBean factory) {
return new SimpleJobExplorer(factory.getJobInstanceDao(), factory.getJobExecutionDao(),
factory.getStepExecutionDao(), factory.getExecutionContextDao());
}
#Bean
public SimpleJobLauncher jobLauncher(JobRepository jobRepository) throws Exception {
SimpleJobLauncher launcher = new SimpleJobLauncher();
launcher.setJobRepository(jobRepository);
launcher.afterPropertiesSet();
return launcher;
}
}

Spring Batch: AsyncItemProcessor and AsyncItemWriter

1) I have a large file (> 100k lines) that needs to be processed. I have a lot of business validation and checks against external systems for each line item. The code is being migrated from a legacy app and i just put these business logic into the AsyncitemProcessor, which also persists the data into the DB. Is this a good practise to create/save records in the ItemProcessor (in lieu of ItemWriter) ?
2) Code is ::
#Configuration
#EnableAutoConfiguration
#ComponentScan(basePackages = "com.liquidation.lpid")
#EntityScan(basePackages = "com.liquidation.lpid.entities")
#EnableTransactionManagement
public class SimpleJobConfiguration {
#Autowired
public JobRepository jobRepository;
#Autowired
private StepBuilderFactory stepBuilderFactory;
#Autowired
#Qualifier("myFtpSessionFactory")
private SessionFactory myFtpSessionFactory;
#Autowired
public JobBuilderFactory jobBuilderFactory;
#Bean
public ThreadPoolTaskExecutor lpidItemTaskExecutor() {
ThreadPoolTaskExecutor tExec = new ThreadPoolTaskExecutor();
tExec.setCorePoolSize(10);
tExec.setMaxPoolSize(10);
tExec.setAllowCoreThreadTimeOut(true);
return tExec;
}
#BeforeStep
public void beforeStep(StepExecution stepExecution){
String name = stepExecution.getStepName();
System.out.println("name: " + name);
}
#Bean
public SomeItemWriterListener someItemWriterListener(){
return new SomeItemWriterListener();
};
#Bean
#StepScope
public FlatFileItemReader<FieldSet> lpidItemReader(#Value("#{stepExecutionContext['fileResource']}") String fileResource) {
System.out.println("itemReader called !!!!!!!!!!! for customer data" + fileResource);
FlatFileItemReader<FieldSet> reader = new FlatFileItemReader<FieldSet>();
reader.setResource(new ClassPathResource("/data/stage/"+ fileResource));
reader.setLinesToSkip(1);
DefaultLineMapper<FieldSet> lineMapper = new DefaultLineMapper<FieldSet>();
DelimitedLineTokenizer tokenizer = new DelimitedLineTokenizer();
reader.setSkippedLinesCallback(new LineCallbackHandler() {
public void handleLine(String line) {
if (line != null) {
tokenizer.setNames(line.split(","));
}
}
});
lineMapper.setLineTokenizer(tokenizer);
lineMapper.setFieldSetMapper(new PassThroughFieldSetMapper());
lineMapper.afterPropertiesSet();
reader.setLineMapper(lineMapper);
return reader;
}
#Bean
public ItemWriter<FieldSet> lpidItemWriter() {
return new LpidItemWriter();
}
#Autowired
private MultiFileResourcePartitioner multiFileResourcePartitioner;
#Bean
public Step masterStep() {
return stepBuilderFactory.get("masterStep")
.partitioner(slaveStep().getName(), multiFileResourcePartitioner)
.step(slaveStep())
.gridSize(4)
.taskExecutor(lpidItemTaskExecutor())
.build();
}
#Bean
public ItemProcessListener<FieldSet,String> processListener(){
return new LpidItemProcessListener();
}
#SuppressWarnings("unchecked")
#Bean
public Step slaveStep() {
return stepBuilderFactory.get("slaveStep")
.<FieldSet,FieldSet>chunk(5)
.faultTolerant()
.listener(new ChunkListener())
.reader(lpidItemReader(null))
.processor(asyncItemProcessor())
.writer(asyncItemWriter()).listener(someItemWriterListener()).build();
}
#Bean
public AsyncItemWriter<FieldSet> asyncItemWriter(){
AsyncItemWriter<FieldSet> asyncItemProcessor = new AsyncItemWriter<>();
asyncItemProcessor.setDelegate(lpidItemWriter());
try {
asyncItemProcessor.afterPropertiesSet();
} catch (Exception e) {
e.printStackTrace();
}
return asyncItemProcessor;
}
#Bean
public ItemProcessor<FieldSet, FieldSet> processor() {
return new lpidCheckItemProcessor();
}
#Bean
public AsyncItemProcessor<FieldSet, FieldSet> asyncItemProcessor() {
AsyncItemProcessor<FieldSet, FieldSet> asyncItemProcessor = new AsyncItemProcessor<FieldSet, FieldSet>();
asyncItemProcessor.setDelegate(processor());
asyncItemProcessor.setTaskExecutor(lpidItemTaskExecutor());
try {
asyncItemProcessor.afterPropertiesSet();
} catch (Exception e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
return asyncItemProcessor;
}
#Bean
public Job job() throws Exception {
return jobBuilderFactory.get("job").incrementer(new RunIdIncrementer()).start(masterStep()).build();
}
}
The itemwriter runs before the itemprocessor has completed. My understanding is: for every chunk, the item reader reads the data, item processor will churn through each item, and at the end of the chunk, the item writer gets called (which in my case,it does not do anything since the itemprocessor persists the data). But the itemwriter gets called before the item processor gets completed and my job never completes. What am i doing incorrectly here? (I looked at previous issues around it and the solution was to wrap the writer around the AsyncItemWriter(), which i am doing) .
Thanks
Sundar

How to run update query in Spring JPA for quartz job

I have a quartz job in spring 4 and I am using JPA hibernate to update database value through quartz job but I am getting javax.persistence.TransactionRequiredException: Executing an update/delete query
I don't understand what kind of configuration is missing in quartz job. I referred to SpringBeanAutowiringSupport example still update is failing but select is working fine.
Below is my code
#Configuration
#ComponentScan("com.stock")
public class QuartzConfiguration {
#Autowired
private ApplicationContext applicationContext;
#Bean
public JobDetailFactoryBean jobDetailBalanceCarryForward(){
JobDetailFactoryBean factory = new JobDetailFactoryBean();
factory.setJobClass(BillingCroneSvcImpl.class);
Map<String,Object> map = new HashMap<String,Object>();
map.put("task", "balanceCarryForward");
factory.setJobDataAsMap(map);
factory.setGroup("BalanceCarryForwardJob");
factory.setName("balance carry forward");
return factory;
}
#Bean
public CronTriggerFactoryBean cronTriggerBalanceCarryForward(){
CronTriggerFactoryBean stFactory = new CronTriggerFactoryBean();
stFactory.setJobDetail(jobDetailBalanceCarryForward().getObject());
stFactory.setStartDelay(3000);
stFactory.setName("balancCarryForwardTrigger");
stFactory.setGroup("balanceCarryForwardgroup");
stFactory.setCronExpression("0 0/1 * 1/1 * ? *");
return stFactory;
}
#Bean
public SpringBeanJobFactory springBeanJobFactory() {
AutoWiringSpringBeanJobFactory jobFactory = new AutoWiringSpringBeanJobFactory();
jobFactory.setApplicationContext(applicationContext);
return jobFactory;
}
#Bean
public SchedulerFactoryBean schedulerFactoryBean() {
SchedulerFactoryBean schedulerFactory = new SchedulerFactoryBean();
schedulerFactory.setJobFactory(springBeanJobFactory());
schedulerFactory.setTriggers(cronTriggerBalanceCarryForward().getObject());
return schedulerFactory;
}
}
Below class where quartz executeInternal method is written
#Service
#PersistJobDataAfterExecution
#DisallowConcurrentExecution
#Autowired
private BillingCroneRepo billingCroneRepo;
public class BillingCroneSvcImpl extends QuartzJobBean implements BillingCroneSvc {
#Override
#Transactional
protected void executeInternal(JobExecutionContext context) throws JobExecutionException {
SpringBeanAutowiringSupport.processInjectionBasedOnCurrentContext(context);
billingCroneRepo.updateBalance();
// this method throws exception javax.persistence.TransactionRequiredException: Executing an update/delete query
}
}
App config class
#EnableWebMvc
#EnableTransactionManagement
#Configuration
#ComponentScan({ "com.stock.*" })
#Import({ SecurityConfig.class })
#PropertySource("classpath:jdbc.properties")
public class AppConfig extends WebMvcConfigurerAdapter {
private static final String PROPERTY_NAME_DATABASE_DRIVER = "db.driver";
private static final String PROPERTY_NAME_DATABASE_PASSWORD = "db.password";
private static final String PROPERTY_NAME_DATABASE_URL = "db.url";
private static final String PROPERTY_NAME_DATABASE_USERNAME = "db.username";
private static final String PROPERTY_NAME_HIBERNATE_DIALECT = "hibernate.dialect";
private static final String PROPERTY_NAME_HIBERNATE_SHOW_SQL = "hibernate.show_sql";
private static final String PROPERTY_NAME_ENTITYMANAGER_PACKAGES_TO_SCAN = "entitymanager.packages.to.scan";
#Resource
private Environment env;
#Bean(name = "dataSource")
public DriverManagerDataSource dataSource() {
DriverManagerDataSource driverManagerDataSource = new DriverManagerDataSource();
driverManagerDataSource.setDriverClassName(env.getRequiredProperty(PROPERTY_NAME_DATABASE_DRIVER));
driverManagerDataSource.setUrl(env.getRequiredProperty(PROPERTY_NAME_DATABASE_URL));
driverManagerDataSource.setUsername(env.getRequiredProperty(PROPERTY_NAME_DATABASE_USERNAME));
driverManagerDataSource.setPassword(env.getRequiredProperty(PROPERTY_NAME_DATABASE_PASSWORD));
return driverManagerDataSource;
}
#Bean
public LocalContainerEntityManagerFactoryBean entityManagerFactory() {
LocalContainerEntityManagerFactoryBean entityManagerFactoryBean = new LocalContainerEntityManagerFactoryBean();
entityManagerFactoryBean.setDataSource(dataSource());
entityManagerFactoryBean.setPersistenceProviderClass(HibernatePersistence.class);
entityManagerFactoryBean.setPackagesToScan(env.getRequiredProperty(PROPERTY_NAME_ENTITYMANAGER_PACKAGES_TO_SCAN));
entityManagerFactoryBean.setJpaProperties(hibProperties());
return entityManagerFactoryBean;
}
private Properties hibProperties() {
Properties properties = new Properties();
properties.put(PROPERTY_NAME_HIBERNATE_DIALECT,env.getRequiredProperty(PROPERTY_NAME_HIBERNATE_DIALECT));
properties.put(PROPERTY_NAME_HIBERNATE_SHOW_SQL,env.getRequiredProperty(PROPERTY_NAME_HIBERNATE_SHOW_SQL));
return properties;
}
#Bean
public PlatformTransactionManager transactionManager() {
JpaTransactionManager transactionManager = new JpaTransactionManager();
transactionManager.setEntityManagerFactory(entityManagerFactory().getObject());
return transactionManager;
}
#Bean
public ReloadableResourceBundleMessageSource messageSource(){
ReloadableResourceBundleMessageSource messageSource=new ReloadableResourceBundleMessageSource();
String[] resources= {"classpath:messages"};
messageSource.setBasenames(resources);
return messageSource;
}
#Bean
public LocaleResolver localeResolver() {
final CookieLocaleResolver ret = new CookieLocaleResolver();
ret.setDefaultLocale(new Locale("en_IN"));
return ret;
}
#Bean
public LocaleChangeInterceptor localeChangeInterceptor(){
LocaleChangeInterceptor localeChangeInterceptor=new LocaleChangeInterceptor();
localeChangeInterceptor.setParamName("language");
return localeChangeInterceptor;
}
#Override
public void addResourceHandlers(ResourceHandlerRegistry registry) {
registry.addResourceHandler("/Angular/**").addResourceLocations("/Angular/");
registry.addResourceHandler("/css/**").addResourceLocations("/css/");
registry.addResourceHandler("/email_templates/**").addResourceLocations("/email_templates/");
registry.addResourceHandler("/fonts/**").addResourceLocations("/fonts/");
registry.addResourceHandler("/img/**").addResourceLocations("/img/");
registry.addResourceHandler("/js/**").addResourceLocations("/js/");
registry.addResourceHandler("/Landing_page/**").addResourceLocations("/Landing_page/");
}
#Bean
public static PropertySourcesPlaceholderConfigurer properties() {
PropertySourcesPlaceholderConfigurer pspc = new PropertySourcesPlaceholderConfigurer();
org.springframework.core.io.Resource[] resources = new ClassPathResource[] { new ClassPathResource("application.properties") };
pspc.setLocations(resources);
pspc.setIgnoreUnresolvablePlaceholders(true);
return pspc;
}
#Bean
public InternalResourceViewResolver viewResolver() {
InternalResourceViewResolver viewResolver = new InternalResourceViewResolver();
viewResolver.setViewClass(JstlView.class);
viewResolver.setPrefix("/WEB-INF/pages/");
viewResolver.setSuffix(".jsp");
return viewResolver;
}
// through below code we directly read properties file in jsp file
#Bean(name = "propertyConfigurer")
public PropertiesFactoryBean mapper() {
PropertiesFactoryBean bean = new PropertiesFactoryBean();
bean.setLocation(new ClassPathResource("application.properties"));
return bean;
}
}
Can anybody please assist me how to resolve transational issue in spring JPA with quartz
Thanks you all for your help. Finally I autowired EntityManagerFactory instead of persitance EntityManager and it is working fine. I tried all scenario but nothing worked to inject spring transactional in quartz so finally autoriwed entitymanagerfactory
Below is my repo class code.
#Repository
public class BillingCroneRepoImpl implements BillingCroneRepo {
/*#PersistenceContext
private EntityManager entityManager;*/
#Autowired
EntityManagerFactory entityManagerFactory;
public boolean updateTable(){
EntityManager entityManager = entityManagerFactory.createEntityManager();
EntityTransaction entityTransaction = entityManager.getTransaction();
entityTransaction.begin(); // this will go in try catch
Query query = entityManager.createQuery(updateSql);
// update table code goes here
entityTransaction.commit(); // this will go in try catch
}
}
I'm not the Spring specialist, but I think new ... doesn't work with #Transactional
#Service
#PersistJobDataAfterExecution
#DisallowConcurrentExecution
public class BillingCroneSvcImpl extends QuartzJobBean implements BillingCroneSvc {
#Autowired
BillingCroneRepo billingCroneRepo;
#Override
#Transactional
protected void executeInternal(JobExecutionContext context) throws JobExecutionException {
SpringBeanAutowiringSupport.processInjectionBasedOnCurrentContext(context);
billingCroneRepo.updateBalance();
}
}
It's because quartz is using the bean instead of the proxy generated for #Transactional.
Use either MethodInvokingJobDetailFactoryBean (instead of inheriting QuartzJob) or use a dedicated wrapper quarz bean (inheriting from QuartzJob) that call the spring bean (not inheriting from QuartzJob) having the #Transactionnal annotation.
EDIT : this is in fact not the problem
The problem is here :
JobDetailFactoryBean factory = new JobDetailFactoryBean();
factory.setJobClass(BillingCroneSvcImpl.class);
By passing the class, I presume that Quartz will instantiate it itself, so Spring won't create it and won't wrap the bean in a Proxy that handle the #Transactionnal behaviour.
Instead you must use something along the line :
#Bean(name = "billingCroneSvc")
public BillingCroneSvc getSvc(){
return new BillingCroneSvcImpl();
}
#Bean
public JobDetailFactoryBean jobDetailBalanceCarryForward(){
JobDetailFactoryBean factory = new JobDetailFactoryBean();
getSvc();// just make sure the bean is instantiated
factory.setBeanName("billingCroneSvc");
...
}

Spring Batch Integration using Java DSL / launching jobs

I've a working spring boot/batch projet containing 2 jobs.
I'm now trying to add Integration to poll files from a remote SFTP using only java configuration / java DSL, and then launch a job.
The file polling is working but I've no idea on how to launch a Job in my flow, despite reading these links :
Spring Batch Integration config using Java DSL
and
Spring Batch Integration job-launching-gateway
some code snippets:
#Bean
public SessionFactory SftpSessionFactory()
{
DefaultSftpSessionFactory sftpSessionFactory = new DefaultSftpSessionFactory();
sftpSessionFactory.setHost("myip");
sftpSessionFactory.setPort(22);
sftpSessionFactory.setUser("user");
sftpSessionFactory.setPrivateKey(new FileSystemResource("path to my key"));
return sftpSessionFactory;
}
#Bean
public IntegrationFlow ftpInboundFlow() {
return IntegrationFlows
.from(Sftp.inboundAdapter(SftpSessionFactory())
.deleteRemoteFiles(Boolean.FALSE)
.preserveTimestamp(Boolean.TRUE)
.autoCreateLocalDirectory(Boolean.TRUE)
.remoteDirectory("remote dir")
.regexFilter(".*\\.txt$")
.localDirectory(new File("C:/sftp/")),
e -> e.id("sftpInboundAdapter").poller(Pollers.fixedRate(600000)))
.handle("FileMessageToJobRequest","toRequest")
// what to put next to process the jobRequest ?
For .handle("FileMessageToJobRequest","toRequest") I use the one described here http://docs.spring.io/spring-batch/trunk/reference/html/springBatchIntegration.html
I would appreciate any help on that, many thanks.
EDIT after Gary comment
I've added, it doesn't compile -of course- because I don't understand how the request is propagated :
.handle("FileMessageToJobRequest","toRequest")
.handle(jobLaunchingGw())
.get();
}
#Bean
public MessageHandler jobLaunchingGw() {
return new JobLaunchingGateway(jobLauncher());
}
#Autowired
private JobLauncher jobLauncher;
#Bean
public JobExecution jobLauncher(JobLaunchRequest req) throws JobExecutionException {
JobExecution execution = jobLauncher.run(req.getJob(), req.getJobParameters());
return execution;
}
I've found a way to launch a job using a #ServiceActivator and adding this to my flow but I'm not sure it's good practice :
.handle("lauchBatchService", "launch")
#Component("lauchBatchService")
public class LaunchBatchService {
private static Logger log = LoggerFactory.getLogger(LaunchBatchService.class);
#Autowired
private JobLauncher jobLauncher;
#ServiceActivator
public JobExecution launch(JobLaunchRequest req) throws JobExecutionException {
JobExecution execution = jobLauncher.run(req.getJob(), req.getJobParameters());
return execution;
}
}
.handle(jobLaunchingGw())
// handle result
...
#Bean
public MessageHandler jobLaunchingGw() {
return new JobLaunchingGateway(jobLauncher());
}
where jobLauncher() is the JobLauncher bean.
EDIT
Your service activator is doing about the same as the JLG; it uses this code.
Your jobLauncher #Bean is wrong.
#Beans are definitions; they don't do runtime stuff like this
#Bean
public JobExecution jobLauncher(JobLaunchRequest req) throws JobExecutionException {
JobExecution execution = jobLauncher.run(req.getJob(), req.getJobParameters());
return execution;
}
Since you are already autowiring a JobLauncher, just use that.
#Autowired
private JobLauncher jobLauncher;
#Bean
public MessageHandler jobLaunchingGw() {
return new JobLaunchingGateway(jobLauncher);
}