problems with ApplicationContext and Spring batch - quartz-scheduler

i'm working with Spring batch, i've done the batch job, configured with an xml file,
i also put all the Quartz configuration in that xml file, (the trigger, schedulerFactoryBean and jobDetail); this is a java project, and i'm trying to load the application context, as an stand alone in a main class; as far as the documentation says, this should make Quartz to start running and is doing it, the problem is when the job runs with the trigger and calls the service, is like all the Autowired beans hadn’t had been loaded, so is giving me an NullpointerException…
this is the code that the job calls after the trigger is fired, and when the JobParametersBuilder is created is when everything crash, Quartz still running though...
could someone helpme with this?
//class called by the job
public class MainJobClass {
private static Logger log = Logger.getLogger(MainJobClass.class);
#Autowired
private SimpleJobLauncher launcher;
#Autowired
private Job job;
public void executeJob(){
try{
log.info("***** Staring job......");
JobParametersBuilder builder = new JobParametersBuilder();
builder.addDate("date", new Date());
builder.addString("sendEmailJob", "Send email to approvers");
JobParameters parameters = builder.toJobParameters();
launcher.run(job, parameters);
}catch(Exception e){
log.error("Error on executing job"+e.fillInStackTrace());
}
}
public void setLauncher(SimpleJobLauncher launcher) {
this.launcher = launcher;
}
public void setJob(Job job) {
this.job = job;
}
simple main method calling App context:
public static void main(String[] args){
ApplicationContext context = new ClassPathXmlApplicationContext("/com/ge/grt/email/grt_email_send.xml");
}
error line:
INFO [DefaultQuartzScheduler_Worker-1] (MainJobClass.java:29) - ***** Staring job......
ERROR [DefaultQuartzScheduler_Worker-1] (MainJobClass.java:40) - Error on executing jobjava.lang.NullPointerException
this are the Quartz beans on the xml file:
<!-- Scheudler Factory bean, the job will run when the context is loaded -->
<bean id="schedulerFactoryBean"
class="org.springframework.scheduling.quartz.SchedulerFactoryBean">
<property name="triggers">
<list>
<ref bean="beanTrigger"></ref>
</list>
</property>
</bean>
<!-- definition of the trigger -->
<!-- defining the execution date: (once every week on monday at 8:00 AM) -->
<bean id="beanTrigger" class="org.springframework.scheduling.quartz.CronTriggerBean">
<property name="jobDetail" ref="jobDetail" />
<property name="misfireInstructionName" value="MISFIRE_INSTRUCTION_FIRE_ONCE_NOW"/>
<!-- <property name="cronExpression" value="0 0 8 ? * MON" /> -->
<property name="cronExpression" value="0 0/1 * * * ?" />
</bean>
<!-- definiton of job detail bean -->
<bean id="jobDetail"
class="org.springframework.scheduling.quartz.MethodInvokingJobDetailFactoryBean">
<property name="targetObject" ref="mainJobClass" />
<property name="targetMethod" value="executeJob" />
<property name="concurrent" value="false"></property>
</bean>

Try org.springframework.scheduling.quartz.JobDetailBean along with jobDataAsMap for job class DI
Ex:
http://static.springsource.org/spring/docs/3.0.x/spring-framework-reference/html/scheduling.html#scheduling-quartz-jobdetail

Related

Quarkus + Camel JPA with multiple Datasources

I'm trying to use Camel to move data from one database to another. We used to use Spring to define two different JPA components as follows:
<bean id="txManager" class="org.springframework.transaction.jta.JtaTransactionManager">
<property name="transactionManager" ref="transactionManager" />
</bean>
<bean id="jpa1" class="org.apache.camel.component.jpa.JpaComponent">
<property name="entityManagerFactory" ref="entityManagerFactory1" />
<property name="transactionManager" ref="txManager" />
</bean>
<bean id="jpa2" class="org.apache.camel.component.jpa.JpaComponent">
<property name="entityManagerFactory" ref="entityManagerFactory2" />
<property name="transactionManager" ref="txManager" />
</bean>
which allows us to use each jpa component separately in the Routes class as follows:
from(jpa("jpa1","org.src.entity.DataSource1"))
-OR-
from(jpa("jpa2","org.src.entity.DataSource2"))
But now in Quarkus I'm unable to find a way to define these JPA components!
Here is my application.properties:
#postgresql 'default datasource'
quarkus.datasource.db-kind=postgresql
quarkus.datasource.username=admin
quarkus.datasource.password=admin
quarkus.datasource.jdbc.url=jdbc:postgresql://localhost:5555/postgres
quarkus.datasource.jdbc.min-size=2
quarkus.datasource.jdbc.max-size=20
quarkus.hibernate-orm.database.generation=drop-and-create
quarkus.hibernate-orm.packages=org.src.target
quarkus.hibernate-orm.log.sql=true
#oracle
quarkus.datasource.oracle.db-kind=oracle
quarkus.datasource.oracle.username=admin
quarkus.datasource.oracle.password=admin
quarkus.datasource.oracle.jdbc.url=jdbc:oracle:thin:#//{{link}}
quarkus.datasource.oracle.jdbc.min-size=2
quarkus.datasource.oracle.jdbc.max-size=20
quarkus.hibernate-orm.oracle.datasource=oracle
quarkus.hibernate-orm.oracle.packages=org.src.source
quarkus.hibernate-orm.oracle.database.default-schema=AAAA
# Quarkus Narayana JTA
quarkus.transaction-manager.object-store-directory=target/narayana
quarkus.transaction-manager.enable-recovery=true
# Camel
camel.rest.context-path=/api
I tried using the oracle datasource as a jpa component but it did not work! I also tried declaring a new JPAComponent in Routes:
#ApplicationScoped
public class Routes extends EndpointRouteBuilder {
#Inject
#PersistenceUnit("oracle")
EntityManagerFactory entityManagerFactory;
#Inject
CamelContext camelContext;
#Override
public void configure() throws Exception {
JpaComponent jpaComponent = new JpaComponent();
jpaComponent.setEntityManagerFactory(entityManagerFactory);
camelContext.addComponent("jpaComponent", jpaComponent);
}
}
But got this error instead: java.lang.ClassNotFoundException: org.springframework.transaction.support.TransactionCallback
I would really appreciate any suggestions.

Stop task when there is an exception thrown in ItemProcessor

I am designing a Spring Batch, which reads multiple csv files. I have used partitioning to read each file in chunk and process it to decrypt a certain column in the csv. Before decrypting if i encounter any validation error , i throw custom exception.
Now what i want is if the processing finds any validation error in the first line, the other lines should not be processed, and the job should end.
How can i achieve this? I tried to implement ProcessorListener too but it has no StepExecution object so that i can call SetTerminateOnly() or ExitStatus=Failed
Also note that i have multiple thread accessing the file in different lines.I want to kill all threads in the event of the first encountered error.
Thanks in advance
So, I identified that running multiple asynchronous concurrent threads (Spring Batch partitioning) was the real issue. Though one of the thread threw an Exception, the other threads were parallely running, and finished executing till the end.
Ath the end, the Job FAILED overall and there was no output processed, but it consumed time to process rest of the data.
Well,the solution to it is as simple as it gets. We just need stop the Job while encountering an error during processing.
The Custom Processor
public class MultiThreadedFlatFileItemProcessor implements ItemProcessor<BinFileVO, BinFileVO>,JobExecutionListener{
private JobExecution jobExecution;
private RSADecrypter decrypter;
public RSADecrypter getDecrypter() {
return decrypter;
}
public void setDecrypter(RSADecrypter decrypter) {
this.decrypter = decrypter;
}
#Override
/**
This method is used process the encrypted data
#param item
* */
public BinFileVO process(BinFileVO item) throws JobException {
if(null!=item.getEncryptedText() && !item.getEncryptedText().isEmpty()){
String decrypted = decrypter.getDecryptedText(item.getEncryptedText());
if(null!=decrypted && !decrypted.isEmpty()){
if(decrypted.matches("[0-9]+")){
if(decrypted.length() >= 12 && decrypted.length() <= 19){
item.setEncryptedText(decrypted);
}else{
this.jobExecution.stop();
throw new JobException(PropertyLoader.getValue(ApplicationConstants.DECRYPTED_CARD_NO_LENGTH_INVALID),item.getLineNumber());
}
}
}else{
this.jobExecution.stop();
throw new JobException(PropertyLoader.getValue(ApplicationConstants.EMPTY_ENCRYPTED_DATA),item.getLineNumber());
}
return item;
}
#Override
public void beforeJob(JobExecution jobExecution) {
this.jobExecution=jobExecution;
}
#Override
public void afterJob(JobExecution jobExecution) {
}
}
The Job xml config
<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
.....>
<!-- JobRepository and JobLauncher are configuration/setup classes -->
<bean id="jobRepository" class="org.springframework.batch.core.repository.support.MapJobRepositoryFactoryBean" />
<bean id="jobLauncher" class="org.springframework.batch.core.launch.support.SimpleJobLauncher">
<property name="jobRepository" ref="jobRepository" />
</bean>
<!-- Job Details -->
<job id="simpleMultiThreadsReaderJob" xmlns="http://www.springframework.org/schema/batch">
<step id="step" >
<partition step="step1" partitioner="partitioner">
<handler grid-size="5" task-executor="taskExecutor"/>
</partition>
</step>
<listeners>
<listener ref="decryptingItemProcessor"/>
</listeners>
</job>
<step id="step1" xmlns="http://www.springframework.org/schema/batch">
<tasklet>
<chunk reader="itemReader" writer="itemWriter" processor="decryptingItemProcessor" commit-interval="500"/>
<listeners>
<listener ref="customItemProcessorListener" />
</listeners>
</tasklet>
</step>
<!-- Processor Details -->
<bean id="decryptingItemProcessor" class="com.test.batch.io.MultiThreadedFlatFileItemProcessor">
<property name="decrypter" ref="rsaDecrypter" />
</bean>
<!-- RSA Decrypter class -->
<bean id="rsaDecrypter" class="test.batch.secure.rsa.client.RSADecrypter"/>
<!-- Partitioner Details -->
<bean class="org.springframework.batch.core.scope.StepScope" />
<bean id="partitioner" class="com.test.batch.partition.FlatFilePartitioner" scope="step">
<property name="resource" ref="inputFile"/>
</bean>
<bean id="taskExecutor"
class="org.springframework.scheduling.concurrent.ThreadPoolTaskExecutor">
<property name="corePoolSize" value="10"/>
</bean>
<!-- Step will need a transaction manager -->
<bean id="transactionManager" class="org.springframework.batch.support.transaction.ResourcelessTransactionManager" />
........
.................
</beans>
Here are the logs
2016-09-01 06:32:40 INFO SimpleJobRepository:273 - Parent JobExecution is stopped, so passing message on to StepExecution
2016-09-01 06:32:43 INFO ThreadStepInterruptionPolicy:60 - Step interrupted through StepExecution
2016-09-01 06:32:43 INFO AbstractStep:216 - Encountered interruption executing step: Job interrupted status detected.
; org.springframework.batch.core.JobInterruptedException
2016-09-01 06:32:45 ERROR CustomJobListener:163 - exception :At line No. 1 : The decrypted card number is less than 12 or greater than 19 in length
2016-09-01 06:32:45 ERROR CustomJobListener:163 - exception :Job interrupted status detected.
2016-09-01 06:32:45 INFO SimpleJobLauncher:135 - Job: [FlowJob: [name=simpleMultiThreadsReaderJob]] completed with the following parameters: [{outputFile=/usr/local/pos/bulktokenization/csv/outputs/cc_output_EDWError_08162016.csv, partitionFile=/usr/local/pos/bulktokenization/csv/partitions/, inputFile=C:\usr\local\pos\bulktokenization\csv\inputs\cc_input_EDWError_08162016.csv, fileName=cc_input_EDWError_08162016}] and the following status: [FAILED]
2016-09-01 06:32:45 INFO BatchLauncher:122 - Exit Status : FAILED
2016-09-01 06:32:45 INFO BatchLauncher:123 - Time Taken : 8969
If we throw Custom Exception in Processor, Spring Batch will terminate and mark the job failed unless you setup 'skipable' exception. You have not mentioned where you perform validate step, are you doing in Processor or Reader? Let me know because it is where Spring Batch decides.
In my project, if I want to stop the job and throw Custom Exception, we put validation logic in a Tasklet or Processor and throw exception as below
private AccountInfoEntity getAccountInfo(Long partnerId) {
if(partnerId != null){
.....
return ....;
} else {
throw new ReportsException("XXXXX");
}
}

Spring-batch Entity Manager becomes null after init

I'm currently implementing a Spring-batch that reads and writes to files BUT also needs to do CRUD operations on a database.
I've tried to simply define an Entity manager in my xml configuration, and use it in my DAO class. However, right after the init, the EntityManager becomes null.
Can anyone help me with this ? (a solution or a link via something usable would be perfect).
My batchContext.xml
<bean id="dataSource" class="org.apache.commons.dbcp.BasicDataSource" destroy-method="close">
<property name="driverClassName" value="${batch.datasource.driverClassName}"/>
<property name="url" value="${batch.datasource.url}"/>
<property name="username" value="${batch.datasource.username}"/>
<property name="password" value="${batch.datasource.password}"/>
<property name="testOnBorrow" value="true"/>
<property name="testOnReturn" value="true"/>
<property name="testWhileIdle" value="true"/>
<property name="timeBetweenEvictionRunsMillis" value="1800000"/>
<property name="numTestsPerEvictionRun" value="3"/>
<property name="minEvictableIdleTimeMillis" value="1800000"/>
</bean>
<bean id="entityManagerFactory" name="entTransactionMgr" class="org.springframework.orm.jpa.LocalContainerEntityManagerFactoryBean">
<!-- <property name="persistenceXmlLocation" value="classpath:/META-INF/spring/persistence.xml" /> -->
<property name="persistenceUnitName" value="persistenceUnit"/>
<property name="packagesToScan" value="${jpa.scan.packages}"/>
<property name="dataSource" ref="dataSource"/>
<property name="jpaVendorAdapter">
<bean class="org.springframework.orm.jpa.vendor.HibernateJpaVendorAdapter"/>
</property>
<property name="loadTimeWeaver">
<bean class="org.springframework.instrument.classloading.InstrumentationLoadTimeWeaver" />
</property>
<!-- Custom jpaDialect pour le deuxieme batch:job-repository-->
<property name="jpaDialect">
<bean class="fr.mma.soecm.batchfacade.util.CustomHibernateJpaDialect" />
</property>
<property name="jpaProperties">
<props>
<!-- multiple props here-->
</props>
</property>
</bean>
<bean id="transactionManager" class="org.springframework.orm.jpa.JpaTransactionManager">
<property name="entityManagerFactory" ref="entityManagerFactory"/>
</bean>
<!-- mode="aspectj" -->
<tx:annotation-driven transaction-manager="transactionManager"/>
<batch:job-repository id="jobRepository" data-source="dataSource" transaction-manager="transactionManager"/>
<!-- Jobs held separatelly -->
<import resource="batchContext-job.xml"/>
My DAO
#Repository("batchJobDao")
#Transactional
public class BatchJobDaoImpl implements BatchJobDao{
#PersistenceContext(unitName="persistenceUnit")
#Autowired
private EntityManager entityManager;
// #PersistenceContext(unitName="persistenceUnit", type=PersistenceContextType.EXTENDED)
// public void setEntityManager(EntityManager entityManager) {
// System.out.println("Setting Entity Manager :" + entityManager);
// this. entityManager = entityManager;
// }
#PostConstruct
public void init(){
if (entityManager == null){
System.out.println(" Entity Manager is null");
} else {
System.out.println(" Entity Manager is not null");
getAllJobExecutions();
}
}
private static final String RECHERCHER_JOB_EXECUTION = "Select bje from BatchJobExecution bje";
#SuppressWarnings("unchecked")
#Override
#Transactional("transactionManager")
public List<BatchJobExecution> getAllJobExecutions(){
// EntityManagerFactory emf=Persistence.createEntityManagerFactory("entTransactionMgr");
// EntityManager em=emf.createEntityManager();
Query query = entityManager.createQuery(RECHERCHER_JOB_EXECUTION, BatchJobExecution.class);
List<BatchJobExecution> executions = (List<BatchJobExecution>) query.getResultList();
System.out.println("EXES : " + executions);
return executions;
}
}
There is some code commented out because I've tried multiple aproaches (changing the persistence context type, recovering the entity manager "manually", having a persistence.xml file for the entityManager) without succes.
My output when running the job is (without all the extra lines..):
Entity Manager is not null
EXES : [fr.mma.soecm.batchfacade.domain.BatchJobExecution#10e6c33,...]
[BatchService] - Synchronous job launch
[AbstractStep] - Encountered an error executing the step
Caused by: java.lang.NullPointerException
And the null pointer, on debug, is throws by the EntityManager being null when I call the "createQuery" in my DAO.
Thanks for your help.
I'll keep searching on my end.. God Speed!
As mentioned in the comment above, my aparent problem was due to the fact that I was trying to call the Service or the DAO in the Constructor of the Step.
After moving the Service call in the "doRead()" method, I could perform all CRUD operations with the EntityManager.
Please let me know if anyone has questions about this/and how to make it work otherwise, as I've not found any explanation on the internet, since I've began searching last week.

spring mvc send email (javamail) exception not getting caught

I am trying to send an email but before I was getting a nullpointerexception error which was due to mailSender not getting set correctly, now I edited the code as it is shown below and I am not getting any exception but the code breaks at the line
MimeMessage message = mailSender.createMimeMessage();
Here is my code (both sendMail() and addNewAlarm() are inside the same class "ElementService"):
public class ElementService implements ApplicationContextAware {
private ApplicationContext ac;
public void sendMail(String toAddress, String subject, String body) throws Exception{
JavaMailSender mailSender = (JavaMailSender) ac.getBean("mailSender");
MimeMessage message = mailSender.createMimeMessage();
try{
MimeMessageHelper helper = new MimeMessageHelper(message, true);
helper.setFrom("xxx#gmail.com");
helper.setTo(toAddress);
helper.setSubject(subject);
helper.setText(body);
}catch (MessagingException e) {
throw new MailParseException(e);
}
try{
mailSender.send(message);
}
catch(Exception e){
throw e;
}
}
//I want an email to be sent every 30 seconds
#Scheduled(fixedDelay = 30*1000)
public void function2RepeatEvery30Seconds()
{
MailService mailer = (MailService) ac.getBean("mailService");
mailer.sendMail("xxx#hotmail.com","subject","body");
//does other stuff..
}
#Override
public void setApplicationContext(ApplicationContext ac) throws BeansException {
this.ac = ac;
}
}
These are the beans in my xml:
<bean id="mailSender" class="org.springframework.mail.javamail.JavaMailSenderImpl">
<!-- SMTP settings -->
<property name="host" value="smtp.gmail.com" />
<property name="port" value="587" />
<property name="username" value="***#gmail.com" />
<property name="password" value="*****" />
<property name="javaMailProperties">
<!-- additional properties specific to JavaMail -->
<props>
<prop key="mail.transport.protocol">smtp</prop>
<prop key="mail.smtp.auth">true</prop>
<prop key="mail.smtp.starttls.enable">true</prop>
</props>
</property>
</bean>
In debug mode I can see that mailSender has been set according to the properties shown on the mailSender bean.
I finally did it!! Thanks to Serge Ballesta of course. I should have been using log4j all along... I researched the Exception MessageRemovedIOException and I found this post java.lang.NoClassDefFoundError: com/sun/mail/util/MailLogger for JUnit test case for Java mail where there is a suggestion in the comments to change
<dependency> <groupId>javax.mail</groupId> <artifactId>javax.mail-api</artifactId> <version>1.5.2</version> </dependency>
to
<dependency> <groupId>com.sun.mail</groupId> <artifactId>javax.mail</artifactId> <version>1.5.2</version> </dependency>
I also had to change this:
<prop key="mail.smtp.starttls.enable">true</prop>
to this:
<prop key="mail.smtp.starttls.enable">false</prop>
Thank you very much for your time and advice #Serge Ballesta!
Ok, obviously the mailSender is null which causes the error. My guess would be that your #Autowired annotation is not processed. Do you have AutowiredAnnotationBeanPostProcessor registered? You can do that for example by using <context:annotation-config /> in your Spring configuration.
Other option (without using #Autowired) is to manually specify the dependency on mailSender in your mailService bean definition like so:
<bean id="mailService" class="gr.mobics.allweb.service.MailService" scope="singleton">
<property name="mailSender" ref="mailSender" />
</bean>
I have not tested this, but it seems like this is the cause of your problem. If these solutions don't work you, leave a comment and I'll try to update the answer.

junit 4 testing with spring 3.0 and Hibernate 3 in Eclipse - LazyInitializationException

I'm getting a LazyInitializationException trying to test my DAO methods using the tool stack defined in the title. My understanding is that my test must be running outside the hibernate session, or it has been closed before I try to read children objects from my DAO. From reading the documentation, I understood that using the #TransactionConfiguration tag would allow me to define the transaction manager in which to run the tests.
I've read the documentation multiple times and a zillion forum posts. Still slamming my head into my keyboard... What am I missing? Thanks for your help!
my unit test class:
#RunWith(SpringJUnit4ClassRunner.class)
#ContextConfiguration(locations = {
"classpath:/WEB-INF/applicationContext-db.xml",
"classpath:/WEB-INF/applicationContext-hibernate.xml",
"classpath:/WEB-INF/applicationContext.xml" })
#TestExecutionListeners({DependencyInjectionTestExecutionListener.class, CleanInsertTestExecutionListener.class})
#DataSetLocation("test/java/com/yada/yada/dao/dbunit-general.xml")
#TransactionConfiguration(transactionManager="transactionManager", defaultRollback = true)
#Transactional
public class RealmDAOJU4Test {
#Autowired
private DbUnitInitializer dbUnitInitializer;
#Autowired
private RealmDAO realmDAO;
#Test
public void testGetById() {
Integer id = 2204;
Realm realm = realmDAO.get(id);
assertEquals(realm.getName().compareToIgnoreCase(
"South Technical Realm"), 0);
assertEquals(8, realm.getRealmRelationships().size());
}
}
my applicationContext-hibernate.xml:
<bean id="sessionFactory" class="org.springframework.orm.hibernate3.LocalSessionFactoryBean">
<property name="dataSource" ref="dataSource"></property>
<property name="useTransactionAwareDataSource" value="true" />
... other properties removed ...
</bean>
<bean id="transactionManager" class="org.springframework.orm.hibernate3.HibernateTransactionManager">
<property name="sessionFactory">
<ref bean="sessionFactory" />
</property>
</bean>
my dao definition in applicationContext.xml
<bean id="realmDAOTarget" class="com.yada.yada.dao.hibernate.RealmDAOImpl">
<property name="sessionFactory" ref="sessionFactory" />
</bean>
<bean id="realmDAO" class="org.springframework.aop.framework.ProxyFactoryBean">
<property name="proxyInterfaces">
<value>com.yada.yada.dao.RealmDAO</value>
</property>
<property name="interceptorNames">
<list>
<value>hibernateInterceptor</value>
<value>realmDAOTarget</value>
</list>
</property>
</bean>
well, for anyone following along at home, here's what I missed:
TransactionalTestExecutionListener
it is required in the #TestExecutionListeners list for the #Transactional annotation to have any effect.