I am setting the maxItemCount property of JdbcPagingItemReader.
I am setting it to 200 but I am getting read/processed/written of 203, 205.
Most of the time I am getting 200 but I get around 200+ commonly.
Why is this happening??
I've checked and there are no same timestamp value for the the sortkey in the 203-205 processed and the max.item.count field is not present in the batch_execution_context entry in the database table.
There is a JdbcPagingItemReader.read.count.max field but it is set to 200.
I am using oracle.
<bean id="batchReader" class="org.springframework.batch.item.database.JdbcPagingItemReader" scope="step">
<property name="dataSource" ref="myDataSource"/>
<property name="queryProvider">
<bean class="org.springframework.batch.item.database.support.SqlPagingQueryProviderFactoryBean">
<property name="dataSource" ref="myDataSource"/>
<property name="selectClause" value="select *" />
<property name="fromClause" value="from TRANSACTION" />
<property name="whereClause" value="where STATUS = 'OK' and TYPE = 200 " />
<property name="sortKey" value="TRANSACTION_TIMESTAMP"/>
</bean>
</property>
<!-- Inject via the ExecutionContext in rangePartitioner -->
<property name="parameterValues">
<map>
</map>
</property>
<property name="maxItemCount" value="200"/>
<property name="pageSize" value="50"/>
<property name="rowMapper">
<bean class="com.mappers.TransactionMapper" scope="step"/>
</property>
</bean>
Related
I have a file to read like below. A record is split into multiple lines.
Each record can have any number of lines; the only way to recognize a new record is when a line starts with "ABC" and there is another line with identifier ABC_SUB. Each line for this record need to have a separate mapper identified by the pattern with the start of the line(e.g., ABC, line2, line3, line, ABC_SUB, line3, line4)
The same pattern line3 and line4 can be present, but they need different mappers based on the previous line type identifier.
In this example,
line3 pattern(with mapper1) exists after a line that starts with ABC and
line3 pattern(with mapper2) exits after a line that starts with ABC_SUB.
How to identify a line3 pattern under ABC vs. a line3 pattern under ABC_SUB?
I tried PatternMatchingCompositeTokenizer, but this gives the first matching mapper.
Is there a way to check a few lines before identifying a subtype(like ABC or ABC_SUB) and giving the respective mapper?
HDR
ABCline1goesonforrecord1 //record starts
line2goesonForRecord1
line3goesonForRecord1 //this requires ABC_line3_mapper
line4goesonForRecord1
ABC_SUBline1goesonforrecord1 //sub-record where it can have same pattern in below lines
line3goesonForRecord1 //this requires ABC_SUB_line3_mapper
line4goesonForRecord1
ABCline2goesOnForRecord2 //record 2 begins
line2goesonForRecord2
line3goesonForRecord2
line4goesonForRecord2
line5goesonForRecord2
ABCline2goesOnForRecord3
line2goesonForRecord3
line3goesonForRecord3
line4goesonForRecord3
TRL
Below is the XML config
<batch:job id="importFileData">
<batch:step id="parseAndLoadData">
<batch:tasklet>
<batch:chunk reader="multiLineReader" writer="writer"
commit-interval="5" skip-limit="100">
<batch:streams>
<batch:stream ref="fileItemReader" />
</batch:streams>
</batch:chunk>
</batch:tasklet>
</batch:step>
</batch:job>
<bean id="fileItemReader"
class="org.springframework.batch.item.file.FlatFileItemReader"
scope="step">
<property name="resource" value="classpath:input/input.txt"></property>
<property name="linesToSkip" value="2" />
<property name="lineMapper">
<bean
class="org.springframework.batch.item.file.mapping.DefaultLineMapper">
<property name="lineTokenizer" ref="customLineTokenizer">
</property>
<property name="fieldSetMapper">
<bean
class="org.springframework.batch.item.file.mapping.PassThroughFieldSetMapper">
</bean>
</property>
</bean>
</property>
</bean>
<bean id="reader" class="sample.MultiLineReader">
<property name="fieldSetReader" ref="fileItemReader" />
<property name="abcMapper" ref="abcMapper" />
<property name="123Mapper" ref="sample123Mapper" />
</bean>
<bean id="orderFileTokenizer"
class="org.springframework.batch.item.file.transform.PatternMatchingCompositeLineTokenizer">
<property name="tokenizers">
<map>
<entry key="ABC*" value-ref="abcLineTokenizer" />
<entry key="123*" value-ref="123LineTokenizer" />
</map>
</property>
</bean>
<bean id="abcLineTokenizer"
class="org.springframework.batch.item.file.transform.FixedLengthTokenizer">
<property name="names" value="NAME, AGE, GENDER" />
<property name="columns" value="1-10,11-15,16-20" />
<property name="strict" value="false" />
</bean>
<bean id="123LineTokenizer"
class="org.springframework.batch.item.file.transform.FixedLengthTokenizer">
<property name="names" value="CONTACT, ALT_CONTACT" />
<property name="columns" value="1-15,16-30" />
<property name="strict" value="false" />
</bean>
<bean id="abcMapper" class="sample.ABCMapper" />
<bean id="sample123Mapper" class="sample.123Mapper" />
</beans>
I'm trying to test spring batch using dbunit but when batch runs HibernateCursorItemReader it doesn't see temporary created objects in the db. But if I create entities using HibernateDaoSupport everything is fine and I can work with test data. Does anybody know how to force batch see my test entities?
My test classes:
#RunWith(SpringEasyMockRunner.class)
#TestExecutionListeners({DependencyInjectionTestExecutionListener.class, TransactionalTestExecutionListener.class, DbUnitTestExecutionListener.class})
#Transactional
#ContextConfiguration({"classpath:/integration-test-context.xml"})
#DbUnitConfiguration(
databaseConnection = {"dbUnitDatabaseConnection"}
)
public abstract class BaseIT implements ApplicationContextAware {
...
}
#RunWith(SpringEasyMockRunner.class)
#ContextConfiguration("classpath:/sopl-test-context.xml")
#DatabaseSetup(value = "classpath:data/SoplTestDataIT.xml", type = DatabaseOperation.CLEAN_INSERT)
#Slf4j
public class SoplOrderTestIT extends BaseIT {
...
}
And configs integration-test-context.xml:
<bean id="dataSource" class="com.mchange.v2.c3p0.ComboPooledDataSource" lazy-init="true" destroy-method="close">
<property name="driverClass" value="${hibernate.driver}"/>
<property name="jdbcUrl" value="${hibernate.connection.url}"/>
<property name="user" value="${hibernate.connection.username}"/>
<property name="password" value="${hibernate.connection.password}"/>
<property name="initialPoolSize" value="${c3p0.initialPoolSize:1}"/>
<property name="minPoolSize" value="${c3p0.minPoolSize:0}"/>
<property name="maxPoolSize" value="${c3p0.maxPoolSize:200}"/>
<property name="maxIdleTime" value="${c3p0.maxIdleTime:120}"/>
<property name="maxStatementsPerConnection" value="${c3p0.maxStatementsPerConnection:100}"/>
<property name="checkoutTimeout" value="${c3p0.checkoutTimeout:30000}"/>
<property name="idleConnectionTestPeriod" value="${c3p0.idleConnectionTestPeriod:3600}"/>
<property name="unreturnedConnectionTimeout" value="${c3p0.unreturnedConnectionTimeout:7200}"/>
<property name="debugUnreturnedConnectionStackTraces"
value="${c3p0.debugUnreturnedConnectionStackTraces:true}"/>
</bean>
<bean name="sessionFactory" class="org.springframework.orm.hibernate4.LocalSessionFactoryBean">
<property name="dataSource" ref="dataSource"/>
<property name="namingStrategy">
<bean class="org.hibernate.cfg.DefaultComponentSafeNamingStrategy"/>
</property>
<property name="configLocations" value="classpath:/hibernate/*-hibernate.cfg.xml"/>
<property name="hibernateProperties">
<props merge="true">
<prop key="hibernate.dialect">org.hibernate.dialect.MySQLInnoDBDialect</prop>
<prop key="hibernate.cache.use_second_level_cache">false</prop>
<prop key="hibernate.show_sql">${hibernate.show_sql:false}</prop>
<prop key="hibernate.format_sql">${hibernate.format_sql:false}</prop>
<prop key="hibernate.use_sql_comments">${hibernate.use_sql_comments:false}</prop>
<prop key="hibernate.generate_statistics">${hibernate.generate_statistics:false}</prop>
<prop key="hibernate.hbm2ddl.auto"/>
<prop key="hibernate.search.autoregister_listeners">false</prop>
</props>
</property>
</bean>
<!-- transaction manager for hibernate -->
<bean id="txManager" class="org.springframework.orm.hibernate4.HibernateTransactionManager" primary="true">
<property name="sessionFactory" ref="sessionFactory"/>
</bean>
<!-- spring-test-dbunit configuration: -->
<bean id="dbUnitDatabaseConfig" class="com.github.springtestdbunit.bean.DatabaseConfigBean">
<property name="qualifiedTableNames" value="true"/>
<property name="metadataHandler">
<bean class="org.dbunit.ext.mysql.MySqlMetadataHandler"/>
</property>
<property name="datatypeFactory">
<bean class="org.dbunit.ext.mysql.MySqlDataTypeFactory"/>
</property>
</bean>
<bean id="dbUnitDatabaseConnection"
class="com.github.springtestdbunit.bean.DatabaseDataSourceConnectionFactoryBean">
<property name="databaseConfig" ref="dbUnitDatabaseConfig"/>
<property name="dataSource" ref="dataSource"/>
</bean>
...
I am trying get job-parameteres-in-to-item-processor-using-spring-batch-annotation.
I implemented it by referring below link; but for me the variable (batchRunName) is coming as null when I try to access it in my Processor class.
Can any one please look at it. I am sure; I am missing some small thing.
How to get Job parameteres in to item processor using spring Batch annotation
public static void main(String[] args) {
contextObj = new ClassPathXmlApplicationContext(springConfig);
jobObj = (Job) contextObj.getBean("XYZ-1001-DD-01");
JobParametersBuilder jobBuilder = new JobParametersBuilder();
System.out.println("args[0] is " + args[0] );
jobBuilder.addString("batchRunName", args[0]);
public class TimeProcessor implements ItemProcessor<Time, TimeMetric> {
private DataSource dataSource;
#Value("#{jobParameters['batchRunNumber']}")
private String batchRunNumber;
public void setBatchRunNumber(String batchRunNumber) {
this.batchRunNumber = batchRunNumber;
}
<bean id="timeProcessor"
class="com.xyz.processor.TimeProcessor" scope="step">
<property name="dataSource" ref="oracledataSource" />
</bean>
=================FULL XML CONFIGURATION========================
<import resource="classpath:/batch/utility/skip/batch_skip.xml" />
<import resource="classpath:/batch/config/context-postgres.xml" />
<import resource="classpath:/batch/config/oracle-database.xml" />
<context:property-placeholder
location="classpath:/batch/jobs/TPF-1001-DD-01/TPF-1001-DD-01.properties" />
<bean id="gridSizePartitioner"
class="com.tpf.partitioner.GridSizePartitioner" />
<task:executor id="taskExecutor" pool-size="${pool.size}" />
<batch:job id="XYZJob" job-repository="jobRepository"
restartable="true">
<batch:step id="XYZSTEP">
<batch:description>Convert TIF files to PDF</batch:description>
<batch:partition partitioner="gridSizePartitioner">
<batch:handler task-executor="taskExecutor"
grid-size="${pool.size}" />
<batch:step>
<batch:tasklet allow-start-if-complete="true">
<batch:chunk commit-interval="${commit.interval}"
skip-limit="${job.skip.limit}">
<batch:reader>
<bean id="timeReader"
class="org.springframework.batch.item.database.JdbcCursorItemReader"
scope="step">
<property name="dataSource" ref="oracledataSource" />
<property name="sql">
<value>
select TIME_ID as timesheetId,count(*),max(CREATION_DATETIME) as creationDateTime , ILN_NUMBER as ilnNumber
from TS_FAKE_NAME
where creation_datetime >= '#{jobParameters['creation_start_date1']} 12.00.00.000000000 AM'
and creation_datetime < '#{jobParameters['creation_start_date2']} 11.59.59.999999999 PM'
and mod(time_id,${pool.size})=#{stepExecutionContext['partition.id']}
group by time_id ,ILN_NUMBER
</value>
</property>
<property name="rowMapper">
<bean
class="org.springframework.jdbc.core.BeanPropertyRowMapper">
<property name="mappedClass"
value="com.tpf.model.Time" />
</bean>
</property>
</bean>
</batch:reader>
<batch:processor>
<bean id="compositeItemProcessor"
class="org.springframework.batch.item.support.CompositeItemProcessor">
<property name="delegates">
<list>
<ref bean="timeProcessor" />
</list>
</property>
</bean>
</batch:processor>
<batch:writer>
<bean id="compositeItemWriter"
class="org.springframework.batch.item.support.CompositeItemWriter">
<property name="delegates">
<list>
<ref bean="timeWriter" />
</list>
</property>
</bean>
</batch:writer>
<batch:skippable-exception-classes>
<batch:include
class="com.utility.skip.BatchSkipException" />
</batch:skippable-exception-classes>
<batch:listeners>
<batch:listener ref="batchSkipListener" />
</batch:listeners>
</batch:chunk>
</batch:tasklet>
</batch:step>
</batch:partition>
</batch:step>
<batch:validator>
<bean
class="org.springframework.batch.core.job.DefaultJobParametersValidator">
<property name="requiredKeys">
<list>
<value>batchRunNumber</value>
<value>creation_start_date1</value>
<value>creation_start_date2</value>
</list>
</property>
</bean>
</batch:validator>
</batch:job>
<bean id="timesheetWriter" class="com.tpf.writer.TimeWriter"
scope="step">
<property name="dataSource" ref="dataSource" />
</bean>
<bean id="timeProcessor"
class="com.tpf.processor.TimeProcessor" scope="step">
<property name="dataSource" ref="oracledataSource" />
</bean>
I think you are facing the issue reported in BATCH-2351.
You can try to provide the job parameter via XML instead of annotation (Since the majority of your config is XML based):
<bean id="timeProcessor" class="com.xyz.processor.TimeProcessor" scope="step">
<property name="dataSource" ref="oracledataSource" />
<property name="batchRunNumber" value="#{jobParameters['batchRunNumber']}" />
</bean>
Hope this helps.
I have a large file which may contain 100K to 500K records. I am planning to use chunk oriented processing and my thought is
1) Split the large file into smaller based on the count let say 10K in each file.
2) If there are 100K records then I will get 10 files each containing 10K reocrds
3) I would like to partition these 10 files and would like to process using 5 threads. I am thinking to use custom MultiResourcePartioner
4) The 5 threads should process all the 10 files created in split process.
5) I don't want to create same number of threads equal to file count as in that case I may face memory issues. What I am looking is whatever the number of files I would like to process them using only 5 threads (I can increase based on my requirements).
Expert could you let me know this can be achieved using spring batch? If yes could you please share pointers or reference implementations
Thanks in advance
The working job-config xml
<description>Spring Batch File Chunk Processing</description>
<import resource="../config/batch-context.xml" />
<batch:job id="file-partition-batch" job-repository="jobRepository" restartable="false">
<batch:step id="master">
<batch:partition partitioner="partitioner" handler="partitionHandler" />
</batch:step>
</batch:job>
<batch:step id="slave">
<batch:tasklet>
<batch:chunk reader="reader" processor="compositeProcessor"
writer="compositeWriter" commit-interval="5">
</batch:chunk>
</batch:tasklet>
</batch:step>
<bean id="partitionHandler" class="org.springframework.batch.core.partition.support.TaskExecutorPartitionHandler">
<property name="taskExecutor" ref="taskExecutor"/>
<property name="step" ref="slave" />
<property name="gridSize" value="5" />
</bean>
<bean id="partitioner" class="com.poc.partitioner.FileMultiResourcePartitioner">
<property name="resources" value="file:/Users/anupghosh/Documents/Spring_Batch/FilePartitionBatch/*.txt" />
<property name="threadName" value="feed-processor" />
</bean>
<bean id="taskExecutor" class="org.springframework.scheduling.concurrent.ThreadPoolTaskExecutor">
<property name="corePoolSize" value="5" />
<property name="maxPoolSize" value="5" />
</bean>
<bean id="reader" class="org.springframework.batch.item.file.FlatFileItemReader" scope="step">
<property name="resource" value="#{stepExecutionContext['fileName']}" />
<property name="lineMapper">
<bean class="org.springframework.batch.item.file.mapping.DefaultLineMapper">
<property name="lineTokenizer">
<bean class="org.springframework.batch.item.file.transform.DelimitedLineTokenizer">
<property name="delimiter" value="|"/>
<property name="names" value="key,docName,docTypCD,itemType,itemNum,launchDate,status" />
</bean>
</property>
<property name="fieldSetMapper">
<bean class="com.poc.mapper.FileRowMapper" />
</property>
</bean>
</property>
</bean>
<bean id="validatingProcessor" class="org.springframework.batch.item.validator.ValidatingItemProcessor">
<constructor-arg ref="feedRowValidator" />
</bean>
<bean id="feedProcesor" class="com.poc.processor.FeedProcessor" />
<bean id="compositeProcessor" class="org.springframework.batch.item.support.CompositeItemProcessor" scope="step">
<property name="delegates">
<list>
<ref bean="validatingProcessor" />
<ref bean="feedProcesor" />
</list>
</property>
</bean>
<bean id="recordDecWriter" class="com.poc.writer.RecordDecWriter" />
<bean id="reconFlatFileCustomWriter" class="com.poc.writer.ReconFileWriter">
<property name="reconFlatFileWriter" ref="reconFlatFileWriter" />
</bean>
<bean id="reconFlatFileWriter" class="org.springframework.batch.item.file.FlatFileItemWriter" scope="step">
<property name="resource" value="file:/Users/anupghosh/Documents/Spring_Batch/recon-#{stepExecutionContext[threadName]}.txt" />
<property name="shouldDeleteIfExists" value="true" />
<property name="lineAggregator">
<bean class="org.springframework.batch.item.file.transform.DelimitedLineAggregator">
<property name="delimiter" value="|" />
<property name="fieldExtractor">
<bean class="org.springframework.batch.item.file.transform.BeanWrapperFieldExtractor">
<property name="names" value="validationError" />
</bean>
</property>
</bean>
</property>
</bean>
<bean id="compositeWriter" class="org.springframework.batch.item.support.CompositeItemWriter">
<property name="delegates">
<list>
<ref bean="recordDecWriter" />
<ref bean="reconFlatFileCustomWriter" />
</list>
</property>
</bean>
<bean id="feedRowValidator" class="org.springframework.batch.item.validator.SpringValidator">
<property name="validator">
<bean class="com.poc.validator.FeedRowValidator"/>
</property>
</bean>
was able to solve this using MultiResourcePartitioner. below are java config
#Bean
public Partitioner partitioner() {
MultiResourcePartitioner partitioner = new MultiResourcePartitioner();
ClassLoader cl = this.getClass().getClassLoader();
ResourcePatternResolver resolver = new PathMatchingResourcePatternResolver(cl);
Resource[] resources = resolver.getResources("file:" + filePath + "/"+"*.csv");
partitioner.setResources(resources);
partitioner.partition(10);
return partitioner;
}
#Bean
public TaskExecutor taskExecutor() {
ThreadPoolTaskExecutor taskExecutor = new ThreadPoolTaskExecutor();
taskExecutor.setMaxPoolSize(4);
taskExecutor.afterPropertiesSet();
return taskExecutor;
}
#Bean
#Qualifier("masterStep")
public Step masterStep() {
return stepBuilderFactory.get("masterStep")
.partitioner(ProcessDataStep())
.partitioner("ProcessDataStep",partitioner())
.taskExecutor(taskExecutor())
.listener(pcStressStepListener)
.build();
}
#Bean
#Qualifier("processData")
public Step processData() {
return stepBuilderFactory.get("processData")
.<pojo, pojo> chunk(5000)
.reader(reader)
.processor(processor())
.writer(writer)
.build();
}
#Bean(name="reader")
#StepScope
public FlatFileItemReader<pojo> reader(#Value("#{stepExecutionContext['fileName']}") String filename) {
FlatFileItemReader<pojo> reader = new FlatFileItemReader<>();
reader.setResource(new UrlResource(filename));
reader.setLineMapper(new DefaultLineMapper<pojo>() {
{
setLineTokenizer(new DelimitedLineTokenizer() {
{
setNames(FILE HEADER);
}
});
setFieldSetMapper(new BeanWrapperFieldSetMapper<pojo>() {
{
setTargetType(pojo.class);
}
});
}
});
return reader;
}
I have this scheduler :
<bean id="odilQuartzScheduler"
class="org.springframework.scheduling.quartz.SchedulerFactoryBean"
lazy-init="false">
<property name="jobFactory">
<bean class="org.springframework.scheduling.quartz.SpringBeanJobFactory"/>
</property>
<property name="schedulerName" value="OdilScheduler"/>
<property name="applicationContextSchedulerContextKey" value="applicationContext"/>
<property name="autoStartup" value="${popo.scheduler}"/>
<property name="startupDelay" value="60"/>
<property name="overwriteExistingJobs" value="true"/>
<property name="configLocation" value="classpath:quartz-odil.properties"/>
<property name="transactionManager" ref="applicationTransactionManager"/>
<property name="schedulerContextAsMap">
<map>
<entry key="globalConfiguration" value-ref="globalConfiguration"/>
<entry key="odilFileImporter" value-ref="odilFileImporter"/>
<entry key="odilRemoteDispatcher" value-ref="odilRemoteDispatcher"/>
<entry key="odilService" value-ref="odilService"/>
<entry key="centreReferenceService" value-ref="centreReferenceService"/>
<entry key="envoiParserFactory" value-ref="envoiParserFactory"/>
<entry key="versionReferenceService" value-ref="versionReferenceService"/>
<entry key="delPublisherService" value-ref="delPublisherService"/>
<entry key="capteurPublisherService" value-ref="capteurPublisherService"/>
</map>
</property>
<property name="triggers">
<list>
<ref bean="piloteOdilTrigger-1"/>
<ref bean="piloteOdilTrigger-2"/>
<ref bean="piloteOdilTrigger-3"/>
<ref bean="piloteOdilTrigger-4"/>
<ref bean="piloteOdilTrigger-5"/>
<ref bean="piloteOdilTrigger-6"/>
</list>
</property>
</bean>
<bean id="piloteOdilTrigger-1" class="org.springframework.scheduling.quartz.SimpleTriggerBean">
<property name="name" value="Trigger 1"/>
<property name="jobDetail" ref="piloteOdilJobBean1"/>
<property name="repeatInterval" value="60000"/>
<property name="group" value="POPO"/>
</bean>
<bean id="piloteOdilTrigger-2" class="org.springframework.scheduling.quartz.CronTriggerBean">
<property name="name" value="Trigger 2"/>
<property name="jobDetail" ref="piloteOdilJobBean2"/>
<property name="cronExpression" value="${popo.odil.rushhour.cronexpression}" />
<property name="group" value="POPO"/>
</bean>
<bean id="piloteOdilTrigger-3" class="org.springframework.scheduling.quartz.CronTriggerBean">
<property name="name" value="Trigger 3"/>
<property name="jobDetail" ref="piloteOdilJobBean3"/>
<property name="cronExpression" value="${popo.odil.rushhour.cronexpression}" />
<property name="group" value="POPO"/>
</bean>
<bean id="piloteOdilTrigger-4" class="org.springframework.scheduling.quartz.CronTriggerBean">
<property name="name" value="Trigger 4"/>
<property name="jobDetail" ref="piloteOdilJobBean4"/>
<property name="cronExpression" value="${popo.odil.rushhour.cronexpression}" />
<property name="group" value="POPO"/>
</bean>
<bean id="piloteOdilTrigger-5" class="org.springframework.scheduling.quartz.CronTriggerBean">
<property name="name" value="Trigger 5"/>
<property name="jobDetail" ref="piloteOdilJobBean5"/>
<property name="cronExpression" value="${popo.odil.rushhour.cronexpression}" />
<property name="group" value="POPO"/>
</bean>
<bean id="piloteOdilTrigger-6" class="org.springframework.scheduling.quartz.CronTriggerBean">
<property name="name" value="Trigger 6"/>
<property name="jobDetail" ref="piloteOdilJobBean6"/>
<property name="cronExpression" value="${popo.odil.rushhour.cronexpression}" />
<property name="group" value="POPO"/>
</bean>
<bean id="piloteOdilJobBean-template" abstract="true"
class="org.springframework.scheduling.quartz.JobDetailBean">
<property name="name" value="Pilote ODIL"/>
<property name="jobClass" value="fr.xxxx.popo.batch.PiloteOdilJob"/> <!-- implements StatefulJob -->
<property name="requestsRecovery" value="true"/>
<property name="group" value="POPO"/>
</bean>
<bean id="piloteOdilJobBean1" parent="piloteOdilJobBean-template" />
<bean id="piloteOdilJobBean2" parent="piloteOdilJobBean-template" />
<bean id="piloteOdilJobBean3" parent="piloteOdilJobBean-template" />
<bean id="piloteOdilJobBean4" parent="piloteOdilJobBean-template" />
<bean id="piloteOdilJobBean5" parent="piloteOdilJobBean-template" />
<bean id="piloteOdilJobBean6" parent="piloteOdilJobBean-template" />
this is it's configuration file
#============================================================================
# Main Scheduler Properties
#============================================================================
org.quartz.scheduler.instanceName = OdilScheduler
org.quartz.scheduler.instanceId = AUTO
org.quartz.scheduler.rmi.export = false
org.quartz.scheduler.rmi.proxy = false
#============================================================================
# ThreadPool
#============================================================================
org.quartz.threadPool.class = org.quartz.simpl.SimpleThreadPool
org.quartz.threadPool.threadCount = 6
org.quartz.threadPool.threadPriority = 5
#============================================================================
# JobStore
#============================================================================
org.quartz.jobStore.class = org.quartz.simpl.RAMJobStore
org.quartz.jobStore.misfireThreshold = 3
#============================================================================
# Plugins
#============================================================================
org.quartz.plugin.shutdownhook.class = org.quartz.plugins.management.ShutdownHookPlugin
org.quartz.plugin.shutdownhook.cleanShutdown = true
From what I understood using a ramJobStore make it non clusterable and thus load balancing should not work.
But in reallity when I have 2 servers running, each one has it's own OdilScheduler, with 6 triggers each.
However a total of only 6 trigger will work at the same time.
3 on instance 1 and 3 on instance 2 for example, I want all the 12 trigger to work simultaniously.
What am I missing?
Thanks
ps : quartz version : 1.8.5
I was mislead, in fact I only tried with 3 trigger on each instance.
1 normal and 2 cron trigger.
The cron trigger were using the RAMJobStore, but because of the template, each trigger had the same name so each instance could only use 1 cron trigger (they are identified by their name).
Which is why it looked like load balancing was happening, in fact what was happening is that since they had the same name only one trigger was triggered per instance.
If I had set 5 cron trigger for each instance I would have had only 1 trigger per instance.