I'm going to transfer data from csv to mysql database.
I any error occurs during CSV->DB conversion I need to rollback the entire transaction and not only data of current chunk.
in my simple-job.xml is
<batch:job id="jobSimple" job-repository="jobRepository" parent="simpleJob">
<batch:step id="step1">
<batch:tasklet transaction-manager="transactionManager">
<batch:transaction-attributes isolation="DEFAULT" propagation="REQUIRED" timeout="30"/>
<batch:chunk reader="cvsItemReader" processor="Pprocess" writer="databaseItemWriter" commit-interval="100"/>
</batch:tasklet>
</batch:step>
</batch:job>
Related
I use the setup below in a project for a job definition.
On the project the batch-jobs are defined in a database. The xml-job definition below serves as a template for creating all these batch jobs at runtime.
This works fine, except in the case of a BeanCreationException in the dataProcessor. When this exception occurs the skip policy is never called and the batch ends immediately instead.
What could be the reason for that? What do I have to do so that every Exception in the dataProcessor is going to use the SkipPolicy?
Thanks a lot in advance
Christian
Version: spring-batch 3.0.7
<batch:job id="MassenGevoJob" restartable="true">
<batch:step id="selectDataStep" parent="selectForMassenGeVoStep" next="executeProcessorStep" />
<batch:step id="executeProcessorStep"
allow-start-if-complete="true" next="decideExitStatus" >
<batch:tasklet>
<batch:chunk reader="dataReader" processor="dataProcessor"
writer="dataItemWriter" commit-interval="10"
skip-policy="batchSkipPolicy">
</batch:chunk>
<batch:listeners>
<batch:listener ref="batchItemListener" />
<batch:listener ref="batchSkipListener" />
<batch:listener ref="batchChunkListener" />
</batch:listeners>
</batch:tasklet>
</batch:step>
<batch:decision decider="failOnPendingObjectsDecider"
id="decideExitStatus">
<batch:fail on="FAILED_PENDING_OBJECTS" exit-code="FAILED_PENDING_OBJECTS" />
<batch:next on="*" to="endFlowStep" />
</batch:decision>
<batch:step id="endFlowStep">
<batch:tasklet ref="noopTasklet"></batch:tasklet>
</batch:step>
<batch:validator ref="batchParameterValidator" />
<batch:listeners>
<batch:listener ref="batchJobListener" />
</batch:listeners>
</batch:job>
A BeanCreationException isn't really skippable because it usually happens before Spring Batch starts. It's also typically a fatal error for your application (Spring couldn't create a component you've defined as being critical to your application). If the creation of that bean is subject to issues and not having it is ok, I'd suggest wrapping it's creation in a factory so that you can control any exceptions that come out of the creation of that bean. For example, if you can't create your custom ItemProcessor, your FactoryBean could return the PassthroughItemProcessor if that's ok.
I have setted the commit interval of 1000 . But, it is not working. Still, i am getting items at random intervals to my item writer . please advise.
<batch:step id="step1" next ="step2">
<batch:tasklet task-executor="simpleTaskExecutor" throttle-limit="20">
<batch:chunk reader="itemReader" processor="itemProcessor" writer="itemWriter" commit-interval="1000" />
<batch:listeners>
<batch:listener ref="appJobExecutionListener" />
<batch:listener ref="appJobFailureLoggerListener" />
<batch:listener ref="customStepListener" />
</batch:listeners>
</batch:tasklet>
</batch:step>
You are using an Async executor. (Guessing, you have not shown in the config) The async executor is set to 20, which means you can have up to 20 threads executing in parallel.
If you want to read records up to the commit interval and then process one chunk at a time, you should drop the task-executor from the config
I created a batch job using spring batch framework, but if the reader return null, the batch job will create an empty file. Anyway to skip to next step and don't create the file.
<batch:job id="MY_BATCH_JOB">
<batch:step id="step1" next="step2">
<batch:tasklet>
<batch:chunk reader="itemReader" writer="itemWriter"
processor="itemProcessor" commit-interval="2">
</batch:chunk>
<batch:listeners>
<batch:listener ref="readListener" />
</batch:listeners>
</batch:tasklet>
</batch:step>
<batch:step id="step2">
<batch:tasklet>
<batch:chunk reader="itemReader2" writer="itemWriter2"
processor="itemProcessor2" commit-interval="2">
</batch:chunk>
</batch:tasklet>
</batch:step>
</batch:job>
If your ItemWriter is a FlatFileItemWriter, you can set the property shouldDeleteIfEmpty to true so that it doesn't create the file if it is empty.
<bean class="org.springframework.batch.item.file.FlatFileItemWriter">
<property name="shouldDeleteIfEmpty" value="true"></property>
</bean>
If your ItemWriter is a custom item writer, then you can apply your own logic to not create the file.
Either way, it will go to next Step.
In Spring Batch we can create one step which is dependent on other like:
<batch:job id="firstJob">
<batch:step id="firstStep" next="secondStep">
<batch:tasklet ref="firstTasklet"/>
</batch:step>
<batch:step id="secondStep">
<batch:tasklet ref="secondTasklet"/>
</batch:step>
</batch:job>
In my case, we have dependency as shown below, task C (child) needs to be executed only when A (parent) and B (parent) both are completed:
Is there any way in Spring Batch where we can say something like:
<batch:job id="firstJob">
<batch:step id="A,B" next="C">
<batch:tasklet ref="firstTasklet"/>
</batch:step>
...
</batch:job>
What I thought of is using listener on A and B, and keep track of both listeners in database. When both listeners gets executed, task C can be invoked.
Please help.
Note: I am using Spring Batch version: 2.1.9-RELEASE, if above requirement is available on higher releases, I can update version as well.
You can use "next" tag as many times as you want to define a chain so:
<batch:step id="A" next="B">
<batch:tasklet ref="firstTasklet"/>
</batch:step>
<batch:step id="B" next="C">
<batch:tasklet ref="secondTasklet"/>
</batch:step>
<batch:step id="C">
<batch:tasklet ref="thirdTasklet"/>
</batch:step>
The chain is: A -> B -> C
C step will be executed after B
Probably not useful anymore but:
<job id="job1">
<split id="split1" task-executor="taskExecutor" next="stepC">
<flow>
<step id="stepA" parent="stepA" />
</flow>
<flow>
<step id="stepB" parent="stepB"/>
</flow>
</split>
<step id="stepC" parent="stepC"/>
</job>
So C will execute once A and B have executed.
http://docs.spring.io/spring-batch/trunk/reference/html/scalability.html#scalabilityParallelSteps
This is my problem:
I want two jobs configured in the same spring batch. There are 2 totally different tasks(jobs - read-process-write) I want to perform based on the arguement that I pass from the command line.
a) Is it possible to have something like this in the same batch config file?
<batch:job id="job1">
<batch:tasklet>
<batch:chunk reader="reader1" writer="writer1"
processor="processor1" commit-interval="1">
</batch:chunk>
</batch:tasklet>
</batch:job>
<batch:job id="job2">
<batch:tasklet>
<batch:chunk reader="reader2" writer="writer2"
processor="processor2" commit-interval="1">
</batch:chunk>
</batch:tasklet>
</batch:job>
b) If yes, how because when I try that this is what I get:
Exception in thread "main" org.springframework.beans.factory.xml.XmlBeanDefinitionStoreException: Line 48 in XML document from class path resource [hefo-job.xml] is invalid; nested exception is org.xml.sax.SAXParseException: cvc-complex-type.2.4.a: Invalid content was found starting with element 'batch:tasklet'.