I need to insert a record into a table and then have the batch terminate with an exception without causing the insert to be rolled back.
I have used but it does not terminate the batch
You can use a fault tolerant step and set the exceptions that should not cause a rollback using FaultTolerantStepBuilder#noRollback.
Then you can use a listener (ItemProcessListener or ItemWriteListener depending on where the exception is thrown) to intercept the exception and terminate the step (and its surrounding job) with StepExecution#setTerminateOnly.
I have used this:
<batch:step id="id">
<batch:tasklet>
<batch:chunk reader="reader" processor="processor" writer="writer" commit-interval="1">
</batch:chunk>
<batch:no-rollback-exception-classes>
<batch:include class="com.exception.myException"></batch:include>
</batch:no-rollback-exception-classes>
</batch:tasklet>
</batch:step>
it allows only to avoid rollback but not terminate batch. I want terminate batch after the execution of myException.
The myException is launched in processor
Related
We have a Spring batch project which is XML based
We need to create a new job and we need to add the job as a nested job to previous XML based job
Is it possible to create the new Job annotation based and add a step to existing XML based job?
I have created a Tasklet Step and tried adding to XML based Job as a Step and am getting.
Cannot convert value of type 'org.springframework.batch.core.step.tasklet.TaskletStep' to required type 'org.springframework.batch.core.step.tasklet.Tasklet' for property 'tasklet': no matching editors or conversion strategy found
A tasklet is not the appropriate type to delegate a step processing to a job, you should use a JobStep instead.
The main job can be defined in XML and refer to the "delegate" job (which could be a bean defined in XML or Java config). Here is an example:
<batch:job id="mainJob">
<batch:step id="step">
<batch:job ref="subjob">
</batch:job>
</batch:step>
</batch:job>
In this example, subjob could be a Spring Batch job defined in XML or Java config.
I am running jobs in parallel. My job execution is always null when I use JobRepositoryFactoryBean. I need to use to use this. If I don't use this, then I will not be able to use metadata tables. Because I want to restart my job when it is not completed because of some failure reason. So, I want previous record which I will be fetching from metadata tables. And if I use MapJobRepositoryFactoryBean, the job execution is not null. But then there will not be insertion in metadata tables.
I referred this link:-
My job is always null. Can't inject a batch job with Spring Batch. Why?
But the link is not working for me.
My congifuration is
<bean id="batchScheduler" class="com.abc.BatchScheduler">
<property name="jobLauncher" ref="jobLauncher" />
<property name="jobtwo" ref="JobTwo" />
</bean>
I searched a lot. Please help me out. I am not able to proceed.
I'm using Asynch jobLaucher for starting spring batch jobs. I tried to execute the same job parallelly but it seems the 2nd started job caused side effect to the 1st job. (It seems that the value of step_output of the 1st job overwritten by the 2nd job.
Everything works well if I use Synch JobLauncher.
My question is. Can I use jobContext in the following way?
MyTaskLetStep (implements StepExecutionListener):
public void beforeStep(StepExecution stepExecution) {
JobExecution jobExecution = stepExecution.getJobExecution();
ExecutionContext jobContext = jobExecution.getExecutionContext();
...
jobContext.put("step_output" ,outList);
...
}
Started Jobs looks like this one:
<batch:job id="TACKJob" restartable="true" incrementer="runIdIncrementer" parent="joblistenerjob">
<batch:step id="F1" next = "F2">
<batch:tasklet ref="N_COMMAND"/>
</batch:step>
<batch:step id="F2">
<batch:tasklet ref="Z_COMMAND"/>
</batch:step>
</batch:job>
The referenced tasklet beans are prototype scoped beans. They implement the referred StepExecutionListener.
Do you have any advices for this problem?
Thanks
Our Spring Batch application is, upon restart of a failed job, processing the same records again, resulting in duplicate rows, and we want to understand how to avoid this.
The Spring Integration poller which starts the batch job is configured to run every couple of hours. When it runs a second time, the job parameters will be the same, but if the previous run failed (for example, because of a DataTruncation exception), Spring Batch will not complain that the job has already completed.
At the point of failure, several hundred thousand records will already have been processed and copied fromn the source table to the destination table. When the job is run a subsequent time, the same rows will be copied to the destination table, resulting in duplicates. Therefore, it appears that the job is not being resumed, but restarted from the beginning.
The Spring Batch database is Derby (file based), this is setup when the application starts, and it appears state is not maintained between restarts of the actual application (because a job can be run again with the same parameters). However, within one application run, state is maintained. For instance, if the job completes succesfully, the next time the poller runs an exception will be thrown because a job (with those parameters) has already completed.
Our job is definition is as follows:
<batch:job id="publisherJob" >
<batch:step id="step1">
<batch:tasklet >
<batch:chunk reader="itemReader" processor="itemProcessor"
writer="itemWriter" commit-interval="${...}" />
</batch:tasklet>
<batch:listeners>
...
</batch:listeners>
</batch:job>
<bean id="itemReader" class="org.springframework.batch.item.database.JdbcCursorItemReader">
<property name="dataSource" ref="dataSource" />
<property name="sql" value="select ${...} from ${...} where ${...}" />
<property name="rowMapper" ref="rowMapper" />
</bean>
The WHERE clause includes ORDER BY.
Our understanding was that Spring Batch would retain the state at which processing failed and proceed from that point (if the error in the source table has been fixed), therefore preventing duplicate rows. What has to be configured for this to happen?
Thanks
Spring Batch maintains state in that it remembers how many records were processed, not specifically which ones. Because of that, it's up to you to guarantee the order of the items is reproducible from run to run so that if we process 100 records in run 1 and fail, when we skip the first 100 records in run 2, those are the right 100 records to skip. You didn't provide the configuration for your JdbcCursorItemReader but my assumption is that you are not using an order by in your SQL. If you want restartability, you need some way to guarantee the order of the items. Using an order by in your SQL is the easiest way to accomplish this (there are others like using the process indicator pattern if that's needed).
I have a very basic question regarding skip. i am using the spring-batch-simple-cli project provided by spring samples, and trying to understand the skip behaviour. this has a very basic example reader that reads from an array of strings ( i have modifed it to read from a list of 10 strings starting from Hellowworld 1 to Hellowworld 10) and a basic writer that logs to console. writer throws java.lang.Exception on each write.
i have added a skip limit of 4 to job configuration. once it reached Hellowworld 5, the job stops as expected.
But whenever writer throws exception, the writer is called back immediately with the same item. My question why is writer called twice? i am expecting this item to be just skipped? is there something i am missing.
<job id="job1" xmlns="http://www.springframework.org/schema/batch" incrementer="jobParametersIncrementer">
<step id="step1" parent="simpleStep">
<tasklet>
<chunk reader="reader" writer="writer" skip-limit="4" >
<skippable-exception-classes>
<include class="java.lang.Exception" />
</skippable-exception-classes>
</chunk>
</tasklet>
</step>
</job>
This is most likely caused by the default functionality, where spring batch rollbacks the chunk and retries each chunk item (in this case there is only one item) again.
https://stackoverflow.com/a/6730807/1627688