Spring Batch Job Execution Listener - Access details of sub-jobs defined by JobStep - spring-batch

I have a job(parent job) of jobs(child). In the jobexecution listener of parent job, I want to access all the details(job name, step names, read/write count etc.) of each child jobs. How can I achieve this?
Below is example code. In afterJob method of parentJobExecutionListener, I want to send an email(already have code for it), containing complete details of JOB1 and JOB2. Below code in afterJob method prints details of JOB-STEP1 and JOB-STEP2. How can I access details of child jobs(JOB1 and JOB2) and their steps?
public class ParentJobConfig {
#Bean
public Job job1() {
Job job = jobBuilderFactory.get("JOB1")
.incrementer(new RunIdIncrementer())
.start(step11())
.next(step12())
.build();
return job;
}
#Bean
public Job job2() {
Job job = jobBuilderFactory.get("JOB2")
.incrementer(new RunIdIncrementer())
.start(step21())
.next(step22())
.build();
return job;
}
#Bean
public Step jobStep1() {
Step step = stepBuilderFactory.get("JOB-STEP1")
.job(job1())
.parametersExtractor(new DefaultJobParametersExtractor())
.build();
return step;
}
#Bean
public Step jobStep2() {
Step step = stepBuilderFactory.get("JOB-STEP2")
.job(job2())
.parametersExtractor(new DefaultJobParametersExtractor())
.build();
return step;
}
#Bean
public Job parentJob() {
Job job = jobBuilderFactory.get("PARENT-JOB")
.incrementer(new RunIdIncrementer())
.start(jobStep1())
.next(jobStep2())
.listener(parentJobExecutionListener())
.build();
return job;
}
#Bean
public JobExecutionListener parentJobExecutionListener() {
return new ParentJobExecutionListener ();
}
}
public class ParentJobExecutionListener extends JobExecutionListenerSupport {
#Override
public void afterJob(JobExecution jobExecution) {
List<StepExecution> stepExecutions = (List<StepExecution>) jobExecution.getStepExecutions();
Collection<StepExecution> stepExecutions = jobExecution.getStepExecutions();
for (StepExecution stepExecution : stepExecutions) {
System.out.println("Step Name: " + stepExecution.getStepName());
System.out.println("Read Count: " + stepExecution.getReadCount());
System.out.println("Skip Count: " + stepExecution.getSkipCount());
System.out.println("Write Count: " + stepExecution.getWriteCount());
}
}
}

I don't see an obvious way to get access to the child job executions created here: https://github.com/spring-projects/spring-batch/blob/d8fc58338d3b059b67b5f777adc132d2564d7402/spring-batch-core/src/main/java/org/springframework/batch/core/step/job/JobStep.java#L119 (Only the status is reflected on the step execution).
What you can do is use a job explorer in your listener and query the repository for child job executions.

Related

How can I enable or disable a Step dynamically reading the enableStatus from the Database?

#Bean
public Step step1(){
return stepBuilderFactory.get("checkInfo").<A, B>chunk(10)
.reader(reader())
.processor(processor())
.writer(writer())
.build();
}
I have created this step called "checkInfo", and i have other steps with other name. In my database I have a table "STEPS" with the name of the step and if it is enabled or disabled.
So i have to chain only the steps enabled to my JOB.
#PostConstruct
public void getActiveSteps(){
stepsList = stepManagementRepository.findAllByActive(true);
for (StepManagement s : stepsList){
System.out.println(s.getDescription());
}
I get all of the active ones in this function. The problem is, how can I get the step i want by the name saved on my DB? (So then i can use the .next() in the job, only if the step is active)
#Bean
public Job runJob(){
SimpleJobBuilder jobBuilder = jobBuilderFactory.get("mainCalculationJob")
.incrementer(new RunIdIncrementer())
.start(step1());
return jobBuilder.build();
}
I solved getting bean by name:
#Bean(name = "checkInfo")
public Step step1(){
return stepBuilderFactory.get("checkInfo").<A, B>chunk(10)
.reader(reader())
.processor(processor())
.writer(writer())
.build();
}
#Bean
public Job runJob(){
SimpleJobBuilder jobBuilder = jobBuilderFactory.get("mainCalculationJob")
.incrementer(new RunIdIncrementer())
.start((Step) context.getBean("checkInfo"));
return jobBuilder.build();
}

Spring Batch - abstract Step definition in Java configuration?

My Spring Batch Job configuration has 5 steps, all of which are identical except for the reader. Is there a way I can abstract out all of the other parts of the step into a "parent" step, so that I don't need to repeat everything? I know this can be done in XML, but I can't figure out the java equivalent.
Here's one of the steps:
public Step quarterlyStep(FileIngestErrorListener listener, ItemReader<DistributionItem> quarterlyReader) {
return stepBuilderFactory.get("quarterlyStep")
.<DistributionItem,DistributionItem>chunk(10)
.reader(quarterlyReader) // The only thing that changes among 5 different steps
.listener(listener.asReadListener())
.processor(processor())
.listener(listener.asProcessListener())
.writer(writer())
.listener(listener.asWriteListener())
.faultTolerant()
.skip(ValidationException.class)
.skip(ExcelFileParseException.class)
.build();
}
Here's the definition of one of the readers:
#Bean
#JobScope
public PoiItemReader<DistributionItem> yearEndReader(#Value("#{jobExecutionContext['filename']}") String filename) {
PoiItemReader<PortfolioFundsDistributionItem> reader = new PoiItemReader<>();
reader.setLinesToSkip(1);
reader.setRowMapper(yearEndRowMapper());
reader.setResource(new FileSystemResource(filename));
return reader;
}
You can do something like:
private StepBuilderFactory stepBuilderFactory;
private SimpleStepBuilder<Integer, Integer> createBaseStep(String stepName) {
return stepBuilderFactory.get(stepName)
.<Integer, Integer>chunk(5)
.processor(itemProcessor())
.writer(itemWriter());
}
#Bean
public Step step1(ItemReader<Integer> itemReader) {
return createBaseStep("step1")
.reader(itemReader)
.build();
}
#Bean
public Step step2(ItemReader<Integer> itemReader) {
return createBaseStep("step2")
.reader(itemReader)
.build();
}

How to quit a tasklet if error in Spring Batch?

I would like to quit a tasklet cleanly if I have an error on it and put and stop the batch without having to resort to a System.exit(1).
Here is my code:
/**
* execution de la tasklet
*
*/
#Override
public RepeatStatus execute(StepContribution arg0, ChunkContext arg1) throws IOException {
if (suiviFluxDao.getNbFileDateTrt(FilenameUtils.getName(resource), Utils.getDateFromStringFormatUS(dateTraitement)) > 0) {
LOGGER.info(PropertiesUtils.getLibelleExcep(Constantes.ERREUR_NB_FILE_SELECT,
new String[]{ConstantesNomsSql.TABLE_STCO_STAU_SUIVI_FLUX, FilenameUtils.getName(resource), dateTraitement, Constantes.NAME_TRT}));
System.exit(1);
} else {
SuiviFluxBO suiviFluxBO = new SuiviFluxBO();
suiviFluxBO.setDateSysteme(Utils.getDateTodayFormatUS());
suiviFluxBO.setDateTrt(Utils.getDateFromStringFormatUS(dateTraitement));
suiviFluxBO.setLibelleTrt("Batch_Java");
suiviFluxBO.setNomficTrt(FilenameUtils.getName(resource));
suiviFluxBO.setNbrrecTrt(Utils.countNbFile(resource));
suiviFluxBO.setNomtabTrt(ConstantesNomsSql.TABLE_STCO_STAU_FIC_ADH);
suiviFluxBO.setNbrlignesTrt(0);
suiviFluxDao.insertSuiviBO(suiviFluxBO);
}
// fin de l'execution
return RepeatStatus.FINISHED;
}
The tasklet implements StepExecutionListener but how to indicate in the IF that contains the error to modify the execution status so that it is in FAILED?
Thank you for your leads.
Based on above requirement , we can build a flow using spring batch FlowBuilder object .
1 . Build a Tasklet which performs required validations and sets ExitStatus based on validation result.
#Component
public class TestTasklet implements StepExecutionListener, Tasklet {
// Any additional properties if required can be added here .
#Override
public void beforeStep(StepExecution stepExecution) {
// Any logic added here will execute before executing step
}
#Override
public ExitStatus afterStep(StepExecution stepExecution) {
// Any logic added here will execute after executing step
return null;
}
#Override
public RepeatStatus execute(StepContribution contribution, ChunkContext chunkContext) throws IOException {
if (suiviFluxDao.getNbFileDateTrt(FilenameUtils.getName(resource),
Utils.getDateFromStringFormatUS(dateTraitement)) > 0) {
LOGGER.info(PropertiesUtils.getLibelleExcep(Constantes.ERREUR_NB_FILE_SELECT,
new String[]{ConstantesNomsSql.TABLE_STCO_STAU_SUIVI_FLUX, FilenameUtils.getName(resource),
dateTraitement, Constantes.NAME_TRT}));
contribution.setExitStatus(ExitStatus.FAILED);
} else {
// any logic goes here .
contribution.setExitStatus(ExitStatus.COMPLETED);
}
return RepeatStatus.FINISHED;
}
}
2 . Below code snippet allows to configure job using flow builder :
#Configuration
public class JobConfigurations {
private StepBuilderFactory stepBuilderFactory;
private JobBuilderFactory jobBuilderFactory;
#Autowired
public JobConfigurations(StepBuilderFactory stepBuilderFactory,
JobBuilderFactory jobBuilderFactory) {
this.stepBuilderFactory = stepBuilderFactory;
this.jobBuilderFactory = jobBuilderFactory;
}
#Bean
public Job job(TestTasklet testTasklet) {
Step validationStep = stepBuilderFactory.get("validationTasklet")
.tasklet(testTasklet).build();
//create another step where you want to perform business logic
//for sake of brevity let us assume it to be businessValidationStep
//Step businessValidationStep = stepBuilderFactory.get("businessvalidationstep")
// .chunk().reader().processor().writer();
return jobBuilderFactory.get("JOB_NAME").incrementer(new RunIdIncrementer())
.start(validationStep)// start your job with validation step
.on(ExitStatus.FAILED.getExitCode()).end()// this will terminate your job cleanly
.from(validationStep)
.on(ExitStatus.COMPLETED.getExitCode())//.to("businessValidationStep")
.to(validationStep).build().build();
}
}
#lasnico37 Hope above code will solve the problem statement .

My spring batch step keeps looping after reaching build

This section reads in the file from our server, processes it, writes it out and archives it.
#Bean
public Step step1() {
log.info("Made if to step1");
System.out.println("Made it to Step 1");
return this.stepBuilderFactory.get("step1")
.<PaymentTransaction, PaymentTransaction>chunk(10)
.reader(paymentTransactionItemReader())
.writer(paymentTransactionItemWriter())
.build();
}
#Bean
public JobExecutionDecider decider() {
System.out.println("Made it to the decider");
return (jobExecution, stepExecution) -> new FlowExecutionStatus("Success"); }
#Bean
public FlowJobBuilder job() {
return jobBuilderFactory.get("BenefitIssuance")
.start(step1())
.next(decider())
.on("Success")
.end()
.build();
}
However when it reaches the build() step at the end, it loops back to the reader
As mentioned in the comments, I don't see why the job() method returns a FlowJobBuilder and not a Job. The following job definition does not loop back on the same step:
#Bean
public Job job() {
return jobs.get("job")
.start(step1())
.next(decider())
.on("Success")
.end()
.build()
.build();
}

Spring Batch restart from last exception

I would like to restart a batch after it was terminated
I stop a batch when a particular exception throws :
public class IntegrationItemProcessorExceptionHandler implements ExceptionHandler {
private static final Logger LOG = LoggerFactory.getLogger(IntegrationItemProcessorExceptionHandler.class);
#Override
public void handleException(RepeatContext context, Throwable throwable) throws Throwable {
LOG.error("handleException", throwable);
if (throwable instanceof CustomResponseException) {
context.setTerminateOnly();
}
}
}
The input is a json I read with a FlatFileItemReader
#Bean
public ItemReader<UserDto> reader() {
FlatFileItemReader<UserDto> reader = new FlatFileItemReader<MerchantDTO>();
reader.setResource(new ClassPathResource("user.json"));
reader.setRecordSeparatorPolicy(new CustomJsonRecordSeparatorPolicy());
reader.setLineMapper(new CustomLineMapper());
return reader;
}
If CustomResponseException is throws by the ItemProcessor, I stop the batch.
After, I would like to restart the batch, but at the same line I stoped it.
What I need to to to have this behavor ??
And my job config :
#Bean
public Job importUserJob(JobBuilderFactory jobs, Step s1) {
return jobs.get("integrationJob")
.incrementer(new RunIdIncrementer())
.flow(s1)
.end()
.build();
}
#Bean
public Step step1(StepBuilderFactory stepBuilderFactory) {
return stepBuilderFactory.get("step1")
.<UserDTO, User>chunk(10)
.reader(reader())
.processor(processor())
.writer(writer())
.listener(new IntegrationItemProcessorListener())
.exceptionHandler(new IntegrationItemProcessorExceptionHandler())
.build();
}