Async Spring Batch job fails to process file - spring-batch

Question

I'm trying to process a file, and upload it into a database, using spring batch, right after uploading it. However the job completes right after it's started, and I'm not too sure of the exact reason. I think it's not doing what it should, in the tasklet.execute. Below is the DEBUG output:
22:25:09.823 [http-nio-127.0.0.1-8080-exec-2] DEBUG o.s.b.c.c.a.SimpleBatchConfiguration$ReferenceTargetSource - Initializing lazy target object
22:25:09.912 [SimpleAsyncTaskExecutor-1] INFO o.s.b.c.l.support.SimpleJobLauncher - Job: [FlowJob: [name=moneyTransactionImport]] launched with the following parameters: [{targetFile=C:\Users\test\AppData\Local\Temp\tomcat.1435325122308787143.8080\uploads\test.csv}]
22:25:09.912 [SimpleAsyncTaskExecutor-1] DEBUG o.s.batch.core.job.AbstractJob - Job execution starting: JobExecution: id=95, version=0, startTime=null, endTime=null, lastUpdated=Tue Sep 16 22:25:09 BST 2014, status=STARTING, exitStatus=exitCode=UNKNOWN;exitDescription=, job=[JobInstance: id=52, version=0, Job=[transactionImport]], jobParameters=[{targetFile=C:\Users\test\AppData\Local\Temp\tomcat.1435325122308787143.8080\uploads\test.csv}]
22:25:09.971 [SimpleAsyncTaskExecutor-1] DEBUG o.s.b.c.job.flow.support.SimpleFlow - Resuming state=transactionImport.step with status=UNKNOWN
22:25:09.972 [SimpleAsyncTaskExecutor-1] DEBUG o.s.b.c.job.flow.support.SimpleFlow - Handling state=transactionImport.step
22:25:10.018 [SimpleAsyncTaskExecutor-1] INFO o.s.batch.core.job.SimpleStepHandler - Executing step: [step]
22:25:10.019 [SimpleAsyncTaskExecutor-1] DEBUG o.s.batch.core.step.AbstractStep - Executing: id=93
22:25:10.072 [SimpleAsyncTaskExecutor-1] DEBUG o.s.batch.core.scope.StepScope - Creating object in scope=step, name=scopedTarget.reader
22:25:10.117 [SimpleAsyncTaskExecutor-1] DEBUG o.s.batch.core.scope.StepScope - Registered destruction callback in scope=step, name=scopedTarget.reader
22:25:10.136 [SimpleAsyncTaskExecutor-1] WARN o.s.b.item.file.FlatFileItemReader - Input resource does not exist class path resource [C:/Users/test/AppData/Local/Temp/tomcat.1435325122308787143.8080/uploads/test.csv]
22:25:10.180 [SimpleAsyncTaskExecutor-1] DEBUG o.s.b.repeat.support.RepeatTemplate - Starting repeat context.
22:25:10.181 [SimpleAsyncTaskExecutor-1] DEBUG o.s.b.repeat.support.RepeatTemplate - Repeat operation about to start at count=1
22:25:10.181 [SimpleAsyncTaskExecutor-1] DEBUG o.s.b.c.s.c.StepContextRepeatCallback - Preparing chunk execution for StepContext: org.springframework.batch.core.scope.context.StepContext#5d85b879
22:25:10.181 [SimpleAsyncTaskExecutor-1] DEBUG o.s.b.c.s.c.StepContextRepeatCallback - Chunk execution starting: queue size=0
22:25:12.333 [SimpleAsyncTaskExecutor-1] DEBUG o.s.b.repeat.support.RepeatTemplate - Starting repeat context.
22:25:12.333 [SimpleAsyncTaskExecutor-1] DEBUG o.s.b.repeat.support.RepeatTemplate - Repeat operation about to start at count=1
22:25:12.334 [SimpleAsyncTaskExecutor-1] DEBUG o.s.b.repeat.support.RepeatTemplate - Repeat is complete according to policy and result value.
22:25:12.334 [SimpleAsyncTaskExecutor-1] DEBUG o.s.b.c.s.item.ChunkOrientedTasklet - Inputs not busy, ended: true
22:25:12.334 [SimpleAsyncTaskExecutor-1] DEBUG o.s.b.core.step.tasklet.TaskletStep - Applying contribution: [StepContribution: read=0, written=0, filtered=0, readSkips=0, writeSkips=0, processSkips=0, exitStatus=EXECUTING]
22:25:12.337 [SimpleAsyncTaskExecutor-1] DEBUG o.s.b.core.step.tasklet.TaskletStep - Saving step execution before commit: StepExecution: id=93, version=1, name=step, status=STARTED, exitStatus=EXECUTING, readCount=0, filterCount=0, writeCount=0 readSkipCount=0, writeSkipCount=0, processSkipCount=0, commitCount=1, rollbackCount=0, exitDescription=
22:25:12.358 [SimpleAsyncTaskExecutor-1] DEBUG o.s.b.repeat.support.RepeatTemplate - Repeat is complete according to policy and result value.
22:25:12.358 [SimpleAsyncTaskExecutor-1] DEBUG o.s.batch.core.step.AbstractStep - Step execution success: id=93
22:25:12.419 [SimpleAsyncTaskExecutor-1] DEBUG o.s.batch.core.step.AbstractStep - Step execution complete: StepExecution: id=93, version=3, name=step, status=COMPLETED, exitStatus=COMPLETED, readCount=0, filterCount=0, writeCount=0 readSkipCount=0, writeSkipCount=0, processSkipCount=0, commitCount=1, rollbackCount=0
22:25:12.442 [SimpleAsyncTaskExecutor-1] DEBUG o.s.b.c.job.flow.support.SimpleFlow - Completed state=transactionImport.step with status=COMPLETED
22:25:12.443 [SimpleAsyncTaskExecutor-1] DEBUG o.s.b.c.job.flow.support.SimpleFlow - Handling state=transactionImport.COMPLETED
22:25:12.443 [SimpleAsyncTaskExecutor-1] DEBUG o.s.b.c.job.flow.support.SimpleFlow - Completed state=transactionImport.COMPLETED with status=COMPLETED
22:25:12.445 [SimpleAsyncTaskExecutor-1] DEBUG o.s.batch.core.job.AbstractJob - Job execution complete: JobExecution: id=95, version=1, startTime=Tue Sep 16 22:25:09 BST 2014, endTime=null, lastUpdated=Tue Sep 16 22:25:09 BST 2014, status=COMPLETED, exitStatus=exitCode=COMPLETED;exitDescription=, job=[JobInstance: id=52, version=0, Job=[transactionImport]], jobParameters=[{targetFile=C:\Users\test\AppData\Local\Temp\tomcat.1435325122308787143.8080\uploads\test.csv}]
22:25:12.466 [SimpleAsyncTaskExecutor-1] INFO o.s.b.c.l.support.SimpleJobLauncher - Job: [FlowJob: [name=transactionImport]] completed with the following parameters: [{targetFile=C:\Users\test\AppData\Local\Temp\tomcat.1435325122308787143.8080\uploads\test.csv}] and the following status: [COMPLETED]
My config is as follows:
#Configuration
#EnableBatchProcessing
public class BatchConfiguration {
#Inject
private TransactionRepository transactionRepository;
#Inject
private JobRepository jobRepository;
#Bean
#StepScope
public FlatFileItemReader<MoneyTransaction> reader(#Value("#{jobParameters[targetFile]}") String file) {
FlatFileItemReader<MoneyTransaction> reader = new FlatFileItemReader<>();
reader.setResource(new ClassPathResource(file));
reader.setLineMapper(new DefaultLineMapper<MoneyTransaction>() {
{
setLineTokenizer(new DelimitedLineTokenizer() {
{
setNames(new String[]{"Number", "Date", "Account", "Payee", "Cleared", "Amount", "Category", "Subcategory", "Memo"});
}
}
);
setFieldSetMapper(new BeanWrapperFieldSetMapper<MoneyTransaction>() {
{
setTargetType(MoneyTransaction.class);
}
});
}
}
);
reader.setStrict(false);
reader.setLinesToSkip(1);
return reader;
}
#Bean
public ItemProcessor<MoneyTransaction, Transaction> processor() {
return new TransactionProcessor();
}
#Bean
public RepositoryItemWriter writer() {
RepositoryItemWriter writer = new RepositoryItemWriter();
writer.setRepository(transactionRepository);
writer.setMethodName("save");
return writer;
}
#Bean
public Step step(StepBuilderFactory stepBuilderFactory, ItemReader<MoneyTransaction> reader,
ItemWriter<Transaction> writer, ItemProcessor<MoneyTransaction, Transaction> processor) {
return stepBuilderFactory.get("step")
.<MoneyTransaction, Transaction>chunk(100)
.reader(reader)
.processor(processor)
.writer(writer)
.build();
}
#Bean
public SimpleAsyncTaskExecutor taskExecutor() {
SimpleAsyncTaskExecutor executor = new SimpleAsyncTaskExecutor();
executor.setConcurrencyLimit(1);
return executor;
}
#Bean
public SimpleJobLauncher jobLauncher() {
SimpleJobLauncher jobLauncher = new SimpleJobLauncher();
jobLauncher.setJobRepository(jobRepository);
jobLauncher.setTaskExecutor(taskExecutor());
return jobLauncher;
}
}
And I save the file, and start processing in the following way:
public JobExecution processFile(String name, MultipartFile file) {
if (!file.isEmpty()) {
try {
byte[] bytes = file.getBytes();
String rootPath = System.getProperty("catalina.home");
File uploadDirectory = new File(rootPath.concat(File.separator).concat("uploads"));
if (!uploadDirectory.exists()) {
uploadDirectory.mkdirs();
}
File uploadFile = new File(uploadDirectory.getAbsolutePath() + File.separator + file.getOriginalFilename());
BufferedOutputStream stream =
new BufferedOutputStream(new FileOutputStream(uploadFile));
stream.write(bytes);
stream.close();
return startImportJob(uploadFile, "transactionImport");
} catch (Exception e) {
logger.error(String.format("Error processing file '%s'.", name), e);
throw new MoneyException(e);
}
} else {
throw new MoneyException("There was no file to process.");
}
}
/**
* #param file
*/
private JobExecution startImportJob(File file, String jobName) {
logger.debug(String.format("Starting job to import file '%s'.", file));
try {
Job job = jobs.get(jobName).incrementer(new MoneyRunIdIncrementer()).flow(step).end().build();
return jobLauncher.run(job, new JobParametersBuilder().addString("targetFile", file.getAbsolutePath()).toJobParameters());
} catch (JobExecutionAlreadyRunningException e) {
logger.error(String.format("Job for processing file '%s' is already running.", file), e);
throw new MoneyException(e);
} catch (JobParametersInvalidException e) {
logger.error(String.format("Invalid parameters for processing of file '%s'.", file), e);
throw new MoneyException(e);
} catch (JobRestartException e) {
logger.error(String.format("Error restarting job, for processing file '%s'.", file), e);
throw new MoneyException(e);
} catch (JobInstanceAlreadyCompleteException e) {
logger.error(String.format("Job to process file '%s' has already completed.", file), e);
throw new MoneyException(e);
}
}
I'm kind of stumped at the minute, and any help would be greatly received.
Thanks.

Answer 1

Found the issue. The problem was with the type of resource ClassPathResource(file), combined with the fact that I was setting the strict property to false.
reader.setResource(new ClassPathResource(file));
I should have used
reader.setResource(new FileSystemResource(file));
Which makes complete sense, as I wasn't uploading the file as a class path resource.