Resources must not be null in MultipleResourceItemReader - spring-batch

Question

I am developing a spring batch job to download the files from S3Bucket first and place it on my local (using Tasklet) and then read the files from my local using MultiResourceItemReader and populating it into work table.
I am calling Tasklet first and then reading the files in the next step. So, that we have the input files available.
But, when I am trying to run the process, I guess because of bean configuration dependency, it's throwing below error : The Resources must not be null.
I am not sure how to handle it. Once the tasklet run is completed, there would be files available but not before that.
Error:
**Caused by: java.lang.IllegalArgumentException: The resources must not be null**
at org.springframework.util.Assert.notNull(Assert.java:201) ~[spring-core-5.3.3.jar:5.3.3]
at org.springframework.batch.item.file.MultiResourceItemReader.setResources(MultiResourceItemReader.java:246) ~[spring-batch-infrastructure-4.3.1.jar:4.3.1]
at com.cspprovemerald.SpringBatchApplication.ItemReader.FileItemReader.providerMultiResourceItemReader(FileItemReader.java:38) ~[classes/:na]
at com.cspprovemerald.SpringBatchApplication.Config.JobStepBuilderConfig.step2(JobStepBuilderConfig.java:64) ~[classes/:na]
at com.cspprovemerald.SpringBatchApplication.Config.JobStepBuilderConfig.job(JobStepBuilderConfig.java:110) ~[classes/:na]
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:na]
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:na]
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:na]
at java.base/java.lang.reflect.Method.invoke(Method.java:566) ~[na:na]
at org.springframework.beans.factory.support.SimpleInstantiationStrategy.instantiate(SimpleInstantiationStrategy.java:154) ~[spring-beans-5.3.3.jar:5.3.3]
... 38 common frames omitted
MultiResouceItemReader :
#Component
public class FileItemReader {
#Value("${local.file.download.path}")
private String localFileDownloadPath;
private static final Logger LOGGER = LoggerFactory.getLogger(FileItemReader.class);
// MultiResourceItemReader to read multiple files sequentially
public MultiResourceItemReader<Provider> providerMultiResourceItemReader() {
String locationPattern = "C:/Users/Desktop/data/in/*.csv";
Resource[] resources = null;
ResourcePatternResolver patternResolver = new PathMatchingResourcePatternResolver();
try {
resources = patternResolver.getResources(locationPattern);
} catch (IOException e) {
e.printStackTrace();
}
MultiResourceItemReader<Provider> multiResourceItemReader = new MultiResourceItemReader<>();
multiResourceItemReader.setResources(resources);
multiResourceItemReader.setDelegate(providerItemReader());
return multiResourceItemReader;
}
}
JobBuilderConfig.java
#Component
#EnableBatchProcessing
public class JobStepBuilderConfig {
#Autowired
JobBuilderFactory jobBuilderFactory;
#Autowired
StepBuilderFactory stepBuilderFactory;
#Autowired
DataSource datasource;
#Autowired
JdbcItemWriter jdbcItemWriter;
#Autowired
JdbcItemReader jdbcItemReader;
#Autowired
FileItemReader fileItemReader;
#Autowired
FileItemWriter fileItemWriter;
#Autowired
TaskletSPExecutor taskletSPExecutor;
#Autowired
TaskletS3DownloadFiles taskletS3DownloadFiles;
public Step step1(){
// step 1 : Read records from custom table and call stored procedure to update facets table
return stepBuilderFactory.get("step1S3ListCopyFiles")
.tasklet(taskletS3DownloadFiles)
.build();
}
public Step step2(){
// step 2 : Read csv files and dump it into a custom table
return stepBuilderFactory.get("step2ReadLoadCSV")
.<Provider, Provider>chunk(1000)
.reader(fileItemReader.providerMultiResourceItemReader())
.writer(jdbcItemWriter.providerJdbcBatchItemWriter())
.build();
}
#Bean
public Job job(){
return jobBuilderFactory.get("jobCSProvMI4275")
.start(step1())
.next(step2())
.incrementer(new RunIdIncrementer())
.build();
}
}

Answer 1

This is because the item reader is created eagerly when the Spring application context is created. At this time, the file is not downloaded yet, hence the error. Spring Batch provides a custom bean scope called the Step scope. This scope allows you to define beans that should be created at runtime only when required.
In your case, you need to make your item reader step-scoped. This means the item reader bean will be created only when the chunk-oriented step requires it (ie after the tasklet has downloaded the file). Here is an example:
#Bean
#StepScope
public MultiResourceItemReader<Provider> providerMultiResourceItemReader() {
// configure your reader here
}
You can find more details about the Step scope in the documentation here.