Reading file dynamically in spring-batch - spring-batch

Question

I am trying to transfer any files (video,txt etc) between different endpoints (pc, s3, dropbox, google drive) using spring-batch on a network. For that, I am getting json file containing list of files location(url) to be transferred (assume I can access those location).
So, how do I tell the reader to read the input once my controller is hit (in which job is created) and not at the time of starting spring-boot application?
I have tried adding "spring.batch.job.enabled=false" which stops spring-batch to start automatically but my concern is where should I write setting my resource line that will be provided to ItemReader :
FlatFileItemReader<String> reader = new FlatFileItemReader<String>();
reader.setResource(someResource);
Because during setting resources I am getting NullPointerException.

Answer 1

The Running Jobs from within a Web Container explains that with a code example. Here is an except:
#Controller
public class JobLauncherController {
#Autowired
JobLauncher jobLauncher;
#Autowired
Job job;
#RequestMapping("/jobLauncher.html")
public void handle() throws Exception{
jobLauncher.run(job, new JobParameters());
}
}
In your case, you need to extract the file name from the request and pass it as a job parameter, something like:
#RequestMapping("/jobLauncher.html")
public void handle() throws Exception{
URL url = // extract url from request
JobParameters parameters = new JobParametersBuilder()
.addString("url", url)
.toJobParameters();
jobLauncher.run(job, parameters);
}
Then make your reader step-scoped and dynamically extract the file from job parameters:
#StepScope
#Bean
public FlatFileItemReader flatFileItemReader(#Value("#{jobParameters['url']}") URL url) {
return new FlatFileItemReaderBuilder<String>()
.resource(new UrlResource(url))
// set other properties
.build();
}
This is explained in the Late Binding of Job and Step Attributes section.