Spring Batch MultiResourceItemReader configuration issue with Custom ItemReader - spring-batch

Question

My current implementation below is working fine as long as in my publicFigureItemReader() I just set the Resource to the file I am trying to read. However, now that I got to this point I am trying to implement a MultiResourceItemReader which it will pass to the publicFigureItemReader() a bunch of files. As I have about 100 files in a single folder.
I have created the MultiResourceItemReader and now if I would use just a classic ItemReader I should just register this into my step .reader(multiResourceItemReader()) but by doing so is not working because I am using a custom ItemReader so I am not sure how to register my MultiResourceItemReader. Any pointers ?
If I try to register like so, the customItemWriter will stop working, I get a null pointer exception.
#Bean
Step associateXmlFileToDatabaseStep() {
return stepBuilderFactory.get("associateXmlFileToDatabaseStep")
.<Associate, Associate>chunk(1000)
.reader(customItemReader())
.writer(associateItemWriter(super.dataSource,super.namedParameterJdbcTemplate))
.stream(multiResourceItemReader())
//.stream((ItemStream) publicFigureItemReader())
.build();
}
If I try without registering the multiResourceItemReader() the exception is obvious java.lang.IllegalArgumentException: The Resource must not be null.
#Bean
Step associateXmlFileToDatabaseStep() {
return stepBuilderFactory.get("associateXmlFileToDatabaseStep")
.<Associate, Associate>chunk(1000)
.reader(customItemReader())
.writer(associateItemWriter(super.dataSource,super.namedParameterJdbcTemplate))
//.stream(multiResourceItemReader())
.stream((ItemStream) publicFigureItemReader())
.build();
}
Here is a copy paste of my current Job Configuration Class.
#Value("classpath*:/data/mydata*.xml")
private Resource[] inputFiles;
#Bean
public MultiResourceItemReader<PublicFigure> multiResourceItemReader() {
MultiResourceItemReader<PublicFigure> reader = new MultiResourceItemReader<>();
reader.setDelegate((ResourceAwareItemReaderItemStream<? extends PublicFigure>) publicFigureItemReader());
reader.setResources(inputFiles);
return reader;
}
#Bean
ItemReader<PublicFigure> publicFigureItemReader() {
StaxEventItemReader<PublicFigure> xmlFileReader = new StaxEventItemReader<>();
//xmlFileReader.setResource(new FileSystemResource("C:\\Users\\da\\myfile1.xml"));
xmlFileReader.setFragmentRootElementName("PublicFigure");
Jaxb2Marshaller publicFigMarshaller = new Jaxb2Marshaller();
publicFigMarshaller.setClassesToBeBound(Associate.class, PublicFigure.class);
publicFigMarshaller.setUnmarshallerListener(new PublicFigureAssociateListener());
xmlFileReader.setUnmarshaller(publicFigMarshaller);
return xmlFileReader;
}
#Bean
ItemReader<Associate> customItemReader() {
return new AssociateItemReader(publicFigureItemReader());
}
#Bean
ItemWriter<Associate> associateItemWriter(DataSource dataSource, NamedParameterJdbcTemplate jdbcTemplate) {
JdbcBatchItemWriter<Associate> databaseItemWriter = new JdbcBatchItemWriter<>();
databaseItemWriter.setDataSource(dataSource);
databaseItemWriter.setJdbcTemplate(jdbcTemplate);
databaseItemWriter.setSql(QUERY_INSERT);
ItemPreparedStatementSetter<Associate> associateItemPreparedStatementSetter = new AssociatePreparedStatementSetter();
databaseItemWriter.setItemPreparedStatementSetter(associateItemPreparedStatementSetter);
return databaseItemWriter;
}
#Bean
Step associateXmlFileToDatabaseStep() {
return stepBuilderFactory.get("associateXmlFileToDatabaseStep")
.<Associate, Associate>chunk(1000)
.reader(customItemReader())
.writer(associateItemWriter(super.dataSource,super.namedParameterJdbcTemplate))
//.stream(multiResourceItemReader())
.stream((ItemStream) publicFigureItemReader())
.build();
}
The above configuration will throw an exception, even after changing the customItemReader to return the multiREsourceItemReader, see below:
#Bean
ItemReader<Associate> customItemReader() {
//return new AssociateItemReader(publicFigureItemReader());
return new AssociateItemReader(multiResourceItemReader());
}
This is the stacktrace as a result
org.springframework.batch.item.ItemStreamException: Failed to initialize the reader
at org.springframework.batch.item.support.AbstractItemCountingItemStreamItemReader.open(AbstractItemCountingItemStreamItemReader.java:147) ~[spring-batch-infrastructure-3.0.9.RELEASE.jar:3.0.9.RELEASE]
at org.springframework.batch.item.support.CompositeItemStream.open(CompositeItemStream.java:96) ~[spring-batch-infrastructure-3.0.9.RELEASE.jar:3.0.9.RELEASE]
at org.springframework.batch.core.step.tasklet.TaskletStep.open(TaskletStep.java:310) ~[spring-batch-core-3.0.9.RELEASE.jar:3.0.9.RELEASE]
at org.springframework.batch.core.step.AbstractStep.execute(AbstractStep.java:197) ~[spring-batch-core-3.0.9.RELEASE.jar:3.0.9.RELEASE]
at org.springframework.batch.core.job.SimpleStepHandler.handleStep(SimpleStepHandler.java:148) [spring-batch-core-3.0.9.RELEASE.jar:3.0.9.RELEASE]
at org.springframework.batch.core.job.flow.JobFlowExecutor.executeStep(JobFlowExecutor.java:64) [spring-batch-core-3.0.9.RELEASE.jar:3.0.9.RELEASE]
at org.springframework.batch.core.job.flow.support.state.StepState.handle(StepState.java:67) [spring-batch-core-3.0.9.RELEASE.jar:3.0.9.RELEASE]
at org.springframework.batch.core.job.flow.support.SimpleFlow.resume(SimpleFlow.java:169) [spring-batch-core-3.0.9.RELEASE.jar:3.0.9.RELEASE]
at org.springframework.batch.core.job.flow.support.SimpleFlow.start(SimpleFlow.java:144) [spring-batch-core-3.0.9.RELEASE.jar:3.0.9.RELEASE]
at org.springframework.batch.core.job.flow.support.state.SplitState$1.call(SplitState.java:93) [spring-batch-core-3.0.9.RELEASE.jar:3.0.9.RELEASE]
at org.springframework.batch.core.job.flow.support.state.SplitState$1.call(SplitState.java:90) [spring-batch-core-3.0.9.RELEASE.jar:3.0.9.RELEASE]
at java.util.concurrent.FutureTask.run(FutureTask.java:266) [na:1.8.0_144]
at java.lang.Thread.run(Thread.java:748) [na:1.8.0_144]
Caused by: java.lang.IllegalArgumentException: The Resource must not be null.
at org.springframework.util.Assert.notNull(Assert.java:134) ~[spring-core-4.3.17.RELEASE.jar:4.3.17.RELEASE]
at org.springframework.batch.item.xml.StaxEventItemReader.doOpen(StaxEventItemReader.java:189) ~[spring-batch-infrastructure-3.0.9.RELEASE.jar:3.0.9.RELEASE]
at org.springframework.batch.item.support.AbstractItemCountingItemStreamItemReader.open(AbstractItemCountingItemStreamItemReader.java:144) ~[spring-batch-infrastructure-3.0.9.RELEASE.jar:3.0.9.RELEASE]
... 12 common frames omitted
The below stacktrace will come up when I register the multiReader and comment the publicFigureItemReader()), like so:
.stream(multiResourceItemReader())
//.stream((ItemStream) publicFigureItemReader())
-This is the stacktrace:
org.springframework.oxm.UncategorizedMappingException: Unknown JAXB exception; nested exception is com.sun.xml.internal.bind.v2.runtime.IllegalAnnotationsException: 1 counts of IllegalAnnotationExceptions
org.springframework.core.io.Resource is an interface, and JAXB can't handle interfaces.
this problem is related to the following location:
at org.springframework.core.io.Resource
at private org.springframework.core.io.Resource io.spring.batch.configuration.jobs.djsplit.djpersonsentities.PublicFigure.resource
at io.spring.batch.configuration.jobs.djsplit.djpersonsentities.PublicFigure
at io.spring.batch.configuration.jobs.djsplit.djpersonsentities.PublicFigure io.spring.batch.configuration.jobs.djsplit.djpersonsentities.Associate.publicFigure
at io.spring.batch.configuration.jobs.djsplit.djpersonsentities.Associate
at org.springframework.oxm.jaxb.Jaxb2Marshaller.convertJaxbException(Jaxb2Marshaller.java:915) ~[spring-oxm-4.3.17.RELEASE.jar:4.3.17.RELEASE]
at org.springframework.oxm.jaxb.Jaxb2Marshaller.getJaxbContext(Jaxb2Marshaller.java:483) ~[spring-oxm-4.3.17.RELEASE.jar:4.3.17.RELEASE]
at org.springframework.oxm.jaxb.Jaxb2Marshaller.createUnmarshaller(Jaxb2Marshaller.java:858) ~[spring-oxm-4.3.17.RELEASE.jar:4.3.17.RELEASE]
at org.springframework.oxm.jaxb.Jaxb2Marshaller.unmarshal(Jaxb2Marshaller.java:761) ~[spring-oxm-4.3.17.RELEASE.jar:4.3.17.RELEASE]
at org.springframework.oxm.jaxb.Jaxb2Marshaller.unmarshal(Jaxb2Marshaller.java:753) ~[spring-oxm-4.3.17.RELEASE.jar:4.3.17.RELEASE]
at org.springframework.batch.item.xml.StaxEventItemReader.doRead(StaxEventItemReader.java:240) ~[spring-batch-infrastructure-3.0.9.RELEASE.jar:3.0.9.RELEASE]
at org.springframework.batch.item.support.AbstractItemCountingItemStreamItemReader.read(AbstractItemCountingItemStreamItemReader.java:88) ~[spring-batch-infrastructure-3.0.9.RELEASE.jar:3.0.9.RELEASE]
at org.springframework.batch.item.file.MultiResourceItemReader.readFromDelegate(MultiResourceItemReader.java:140) ~[spring-batch-infrastructure-3.0.9.RELEASE.jar:3.0.9.RELEASE]
at org.springframework.batch.item.file.MultiResourceItemReader.readNextItem(MultiResourceItemReader.java:119) ~[spring-batch-infrastructure-3.0.9.RELEASE.jar:3.0.9.RELEASE]
at org.springframework.batch.item.file.MultiResourceItemReader.read(MultiResourceItemReader.java:108) ~[spring-batch-infrastructure-3.0.9.RELEASE.jar:3.0.9.RELEASE]
at io.spring.batch.configuration.jobs.djsplit.djpersonsentities.AssociateItemReader.read(AssociateItemReader.java:23) ~[classes/:na]
at io.spring.batch.configuration.jobs.djsplit.djpersonsentities.AssociateItemReader.read(AssociateItemReader.java:11) ~[classes/:na]
at org.springframework.batch.core.step.item.SimpleChunkProvider.doRead(SimpleChunkProvider.java:91) ~[spring-batch-core-3.0.9.RELEASE.jar:3.0.9.RELEASE]
at org.springframework.batch.core.step.item.SimpleChunkProvider.read(SimpleChunkProvider.java:157) ~[spring-batch-core-3.0.9.RELEASE.jar:3.0.9.RELEASE]
at org.springframework.batch.core.step.item.SimpleChunkProvider$1.doInIteration(SimpleChunkProvider.java:116) ~[spring-batch-core-3.0.9.RELEASE.jar:3.0.9.RELEASE]
at org.springframework.batch.repeat.support.RepeatTemplate.getNextResult(RepeatTemplate.java:374) ~[spring-batch-infrastructure-3.0.9.RELEASE.jar:3.0.9.RELEASE]
at org.springframework.batch.repeat.support.RepeatTemplate.executeInternal(RepeatTemplate.java:215) ~[spring-batch-infrastructure-3.0.9.RELEASE.jar:3.0.9.RELEASE]
at org.springframework.batch.repeat.support.RepeatTemplate.iterate(RepeatTemplate.java:144) ~[spring-batch-infrastructure-3.0.9.RELEASE.jar:3.0.9.RELEASE]
at org.springframework.batch.core.step.item.SimpleChunkProvider.provide(SimpleChunkProvider.java:110) ~[spring-batch-core-3.0.9.RELEASE.jar:3.0.9.RELEASE]
at org.springframework.batch.core.step.item.ChunkOrientedTasklet.execute(ChunkOrientedTasklet.java:69) ~[spring-batch-core-3.0.9.RELEASE.jar:3.0.9.RELEASE]
at org.springframework.batch.core.step.tasklet.TaskletStep$ChunkTransactionCallback.doInTransaction(TaskletStep.java:406) ~[spring-batch-core-3.0.9.RELEASE.jar:3.0.9.RELEASE]
at org.springframework.batch.core.step.tasklet.TaskletStep$ChunkTransactionCallback.doInTransaction(TaskletStep.java:330) ~[spring-batch-core-3.0.9.RELEASE.jar:3.0.9.RELEASE]
at org.springframework.transaction.support.TransactionTemplate.execute(TransactionTemplate.java:133) ~[spring-tx-4.3.17.RELEASE.jar:4.3.17.RELEASE]
at org.springframework.batch.core.step.tasklet.TaskletStep$2.doInChunkContext(TaskletStep.java:272) ~[spring-batch-core-3.0.9.RELEASE.jar:3.0.9.RELEASE]
at org.springframework.batch.core.scope.context.StepContextRepeatCallback.doInIteration(StepContextRepeatCallback.java:81) ~[spring-batch-core-3.0.9.RELEASE.jar:3.0.9.RELEASE]
at org.springframework.batch.repeat.support.RepeatTemplate.getNextResult(RepeatTemplate.java:374) ~[spring-batch-infrastructure-3.0.9.RELEASE.jar:3.0.9.RELEASE]
at org.springframework.batch.repeat.support.RepeatTemplate.executeInternal(RepeatTemplate.java:215) ~[spring-batch-infrastructure-3.0.9.RELEASE.jar:3.0.9.RELEASE]
at org.springframework.batch.repeat.support.RepeatTemplate.iterate(RepeatTemplate.java:144) ~[spring-batch-infrastructure-3.0.9.RELEASE.jar:3.0.9.RELEASE]
at org.springframework.batch.core.step.tasklet.TaskletStep.doExecute(TaskletStep.java:257) ~[spring-batch-core-3.0.9.RELEASE.jar:3.0.9.RELEASE]
at org.springframework.batch.core.step.AbstractStep.execute(AbstractStep.java:200) ~[spring-batch-core-3.0.9.RELEASE.jar:3.0.9.RELEASE]
at org.springframework.batch.core.job.SimpleStepHandler.handleStep(SimpleStepHandler.java:148) [spring-batch-core-3.0.9.RELEASE.jar:3.0.9.RELEASE]
at org.springframework.batch.core.job.flow.JobFlowExecutor.executeStep(JobFlowExecutor.java:64) [spring-batch-core-3.0.9.RELEASE.jar:3.0.9.RELEASE]
at org.springframework.batch.core.job.flow.support.state.StepState.handle(StepState.java:67) [spring-batch-core-3.0.9.RELEASE.jar:3.0.9.RELEASE]
at org.springframework.batch.core.job.flow.support.SimpleFlow.resume(SimpleFlow.java:169) [spring-batch-core-3.0.9.RELEASE.jar:3.0.9.RELEASE]
at org.springframework.batch.core.job.flow.support.SimpleFlow.start(SimpleFlow.java:144) [spring-batch-core-3.0.9.RELEASE.jar:3.0.9.RELEASE]
at org.springframework.batch.core.job.flow.support.state.SplitState$1.call(SplitState.java:93) [spring-batch-core-3.0.9.RELEASE.jar:3.0.9.RELEASE]
at org.springframework.batch.core.job.flow.support.state.SplitState$1.call(SplitState.java:90) [spring-batch-core-3.0.9.RELEASE.jar:3.0.9.RELEASE]
at java.util.concurrent.FutureTask.run(FutureTask.java:266) [na:1.8.0_144]
at java.lang.Thread.run(Thread.java:748) [na:1.8.0_144]
Caused by: com.sun.xml.internal.bind.v2.runtime.IllegalAnnotationsException: 1 counts of IllegalAnnotationExceptions
at com.sun.xml.internal.bind.v2.runtime.IllegalAnnotationsException$Builder.check(IllegalAnnotationsException.java:91) ~[na:1.8.0_144]
at com.sun.xml.internal.bind.v2.runtime.JAXBContextImpl.getTypeInfoSet(JAXBContextImpl.java:445) ~[na:1.8.0_144]
at com.sun.xml.internal.bind.v2.runtime.JAXBContextImpl.<init>(JAXBContextImpl.java:277) ~[na:1.8.0_144]
at com.sun.xml.internal.bind.v2.runtime.JAXBContextImpl.<init>(JAXBContextImpl.java:124) ~[na:1.8.0_144]
at com.sun.xml.internal.bind.v2.runtime.JAXBContextImpl$JAXBContextBuilder.build(JAXBContextImpl.java:1123) ~[na:1.8.0_144]
at com.sun.xml.internal.bind.v2.ContextFactory.createContext(ContextFactory.java:147) ~[na:1.8.0_144]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_144]
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_144]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_144]
at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_144]
at javax.xml.bind.ContextFinder.newInstance(ContextFinder.java:247) ~[na:1.8.0_144]
at javax.xml.bind.ContextFinder.newInstance(ContextFinder.java:234) ~[na:1.8.0_144]
at javax.xml.bind.ContextFinder.find(ContextFinder.java:462) ~[na:1.8.0_144]
at javax.xml.bind.JAXBContext.newInstance(JAXBContext.java:641) ~[na:1.8.0_144]
at javax.xml.bind.JAXBContext.newInstance(JAXBContext.java:584) ~[na:1.8.0_144]
at org.springframework.oxm.jaxb.Jaxb2Marshaller.createJaxbContextFromClasses(Jaxb2Marshaller.java:523) ~[spring-oxm-4.3.17.RELEASE.jar:4.3.17.RELEASE]
at org.springframework.oxm.jaxb.Jaxb2Marshaller.getJaxbContext(Jaxb2Marshaller.java:476) ~[spring-oxm-4.3.17.RELEASE.jar:4.3.17.RELEASE]
... 37 common frames omitted