I have a project in spring batch, in which I read from a txt file (input data) and according to a validation of the item I read, it should be written in a txt file (output 1) or in another txt file (output 2) I think for this i should use a ClassifierCompositeItemwriter, how i can do to additionally write all items that I read in a database (output 3)?
I must keep in mind that the three outputs have different formats
Thanks!
You can use CompositeItemWriter which will delegate writing to a list of ItemWriter . The order of the ItemWriter list is important as it will call the ItemWriter in order. So make sure ClassifierCompositeItemwriter is before the ItemWriter for writing to database. :
#Bean
public ItemWriter itemWriter(ClassifierCompositeItemwriter classiferWriter ,JdbcBatchItemWriter jdbcWriter){
CompositeItemWriter writer = new CompositeItemWriter();
writer.setDelegates(List.of(classiferWriter,jdbcWriter));
return writer;
}
Related
I need to write a Step with :
reader JpaPagingItemReader
processor that return a List
writer that writer foreach student that returned from processor a line in csv file.
How it is possible? how write a costum "FlatFileItemWriter" that consume a list and for each object write a line into FileResource?
Thanks
How to call a rest service in listitemreader then write those list items into a flat file using spring batch.
You batch job steps needs to have the following itemReader, itemProcesor and itemWriter.
listitemreader - read the element of the list
itemProcessor - create an item processor that processes the item read - by making a test call
FlatFileItemWriter - writes the processor record to the file
The Spring batch guide explains the user of reader,writer and processor in good detail - http://docs.spring.io/spring-batch/reference/html/readersAndWriters.html
Following is my use case for spring batch.
Reads the input from web service. Web service will return all records.
Process the records.
Write the processed records one by one.
I'm clear about step 2 and 3 but not able to figure out how to implement a reader which can read all the records in one go. How to pass the records one by one to item processor /writer?
Should I be using tasklet instead of reader/writer?
What will your WebService Returns? A collection of object i guess!
Your ItemReader need to loop on this collection and remove items one-by-one then return null when they are all processed.
What #Kik was saying is the rest is handled by Spring batch based on your commit-interval. if you have a commit0interval of 10 for example, your reader will read 10 items, passed those 10 items to the ItemProc. then pass them again after to the writer.
Hope it clarify
EDIT: 1) In Spring Batch you have more than one option to do what you need.
Easy Option, create a custom MyWsItemReader that implements the ItemReader interface.
-Define a method init() in this class that will call your webService and put the results in a collection attribute of MyWsItemReader.
-Implements the method read() from the interface. (read carfully the contact in the doc - you must return null when you passed all the elements of the collection)
-Then, configure a stepListener around the step and implement the beforeStep() method to call the init() of your MyWsItemReader. You can autowire the reader in the listener to accomplish this.
Alternatively, your MyWsItemReader could also implements the InitializingBean. then you would have to implement the afterPropertySet() where you could call the ws and store the result in a private attribute of MyWsItemReader
regards
I have to create a batch job to do financial reconciliation. Right now i have 3 steps:
step 1 : Read an XML from the third party , convert this in our domains object, write in DB(table 1)
step 2 : Read a flatFile from our transactions datastore, write in DB (Table2)
step 3 : Read both table1 and table 2 in an aggregatorObject, process both list to find differences and set status code, write a status code in table 2
My problem is with step3. I can't find a good solution to have my ItemReader reading from 2 SQL.
I started with a custom ItemReader like this :
package batch.concilliation.readers;
#Component("conciliationReader")
public class TransactionReader implements ItemReader<TransactionsAgragegator>{
private final Logger log = Logger.getLogger(TransactionReader.class);
#Autowired
private ConciliationContext context;
#Autowired
private ServiceSommaireConciliation serviceTransactionThem;
#Autowired
private ServiceTransactionVirement serviceTransactionUs;
#Override
public TransactionsAgragegator read() throws Exception, UnexpectedInputException, ParseException, NonTransientResourceException {
TransactionsAgragegator agregator = new TransactionsAgragegator();
SommaireConciliationVirementInterac sommaire = serviceSommaireThem.findByRunNo(context.getRunNo());
List<TransactionVirement> journalSic = serviceTransactionUs.findByTimestamp(sommaire.getBeginDate(), sommaire.getEndDate());
// on place ces deux listes dans l'objet agregteur.
agregator.setListeTransactionThem(sommaire.getPayments());
agregator.setListeTransactionsUs(journalSic);
return aggregator;
}
}
This Reader use two services already implemented (DAO) that read both tables and return domain objects. I take the two lists of transaction from us and from them and put them in an aggregator object. This object would be passed to the ItemProcessor and i could do my business logic... but this reader start an infinite loop since it will never read null.
I read about ItemReaderAdapter, but i still have the same problem of looping over a collection until i get a null.
So in summary, i want to read 2 different tables and get 2 List:
List<TransactionThirdParty>
List<TransactionHome>
then My ItemProcesssor would check to see if both lists are equals or not, is one has more or less transactions then the other..etc
Any Spring Batch expert can suggest something?
The problem here is that your first two steps are chunk oriented but the third one is not. While the first two may have the usual read-process-write cycle, the third step while dependent on the first two is a one time operation. It is no more different then copying a file in batch domain.
So you should not use the ItemReader way here, because you do not have an exit criteria (that is why you never get nulls from the reader, it cannot know when the source is exhausted since it does not deal with a line or record.
That is where TaskletStep helps
The Tasklet is a simple interface that has one method, execute, which
will be a called repeatedly by the TaskletStep until it either returns
RepeatStatus.FINISHED or throws an exception to signal a failure.
So implement your third step as a Tasklet instead of chunk oriented way.
I am new to Spring Batch. I have following question.
I am using Spring Batch for developing a batch process.
I have a java array with some 'process_id' values in it. What I want to do is for each 'process_id' I need to call database stored procedure using a ItemReader. Can anyone help me to write ItemReader to achieve this?
Thanks for your help.
You should think about your reader as designed :
The reader should provide the ID collection.
It will pass each ID to a writer that will call you stored procedure.
The goal of the reader is to find data to process.
Each data is then send to a writer (or a processor, depends on your batch design).