Talend load fails in midway resulting rollback - talend

Question

I have a tfileinputdelimited component and a tmap and the result is pass to the tfileoutdelimited which creates a csv file.
now in middle of the job some times data load fails resulting rollback of the destination file.
It cause wastage of resources and time.
can anyone provide a way of approach so that once a job fails in between so that data that is passed will go to save state and the next time when job runs then it again start from the point of failure only.

Answer 1

Talend won't rollback a process when writing in tFileOutputDelimited. If you got empty output file, it means that your job died prematurely and no record were written in the output buffer.
If an error occurs while writing in the file, then the following code (generated by tFileOutputDelimited) close the outputBuffer and flush the data successfully inserted before the error :
...
} finally {
if (outtFileOutputDelimited_1 != null) {
outtFileOutputDelimited_1.flush();
outtFileOutputDelimited_1.close();
}
...
}
...
There's no real "resume" feature in Talend, but you can create your own die&resume process in the job as following :
tFileInput1 ==> tHashOutput
tFileInput2 = main => tMap ==> tFileOutput1
tHashInput =lookup=> tMap
tFileInput1 : reads the data generated by the last run of your job and is stored in memory with tHashOutput
tFileInput2 : reads your input file
tFileOutput1 : stores the output data
tHashInput : reads the data in memory and serves as lookup in the tMap
In your tMap, create an inner join between tFileInput1 and tHashInput. Then, for your output schema, select catch lookup inner join reject to process all the record that are not in tHashInput.
Not sure that it will save resource and time. The best way to manage errors is to identify them and do all the checks in the job to avoid them !
For more clarity, could you give an example of error that occurs when you run the job ?