Spring batch absolute path - spring-batch

I have a Spring Batch process
I want to use a relative path to writre spring batch exception usong a csv file, but it refuse, it missing an absolute path, why?
the file is : fo
My code :
public class MyJob {
File fo=new File("C:\\Users\\m.youneb\\Documents\\icdc\\cecWorkplace\\saveLines\\src\\main\\resources\\csv\\skip.csv");
#Bean
public Step step() throws IOException {
return steps.get("step")
.<Person, Person>chunk(5)
.reader(itemReader())
.processor(itemProcessor())
.writer(itemWriter())
.faultTolerant()
.skip(IllegalArgumentException.class)
.skip(FlatFileParseException.class)
.skipLimit(100)
.listener(new MySkipListener(fo))
.skip(Exception.class)
.build();
}
public static class MySkipListener implements SkipListener<Person, Person> {
//private FileWriter fileWriter;
private BufferedWriter bw = null;
public MySkipListener(File file) throws IOException {
//this.fileWriter = new FileWriter(file);
bw= new BufferedWriter(new FileWriter(file, true));
System.out.println("MySkipListener =========> :"+file);
}
#Override
public void onSkipInRead(Throwable throwable) {
if (throwable instanceof FlatFileParseException) {
FlatFileParseException flatFileParseException = (FlatFileParseException) throwable;
System.out.println("onSkipInRead =========> :");
try {
bw.write(flatFileParseException.getInput()+"Vérifiez les colonnes!!");
bw.newLine();
bw.flush();
// fileWriter.close();
} catch (IOException e) {
System.err.println("Unable to write skipped line to error file");
}
}
}
}
I need to work with relative path, Thanks.

To work with a relative path to your file, you can use:
File fo = new File("src/main/resources/csv/skip.csv"); // should work on windows
This assumes you are running your JVM in the directory containing src/main/resources, which is usually the root directory of a typical maven project.
Hope this helps.

Related

Spring batch write exception with file parameters

I want to write the exceptions in the same file as the item writer but do not work
// MyLitner
public static class MySkipListener implements SkipListener<CSCiviqueDTO, CSCiviqueDTO> {
private BufferedWriter bw = null;
public MySkipListener(File file) throws IOException {
bw= new BufferedWriter(new FileWriter(file, true));
}
#Override
public void onSkipInRead(Throwable throwable) {
if (throwable instanceof FlatFileParseException) {
FlatFileParseException flatFileParseException = (FlatFileParseException) throwable;
System.out.println("onSkipInRead =========> :");
try {
bw.write(flatFileParseException.getInput() + "; Step Vérifiez les colonnes !!");
bw.newLine();
bw.flush();
// fileWriter.close();
} catch (IOException e) {
System.err.println("Unable to write skipped line to error file");
}
}
}

Spring batch pass dynamic file name to onSkipInRead

I want to pass a dynamic file name to onSkipInRead.
example:
public MySkipListener(#Value("#{jobParameters['file']}") String file) throws IOException {
bw= new BufferedWriter(new FileWriter(file);
System.out.println("MySkipListener =========> :"+bw);
}
#Override
public void onSkipInRead(Throwable throwable) {
if (throwable instanceof FlatFileParseException) {
FlatFileParseException flatFileParseException = (FlatFileParseException) throwable;
try {
bw.write(flatFileParseException.getInput()+"; Step Vérifiez les colonnes !!");
bw.newLine();
bw.flush();
} catch (IOException e) {
System.err.println("Unable to write skipped line to error file"); }
}
}
Thank you.
You can create a constructor in your listener with a file and pass the file to it when you define the step. Here is an example:
class MySkipListener implements SkipListener<Integer, Integer> {
private FileWriter fileWriter;
public MySkipListener(File file) throws IOException {
this.fileWriter = new FileWriter(file);
}
// your onSkipInRead method
}
Then you pass the job parameter file to your listener when you define the step:
#Bean
#JobScope
public Step step(#Value("#{jobParameters['file']}") String file) throws IOException {
return stepBuilderFactory.get("step")
.<Integer, Integer>chunk(5)
.reader(itemReader())
.writer(itemWriter())
.listener(new MySkipListener(new File(file)))
.build();
}
Hope this helps.

Spring batch write skipper item into csv file

I want to write skipper lines in first csv file and the result of processor in second file in one step but it not works !
My code :
// => Step cecStep1
#Bean
public Step cecStep1(StepBuilderFactory stepBuilders) throws IOException {
return stepBuilders.get("fileDecrypt")
.<CSCivique, String>chunk(100)
.reader(reader1())
.processor(processor1FileDecrypt())
.writer(writer1())
.faultTolerant()
.skip(Exception.class)
.skipLimit(100)
.listener(new MySkipListener())
.build();
}
// ##################################### Step SkipListener ###################################################
public static class MySkipListener implements SkipListener {
private BufferedWriter bw = null;
public MySkipListener(File file) throws IOException {
//this.fileWriter = new FileWriter(file);
bw= new BufferedWriter(new FileWriter(file, true));
System.out.println("MySkipListener =========> :"+file);
}
#Override
public void onSkipInRead(Throwable throwable) {
if (throwable instanceof FlatFileParseException) {
FlatFileParseException flatFileParseException = (FlatFileParseException) throwable;
System.out.println("onSkipInRead =========> :");
try {
bw.write(flatFileParseException.getInput()+"; Vérifiez les colonnes !!");
bw.newLine();
bw.flush();
// fileWriter.close();
} catch (IOException e) {
System.err.println("Unable to write skipped line to error file");
}
}
}
#Override
public void onSkipInWrite(CSCivique item, Throwable t) {
System.out.println("Item " + item + " was skipped due to: " + t.getMessage());
}
#Override
public void onSkipInProcess(CSCivique item, Throwable t) {
System.out.println("Item " + item + " was skipped due to: " + t.getMessage());
}
}
#Bean
public FlatFileItemWriter<String> writer1() {
return new FlatFileItemWriterBuilder<String>().name(greetingItemWriter)
.resource(new FileSystemResource("target/test-outputs/greetings.csv"))
.lineAggregator(new PassThroughLineAggregator<>()).build();
}
Tankyou !
In your processor, you can:
throw a skippable exception for invalid items so that the skip listener intercepts them and writes them to the specified file
let valid items go to the writer so that they are written as configured in the item writer
For example:
class MyItemProcessor implements ItemProcessor<Object, Object> {
#Override
public Object process(Object item) throws Exception {
if (shouldBeSkipped(item)) {
throw new MySkippableException();
}
// process item
return item;
}
}
Hope this helps.

Eclipse Project dependencies dont work when exported

I am currently facing following big problem:
I have a Framework-Project (maven), where a PropertyReader is included (reads "config.properties" in the same package and returns its values):
This is the Framework-Project:
public class PropertyReaderFramework {
private static Properties props;
private static void init(){
String filename = "com/ks/framework/properties/config.properties";
InputStream input = PropertyReaderFramework.class.getClassLoader()
.getResourceAsStream(filename);
if (input == null) {
System.out.println("Sorry, unable to find " + filename);
props = null;
} else {
props = new Properties();
}
try {
props.load(input);
} catch (IOException e) {
e.printStackTrace();
}
}
public static String getProperty(String key){
if(props == null) init();
return props.getProperty(key);
}
public static Properties getProperties(){
if(props == null) init();
return props;
}
}
And my main-project, where I need the information of the properties-file just has one class (for demonstation):
package testmsg;
import com.ks.framework.properties.PropertyReaderFramework;
public class main {
public static void main(String[] args) throws InterruptedException {
try {
String basepath = PropertyReaderFramework.getProperty("remoteFileAccess.script.location");
System.out.println(basepath);
} catch (Exception e) {
e.printStackTrace();
} finally {
Thread.sleep(5000);
}
}
}
The funny thing is, that if I execute the main() class in eclipse, it reads the value from the properties correctly.
But when I export it as a runnable JAR, it throws me following error:
Can anyone help me to solve this problem? I cannot figure out why it behaves like that...

CAS consumer not working as expected

I have a CAS consumer AE which is expected to iterates over CAS objects in a pipeline, serialize them and add the serialized CASs to an xml file.
public class DataWriter extends JCasConsumer_ImplBase {
private File outputDirectory;
public static final String PARAM_OUTPUT_DIRECTORY = "outputDir";
#ConfigurationParameter(name=PARAM_OUTPUT_DIRECTORY, defaultValue=".")
private String outputDir;
CasToInlineXml cas2xml;
public void initialize(UimaContext context) throws ResourceInitializationException {
super.initialize(context);
ConfigurationParameterInitializer.initialize(this, context);
outputDirectory = new File(outputDir);
if (!outputDirectory.exists()) {
outputDirectory.mkdirs();
}
}
#Override
public void process(JCas jCas) throws AnalysisEngineProcessException {
String file = fileCollectionReader.fileName;
File outFile = new File(outputDirectory, file + ".xmi");
FileOutputStream out = null;
try {
out = new FileOutputStream(outFile);
String xmlAnnotations = cas2xml.generateXML(jCas.getCas());
out.write(xmlAnnotations.getBytes("UTF-8"));
/* XmiCasSerializer ser = new XmiCasSerializer(jCas.getCas().getTypeSystem());
XMLSerializer xmlSer = new XMLSerializer(out, false);
ser.serialize(jCas.getCas(), xmlSer.getContentHandler());*/
if (out != null) {
out.close();
}
}
catch (IOException e) {
throw new AnalysisEngineProcessException(e);
}
catch (CASException e) {
throw new AnalysisEngineProcessException(e);
}
}
I am using it inside a pipeline after all my annotators, but it couldn't read CAS objects (I am getting NullPointerException at jCas.getCas()). It looks like I don't seem to understand the proper usage of CAS consumer. I appreciate any suggestions.