Springbatch FlatfileItemReader problem with & in line - spring-batch

Question

I created an application using Springbatch. The batch read a csv file and do some stuff after. Everything works fine except when a line in the file contains the character &.
For example:
"BB1222";"Myexample & blabla";"tayoo"
I don't understand why and how to fix it, but the batch fail and can not convert the line in my object. It throw java.lang.IndexOutOfBoundsException: start ....
I defined my reader like this:
#Bean
public FlatFileItemReader<Bank> bankReader() {
FlatFileItemReader<Bank> reader = new FlatFileItemReader<Bank>();
reader.setLinesToSkip(1);
reader.setStrict(false);
reader.setEncoding("UTF-8");
reader.setLineMapper(new DefaultLineMapper<Bank>() {
{
setLineTokenizer(new DelimitedLineTokenizer() {
{
setNames(new String[]{
...
});
setDelimiter(";");
}
});
setFieldSetMapper(new BeanWrapperFieldSetMapper<Bank>() {
{
setTargetType(Bank.class);
}
});
}
});
return reader;
}
Can you help me ?
Thanks in advance !

Answer 1

Your issue does not seem to be related to the presence of &. Here is a passing test with your example:
import java.nio.charset.Charset;
import java.nio.charset.StandardCharsets;
import org.junit.jupiter.api.Assertions;
import org.junit.jupiter.api.Test;
import org.springframework.batch.item.ExecutionContext;
import org.springframework.batch.item.file.FlatFileItemReader;
import org.springframework.batch.item.file.mapping.BeanWrapperFieldSetMapper;
import org.springframework.batch.item.file.mapping.DefaultLineMapper;
import org.springframework.batch.item.file.transform.DelimitedLineTokenizer;
import org.springframework.core.io.ByteArrayResource;
class DemoApplicationTests {
#Test
void testFlatFileItemReaderWithAmpersandInInput() throws Exception {
// given
String line = "\"BB1222\";\"Myexample & blabla\";\"tayoo\"";
Charset encoding = StandardCharsets.UTF_8;
FlatFileItemReader<Bank> reader = new FlatFileItemReader<>();
reader.setStrict(false);
reader.setEncoding(encoding.name());
reader.setLineMapper(new DefaultLineMapper<Bank>() {
{
setLineTokenizer(new DelimitedLineTokenizer() {
{
setNames("code", "name", "address");
setDelimiter(";");
}
});
setFieldSetMapper(new BeanWrapperFieldSetMapper<Bank>() {
{
setTargetType(Bank.class);
}
});
}
});
reader.setResource(new ByteArrayResource(line.getBytes(encoding)));
// when
reader.open(new ExecutionContext());
Bank item1 = reader.read();
Bank item2 = reader.read();
reader.close();
// then
Assertions.assertNotNull(item1);
Assertions.assertEquals("BB1222", item1.getCode());
Assertions.assertEquals("Myexample & blabla", item1.getName());
Assertions.assertEquals("tayoo", item1.getAddress());
Assertions.assertNull(item2);
}
public static class Bank {
String code;
String name;
String address;
public String getCode() {
return code;
}
public void setCode(String code) {
this.code = code;
}
public String getName() {
return name;
}
public void setName(String name) {
this.name = name;
}
public String getAddress() {
return address;
}
public void setAddress(String address) {
this.address = address;
}
}
}
You didn't share the full stacktrace, but you probably have a line with less or more tokens than expected. Moreover, since you configured the reader with UTF-8, you need to make sure the input file is encoded in UTF-8.