Spring batchUpdate with IN clause - postgresql

Question

Given a simple table
create table car (
make varchar
model varchar
)
And the following DAO code
NamedParameterJdbcTemplate template;
String SQL = "delete from car where make = :make and model in (:model)";
void batchDelete(final Map<String, Collection<String>> map) {
SqlParameterSource[] params = map.entrySet().stream()
.map(entry -> toParams(entry.getKey(), entry.getValue()))
.toArray(SqlParameterSource[]::new);
template.batchUpdate(SQL, params);
}
void delete(final Map<String, Collection<String>> map) {
map.forEach((make, models) -> {
SqlParameterSource params = toParams(make, models);
template.update(SQL, params);
});
}
SqlParameterSource toParams(final String make, final Collection<String> models) {
return new MapSqlParameterSource("make", make)
.addValue("model", new ArrayList<>(models));
}
The batch delete function fails when the maps has 2 keys with different number of values for the IN clause in a batch. Assume Map.of creates and ordered Map.
// runs fine - 2 values for each key
batchDelete(Map.of("VW", Arrays.asList("Polo", "Golf"), "Toyota", Arrays.asList("Yaris", "Camry")));
// fails - first key has 1 value, second key has 2 values
batchDelete(Map.of("Toyota", Arrays.asList("Yaris"), "VW", Arrays.asList("Polo", "Golf")));
// runs fine - key with bigger list comes first
batchDelete(Map.of("VW", Arrays.asList("Polo", "Golf"), "Toyota", Arrays.asList("Yaris")));
// non batch delete runs fine either way
delete(Map.of("Toyota", Arrays.asList("Yaris"), "VW", Arrays.asList("Polo", "Golf")));
Spring documentation sort of alludes to that
https://docs.spring.io/spring/docs/current/spring-framework-reference/data-access.html#jdbc-in-clause
The SQL standard allows for selecting rows based on an expression that includes a variable list of values. A typical example would be select * from T_ACTOR where id in (1, 2, 3). This variable list is not directly supported for prepared statements by the JDBC standard; you cannot declare a variable number of placeholders. You need a number of variations with the desired number of placeholders prepared, or you need to generate the SQL string dynamically once you know how many placeholders are required. The named parameter support provided in the NamedParameterJdbcTemplate and JdbcTemplate takes the latter approach.
The error message is
The column index is out of range: 3, number of columns: 2.; nested exception is org.postgresql.util.PSQLException: The column index is out of range: 3, number of columns: 2.
What happens is the following line in NamedParameterJdbcTemplate # batchUpdate:
PreparedStatementCreatorFactory pscf = getPreparedStatementCreatorFactory(parsedSql, batchArgs[0]);
will create a dynamic sql out of the first batch arg length:
delete from car where make = ? and model in (?)
So the 2nd batch item which has 2 models will fail as there is only 1 placeholder.
What would be a workaround ? (other than grouping map entries by number of values)
Solution
Went back to plain old PreparedStatement
SQL - use ANY instead of IN
delete from car where make = ? and model = any (?)
DAO
Connection con;
PreparedStatement ps = con.prepareStatement("SQL");
map.forEach((make, models) -> {
int col = 0;
ps.setString(++col, make);
ps.setArray(++col, con.createArrayOf("text", models));
ps.addBatch();
});
ps.executeBatch();

Answer 1

I would recommend changing the SQL to look something more like this:
String SQL = "DELETE FROM car WHERE (make, model) IN (:ids)";
If you do it this way then you can use something similar to the answer I gave on this question: NamedJDBCTemplate Parameters is list of lists. Doing it this way means you can use NamedParameterJdbcTemplate.update(String sql, Map<String, ?> paramMap). Where in your paramMap the key would be "ids" and the value would be an instance of Collection<Object[]> where each entry in the collection is an array containing the value pairs you want to delete:
List<Object[]> params = new ArrayList<>();//you can make this any instance of collection you want
for (Car car : cars) {
params.add(new Object[] { car.getMake(), car.getModel() });
//this is just to provide an example of what I mean, obviously this will probably be different in your app.
}