I'm having problems understanding how MyBatis maps parameters passed in, and then marshalling the results back to the caller. I'd like to pass a POJO in, use properties set in there as part of the query and pass the results back in a map form.
My Mapping file function (without using the parameters):
<select id="getTotalUniqueUserDetails"
resultType="map"
parameterType="RequestFilter">
select KEY,sum(SINGLE_VALUE) as TOTAL from (
select schut.user_type_name as KEY, schuc.usage_count AS SINGLE_VALUE
from usage_count schuc
left join users schu
on schuc.user_key=schu.user_key
left join user_types schut
on schut.user_type_id=schu.user_type_id
)
GROUP by KEY;
</select>
My Mapping file function (with the parameters)
<select id="getTotalUniqueUserDetails"
resultType="map"
parameterType="RequestFilter">
select KEY,sum(SINGLE_VALUE) as TOTAL from (
select schut.user_type_name as KEY, schuc.usage_count AS SINGLE_VALUE
from usage_count schuc
left join users schu
on schuc.user_key=schu.user_key
left join user_types schut
on schut.user_type_id=schu.user_type_id
where schuc.year_month between
to_date('#{fromDate}','yyyy-mm-dd')
and to_date('#{toDate}','yyyy-mm-dd')
)
GROUP by KEY;
</select>
My mapper interface
public interface TotalUniqueUsers {
Object getTotalUniqueUserDetails(RequestFilter filter);
}
public class RequestFilter {
private String fromDate;
private String toDate;
... getters and setters for both above
}
The code that calls the query:
SqlSession sqlSession = DBConnection.getInstance().getSqlSession();
System.out.println("####"+filter.getFromDate());
System.out.println("####"+filter.getToDate());
TotalUniqueUsers testMapper = sqlSession.getMapper(TotalUniqueUsers.class);
return testMapper.getTotalUniqueUserDetails(filter);
With no parameters used in mapping file I get this error:
## The error occurred while setting parameters
### SQL: select KEY,sum(SINGLE_VALUE) as TOTAL from ( select schut.user_type_name as KEY, schuc.usage_count AS SINGLE_VALUE from usage_count schuc left join users schu on schuc.user_key=schu.user_key left join user_types schut on schut.user_type_id=schu.user_type_id ) GROUP by KEY;
### Cause: java.sql.SQLSyntaxErrorException: ORA-00933: SQL command not properly ended
With parameters referenced in mapping file I get this error:
### Cause: org.apache.ibatis.type.TypeException: Could not set parameters for mapping: ParameterMapping{property='fromDate', mode=IN, javaType=class java.lang.String, jdbcType=null, numericScale=null, resultMapId='null', jdbcTypeName='null', expression='null'}. Cause: org.apache.ibatis.type.TypeException: Error setting non null for parameter #1 with JdbcType null . Try setting a different JdbcType for this parameter or a different configuration property. Cause: java.sql.SQLException: Invalid column index
If you've read this much, thank you from the Woodsman.
In short, if I use the parameter passed in the object, it says it can't use a null value. If I don't then it gives me some other error.
The result set should be something like
KEY TOTAL
GROUP1 33
GROUP2 55
I was hoping I could just let it stuff the result set into a map, with the keys being "GROUP1", "GROUP2", with their respective values.
Please tell me how to properly refer to the properties in the object in the map and how to marshall it back. Should I use a custom object?
You need to remove...
The semicolon at the end of the statement. Although not all drivers reject it, it is a common cause for ORA-00933 with Oracle's JDBC driver.
The single quotes surrounding the parameter placeholders e.g. '#{toDate}' to #{toDate}.
With single quotes, the first argument of to_date function becomes a literal instead of a java.sql.PreparedStatement placeholder (e.g. to_date('?', 'yyyy-mm-dd') instead of to_date(?, 'yyyy-mm-dd')). When MyBatis tries to call PreparedStatement#setString(int parameterIndex, String parameterValue), there is no placeholder in the statement and the driver throws the exception.
Related
I've been trying to dynamically use a PostgreSQL 13 native query:
public interface TasksRepository extends JpaRepository<Task, Long>, JpaSpecificationExecutor<Task> {
}
#AllArgsConstructor
public class TaskSpecification implements Specification<Task> {
private final String entityCode;
private final UUID entityId;
#Override
public Predicate toPredicate(Root<Task> root, CriteriaQuery<?> query, CriteriaBuilder builder) {
// see https://www.postgresql.org/docs/13/functions-json.html
// jsonb_path_exists ( target jsonb, path jsonpath [, vars jsonb [, silent boolean ]] ) → boolean
String template = "$[*] ? (#.entityCode == $code && #.entityId == $id)";
String variable = "{\"code\":\"?1\", \"id\":\"?2\"}"
.replace("?1", this.entityCode)
.replace("?2", this.entityId.toString());
return builder.isTrue(
builder.function("jsonb_path_exists", Boolean.class,
/* target */ root.<List<RelatedEntity>>get("taskTags"),
/* path */ builder.literal("'" + template + "'::jsonpath"), //DEBUG CAST
/* vars */ builder.literal("'" + variable + "'::jsonb"), //DEBUG CAST
/* silent */ builder.literal(Boolean.FALSE)
));
}
}
But ended up with traumatic errors, despite my casting attempt:
Hibernate:
select
task0_.id as id1_0_,
task0_.business_unit as business2_0_,
task0_.due_date as due_date3_0_,
task0_.is_urgent as is_urgen4_0_,
task0_.task_tags as task_tag5_0_,
task0_.task_text as task_tex6_0_,
task0_.task_type as task_typ7_0_
from
tasks_table task0_
where
jsonb_path_exists(task0_.task_tags,?,?,?)=true
binding parameter [1] as [VARCHAR] - ['$[*] ? (#.entityCode == $code && #.entityId == $id)'::jsonpath]
binding parameter [2] as [VARCHAR] - ['{"code":"ETY", "id":"bedb1903-3827-4507-883b-d41888d2ed68"}'::jsonb]
binding parameter [3] as [BOOLEAN] - [false]
SQL Error: 0, SQLState: 42883
ERROR: function jsonb_path_exists(jsonb, character varying, character varying, boolean) does not exist
Indice : No function matches the given name and argument types. You might need to add explicit type casts.
I've tried to cast the above inner query parameters, but I suspect that this is a JPA-level issue; but I couldn't find the corresponding types to cast (jsonpath, jsonb) in my dependencies for them to applied with builder/Expression#as
Maybe the function is not visible (with schema issue or something alike?)
Thanks for any help
Try removing the cast ::jsonpath and give it a try.
A json path is not a data type, so no need for the cast. Instead this part root.<List<RelatedEntity>>get("taskTags") shoud be a valid json object, I am not sure how this is rendered in the query.
To verify what is rendered and the values binded to the query, enable logging for hibernate as such in application.properties;
logging.level.org.hibernate.SQL=trace
logging.level.org.hibernate.type.descriptor.sql.BasicBinder=trace
This will show you the query and the value passed in the query.
I try to run with for-loops through different parameter names and try to store their value in a table with UPDATE. The "+AktuellerParameterWert+" statement in the UPDATE statement just shows the name of the parameter, not the value of the parameter as intended. The parameters are declared on the graphical interface of Anylogic, have names like "Parameter12" and contain int values. I guess the UPDATE statement can't make a connection to the parameter on the graphical interface. Any help regarding those two problems would be greatly appreciated :)
Error: root:
Error occurred when executing SQL query in modify() method
Caused by: user lacks privilege or object not found: PARAMETER11 in statement [UPDATE konfigurations_matrix SET station1=Parameter11 WHERE produktionsschritte = 'A']
int AnzahlAvgs=4;
int AnzahlStationen=2;
int AvgCount;
int StationenCount;
String ParameterName = "Parameter";
String AktuellerParameterWert;
String Station = "station";
String AktuelleStation;
String Produktionsschritt;
for (StationenCount=1; StationenCount<=AnzahlStationen; StationenCount++) {
for (AvgCount=1; AvgCount<=AnzahlAvgs; AvgCount++) {
AktuellerParameterWert = ParameterName + StationenCount + AvgCount;
System.out.println(AktuellerParameterWert);
AktuelleStation = Station + StationenCount;
Produktionsschritt = String.valueOf((char)(StationenCount + 64));
executeStatement("UPDATE konfigurations_matrix SET '"+AktuelleStation+"'="+AktuellerParameterWert+" WHERE produktionsschritte = '"+Produktionsschritt+"'");
}
}
You are not referencing the parameter properly from where you call the SQL code.
You need to refer to your parameter in its proper location, relative to from where you are calling the SQL statement. This is basic OOP and very important to understand. Search the AnyLogic help for "where am I and how do I get to" to learn more.
Given a simple table
create table car (
make varchar
model varchar
)
And the following DAO code
NamedParameterJdbcTemplate template;
String SQL = "delete from car where make = :make and model in (:model)";
void batchDelete(final Map<String, Collection<String>> map) {
SqlParameterSource[] params = map.entrySet().stream()
.map(entry -> toParams(entry.getKey(), entry.getValue()))
.toArray(SqlParameterSource[]::new);
template.batchUpdate(SQL, params);
}
void delete(final Map<String, Collection<String>> map) {
map.forEach((make, models) -> {
SqlParameterSource params = toParams(make, models);
template.update(SQL, params);
});
}
SqlParameterSource toParams(final String make, final Collection<String> models) {
return new MapSqlParameterSource("make", make)
.addValue("model", new ArrayList<>(models));
}
The batch delete function fails when the maps has 2 keys with different number of values for the IN clause in a batch. Assume Map.of creates and ordered Map.
// runs fine - 2 values for each key
batchDelete(Map.of("VW", Arrays.asList("Polo", "Golf"), "Toyota", Arrays.asList("Yaris", "Camry")));
// fails - first key has 1 value, second key has 2 values
batchDelete(Map.of("Toyota", Arrays.asList("Yaris"), "VW", Arrays.asList("Polo", "Golf")));
// runs fine - key with bigger list comes first
batchDelete(Map.of("VW", Arrays.asList("Polo", "Golf"), "Toyota", Arrays.asList("Yaris")));
// non batch delete runs fine either way
delete(Map.of("Toyota", Arrays.asList("Yaris"), "VW", Arrays.asList("Polo", "Golf")));
Spring documentation sort of alludes to that
https://docs.spring.io/spring/docs/current/spring-framework-reference/data-access.html#jdbc-in-clause
The SQL standard allows for selecting rows based on an expression that includes a variable list of values. A typical example would be select * from T_ACTOR where id in (1, 2, 3). This variable list is not directly supported for prepared statements by the JDBC standard; you cannot declare a variable number of placeholders. You need a number of variations with the desired number of placeholders prepared, or you need to generate the SQL string dynamically once you know how many placeholders are required. The named parameter support provided in the NamedParameterJdbcTemplate and JdbcTemplate takes the latter approach.
The error message is
The column index is out of range: 3, number of columns: 2.; nested exception is org.postgresql.util.PSQLException: The column index is out of range: 3, number of columns: 2.
What happens is the following line in NamedParameterJdbcTemplate # batchUpdate:
PreparedStatementCreatorFactory pscf = getPreparedStatementCreatorFactory(parsedSql, batchArgs[0]);
will create a dynamic sql out of the first batch arg length:
delete from car where make = ? and model in (?)
So the 2nd batch item which has 2 models will fail as there is only 1 placeholder.
What would be a workaround ? (other than grouping map entries by number of values)
Solution
Went back to plain old PreparedStatement
SQL - use ANY instead of IN
delete from car where make = ? and model = any (?)
DAO
Connection con;
PreparedStatement ps = con.prepareStatement("SQL");
map.forEach((make, models) -> {
int col = 0;
ps.setString(++col, make);
ps.setArray(++col, con.createArrayOf("text", models));
ps.addBatch();
});
ps.executeBatch();
I would recommend changing the SQL to look something more like this:
String SQL = "DELETE FROM car WHERE (make, model) IN (:ids)";
If you do it this way then you can use something similar to the answer I gave on this question: NamedJDBCTemplate Parameters is list of lists. Doing it this way means you can use NamedParameterJdbcTemplate.update(String sql, Map<String, ?> paramMap). Where in your paramMap the key would be "ids" and the value would be an instance of Collection<Object[]> where each entry in the collection is an array containing the value pairs you want to delete:
List<Object[]> params = new ArrayList<>();//you can make this any instance of collection you want
for (Car car : cars) {
params.add(new Object[] { car.getMake(), car.getModel() });
//this is just to provide an example of what I mean, obviously this will probably be different in your app.
}
I'm trying to use jOOQ to create a function akin to arrayRemove but that allows removing several elements at once from a PostgreSQL column of type uuid[].
So my first attempt was:
private Field<UUID[]> arrayRemoveAll(final Field<UUID[]> field, final Set<UUID> elements) {
return select(field("array_agg(tab.col)", UUID[].class))
.from(unnest(field).as("tab", "col"))
.where(field("tab.col", UUID.class).notIn(elements))
.asField();
}
Which succeeds at removing every requested element, but has the problem of returning null instead of an empty array if I attempt to remove every element.
So I added a coalesce to my code to make it return an empty array:
private Field<UUID[]> arrayRemoveAll(final Field<UUID[]> field, final Set<UUID> elements) {
final Field<UUID[]> newArray = select(field("array_agg(tab.col)", UUID[].class))
.from(unnest(field).as("tab", "col"))
.where(field("tab.col", UUID.class).notIn(elements))
.asField();
return coalesce(newArray, field("{}", UUID[].class));
}
But running this code threw this exception:
org.jooq.exception.DataAccessException: SQL [<<confidential SQL removed>>]
Caused by: org.postgresql.util.PSQLException: ERROR: syntax error at or near ")"
This is the part of the SQL exception it is complaining about (notice the trailing comma and missing 2nd parameter in the coalesce):
coalesce((select array_agg(tab.col)
from unnest("my_schema"."my_table"."my_field") as "tab"("col")
where tab.col not in (?, ?)), )
Is this a bug in jOOQ?
I found that I had a mix of field and val in the code above, changing field("{}", UUID[].class) to val(new UUID[0]) solves the problem.
Also check Lukas Eder's answer about how to solve the issue using field.
So the final code, with generics, looks like this:
private <T> Field<T[]> arrayRemoveAll(final Field<T[]> field, final Set<T> elements, final T[] emptyArray) {
final Field<T[]> newArray = select(field("array_agg(tab.col)"))
.from(unnest(field).as("tab", "col"))
.where(field("tab.col").notIn(elements))
.asField();
return coalesce(newArray, val(emptyArray));
}
And you can use it in your statements like this:
using(configuration)
.update(MY_TABLE)
.set(MY_TABLE.MY_COLUMN,
arrayRemoveAll(MY_TABLE.MY_COLUMN, someElements, new UUID[0]))
.where(MY_TABLE.ID.eq(...))
.execute();
Your field("{}") does not generate the {} string in the SQL, but is considered a part of jOOQ's plain SQL templating language, which unfortunately doesn't allow for escaping those braces:
https://www.jooq.org/doc/latest/manual/sql-building/plain-sql-templating
Luckily, PostgreSQL supports a more formal, standards-compliant way to create an empty array literal:
field("array[]::uuid[]", UUID.class)
This is the query I am trying to run in PostgreSQL:
SELECT * FROM message WHERE id IN (
SELECT unnest(message_ids) "mid"
FROM session_messages WHERE session_id = '?' ORDER BY "mid" ASC
);
However, I am not able do something:
create.selectFrom(Tables.MESSAGE).where(Tables.MESSAGE.ID.in(
create.select(DSL.unnest(..))
Because DSL.unnest is a Table<?>, which makes sense since it is trying to take a List-like object (mostly a literal) and convert it to table.
I have a feeling that I need to find a way to wrap the function around my field name, but I have no clue as to how to proceed.
NOTE. The field message_ids is of type bigint[].
EDIT
So, this is how I am doing it now, and it works exactly as expected, but I am not sure if this is the best way to do it:
Field<Long> unnestMessageIdField = DSL.field(
"unnest(" + SESSION_MESSAGES.MESSAGE_IDS.getName() + ")",
Long.class)
.as("mid");
Field<Long> messageIdField = DSL.field("mid", Long.class);
MESSAGE.ID.in(
ctx.select(messageIdField).from(
ctx.select(unnestMessageIdField)
.from(Tables.CHAT_SESSION_MESSAGES)
.where(Tables.CHAT_SESSION_MESSAGES.SESSION_ID.eq(sessionId))
)
.where(condition)
)
EDIT2
After going through the code on https://github.com/jOOQ/jOOQ/blob/master/jOOQ/src/main/java/org/jooq/impl/DSL.java I guess the right way to do this would be:
DSL.function("unnest", SQLDataTypes.BIGINT.getArrayType(), SESSION_MESSAGES.MESSAGE_IDS)
EDIT3
Since as always lukas is here for my jOOQ woes, I am going to capitalize on this :)
Trying to generalize this function, in a signature of sort
public <T> Field<T> unnest(Field<T[]> arrayField) {
return DSL.function("unnest", <??>, arrayField);
}
I don't know how I can fetch the datatype. There seems to be a way to get DataType<T[]> from DataType<T> using DataType::getArrayDataType(), but the reverse is not possible. There is this class I found ArrayDataType, but it seems to be package-private, so I cannot use it (and even if I could, it does not expose the field elementType).
Old PostgreSQL versions had this funky idea that it is OK to produce a table from within the SELECT clause, and expand it into the "outer" table, as if it were declared in the FROM clause. That is a very obscure PostgreSQL legacy, and this example is a good chance to get rid of it, and use LATERAL instead. Your query is equivalent to this one:
SELECT *
FROM message
WHERE id IN (
SELECT "mid"
FROM session_messages
CROSS JOIN LATERAL unnest(message_ids) AS t("mid")
WHERE session_id = '?'
);
This can be translated to jOOQ much more easily as:
DSL.using(configuration)
.select()
.from(MESSAGE)
.where(MESSAGE.ID).in(
select(field(name("mid"), MESSAGE.ID.getDataType()))
.from(SESSION_MESSAGES)
.crossJoin(lateral(unnest(SESSION_MESSAGES.MESSAGE_IDS)).as("t", "mid"))
.where(SESSION_MESSAGES.SESSION_ID.eq("'?'"))
)
The Edit3 in the question is quite close to a decent solution for this problem.
We can create a custom generic unnest method for jOOQ which accepts Field and use it in jOOQ query normally.
Helper method:
public static <T> Field<T> unnest(Field<T[]> field) {
var type = (Class<T>) field.getType().getComponentType();
return DSL.function("unnest", type, field);
}
Usage:
public void query(SessionId sessionId) {
var field = unnest(SESSION_MESSAGES.MESSAGE_IDS, UUID.class);
dsl.select().from(MESSAGE).where(
MESSAGE.ID.in(
dsl.select(field).from(SESSION_MESSAGES)
.where(SESSION_MESSAGES.SESSION_ID.eq(sessionId.id))
.orderBy(field)
)
);
}