Escaping hstore contains operators in a JDBC Prepared statement - postgresql

I am using PostgreSQL 9.1.4 with hstore and the PostgreSQL JDBC driver (9.1-901.jdbc4).
I am trying to use the contains operators (?, ?&, ?|) in a PreparedStatement, however the ? character is parsed as a variable placeholder. Can this character be escaped to send the correct operator in the query?
An example:
PreparedStatement stmt = conn.prepareStatement("SELECT a, b FROM table1 WHERE c ? 'foo' AND d = ?");
stmt.setInt(1, dValue);
stmt.executeQuery();
In this form the following example would raise an exception:
org.postgresql.util.PSQLException: No value specified for parameter 2.
Update:
After investigating the query parser in the pgjdbc driver this snippet seems to indicate that it is not possible to escape the ? character. The questions that remain are:
Is there anything in the JDBC spec which allows a ? to be escaped and be anything other than a parameter placeholder?
Is there any better work around for this issue than just using plain Statements with variables manually inserted into the query string?

Effectively, it looks like the java SQL parser is not hstore compliant.
But since the syntax c ? 'foo' is equivalent to exist(c, 'foo'), you can easily workaround this problem. Have a look at the following page to see what the verbose operators for hstore are.
Postgres hstore documentation

There is a discussion about this issue on pgsql-hackers mailing list: http://grokbase.com/t/postgresql/pgsql-hackers/1325c6ys9n/alias-hstores-to-so-that-it-works-with-jdbc
For now I like most this workaround which also supports indexes:
CREATE FUNCTION exist_inline (hstore, text) RETURNS bool AS $$ SELECT $1 ? $2; $$ LANGUAGE sql;

You can use this query to find the function backing an operator in PostgreSQL like this. In your example:
SELECT
oprname,
oprcode || '(' || format_type(oprleft, NULL::integer) || ', '
|| format_type(oprright, NULL::integer) || ')' AS function
FROM pg_operator
WHERE oprname LIKE '?%'
AND (SELECT oid FROM pg_type WHERE typname = 'hstore') IN (oprleft, oprright);
This produces:
|oprname|function |
|-------|--------------------------|
|? |exist(hstore, text) |
|?| |exists_any(hstore, text[])|
|?& |exists_all(hstore, text[])|
See also a related question about using JSON operators containing ?. Note that the function usage may not profit from the same indexing capability when using a GIN index on your HSTORE column.

If you'd like to add multiple key-value pairs using PreparedStatement then you can do:
PreparedStatement ps = c.prepareStatement(
"insert into xyz(id, data) values(?, hstore(?, ?))");
ps.setLong(1, 23456L);
ps.setArray(2, c.createArrayOf("text", new String[]{"name", "city"}));
ps.setArray(3, c.createArrayOf("text", new String[]{"Duke", "Valley"}));
This will insert: 23456, 'name=>Duke, city=>Valley'

Related

DB2 Assign result of with clause to variable

I am using DB2 LUW and want to a assign a result of a With clause to a variable in a stored procedure.
I got the exception
{0:0} An unexpected token "AS" was found following "l = (WITH BASE". Expected tokens may include: "JOIN".. SQLCODE=-104, SQLSTATE=42601, DRIVER=4.28.11
Is it possible to assign the result on this way or should I have to solve it with a cursor?
DECLARE result CLOB(8M);
SET result = (WITH BASE AS (
xxx
)
SELECT JSON_ARRAY (select json_objects FROM ITEMS format json) FROM SYSIBM.SYSDUMMY1);
Use instead the syntax style:
with ctename AS ( ... ) SELECT ... INTO ... FROM ctename;

Approaches to execute PostgreSQL's concat() instead of || in JOOQ?

The ||-operator and the concat(...)-function in PostgreSQL behave differently.
select 'ABC'||NULL||'def';
-- Result: NULL
select concat('ABC', NULL, 'def');
-- Result: 'ABCdef'
concat(...) ignores NULL values, but a NULL whithin a || expression makes the whole result become NULL.
In JOOQ, the DSL.concat() in the PostgreSQL dialect renders expressions using the ||-operator:
Java: dsl.select(
DSL.concat(
DSL.inline("ABC"),
DSL.inline(null, SQLDataType.VARCHAR),
DSL.inline("def"))
).execute();
SQL: select ('ABC' || null || 'def')
Result: NULL
I am looking for (elegant?) ways to invoke the concat(...)-function instead of the ||-operator via JOOQ in PostgreSQL:
Java: dsl.select(???).execute();
SQL: select concat('ABC', null, 'def')
Result: 'ABCdef'
I found two ways to achieve the posed objective.
Approach #1:
dsl.select(
field(
"concat({0})",
SQLDataType.VARCHAR,
list(
inline("ABC"),
inline(null, SQLDataType.VARCHAR),
inline("def")
)
)
).execute();
This has to the intended behavior, but necessitates the in my eyes ugly "concat({0})". A more elegant approach from my point of view is:
Approach #2:
dsl.select(
function(
"concat",
SQLDataType.VARCHAR,
inline("ABC"),
inline(null, SQLDataType.VARCHAR),
inline("def")
)
).execute();
This solution does not involve inline SQL with placeholders as approach #1. Why JOOQ generates || instead of concat(...) in the first place is still to be expounded, though.

DB2 SQL Error: SQLCODE=-302 while executing prepared statement

I have a SQL query which takes user inputs hence security flaw is present.
The existing query is:
SELECT BUS_NM, STR_ADDR_1, CITY_NM, STATE_CD, POSTAL_CD, COUNTRY_CD,
BUS_PHONE_NB,PEG_ACCOUNT_ID, GDN_ALERT_ID, GBIN, GDN_MON_REF_NB,
ALERT_DT, ALERT_TYPE, ALERT_DESC,ALERT_PRIORITY
FROM ( SELECT A.BUS_NM, AE.STR_ADDR_1, A.CITY_NM, A.STATE_CD, A.POSTAL_CD,
CC.COUNTRY_CD, A.BUS_PHONE_NB, A.PEG_ACCOUNT_ID, 'I' ||
LPAD(INTL_ALERT_DTL_ID, 9,'0') GDN_ALERT_ID,
LPAD(IA.GBIN, 9,'0') GBIN, IA.GDN_MON_REF_NB,
DATE(IAD.ALERT_TS) ALERT_DT,
XMLCAST(XMLQUERY('$A/alertTypeConfig/biqCode/text()' passing
IAC.INTL_ALERT_TYPE_CONFIG as "A") AS CHAR(4)) ALERT_TYPE,
, ROW_NUMBER() OVER () AS "RN"
FROM ACCOUNT A, Other tables
WHERE IA.GDN_MON_REF_NB = '100'
AND A.PEG_ACCOUNT_ID = IAAR.PEG_ACCOUNT_ID
AND CC.COUNTRY_CD = A.COUNTRY_ISO3_CD
ORDER BY IA.INTL_ALERT_ID ASC )
WHERE ALERT_TYPE IN (" +TriggerType+ ");
I changed it to accept TriggerType from setString like:
SELECT BUS_NM, STR_ADDR_1, CITY_NM, STATE_CD, POSTAL_CD, COUNTRY_CD,
BUS_PHONE_NB,PEG_ACCOUNT_ID, GDN_ALERT_ID, GBIN, GDN_MON_REF_NB,
ALERT_DT, ALERT_TYPE, ALERT_DESC,ALERT_PRIORITY
FROM ( SELECT A.BUS_NM, AE.STR_ADDR_1, A.CITY_NM, A.STATE_CD, A.POSTAL_CD,
CC.COUNTRY_CD, A.BUS_PHONE_NB, A.PEG_ACCOUNT_ID,
'I' || LPAD(INTL_ALERT_DTL_ID, 9,'0') GDN_ALERT_ID,
LPAD(IA.GBIN, 9,'0') GBIN, IA.GDN_MON_REF_NB,
DATE(IAD.ALERT_TS) ALERT_DT,
XMLCAST(XMLQUERY('$A/alertTypeConfig/biqCode/text()' passing
IAC.INTL_ALERT_TYPE_CONFIG as "A") AS CHAR(4)) ALERT_TYPE,
ROW_NUMBER() OVER () AS "RN"
FROM ACCOUNT A, other tables
WHERE IA.GDN_MON_REF_NB = '100'
AND A.PEG_ACCOUNT_ID = IAAR.PEG_ACCOUNT_ID
AND CC.COUNTRY_CD = A.COUNTRY_ISO3_CD
ORDER BY IA.INTL_ALERT_ID ASC )
WHERE ALERT_TYPE IN (?);
Setting trigger type as below:
if (StringUtils.isNotBlank(request.getTriggerType())) {
preparedStatement.setString(1, triggerType != null ? triggerType.toString() : "");
}
Getting error as
Caused by: com.ibm.db2.jcc.am.SqlDataException: DB2 SQL Error: SQLCODE=-302, SQLSTATE=22001, SQLERRMC=null, DRIVER=4.19.26
The -302 SQLCODE indicates a conversion error of some sort.
SQLSTATE 22001 narrows that down a bit by telling us that you are trying to force a big string into a small variable. Given the limited information in your question, I am guessing it is the XMLCAST that is the culprit.
DB2 won't jam 30 pounds of crap into a 4 pound bag so to speak, it gives you an error. Maybe giving XML some extra room in the cast might be a help. If you need to make sure it ends up being only 4 characters long, you could explicitly do a LEFT(XMLCAST( ... AS VARCHAR(64)), 4). That way the XMLCAST has the space it needs, but you cut it back to fit your variable on the fetch.
The other thing could be that the variable being passed to the parameter marker is too long. DB2 will guess the type and length based on the length of ALERT_TYPE. Note that you can only pass a single value through a parameter marker. If you pass a comma separated list, it will not behave as expected (unless you expect ALERT_TYPE to also contain a comma separated list). If you are getting the comma separated list from a table, you can use a sub-select instead.
Wrong IN predicate use with a parameter.
Do not expect that IN ('AAAA, M250, ABCD') (as you try to do passing a comma-separated string as a single parameter) works as IN ('AAAA', 'M250', 'ABCD') (as you need). These predicates are not equivalent.
You need some "string tokenizer", if you want to pass such a comma-separated string like below.
select t.*
from
(
select XMLCAST(XMLQUERY('$A/alertTypeConfig/biqCode/text()' passing IAC.INTL_ALERT_TYPE_CONFIG as "A") AS CHAR(4)) ALERT_TYPE
from table(values xmlparse(document '<alertTypeConfig><biqCode>M250, really big code</biqCode></alertTypeConfig>')) IAC(INTL_ALERT_TYPE_CONFIG)
) t
--WHERE ALERT_TYPE IN ('AAAA, M250, ABCD')
join xmltable('for $id in tokenize($s, ",\s?") return <i>{string($id)}</i>'
passing cast('AAA, M250 , ABCD' as varchar(200)) as "s"
columns token varchar(200) path '.') x on x.token=t.ALERT_TYPE
;
Run the statement as is. Then you may uncomment the string with WHERE clause and comment out the rest to see what you try to do.
P.S.:
The error you get is probably because you don't specify the data type of the parameter (you don't use something like IN (cast(? as varchar(xxx))), and db2 compiler assumes that its length is equal to the length of the ALERT_TYPE expression (4 bytes).

Postgresql SQLSTATE[42P18]: Indeterminate datatype with PDO and CONCAT

I'm having issues with CONCAT() when used on a WHERE, in PDO.
The code:
<?php
require_once('config.php');
$fdate = '01/01/2010';
$tdate = '31/12/2030';
$identification = '';
$count = "SELECT count(*) as total FROM ( select time_id from doc_sent WHERE date >= :fdate AND date <= :tdate AND identification LIKE concat('%',:identification,'%') ) x;";
//$count = "SELECT count(*) as total FROM ( select time_id from doc_sent WHERE date >= :fdate AND date <= :tdate ) x;";
$stmt_count_row_main_table = $pdo->prepare($count);
$stmt_count_row_main_table->execute(['fdate' => $fdate, 'tdate' => $tdate, 'identification' => $identification]);
//$stmt_count_row_main_table->execute(['fdate' => $fdate, 'tdate' => $tdate]);
$count_row_main_table = $stmt_count_row_main_table->fetch();
print_r( $count_row_main_table);
?>
The code works when the 'identification' part is commented.
When I'm trying to use CONCAT(), it doesn't.
I tried many "version" of CONCAT() (and read many other questions, like this one: How do I create a PDO parameterized query with a LIKE statement? ) but I am always referring to the main documentation:
https://www.postgresql.org/docs/9.1/static/functions-string.html
Which say:
concat('abcde', 2, NULL, 22) --> abcde222
The FULL error when I use CONCAT() is:
PHP Fatal error: Uncaught PDOException: SQLSTATE[42P18]: Indeterminate datatype: 7 ERROR: could not determine data type of parameter $3 in /var/www/pdo-reporter/show.php:17\nStack trace:\n#0 /var/www/pdo-reporter/show.php(17): PDOStatement->execute(Array)\n#1 {main}\n thrown in /var/www/pdo-reporter/show.php on line 17
What's wrong with my code?
CONCAT is a function that takes a VARIADIC argument list, which means that internally postgres will convert them into an array of the same type.
postgres=# \df concat
List of functions
Schema | Name | Result data type | Argument data types | Type
------------+--------+------------------+---------------------+------
pg_catalog | concat | text | VARIADIC "any" | func
When trying to resolve the input type to a single type, the SQL parser fails. It can be reproduced in this simpler form:
postgres=# PREPARE p AS select concat('A', $1);
ERROR: could not determine data type of parameter $1
The parser can't figure out the datatype of $1 so it errs on the side of caution.
One easy solution is to cast the parameter as text:
postgres=# PREPARE p AS select concat($1::text);
PREPARE
or with the CAST operator:
postgres=# PREPARE p AS select concat(cast($1 as text));
PREPARE
I haven't tested with PDO but presumably it would work (given how it deals with parameters to produce prepared statements) to change the query to:
"...identification LIKE '%' || :identification || '::text%'..."
or use the '||' operator instead of concat in the query:
identification LIKE '%' || :identification || '%'
EDIT: BTW if you want to find that a parameter :X is a substring of identification, this clause is more secure: strpos(identification, :X) > 0, because :X may contain '%' or '_' without causing any side-effect in the match, contrary to what happens with LIKE.

Pgsql error: You might need to add explicit type casts

My website is just working fine til i deployed it to heroku and the problem is heroku uses pgsql and I'm using mysql and laravel framework.
my query is
$patient = Patient::where('patient_address', 'ILIKE' ,'%' . $request->input)->where('patient_sex', 'ILIKE' ,'%' . $request->gender)->whereHas('users', function($q) use($vaccine_id){
$q->where('vaccine_id','ILIKE','%' . $vaccine_id);
})->get();
here's what I'm getting when I deploy it to heroku
SQLSTATE[42883]: Undefined function: 7 ERROR: operator does not exist: integer ~~* unknown
LINE 1: ...ient_id" = "patients"."PatientID" and "vaccine_id" ILIKE $3)
HINT: No operator matches the given name and argument type(s). You might need to add explicit type casts. (SQL: select * from "patients" where "patient_address" ILIKE %San Francisco and "patient_sex" ILIKE % and exists (select * from "vaccines" inner join "immunizations" on "vaccines"."VaccineID" = "immunizations"."vaccine_id" where "immunizations"."patient_id" = "patients"."PatientID" and "vaccine_id" ILIKE %))
I have tried using cast like CAST(vaccine_id AS VARCHAR) and I' not getting the error but it doesnt return any result.
The problem is here:
$q->where('vaccine_id','ILIKE','%' . $vaccine_id)
looks like vaccine_id is integer, and you can not use operator ILIKE to integer. Try just '='
If you want to use LIKE, ILIKE or other text operator you must cast your data to text. In SQL it must looks like:
WHERE "vaccine_id"::text ILIKE val
instead
WHERE "vaccine_id" ILIKE val
You could do this:
$q->where('cast(vaccine_id AS VARCHAR)','LIKE','%' . $vaccine_id)
OR
$q->where('cast(vaccine_id AS TEXT)','LIKE','%' . $vaccine_id)