Postgresql SQLSTATE[42P18]: Indeterminate datatype with PDO and CONCAT - postgresql

I'm having issues with CONCAT() when used on a WHERE, in PDO.
The code:
<?php
require_once('config.php');
$fdate = '01/01/2010';
$tdate = '31/12/2030';
$identification = '';
$count = "SELECT count(*) as total FROM ( select time_id from doc_sent WHERE date >= :fdate AND date <= :tdate AND identification LIKE concat('%',:identification,'%') ) x;";
//$count = "SELECT count(*) as total FROM ( select time_id from doc_sent WHERE date >= :fdate AND date <= :tdate ) x;";
$stmt_count_row_main_table = $pdo->prepare($count);
$stmt_count_row_main_table->execute(['fdate' => $fdate, 'tdate' => $tdate, 'identification' => $identification]);
//$stmt_count_row_main_table->execute(['fdate' => $fdate, 'tdate' => $tdate]);
$count_row_main_table = $stmt_count_row_main_table->fetch();
print_r( $count_row_main_table);
?>
The code works when the 'identification' part is commented.
When I'm trying to use CONCAT(), it doesn't.
I tried many "version" of CONCAT() (and read many other questions, like this one: How do I create a PDO parameterized query with a LIKE statement? ) but I am always referring to the main documentation:
https://www.postgresql.org/docs/9.1/static/functions-string.html
Which say:
concat('abcde', 2, NULL, 22) --> abcde222
The FULL error when I use CONCAT() is:
PHP Fatal error: Uncaught PDOException: SQLSTATE[42P18]: Indeterminate datatype: 7 ERROR: could not determine data type of parameter $3 in /var/www/pdo-reporter/show.php:17\nStack trace:\n#0 /var/www/pdo-reporter/show.php(17): PDOStatement->execute(Array)\n#1 {main}\n thrown in /var/www/pdo-reporter/show.php on line 17
What's wrong with my code?

CONCAT is a function that takes a VARIADIC argument list, which means that internally postgres will convert them into an array of the same type.
postgres=# \df concat
List of functions
Schema | Name | Result data type | Argument data types | Type
------------+--------+------------------+---------------------+------
pg_catalog | concat | text | VARIADIC "any" | func
When trying to resolve the input type to a single type, the SQL parser fails. It can be reproduced in this simpler form:
postgres=# PREPARE p AS select concat('A', $1);
ERROR: could not determine data type of parameter $1
The parser can't figure out the datatype of $1 so it errs on the side of caution.
One easy solution is to cast the parameter as text:
postgres=# PREPARE p AS select concat($1::text);
PREPARE
or with the CAST operator:
postgres=# PREPARE p AS select concat(cast($1 as text));
PREPARE
I haven't tested with PDO but presumably it would work (given how it deals with parameters to produce prepared statements) to change the query to:
"...identification LIKE '%' || :identification || '::text%'..."
or use the '||' operator instead of concat in the query:
identification LIKE '%' || :identification || '%'
EDIT: BTW if you want to find that a parameter :X is a substring of identification, this clause is more secure: strpos(identification, :X) > 0, because :X may contain '%' or '_' without causing any side-effect in the match, contrary to what happens with LIKE.

Related

The column index is out of range: 2, number of columns: 1 error while updating jsonb column

I am trying to update jsonb column in java with mybatis.
Following is my mapper method
#Update("update service_user_assn set external_group = external_group || '{\"service_name\": \"#{service_name}\" }' where user=#{user} " +
" and service_name= (select service_name from services where service_name='Google') " )
public int update(#Param("service_name")String service_name,#Param("user") Integer user);
I am getting the following error while updating the jsonb (external_group) cloumn.
### Error updating database. Cause: org.postgresql.util.PSQLException: The column index is out of range: 2, number of columns: 1.
### The error may involve com.apds.mybatis.mapper.ServiceUserMapper.update-Inline
I am able to update with the same way for non-jsonb columns.
Also if I am putting hardcoded value it's working for jsonb columns.
How to solve this error while updating jsonb column?
You should not enclose #{} in single quotes because it will become part of a literal rather than a placeholder. i.e.
external_group = external_group || '{"service_name": "?"}' where ...
So, there will be only one placeholder in the PreparedStatement and you get the error.
The correct way is to concatenate the #{} in SQL.
You may also need to cast the literal to jsonb type explicitly.
#Update({
"update service_user_assn set",
"external_group = external_group",
"|| ('{\"service_name\": \"' || #{service_name} || '\" }')::jsonb",
"where user=#{user} and",
"service_name= (select service_name from services where service_name='Google')"})
The SQL being executed would look as follows.
external_group = external_group || ('{"service_name": "' || ? || '"}')::jsonb where ...

DB2 SQL Error: SQLCODE=-302 while executing prepared statement

I have a SQL query which takes user inputs hence security flaw is present.
The existing query is:
SELECT BUS_NM, STR_ADDR_1, CITY_NM, STATE_CD, POSTAL_CD, COUNTRY_CD,
BUS_PHONE_NB,PEG_ACCOUNT_ID, GDN_ALERT_ID, GBIN, GDN_MON_REF_NB,
ALERT_DT, ALERT_TYPE, ALERT_DESC,ALERT_PRIORITY
FROM ( SELECT A.BUS_NM, AE.STR_ADDR_1, A.CITY_NM, A.STATE_CD, A.POSTAL_CD,
CC.COUNTRY_CD, A.BUS_PHONE_NB, A.PEG_ACCOUNT_ID, 'I' ||
LPAD(INTL_ALERT_DTL_ID, 9,'0') GDN_ALERT_ID,
LPAD(IA.GBIN, 9,'0') GBIN, IA.GDN_MON_REF_NB,
DATE(IAD.ALERT_TS) ALERT_DT,
XMLCAST(XMLQUERY('$A/alertTypeConfig/biqCode/text()' passing
IAC.INTL_ALERT_TYPE_CONFIG as "A") AS CHAR(4)) ALERT_TYPE,
, ROW_NUMBER() OVER () AS "RN"
FROM ACCOUNT A, Other tables
WHERE IA.GDN_MON_REF_NB = '100'
AND A.PEG_ACCOUNT_ID = IAAR.PEG_ACCOUNT_ID
AND CC.COUNTRY_CD = A.COUNTRY_ISO3_CD
ORDER BY IA.INTL_ALERT_ID ASC )
WHERE ALERT_TYPE IN (" +TriggerType+ ");
I changed it to accept TriggerType from setString like:
SELECT BUS_NM, STR_ADDR_1, CITY_NM, STATE_CD, POSTAL_CD, COUNTRY_CD,
BUS_PHONE_NB,PEG_ACCOUNT_ID, GDN_ALERT_ID, GBIN, GDN_MON_REF_NB,
ALERT_DT, ALERT_TYPE, ALERT_DESC,ALERT_PRIORITY
FROM ( SELECT A.BUS_NM, AE.STR_ADDR_1, A.CITY_NM, A.STATE_CD, A.POSTAL_CD,
CC.COUNTRY_CD, A.BUS_PHONE_NB, A.PEG_ACCOUNT_ID,
'I' || LPAD(INTL_ALERT_DTL_ID, 9,'0') GDN_ALERT_ID,
LPAD(IA.GBIN, 9,'0') GBIN, IA.GDN_MON_REF_NB,
DATE(IAD.ALERT_TS) ALERT_DT,
XMLCAST(XMLQUERY('$A/alertTypeConfig/biqCode/text()' passing
IAC.INTL_ALERT_TYPE_CONFIG as "A") AS CHAR(4)) ALERT_TYPE,
ROW_NUMBER() OVER () AS "RN"
FROM ACCOUNT A, other tables
WHERE IA.GDN_MON_REF_NB = '100'
AND A.PEG_ACCOUNT_ID = IAAR.PEG_ACCOUNT_ID
AND CC.COUNTRY_CD = A.COUNTRY_ISO3_CD
ORDER BY IA.INTL_ALERT_ID ASC )
WHERE ALERT_TYPE IN (?);
Setting trigger type as below:
if (StringUtils.isNotBlank(request.getTriggerType())) {
preparedStatement.setString(1, triggerType != null ? triggerType.toString() : "");
}
Getting error as
Caused by: com.ibm.db2.jcc.am.SqlDataException: DB2 SQL Error: SQLCODE=-302, SQLSTATE=22001, SQLERRMC=null, DRIVER=4.19.26
The -302 SQLCODE indicates a conversion error of some sort.
SQLSTATE 22001 narrows that down a bit by telling us that you are trying to force a big string into a small variable. Given the limited information in your question, I am guessing it is the XMLCAST that is the culprit.
DB2 won't jam 30 pounds of crap into a 4 pound bag so to speak, it gives you an error. Maybe giving XML some extra room in the cast might be a help. If you need to make sure it ends up being only 4 characters long, you could explicitly do a LEFT(XMLCAST( ... AS VARCHAR(64)), 4). That way the XMLCAST has the space it needs, but you cut it back to fit your variable on the fetch.
The other thing could be that the variable being passed to the parameter marker is too long. DB2 will guess the type and length based on the length of ALERT_TYPE. Note that you can only pass a single value through a parameter marker. If you pass a comma separated list, it will not behave as expected (unless you expect ALERT_TYPE to also contain a comma separated list). If you are getting the comma separated list from a table, you can use a sub-select instead.
Wrong IN predicate use with a parameter.
Do not expect that IN ('AAAA, M250, ABCD') (as you try to do passing a comma-separated string as a single parameter) works as IN ('AAAA', 'M250', 'ABCD') (as you need). These predicates are not equivalent.
You need some "string tokenizer", if you want to pass such a comma-separated string like below.
select t.*
from
(
select XMLCAST(XMLQUERY('$A/alertTypeConfig/biqCode/text()' passing IAC.INTL_ALERT_TYPE_CONFIG as "A") AS CHAR(4)) ALERT_TYPE
from table(values xmlparse(document '<alertTypeConfig><biqCode>M250, really big code</biqCode></alertTypeConfig>')) IAC(INTL_ALERT_TYPE_CONFIG)
) t
--WHERE ALERT_TYPE IN ('AAAA, M250, ABCD')
join xmltable('for $id in tokenize($s, ",\s?") return <i>{string($id)}</i>'
passing cast('AAA, M250 , ABCD' as varchar(200)) as "s"
columns token varchar(200) path '.') x on x.token=t.ALERT_TYPE
;
Run the statement as is. Then you may uncomment the string with WHERE clause and comment out the rest to see what you try to do.
P.S.:
The error you get is probably because you don't specify the data type of the parameter (you don't use something like IN (cast(? as varchar(xxx))), and db2 compiler assumes that its length is equal to the length of the ALERT_TYPE expression (4 bytes).

Postgres SQL - different results from LIKE query using OR vs ||

I have a table with an integer column. It has 12 records numbered 1000 to 1012. Remember, these are ints.
This query returns, as expected, 12 results:
select count(*) from proposals where qd_number::text like '%10%'
as does this:
SELECT COUNT(*) FROM "proposals" WHERE (lower(first_name) LIKE '%10%' OR qd_number::text LIKE '%10%' )
but this query returns 2 records:
SELECT COUNT(*) FROM "proposals" WHERE (lower(first_name) || ' ' || qd_number::text LIKE '%10%' )
which implies using || in concatenated where expressions is not equivalent to using OR. Is that correct or am I missing something else here?
You probably have nulls in first_name. For these records (lower(first_name) || ' ' || qd_number::text results in null, so you don't find the numbers any longer.
using || in concatenated where expressions is not equivalent to using ORIs that correct or am I missing something else here?
That is correct.
|| is the string concatenation operator in SQL, not the OR operator.

Pgsql error: You might need to add explicit type casts

My website is just working fine til i deployed it to heroku and the problem is heroku uses pgsql and I'm using mysql and laravel framework.
my query is
$patient = Patient::where('patient_address', 'ILIKE' ,'%' . $request->input)->where('patient_sex', 'ILIKE' ,'%' . $request->gender)->whereHas('users', function($q) use($vaccine_id){
$q->where('vaccine_id','ILIKE','%' . $vaccine_id);
})->get();
here's what I'm getting when I deploy it to heroku
SQLSTATE[42883]: Undefined function: 7 ERROR: operator does not exist: integer ~~* unknown
LINE 1: ...ient_id" = "patients"."PatientID" and "vaccine_id" ILIKE $3)
HINT: No operator matches the given name and argument type(s). You might need to add explicit type casts. (SQL: select * from "patients" where "patient_address" ILIKE %San Francisco and "patient_sex" ILIKE % and exists (select * from "vaccines" inner join "immunizations" on "vaccines"."VaccineID" = "immunizations"."vaccine_id" where "immunizations"."patient_id" = "patients"."PatientID" and "vaccine_id" ILIKE %))
I have tried using cast like CAST(vaccine_id AS VARCHAR) and I' not getting the error but it doesnt return any result.
The problem is here:
$q->where('vaccine_id','ILIKE','%' . $vaccine_id)
looks like vaccine_id is integer, and you can not use operator ILIKE to integer. Try just '='
If you want to use LIKE, ILIKE or other text operator you must cast your data to text. In SQL it must looks like:
WHERE "vaccine_id"::text ILIKE val
instead
WHERE "vaccine_id" ILIKE val
You could do this:
$q->where('cast(vaccine_id AS VARCHAR)','LIKE','%' . $vaccine_id)
OR
$q->where('cast(vaccine_id AS TEXT)','LIKE','%' . $vaccine_id)

Escaping hstore contains operators in a JDBC Prepared statement

I am using PostgreSQL 9.1.4 with hstore and the PostgreSQL JDBC driver (9.1-901.jdbc4).
I am trying to use the contains operators (?, ?&, ?|) in a PreparedStatement, however the ? character is parsed as a variable placeholder. Can this character be escaped to send the correct operator in the query?
An example:
PreparedStatement stmt = conn.prepareStatement("SELECT a, b FROM table1 WHERE c ? 'foo' AND d = ?");
stmt.setInt(1, dValue);
stmt.executeQuery();
In this form the following example would raise an exception:
org.postgresql.util.PSQLException: No value specified for parameter 2.
Update:
After investigating the query parser in the pgjdbc driver this snippet seems to indicate that it is not possible to escape the ? character. The questions that remain are:
Is there anything in the JDBC spec which allows a ? to be escaped and be anything other than a parameter placeholder?
Is there any better work around for this issue than just using plain Statements with variables manually inserted into the query string?
Effectively, it looks like the java SQL parser is not hstore compliant.
But since the syntax c ? 'foo' is equivalent to exist(c, 'foo'), you can easily workaround this problem. Have a look at the following page to see what the verbose operators for hstore are.
Postgres hstore documentation
There is a discussion about this issue on pgsql-hackers mailing list: http://grokbase.com/t/postgresql/pgsql-hackers/1325c6ys9n/alias-hstores-to-so-that-it-works-with-jdbc
For now I like most this workaround which also supports indexes:
CREATE FUNCTION exist_inline (hstore, text) RETURNS bool AS $$ SELECT $1 ? $2; $$ LANGUAGE sql;
You can use this query to find the function backing an operator in PostgreSQL like this. In your example:
SELECT
oprname,
oprcode || '(' || format_type(oprleft, NULL::integer) || ', '
|| format_type(oprright, NULL::integer) || ')' AS function
FROM pg_operator
WHERE oprname LIKE '?%'
AND (SELECT oid FROM pg_type WHERE typname = 'hstore') IN (oprleft, oprright);
This produces:
|oprname|function |
|-------|--------------------------|
|? |exist(hstore, text) |
|?| |exists_any(hstore, text[])|
|?& |exists_all(hstore, text[])|
See also a related question about using JSON operators containing ?. Note that the function usage may not profit from the same indexing capability when using a GIN index on your HSTORE column.
If you'd like to add multiple key-value pairs using PreparedStatement then you can do:
PreparedStatement ps = c.prepareStatement(
"insert into xyz(id, data) values(?, hstore(?, ?))");
ps.setLong(1, 23456L);
ps.setArray(2, c.createArrayOf("text", new String[]{"name", "city"}));
ps.setArray(3, c.createArrayOf("text", new String[]{"Duke", "Valley"}));
This will insert: 23456, 'name=>Duke, city=>Valley'