Approaches to execute PostgreSQL's concat() instead of || in JOOQ? - postgresql

The ||-operator and the concat(...)-function in PostgreSQL behave differently.
select 'ABC'||NULL||'def';
-- Result: NULL
select concat('ABC', NULL, 'def');
-- Result: 'ABCdef'
concat(...) ignores NULL values, but a NULL whithin a || expression makes the whole result become NULL.
In JOOQ, the DSL.concat() in the PostgreSQL dialect renders expressions using the ||-operator:
Java: dsl.select(
DSL.concat(
DSL.inline("ABC"),
DSL.inline(null, SQLDataType.VARCHAR),
DSL.inline("def"))
).execute();
SQL: select ('ABC' || null || 'def')
Result: NULL
I am looking for (elegant?) ways to invoke the concat(...)-function instead of the ||-operator via JOOQ in PostgreSQL:
Java: dsl.select(???).execute();
SQL: select concat('ABC', null, 'def')
Result: 'ABCdef'

I found two ways to achieve the posed objective.
Approach #1:
dsl.select(
field(
"concat({0})",
SQLDataType.VARCHAR,
list(
inline("ABC"),
inline(null, SQLDataType.VARCHAR),
inline("def")
)
)
).execute();
This has to the intended behavior, but necessitates the in my eyes ugly "concat({0})". A more elegant approach from my point of view is:
Approach #2:
dsl.select(
function(
"concat",
SQLDataType.VARCHAR,
inline("ABC"),
inline(null, SQLDataType.VARCHAR),
inline("def")
)
).execute();
This solution does not involve inline SQL with placeholders as approach #1. Why JOOQ generates || instead of concat(...) in the first place is still to be expounded, though.

Related

DB2 SQL Error: SQLCODE=-302 while executing prepared statement

I have a SQL query which takes user inputs hence security flaw is present.
The existing query is:
SELECT BUS_NM, STR_ADDR_1, CITY_NM, STATE_CD, POSTAL_CD, COUNTRY_CD,
BUS_PHONE_NB,PEG_ACCOUNT_ID, GDN_ALERT_ID, GBIN, GDN_MON_REF_NB,
ALERT_DT, ALERT_TYPE, ALERT_DESC,ALERT_PRIORITY
FROM ( SELECT A.BUS_NM, AE.STR_ADDR_1, A.CITY_NM, A.STATE_CD, A.POSTAL_CD,
CC.COUNTRY_CD, A.BUS_PHONE_NB, A.PEG_ACCOUNT_ID, 'I' ||
LPAD(INTL_ALERT_DTL_ID, 9,'0') GDN_ALERT_ID,
LPAD(IA.GBIN, 9,'0') GBIN, IA.GDN_MON_REF_NB,
DATE(IAD.ALERT_TS) ALERT_DT,
XMLCAST(XMLQUERY('$A/alertTypeConfig/biqCode/text()' passing
IAC.INTL_ALERT_TYPE_CONFIG as "A") AS CHAR(4)) ALERT_TYPE,
, ROW_NUMBER() OVER () AS "RN"
FROM ACCOUNT A, Other tables
WHERE IA.GDN_MON_REF_NB = '100'
AND A.PEG_ACCOUNT_ID = IAAR.PEG_ACCOUNT_ID
AND CC.COUNTRY_CD = A.COUNTRY_ISO3_CD
ORDER BY IA.INTL_ALERT_ID ASC )
WHERE ALERT_TYPE IN (" +TriggerType+ ");
I changed it to accept TriggerType from setString like:
SELECT BUS_NM, STR_ADDR_1, CITY_NM, STATE_CD, POSTAL_CD, COUNTRY_CD,
BUS_PHONE_NB,PEG_ACCOUNT_ID, GDN_ALERT_ID, GBIN, GDN_MON_REF_NB,
ALERT_DT, ALERT_TYPE, ALERT_DESC,ALERT_PRIORITY
FROM ( SELECT A.BUS_NM, AE.STR_ADDR_1, A.CITY_NM, A.STATE_CD, A.POSTAL_CD,
CC.COUNTRY_CD, A.BUS_PHONE_NB, A.PEG_ACCOUNT_ID,
'I' || LPAD(INTL_ALERT_DTL_ID, 9,'0') GDN_ALERT_ID,
LPAD(IA.GBIN, 9,'0') GBIN, IA.GDN_MON_REF_NB,
DATE(IAD.ALERT_TS) ALERT_DT,
XMLCAST(XMLQUERY('$A/alertTypeConfig/biqCode/text()' passing
IAC.INTL_ALERT_TYPE_CONFIG as "A") AS CHAR(4)) ALERT_TYPE,
ROW_NUMBER() OVER () AS "RN"
FROM ACCOUNT A, other tables
WHERE IA.GDN_MON_REF_NB = '100'
AND A.PEG_ACCOUNT_ID = IAAR.PEG_ACCOUNT_ID
AND CC.COUNTRY_CD = A.COUNTRY_ISO3_CD
ORDER BY IA.INTL_ALERT_ID ASC )
WHERE ALERT_TYPE IN (?);
Setting trigger type as below:
if (StringUtils.isNotBlank(request.getTriggerType())) {
preparedStatement.setString(1, triggerType != null ? triggerType.toString() : "");
}
Getting error as
Caused by: com.ibm.db2.jcc.am.SqlDataException: DB2 SQL Error: SQLCODE=-302, SQLSTATE=22001, SQLERRMC=null, DRIVER=4.19.26
The -302 SQLCODE indicates a conversion error of some sort.
SQLSTATE 22001 narrows that down a bit by telling us that you are trying to force a big string into a small variable. Given the limited information in your question, I am guessing it is the XMLCAST that is the culprit.
DB2 won't jam 30 pounds of crap into a 4 pound bag so to speak, it gives you an error. Maybe giving XML some extra room in the cast might be a help. If you need to make sure it ends up being only 4 characters long, you could explicitly do a LEFT(XMLCAST( ... AS VARCHAR(64)), 4). That way the XMLCAST has the space it needs, but you cut it back to fit your variable on the fetch.
The other thing could be that the variable being passed to the parameter marker is too long. DB2 will guess the type and length based on the length of ALERT_TYPE. Note that you can only pass a single value through a parameter marker. If you pass a comma separated list, it will not behave as expected (unless you expect ALERT_TYPE to also contain a comma separated list). If you are getting the comma separated list from a table, you can use a sub-select instead.
Wrong IN predicate use with a parameter.
Do not expect that IN ('AAAA, M250, ABCD') (as you try to do passing a comma-separated string as a single parameter) works as IN ('AAAA', 'M250', 'ABCD') (as you need). These predicates are not equivalent.
You need some "string tokenizer", if you want to pass such a comma-separated string like below.
select t.*
from
(
select XMLCAST(XMLQUERY('$A/alertTypeConfig/biqCode/text()' passing IAC.INTL_ALERT_TYPE_CONFIG as "A") AS CHAR(4)) ALERT_TYPE
from table(values xmlparse(document '<alertTypeConfig><biqCode>M250, really big code</biqCode></alertTypeConfig>')) IAC(INTL_ALERT_TYPE_CONFIG)
) t
--WHERE ALERT_TYPE IN ('AAAA, M250, ABCD')
join xmltable('for $id in tokenize($s, ",\s?") return <i>{string($id)}</i>'
passing cast('AAA, M250 , ABCD' as varchar(200)) as "s"
columns token varchar(200) path '.') x on x.token=t.ALERT_TYPE
;
Run the statement as is. Then you may uncomment the string with WHERE clause and comment out the rest to see what you try to do.
P.S.:
The error you get is probably because you don't specify the data type of the parameter (you don't use something like IN (cast(? as varchar(xxx))), and db2 compiler assumes that its length is equal to the length of the ALERT_TYPE expression (4 bytes).

Postgres SQL - different results from LIKE query using OR vs ||

I have a table with an integer column. It has 12 records numbered 1000 to 1012. Remember, these are ints.
This query returns, as expected, 12 results:
select count(*) from proposals where qd_number::text like '%10%'
as does this:
SELECT COUNT(*) FROM "proposals" WHERE (lower(first_name) LIKE '%10%' OR qd_number::text LIKE '%10%' )
but this query returns 2 records:
SELECT COUNT(*) FROM "proposals" WHERE (lower(first_name) || ' ' || qd_number::text LIKE '%10%' )
which implies using || in concatenated where expressions is not equivalent to using OR. Is that correct or am I missing something else here?
You probably have nulls in first_name. For these records (lower(first_name) || ' ' || qd_number::text results in null, so you don't find the numbers any longer.
using || in concatenated where expressions is not equivalent to using ORIs that correct or am I missing something else here?
That is correct.
|| is the string concatenation operator in SQL, not the OR operator.

PostgreSQL ERROR: invalid input syntax for integer: "1e+06"

The full error message is:
ERROR: invalid input syntax for integer: "1e+06"
SQL state: 22P02
Context: In PL/R function sample
The query I'm using is:
WITH a as
(
SELECT a.tract_id_alias,
array_agg(a.pgid ORDER BY a.pgid) as pgids,
array_agg(a.sample_weight_geo ORDER BY a.pgid) as block_weights
FROM results_20161109.block_microdata_res_joined a
WHERE a.tract_id_alias in (66772, 66773, 66785, 66802, 66805, 66806, 66813)
AND a.bldg_count_res > 0
GROUP BY a.tract_id_alias
)
SELECT NULL::INTEGER agent_id,
a.tract_id_alias,
b.year,
unnest(shared.sample(a.pgids,
b.n_agents,
1 * b.year,
True,
a.block_weights)
) as pgid
FROM a
LEFT JOIN results_20161109.initial_agent_count_by_tract_res_11 b
ON a.tract_id_alias = b.tract_id_alias
ORDER BY b.year, a.tract_id_alias, pgid;
And the shared.sample function I'm using is:
CREATE OR REPLACE FUNCTION shared.sample(ids bigint[], size integer, seed integer DEFAULT 1, with_replacement boolean DEFAULT false, probabilities numeric[] DEFAULT NULL::numeric[])
RETURNS integer[] AS
$BODY$
set.seed(seed)
if (length(ids) == 1) {
s = rep(ids,size)
} else {
s = sample(ids,size, with_replacement,probabilities)
}
return(s)
$BODY$
LANGUAGE plr VOLATILE
COST 100;
ALTER FUNCTION shared.sample(bigint[], integer, integer, boolean, numeric[])
OWNER TO "server-superusers";
I'm pretty new to this stuff, so any help would be appreciated.
Not a problem of the function. Like the error messages says: The string '1e+06' cannot be cast to integer.
Obviously, the columns n_agents in your table results_20161109.initial_agent_count_by_tract_res_11 is not an integer column. Probably type text or varchar? (That info would help in your question.)
Either way, the assignment cast does not work for the target type integer. But it does for numeric:
Does not work:
SELECT '1e+06'::text::int; -- error as in question
Works:
SELECT '1e+06'::text::numeric::int;
If my assumptions hold, you can use this as stepping stone.
Replace b.n_agents in your query with b.n_agents::numeric::int.
It's your responsibility that numbers stay in integer range, or you get the next exception.
If that did not nail it, you need to look into function overloading:
Is there a way to disable function overloading in Postgres
And function type resolution:
PostgreSQL function call
The schema search path is relevant in many related cases, but you did schema-qualify all objects, so we can rule that out.
How does the search_path influence identifier resolution and the "current schema"
Your query generally looks good. I had a look and only found minor improvements:
SELECT NULL::int AS agent_id -- never omit the AS keyword for column alias
, a.tract_id_alias
, b.year
, s.pgid
FROM (
SELECT tract_id_alias
, array_agg(pgid) AS pgids
, array_agg(sample_weight_geo) AS block_weights
FROM ( -- use a subquery, cheaper than CTE
SELECT tract_id_alias
, pgid
, sample_weight_geo
FROM results_20161109.block_microdata_res_joined
WHERE tract_id_alias IN (66772, 66773, 66785, 66802, 66805, 66806, 66813)
AND bldg_count_res > 0
ORDER BY pgid -- sort once in a subquery. cheaper.
) sub
GROUP BY 1
) a
LEFT JOIN results_20161109.initial_agent_count_by_tract_res_11 b USING (tract_id_alias)
LEFT JOIN LATERAL
unnest(shared.sample(a.pgids
, b.n_agents
, b.year -- why "1 * b.year"?
, true
, a.block_weights)) s(pgid) ON true
ORDER BY b.year, a.tract_id_alias, s.pgid;

Escaping hstore contains operators in a JDBC Prepared statement

I am using PostgreSQL 9.1.4 with hstore and the PostgreSQL JDBC driver (9.1-901.jdbc4).
I am trying to use the contains operators (?, ?&, ?|) in a PreparedStatement, however the ? character is parsed as a variable placeholder. Can this character be escaped to send the correct operator in the query?
An example:
PreparedStatement stmt = conn.prepareStatement("SELECT a, b FROM table1 WHERE c ? 'foo' AND d = ?");
stmt.setInt(1, dValue);
stmt.executeQuery();
In this form the following example would raise an exception:
org.postgresql.util.PSQLException: No value specified for parameter 2.
Update:
After investigating the query parser in the pgjdbc driver this snippet seems to indicate that it is not possible to escape the ? character. The questions that remain are:
Is there anything in the JDBC spec which allows a ? to be escaped and be anything other than a parameter placeholder?
Is there any better work around for this issue than just using plain Statements with variables manually inserted into the query string?
Effectively, it looks like the java SQL parser is not hstore compliant.
But since the syntax c ? 'foo' is equivalent to exist(c, 'foo'), you can easily workaround this problem. Have a look at the following page to see what the verbose operators for hstore are.
Postgres hstore documentation
There is a discussion about this issue on pgsql-hackers mailing list: http://grokbase.com/t/postgresql/pgsql-hackers/1325c6ys9n/alias-hstores-to-so-that-it-works-with-jdbc
For now I like most this workaround which also supports indexes:
CREATE FUNCTION exist_inline (hstore, text) RETURNS bool AS $$ SELECT $1 ? $2; $$ LANGUAGE sql;
You can use this query to find the function backing an operator in PostgreSQL like this. In your example:
SELECT
oprname,
oprcode || '(' || format_type(oprleft, NULL::integer) || ', '
|| format_type(oprright, NULL::integer) || ')' AS function
FROM pg_operator
WHERE oprname LIKE '?%'
AND (SELECT oid FROM pg_type WHERE typname = 'hstore') IN (oprleft, oprright);
This produces:
|oprname|function |
|-------|--------------------------|
|? |exist(hstore, text) |
|?| |exists_any(hstore, text[])|
|?& |exists_all(hstore, text[])|
See also a related question about using JSON operators containing ?. Note that the function usage may not profit from the same indexing capability when using a GIN index on your HSTORE column.
If you'd like to add multiple key-value pairs using PreparedStatement then you can do:
PreparedStatement ps = c.prepareStatement(
"insert into xyz(id, data) values(?, hstore(?, ?))");
ps.setLong(1, 23456L);
ps.setArray(2, c.createArrayOf("text", new String[]{"name", "city"}));
ps.setArray(3, c.createArrayOf("text", new String[]{"Duke", "Valley"}));
This will insert: 23456, 'name=>Duke, city=>Valley'

T-SQL where clause case statement

Need some small help with some SQL. I am using a VARCHAR value to determine in a case which logic to use in WHERE clause but I am having some issues writing this case statement.
where
(CASE WHEN #p_flag = 'ATA'
THEN (#p_start_ata IS NULL AND #p_end_ata IS NULL) OR (vso.poa_ata between #p_start_ata and #p_end_ata)
ELSE (#p_start_atd IS NULL AND #p_end_atd IS NULL) OR (vso.pol_atd between #p_start_atd and #p_end_atd)
)
Line 93 Incorrect syntax near the keyword 'IS'.
Its probably a minor error but its frustrating me. Maybe there is a better way of writing this? Thanks!
Like said before you can not use a case statement like you are doing it. So this might be a suggestion:
WHERE
(
#p_flag = 'ATA'
AND
(
(#p_start_ata IS NULL AND #p_end_ata)
OR (vso.poa_ata between #p_start_ata and #p_end_ata)
)
)
OR
(
NOT #p_flag = 'ATA'
AND
(
(#p_start_atd IS NULL AND #p_end_atd IS NULL)
OR (vso.pol_atd between #p_start_atd and #p_end_atd)
)
)
You can't use the CASE like that!
It's use to get values and not restrictions. Check this to view how to properly use CASE.
In your case you've to change your logic probably using IF's and making the wanted SELECT's.