I'm trying to escape the ?| operator in this query:
val data = sql"""
SELECT ......
FROM .......
WHERE table.column ?| array['23', '12']
""".as[Int].head
db.run(data)
however the ?| operator gets translated as $1| in the query (checked in the DB query log) and it obviously generates the error
ERROR: syntax error at or near "$1" at character 735
I've tried with #?| and $?| without success
? is a placeholder for a parameter in JDBC (that's the level after Slick). You can escape ? specifically for PostgreSQL as ??|. There's useful discussion of this in SO 14779896 - Does the JDBC spec prevent '?' from being used as an operator.
An alternative to this convention would be to use non-symbolic alternatives: jsonb_exists_any. E.g.,
WHERE jsonb_exists_any(table.column, array['23', '12'])
Related
I am trying to run this query:
select *
from my_table
where column_one=${myValue}
I get the following error in Datagrip:
[42883] ERROR: operator does not exist: character varying = bigint Hint: No operator matches the given name and argument types. You might need to add explicit type casts.
Now, I have found this question, and I can fix the error by putting a string like this:
select *
from my_table
where column_one='123'
What I need is a way to pass in the '123' as a parameter. I usually do this ${myValue} and it works, but I am not sure how to keep my variable there as an input so I can run dynamic queries in code and let Postgres understand I want to pass in a string and not a number.
Any suggestions?
Here's a screenshot of how I am putting the parameter value in DataGrip...:
Ok, so, I just tried to put quotes in the data grip parameters input field for myValue #thirumal's answer things work. I didn't know I have to quote the value for it to work.
This is what it looks like:
Type cast ${myValue} using SQL Standard,
cast(${myValue} AS varchar)
or using Postgres Syntax:
${myValue}::varchar
Why is there a need to have the word foo at the end of the query.
queryTwo = """(SELECT max(lst_updt_dt_tm) as maxDt FROM table) foo """
theTable = sqlContext.read.format("jdbc").options(url="the url", driver = "the td driver", dbtable = queryTwo, user="xxxx", password="xxxx").load()
theTable.show();
If I remove the word foo, the code fails.
Reverse engineering what is happening here from the error in your comment:
java.sql.SQLException: [Teradata Database] [TeraJDBC 15.10.00.33] [Error 3707] [SQLState 42000] Syntax error, expected something like a name or a Unicode delimited identifier or an 'UDFCALLNAME' keyword or '(' between the 'FROM' keyword and the 'SELECT' keyword.
It's pretty clear that your dbtable parameter requires a SUBQUERY as input. Whatever is happening in sqlContext.read.format("jdbc").options is rewriting the query as :
SELECT <something> FROM <dbtable> <possibly something more here>;
As such it requires that dbtable be a subquery which is why you must wrap it in parentheses and give it an alias foo.
If you read up on the documentation for SPARK SQL You will see:
dbtable: The JDBC table that should be read. Note that anything that is
valid in a FROM clause of a SQL query can be used. For example,
instead of a full table you could also use a subquery in parentheses.
So you could give it a table, or, optionally, you can give it a subquery. Since it doesn't specify that the subqueries alias needs to be foo you could really write any legal alias name here like mysubquery or "this is a poor choice for a subquery alias" or "f" if you feel so inclined..
Below statmen generated from h2
INSERT INTO EPSG_ALIAS
(ALIAS_CODE, OBJECT_TABLE_NAME, OBJECT_CODE, NAMING_SYSTEM_CODE, ALIAS, REMARKS)
VALUES
(1431, 'Datum', 6123, 7300, STRINGDECODE('Kartastokoordinaattij\ufffdrjestelm\ufffd (1966)'), NULL);
When i execute the above statment in postgresql i am getting following error.
ERROR: function stringdecode(unknown) does not exist
SQL state: 42883
Hint: No function matches the given name and argument types. You might need to add explicit type casts.
Character: 140
STRINGDECODE is the way of H2 to read 'ASCII written UTF-8 characters'.
For Postgres, it's a simple 'E' before your string:
STRINGDECODE('Kartastokoordinaattij\ufffdrjestelm\ufffd (1966)')
Becomes
E'Kartastokoordinaattij\ufffdrjestelm\ufffd (1966)'
I am using PostgreSQL 9.4 and the awesome JSONB field type. I am trying to query against a field in a document. The following works in the psql CLI
SELECT id FROM program WHERE document -> 'dept' ? 'CS'
When I try to run the same query via my Scala app, I'm getting the error below. I'm using Play framework and Anorm, so the query looks like this
SQL(s"SELECT id FROM program WHERE document -> 'dept' ? {dept}")
.on('dept -> "CS")
....
SQLException: : No value specified for parameter 5.
(SimpleParameterList.java:223)
(in my actual queries there are more parameters)
I can get around this by casting my parameter to type jsonb and using the #> operator to check containment.
SQL(s"SELECT id FROM program WHERE document -> 'dept' #> {dept}::jsonb")
.on('dept -> "CS")
....
I'm not too keen on the work around. I don't know if there are performance penalties for the cast, but it's extra typing, and non-obvious.
Is there anything else I can do?
As a workaround to avoid the ? operator, you could create a new operator doing exactly the same.
This is the code of the original operator:
CREATE OPERATOR ?(
PROCEDURE = jsonb_exists,
LEFTARG = jsonb,
RIGHTARG = text,
RESTRICT = contsel,
JOIN = contjoinsel);
SELECT '{"a":1, "b":2}'::jsonb ? 'b'; -- true
Use a different name, without any conflicts, like #-# and create a new one:
CREATE OPERATOR #-#(
PROCEDURE = jsonb_exists,
LEFTARG = jsonb,
RIGHTARG = text,
RESTRICT = contsel,
JOIN = contjoinsel);
SELECT '{"a":1, "b":2}'::jsonb #-# 'b'; -- true
Use this new operator in your code and it should work.
Check pgAdmin -> pg_catalog -> Operators for all the operators that use a ? in the name.
In JDBC (and standard SQL) the question mark is reserved as a parameter placeholder. Other uses are not allowed.
See Does the JDBC spec prevent '?' from being used as an operator (outside of quotes)? and the discussion on jdbc-spec-discuss.
The current PostgreSQL JDBC driver will transform all occurrences (outside text or comments) of a question mark to a PostgreSQL specific parameter placeholder. I am not sure if the PostgreSQL JDBC project has done anything (like introducing an escape as discussed in the links above) to address this yet. A quick look at the code and documentation suggests they didn't, but I didn't dig too deep.
Addendum: As shown in the answer by bobmarksie, current versions of the PostgreSQL JDBC driver now support escaping the question mark by doubling it (ie: use ?? instead of ?).
I had the same issue a couple of days ago and after some investigation I found this.
https://jdbc.postgresql.org/documentation/head/statement.html
In JDBC, the question mark (?) is the placeholder for the positional parameters of a PreparedStatement. There are, however, a number of PostgreSQL operators that contain a question mark. To keep such question marks in a SQL statement from being interpreted as positional parameters, use two question marks (??) as escape sequence. You can also use this escape sequence in a Statement, but that is not required. Specifically only in a Statement a single (?) can be used as an operator.
Using 2 question marks seemed to work well for me - I was using the following driver (illustrated using maven dependency) ...
<dependency>
<groupId>org.postgresql</groupId>
<artifactId>postgresql</artifactId>
<version>9.4-1201-jdbc41</version>
</dependency>
... and MyBatis for creating the SQL queries and it seemed to work well. Seemed easier / cleaner than creating an PostgreSQL operator.
SQL went from e.g.
select * from user_docs where userTags ?| array['sport','property']
... to ...
select * from user_docs where userTags ??| array['sport','property']
Hopefully this works with your scenario!
As bob said just use ?? instead of ?
SQL(s"SELECT id FROM program WHERE document -> 'dept' ?? {dept}")
.on('dept -> "CS")
My database has implicit cast from integer to text, so I'm getting some 'operator is not unique' issues. I'm trying to delete the operator || (text, anynonarray) with no success, the error message is
ERROR: cannot drop operator ||(text,anynonarray) because it is required by the database system
SQL state: 2BP01
I have another database that doesn't have this operator, so I think is possible somehow. I cannot add explicit casts in hundreds of queries, neither remove the implicit casts. There's a way to force the remotion of this operator?
Seems I got it. The trick is to delete right from the pg_operator table.
delete from pg_operator where oprname = '||' and (oprleft = 25 or oprleft = 2776 ) and oprleft != oprright
Because I wanted to delete these operators:
OPERATOR ||("text", anynonarray);
OPERATOR ||(anynonarray, "text");
I don't know if there are any implications though, seems ok till now.