PostgreSQL jsonb, `?` and JDBC - postgresql

I am using PostgreSQL 9.4 and the awesome JSONB field type. I am trying to query against a field in a document. The following works in the psql CLI
SELECT id FROM program WHERE document -> 'dept' ? 'CS'
When I try to run the same query via my Scala app, I'm getting the error below. I'm using Play framework and Anorm, so the query looks like this
SQL(s"SELECT id FROM program WHERE document -> 'dept' ? {dept}")
.on('dept -> "CS")
....
SQLException: : No value specified for parameter 5.
(SimpleParameterList.java:223)
(in my actual queries there are more parameters)
I can get around this by casting my parameter to type jsonb and using the #> operator to check containment.
SQL(s"SELECT id FROM program WHERE document -> 'dept' #> {dept}::jsonb")
.on('dept -> "CS")
....
I'm not too keen on the work around. I don't know if there are performance penalties for the cast, but it's extra typing, and non-obvious.
Is there anything else I can do?

As a workaround to avoid the ? operator, you could create a new operator doing exactly the same.
This is the code of the original operator:
CREATE OPERATOR ?(
PROCEDURE = jsonb_exists,
LEFTARG = jsonb,
RIGHTARG = text,
RESTRICT = contsel,
JOIN = contjoinsel);
SELECT '{"a":1, "b":2}'::jsonb ? 'b'; -- true
Use a different name, without any conflicts, like #-# and create a new one:
CREATE OPERATOR #-#(
PROCEDURE = jsonb_exists,
LEFTARG = jsonb,
RIGHTARG = text,
RESTRICT = contsel,
JOIN = contjoinsel);
SELECT '{"a":1, "b":2}'::jsonb #-# 'b'; -- true
Use this new operator in your code and it should work.
Check pgAdmin -> pg_catalog -> Operators for all the operators that use a ? in the name.

In JDBC (and standard SQL) the question mark is reserved as a parameter placeholder. Other uses are not allowed.
See Does the JDBC spec prevent '?' from being used as an operator (outside of quotes)? and the discussion on jdbc-spec-discuss.
The current PostgreSQL JDBC driver will transform all occurrences (outside text or comments) of a question mark to a PostgreSQL specific parameter placeholder. I am not sure if the PostgreSQL JDBC project has done anything (like introducing an escape as discussed in the links above) to address this yet. A quick look at the code and documentation suggests they didn't, but I didn't dig too deep.
Addendum: As shown in the answer by bobmarksie, current versions of the PostgreSQL JDBC driver now support escaping the question mark by doubling it (ie: use ?? instead of ?).

I had the same issue a couple of days ago and after some investigation I found this.
https://jdbc.postgresql.org/documentation/head/statement.html
In JDBC, the question mark (?) is the placeholder for the positional parameters of a PreparedStatement. There are, however, a number of PostgreSQL operators that contain a question mark. To keep such question marks in a SQL statement from being interpreted as positional parameters, use two question marks (??) as escape sequence. You can also use this escape sequence in a Statement, but that is not required. Specifically only in a Statement a single (?) can be used as an operator.
Using 2 question marks seemed to work well for me - I was using the following driver (illustrated using maven dependency) ...
<dependency>
<groupId>org.postgresql</groupId>
<artifactId>postgresql</artifactId>
<version>9.4-1201-jdbc41</version>
</dependency>
... and MyBatis for creating the SQL queries and it seemed to work well. Seemed easier / cleaner than creating an PostgreSQL operator.
SQL went from e.g.
select * from user_docs where userTags ?| array['sport','property']
... to ...
select * from user_docs where userTags ??| array['sport','property']
Hopefully this works with your scenario!

As bob said just use ?? instead of ?
SQL(s"SELECT id FROM program WHERE document -> 'dept' ?? {dept}")
.on('dept -> "CS")

Related

How to use Postgresql ts_delete function

I am trying to use Postgresql Full Text Search. I read that the stop words (words ignored for indexing) are implemented via dictionary. But I would like to give the user a limited control over the stop words (insert new ones), so I grouped then in a table.
From the example below:
select strip(to_tsvector('simple', texto)) from longtxts where id = 23;
I can get the vector:
{'alta' 'aluno' 'cada' 'do' 'em' 'leia' 'livro' 'pedir' 'que' 'trecho' 'um' 'voz'}
And now I would like to remove the elements from the stopwords table:
select array(select palavra_proibida from stopwords);
That returns the array:
{a,as,ao,aos,com,default,e,eu,o,os,da,das,de,do,dos,em,lhe,na,nao,nas,no,nos,ou,por,para,pra,que,sem,se,um,uma}
Then, following documentation:
ts_delete(vector tsvector, lexemes text[]) tsvector remove any occurrence of lexemes in lexemes from vector ts_delete('fat:2,4 cat:3 rat:5A'::tsvector, ARRAY['fat','rat'])
I tried a lot. For example:
select ts_delete((select strip(to_tsvector('simple', texto)) from longtxts where id = 23), array[(select palavra_proibida from stopwords)]);
But I always receive the error:
ERROR: function ts_delete(tsvector, character varying[]) does not exist
LINE 1: select ts_delete((select strip(to_tsvector('simple', texto))...
^
HINT: No function matches the given name and argument types. You might need to add explicit type casts.
Could anyone help me? Thanks in advance!
ts_delete was introduced in PostgreSQL 9.6. Based on the error message, you're using an earlier version. You may try select version(); to be sure.
When you land on the PostgreSQL online documentation with a web search, it may correspond to any version. The version is in the URL and there's a "This page in another version" set of links at the top of each page to help switching to the equivalent doc for a different version.

How to cast a JSON value to its correspondent datatype?

JSON-string is SQL-text, JSON-number is SQL-numeric, JSON-boolean is SQL-boolean... So I can do "real good cast"... But:
SELECT to_jsonb('hello'::text)::text
is not good. Returning with quotes. Worst case:
SELECT to_jsonb(1::int)::int;
-- ERROR: cannot cast type jsonb to integer
So, how to do casting?
  ((this is a Wiki, you can edit!))
(update for) Modern PostgreSQL do direct cast!
Example using pg 13.5:
SELECT (('{"x":123}'::jsonb)->'x')::int; -- ok no error!
SELECT (('{"x":123.44}'::jsonb)->'x')::float; -- ok no error!
SELECT (('{"x":123.44}'::jsonb)->'x')::numeric; -- ok no error!
SELECT (('{"x":"hello !"}'::jsonb)->'x')::text; -- no error, but quotes
SELECT ('{"x":"hello !"}'::jsonb)->>'x'; -- ok!
-- Modern versions not need cast for simple operations:
SELECT (('{"x":123}'::jsonb)->'x')::int + 3; -- Ok, but...
SELECT jsonb_path_query('{"x":123}', '$.x + 3'); -- How about this!
OLD answers for pg 9.4
Ugly ways only...
Today PostgreSQL is not so serious about JSON... So, let's workaround.
Sorry there are questions (mine!) and answers. The answer with the most simple solution is this one:
In pg v9.4.4+ using the #>> operator works for me: select to_json('test'::text)#>>'{}';
So, using the question's example:
SELECT (to_json(1::int)#>>'{}')::int; -- the cast works
JSONb question... Internal binary value was reused?
SELECT (to_jsonB(10.7::numeric)#>>'{}')::numeric; -- internal repres. preserved?
SELECT (to_jsonB(10.7::float)#>>'{}')::float; -- internal repres. preserved?
-- internal JSONb was float or numeric?
SELECT (to_jsonB(true)#>>'{}')::boolean; -- internal representation preserved?
... there are no guarantee that PostgreSQL's internal parser is doing the correct thing, reusing internal numerical representation of JSONb to avoid CPU-time consuption (converting jsonb-number to SQL-text, them casting text to SQL-number).
"JSONb cast optimization" or "SQL datatypes/JSONb datatypes convertion optionmization" seems to still be a big gap for PostgreSQL.
Here's a simpler solution that I've found for PostgreSQL.
Assuming:
{
"additionaldata" :
{
"Duration" : "123.44"
}
}
The following select works:
select
(additionaldata -> 'Duration')::jsonb::text::numeric as durationSeconds
from ...

How to insert similar value into multiple locations of a psycopg2 query statement using dict? [duplicate]

I have a Python script that runs a pgSQL file through SQLAlchemy's connection.execute function. Here's the block of code in Python:
results = pg_conn.execute(sql_cmd, beg_date = datetime.date(2015,4,1), end_date = datetime.date(2015,4,30))
And here's one of the areas where the variable gets inputted in my SQL:
WHERE
( dv.date >= %(beg_date)s AND
dv.date <= %(end_date)s)
When I run this, I get a cryptic python error:
sqlalchemy.exc.ProgrammingError: (psycopg2.ProgrammingError) argument formats can't be mixed
…followed by a huge dump of the offending SQL query. I've run this exact code with the same variable convention before. Why isn't it working this time?
I encountered a similar issue as Nikhil. I have a query with LIKE clauses which worked until I modified it to include a bind variable, at which point I received the following error:
DatabaseError: Execution failed on sql '...': argument formats can't be mixed
The solution is not to give up on the LIKE clause. That would be pretty crazy if psycopg2 simply didn't permit LIKE clauses. Rather, we can escape the literal % with %%. For example, the following query:
SELECT *
FROM people
WHERE start_date > %(beg_date)s
AND name LIKE 'John%';
would need to be modified to:
SELECT *
FROM people
WHERE start_date > %(beg_date)s
AND name LIKE 'John%%';
More details in the pscopg2 docs: http://initd.org/psycopg/docs/usage.html#passing-parameters-to-sql-queries
As it turned out, I had used a SQL LIKE operator in the new SQL query, and the % operand was messing with Python's escaping capability. For instance:
dv.device LIKE 'iPhone%' or
dv.device LIKE '%Phone'
Another answer offered a way to un-escape and re-escape, which I felt would add unnecessary complexity to otherwise simple code. Instead, I used pgSQL's ability to handle regex to modify the SQL query itself. This changed the above portion of the query to:
dv.device ~ E'iPhone.*' or
dv.device ~ E'.*Phone$'
So for others: you may need to change your LIKE operators to regex '~' to get it to work. Just remember that it'll be WAY slower for large queries. (More info here.)
For me it's turn out I have % in sql comment
/* Any future change in the testing size will not require
a change here... even if we do a 100% test
*/
This works fine:
/* Any future change in the testing size will not require
a change here... even if we do a 100pct test
*/

how to use for each loop in Progress?

Basically i'm trying to do a simple join. I'm a beginner in progress and even if i'm reading always the same things... my problem still unresolved ! :'(
I'm using unixodbc to communicate with my base and this is working like a charm when i'm using simple command like : SELECT * from PUB."Art"
I understood I have to do something who looks like that to join 2 tables :
FOR EACH PUB."Art" WHERE (PUB."Art".IdArt = 16969) ,
EACH PUB."ArtDet" WHERE (PUB."ArtDet".IdArt = PUB."Art".IdArt)
END
But this only return me [ISQL]ERROR: Could not SQLPrepare
I then try to simplify the thing with :
for each PUB."Art": display PUB."Art".IdArt end.
I try to put colon (or not) after the for each loop, using point / comma etc... but I never use the right syntax apparently... or I'm missing a thing to execute this command !
Is anyone can advice me ?
Thx a lot !
You appear to mixing SQL and 4GL syntax.
"FOR EACH" is 4GL. The SQL equivalent is "SELECT".
(If you are using 4GL you do not need then "PUB" prefix and quoting table and field names will not work.)
To do a join with SQL (or the 4GL) use a "," between the table names. For SQL your syntax would look something like:
SELECT * from PUB."Art", PUB."ArtDet"
Gory details regarding WHERE clauses, SQL INNER & OUTER joins etc. can be found in the online documentation:
https://community.progress.com/community_groups/openedge_general/w/openedgegeneral/1329.openedge-product-documentation-overview
You will want to navigate to your specific release and then find the "SQL" guide.

GreenPlum Substring - Getting part of a long text

say I have a long URL
xyz = 'www.google.com/xyz?para1=value1&para2=value2&para3=value3....'
I am trying to get the 'para1' out of this long URL
So, I have
select TRIM(Leading '?' from Substring(xyz from '%#"?%=#"%' for '#'))
The answer I get for this particular statement is
para1=value1&para2=value2&para3=
How can I get just 'para1' using the select statement above (or any other similar method?)
I am using Greenplum (as mentioned in the topic heading)
Since you apparently have the regexp_ functions (I didn't think Greenplum supported them) use:
select (regexp_matches(
'www.google.com/xyz?para1=value1&para2=value2&para3=value3....',
'\?([^&]+)='
))[1];