Having a problem with PostgreSQL dialect in PyCharm. I have the below SELECT query:
"SELECT * FROM table WHERE ST_DWithin(table.geog_column,
ST_GeographyFromText(\'SRID=4326;POINT(%s %s)\'), %s)"
The query performs as expected in a query editor but Pycharm complains <expression> expected, got '%'. I have set the dialect detection to PostgreSQL.
I believe there is an issue with the parameter binding but not able to figure out what the issue is. Any help would be appreciated.
EDIT: I somehow missed the clear warnings on psycopg2 documentation about using python string interpolation and concatenation.
The right way of doing it is to use SQLAlchemy to construct raw SQL queries:
from sqlalchemy import text
sql = text("SELECT * FROM table WHERE ST_DWithin(table.geog_column,
ST_GeographyFromText(\'SRID=4326;POINT(:long :lat)\'), :distance)")
data = {'long': longitude, 'lat': latitude, 'distance': distance}
result = conn.execute(sql, data)
The below approach is WRONG and is susceptible to SQL injections. I have left it here for reference only.
I just found the mistake and for anyone else who is caffeine starved, you need to add the %s within single quotes. Elementary but can easily be missed.
"SELECT * FROM table WHERE ST_DWithin(table.geog_column,
ST_GeographyFromText(\'SRID=4326;POINT('%s' '%s')\'), *'%s'*)"
The quotes fixed the issue for me but I am not entirely sure if this is the right approach and hence leaving it here to get some input.
Related
I have a Python script that runs a pgSQL file through SQLAlchemy's connection.execute function. Here's the block of code in Python:
results = pg_conn.execute(sql_cmd, beg_date = datetime.date(2015,4,1), end_date = datetime.date(2015,4,30))
And here's one of the areas where the variable gets inputted in my SQL:
WHERE
( dv.date >= %(beg_date)s AND
dv.date <= %(end_date)s)
When I run this, I get a cryptic python error:
sqlalchemy.exc.ProgrammingError: (psycopg2.ProgrammingError) argument formats can't be mixed
…followed by a huge dump of the offending SQL query. I've run this exact code with the same variable convention before. Why isn't it working this time?
I encountered a similar issue as Nikhil. I have a query with LIKE clauses which worked until I modified it to include a bind variable, at which point I received the following error:
DatabaseError: Execution failed on sql '...': argument formats can't be mixed
The solution is not to give up on the LIKE clause. That would be pretty crazy if psycopg2 simply didn't permit LIKE clauses. Rather, we can escape the literal % with %%. For example, the following query:
SELECT *
FROM people
WHERE start_date > %(beg_date)s
AND name LIKE 'John%';
would need to be modified to:
SELECT *
FROM people
WHERE start_date > %(beg_date)s
AND name LIKE 'John%%';
More details in the pscopg2 docs: http://initd.org/psycopg/docs/usage.html#passing-parameters-to-sql-queries
As it turned out, I had used a SQL LIKE operator in the new SQL query, and the % operand was messing with Python's escaping capability. For instance:
dv.device LIKE 'iPhone%' or
dv.device LIKE '%Phone'
Another answer offered a way to un-escape and re-escape, which I felt would add unnecessary complexity to otherwise simple code. Instead, I used pgSQL's ability to handle regex to modify the SQL query itself. This changed the above portion of the query to:
dv.device ~ E'iPhone.*' or
dv.device ~ E'.*Phone$'
So for others: you may need to change your LIKE operators to regex '~' to get it to work. Just remember that it'll be WAY slower for large queries. (More info here.)
For me it's turn out I have % in sql comment
/* Any future change in the testing size will not require
a change here... even if we do a 100% test
*/
This works fine:
/* Any future change in the testing size will not require
a change here... even if we do a 100pct test
*/
Basically i'm trying to do a simple join. I'm a beginner in progress and even if i'm reading always the same things... my problem still unresolved ! :'(
I'm using unixodbc to communicate with my base and this is working like a charm when i'm using simple command like : SELECT * from PUB."Art"
I understood I have to do something who looks like that to join 2 tables :
FOR EACH PUB."Art" WHERE (PUB."Art".IdArt = 16969) ,
EACH PUB."ArtDet" WHERE (PUB."ArtDet".IdArt = PUB."Art".IdArt)
END
But this only return me [ISQL]ERROR: Could not SQLPrepare
I then try to simplify the thing with :
for each PUB."Art": display PUB."Art".IdArt end.
I try to put colon (or not) after the for each loop, using point / comma etc... but I never use the right syntax apparently... or I'm missing a thing to execute this command !
Is anyone can advice me ?
Thx a lot !
You appear to mixing SQL and 4GL syntax.
"FOR EACH" is 4GL. The SQL equivalent is "SELECT".
(If you are using 4GL you do not need then "PUB" prefix and quoting table and field names will not work.)
To do a join with SQL (or the 4GL) use a "," between the table names. For SQL your syntax would look something like:
SELECT * from PUB."Art", PUB."ArtDet"
Gory details regarding WHERE clauses, SQL INNER & OUTER joins etc. can be found in the online documentation:
https://community.progress.com/community_groups/openedge_general/w/openedgegeneral/1329.openedge-product-documentation-overview
You will want to navigate to your specific release and then find the "SQL" guide.
I get this error when trying out this command in the BIRT Classic Models sample database in Data Studio
select xmlelement(name "custno", customers.customernumber) from customers
Syntax error: Encountered "\"custno\"" at line 1, column 24.
I do not know how to correct it.
Thanks.
I'm not familiar with db2, but according to this your statement looks quite alrigth (although I'd place an alias to name this field...)
But this
Syntax error: Encountered "\"custno\"" at line 1, column 24.
seems to be a quite clear hint, that your error is connected to the NAME of the element.
I'm pretty sure, that this statement was created on string level.
Did you try to escape the "-characters with \"?
The SQL reaching the engine might look like
select xmlelement(name \"custno\", customers.customernumber) from customers
or
select xmlelement(name "\"custno"\", customers.customernumber) from customers
... which is wrong of course...
But to be honest: just guessing...
I am using PostgreSQL 9.4 and the awesome JSONB field type. I am trying to query against a field in a document. The following works in the psql CLI
SELECT id FROM program WHERE document -> 'dept' ? 'CS'
When I try to run the same query via my Scala app, I'm getting the error below. I'm using Play framework and Anorm, so the query looks like this
SQL(s"SELECT id FROM program WHERE document -> 'dept' ? {dept}")
.on('dept -> "CS")
....
SQLException: : No value specified for parameter 5.
(SimpleParameterList.java:223)
(in my actual queries there are more parameters)
I can get around this by casting my parameter to type jsonb and using the #> operator to check containment.
SQL(s"SELECT id FROM program WHERE document -> 'dept' #> {dept}::jsonb")
.on('dept -> "CS")
....
I'm not too keen on the work around. I don't know if there are performance penalties for the cast, but it's extra typing, and non-obvious.
Is there anything else I can do?
As a workaround to avoid the ? operator, you could create a new operator doing exactly the same.
This is the code of the original operator:
CREATE OPERATOR ?(
PROCEDURE = jsonb_exists,
LEFTARG = jsonb,
RIGHTARG = text,
RESTRICT = contsel,
JOIN = contjoinsel);
SELECT '{"a":1, "b":2}'::jsonb ? 'b'; -- true
Use a different name, without any conflicts, like #-# and create a new one:
CREATE OPERATOR #-#(
PROCEDURE = jsonb_exists,
LEFTARG = jsonb,
RIGHTARG = text,
RESTRICT = contsel,
JOIN = contjoinsel);
SELECT '{"a":1, "b":2}'::jsonb #-# 'b'; -- true
Use this new operator in your code and it should work.
Check pgAdmin -> pg_catalog -> Operators for all the operators that use a ? in the name.
In JDBC (and standard SQL) the question mark is reserved as a parameter placeholder. Other uses are not allowed.
See Does the JDBC spec prevent '?' from being used as an operator (outside of quotes)? and the discussion on jdbc-spec-discuss.
The current PostgreSQL JDBC driver will transform all occurrences (outside text or comments) of a question mark to a PostgreSQL specific parameter placeholder. I am not sure if the PostgreSQL JDBC project has done anything (like introducing an escape as discussed in the links above) to address this yet. A quick look at the code and documentation suggests they didn't, but I didn't dig too deep.
Addendum: As shown in the answer by bobmarksie, current versions of the PostgreSQL JDBC driver now support escaping the question mark by doubling it (ie: use ?? instead of ?).
I had the same issue a couple of days ago and after some investigation I found this.
https://jdbc.postgresql.org/documentation/head/statement.html
In JDBC, the question mark (?) is the placeholder for the positional parameters of a PreparedStatement. There are, however, a number of PostgreSQL operators that contain a question mark. To keep such question marks in a SQL statement from being interpreted as positional parameters, use two question marks (??) as escape sequence. You can also use this escape sequence in a Statement, but that is not required. Specifically only in a Statement a single (?) can be used as an operator.
Using 2 question marks seemed to work well for me - I was using the following driver (illustrated using maven dependency) ...
<dependency>
<groupId>org.postgresql</groupId>
<artifactId>postgresql</artifactId>
<version>9.4-1201-jdbc41</version>
</dependency>
... and MyBatis for creating the SQL queries and it seemed to work well. Seemed easier / cleaner than creating an PostgreSQL operator.
SQL went from e.g.
select * from user_docs where userTags ?| array['sport','property']
... to ...
select * from user_docs where userTags ??| array['sport','property']
Hopefully this works with your scenario!
As bob said just use ?? instead of ?
SQL(s"SELECT id FROM program WHERE document -> 'dept' ?? {dept}")
.on('dept -> "CS")
say I have a long URL
xyz = 'www.google.com/xyz?para1=value1¶2=value2¶3=value3....'
I am trying to get the 'para1' out of this long URL
So, I have
select TRIM(Leading '?' from Substring(xyz from '%#"?%=#"%' for '#'))
The answer I get for this particular statement is
para1=value1¶2=value2¶3=
How can I get just 'para1' using the select statement above (or any other similar method?)
I am using Greenplum (as mentioned in the topic heading)
Since you apparently have the regexp_ functions (I didn't think Greenplum supported them) use:
select (regexp_matches(
'www.google.com/xyz?para1=value1¶2=value2¶3=value3....',
'\?([^&]+)='
))[1];