I have this function where I take a string and split it into an array of words. It works great in Postgres but I want to convert it to SQLAlchemy and I haven't been able to find a good alternative to STRING_TO_ARRAY. Are there any good work arounds that people have found?
Here is my basic query for reference:
SELECT type,
UNNEST(STRING_TO_ARRAY(description, ' ')) AS word
FROM item
From SQLAlchemy's documentation on SQL and Generic Functions, you should be able to use the function by name directly even if SQLAlchemy doesn't know about it.
Note that any name not known to func generates the function name as is - there is no restriction on what SQL functions can be called, known or unknown to SQLAlchemy, built-in or user defined.
from sqlalchemy import select, func, Table, Column, MetaData, String
metadata_obj = MetaData()
item = Table("item", metadata_obj, Column("description", String))
stmt = select(
func.unnest(
func.string_to_array(item.c.description, " ")
).label("word")
)
print(stmt.compile(compile_kwargs={"literal_binds": True}))
# SELECT unnest(string_to_array(item.description, ' ')) AS word
# FROM item
This remains a PostgreSQL specific query, but I don't see how to make it agnostic with both UNNEST and STRING_TO_ARRAY.
Related
I'm trying to use a query where the schema and table are passed as parameters into a prepared statement using pg_query_params like this:
$searchSchema = 'mySchema';
$searchTable = 'myTable';
$searchColumn = 'searchColumn';
$searchValue = 'some value';
$selQuery = "SELECT *
FROM $1.$2 --fails here
WHERE someColumn like $3;";
$rs = pg_query_params($db, $selQuery , array($searchSchema, $searchTable, $searchColumn, $searchValue));
The issue is with the schema and table which need to be set dynamically - as in the code above.
In a parameterized SQL statement (which is a prepared statement in PostgreSQL), parameters can only stand for constant values, not for table or column names.
This limitation is enforced by PostgreSQL, and there is no way around it, no matter what programming language or driver you use. This is also intentional and not a bug.
You will have to compose a string that contains the complete SQL statement with table and column names substituted and execute that. Beware of SQL injection – use functions like pg_escape_identifier to escape names.
I want to call a function that I created in my PostgreSQL database. I've looked at the official SQLAlchemy documentation as well as several questions here on SO, but nobody seems to explain how to set up the function in SQLAlchemy.
I did find this question, but am unsure how to compile the function as the answer suggests. Where does that code go? I get errors when I try to put this in both my view and model scripts.
Edit 1 (8/11/2016)
As per the community's requests and requirements, here are all the details I left out:
I have a table called books whose columns are arranged with information regarding the general book (title, author(s), publication date...).
I then have many tables all of the same kind whose columns contain information regarding all the chapters in each book (chapter name, length, short summary...). It is absolutely necessary for each book to have its own table. I have played around with one large table of all the chapters, and found it ill suited to my needs, not too mention extremely unwieldy.
My function that I'm asking about queries the table of books for an individual book's name, and casts the book's name to a regclass. It then queries the regclass object for all its data, returns all the rows as a table like the individual book tables, and exits. Here's the raw code:
CREATE OR REPLACE FUNCTION public.get_book(bookName character varying)
RETURNS TABLE(/*columns of individual book table go here*/)
LANGUAGE plpgsql
AS $function$
declare
_tbl regclass;
begin
for _tbl in
select name::regclass
from books
where name=bookName
loop
return query execute '
select * from ' ||_tbl;
end loop;
end;
$function$
This function has been tested several times in both the command line and pgAdmin. It works as expected.
My intention is to have a view in my Flask app whose route is #app.route('/book/<string:bookName>') and calls the above function before rendering the template. The exact view is as follows:
#app.route('/book/<string:bookName>')
def book(bookName):
chapterList = /*call function here*/
return render_template('book.html', book=bookName, list=chapterList)
This is my question: how do I set up my app in such a way that SQLAlchemy knows about and can call the function I have in my database? I am open to other suggestions of achieving the same result as well.
P.S. I only omitted this information with the intention of keeping my question as abstract as possible, not knowing that the rules of the forum dictate a requirement for a very specific question. Please forgive me my lack of knowledge.
If you want to do it without raw sql, you can use func from sqlalchemy:
from sqlalchemy import func
data = db.session.query(func.your_schema.your_function_name()).all()
You can use func
Syntax:
from sqlalchemy import func
func.function_name(column)
Example:
from sqlalchemy import func
result = db.session.query(func.lower(Student.name)).all()
I found a solution to execute the function with raw SQL:
Create a connection
Call the function as you normally would in the database GUI. E.g. for the function add_apples():
select add_apples();
Execute this statement, which should be a string.
Example code:
transaction = connection.begin()
sql = list() # Allows multiple queries
sql.append('select add_apples();')
print('Printing the queries.')
for i in sql:
print(i)
# Now, we iterate through the sql statements executing them one after another. If there is an exception on one of them, we stop the execution
# of the program.
for i in sql:
# We execute the corresponding command
try:
r = connection.execute(i)
print('Executed ----- %r' % i)
except Exception as e:
print('EXCEPTION!: {}'.format(e))
transaction.rollback()
exit(-1)
transaction.commit()
from sqlalchemy.sql import text
with engine.connect() as con:
statement = text("""your function""")
con.execute(statement)
You must execute raw sql through sqlalchemy
I have postgresql database where I'd like to run full text search queries by using JPA/Hibernate criteria API. My issue is that I don't know which type should I use for tsvector type and what is a replacement for ## operator?
I created a database view which combines data from two tables and makes a concatenated tsvector:
CREATE OR REPLACE VIEW node_name_description_tags AS
SELECT nodeId, document
FROM (SELECT node.id as nodeId,
to_tsvector('english', node.name) ||
to_tsvector('english', coalesce(node.description, ' ')) ||
to_tsvector('english', coalesce(string_agg(tag.name, ' '), ' ')) as document
FROM node
JOIN tag_node ON node.id = tag_node.node_id
JOIN tag ON tag.id = tag_node.tag_id
GROUP BY nodeId) as documents
Then I can run queries on it like this and it returns what I expect:
SELECT * FROM node_name_description_tags WHERE document ## PLAINTO_TSQUERY('english', 'integration user administration file')
What I was going to do next - create hibernate entity mapped to this view, but I don't know which type to use for column tsvector. Then I was going to to create a hibernate specification with where clause, but I don't know how operator ## is implemented in hibernate. It seems that this functionality isn't supported at all!
I found on the internet that people propose to use custom dialect with added full-text-search function which generates ## where clause. That's, basically, everything what I have now.
Any advice on how to make this working from Hibernate?
In my case, I don't want to make a full text search across all the fields. As a quick as working solution I added custom column, search queries to be run accordingly to:
#Column(name = "full_text_search_index", columnDefinition = "TEXT", nullable = false)
In this column I store and keep consistent (on entity updates) needed fields for search (some kind of denormalization), then search queries just run in 'contains' like mode. This column is indexes, so it is working quite good.
I have a string that I want to pass to SQL. To prevent SQL injection and other quoting and escaping problems, the best practice is to use a PreparedStatement with ?. For example:
val ps = conn.prepareStatement("select * from foo where name = ?")
ps.setString(1, name)
But for some SQL code, this won't work. For example, here is PostgreSQL, trying to create a view.
val ps = conn.prepareStatement("create temp view v1 as select * from foo where name = ?")
ps.setString(1, name)
val rs = ps.execute()
This throws an error:
org.postgresql.util.PSQLException: ERROR: there is no parameter $1
It apparently doesn't allow parameters to create view. How do you get around this and safely escape the string?
Prepared statements are used to plan a complex statement once and then execute it multiple (= very many) times with different parameter values. Simple statements have no noticeable benefit from using a prepared statement because planning them is trivial. DDL statements are not supported at all, so that is most likely the cause of the error, although the error message is confusing.
From the documentation:
PREPARE name [ ( data_type [, ...] ) ] AS statement
statement
Any SELECT, INSERT, UPDATE, DELETE, or VALUES statement.
The PreparedStatement class does document that you can use DDL in the executeUpdate() method, but from a logical standpoint that is just nonsense, at least in PostgreSQL.
Instead, you should use a Statement and then call execute() or executeUpdate() (a bit odd that the latter method would support DDL statements because there is no update being performed).
Preventing SQL-injection
In order to prevent SQL-injection you can use a few PostgreSQL functions:
quote_literal() - As can be expected, this will quote a literal parameter value to be safe in the query. Not only does this prevent you from Bobby Tables, but also from the likes of Mr. O'Brien.
quote_nullable() - For literals like above, but will generate proper code when the parameter IS NULL.
quote_identifier() - Will double quote any table or column name that might cause problems for the planner, like table name from with columns int, type and from: SELECT int, type, from FROM from WHERE int = type becomes SELECT "int", "type", "from" FROM "from" WHERE "int" = "type".
You can use these functions directly in your SQL statements and then let PostgreSQL deal with nasty input.
I was looking for a way to combine multiple returned rows into a single row on a db2 database (I have an application that can query a database, but will only work if a single row is returned). I found this solution which worked pretty well and was a lot easier than using recursive SQL. However, I ran into a problem when I tried to include a column that was set as TIMESTAMP instead of VARCHAR.
So how can I make this work if a column is a TIMESTAMP type?
Error:
SQL0440N No authorized routine named "XMLTEXT" of type "FUNCTION" having
compatible arguments was found. SQLSTATE=42884
SQL0440N No authorized routine named "XMLTEXT" of type "FUNCTION " having compatible arguments was found.
".
Example:
select xmlserialize(
xmlagg(
xmlconcat(
xmltext(column_name),
xmltext(':'),
xmltext(content),
xmltext(','),
xmltext(DATETIMESTAMP),
xmltext(',')
)
) as varchar(10000)
)
from
yourtable
Instead of the suggested CAST you could wrap the TOCHAR` function around the timestamp value:
select xmlserialize(
xmlagg(
xmlconcat(
xmltext(column_name),
xmltext(':'),
xmltext(content),
xmltext(','),
xmltext(TO_CHAR(DATETIMESTAMP)),
xmltext(',')
)
) as varchar(10000)
)
from
yourtable
If you are on a recent version of DB2 and have LISTAGG available I would recommend to use that function. It is much faster than converting the SQL input to XML types and then converting it back. It requires some CPU cycles due to all the official rules involved.