I have a multi-tenant database where each tenant gets their own schema. Each schema has a set of materialized views used in full-text searches.
The following function takes a schema name and a table name and concatenates them into schema.table_name format:
CREATE OR REPLACE FUNCTION create_table_name(_schema text, _tbl text, OUT result text)
AS 'select $1 || ''.'' || $2'
LANGUAGE SQL
It works as expected in PGAdmin:
I'm trying to use this function in a prepared statement, like this:
SELECT p.id AS id,
ts_rank(
p.document, plainto_tsquery(unaccent(?))
) AS rank
FROM create_table_name(?, 'project_search') AS p
WHERE p.document ## plainto_tsquery(unaccent(?))
OR p.name ILIKE ?
However, when I run it, I get the following error:
ERROR 42703 (undefined_column) column p.id does not exist
If I "hard-code" the schema and table name though, it works.
Why am I getting this error?
P.S. I should note that I am aware of the dangers of this approach, but the schema name always comes from inside my application so I'm not worried about SQL injection.
You want to use the function result as table name in a query, but what you are actually doing is using the function as a table function. This “table” has only one row and one column called result, which explains the error message.
You need dynamic SQL for that, for example by using PL/pgSQL code in a DO statement:
DO
$$DECLARE
...
BEGIN
EXECUTE
format(
E'SELECT p.id AS id,\n'
' ts_rank(\n'
' p.document,\n'
' plainto_tsquery(unaccent(?))\n'
' ) AS rank\n'
'FROM %I.project_search AS p\n'
'WHERE p.document ## plainto_tsquery(unaccent($1))\n'
'OR p.name ILIKE $2',
schema_name
)
USING fts_query, like_pattern
INTO var1, ...;
...
$$;
To handle more than one result row, you'd use a FOR loop — this is just a simple example to show the principle.
Note how I use format with the %I pattern to avoid SQL injection. Your function is vulnerable.
Related
How can I write a dynamic SELECT INTO query inside a PL/pgSQL function in Postgres?
Say I have a variable called tb_name which is filled in a FOR loop from information_schema.tables. Now I have a variable called tc which will be taking the row count for each table. I want something like the following:
FOR tb_name in select table_name from information_schema.tables where table_schema='some_schema' and table_name like '%1%'
LOOP
EXECUTE FORMAT('select count(*) into' || tc 'from' || tb_name);
END LOOP
What should be the data type of tb_name and tc in this case?
CREATE OR REPLACE FUNCTION myfunc(_tbl_pattern text, _schema text = 'public')
RETURNS void AS -- or whatever you want to return
$func$
DECLARE
_tb_name information_schema.tables.table_name%TYPE; -- currently varchar
_tc bigint; -- count() returns bigint
BEGIN
FOR _tb_name IN
SELECT table_name
FROM information_schema.tables
WHERE table_schema = _schema
AND table_name ~ _tbl_pattern -- see below!
LOOP
EXECUTE format('SELECT count(*) FROM %I.%I', _schema, _tb_name)
INTO _tc;
-- do something with _tc
END LOOP;
END
$func$ LANGUAGE plpgsql;
Notes
I prepended all parameters and variables with an underscore (_) to avoid naming collisions with table columns. Just a useful convention.
_tc should be bigint, since that's what the aggregate function count() returns.
The data type of _tb_name is derived from its parent column dynamically: information_schema.tables.table_name%TYPE. See the chapter Copying Types in the manual.
Are you sure you only want tables listed in information_schema.tables? Makes sense, but be aware of implications. See:
How to check if a table exists in a given schema
a_horse already pointed to the manual and Andy provided a code example. This is how you assign a single row or value returned from a dynamic query with EXECUTE to a (row) variable. A single column (like count in the example) is decomposed from the row type automatically, so we can assign to the scalar variable tc directly - in the same way we would assign a whole row to a record or row variable. Related:
How to get the value of a dynamically generated field name in PL/pgSQL
Schema-qualify the table name in the dynamic query. There may be other tables of the same name in the current search_path, which would result in completely wrong (and very confusing!) results without schema-qualification. Sneaky bug! Or this schema is not in the search_path at all, which would make the function raise an exception immediately.
How does the search_path influence identifier resolution and the "current schema"
Always quote identifiers properly to defend against SQL injection and random errors. Schema and table have to be quoted separately! See:
Table name as a PostgreSQL function parameter
Truncating all tables in a Postgres database
I use the regular expression operator ~ in table_name ~ _tbl_pattern instead of table_name LIKE ('%' || _tbl_pattern || '%'), that's simpler. Be wary of special characters in the pattern parameter either way! See:
PostgreSQL Reverse LIKE
Escape function for regular expression or LIKE patterns
Pattern matching with LIKE, SIMILAR TO or regular expressions in PostgreSQL
I set a default for the schema name in the function call: _schema text = 'public'. Just for convenience, you may or may not want that. See:
Assigning default value for type
Addressing your comment: to pass values, use the USING clause like:
EXECUTE format('SELECT count(*) FROM %I.%I
WHERE some_column = $1', _schema, _tb_name,column_name)
USING user_def_variable;
Related:
INSERT with dynamic table name in trigger function
It looks like you want the %I placeholder for FORMAT so that it treats your variable as an identifier. Also, the INTO clause should go outside the prepared statement.
FOR tb_name in select table_name from information_schema.tables where table_schema='some_schema' and table_name like '%1%'
LOOP
EXECUTE FORMAT('select count(*) from %I', tb_name) INTO tc;
END LOOP
I am trying to create a set of functions for a PostgreSQL database I have, and I am facing this problem. What I am trying to do is to create a function, which takes as a parameter the name of his column a user wants to change, and the new value they wish to insert.
I thought that the simplest way would be to create the queries as strings internally and execute them (sql injections are not of concern right now).
Searching a bit, I tried to use the EXECUTE command, with no success. Any arrangement of the queries I tried did not work, with or without SET, and even a super simple query shows a syntax error, inside the function code or in the pgadmin sql editor:
EXECUTE "SELECT * FROM user_data.users;";
ERROR: prepared statement "SELECT * FROM user_data.users;" does not exist
********** Error **********
ERROR: prepared statement "SELECT * FROM user_data.users;" does not exist
SQL state: 26000
Any suggestions to solve this?
To alter a column name you could use a function like this.
It takes the table name, column name, and new column name.
CREATE OR REPLACE FUNCTION foo(_t regclass, _ocn text, _ncn text)
RETURNS void AS
$func$
BEGIN
EXECUTE 'ALTER TABLE '|| _t ||'
RENAME COLUMN '|| quote_ident(_ocn) ||' TO '|| quote_ident(_ncn) ||'';
END
$func$ LANGUAGE plpgsql;
To alter the type is a bit more work and may add to the server overheads.
Lets use this table as an example
CREATE TABLE foobar (id serial primary key, name text);
Ref Postgresql Type Conversion
In many cases a user does not need to understand the details of the
type conversion mechanism. However, implicit conversions done by
PostgreSQL can affect the results of a query. When necessary, these
results can be tailored by using explicit type conversion.
Firstly we would have to get the column data type(s) from the information The Information Schema Ref postgresql 9.4
select data_type from information_schema.columns where table_name = 'foobar' and column_name = 'name';
We could cast types as mentioned in type conversion::Table 8-1. Data Types Ref Postgresql 9.4
ALTER TABLE foobar ALTER COLUMN name TYPE varchar(20) USING name::text;
I hope this helps
In Postgres, you can link to your other databases using dblink like so:
SELECT *
FROM dblink (
'dbname=name port=1234 host=host user=user password=password',
'select * from table'
) AS users([insert each column name and its type here]);
But this is quite verbose.
I've shortened it up by using dblink_connect and dblink_disconnect to abstract the connection string from my dblink queries. However, that still leaves me with the manual table definition (i.e., [insert each column name and its type here]).
Instead of defining the table manually, is there a way I can define it with a TYPE or anything else that'd be re-usable?
In my case, the number of remote tables I have to join and the number of columns involved makes my query massive.
I tried something along the lines of:
SELECT *
FROM dblink (
'myconn',
'select * from table'
) AS users(postgres_pre_defined_type_here);
But I received the following error:
ERROR: a column definition list is required for functions returning "record"
As you considered creating several types for dblink, you can accept creating several functions as well. The functions will be well defined and very easy to use.
Example:
create or replace function dblink_tables()
returns table (table_schema text, table_name text)
language plpgsql
as $$
begin
return query select * from dblink (
'dbname=test password=mypassword',
'select table_schema, table_name from information_schema.tables')
as tables (table_schema text, table_name text);
end $$;
select table_name
from dblink_tables()
where table_schema = 'public'
order by 1
I have a plpgsql function that creates an execute statement and I want it to execute it into a table. The standard EXECUTE ... INTO table_name didn't work, so I was looking for an alternative. Basically, the select statement returns three columns which I want to save to a table.
Here's some sample code of the execute:
query = 'SELECT name, work, phone FROM info WHERE name = '
|| quote_literal(inName) || ' ORDER BY phone';
Ideally, if I was just running the query myself I would just put a SELECT INTO tablename, but that didn't work with the execute.
Any ideas?
Use CREATE TABLE AS for that:
EXECUTE 'CREATE TABLE foo AS
SELECT name, work, phone
FROM info WHERE name = ' || quote_literal(in_name) || ' ORDER BY phone';
SELECT INTO is discouraged for that purpose:
Combine two tables into a new one so that select rows from the other one are ignored
SELECT / EXECUTE .. INTO .. is meant for single rows, not for whole tables in plpgsql.
And the assignment operator in plpgsql is :=, not =.
And a single quote was missing.
And don't use unquoted mixed case identifiers in Postgres.
I was attempting to use Dynamic SQL to run some queries in postgres.
Example:
EXECUTE format('SELECT * from result_%s_table', quote_ident((select id from ids where condition = some_condition)))
I have to query a table, which is of the form result_%s_table wherein, I need to substitute the correct table name (an id) from an another table.
I get the error ERROR: prepared statement "format" does not exist
Link: string substitution with query result postgresql
EXECUTE ... USING only works in PL/PgSQL - ie within functions or DO blocks written in the PL/PgSQL language. It does not work in plain SQL; the EXECUTE in plain SQL is completely different, for executing prepared statements. You cannot use dynamic SQL directly in PostgreSQL's SQL dialect.
Compare:
PL/PgSQL's EXECUTE ... USING; to
SQL's EXECUTE
See the 2nd last par in my prior answer.
In addition to not running except in PL/PgSQL your SQL statement is wrong, it won't do what you expect. If (select id from ids where condition = some_condition) returns say 42, the statement would fail if id is an integer. If it's cast to text you'd get:
EXECUTE format('SELECT * from result_%s_table', quote_ident('42'));
EXECUTE format('SELECT * from result_%s_table', '"42"');
EXECUTE 'SELECT * from result_"42"_table';
That's invalid. You actually want result_42_table or "result_42_table". You'd have to write something more like:
EXECUTE format('SELECT * from %s', quote_ident('result_'||(select id from ids where condition = some_condition)||'_table'))
... if you must use quote_ident.
CREATE OR REPLACE FUNCTION public.exec(
text)
RETURNS SETOF RECORD
LANGUAGE 'plpgsql'
AS $BODY$
BEGIN
RETURN QUERY EXECUTE $1 ;
END
$BODY$;
usage:
select * from exec('select now()') as t(dt timestamptz)
Try using
RETURN QUERY EXECUTE '<SQL Command>'
This will return data into form of table. You have to use this into stored function of PostgreSQL.
I have already created on full demonstration on custom filter and custom sorting using dynamic query of PostgreSQL.
Please visit this url:
http://www.dbrnd.com/2015/05/postgresql-dynamic-sql/
These all look more complicated than the OP's question. A different formatting should do the trick.. but it could absolutely the case that I don't understand.
From how I read OP's question, I think others in a similar situation may benefit from how I got it.
I am using Postgre on Redshift, and I ran into this issue and found a solution.
I was trying to create a dynamic query, putting in my own date.
date = dt.date(2018, 10, 30)
query = ''' select * from table where date >= ''' + str(my_date) + ''' order by date '''
But, the query entirely ignores the condition when typing it this way.
However, if you use the percent sign (%), you can insert the date correctly.
One correct way to write the above statement is:
query = ''' select * from table where date >= ''' + ''' '%s' ''' % my_date + ''' order by date '''
So, maybe this is helpful, or maybe it is not. I hope it helps at least one person in my situation!
Best wishes.
EXECUTE will work only on pl/pqsql environment.
instead of EXECUTE try with SELECT
SELECT format('SELECT * from result_%s_table', quote_ident((select id from ids where condition = some_condition))
output would be the dynamic query.