I am building a trigger function in plpgsql for my partitioned table and I've got all of my logic working, but am having trouble inserting the actual record into my table.
I have to reference my specific table by a variable reference, so that (as far as I understand) forces me to use an EXECUTE command, as so:
EXECUTE 'INSERT INTO ' || tablename || ' VALUES ' || NEW.*;
However, this does not handle unpacking the record stored in NEW in a way that Postgres' INSERT function can understand. It converts the record into a string while preserving all of the double quotes within. I.e. the above turns into the following upon execution:
INSERT INTO cfs_hgt_05152016_05202016
VALUES ("2016-05-16 00:00:00","2016-05-12 06:00:00",HGT,"1000 mb",9,-71,-38.5371)
The problem with this is Postgres thinks these values are now columns due to the double quotes.
COPY cfs_hgt_master, line 1: ""2016-05-16 00:00:00","2016-05-12 06:00:00","HGT","1000 mb",9,-71,-38.5371"
ERROR: column "2016-05-16 00:00:00" does not exist
I tried to remedy this by the following:
record_text := regexp_replace(NEW.*, '"', '\'', 'gi');
EXECUTE 'INSERT INTO ' || tablename || ' VALUES ' || record_text;
But escaping the single quote like that produces an error:
psql:cfs_hgt_trigger_function.sql:36: ERROR: unterminated quoted string at or near "' || record_text;
LINE 30: ... EXECUTE 'INSERT INTO ' || tablename || ' VALUES ' || recor...
^
Could someone help me figure out how to either properly escape that single quote, or suggest an alternative means of accomplishing my task?
Don't convert values to their text representation at all. Pass values with the USING clause of EXECUTE.
And escape table names properly. You can use format() for this:
EXECUTE format('INSERT INTO %I SELECT $1.*', tablename)
USING NEW;
Details:
INSERT with dynamic table name in trigger function
How to pass NEW.* to EXECUTE in trigger function
Related
May I know on how to call an array in stored procedure? I tried to enclosed it with a bracket to put the column_name that need to be insert in the new table.
CREATE OR REPLACE PROCEDURE data_versioning_nonull(new_table_name VARCHAR(100),column_name VARCHAR(100)[], current_table_name VARCHAR(100))
language plpgsql
as $$
BEGIN
EXECUTE ('CREATE TABLE ' || quote_ident(new_table_name) || ' AS SELECT ' || quote_ident(column_name) || ' FROM ' || quote_ident(current_table_name));
END $$;
CALL data_versioning_nonull('sales_2019_sample', ['orderid', 'product', 'address'], 'sales_2019');
Using execute format() lets you replace all the quote_ident() with %I placeholders in a single text instead of a series of concatenated snippets. %1$I lets you re-use the first argument.
It's best if you use ARRAY['a','b','c']::VARCHAR(100)[] to explicitly make it an array of your desired type. '{"a","b","c"}'::VARCHAR(100)[] works too.
You'll need to convert the array into a list of columns some other way, because when cast to text, it'll get curly braces which are not allowed in the column list syntax. Demo
It's not a good practice to introduce random limitations - PostgreSQL doesn't limit identifier lengths to 100 characters, so you don't have to either. The default limit is 63 bytes, so you can go way, way longer than 100 characters (demo). You can switch that data type to a regular text. Interestingly, exceeding specified varchar length would just convert it to unlimited varchar, making it just syntax noise.
DBFiddle online demo
CREATE TABLE sales_2019(orderid INT,product INT,address INT);
CREATE OR REPLACE PROCEDURE data_versioning_nonull(
new_table_name TEXT,
column_names TEXT[],
current_table_name TEXT)
LANGUAGE plpgsql AS $$
DECLARE
list_of_columns_as_quoted_identifiers TEXT;
BEGIN
SELECT string_agg(quote_ident(name),',')
INTO list_of_columns_as_quoted_identifiers
FROM unnest(column_names) name;
EXECUTE format('CREATE TABLE %1$I.%2$I AS SELECT %3$s FROM %1$I.%4$I',
current_schema(),
new_table_name,
list_of_columns_as_quoted_identifiers,
current_table_name);
END $$;
CALL data_versioning_nonull(
'sales_2019_sample',
ARRAY['orderid', 'product', 'address']::text[],
'sales_2019');
Schema awareness: currently the procedure creates the new table in the default schema, based on a table in that same default schema - above I made it explicit, but that's what it would do without the current_schema() calls anyway. You could add new_table_schema and current_table_schema parameters and if most of the time you don't expect them to be used, you can hide them behind procedure overloads for convenience, using current_schema() to keep the implicit behaviour. Demo
First, change your stored procedure to convert selected columns from array to csv like this.
CREATE OR REPLACE PROCEDURE data_versioning_nonull(new_table_name VARCHAR(100),column_name VARCHAR(100)[], current_table_name VARCHAR(100))
language plpgsql
as $$
BEGIN
EXECUTE ('CREATE TABLE ' || quote_ident(new_table_name) || ' AS SELECT ' || array_to_string(column_name, ',') || ' FROM ' || quote_ident(current_table_name));
END $$;
Then call it as:
CALL data_versioning_nonull('sales_2019_sample', '{"orderid", "product", "address"}', 'sales_2019');
I'm struggling with a stored procedure which heavily uses dynamic queries. Among others I need to store maximum value of an existing column into a variable.
Postgres documents state "if you want to use dynamically determined table or column names, you must insert them into the command string textually". Based on that I've come up with following statement:
EXECUTE 'SELECT MAX(' || pkColumn::regclass || ') FROM ' ||
tableName::regclass INTO maxValue;
Table name seems to be OK, column name triggers error.
What am I doing wrong ?
Pavel
EXECUTE 'SELECT MAX(' || pkColumn ||'::regclass) FROM ' || ...
::regclass is a cast done inside query. You can also skip it, or put " - which in PG works the same. So please try one of:
EXECUTE 'SELECT MAX(' || pkColumn || ') FROM ' || ...
or
EXECUTE 'SELECT MAX("' || pkColumn || '") FROM ' || ...
All tree should work. If not - just let me know. In that case it is my fault, postgresql simply works.
There is no reason to cast parameters as they are just identifiers. For better control and readability use the function format(), e.g.:
declare
pkcolumn text = 'my_column';
tablename text = 'my_table';
...
execute format('select max(%I) from %I', pkcolumn, tablename)
into maxvalue;
I am trying to execute the following dynamic sql, but I could not figure out how to do it:
DROP FUNCTION f_mycross(text, text);
EXECUTE ('CREATE OR REPLACE FUNCTION f_mycross(text, text)
RETURNS TABLE ("registration_id" integer, '
|| (SELECT string_agg(DISTINCT pivot_headers, ',' order by pivot_headers)
FROM (SELECT DISTINCT '"' || qid::text || '" text' AS pivot_headers
FROM answers) x)
|| ') AS ''$libdir/tablefunc'',''crosstab_hash'' LANGUAGE C STABLE STRICT;')
I am relatively new to PostgreSQL.
Like a_horse commented, EXECUTE is not an SQL command. It's a PL/pgSQL command and can only be used in a function body or DO statement using this procedural language. Like:
DROP FUNCTION IF EXISTS f_mycross(text, text);
DO
$do$
BEGIN
EXECUTE (
SELECT 'CREATE OR REPLACE FUNCTION f_mycross(text, text)
RETURNS TABLE (registration_id integer, '
|| string_agg(pivot_header || ' text', ', ')
|| $$) AS '$libdir/tablefunc', 'crosstab_hash' LANGUAGE C STABLE STRICT$$
FROM (SELECT DISTINCT quote_ident(qid::text) AS pivot_header FROM answers ORDER BY 1) x
);
END
$do$; -- LANGUAGE plpgsql is the default
I added some improvements and simplified the nested SELECT query.
Major points
Add IF EXISTS to DROP FUNCTION unless you are certain the function exists or you want to raise an exception if it does not.
DISTINCT in the subquery is enough, no need for another DISTINCT in the outer SELECT.
Use quote_ident() to automatically double-quote identifiers where necessary.
No parentheses required around the string we feed to EXECUTE.
Simpler nested quoting with $-quotes.
Insert text with single quotes in PostgreSQL
We can apply ORDER BY in the subquery, which is typically much faster than adding ORDER BY in the outer aggregate function.
I have a PL/pgSQL function that takes table name as dynamic parameter. As I am updating an existing query to take table name as dynamic parameter, this is what I have as my function:
DECLARE rec RECORD;
BEGIN
EXECUTE 'insert into stat_300_8_0(ts, target, data)
select distinct timestamp-(timestamp%3600) as wide_row_ts,
target, array[]::real[] as data
from ' || temp_table_name || ' as temp
where class_id=8
and subclass_id=0
and not exists (select ts from stat_300_8_0
where ts=temp.timestamp-(temp.timestamp%3600)
and target=temp.target)';
FOR rec IN EXECUTE 'SELECT DISTINCT timestamp AS ts
FROM ' || temp_table_name ||
' WHERE class_id=8'
LOOP
EXECUTE 'update stat_300_8_0 as disk_table
set data[new_data.start_idx:new_data.end_idx] = array[data_0,data_1]
from (select timestamp-(timestamp%3600) as wide_row_ts,
(timestamp%3600)/300 * 2 + 1 as start_idx,
((timestamp%3600 / 300) + 1) * 2 as end_idx,
target, data_0, data_1
from ' || temp_table_name ||
' where class_id=8 and subclass_id=0
and timestamp=rec.ts) as new_data
where disk_table.ts=new_data.wide_row_ts
and disk_table.target=new_data.target';
END LOOP;
END;
However, when this function is executed I get an error saying
ERROR: missing FROM-clause entry for table "rec"
However, rec is declared in the first line of the above code. I am not able to figure what is wrong with my queries. Any help is appreciated.
Supplemental to Eelke's answer:
Assuming temp_table_name is an argument, you really, really want to run it through quote_ident() because otherwise someone could create a table with a name that could inject sql into your function.
Instead of the change suggested there, you are better off using EXECUTE...USING since that gives you parameterization regarding values (and hence protection against SQL injection). You would change rec.ts to $1 and then add to the end USING ts.rec (outside the quoted execute string). This gives you a parameterized statement inside your execute which is safer. However parameters cannot include table names, so it doesn't spare you from the first point above.
I have a number of tables that use the Postgres "Partitioning" feature. I want to define a common BEFORE INSERT OF ROW trigger on each table that will 1) dynamically create the partition should the insert occur against the parent table and 2) re-execute the insert against the partition.
Something like:
CREATE OR REPLACE FUNCTION partition_insert_redirect( )
RETURNS trigger AS $BODY$
BEGIN
... create the new partition and set up the redirect Rules ...
/* Redo the INSERT dynamically. The new RULE will redirect it to the child table */
EXECUTE 'INSERT INTO ' || quote_ident(TG_TABLE_SCHEMA) || '.' || quote_ident(TG_TABLE_NAME) ||
' SELECT NEW.*'
END
But the "NEW" record isn't visible inside the EXECUTE SQL. How can I make this work as simply as possible?
As an alternative, can I iterate over the fields in the NEW record somehow?
I've thought of using a temp-table:
EXECUTE 'CREATE TEMPORARY TABLE new_row (LIKE ' ||
quote_ident(TG_TABLE_SCHEMA) || '.' || quote_ident(TG_TABLE_NAME) ||
') ON COMMIT DROP';
INSERT INTO new_row SELECT NEW.*;
EXECUTE 'INSERT INTO ' || quote_ident(TG_TABLE_SCHEMA) || '.' || quote_ident(TG_TABLE_NAME) ||
' SELECT * FROM new_row';
DROP TABLE new_row;
But this also doesn't work because of the cached reference to a temp-table: Why do I get "relation with OID ##### does not exist" errors when accessing temporary tables in PL/PgSQL functions?
I'm using Postgres 8.2 and I can't change to any other version.
EDIT:
As #alvherre pointed out, this can probably be done in Postgres 8.4 with the EXECUTE ... USING syntax. See an example at http://wiki.postgresql.org/wiki/PL/pgSQL_Dynamic_Triggers
You can use EXECUTE USING to pass NEW to it. Your example would be
EXECUTE 'INSERT INTO ' || TG_RELID || '::regclass SELECT $1' USING NEW;
(Note that I use TG_RELID casted to regclass instead of fiddling with TG_TABLE_SCHEMA and TABLE_NAME because it is easier to use, if nonstandard. But then, plpgsql is nonstandard anyway.)
Yes, you can use EXECUTE ... USING in 8.4. For example:
EXECUTE 'INSERT INTO ' || table_name || ' SELECT $1.*' USING NEW;
In lower versions (I've only tested in 8.3), you can use:
EXECUTE 'INSERT INTO ' || table_name ||
' SELECT (' || quote_literal(NEW) || '::' || TG_RELID::regclass || ').*';
I've managed to get this to work by dynamically compiling a function that accepts the NEW row as a parameter:
EXECUTE 'create or replace function partition_insert(r ' || TG_TABLE_NAME || ') RETURNS void AS $FUNC$' ||
'BEGIN ' ||
'insert into ' || TG_TABLE_NAME || ' SELECT r.*; ' ||
'END $FUNC$ LANGUAGE plpgsql VOLATILE';
PERFORM partition_insert(NEW);
As Postgres functions are polymorphic, this will generate a different function for each table that uses this trigger.
Despite being an ugly kludge, this seems to do the job.
Although it looks like I could define each polymorphic variation up front when I build the system, because of caching, I must recompile the function whenever I create or drop a child table so that the function uses the latest insert RULE.
EDIT: Additional wrinkles
There's a little gotcha with this technique: If this EXECUTE/PERFORM action is rolled-back on the first attempt due to another error (for example, in my case a CHECK constraint failure) then the function containing this code seems to cache a reference to the rolled-back partition_insert() function it created using the EXECUTE and subsequent calls fail due to a cached object not being found.
I resolved this by pre-creating stub versions of the function for each required table-type parameter when I define the database.