I have a simple mission is inserting huge MD5 values into tables (partitioned table), and have created a trigger and also a trigger function to instead of INSERT operation. And in function I checked the first two characters of NEW.md5 to determine which table should be inserted.
DECLARE
tb text;
BEGIN
IF TG_OP = 'INSERT' THEN
tb = 'samples_' || left(NEW.md5, 2);
EXECUTE(format('INSERT INTO %s VALUES (%s);', tb, NEW.*)); <- WRONG
END IF;
RETURN NULL;
END;
The question is how to concat the NEW.* into the SQL statement?
Best with the USING clause of EXECUTE:
CREATE FUNCTION foo ()
RETURNS trigger AS
$func$
BEGIN
IF TG_OP = 'INSERT' THEN
EXECUTE format('INSERT INTO %s SELECT $1.*'
, 'samples_' || left(NEW.md5, 2);
USING NEW;
END IF;
RETURN NULL;
END
$func$ LANGUAGE plpgsql;
And EXECUTE does not require parentheses.
And you are aware that identifiers are folded to lower case unless quoted where necessary (%I instead of %s in format()).
More details:
INSERT with dynamic table name in trigger function
How to dynamically use TG_TABLE_NAME in PostgreSQL 8.2?
Related
I have some function on PostgreSQL 9.6 returning a cursor (refcursor):
CREATE OR REPLACE FUNCTION public.test_returning_cursor()
RETURNS refcursor
IMMUTABLE
LANGUAGE plpgsql
AS $$
DECLARE
_ref refcursor = 'test_returning_cursor_ref1';
BEGIN
OPEN _ref FOR
SELECT 'a' :: text AS col1
UNION
SELECT 'b'
UNION
SELECT 'c';
RETURN _ref;
END
$$;
I need to write another function in which a temp table is created and all data from this refcursor are inserted to it. But INSERT INTO ... FETCH ALL FROM ... seems to be impossible. Such function can't be compiled:
CREATE OR REPLACE FUNCTION public.test_insert_from_cursor()
RETURNS table(col1 text)
IMMUTABLE
LANGUAGE plpgsql
AS $$
BEGIN
CREATE TEMP TABLE _temptable (
col1 text
) ON COMMIT DROP;
INSERT INTO _temptable (col1)
FETCH ALL FROM "test_returning_cursor_ref1";
RETURN QUERY
SELECT col1
FROM _temptable;
END
$$;
I know that I can use:
FOR _rec IN
FETCH ALL FROM "test_returning_cursor_ref1"
LOOP
INSERT INTO ...
END LOOP;
But is there better way?
Unfortunately, INSERT and SELECT don't have access to cursors as a whole.
To avoid expensive single-row INSERT, you could have intermediary functions with RETURNS TABLE and return the cursor as table with RETURN QUERY. See:
Return a query from a function?
CREATE OR REPLACE FUNCTION f_cursor1_to_tbl()
RETURNS TABLE (col1 text) AS
$func$
BEGIN
-- MOVE BACKWARD ALL FROM test_returning_cursor_ref1; -- optional, see below
RETURN QUERY
FETCH ALL FROM test_returning_cursor_ref1;
END
$func$ LANGUAGE plpgsql; -- not IMMUTABLE
Then create the temporary table(s) directly like:
CREATE TEMP TABLE t1 ON COMMIT DROP
AS SELECT * FROM f_cursor1_to_tbl();
See:
Creating temporary tables in SQL
Still not very elegant, but much faster than single-row INSERT.
Note: Since the source is a cursor only the first call succeeds. Executing the function a second time would return an empty set. You would need a cursor with the SCROLL option and move to the start for repeated calls.
This function does INSERT INTO from refcursor. It is universal for all the tables. The only requirement is that all columns of table corresponds to columns of refcursor by types and order (not necessary by names).
to_json() does the trick to convert any primitive data types to string with double-quotes "", which are later replaced with ''.
CREATE OR REPLACE FUNCTION public.insert_into_from_refcursor(_table_name text, _ref refcursor)
RETURNS void
LANGUAGE plpgsql
AS $$
DECLARE
_sql text;
_sql_val text = '';
_row record;
_hasvalues boolean = FALSE;
BEGIN
LOOP --for each row
FETCH _ref INTO _row;
EXIT WHEN NOT found; --there are no rows more
_hasvalues = TRUE;
SELECT _sql_val || '
(' ||
STRING_AGG(val.value :: text, ',') ||
'),'
INTO _sql_val
FROM JSON_EACH(TO_JSON(_row)) val;
END LOOP;
_sql_val = REPLACE(_sql_val, '"', '''');
_sql_val = TRIM(TRAILING ',' FROM _sql_val);
_sql = '
INSERT INTO ' || _table_name || '
VALUES ' || _sql_val;
--RAISE NOTICE 'insert_into_from_refcursor(): SQL is: %', _sql;
IF _hasvalues THEN --to avoid error when trying to insert 0 values
EXECUTE (_sql);
END IF;
END;
$$;
Usage:
CREATE TABLE public.table1 (...);
PERFORM my_func_opening_refcursor();
PERFORM public.insert_into_from_refcursor('public.table1', 'name_of_refcursor_portal'::refcursor);
where my_func_opening_refcursor() contains
DECLARE
_ref refcursor = 'name_of_refcursor_portal';
OPEN _ref FOR
SELECT ...;
I am creating trigger in PostgresSQL. On update I would like to compare all of the values in a Hstore column and update changes in my mirror table. I managed to get names of my columns in variable k but I am not able to get values using it from NEW and OLD.
CREATE OR REPLACE FUNCTION function_replication() RETURNS TRIGGER AS
$BODY$
DECLARE
k text;
BEGIN
FOR k IN SELECT key FROM EACH(hstore(NEW)) LOOP
IF NEW.k != OLD.k THEN
EXECUTE 'UPDATE ' || TG_TABLE_NAME || '_2' || 'SET ' || k || '=' || new.k || ' WHERE ID=$1.ID;' USING OLD;
END IF;
END LOOP;
RETURN NEW;
END;
$BODY$
language plpgsql;
You should operate on hstore representations of the records new and old. Also, use the format() function for better control and readibility.
create or replace function function_replication()
returns trigger as
$body$
declare
newh hstore = hstore(new);
oldh hstore = hstore(old);
key text;
begin
foreach key in array akeys(newh) loop
if newh->key != oldh->key then
execute format(
'update %s_2 set %s = %L where id = %s',
tg_table_name, key, newh->key, oldh->'id');
end if;
end loop;
return new;
end;
$body$
language plpgsql;
Another version - with minimalistic numbers of updates - in partially functional design (where it is possible).
This trigger should be AFTER trigger, to be ensured correct behave.
CREATE OR REPLACE FUNCTION function_replication()
RETURNS trigger AS $$
DECLARE
newh hstore;
oldh hstore;
update_vec text[];
pair text[];
BEGIN
IF new IS DISTINCT FROM old THEN
IF new.id <> old.id THEN
RAISE EXCEPTION 'id should be immutable';
END IF;
newh := hstore(new); oldh := hstore(old); update_vec := '{}';
FOREACH pair SLICE 1 IN ARRAY hstore_to_matrix(newh - oldh)
LOOP
update_vec := update_vec || format('%I = %L', pair[1], pair[2]);
END LOOP;
EXECUTE
format('UPDATE %I SET %s WHERE id = $1',
tg_table_name || '_2',
array_to_string(update_vec, ', '))
USING old.id;
END IF;
RETURN NEW; -- the value is not important in AFTER trg
END;
$$ LANGUAGE plpgsql;
CREATE TABLE foo(id int PRIMARY KEY, a int, b int);
CREATE TABLE foo_2(LIKE foo INCLUDING ALL);
CREATE TRIGGER xxx AFTER UPDATE ON foo
FOR EACH ROW EXECUTE PROCEDURE function_replication();
INSERT INTO foo VALUES(1, NULL, NULL);
INSERT INTO foo VALUES(2, 1,1);
INSERT INTO foo_2 VALUES(1, NULL, NULL);
INSERT INTO foo_2 VALUES(2, 1,1);
UPDATE foo SET a = 20, b = 30 WHERE id = 1;
UPDATE foo SET a = NULL WHERE id = 1;
This code is little bit more complex, but all what should be escaped is escaped and reduce number of executed UPDATE commands. UPDATE is full SQL command and the overhead of full SQL commands should be significantly higher than code that reduce number of full SQL commands.
I wrote a PL/pgsql to batch create index on tables
CREATE OR REPLACE FUNCTION create_index() RETURNS void AS
$BODY$
DECLARE
r INTEGER;
BEGIN
FOR r IN 1..1000
LOOP
EXECUTE format(
' CREATE INDEX idx_abc_id_' || r::text ||
' ON abc_id_' || r::text ||
' USING btree
(key);');
END LOOP;
RETURN;
END
$BODY$
LANGUAGE plpgsql;
it has one problem, if partition abc_500 doesn't exist, then the how create index function will fail and do nothing.
How to make loop keep going through even if create_index made an error on one of the table in between?
I think a better approach would be to not hardcode the number for the loop, but iterate over the existing tables:
CREATE OR REPLACE FUNCTION create_index() RETURNS void AS
$BODY$
DECLARE
r record;
BEGIN
FOR r IN select tablename, regexp_replace(tablename, '[^0-9]+','') as idx_nr
from pg_tables
where tablename ~ 'abc_id_[0-9]+'
LOOP
EXECUTE format('CREATE INDEX %I ON %I USING btree (key)',
'idx_abc_id_'||r.idx_nr,
r.tablename);
END LOOP;
RETURN;
END
$BODY$
LANGUAGE plpgsql;
When you use the format() function is better to use the proper place holders for identifiers.
If you also want to ignore any error when creating the index on an existing table, you need to catch the exception and ignore it:
CREATE OR REPLACE FUNCTION create_index() RETURNS void AS
$BODY$
DECLARE
r record;
msg text;
BEGIN
FOR r IN select tablename, regexp_replace(tablename, '[^0-9]+','') as idx_nr
from pg_tables
where tablename ~ 'abc_id_[0-9]+'
LOOP
BEGIN
EXECUTE format('CREATE INDEX %I ON %I USING btree (key)',
'idx_abc_id_'||r.idx_nr,
r.tablename);
EXCEPTION
WHEN OTHERS THEN
GET STACKED DIAGNOSTICS msg = MESSAGE_TEXT;
RAISE NOTICE 'Could not create index for: %, %', r.idx_nr, msg;
END;
END LOOP;
RETURN;
END
$BODY$
LANGUAGE plpgsql;
This is a trigger that is called by either an insert, update or a delete on a table. It is guaranteed the calling table has all the columns impacted and a deletes table also exists.
CREATE OR REPLACE FUNCTION sample_trigger_func() RETURNS TRIGGER AS $$
DECLARE
operation_code char;
table_name varchar(50);
delete_table_name varchar(50);
old_id integer;
BEGIN
table_name = TG_TABLE_NAME;
delete_table_name = TG_TABLE_NAME || '_deletes';
SELECT SUBSTR(TG_OP, 1, 1)::CHAR INTO operation_code;
IF TG_OP = 'DELETE' THEN
OLD.mod_op = operation_code;
OLD.mod_date = now();
RAISE INFO 'OLD: %', (OLD).name;
EXECUTE format('INSERT INTO %s VALUES %s', delete_table_name, (OLD).*);
ELSE
EXECUTE format('UPDATE TABLE %s SET mod_op = %s AND mod_date = %s'
, TG_TABLE_NAME, operation_code, now());
END IF;
RETURN NEW;
END;
$$ LANGUAGE plpgsql;
The ELSE branch triggers an endless loop. There may be more problems.
How to fix it?
The ELSE branch can be radically simplified. But a couple more things are inefficient / inaccurate / dangerous:
CREATE OR REPLACE FUNCTION sample_trigger_func()
RETURNS TRIGGER AS
$func$
BEGIN
IF TG_OP = 'DELETE' THEN
RAISE INFO 'OLD: %', OLD.name;
EXECUTE format('INSERT INTO %I SELECT ($1).*', TG_TABLE_NAME || '_deletes')
USING OLD #= hstore('{mod_op, mod_datetime}'::text[]
, ARRAY[left(TG_OP, 1), now()::text]);
RETURN OLD;
ELSE -- insert, update
NEW.mod_op := left(TG_OP, 1);
NEW.mod_datetime := now();
RETURN NEW;
END IF;
END
$func$ LANGUAGE plpgsql;
In the ELSE branch just assign to NEW directly. No need for more dynamic SQL - which would fire the same trigger again causing an endless loop. That's the primary error.
RETURN NEW; outside the IF construct would break your trigger function for DELETE, since NEW is not assigned for DELETEs.
A key feature is the use of hstore and the hstore operator #= to dynamically change two selected fields of the well-known row type - that is unknown at the time of writing the code. This way you do not tamper with the original OLD value, which might have surprising side effect if you have more triggers down the chain of events.
OLD #= hstore('{mod_op, mod_datetime}'::text[]
, ARRAY[left(TG_OP, 1), now()::text]);
The additional module hstore must be installed. Details:
How to set value of composite variable field using dynamic SQL
Passing column names dynamically for a record variable in PostgreSQL
Using the hstore(text[], text[]) variant here to construct an hstore value with multiple fields on the fly.
The assignment operator in plpgsql is :=:
The forgotten assignment operator "=" and the commonplace ":="
Note that I used the column name mod_datetime instead of the misleading mod_date, since the column is obviously a timestamp and not a date.
I added a couple of other improvements while being at it. And the trigger itself should look like this:
CREATE TRIGGER insupdel_bef
BEFORE INSERT OR UPDATE OR DELETE ON table_name
FOR EACH ROW EXECUTE PROCEDURE sample_trigger_func();
SQL Fiddle.
I am trying to write a trigger function that will input values into separate child tables, however I am getting an error I have not seen before.
Here is an example set up:
-- create initial table
CREATE TABLE public.testlog(
id serial not null,
col1 integer,
col2 integer,
col3 integer,
name text
);
-- create child table
CREATE TABLE public.testlog_a (primary key(id)) INHERITS(public.testlog);
-- make trigger function for insert
CREATE OR REPLACE FUNCTION public.test_log() RETURNS trigger AS
$$
DECLARE
qry text;
BEGIN
qry := 'INSERT INTO public.testlog_' || NEW.name || ' SELECT ($1).*';
EXECUTE qry USING NEW.*;
RETURN OLD;
END
$$
LANGUAGE plpgsql VOLATILE SECURITY DEFINER;
-- add function to table
CREATE TRIGGER test_log_sorter BEFORE INSERT
ON public.testlog FOR EACH ROW
EXECUTE PROCEDURE public.test_log();
and the query:
INSERT INTO public.testlog (col1, col2, col3, name) values (1, 2, 3, 'a');
error message:
[Err] ERROR: query "SELECT NEW.*" returned 5 columns
CONTEXT: PL/pgSQL function test_log() line 7 at EXECUTE statement
5 columns is exactly what I am looking for it to return, so clearly there is something I am not understanding but the error message seems to make no sense.
Can anybody explain why I am getting this?
Your solution fixes the passing of the row-typed NEW variable. However, you have a sneaky SQL-injection hole in your code, that's particularly dangerous in a SECURITY DEFINER function. User input must never be converted to SQL code unescaped.
Sanitize like this:
CREATE OR REPLACE FUNCTION trg_test_log()
RETURNS trigger AS
$$
BEGIN
EXECUTE 'INSERT INTO public.' || quote_ident('testlog_' || NEW.name)
|| ' SELECT ($1).*'
USING NEW;
RETURN NULL;
END
$$
LANGUAGE plpgsql SECURITY DEFINER;
Also:
OLD is not defined in an INSERT trigger.
You don't need a variable. Assignments are comparatively expensive in plpgsql.
The EXECUTE qry USING NEW.* passes in the NEW.* as the arguments to the query. Since NEW.* returns five columns, the query should have $1, $2, $3, $4 and $5 in order to bind the five columns.
You are expecting a single argument ($1) which has five columns in it. I believe that if you change the the line to
EXECUTE qry USING NEW;
it will work as you expect.
With regards to Robert M. Lefkowitz' response, the answer is so simple: NEW as opposed to NEW.*
CREATE OR REPLACE FUNCTION public.test_log() RETURNS trigger AS
$$
DECLARE
qry text;
BEGIN
qry := 'INSERT INTO public.testlog_' || NEW.name || ' SELECT ($1).*';
EXECUTE qry USING NEW;
RETURN OLD;
END
$$
LANGUAGE plpgsql VOLATILE SECURITY DEFINER
COST 100;
thanks.