I have a spatial table in a Postgres database from which I create three separate materialized views (each based on specific spatial queries). I want to create trigger functions to refresh each of the materialized views on updates, inserts, and deletes.
I have created three separate functions and triggers, but the performance (as to be expected) is horrendous. If I run a single trigger on update, insert, or delete, it performs fine. Below is a sample function and trigger I am using:
CREATE OR REPLACE FUNCTION refresh_mvw_taxa_hex5km()
RETURNS trigger
AS $BODY$
BEGIN
refresh materialized view mvw_taxa_hex5km;
return new;
END;
$BODY$ LANGUAGE plpgsql;
CREATE TRIGGER refresh_mvw_taxa_hex5km
AFTER INSERT OR UPDATE OR DELETE ON occurrence
FOR EACH STATEMENT
EXECUTE PROCEDURE refresh_mvw_taxa_hex5km();
Is there a more efficient way to do this? I considered running scheduled tasks, but I really need the refresh on changes to the table. I have read a little about "concurrently," but not sure if this is the answer.
Related
This question already has an answer here:
Drop trigger/function at end of session in PostgreSQL?
(1 answer)
Closed last month.
I'm trying to create a system in Postgres where each client can create its own subscriptions via listen + notify + triggers. I.e. the client will listen to a channel, and then create a trigger which runs notify on that channel when its conditions have been met. The issue is that I want Postgres to clean up properly in case of improper client termination (e.g. the remote client process dies). To be more specific, I want that trigger which is calling the notify to be removed as there is no longer a listener anyways. How can I accomplish this?
I've thought about having a table to map triggers to client ids and then using that to remove triggers where the client is gone, but it seems like a not so great solution.
I found an answer to this in another question: Drop trigger/function at end of session in PostgreSQL?
In reasonably recent Postgres versions you can create a function in
pg_temp schema:
create function pg_temp.get_true() returns boolean language sql as $$ select true; $$;
select pg_temp.get_true();
This is the schema in which temporary tables are created. All its
contents, including your function, will be deleted on end of session.
You can also create triggers using temporary functions on tables. I've
just tested this and it works as expected:
create function pg_temp.ignore_writes() returns trigger language plpgsql as $$
begin
return NULL;
end;
$$;
create table test (id int);
create trigger test_ignore_writes
before insert, update, delete on test
for each row execute procedure pg_temp.ignore_writes();
Because this trigger function always returns NULL and is before [event] it should make any writes to this table to be ignored. And
indeed:
insert into test values(1);
select count(*) from test;
count
-------
0
But after logout and login this function and the trigger would not be
present anymore, so writes would work:
insert into test values(1);
select count(*) from test;
count
-------
1
But you should be aware that this is somewhat hackish — not often used
and might not be very thoroughly tested.
That's not how it works. CREATE TRIGGER requires that you either own the table or have the TRIGGER privilege on it (which nobody in their right mind will give you, because it enables you to run arbitrary code in their name). Moreover, CREATE TRIGGER requires an SHARE ROW EXCLUSIVE lock on the table and DROP TRIGGER requires an ACCESS EXCLUSIVE lock, which can be disruptive.
Create a single trigger and keep that around.
I am trying to trigger a refresh for a materialised view after an Insert,update or delete on another materialised view. I am trying to use a triggered function to achieve this. Script is as follows;
CREATE FUNCTION aza_ods_version1.populate_fact_churn_m()
RETURNS TRIGGER AS $$
BEGIN
REFRESH MATERIALIZED VIEW aza_ods_version1.fact_churn_monthly;
RETURN NULL;
END;
$$ LANGUAGE plpgsql;
CREATE TRIGGER refresh_fact_churn_monthly
AFTER INSERT OR UPDATE OR DELETE OR TRUNCATE
ON aza_ods_version1.fact_transactions
FOR EACH ROW
WHEN (OLD.* IS DISTINCT FROM NEW.*)
EXECUTE PROCEDURE aza_ods_version1.populate_fact_churn_m();
On attempting to create the trigger, I get this error;
ERROR: "fact_transactions" is not a table or view
SQL state: 42809
I think its because fact_transactions is a materialized view and not a view or table.
Is there a way i can achieve what i am trying to??
You can't.
The documentation for CREATE TRIGGER doesn't mention materialized views:
table_name The name (optionally schema-qualified) of the table, view, or foreign table the trigger is for.
You cannot put a trigger on a materialized view, you will need to put the trigger on each table references in the query that generated the materialized. Further for TRUNCATE you will need to create a separate trigger:
In addition, triggers may be defined to fire for TRUNCATE, though only
FOR EACH STATEMENT.
I have been using Postgres for a while now, but I have not implemented any triggers yet. I wanted to check if this will do what I intend it to do.
On a daily basis, I am adding new rows to a table (COPY) and also updating existing rows if there is a primary key conflict (ON CONFLICT DO UPDATE SET). I then have a materialized view using that table and a few other joins, and this view is used for a lot of reporting.
I want the materialized view to update when the original table has been updated, without me needing to schedule it or run it manually. (Right now I have it scheduled with a Python psycopg2 execute command).
CREATE OR REPLACE FUNCTION refresh_mat_view()
RETURNS TRIGGER LANGUAGE plpgsql
AS $$
BEGIN
REFRESH MATERIALIZED VIEW schema_name.materialized_view_name;
RETURN NULL;
END $$;
CREATE TRIGGER refresh_view
AFTER INSERT OR UPDATE OR DELETE OR TRUNCATE
ON sutherland.dimension_peoplesoft FOR EACH STATEMENT
EXECUTE PROCEDURE refresh_mat_view();
Would that refresh the view for every single row which is updated too? I am just imagining it trigger a refresh for each individual row which might be 100k+. It would be better to happen AFTER all inserts have been done (I have Python looping through each row in a pandas DataFrame to UPSERT into the database).
When you use a pure Materialized View, every time you refresh it, it will rebuild the whole thing. So if your data changes a lot and you need the data available fast it's not an optimized choice.
You should use Eager Materialized Views or Lazy Materialized Views that basically are "A well use of triggers". Obviously it's harder to do than a pure materialized view, but the results are better (depending on the case of use).
You should check this article Materialized View Strategies Using PostgreSQL
In my PostgreSQL 9.1 database I have multiple tables and one trigger function.
Right now I am creating the trigger for each table by using that trigger function.
This methodology working fine. My boss has asked me to create the trigger commonly (only one time) by re-using that trigger function. That one trigger function should get used by all the tables in my database.
You can find an example of creating a trigger with dynamic SQL using PL/PgSQL in the Audit Trigger sample for PostgreSQL. The same approach will work with any other DDL.
See the function audit.audit_table and use of format and EXECUTE there.
That said, needing to create tables procedurally can be (but isn't always) a sign of questionable schema design.
Simple example of dynamic SQL creating a table:
CREATE OR REPLACE FUNCTION demo_dynamic_table(tablename text) RETURNS void AS $$
BEGIN
EXECUTE format('CREATE TABLE %I (id serial primary key);', tablename);
END;
$$ LANGUAGE plpgsql;
The same approach works for trigger creation, etc.
You can create PL/pgSQL procedure for table creation and move your trigger creation code inside it
i have a table "demand_details"
on update or delete i want to store values of each row in another table "demand_details_log"
my functions is as follows
CREATE OR REPLACE FUNCTION water_blogb() RETURNS trigger AS
$BODY$
BEGIN
IF (TG_OP='UPDATE') THEN
INSERT INTO demand_details_log VALUES ('U',now(),OLD.*);
RETURN NEW;
END IF;
IF (TG_OP='DELETE') THEN
INSERT INTO demand_details_log VALUES ('D',now(),OLD.*);
RETURN OLD;
END IF;
END;
$BODY$ LANGUAGE plpgsql
my trigger is as follows
CREATE TRIGGER water_btrg_b
AFTER UPDATE OR DELETE
ON demand_details
FOR EACH ROW
EXECUTE PROCEDURE water_blogb();
MY problem is the same trigger and functions works well on other table (by changing table,trigger and function name) but not working with demand table. I tried with "RAISE NOTICE 'working...'" in both in other table trigger gets fired but in demand table its not fired at all.
As you found, triggers are not inherited. This leads to some difficulties in managing triggers in inherited table structures. You may want to read up on some of the issues involved at http://ledgersmbdev.blogspot.com/2012/08/postgresql-or-modelling-part-3-table.html and http://ledgersmbdev.blogspot.com/2012/08/or-modelling-32-setsubset-modelling.html.
Now those do not address table partitioning directly which may be what you are trying to do here. I would recommend that you build in some additional tests that you can run to check and make sure that triggers are properly installed on all subtables. I would suggest taking a look at How to find inherited tables programatically in PostgreSQL? and also the pg_trigger table so that you can build a report of child tables which do not share the triggers of their parents.