Having multiple trigger events when redirecting insertions to partition tables - postgresql

I am trying to set up triggers for insert and update events for the master table of some partition tables in PostgreSQL. Each time an insertion is made into the master table, the insert trigger event will redirect it into the correct partition table. Consequently, I will need to return NULL from this function call, since I don't want the master table to be populated as well. If the master table receives an update event, it will update a timestamp before making the change in the table. The problem is that the update trigger is never fired. I am using PostgreSQL version 9.6.
I have tried to combine the trigger functions into one, and merged the called trigger procedures into one as well, but the results are the same. The update trigger is only triggered if I return NEW from the insertion trigger function (which populates the master table), or if I comment out the insertion trigger function altogether.
DROP SCHEMA IF EXISTS test CASCADE;
CREATE SCHEMA test;
SET SCHEMA 'test';
CREATE TYPE test_type AS ENUM ('unit', 'performance');
CREATE TABLE test (
type test_type NOT NULL,
score INTEGER NOT NULL CHECK (score > 0),
id SERIAL PRIMARY KEY,
updated_at TIMESTAMP DEFAULT current_timestamp
);
CREATE TABLE performance_test (
CHECK (type = 'performance')
) INHERITS (test);
CREATE FUNCTION insert_test()
RETURNS trigger AS
$$
BEGIN
INSERT INTO performance_test VALUES (NEW.*);
RETURN NULL;
END;
$$ LANGUAGE plpgsql;
CREATE FUNCTION update_timestamp()
RETURNS trigger AS
$$
BEGIN
RAISE NOTICE 'This is never reached.';
UPDATE performance_test
SET updated_at = current_timestamp
WHERE id = NEW.id;
RETURN NULL;
END;
$$ LANGUAGE plpgsql;
CREATE TRIGGER test_insertion BEFORE INSERT ON test
FOR EACH ROW EXECUTE PROCEDURE insert_test();
CREATE TRIGGER test_update BEFORE UPDATE ON test
FOR EACH ROW EXECUTE PROCEDURE update_timestamp();
---------------------------------------------------------------------------
INSERT INTO test VALUES ('performance', 10);
SELECT * FROM performance_test;
UPDATE test SET score = 20 WHERE id = 1;
SELECT * FROM performance_test;
I am not sure if it is possible to achieve what I want with this method, so I'm reaching out here for any advice. Thanks in advance!
/ Hampus

Row triggers must be defined on individual partitions, not the partitioned table. See https://www.postgresql.org/docs/10/ddl-partitioning.html#DDL-PARTITIONING-DECLARATIVE-LIMITATIONS
I don't know why the documentation for 9.6 doesn't mention this
working update trigger:
CREATE FUNCTION update_timestamp()
RETURNS trigger AS
$$
BEGIN
NEW.updated_at = now();
RETURN NEW;
END;
$$ LANGUAGE plpgsql;
CREATE TRIGGER test_update BEFORE UPDATE ON performance_test
FOR EACH ROW EXECUTE PROCEDURE update_timestamp();
if you do UPDATE test SET score = 30, updated_at=DEFAULT; or UPDATE test SET score = 30, updated_at=current_timestamp; you might not need the update trigger.
Partitioning is not a free lunch because it has non-obvious effects on both behavior and performance, as you noticed by the trigger not behaving as you expected. If you make a mistake it can easily lead to failing queries and even bad data.
If you are really sure you need it you should make sure you understand it in detail and otherwise I'd recommend you to avoid it, most issues with slow queries can be solved by making sure the table statistics is up to date, using the right indexes, optimizing queries, changing Postgres configuration or adding more hardware.

Related

SELECT in cascaded AFTER DELETE trigger returning stale data in Postgres 11

I have an AFTER INSERT/UPDATE/DELETE trigger function which runs after any change to table campaigns and triggers an update on table contracts:
CREATE OR REPLACE FUNCTION update_campaign_target() RETURNS trigger AS $update_campaign_target$
BEGIN
UPDATE contracts SET updated_at = now() WHERE contracts.contract_id = NEW.contract_id;
END;
$update_campaign_target$ LANGUAGE plpgsql;
DROP TRIGGER IF EXISTS update_campaign_target ON campaigns;
CREATE TRIGGER update_campaign_target AFTER INSERT OR UPDATE OR DELETE ON campaigns
FOR EACH ROW EXECUTE PROCEDURE update_campaign_target();
I have another trigger on table contracts that runs BEFORE UPDATE. The goal is to generate a computed column target which displays either contracts.manual_target (if set) or SUM(campaigns.target) WHERE campaign.contract_id = NEW.contract_id.
CREATE OR REPLACE FUNCTION update_contract_manual_target() RETURNS trigger AS $update_contract_manual_target$
DECLARE
campaign_target_count int;
BEGIN
IF NEW.manual_target IS NOT NULL
THEN
NEW.target := NEW.manual_target;
RETURN NEW;
ELSE
SELECT SUM(campaigns.target) INTO campaign_target_count
FROM campaigns
WHERE campaigns.contract_id = NEW.contract_id;
NEW.target := campaign_target_count;
RETURN NEW;
END IF;
END;
$update_contract_manual_target$ LANGUAGE plpgsql;
DROP TRIGGER IF EXISTS update_contract_manual_target ON contracts;
CREATE TRIGGER update_contract_manual_target BEFORE INSERT OR UPDATE ON contracts
FOR EACH ROW EXECUTE PROCEDURE update_contract_manual_target();
This works as expected on INSERT and UPDATE on campaigns, but does not work on DELETE. When a campaign is deleted, the result of SUM(campaigns.target) in the second trigger includes the deleted campaign's target, and thus does not update the contracts.target column to the expected value. A second update of contracts will correctly set the value.
Three questions:
Why doesn't this work?
Is there a way to achieve the behavior I'm looking for using triggers?
For this type of data synchronization, is it better to achieve this using triggers or views? Triggers make sense to me because this is a table that we will read many magnitudes of times more than we'll write to it, but I'm not sure what the best practices are.
The reason this doesn't work is the usage of NEW.contract_id in the AFTER DELETE trigger:
UPDATE contracts SET updated_at = now() WHERE contracts.contract_id = NEW.contract_id;
Per the Triggers on Data Changes documentation, NEW is NULL for DELETE triggers.
Updating the code to use OLD instead of NEW fixes the issue:
CREATE OR REPLACE FUNCTION update_campaign_target() RETURNS trigger AS $update_campaign_target$
BEGIN
IF TG_OP = 'DELETE'
THEN
UPDATE contracts SET updated_at = now() WHERE contracts.contract_id = OLD.contract_id;
ELSE
UPDATE contracts SET updated_at = now() WHERE contracts.contract_id = NEW.contract_id;
END IF;
RETURN NULL;
END;
$update_campaign_target$ LANGUAGE plpgsql;
Thanks to Anthony Sotolongo and Belayer for your help!

PostgreSQL Trigger that updates table where original trigger runs from

I have two tables in this scenario. One is my "hot sync" table which is near-realtime bi-directional sync of data from my Salesforce Org to a Postgres table. As data changes in the source system (Salesforce), it updates that table on Postgres.
On this table in Postgres, I have a trigger that runs some logic. It basically checks to see if the record triggering it has a sent date that meets some business logic, copy that row into another schema/table to "archive" it.
This all works fine.
What I need to do however is once this row has been copied into the other table, I need to update the status of the record hot sync table. Since it is bi-directional, this will allow the data in Salesforce to reflect the changes I make from the Postgres side.
Can I place this update statement within the originating trigger or is this going to cause recursion issues?
CREATE FUNCTION salesforce.archivelogicfunc()
RETURNS trigger
LANGUAGE 'plpgsql'
COST 100
VOLATILE NOT LEAKPROOF
AS $BODY$ BEGIN
IF (DATE(NEW.et4ae5__datesent__c) < NOW() - INTERVAL '180 days'
AND DATE(NEW.et4ae5__datesent__c) > NOW() - INTERVAL '540 days')
THEN
INSERT INTO archive.individualemailresult__c
(dateopened__c,
numberoftotalclicks__c,
datebounced__c,
fromname__c,
hardbounce__c,
fromaddress__c,
softbounce__c,
name,
lastmodifieddate,
opened__c,
ownerid,
subjectline__c,
isdeleted,
contact__c,
systemmodstamp,
lastmodifiedbyid,
datesent__c,
dateunsubscribed__c,
createddate,
createdbyid,
lead__c,
tracking_as_of__c,
numberofuniqueclicks__c,
senddefinition__c,
mergeid__c,
triggeredsenddefinition__c,
sfid,
id,
_hc_lastop,
_hc_err,
isarchived)
VALUES
(NEW.et4ae5__dateopened__c,
NEW.et4ae5__numberoftotalclicks__c,
NEW.et4ae5__datebounced__c,
NEW.et4ae5__fromname__c,
NEW.et4ae5__hardbounce__c,
NEW.et4ae5__fromaddress__c,
NEW.et4ae5__softbounce__c,
NEW.name,
NEW.lastmodifieddate,
NEW.et4ae5__opened__c,
NEW.ownerid,
NEW.et4ae5__subjectline__c,
NEW.isdeleted,
NEW.et4ae5__contact__c,
NEW.systemmodstamp,
NEW.lastmodifiedbyid,
NEW.et4ae5__datesent__c,
NEW.et4ae5__dateunsubscribed__c,
NEW.createddate,
NEW.createdbyid,
NEW.et4ae5__lead__c,
NEW.et4ae5__tracking_as_of__c,
NEW.et4ae5__numberofuniqueclicks__c,
NEW.et4ae5__senddefinition__c,
NEW.et4ae5__mergeid__c,
NEW.et4ae5__triggeredsenddefinition__c,
NEW.sfid,
NEW.id,
NEW._hc_lastop,
NEW._hc_err,
NEW.isarchived__c)
ON CONFLICT (id)
DO NOTHING;
-- Update SF to reflect the archive
UPDATE salesforce."et4ae5__individualemailresult__c" SET isarchived__c = true, isdeleted = true WHERE id = NEW.id;
END IF;
RETURN NULL;
END;
$BODY$;
ALTER FUNCTION salesforce.archivelogicfunc()
OWNER TO ....;
My understanding is that the NEW.* is only going to contain the rows that caused the trigger to fire in the first place. Therefore if my trigger was fired for a single record, the update statement NEW.id should only update one record on the source table?
Trying to ensure the trigger isn't going to fire again with the update statement causing some recursive loop that I am not expecting.
My concern is:
Record is Updated
Trigger Fires and inserts record into an archive table
Update runs on the source table to update the record for the new.id
This update causes the trigger to run again. The insert would fail due to the on conflict, but the update would then run again, and again etc..
The original trigger is fired AFTER INSERT/UPDATE.
TRIGGER:
CREATE TRIGGER archivelogic_firetrigger
AFTER INSERT OR UPDATE
ON salesforce.et4ae5__individualemailresult__c
FOR EACH ROW
EXECUTE PROCEDURE salesforce.archivelogicfunc();
UPDATE:
I added a WHEN condition to my trigger. It appeared to work on a basic test, but willing to take any other advice if suggested.
CREATE TRIGGER archivelogic_firetrigger
AFTER INSERT OR UPDATE
ON salesforce.et4ae5__individualemailresult__c
FOR EACH ROW
WHEN (pg_trigger_depth() = 0) // <-- Added to prevent recursion
EXECUTE PROCEDURE salesforce.archivelogicfunc();
The easiest would be to make it a before trigger, and to replace the update by
NEW.isarchived__c = true;
NEW. isdeleted = true;
[...]
RETURN NEW;
Otherwise, you can filter the rows before running the trigger: it will be called only when isarchived__c and isdeleted have NOT changed (it may be dangerous though, just imagine someone updating ALL fields)
CREATE TRIGGER archivelogic_firetrigger
AFTER INSERT OR UPDATE
ON salesforce.et4ae5__individualemailresult__c
FOR EACH ROW
WHEN (NEW.isarchived__c IS NOT DISTINCT FROM OLD.isarchived__c
AND NEW.isdeleted IS NOT DISTINCT FROM OLD.isdeleted )
EXECUTE PROCEDURE salesforce.archivelogicfunc();

postgres trigger function only inserts few records in another table

I have 2 tables hourly and daily and my aim is to calculate average of values from hourly table and save it to daily table. I have written a trigger function like this -
CREATE OR REPLACE FUNCTION public.calculate_daily_avg()
RETURNS trigger AS
$BODY$
DECLARE chrly CURSOR for
SELECT device, date(datum) datum, avg(cpu_util) cpu_util
FROM chourly WHERE date(datum) = current_date group by device, date(datum);
BEGIN
FOR chrly_rec IN chrly
LOOP
insert into cdaily (device, datum, cpu_util)
values (chrly_rec.device, chrly_rec.datum, chrly_rec.cpu_util)
on conflict (device, datum) do update set
device=chrly_rec.device, datum=chrly_rec.datum, cpu_util=chrly_rec.cpu_util;
return NEW;
END LOOP;
EXCEPTION
WHEN NO_DATA_FOUND THEN
RAISE NOTICE 'NO DATA IN chourly FOR %', current_date;
END;
$BODY$
LANGUAGE plpgsql VOLATILE
COST 100;
ALTER FUNCTION public.calculate_daily_avg()
OWNER TO postgres;
and a trigger on hourly table like this -
CREATE TRIGGER calculate_daily_avg_trg
BEFORE INSERT OR UPDATE
ON public.chourly
FOR EACH ROW
EXECUTE PROCEDURE public.calculate_daily_avg();
But when I try to insert or update about 3000 records in the hourly table only 3 or 4 devices are inserted and not 3000. (also in trigger I have already tried AFTER INSERT OR UPDATE but even that gives the same result) What I am doing wrong here? Please suggest any better way to write the trigger if you feel I have written it wrongly. Thanks!
I don't suggest using TRIGGER for calculation when INSERT. Try different approach using function execute by cron hourly or daily.
WHY?
Because everytime you INSERT one row. the postgres will always do aggregate function AVG() and LOOPING for insert (based on your flow).
That mean another INSERT statement will waiting until Previous INSERT commited that will suffer your database performance for highly INSERT Transaction. If somehow you manage to BREAK the rule (maybe from configuration) you will get inconsistent data like what happened to you right now.

PgSQL log table update time

I've created the following table:
CREATE TABLE updates
(
"table" text,
last_update timestamp without time zone
)
I want to update it whenever any table is updated, the problem is I don't know how, could someone please help me turn this pseudocode into a trigger?
this = current table on whitch operation is performed
ON ALTER,INSERT,DELETE {
IF (SELECT COUNT(*) FROM updates where table = this) = 1
THEN
UPDATE updates SET last_update = timeofday()::timestamp WHERE `table`=this
ELSE
INSERT INTO updates VALUES (this,timeofday()::timestamp);
}
You need a trigger function that is called whenever one of your tables is "updated", assuming that you mean that an INSERT, UPDATE, or DELETE is successfully executed. That trigger function would look like this:
CREATE FUNCTION log_update() RETURNS trigger AS $$
BEGIN
UPDATE updates SET last_update = now() WHERE "table" = TG_TABLE_NAME;
IF NOT FOUND THEN
INSERT INTO updates VALUES (TG_TABLE_NAME, now());
END IF;
IF (TG_OP = 'DELETE') THEN
RETURN OLD;
ELSE
RETURN NEW;
END IF;
END; $$ LANGUAGE PLPGSQL;
Every table that has to be logged this way needs to have a trigger associated with it like this:
CREATE TRIGGER ZZZ_mytable_log_updates
AFTER INSERT OR UPDATE OR DELETE ON mytable
FOR EACH ROW EXECUTE PROCEDURE log_update();
A few comments:
Trigger functions are created with PL/PgSQL; see chapter 40 in the documentation. Trigger functions come with some automatic parameters such as TG_TABLE_NAME.
Don't use reserved words ("table" in your case) as column names. Actually, in this case you are better off using the oid of the table, with the associated TG_RELID automatic parameter. It takes up less storage, it is faster, and it avoids confusion between tables with the same name in different schemas of your database. You can use the pg_tables system catalog table to look up the table name from the oid.
You must return the proper value depending on the operation, or the operation may fail. INSERT and UPDATE operations need to have NEW returned; DELETE needs to have OLD returned.
The name of the trigger starts with "ZZZ" to make sure that it fires after any other triggers on the same table have succeeded (they are fired in alphabetical order). If a prior trigger fails, this trigger function will not be called, which is the proper behaviour because the insert, update or delete will not take place either.

How to create a Trigger in PostgreSql?

TRIGEER-->To get a column value from one table to other table when i insert values?
I am having two tables(customer_details and loan_balance).
What i need is, I must get the column (custid)of customer_details table to the loan_balance table when i insert the data into the loan_balance table.
This is the full set up of my query : SQL FIDDLE
So i need a trigger to be raised and the data should be updated automatically without dynamic insertion of custid.
Postgres has an unconventional way of creating triggers:
create a function that returns type trigger and return the NEW row record
create a trigger that executes the function
Here's the code you need:
CREATE FUNCTION synch_custid_proc()
RETURNS trigger AS $$
BEGIN
NEW.custid = (
select max(custid)
from customer_details
where creditid = NEW.creditid
);
RETURN NEW;
END;
$$ LANGUAGE plpgsql
CREATE TRIGGER synch_custid_trig
BEFORE INSERT ON loan_amount
FOR EACH ROW
EXECUTE PROCEDURE synch_custid_proc();
I chosen to select max(custid) rather than simply custid when finding the value in case there are multiple rows that match. You might have to adjust this logic to suit your data.
See a live demo on SQLFiddle