how to write trigger in postgresql which maintain old value, new value and table name.
I have 5 tables and each tables is different data structure i want to maintain audit details in single table with old value new value and table name.old value and new value contain multiple columns in json format with column name and value.
example
audit_details
----------------------------------------------------------------
date_time|table_name|old_data|new_data|user|primary_key_of_table
----------------------------------------------------------------
I wrote sample trigger function for you. You can make additional changes yourself.
CREATE TABLE test.log_table_data (
id serial not null,
schema_name varchar(100) NOT NULL,
table_name varchar(100) NOT NULL,
action_date timestamp NOT NULL DEFAULT now(),
action_type varchar(10) NOT NULL,
table_id int4 NOT NULL,
old_data jsonb NULL,
new_data jsonb NULL,
CONSTRAINT log_table_data_pk PRIMARY KEY (id)
);
create or replace function test.register_as_log()
returns trigger
language plpgsql
security definer
as $function$
declare
v_old_data json;
v_new_data json;
shemaname varchar;
tablename varchar;
begin
shemaname = tg_table_schema;
tablename = tg_table_name;
/* json_strip_nulls - removes null values */
if (tg_op = 'UPDATE') then
v_old_data = (select json_strip_nulls(row_to_json(old.*)));
v_new_data = (select json_strip_nulls(row_to_json(new.*)));
insert into test.log_table_data (schema_name, table_name, action_type, table_id, old_data, new_data)
values (shemaname, tablename, 'update', old.id, v_old_data, v_new_data);
return new;
end if;
if (tg_op = 'DELETE') then
v_old_data = (select json_strip_nulls(row_to_json(old.*)));
insert into test.log_table_data (schema_name, table_name, action_type, table_id, old_data, new_data)
values (shemaname, tablename, 'delete', old.id, v_old_data, null);
return old;
end if;
end;
$function$
;
CREATE FUNCTION create_child1()
RETURNS TABLE(sys_user_id integer,
sys_service_id integer
)
LANGUAGE 'plpgsql'
COST 100
VOLATILE
ROWS 1000
AS $BODY$
DECLARE
curr_id CURSOR IS
SELECT id FROM users WHERE id in (3089,3090,3091,3092);
v_id bigint;
BEGIN
OPEN curr_id;
LOOP
FETCH curr_id INTO v_id;
EXIT WHEN not found ;
EXECUTE format('
CREATE TABLE IF NOT EXISTS %I (
sys_user_id integer,
sys_service_id integer
id bigint NOT NULL primary key
)
INHERITS (telemetry_master)
WITH (
OIDS=FALSE
)', 'telemetry_' || v_id);
end loop;
close curr_id;
fetch next from curr_id into v_id;
END
$BODY$ LANGUAGE plpgsql;
You do not need an explicit cursor in your function. You can use a simple FOR ... IN ... LOOP.
It is unclear what you want to return from the function. For example, it can return a readable text about each created table.
CREATE OR REPLACE FUNCTION create_child1()
RETURNS SETOF text LANGUAGE plpgsql
AS $BODY$
DECLARE
v_id int;
BEGIN
FOR v_id IN 3089..3092 LOOP
EXECUTE format('
CREATE TABLE IF NOT EXISTS telemetry_%s (
sys_user_id integer,
sys_service_id integer,
id bigint NOT NULL primary key
)
INHERITS (telemetry_master)', v_id);
RETURN NEXT format('telemetry_%s created.', v_id);
END LOOP;
END $BODY$;
Use:
SELECT create_child1();
create_child1
-------------------------
telemetry_3089 created.
telemetry_3090 created.
telemetry_3091 created.
telemetry_3092 created.
(4 rows)
If the ids are not consecutive you can use unnest(), e.g.:
FOR v_id IN SELECT id FROM unnest(array[3000,3001,3020,3021]) AS id LOOP
I have the following small MVWE for a basic micro-auditing system which works fine but lacks a functionality:
DROP TABLE IF EXISTS audit CASCADE;
CREATE TABLE audit(
Id BIGSERIAL NOT NULL
,TimeValue TIMESTAMP NOT NULL
,RoleName NAME NOT NULL
,Operation NAME NOT NULL
,SchemaName NAME NOT NULL
,TableName NAME NOT NULL
,Identifiers BIGINT[]
---
,PRIMARY KEY(Id)
);
-- Audit Trigger:
DROP FUNCTION IF EXISTS audit_trigger() CASCADE;
CREATE OR REPLACE FUNCTION audit_trigger()
RETURNS TRIGGER AS
$BODY$
BEGIN
INSERT INTO audit(TimeValue, RoleName, Operation, SchemaName, TableName) VALUES
(now()::TIMESTAMP, current_user, TG_OP, TG_TABLE_SCHEMA, TG_RELNAME);
RETURN NULL;
END;
$BODY$
LANGUAGE plpgsql SECURITY DEFINER;
-- Channels:
DROP TABLE IF EXISTS channels CASCADE;
CREATE TABLE channels(
Id INTEGER NOT NULL
,UserKey TEXT NOT NULL
,Active BOOLEAN NOT NULL DEFAULT(TRUE)
---
,PRIMARY KEY(Id)
,UNIQUE(UserKey)
);
CREATE TRIGGER channel_audit_trigger BEFORE INSERT OR UPDATE OR DELETE ON channels
FOR EACH STATEMENT EXECUTE PROCEDURE audit_trigger();
-- Perform some operations:
INSERT INTO channels(
SELECT C.Id, 'Channel-' || C.Id
FROM generate_series(1, 300, 10) AS C(Id)
);
DELETE FROM channels WHERE id < 10;
UPDATE channels
SET UserKey = 'wild channel'
WHERE id = 21;
I would like to add into the last column of audit table, identifiers of rows that have been inserted/updated/deleted in channels.
I have used STATEMENT level because I just need to collects identifiers in a array. But I do not find how to access DML statistics. Conversely at the ROW level I must handle OLD and NEW cases and I cannot succeed in aggregate all touched identifier.
How can I proceed in order to fill the last column of audit table with touched identifiers?
Update
Finally I reached my goal, but this solution might not be scalable and may have some unwanted drawbacks (I am open to any constructive feedback and advice).
Basically, how I have solved my problem:
Log at ROW level BEFORE DML is performed into a table audit_rowlevel;
Aggregate the freshly added content of audit_rowlevel into audit_statementlevel at STATEMENT level AFTER DML is performed;
Minimal Working Example is now:
DROP TABLE IF EXISTS audit_rowlevel CASCADE;
CREATE TABLE audit_rowlevel(
Id BIGSERIAL NOT NULL
,Aggregated BOOLEAN NOT NULL DEFAULT(FALSE)
,TimeValue TIMESTAMP NOT NULL
-- https://www.postgresql.org/docs/current/static/functions-info.html
,RoleName NAME NOT NULL
,ClientIP INET NOT NULL
,ClientPid INTEGER NOT NULL
-- https://www.postgresql.org/docs/current/static/plpgsql-trigger.html
,Operation TEXT NOT NULL
,SchemaName NAME NOT NULL
,TableName NAME NOT NULL
,RowId BIGINT NOT NULL
-- https://www.postgresql.org/docs/current/static/functions-json.html
,OldValue JSONB
,NewValue JSONB
---
,PRIMARY KEY(Id)
);
-- Row Level Trigger:
DROP FUNCTION IF EXISTS audit_rowlevel_trigger() CASCADE;
CREATE OR REPLACE FUNCTION audit_rowlevel_trigger()
RETURNS TRIGGER AS
$BODY$
DECLARE
history BOOLEAN := (TG_NARGS > 0) AND (TG_ARGV[0]::BOOLEAN);
rowid BIGINT;
oldvalue JSONB;
newvalue JSONB;
BEGIN
-- Handle NEW:
IF TG_OP = ANY('{INSERT,UPDATE}') THEN
IF history THEN
newvalue := to_jsonb(NEW);
END IF;
rowid := NEW.Id::BIGINT;
END IF;
-- Handle OLD:
IF TG_OP = ANY('{UPDATE,DELETE}') THEN
IF history THEN
oldvalue := to_jsonb(OLD);
END IF;
rowid := OLD.Id::BIGINT;
END IF;
-- INSERT:
INSERT INTO audit_rowlevel(TimeValue, RoleName, ClientIP, ClientPID, Operation, SchemaName, TableName, RowId, NewValue, OldValue) VALUES
(now()::TIMESTAMP, current_user, inet_client_addr(), pg_backend_pid(), TG_OP, TG_TABLE_SCHEMA, TG_RELNAME, RowId, NewValue, OldValue);
-- RETURN:
IF TG_OP = ANY('{INSERT,UPDATE}') THEN
RETURN NEW;
ELSIF TG_OP = 'DELETE' THEN
RETURN OLD;
ELSE
RETURN NULL;
END IF;
END;
$BODY$
LANGUAGE plpgsql SECURITY DEFINER;
-- Statement Level Trigger:
DROP TABLE IF EXISTS audit_statementlevel CASCADE;
CREATE TABLE audit_statementlevel(
Id BIGSERIAL NOT NULL
,TimeValue TIMESTAMP NOT NULL
,RoleName NAME NOT NULL
,ClientIP INET NOT NULL
,ClientPid INTEGER NOT NULL
,Operation TEXT NOT NULL
,SchemaName NAME NOT NULL
,TableName NAME NOT NULL
,RowCount BIGINT NOT NULL
,RowIds BIGINT[] NOT NULL
,AuditIds BIGINT[] NOT NULL
---
,PRIMARY KEY(Id)
);
-- Row Level Trigger:
DROP FUNCTION IF EXISTS audit_statementlevel_trigger() CASCADE;
CREATE OR REPLACE FUNCTION audit_statementlevel_trigger()
RETURNS TRIGGER AS
$BODY$
DECLARE
rowcount BIGINT;
BEGIN
WITH
A AS (
SELECT
TimeValue, RoleName, ClientIP, ClientPid, Operation, SchemaName, TableName
,COUNT(*)
,array_agg(RowId)
,array_agg(Id)
FROM
audit_rowlevel
WHERE
NOT Aggregated
GROUP BY
TimeValue, RoleName, ClientIP, ClientPid, Operation, SchemaName, TableName
ORDER BY
TimeValue
),
B AS (
INSERT INTO audit_statementlevel(TimeValue, RoleName, ClientIP, ClientPid, Operation, SchemaName, TableName, RowCount, RowIds, AuditIds)
(SELECT * FROM A)
RETURNING AuditIds
),
C AS (
SELECT array_agg(DISTINCT T.id) AS Ids FROM B, unnest(B.AuditIds) AS T(id)
)
UPDATE
audit_rowlevel
SET
Aggregated = TRUE
FROM
C
WHERE
Id = ANY(C.Ids);
RETURN NULL;
END;
$BODY$
LANGUAGE plpgsql SECURITY DEFINER;
-- Channels:
DROP TABLE IF EXISTS channels CASCADE;
CREATE TABLE channels(
Id INTEGER NOT NULL
,UserKey TEXT NOT NULL
,Active BOOLEAN NOT NULL DEFAULT(TRUE)
---
,PRIMARY KEY(Id)
,UNIQUE(UserKey)
);
CREATE TRIGGER channel_audit_rowlevel_trigger BEFORE INSERT OR UPDATE OR DELETE ON channels
FOR EACH ROW EXECUTE PROCEDURE audit_rowlevel_trigger(TRUE);
CREATE TRIGGER channel_audit_statementlevel_trigger AFTER INSERT OR UPDATE OR DELETE ON channels
FOR EACH STATEMENT EXECUTE PROCEDURE audit_statementlevel_trigger();
-- Perform some operations:
INSERT INTO channels(
SELECT C.Id, 'Channel-' || C.Id
FROM generate_series(1, 300, 10) AS C(Id)
);
DELETE FROM channels WHERE id < 10;
UPDATE channels
SET UserKey = 'wild channel'
WHERE id = 21;
I am interested knowing if this solution looks right to professional developers.
Am I going into the good direction or is this solution evil?
CREATE OR REPLACE FUNCTION public.merge_test (
r_obj refcursor,
_ldeptid character varying
) RETURNS refcursor
LANGUAGE 'plpgsql' COST 100.0 VOLATILE AS $function$
BEGIN
DROP TABLE IF EXISTS tblCumulate;
create temp table tblCumulate (
lCompid varchar(10),
lOpenCount int default 0,
lClosedCount int default 0
);
DROP TABLE IF EXISTS tblOpen;
create temp table tblOpen (
lOSID SERIAL,
lCount numeric(24,0),
lCompid varchar(100)
);
MERGE into tblCumulate CUM using (select lcompid,lCount from tblopen) as OP
on CUM.lcompid=OP.lcompid
when matched
then update set cum.lOpenCount=op.lcount
when not matched
then insert (lCompid,lOpenCount) values op.lcompid,op.lcount);
open r_obj for
select * from tblCumulate;
return r_obj;
END;
$function$;
when I execute (Run) this procedure showing following error.
ERROR: "tblcumulate" is not a known variable
LINE 41: MERGE into tblCumulate CUM temp
There is no MERGE statement in PostgreSQL.
Consider using INSERT ... ON CONFLICT.
Here is the complete example code:
CREATE TABLE testtbl (
id integer NOT NULL,
intval integer,
strval varchar(64)
);
CREATE SEQUENCE testtbl_id_seq
START WITH 1 INCREMENT BY 1
NO MINVALUE NO MAXVALUE CACHE 1;
ALTER SEQUENCE testtbl_id_seq OWNED BY testtbl.id;
ALTER TABLE ONLY testtbl ALTER COLUMN id SET DEFAULT
nextval('testtbl_id_seq'::regclass);
ALTER TABLE ONLY testtbl ADD CONSTRAINT testtbl_pkey PRIMARY KEY (id);
CREATE FUNCTION insert_testtbl (p_intval integer, p_strval varchar(64))
RETURNS integer AS $$
DECLARE
v_new_id integer;
BEGIN
INSERT INTO testtbl (intval, strval) VALUES (p_intval, p_strval)
RETURNING v_new_id;
RETURN v_new_id;
END;
$$ LANGUAGE plpgsql;
SELECT insert_testtbl(1, 'One');
When I run this (PostgreSQL version is 9.6.1), I get:
ERROR: query has no destination for result data
CONTEXT: PL/pgSQL function insert_testtbl(integer,character varying) line 5 at SQL statement
This doesn't make sense; I AM specifying a destination for the result!
What am I doing wrong here? Thanks!!!
I am specifying a destination for the result!
No you are not.
RETURNING v_new_id; simply means:
"return the current value of the variable v_new_id from this insert statement"
(which is null as the variable was never assigned a value)
You are not storing the generated value anywhere.
You either need to use an into clause:
CREATE FUNCTION insert_testtbl (p_intval integer, p_strval varchar(64))
RETURNS integer AS $$
DECLARE
v_new_id integer;
BEGIN
INSERT INTO testtbl (intval, strval) VALUES (p_intval, p_strval)
RETURNING id
INTO v_new_id; --<<< HERE
RETURN v_new_id;
END;
$$ LANGUAGE plpgsql;
Or convert everything it simple SQL function:
CREATE FUNCTION insert_testtbl (p_intval integer, p_strval varchar(64))
RETURNS integer AS
$$
INSERT INTO testtbl (intval, strval) VALUES (p_intval, p_strval)
RETURNING id;
$$ LANGUAGE sql;