We are looking to convert our database over to Postgres (9.3.5), which I have no experience with, and I am trying to get our audit tables up and running. I understand that each table will need its own trigger, but all triggers can call a single function.
The trigger on the table is passing a list of the columns that need to be audited since some of our columns are not tracked.
Here are some of the posts I followed:
- https://stackoverflow.com/a/7915100/229897
- http://www.postgresql.org/docs/9.3/static/plpgsql-statements.html
- http://www.postgresql.org/docs/9.4/static/plpgsql-trigger.html
When I run this I get the error: ERROR: syntax error at or near "$1"
DROP TABLE IF EXISTS people;
DROP TABLE IF EXISTS a_people;
CREATE TABLE IF NOT EXISTS people (
record_id SERIAL PRIMARY KEY NOT NULL,
first_name VARCHAR NOT NULL,
last_name VARCHAR NOT NULL,
last_updated_on TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP
);
CREATE TABLE IF NOT EXISTS a_people (
record_id SERIAL PRIMARY KEY NOT NULL,
a_record_id INT,
first_name VARCHAR NULL,
last_name VARCHAR NULL,
last_updated_on TIMESTAMP
);
/******************************************************/
--the function
CREATE OR REPLACE FUNCTION audit_func()
RETURNS TRIGGER AS
$BODY$
DECLARE
audit TEXT := TG_TABLE_SCHEMA || '.a_' || TG_TABLE_NAME;
cols TEXT := TG_ARGV[0];
BEGIN
EXECUTE format('INSERT INTO %1$s(a_%2$s) SELECT %2$s FROM ($1)', audit, cols) USING OLD;
NEW.last_updated_on = CURRENT_TIMESTAMP;
RETURN NEW;
END;
$BODY$
LANGUAGE plpgsql;
/******************************************************/
--the trigger calling the function to update inbound records
CREATE TRIGGER build_user_full_name_trg
BEFORE UPDATE
ON people
FOR EACH ROW WHEN (OLD.* IS DISTINCT FROM NEW.*)
EXECUTE PROCEDURE audit_func('record_id,first_name,last_name');
/******************************************************/
INSERT INTO people (first_name, last_name) VALUES ('George','Lincoln');
UPDATE people SET last_name = 'Washington' WHERE first_name = 'George';
SELECT * FROM people;
I welcome your assistance (and patience)!
This subselect should work:
EXECUTE format('INSERT INTO %1$s(a_%2$s) SELECT %2$s FROM (select ($1).*) XX', audit, cols) USING OLD;
Related
Let's say I have several users, each with their own set of contacts. Users get to select which 3rd parties have access to their contacts.
I could maybe create 1 large contact table and have an 'owner' column so that I can identify which contacts belong to which users. However, I'd then need to maintain row-level permissions so that I can restrict which 3rd parties do/don't have access to specific user contacts.
Instead, I think (and I could be mistaken here) it makes more sense to have a contact table for each user, e.g.: contact_e878df81_eba1_4a61_b592_30ac7100362a. I could then manage permissions in a separate table.
To create these 'dynamic contact tables', I have the following function:
CREATE OR REPLACE FUNCTION create_contact_table(IN tbl_name text) RETURNS INT AS $$
DECLARE
table_name text;
BEGIN
table_name = tbl_name;
create table IF NOT EXISTS table_name
(
id serial not null
constraint test_pkey
primary key,
firstname varchar not null,
lastname varchar not null,
age int not null,
address varchar not null,
email varchar not null,
created timestamp default CURRENT_TIMESTAMP
);
alter table table_name
owner to postgres;
RETURN 1;
END ;
$$ LANGUAGE plpgsql;
When I execute the function (create_contact_table(contact_e878df81_eba1_4a61_b592_30ac7100362a), the table is created, but the name of the table is table_name...
Why is that? How can I fix it so that the table name is correct?
You can do it only using dynamic SQL, for example:
CREATE OR REPLACE FUNCTION create_contact_table(tbl_name text)
RETURNS boolean
LANGUAGE plpgsql
AS $function$
declare
v_sql text;
t1 text;
begin
v_sql = '
create table %I
(
id serial not null
constraint test_pkey
primary key,
firstname varchar not null,
lastname varchar not null,
age int not null,
address varchar not null,
email varchar not null,
created timestamp default CURRENT_TIMESTAMP
);
';
raise notice 'SQL:: %', v_sql;
EXECUTE format(v_sql, tbl_name);
return true;
END;
$function$
;
I want to use PostgreSQL's declarative table partitioning on a column with UUID values. I want it partitioned by values that are returned by a SELECT.
CREATE TABLE foo (
id uuid NOT NULL,
type_id uuid NOT NULL,
-- other columns
PRIMARY KEY (id, type_id)
) PARTITION BY LIST (type_id);
CREATE TABLE foo_1 PARTITION OF foo
FOR VALUES IN (SELECT id FROM type_ids WHERE type_name = 'type1');
CREATE TABLE foo_2 PARTITION OF foo
FOR VALUES IN (SELECT id FROM type_ids WHERE type_name = 'type2');
I don't want to use specific UUIDs in FOR VALUES IN ('uuid'), since the UUIDs may differ by environment (dev, qa, prod). However, the SELECT syntax doesn't seem to be accepted by Postgres. Any suggestions?
I just wanted the SELECT to be evaluated at the table creation time
You should have made that clear in the question, not in a comment.
In that case - if this is a one time thing, you can use dynamic SQL, e.g. wrapped into a procedure
create procedure create_partition(p_part_name text, p_type_name text)
as
$$
declare
l_sql text;
l_id uuid;
begin
select id
into l_id
from type_ids
where type_name = p_type_name;
l_sql := format('CREATE TABLE %I PARTITION OF foo FOR VALUES IN (%L)', p_part_name, l_id);
execute l_sql;
end;
$$
language plpgsql;
Then you can do:
call create_partition('foo_1', 'type1');
call create_partition('foo_2', 'type2');
I am trying to write a stored procedure where the table name comes dynamically.
Also it has to check whether the table already exists and create only if it does not exist.
Then later I am trying to insert data into the table like below.
Here I am passing pkey and filedata as parameters to insert query in which pkey is a string and filedata is a json data which looks like { "customer": "John Doe", "items": {"product": "Beer","qty": 6}}
I have tried the below query but the table is not getting created it is giving the message
Notice: identifier public.tablename_11111 will be truncated to public.tablename_11111
here the table name is public.tablename_11111
CREATE OR REPLACE FUNCTION public.generate_table(tb_name text)
RETURNS text LANGUAGE 'plpgsql'
COST 100 VOLATILE AS $BODY$
BEGIN
EXECUTE format('
CREATE TABLE IF NOT EXISTS %I(
id serial PRIMARY KEY,
pkey VARCHAR (250) NULL,
fpo_data TEXT NULL
)', tb_name || '_pk');
EXECUTE 'INSERT INTO' || tb_name || '_pk (pkey, fpo_data) VALUES
('|| pkey ||', '|| filedata ||')';
END;
$BODY$;
First: %I, when used with a name like public.tablename_11111, won't do what you want.
You will end up with a table called "public.tablename_11111", not with a table tablename_11111 in schema public. For that, you should separate schema and table name and use the format %I.%I:
EXECUTE
format(
'CREATE TABLE %I.%I (...)',
schema_name, tb_name || '_pk'
);
Second, your INSERT statement is vulnerable to SQL injection. You must use the format function there as well, just like in CREATE TABLE.
Ideally you should pass the schema name and table name as two separate values. And it's better to not concatenate values into a SQL string, but use placeholders. Mainly so that you don't need to worry about formatting them correctly.
Something like the following:
CREATE OR REPLACE FUNCTION public.generate_table(tb_schema text, tb_name text, ???)
RETURNS text
LANGUAGE plpgsql --<< the language name is an identifier, don't quote it
COST 100 VOLATILE
AS $BODY$
BEGIN
tb_name := tb_name ||'_pk';
EXECUTE format('
CREATE TABLE IF NOT EXISTS %I.%I (
id serial PRIMARY KEY,
pkey VARCHAR (250) NULL,
fpo_data TEXT NULL
)', tb_schema, tb_name);
-- where do pkey and filedata come from?
EXECUTE format('INSERT INTO %I.%I (pkey, fpo_data) VALUES (:1, :2)',
tb_schema, tb_name)
using pkey, filedata;
END;
$BODY$;
I have the following small MVWE for a basic micro-auditing system which works fine but lacks a functionality:
DROP TABLE IF EXISTS audit CASCADE;
CREATE TABLE audit(
Id BIGSERIAL NOT NULL
,TimeValue TIMESTAMP NOT NULL
,RoleName NAME NOT NULL
,Operation NAME NOT NULL
,SchemaName NAME NOT NULL
,TableName NAME NOT NULL
,Identifiers BIGINT[]
---
,PRIMARY KEY(Id)
);
-- Audit Trigger:
DROP FUNCTION IF EXISTS audit_trigger() CASCADE;
CREATE OR REPLACE FUNCTION audit_trigger()
RETURNS TRIGGER AS
$BODY$
BEGIN
INSERT INTO audit(TimeValue, RoleName, Operation, SchemaName, TableName) VALUES
(now()::TIMESTAMP, current_user, TG_OP, TG_TABLE_SCHEMA, TG_RELNAME);
RETURN NULL;
END;
$BODY$
LANGUAGE plpgsql SECURITY DEFINER;
-- Channels:
DROP TABLE IF EXISTS channels CASCADE;
CREATE TABLE channels(
Id INTEGER NOT NULL
,UserKey TEXT NOT NULL
,Active BOOLEAN NOT NULL DEFAULT(TRUE)
---
,PRIMARY KEY(Id)
,UNIQUE(UserKey)
);
CREATE TRIGGER channel_audit_trigger BEFORE INSERT OR UPDATE OR DELETE ON channels
FOR EACH STATEMENT EXECUTE PROCEDURE audit_trigger();
-- Perform some operations:
INSERT INTO channels(
SELECT C.Id, 'Channel-' || C.Id
FROM generate_series(1, 300, 10) AS C(Id)
);
DELETE FROM channels WHERE id < 10;
UPDATE channels
SET UserKey = 'wild channel'
WHERE id = 21;
I would like to add into the last column of audit table, identifiers of rows that have been inserted/updated/deleted in channels.
I have used STATEMENT level because I just need to collects identifiers in a array. But I do not find how to access DML statistics. Conversely at the ROW level I must handle OLD and NEW cases and I cannot succeed in aggregate all touched identifier.
How can I proceed in order to fill the last column of audit table with touched identifiers?
Update
Finally I reached my goal, but this solution might not be scalable and may have some unwanted drawbacks (I am open to any constructive feedback and advice).
Basically, how I have solved my problem:
Log at ROW level BEFORE DML is performed into a table audit_rowlevel;
Aggregate the freshly added content of audit_rowlevel into audit_statementlevel at STATEMENT level AFTER DML is performed;
Minimal Working Example is now:
DROP TABLE IF EXISTS audit_rowlevel CASCADE;
CREATE TABLE audit_rowlevel(
Id BIGSERIAL NOT NULL
,Aggregated BOOLEAN NOT NULL DEFAULT(FALSE)
,TimeValue TIMESTAMP NOT NULL
-- https://www.postgresql.org/docs/current/static/functions-info.html
,RoleName NAME NOT NULL
,ClientIP INET NOT NULL
,ClientPid INTEGER NOT NULL
-- https://www.postgresql.org/docs/current/static/plpgsql-trigger.html
,Operation TEXT NOT NULL
,SchemaName NAME NOT NULL
,TableName NAME NOT NULL
,RowId BIGINT NOT NULL
-- https://www.postgresql.org/docs/current/static/functions-json.html
,OldValue JSONB
,NewValue JSONB
---
,PRIMARY KEY(Id)
);
-- Row Level Trigger:
DROP FUNCTION IF EXISTS audit_rowlevel_trigger() CASCADE;
CREATE OR REPLACE FUNCTION audit_rowlevel_trigger()
RETURNS TRIGGER AS
$BODY$
DECLARE
history BOOLEAN := (TG_NARGS > 0) AND (TG_ARGV[0]::BOOLEAN);
rowid BIGINT;
oldvalue JSONB;
newvalue JSONB;
BEGIN
-- Handle NEW:
IF TG_OP = ANY('{INSERT,UPDATE}') THEN
IF history THEN
newvalue := to_jsonb(NEW);
END IF;
rowid := NEW.Id::BIGINT;
END IF;
-- Handle OLD:
IF TG_OP = ANY('{UPDATE,DELETE}') THEN
IF history THEN
oldvalue := to_jsonb(OLD);
END IF;
rowid := OLD.Id::BIGINT;
END IF;
-- INSERT:
INSERT INTO audit_rowlevel(TimeValue, RoleName, ClientIP, ClientPID, Operation, SchemaName, TableName, RowId, NewValue, OldValue) VALUES
(now()::TIMESTAMP, current_user, inet_client_addr(), pg_backend_pid(), TG_OP, TG_TABLE_SCHEMA, TG_RELNAME, RowId, NewValue, OldValue);
-- RETURN:
IF TG_OP = ANY('{INSERT,UPDATE}') THEN
RETURN NEW;
ELSIF TG_OP = 'DELETE' THEN
RETURN OLD;
ELSE
RETURN NULL;
END IF;
END;
$BODY$
LANGUAGE plpgsql SECURITY DEFINER;
-- Statement Level Trigger:
DROP TABLE IF EXISTS audit_statementlevel CASCADE;
CREATE TABLE audit_statementlevel(
Id BIGSERIAL NOT NULL
,TimeValue TIMESTAMP NOT NULL
,RoleName NAME NOT NULL
,ClientIP INET NOT NULL
,ClientPid INTEGER NOT NULL
,Operation TEXT NOT NULL
,SchemaName NAME NOT NULL
,TableName NAME NOT NULL
,RowCount BIGINT NOT NULL
,RowIds BIGINT[] NOT NULL
,AuditIds BIGINT[] NOT NULL
---
,PRIMARY KEY(Id)
);
-- Row Level Trigger:
DROP FUNCTION IF EXISTS audit_statementlevel_trigger() CASCADE;
CREATE OR REPLACE FUNCTION audit_statementlevel_trigger()
RETURNS TRIGGER AS
$BODY$
DECLARE
rowcount BIGINT;
BEGIN
WITH
A AS (
SELECT
TimeValue, RoleName, ClientIP, ClientPid, Operation, SchemaName, TableName
,COUNT(*)
,array_agg(RowId)
,array_agg(Id)
FROM
audit_rowlevel
WHERE
NOT Aggregated
GROUP BY
TimeValue, RoleName, ClientIP, ClientPid, Operation, SchemaName, TableName
ORDER BY
TimeValue
),
B AS (
INSERT INTO audit_statementlevel(TimeValue, RoleName, ClientIP, ClientPid, Operation, SchemaName, TableName, RowCount, RowIds, AuditIds)
(SELECT * FROM A)
RETURNING AuditIds
),
C AS (
SELECT array_agg(DISTINCT T.id) AS Ids FROM B, unnest(B.AuditIds) AS T(id)
)
UPDATE
audit_rowlevel
SET
Aggregated = TRUE
FROM
C
WHERE
Id = ANY(C.Ids);
RETURN NULL;
END;
$BODY$
LANGUAGE plpgsql SECURITY DEFINER;
-- Channels:
DROP TABLE IF EXISTS channels CASCADE;
CREATE TABLE channels(
Id INTEGER NOT NULL
,UserKey TEXT NOT NULL
,Active BOOLEAN NOT NULL DEFAULT(TRUE)
---
,PRIMARY KEY(Id)
,UNIQUE(UserKey)
);
CREATE TRIGGER channel_audit_rowlevel_trigger BEFORE INSERT OR UPDATE OR DELETE ON channels
FOR EACH ROW EXECUTE PROCEDURE audit_rowlevel_trigger(TRUE);
CREATE TRIGGER channel_audit_statementlevel_trigger AFTER INSERT OR UPDATE OR DELETE ON channels
FOR EACH STATEMENT EXECUTE PROCEDURE audit_statementlevel_trigger();
-- Perform some operations:
INSERT INTO channels(
SELECT C.Id, 'Channel-' || C.Id
FROM generate_series(1, 300, 10) AS C(Id)
);
DELETE FROM channels WHERE id < 10;
UPDATE channels
SET UserKey = 'wild channel'
WHERE id = 21;
I am interested knowing if this solution looks right to professional developers.
Am I going into the good direction or is this solution evil?
I would like to add a constraint that will check values from related table.
I have 3 tables:
CREATE TABLE somethink_usr_rel (
user_id BIGINT NOT NULL,
stomethink_id BIGINT NOT NULL
);
CREATE TABLE usr (
id BIGINT NOT NULL,
role_id BIGINT NOT NULL
);
CREATE TABLE role (
id BIGINT NOT NULL,
type BIGINT NOT NULL
);
(If you want me to put constraint with FK let me know.)
I want to add a constraint to somethink_usr_rel that checks type in role ("two tables away"), e.g.:
ALTER TABLE somethink_usr_rel
ADD CONSTRAINT CH_sm_usr_type_check
CHECK (usr.role.type = 'SOME_ENUM');
I tried to do this with JOINs but didn't succeed. Any idea how to achieve it?
CHECK constraints cannot currently reference other tables. The manual:
Currently, CHECK expressions cannot contain subqueries nor refer to
variables other than columns of the current row.
One way is to use a trigger like demonstrated by #Wolph.
A clean solution without triggers: add redundant columns and include them in FOREIGN KEY constraints, which are the first choice to enforce referential integrity. Related answer on dba.SE with detailed instructions:
Enforcing constraints “two tables away”
Another option would be to "fake" an IMMUTABLE function doing the check and use that in a CHECK constraint. Postgres will allow this, but be aware of possible caveats. Best make that a NOT VALID constraint. See:
Disable all constraints and table checks while restoring a dump
A CHECK constraint is not an option if you need joins. You can create a trigger which raises an error instead.
Have a look at this example: http://www.postgresql.org/docs/9.1/static/plpgsql-trigger.html#PLPGSQL-TRIGGER-EXAMPLE
CREATE TABLE emp (
empname text,
salary integer,
last_date timestamp,
last_user text
);
CREATE FUNCTION emp_stamp() RETURNS trigger AS $emp_stamp$
BEGIN
-- Check that empname and salary are given
IF NEW.empname IS NULL THEN
RAISE EXCEPTION 'empname cannot be null';
END IF;
IF NEW.salary IS NULL THEN
RAISE EXCEPTION '% cannot have null salary', NEW.empname;
END IF;
-- Who works for us when she must pay for it?
IF NEW.salary < 0 THEN
RAISE EXCEPTION '% cannot have a negative salary', NEW.empname;
END IF;
-- Remember who changed the payroll when
NEW.last_date := current_timestamp;
NEW.last_user := current_user;
RETURN NEW;
END;
$emp_stamp$ LANGUAGE plpgsql;
CREATE TRIGGER emp_stamp BEFORE INSERT OR UPDATE ON emp
FOR EACH ROW EXECUTE PROCEDURE emp_stamp();
...i did it so (nazwa=user name, firma = company name) :
CREATE TABLE users
(
id bigserial CONSTRAINT firstkey PRIMARY KEY,
nazwa character varying(20),
firma character varying(50)
);
CREATE TABLE test
(
id bigserial CONSTRAINT firstkey PRIMARY KEY,
firma character varying(50),
towar character varying(20),
nazwisko character varying(20)
);
ALTER TABLE public.test ENABLE ROW LEVEL SECURITY;
CREATE OR REPLACE FUNCTION whoIAM3() RETURNS varchar(50) as $$
declare
result varchar(50);
BEGIN
select into result users.firma from users where users.nazwa = current_user;
return result;
END;
$$ LANGUAGE plpgsql;
CREATE POLICY user_policy ON public.test
USING (firma = whoIAM3());
CREATE FUNCTION test_trigger_function()
RETURNS trigger AS $$
BEGIN
NEW.firma:=whoIam3();
return NEW;
END
$$ LANGUAGE 'plpgsql'
CREATE TRIGGER test_trigger_insert BEFORE INSERT ON test FOR EACH ROW EXECUTE PROCEDURE test_trigger_function();