I have a Web Application that has a modified field in the important tables to be able to track back when any modification was done e.g. (never mind the ;; it is there because this postgres sql code is executed from a Scala framework that uses ; as separator and ;; escapes it)
CREATE TABLE security_permission (
id BIGSERIAL,
value VARCHAR(255) NOT NULL,
modified TIMESTAMP DEFAULT now(),
PRIMARY KEY (id)
);
CREATE OR REPLACE FUNCTION update_modified()
RETURNS TRIGGER AS $$
BEGIN
NEW.modified = now();;
RETURN NEW;;
END;;
$$ language 'plpgsql';
CREATE TRIGGER update_modified_security_permission BEFORE UPDATE ON security_permission FOR EACH ROW EXECUTE PROCEDURE update_modified();
The problem is this works only if the field is NOT specified in the insert/update statement. If the field is specified even with NULL then the modified is not set. I do not have full control of the generated statements because they are part of an ORM framework that generates them automatically but I'd like to nevertheless always set the modified field. How can I do that?
I have tried using BEFORE INSERT OR UPDATE ON and AFTER INSERT OR UPDATE ON but nothing seems to work if the field is populated in the insert/update statement even if NULL. How can I do this?
Define the trigger as before update or insert:
CREATE TRIGGER update_modified_security_permission
BEFORE UPDATE OR INSERT ON security_permission
FOR EACH ROW EXECUTE PROCEDURE update_modified();
See a working example here.
Related
I want to audit created_by, created_timestamp, modified_by, and modified_timestamp columns in my PostgreSQL table with triggers. Creating BEFORE INSERT and BEFORE UPDATE triggers to set these values to current_user and now() is reasonably straightforward.
However, if someone tries to do:
INSERT INTO SOMETABLE(someColumn, created_by) VALUES ('test', 'someOtherUser');
I'd rather throw an exception like, 'Manually setting created_by in an INSERT query is not allowed." instead of having the trigger silently change 'someOtherUser' to current_user.
I thought I could accomplish this in the trigger with:
if new.created_by is not null then raise exception 'Manually setting created_by in an INSERT query is not allowed.'; end if;
This works as expected for INSERT queries and triggers.
However, using the same strategy for UPDATE triggers, I'm finding it a bit more difficult, because the NEW record has the unchanged values from the existing row in addition to the changed values in the UPDATE query. (At least, I think that's what's happening.)
I can compare new.created_by to old.created_by to ensure they're the same, thus preventing the query from changing the value, but even though the end result is similar (i.e. the value in the table doesn't get changed), this really isn't the same as disallowing the column from being in the UPDATE query at all.
Is there an elegant way to determine if a column is present in the INSERT or UPDATE query? I've seen some suggestions here to convert to JSON and test that way, but that seems to be a rather ugly solution to me.
Are there other solutions to ensurevthese columns (created_by, created_timestamp, etc.) are only set by the trigger functions and are not manually settable in INSERT and UPDATE queries?
Create a special trigger for UPDATE with a name that is early in the alphabet, so that it is called before your other trigger:
CREATE FUNCTION yell() RETURNS trigger
LANGUAGE plpgsql AS
$$BEGIN
RAISE EXCEPTION 'direct update of "created_by" is forbidden';
END;$$;
CREATE TRIGGER aa_nosuchupdate
BEFORE UPDATE OF created_by FOR EACH ROW
EXECUTE PROCEDURE yell();
The INSERT case can be handled in your other trigger.
So I found this example:
create function eager.account_insert() returns trigger
security definer
language plpgsql
as $$
begin
insert into eager.account_balances(name) values(new.name);
return new;
end;
$$;
create trigger account_insert after insert on accounts
for each row execute procedure eager.account_insert();
The thing I can't understand: function eager.account_insert() does not take any arguments, however, it operates with variable new. It returns it, but should't it return trigger?
Also, this: insert into eager.account_balances(name), this is not some certain record chosen, what it this?
The new (and old when it's an update statement) is the RECORD you're inserting or updating. You can get columns from it and do whatever you want with them. Many times BEFORE INSERT triggers check for valid values etc.
The function must return a RECORD with the same columns as the table, or NULL if the insert should not happen (usually for INSTEAD triggers).
The insert statement is just a regular insert with one column specified of the table and the value is taken from the newly inserted RECORD's column name.
The documentation explains the triggers very well.
I am using PostgreSQL as my database for a project at work. We use triggers in quite a few places to either maintain computed columns, or tables that essentially act as a materialized view.
All this worked just fine when simply utilizing row level triggers to keep all this in sync. However when we wrote scripts to periodically import our customers data into the database, we ran into issues with either performance or problems with number of locks in a single transaction.
To alleviate this I wanted to create a statement-level trigger with access to the modified rows (inserted, updated or deleted). However as this is not possible I instead created a BEFORE statement-level trigger that would create a temporary table. Then an AFTER row-level trigger that would insert the changed data into the temporary table. At last an AFTER statement-level trigger that would read the changes and perform necessary updates, and then drop the temporary table.
All this works just fine, assuming that within the triggers, no one would re-trigger the same flow again (as the temporary table would then already exist).
However I then learned that when using foreign key constraints with ON DELETE SET NULL, it is simply implemented with a system trigger that sets the column to NULL. This of course is not a problem at all, except for the fact that when you have several foreign key constraints like this on a single table, all referencing the same table (let's just call this files). When deleting a row from the files table, all these system level triggers to handle the ON DELETE SET NULL clause all fire at the same time, that is in parallel. Which presents a serious issue for me.
How would I go about implementing something like this? Here is a short SQL script to illustrate the problem:
CREATE TABLE files (
id serial PRIMARY KEY,
"name" TEXT NOT NULL
);
CREATE TABLE profiles (
id serial PRIMARY KEY,
NAME TEXT NOT NULL,
cv_file_id INT REFERENCES files(id) ON DELETE SET NULL,
photo_file_id INT REFERENCES files(id) ON DELETE SET NULL
);
CREATE TABLE profile_audit (
profile_id INT NOT NULL,
modified_at timestamptz NOT NULL
);
CREATE FUNCTION pre_stmt_create_temp_table()
RETURNS TRIGGER
AS $$
BEGIN
CREATE TEMPORARY TABLE tmp_modified_profiles (
id INT NOT NULL
) ON COMMIT DROP;
RETURN NULL;
END;
$$ LANGUAGE 'plpgsql';
CREATE FUNCTION insert_modified_profile_to_temp_table()
RETURNS TRIGGER
AS $$
BEGIN
INSERT INTO tmp_modified_profiles(id) VALUES (NEW.id);
RETURN NULL;
END;
$$ LANGUAGE 'plpgsql';
CREATE FUNCTION post_stmt_insert_rows_and_drop_temp_table()
RETURNS TRIGGER
AS $$
BEGIN
INSERT INTO profile_audit (id, modified_at)
SELECT t.id, CURRENT_TIMESTAMP FROM tmp_modified_profiles t;
DROP TABLE tmp_modified_profiles;
RETURN NULL;
END;
$$ LANGUAGE 'plpgsql';
CREATE TRIGGER tr_create_working_table BEFORE UPDATE ON profiles FOR EACH STATEMENT EXECUTE PROCEDURE pre_stmt_create_temp_table();
CREATE TRIGGER tr_insert_row_to_working_table AFTER UPDATE ON profiles FOR EACH ROW EXECUTE PROCEDURE insert_modified_profile_to_temp_table();
CREATE TRIGGER tr_insert_modified_rows_and_drop_working_table AFTER UPDATE ON profiles FOR EACH STATEMENT EXECUTE PROCEDURE post_stmt_insert_rows_and_drop_temp_table();
INSERT INTO files ("name") VALUES ('photo.jpg'), ('my_cv.pdf');
INSERT INTO profiles ("name") VALUES ('John Doe');
DELETE FROM files WHERE "name" = 'photo.jpg';
It would be a serious hack, but meanwhile, until PostgreSQL 9.5 is out, I would try to use CONSTRAINT triggers deferred to the end of the transaction. I am not really sure this will work, but might be worth trying.
You could use a status column to track inserts and updates for your statement-level triggers.
In a BEFORE INSERT OR UPDATE row-level trigger:
SET NEW.status = TG_OP;
Now you can use statement-level AFTER triggers:
BEGIN
DO FUNNY THINGS
WHERE status = 'INSERT';
-- reset the status
UPDATE mytable
SET status = NULL
WHERE status = 'INSERT';
END;
However, if you want to deal with deletes as well, you'll need something like this in your row-level trigger:
INSERT INTO status_table (table_name, op, id) VALUES (TG_TABLE_NAME, TG_OP, OLD.id);
Then, in your statement-level AFTER trigger, you can go like:
BEGIN
DO FUNNY THINGS
WHERE id IN (SELECT id FROM status_table
WHERE table_name = TG_TABLE_NAME AND op = TG_OP); -- just an example
-- reset the status
DELETE FROM status_table
WHERE table_name = TG_TABLE_NAME AND op = TG_OP;
END;
I'm looking for a way to manipulate rows automatically before adding them to a table in postgreSQL. Say for instance we have the following table:
CREATE TABLE foo (
id serial NOT NULL,
value integer NOT NULL,
CONSTRAINT "Foo_pkey" PRIMARY KEY (id),
CONSTRAINT "Foo_value_check" CHECK (value >= 0)
)
Now one can insert rows:
INSERT INTO foo (id,value) VALUES ('0','2')
And when one enters:
INSERT INTO foo (id,value) VALUES ('1','-2')
An error will occur. Is it possible to define a "rewrite rule" that given the value column contains a value less than zero, zero is used (for instance)?
Yes, it is possible. One way is to use triggers. A trigger causes a procedure to be run on particular actions, which can allow you to modify the data to be inserted (amongst other things).
To set up a trigger, you first create a function that will perform the checks and modifications you want. The variable new in your function will be implicitly declared and contain the new row to be inserted / updated so you can check and modify the values before they reach the table.
You then specify that this function is to be called before insert or update on one or more tables.
Example:
CREATE FUNCTION validate_foo_row()
RETURNS TRIGGER AS $$
BEGIN
IF new.value<0 THEN
new.value=0;
END IF;
RETURN NEW;
END
$$ LANGUAGE 'plpgsql';
CREATE TRIGGER trig_validate_foo BEFORE INSERT ON foo
FOR EACH ROW EXECUTE PROCEDURE validate_foo_row();
SqlFiddle Here
The above simplistic example only triggers for inserts, you might want to have it trigger for updates as well.
You can read more about triggers in the postgresql manual. They are powerful and are capable of a lot more than this simple example shows.
I have the following table in postgres:
CREATE TABLE "test" (
"id" serial NOT NULL PRIMARY KEY,
"value" text
)
I am doing following insertions:
insert into test (id, value) values (1, 'alpha')
insert into test (id, value) values (2, 'beta')
insert into test (value) values ('gamma')
In the first 2 inserts I am explicitly mentioning the id. However the table's auto increment pointer is not updated in this case. Hence in the 3rd insert I get the error:
ERROR: duplicate key value violates unique constraint "test_pkey"
DETAIL: Key (id)=(1) already exists.
I never faced this problem in Mysql in both MyISAM and INNODB engines. Explicit or not, mysql always update autoincrement pointer based on the max row id.
What is the workaround for this problem in postgres? I need it because I want a tighter control for some ids in my table.
UPDATE:
I need it because for some values I need to have a fixed id. For other new entries I dont mind creating new ones.
I think it may be possible by manually incrementing the nextval pointer to max(id) + 1 whenever I am explicitly inserting the ids. But I am not sure how to do that.
That's how it's supposed to work - next_val('test_id_seq') is only called when the system needs a value for this column and you have not provided one. If you provide value no such call is performed and consequently the sequence is not "updated".
You could work around this by manually setting the value of the sequence after your last insert with explicitly provided values:
SELECT setval('test_id_seq', (SELECT MAX(id) from "test"));
The name of the sequence is autogenerated and is always tablename_columnname_seq.
In the recent version of Django, this topic is discussed in the documentation:
Django uses PostgreSQL’s SERIAL data type to store auto-incrementing
primary keys. A SERIAL column is populated with values from a sequence
that keeps track of the next available value. Manually assigning a
value to an auto-incrementing field doesn’t update the field’s
sequence, which might later cause a conflict.
Ref: https://docs.djangoproject.com/en/dev/ref/databases/#manually-specified-autoincrement-pk
There is also management command manage.py sqlsequencereset app_label ... that is able to generate SQL statements for resetting sequences for the given app name(s)
Ref: https://docs.djangoproject.com/en/dev/ref/django-admin/#django-admin-sqlsequencereset
For example these SQL statements were generated by manage.py sqlsequencereset my_app_in_my_project:
BEGIN;
SELECT setval(pg_get_serial_sequence('"my_project_aaa"','id'), coalesce(max("id"), 1), max("id") IS NOT null) FROM "my_project_aaa";
SELECT setval(pg_get_serial_sequence('"my_project_bbb"','id'), coalesce(max("id"), 1), max("id") IS NOT null) FROM "my_project_bbb";
SELECT setval(pg_get_serial_sequence('"my_project_ccc"','id'), coalesce(max("id"), 1), max("id") IS NOT null) FROM "my_project_ccc";
COMMIT;
It can be done automatically using a trigger. This way you are sure that the largest value is always used as the next default value.
CREATE OR REPLACE FUNCTION set_serial_id_seq()
RETURNS trigger AS
$BODY$
BEGIN
EXECUTE (FORMAT('SELECT setval(''%s_%s_seq'', (SELECT MAX(%s) from %s));',
TG_TABLE_NAME,
TG_ARGV[0],
TG_ARGV[0],
TG_TABLE_NAME));
RETURN OLD;
END;
$BODY$
LANGUAGE plpgsql;
CREATE TRIGGER set_mytable_id_seq
AFTER INSERT OR UPDATE OR DELETE
ON mytable
FOR EACH STATEMENT
EXECUTE PROCEDURE set_serial_id_seq('mytable_id');
The function can be reused for multiple tables. Change "mytable" to the table of interest.
For more info regarding triggers:
https://www.postgresql.org/docs/9.1/plpgsql-trigger.html
https://www.postgresql.org/docs/9.1/sql-createtrigger.html