I have a function with several queries inside, like this:
CREATE OR REPLACE FUNCTION public.myfunction(winningid integer, losingid integer)
RETURNS void
LANGUAGE plpgsql
AS $function$
begin
update partecipants set id_be=winningId where id_be=losingId;
...
many other updates and deletes regarding other tables
...
delete from business_entity where id_be=losingId;
end;
$function$
;
There is a foreign key between partecipants and business_entity:
ALTER TABLE partecipants ADD CONSTRAINT partecipants_fk FOREIGN KEY (id_be) REFERENCES business_entity(id_be)
Sometimes (like 1 in 1000 times) this function hangs with error:
error: update or delete on table "business_entity" violates foreign key constraint "partecipants_fk" on table "partecipants"
detail:
'Key (id_be)=(315017) is still referenced from table "partecipants".
If I run the same function a second time, ends without error.
How is this possible?
As a background information, other processes are using (often also with locks) the tables involved in the function while is running.
Concurrent transactions could add or modify rows in partecipants between your UPDATE and your DELETE so that the latter fails.
If you want to avoid that, do both in a single statement:
WITH dummy AS (
update partecipants set id_be=winningId where id_be=losingId
)
delete from business_entity where id_be=losingId;
Related
I would like to create trigger to execute function to truncate local database table and insert new data.
Trigger execution must start after new row have insert in foreign database table.
I have read a lot about creating triggers on foreign table, but for me its not working. Trigger seems to not execute function when new row will be inserted in foreign table. It seems like trigger cant see this new row insert event.
What I did:
Created foreign table in my local database, lets call it 'foreign_table'. I tested, I can read data.
Created function to truncate local table and insert new data:
CREATE or replace FUNCTION public.reset_insert_table()
RETURNS TRIGGER
LANGUAGE 'plpgsql'
SET search_path=public
AS $BODY$
BEGIN
create temporary table temporary_table_tmp
as select * from public.table1;
TRUNCATE TABLE public.table2;
insert into table2
select * from temporary_table_tmp;
DROP table temporary_table_tmp;
END;
$BODY$;
Created trigger to launch function 'reset_insert_table()'
CREATE TRIGGER local_table_update
AFTER INSERT
ON 'foreign_table'
FOR EACH ROW EXECUTE PROCEDURE reset_insert_table();
Made test: inserted new row in foreign database table 'foreign_table', but I cant see that table is truncated and new data is not inserted. Insertion to foreign_tale was done in foreign database.
Problem was also testing does this trigger function work, executing manually will produce error:
EXECUTE PROCEDURE reset_insert_table();
ERROR: syntax error at or near "execute"
Tried also CALL and SELECT.
I created same function for testing but instead defining 'RETURNS TRIGGER'used 'RETURNS VOID' and function is working.
Can anyone tell why my solution is not working and does trigger on foreign tables must see events happening in foreign tables?
According to your comments, you seem to be using logical replication.
While data modifications are replayed on the standby with logical replication, the parameter session_replication_role is set to replica to keep triggers and foreign key constraints from working.
If you want a trigger to be triggered by the replay of data via logical replication, you have to declare it as a replica trigger:
ALTER TABLE a2 ENABLE REPLICA TRIGGER trigger_name;
When calling a SQL function that performs an insert, from a PGPLSQL function with an EXCEPTION WHEN OTHERS block, the exception is raised rather than caught if the violated foreign key constraint is deferrable.
I'm using Amazon Aurora PostgreSQL compatible (v 10.4). I found that my exception handler wasn't always catching exceptions, which were instead being raised to the application (in my case an AWS Lambda Python function using Pyscopg2).
Took a lot of troubleshooting to narrow it down to the deferrable constraint, so I created a test function that reproduces the problem pretty reliably.
I've also reproduced the same behaviour on an RDS (non-Aurora) instance running version 10.5, and also on a 9.6.6 RDS instance so it's not Aurora and it's not specific to version 10.4.
Is this a bug? Or am I missing something that's documented with deferrable constraints?
Here are two tables and two functions.
CREATE TABLE public.load (
load_id BIGINT NOT NULL,
created_timestamp TIMESTAMP WITH TIME ZONE DEFAULT now() NOT NULL,
ds_code TEXT NOT NULL,
route TEXT,
file_name TEXT,
user_name TEXT,
staged BOOLEAN,
staging_duration INTERVAL,
CONSTRAINT load_pkey
PRIMARY KEY (load_id));
CREATE TABLE load_content (
load_id BIGINT NOT NULL,
load_content TEXT,
CONSTRAINT data_load_pk
PRIMARY KEY (load_id),
CONSTRAINT data_load_load_load_id_fk
FOREIGN KEY (load_id) REFERENCES public.load
)
;
CREATE FUNCTION insert_something() RETURNS void
LANGUAGE SQL
AS
$$
INSERT INTO public.load_content values (1);
$$
;
CREATE FUNCTION test_load() RETURNS TEXT
LANGUAGE plpgsql
AS
$$
BEGIN
PERFORM public.insert_something();
RETURN 'success';
EXCEPTION
WHEN OTHERS THEN
RETURN 'failure';
END
;
$$
;
Executing public.test_load() returns a single row 'failure'.
If you then do this:
alter table public.load_content
drop constraint data_load_load_load_id_fk;
ALTER TABLE public.load_content
ADD CONSTRAINT data_load_load_load_id_fk
FOREIGN KEY (load_id) REFERENCES public.load
DEFERRABLE INITIALLY DEFERRED
;
Then execute public.test_load() the exception just happens:
[2019-01-14 16:36:32] [23503] ERROR: insert or update on table "load_content" violates foreign key constraint "data_load_load_load_id_fk"
[2019-01-14 16:36:32] Detail: Key (load_id)=(1) is not present in table "load".
Obviously, I've simplified this considerably for test purposes - what's actually happening in my system is a trigger on the "load" table that fires to normalise raw data (usually JSON) into a staging data model.
A DEFERRED constraint check is postponed to the commit, an IMMEDIATE constraint check is executed as the name says, immediatly.
The behaviour of a DEFERRABLE constraint can be changed within the current transaction with the SET CONSTRAINTS { ALL | name [, ...] } { DEFERRED | IMMEDIATE } (link to docs).
Should you want to change the behaviour of the test_load() function, here is the code example.
CREATE FUNCTION test_load() RETURNS TEXT
LANGUAGE plpgsql
AS
$$
BEGIN
SET CONSTRAINTS data_load_load_load_id_fk IMMEDIATE;
PERFORM public.insert_something();
RETURN 'success';
EXCEPTION
WHEN OTHERS THEN
RETURN 'failure';
END
;
$$
;
I'm looking for a way to manipulate rows automatically before adding them to a table in postgreSQL. Say for instance we have the following table:
CREATE TABLE foo (
id serial NOT NULL,
value integer NOT NULL,
CONSTRAINT "Foo_pkey" PRIMARY KEY (id),
CONSTRAINT "Foo_value_check" CHECK (value >= 0)
)
Now one can insert rows:
INSERT INTO foo (id,value) VALUES ('0','2')
And when one enters:
INSERT INTO foo (id,value) VALUES ('1','-2')
An error will occur. Is it possible to define a "rewrite rule" that given the value column contains a value less than zero, zero is used (for instance)?
Yes, it is possible. One way is to use triggers. A trigger causes a procedure to be run on particular actions, which can allow you to modify the data to be inserted (amongst other things).
To set up a trigger, you first create a function that will perform the checks and modifications you want. The variable new in your function will be implicitly declared and contain the new row to be inserted / updated so you can check and modify the values before they reach the table.
You then specify that this function is to be called before insert or update on one or more tables.
Example:
CREATE FUNCTION validate_foo_row()
RETURNS TRIGGER AS $$
BEGIN
IF new.value<0 THEN
new.value=0;
END IF;
RETURN NEW;
END
$$ LANGUAGE 'plpgsql';
CREATE TRIGGER trig_validate_foo BEFORE INSERT ON foo
FOR EACH ROW EXECUTE PROCEDURE validate_foo_row();
SqlFiddle Here
The above simplistic example only triggers for inserts, you might want to have it trigger for updates as well.
You can read more about triggers in the postgresql manual. They are powerful and are capable of a lot more than this simple example shows.
I have the following table in postgres:
CREATE TABLE "test" (
"id" serial NOT NULL PRIMARY KEY,
"value" text
)
I am doing following insertions:
insert into test (id, value) values (1, 'alpha')
insert into test (id, value) values (2, 'beta')
insert into test (value) values ('gamma')
In the first 2 inserts I am explicitly mentioning the id. However the table's auto increment pointer is not updated in this case. Hence in the 3rd insert I get the error:
ERROR: duplicate key value violates unique constraint "test_pkey"
DETAIL: Key (id)=(1) already exists.
I never faced this problem in Mysql in both MyISAM and INNODB engines. Explicit or not, mysql always update autoincrement pointer based on the max row id.
What is the workaround for this problem in postgres? I need it because I want a tighter control for some ids in my table.
UPDATE:
I need it because for some values I need to have a fixed id. For other new entries I dont mind creating new ones.
I think it may be possible by manually incrementing the nextval pointer to max(id) + 1 whenever I am explicitly inserting the ids. But I am not sure how to do that.
That's how it's supposed to work - next_val('test_id_seq') is only called when the system needs a value for this column and you have not provided one. If you provide value no such call is performed and consequently the sequence is not "updated".
You could work around this by manually setting the value of the sequence after your last insert with explicitly provided values:
SELECT setval('test_id_seq', (SELECT MAX(id) from "test"));
The name of the sequence is autogenerated and is always tablename_columnname_seq.
In the recent version of Django, this topic is discussed in the documentation:
Django uses PostgreSQL’s SERIAL data type to store auto-incrementing
primary keys. A SERIAL column is populated with values from a sequence
that keeps track of the next available value. Manually assigning a
value to an auto-incrementing field doesn’t update the field’s
sequence, which might later cause a conflict.
Ref: https://docs.djangoproject.com/en/dev/ref/databases/#manually-specified-autoincrement-pk
There is also management command manage.py sqlsequencereset app_label ... that is able to generate SQL statements for resetting sequences for the given app name(s)
Ref: https://docs.djangoproject.com/en/dev/ref/django-admin/#django-admin-sqlsequencereset
For example these SQL statements were generated by manage.py sqlsequencereset my_app_in_my_project:
BEGIN;
SELECT setval(pg_get_serial_sequence('"my_project_aaa"','id'), coalesce(max("id"), 1), max("id") IS NOT null) FROM "my_project_aaa";
SELECT setval(pg_get_serial_sequence('"my_project_bbb"','id'), coalesce(max("id"), 1), max("id") IS NOT null) FROM "my_project_bbb";
SELECT setval(pg_get_serial_sequence('"my_project_ccc"','id'), coalesce(max("id"), 1), max("id") IS NOT null) FROM "my_project_ccc";
COMMIT;
It can be done automatically using a trigger. This way you are sure that the largest value is always used as the next default value.
CREATE OR REPLACE FUNCTION set_serial_id_seq()
RETURNS trigger AS
$BODY$
BEGIN
EXECUTE (FORMAT('SELECT setval(''%s_%s_seq'', (SELECT MAX(%s) from %s));',
TG_TABLE_NAME,
TG_ARGV[0],
TG_ARGV[0],
TG_TABLE_NAME));
RETURN OLD;
END;
$BODY$
LANGUAGE plpgsql;
CREATE TRIGGER set_mytable_id_seq
AFTER INSERT OR UPDATE OR DELETE
ON mytable
FOR EACH STATEMENT
EXECUTE PROCEDURE set_serial_id_seq('mytable_id');
The function can be reused for multiple tables. Change "mytable" to the table of interest.
For more info regarding triggers:
https://www.postgresql.org/docs/9.1/plpgsql-trigger.html
https://www.postgresql.org/docs/9.1/sql-createtrigger.html
I'm converting a MySQL table to PostgreSQL for the first time in my life and running into the traditional newbie problem of having no auto_increment.
Now I've found out that the postgres solution is to use a sequence and then request the nextval() of this sequence as the default value every time you insert. I've also read that the SERIAL type creates a sequence and a primary key automatically, and that nextval() increments the counter even when called inside transactions to avoid locking the sequence.
What I can't find addressed is the issue of what happens when you manually insert values into a field with a UNIQUE or PRIMARY constraint and a nextval() of a sequence as default. As far as I can see, this causes the INSERT to fail when the sequence reaches that value.
Is there a simple (or common) way to fix this ?
A clear explanation would be very much appreciated.
Update: If you feel I shouldn't do this, will never be able to fix this or am making some flawed assumptions, please feel free to point them out in your answers. Above all, please tell me what to do instead to offer programmers a stable and robust database that can't be corrupted with a simple insert (preferably without hiding everything behind stored procedures)
If you're migrating your data then I would drop the sequence constraint on the column, perform all of your inserts, use setval() to set the sequence to the maximum value of your data and then reinstate your column sequence nextval() default.
You can create a trigger which will check if currval('id_sequence_name')>=NEW.id.
If your transaction did not use default value or nextval('id_sequence_name'), then a currval function will throw an error, as it works only when sequence was updated in current session. If you use nextval and then try to insert bigger primary key then it will throw another error. A transaction will be then aborted.
This would prevent inserting any bad primary keys which would break serial.
Example code:
create table test (id serial primary key, value text);
create or replace function test_id_check() returns trigger language plpgsql as
$$ begin
if ( currval('test_id_seq')<NEW.id ) then
raise exception 'currval(test_id_seq)<id';
end if;
return NEW;
end; $$;
create trigger test_id_seq_check before insert or update of id on test
for each row execute procedure test_id_check();
Then inserting with default primary key will work fine:
insert into test(value) values ('a'),('b'),('c'),('d');
But inserting too big primary key will error out and abort:
insert into test(id, value) values (10,'z');
To expand on Tometzky's great answer, here is a more general version:
CREATE OR REPLACE FUNCTION check_serial() RETURNS trigger AS $$
BEGIN
IF currval(TG_TABLE_SCHEMA || '.' || TG_TABLE_NAME || '_' || TG_ARGV[0] || '_seq') <
(row_to_json(NEW)->>TG_ARGV[0])::bigint
THEN RAISE SQLSTATE '55000'; -- same as currval() of uninitialized sequence
END IF;
RETURN NULL;
EXCEPTION
WHEN SQLSTATE '55000'
THEN RAISE 'manual entry of serial field %.%.% disallowed',
TG_TABLE_SCHEMA, TG_TABLE_NAME, TG_ARGV[0]
USING HINT = 'use DEFAULT instead of specifying value manually',
SCHEMA = TG_TABLE_SCHEMA, TABLE = TG_TABLE_NAME, COLUMN = TG_ARGV[0];
END;
$$ LANGUAGE plpgsql;
Which you can apply to any column, say test.id, thusly:
CREATE CONSTRAINT TRIGGER test_id_check
AFTER INSERT OR UPDATE OF id ON test
FOR EACH ROW EXECUTE PROCEDURE check_serial(id);
I don't exactly understand you question, but if your goal is just to do the insert, and have a valid field (e.g. an id), then insert the values without the id field, that's what "default" stands for. It will work.
E.g. havin a id serial NOT NULL and a CONSTRAINT table_pkey PRIMARY KEY(id) in the table definition will auto-set the id and auto-increment a sequence table_id_seq.
What about using a CHECK?
CREATE SEQUENCE pk_test
INCREMENT 1
MINVALUE 1
MAXVALUE 9223372036854775807
START 1
CACHE 1;
CREATE TABLE test (
id INT PRIMARY KEY CHECK (id=currval('pk_test')) DEFAULT nextval('pk_test'),
num int not null
);
ALTER SEQUENCE pk_test OWNED BY test.id;
-- Testing:
INSERT INTO test (num) VALUES (3) RETURNING id, num;
1,3 -- OK
2,3 -- OK
INSERT INTO test (id, num) values (30,3) RETURNING id, num;
/*
ERROR: new row for relation "test" violates check constraint "test_id_check"
DETAIL: Failing row contains (30, 3).
********** Error **********
ERROR: new row for relation "test" violates check constraint "test_id_check"
SQL state: 23514
Detail: Failing row contains (30, 3).
*/
DROP TABLE test;