I am now using Alembic 1.8, SQLAlchemy 1.4 and PostgreSQL
I would like to set my FK constraints as "DEFERRABLE INITIALLY IMMEDIATE".
I passed the FK options as shown below:
sa.Column(
"group_id",
sa.BigInteger,
sa.ForeignKey(
"auth_group.id",
onupdate="CASCADE",
ondelete="CASCADE",
deferrable=True,
initially="IMMEDIATE"
),
index=True,
),
It generates my "create table" SQL like this:
CONSTRAINT auth_user_groups_group_id_fkey FOREIGN KEY (group_id)
REFERENCES public.auth_group (id) MATCH SIMPLE
ON UPDATE CASCADE
ON DELETE CASCADE,
DEFERRABLE
I expected "DEFERRABLE INITIALLY IMMEDIATE" instead of "DEFERRABLE".
Please, let me know how to make the constraint as "DEFERRABLE INITIALLY IMMEDIATE".
Thank you.
This is an issue (or design choice) of your DDL -> text generator -not with sqlalchemy.
The actual constraint information Postgres uses is stored in the table pg_catalog.pg_constraint. If you take a look at the docs for the pg_constraint table, you'll notice that the concept of deferability is (v7.2-v15+) controlled entirely by the two boolean columns condeferrable and condeferred.
Thus, if a constraint is DEFERRABLE and it's not INITIALLY DEFERRED (checked end of transaction), it can only be INITIALLY IMMEDIATE (checked end of statement).
If you want to be absolutely certain, you can run this simple query:
SELECT
pgc.conname constraint_name,
pgc.confrelid::regclass tbl_name,
CASE
WHEN pgc.condeferrable
THEN
CASE
WHEN pgc.condeferred
THEN 'DEFERRABLE INITIALLY DEFERRED'
ELSE 'DEFERRABLE INITIALLY IMMEDIATE'
END
ELSE 'NOT DEFERRABLE'
END deferrability
FROM
pg_catalog.pg_constraint pgc
WHERE
conname = 'auth_user_groups_group_id_fkey'
Related
I have a column sort_order with a unique constraint on it.
The following SQL fails on Postgres 9.5:
UPDATE test
SET sort_order = sort_order + 1;
-- [23505] ERROR: duplicate key value violates unique constraint "test_sort_order_key"
-- Detail: Key (sort_order)=(2) already exists.
Clearly, if the sort_order values were unique before the update, they will still be unique after the update. Why is this?
The same statement works fine on Oracle and MS SQL, but also fails on MySQL and SQLite.
Here's the complete setup code for a SQL fiddle:
DROP TABLE IF EXISTS test;
CREATE TABLE test (
val TEXT,
sort_order INTEGER NOT NULL UNIQUE
);
INSERT INTO test
VALUES ('A', 1), ('B', 2);
Postgres decides to check constraints of type IMMEDIATELY at a different time than proposed in the SQL standard.
Specifically, the documentation for SET CONSTRAINTS states (emphasis mine):
NOT NULL and CHECK constraints are always checked immediately when a row is inserted or modified (not at the end of the statement). Uniqueness and exclusion constraints that have not been declared DEFERRABLE are also checked immediately.
Postgres chooses to execute this query using a plan that results in a temporary collision for sort_order and IMMEDIATELY fails. Note that means that for the same schema and the same data, the same query may work or fail depending on the execution plan.
You'll have to make the constraint DEFERRABLE or DEFERRABLE INITIALLY DEFERRED, which delays verification of the constraint until the end of the transaction or up to the point where a statement SET CONSTRAINTS ... IMMEDIATE is executed.
Addendum from #HansGinzel's comment:
for COPY, it seems, that (even IMMEDIATE) constraints are tested after all data are COPYied.
I have a transaction where we insert a row of table foo and then a row of table bar. This ensures we either write both rows or neither. The trouble with this is bar has a foreign key into foo. Because we don't know the id of foo at the time of the bar insert, this fails the foreign key constraint.
Previously I've used tools like SQLAlchemy, when writing Python backends, that include the capability of flushing a session before the transaction is committed--this allows the user to derive the id of foo and pass it along to the INSERT into bar before actually writing anything.
My question is, in the context of JDBC and its Clojure wrapper, how can this be done?
Previously I was attempting to use (jdbc/query db-spec ["select id from foo where name='my_foo'"]) within the transaction to derive the dependent foo row ID. This was returning nil and so it seemed like the obvious method didn't work. However it turned out I was using db-spec and not the transaction connection, which if you use jdbc/with-db-transaction is bound in the vector.
For example:
(jdbc/with-db-transaction [t-conn db-spec]
(jdbc/insert! t-conn :foo {:name "my_foo"})
(jdbc/query t-conn ["select id from foo where name='my_foo'"]))
The query in the above form will yield the correct row ID.
You can insert values into both tables in one query, e.g.:
create table foo (
foo_id serial primary key,
name text);
create table bar (
bar_id serial primary key,
foo_id int references foo,
name text);
with insert_into_foo as (
insert into foo (name)
values ('some foo')
returning foo_id
)
insert into bar (foo_id, name)
select foo_id, 'some bar'
from insert_into_foo;
This is part of what DEFERRABLE foreign key constraints are for.
ALTER TABLE mytable
DROP CONSTRAINT the_fk_name;
ALTER TABLE
ADD CONSTRAINT the_fk_name
FOREIGN KEY (thecol) REFERENCES othertable(othercol)
DEFERRABLE INITIALLY IMMEDIATE;
then
BEGIN;
SET CONSTRAINTS DEFERRED;
INSERT thetable ...;
INSERT INTO othertable ...;
-- optional, but if you do this you get any errors BEFORE commit
SET CONSTRAINTS IMMEDIATE;
COMMIT;
I suggest using initially immediate and set constraints so that the rest of the time you don't create queued triggers. It's better for performance and memory use, plus it won't confuse apps that don't understand and expect deferred cosntraints.
If your framework can't cope with this you can use DEFERRABLE INITIALLY DEFERRED instead.
I tried to change all foreign keys in PostgreSQL at once to cascade on delete:
UPDATE pg_catalog.pg_constraint
SET confupdtype='c', confdeltype='c', confmatchtype='u'
WHERE connamespace=2200;
There are no errors, and when I inspect the tables with pgadmin, it looks right, but when I try to delete a referenced table-line it comes to a constraint error. Just the SQL statement works:
ALTER TABLE tblname
DROP CONSTRAINT IF EXISTS fk3e2e4a8ff123848a;
ALTER TABLE tblname
ADD CONSTRAINT fk3e2e4a8ff123848a FOREIGN KEY (field)
REFERENCES othertable (id) MATCH SIMPLE
ON UPDATE CASCADE ON DELETE CASCADE;
Any idea why changing pg_catalog.pg_constraint is not working? even restarting the service after didn't help.
Really you shouldn't be updating pg_* tables.
Use a command like
ALTER TABLE YOURTABLE DISABLE TRIGGER;
Check this link out.
http://archives.postgresql.org/pgsql-general/2011-10/msg00802.php
This is the situation: I have two tables where the one references the other (say, table2 references table1). When creating these tables, I did set the foreign key constraint as DEFERRABLE and the ON UPDATE and ON DELETE clauses as NO ACTION (which is the default).
But still, when running the transaction below, I get the following error.
Transaction:
START TRANSACTION;
SET CONSTRAINTS ALL DEFERRED;
UPDATE table1 SET blah blah;
UPDATE table2 SET blah blah;
COMMIT;
Error:
ERROR: update or delete on table "table1" violates foreign key constraint "table1_column_fkey" on table "table2"
DETAIL: Key (column1)=(blahblah) is still referenced from table "table2".
And table construction:
CREATE TABLE table1(
column1 CHAR(10),
[...]
PRIMARY KEY (column1)
);
CREATE TABLE table2(
primkey CHAR(9),
[...]
column2 CHAR(10) NOT NULL,
PRIMARY KEY(primkey),
FOREIGN KEY(column2) REFERENCES table1(column1) DEFERRABLE
);
What I want to do is to defer the foreign key checking while the transaction is in progress, until it commits. I just can't see why is this error returning and how can I make the transaction work.
The problem was indeed a foreign key constraint violation. I mean, the constraints were indeed deferred within the transaction, but the problem was that at the end of the transaction, after table1 and table2 were updated, the new data were violating a foreign key constraint. I was updating the primary key of a table1 row, which was still being referenced by some table2 rows. These rows I had to update them too, so that the referencing column of table2 rows matched the updated primary key of table1's row. I changed the 'UPDATE' queries within the transaction and the problem got solved.
Sorry to put you into this. The solution was so simple, but that day I coudn't see it.
I've created some tables in postgres, added a foreign key from one table to another and set ON DELETE to CASCADE. Strangely enough, I have some fields that appear to be violating this constraint.
Is this normal behaviour? And if so, is there a way to get the behaviour I want (no violations possible)?
Edit:
I orginaly created the foreign key as part of CREATE TABLE, just using
... REFERENCES product (id) ON UPDATE CASCADE ON DELETE CASCADE
The current code pgAdmin3 gives is
ALTER TABLE cultivar
ADD CONSTRAINT cultivar_id_fkey FOREIGN KEY (id)
REFERENCES product (id) MATCH SIMPLE
ON UPDATE CASCADE ON DELETE CASCADE;
Edit 2:
To Clarify, I have a sneaking suspicion that the constraints are only checked when updates/inserts happen but are then never looked at again. Unfortunately I don't know enough about postgres to find out if this is true or how fields could end up in the database without those checks being run.
If this is the case, is there some way to check all the foreign keys and fix those problems?
Edit 3:
A constraint violation can be caused by a faulty trigger, see below
I tried to create a simple example that shows foreign key constraint being enforced. With this example I prove I'm not allowed to enter data that violates the fk and I prove that if the fk is not in place during insert, and I enable the fk, the fk constraint throws an error telling me data violates the fk. So I'm not seeing how you have data in the table that violates a fk that is in place. I'm on 9.0, but this should not be different on 8.3. If you can show a working example that proves your issue that might help.
--CREATE TABLES--
CREATE TABLE parent
(
parent_id integer NOT NULL,
first_name character varying(50) NOT NULL,
CONSTRAINT pk_parent PRIMARY KEY (parent_id)
)
WITH (
OIDS=FALSE
);
ALTER TABLE parent OWNER TO postgres;
CREATE TABLE child
(
child_id integer NOT NULL,
parent_id integer NOT NULL,
first_name character varying(50) NOT NULL,
CONSTRAINT pk_child PRIMARY KEY (child_id),
CONSTRAINT fk1_child FOREIGN KEY (parent_id)
REFERENCES parent (parent_id) MATCH SIMPLE
ON UPDATE CASCADE ON DELETE CASCADE
)
WITH (
OIDS=FALSE
);
ALTER TABLE child OWNER TO postgres;
--CREATE TABLES--
--INSERT TEST DATA--
INSERT INTO parent(parent_id,first_name)
SELECT 1,'Daddy'
UNION
SELECT 2,'Mommy';
INSERT INTO child(child_id,parent_id,first_name)
SELECT 1,1,'Billy'
UNION
SELECT 2,1,'Jenny'
UNION
SELECT 3,1,'Kimmy'
UNION
SELECT 4,2,'Billy'
UNION
SELECT 5,2,'Jenny'
UNION
SELECT 6,2,'Kimmy';
--INSERT TEST DATA--
--SHOW THE DATA WE HAVE--
select parent.first_name,
child.first_name
from parent
inner join child
on child.parent_id = parent.parent_id
order by parent.first_name, child.first_name asc;
--SHOW THE DATA WE HAVE--
--DELETE PARENT WHO HAS CHILDREN--
BEGIN TRANSACTION;
delete from parent
where parent_id = 1;
--Check to see if any children that were linked to Daddy are still there?
--None there so the cascade delete worked.
select parent.first_name,
child.first_name
from parent
right outer join child
on child.parent_id = parent.parent_id
order by parent.first_name, child.first_name asc;
ROLLBACK TRANSACTION;
--TRY ALLOW NO REFERENTIAL DATA IN--
BEGIN TRANSACTION;
--Get rid of fk constraint so we can insert red headed step child
ALTER TABLE child DROP CONSTRAINT fk1_child;
INSERT INTO child(child_id,parent_id,first_name)
SELECT 7,99999,'Red Headed Step Child';
select parent.first_name,
child.first_name
from parent
right outer join child
on child.parent_id = parent.parent_id
order by parent.first_name, child.first_name asc;
--Will throw FK check violation because parent 99999 doesn't exist in parent table
ALTER TABLE child
ADD CONSTRAINT fk1_child FOREIGN KEY (parent_id)
REFERENCES parent (parent_id) MATCH SIMPLE
ON UPDATE CASCADE ON DELETE CASCADE;
ROLLBACK TRANSACTION;
--TRY ALLOW NO REFERENTIAL DATA IN--
--DROP TABLE parent;
--DROP TABLE child;
Everything I've read so far seems to suggest that constraints are only checked when the data is inserted. (Or when the constraint is created) For example the manual on set constraints.
This makes sense and - if the database works properly - should be good enough. I'm still curious how I managed to circumvent this or if I just read the situation wrong and there was never a real constraint violation to begin with.
Either way, case closed :-/
------- UPDATE --------
There was definitely a constraint violation, caused by a faulty trigger. Here's a script to replicate:
-- Create master table
CREATE TABLE product
(
id INT NOT NULL PRIMARY KEY
);
-- Create second table, referencing the first
CREATE TABLE example
(
id int PRIMARY KEY REFERENCES product (id) ON DELETE CASCADE
);
-- Create a (broken) trigger function
--CREATE LANGUAGE plpgsql;
CREATE OR REPLACE FUNCTION delete_product()
RETURNS trigger AS
$BODY$
BEGIN
DELETE FROM product WHERE product.id = OLD.id;
-- This is an error!
RETURN null;
END;
$BODY$
LANGUAGE plpgsql;
-- Add it to the second table
CREATE TRIGGER example_delete
BEFORE DELETE
ON example
FOR EACH ROW
EXECUTE PROCEDURE delete_product();
-- Now lets add a row
INSERT INTO product (id) VALUES (1);
INSERT INTO example (id) VALUES (1);
-- And now lets delete the row
DELETE FROM example WHERE id = 1;
/*
Now if everything is working, this should return two columns:
(pid,eid)=(1,1). However, it returns only the example id, so
(pid,eid)=(0,1). This means the foreign key constraint on the
example table is violated.
*/
SELECT product.id AS pid, example.id AS eid FROM product FULL JOIN example ON product.id = example.id;