I am looking for a way to add on delete cascade to all foreign keys in my PostgreSQL database (preferably limitable to a schema). I have found a script here, but it doesn't seem to work for PostgreSQL. The solution doesn't have to use a script, a GUI tool would also be fine.
Thank you.
The recommended and supported way to do this is to add new constraints with the same definition and ON DELETE CASCADE, then drop the original constraints.
If you are ready to risk breaking your database with a catalog modification, and you trust a random SQL statement from the internet, you could try
WITH tables(oid) AS (
UPDATE pg_constraint
SET confdeltype = 'c'
WHERE contype = 'f'
AND confdeltype <> 'c'
AND connamespace = 'myschema'::regnamespace
RETURNING confrelid
)
UPDATE pg_trigger
SET tgfoid = '"RI_FKey_cascade_del"()'::regprocedure
FROM tables
WHERE tables.oid = pg_trigger.tgrelid
AND tgtype = 9;
Test well before using!
Related
I am basically trying to replicate data from a table on one server to another.
I have two identical databases on the servers. I created a foreign table called opentickets_aux1 to represent the opentickets table on the secondary server on the primary server. Both have a primary key of incidentnumber. I can access the data in the foreign table just fine but when I try the following SQL,I get "ERROR: there is no unique or exclusion constraint matching the ON CONFLICT specification."
INSERT INTO opentickets_aux1 (SELECT * FROM opentickets)
ON CONFLICT (incidentnumber)
DO
UPDATE SET
status = EXCLUDED.status,
lastmodifieddate = EXCLUDED.lastmodifieddate
I want to update a few columns if the primary key exist. I use this statement for other queries and they work when its a local table. Any ideas?
A foreign table cannot have a primary key constraint, because PostgreSQL wouldn't be able to enforce its integrity. Therefore, you cannot use INSERT ... ON CONFLICT with foreign tables.
Your idea also does not handle rows that are deleted on the foreign server, but maybe that's intentional.
If you want a local copy of a foreign table, the easiest way would be to create a materialized view on the foreign table.
If that is not your desire (perhaps because you don't want to copy deletions), you'd have to use statements like
INSERT INTO localtable
SELECT * FROM foreigntable f
WHERE NOT EXISTS
(SELECT 1 FROM localtable l
WHERE f.id = l.id);
UPDATE localtable l
SET /* all columns from f */
FROM foreigntable f
WHERE f.id = l.id
AND (f.*) <> (l.*);
My use case is the following:
I have big table users (~200 millions rows) of users with user_id as the primary key. users is referenced by several other tables using foreign key with ON DELETE CASCADE.
Every day I have to replace the whole content of users using a lot of csv files. (Please don't ask why I have to do that, I just have to...)
My idea was to set the primary key and all foreign keys as DEFERRED, then, in the same transaction, DELETE the whole table and copying all the csvs using the COPY command. The expected result was that all check and index calculation would happen at the end of the transaction.
But actually the insert process is super slow (4hours, against 10min if I insert and the put the primary key) AND no foreign key can refer to a deferrable primary.
I can't remove the primary key during the insertion because of the foreign keys. I don't want get rid of the foreign key either because I would have to simulate the behavior of ON DELETE CASCADE manually.
So basically I am looking for a way to tell postgres to not care about primary key index or foreign key check until the very end of the transaction.
PS1: I made up the users table, I am actually working with very different kind of data but it's not really relevant to the problem.
PS2: As a rough estimation I would say that every day, on my 200+ millions records, I have 10 records removed, 1million updated and 1million added.
A full delete + a full insert will cause a flood of cascading FK,
which will have to be postponed by DEFERRED,
which will cause an avalanche of aftermath for the DBMS at commit time.
Instead, dont {delete+create} keys, but keep them right where they are.
Also, dont touch records that dont need to be touched.
-- staging table
CREATE TABLE tmp_users AS SELECT * FROM big_users WHERE 1=0;
COPY TABLE tmp_users (...) FROM '...' WITH CSV;
-- ... and more copying ...
-- ... from more files ...
-- If this fails, you have a problem!
ALTER TABLE tmp_users
ADD PRIMARY KEY (id);
-- [EDIT]
-- I added this later, because the user_comments table
-- was not present in the original question.
DELETE FROM user_comments c
WHERE NOT EXISTS (
SELECT * FROM tmp_users u WHERE u.id = c.user_id
);
-- These deletes are allowed to cascade
-- [we assume that the mport of the CSV files was complete, here ...]
DELETE FROM big_users b
WHERE NOT EXISTS (
SELECT *
FROM tmp_users t
WHERE t.id = b.id
);
-- Only update the records that actually **change**
-- [ updates are expensive in terms of I/O, because they create row-versions
-- , and the need to delete the old row-versions, afterwards ]
-- Note that the key (id) does not change, so there will be no cascading.
-- ------------------------------------------------------------
UPDATE big_users b
SET name_1 = t.name_1
, name_2 = t.name_2
, address = t.address
-- , ... ALL THE COLUMNS here, except the key(s)
FROM tmp_users t
WHERE t.id = b.id
AND (t.name_1, t.name_2, t.address, ...) -- ALL THE COLUMNS, except the key(s)
IS DISTINCT FROM
(b.name_1, b.name_2, b.address, ...)
;
-- Maybe there were some new records in the CSV files. Add them.
INSERT INTO big_users (id,name_1,name_2,address, ...)
SELECT id,name_1,name_2,address, ...
FROM tmp_users t
WHERE NOT EXISTS (
SELECT *
FROM big_users x
WHERE x.id = t.id
);
I found a hacky solution :
update pg_index set indisvalid = false, indisready=false where indexrelid= 'users_pkey'::regclass;
DELETE FROM users;
COPY TABLE users FROM 'file.csv';
REINDEX INDEX users_pkey;
DELETE FROM user_comments c WHERE NOT EXISTS (SELECT * FROM users u WHERE u.id = c.user_id )
commit;
The magic dirty hack is to disable the primary key index in the postgres catalog and at the end to force the reindexing (which will override what we changed). I can't use foreign key with ON DELETE CASCADE because for some reason it makes the constraint being executed immediatly... So instead my foreign keys are ON DELETE NO ACTION DEFERRABLE INITIALLY DEFERRED and I have to do the delete myself.
This works well in my case because only a few users are being referred in other tables.
I wish there was a cleaner solution though...
I've tried searching for about an hour through all the Foreign Key / Collation Questions but I can't find something even remotely close to my question.
I have 2 tables from two different software vendors sitting in the same database. One vendor hard codes their collations to Latin1_General_BIN while the other uses the Database Default (in this case Latin1_General_CI_AS). Is it possible, without altering any of the columns, to create Foreign Keys between these two Collation Types?
Typically you would simply change one but in this case the Tables are very sensitive and I'm not allowed to make such a change but I have to create a Foreign Key due to a piece of logic in a Trigger that reads data between these two tables only if it finds a Foreign Key:
SELECT COLUMN_NAME FROM INFORMATION_SCHEMA.CONSTRAINT_COLUMN_USAGE
WHERE CONSTRAINT_NAME =
(
SELECT name FROM sys.foreign_keys
WHERE parent_object_id = OBJECT_ID( 'Table1' )
AND referenced_object_id = OBJECT_ID( 'Table2' )
)
Any help would really be appreciate
P.S. I just can't seem to figure out how this code thing works if anyone would help me out, I put in the 4 required spaces but it's still just displaying my code as text :(
Adding a foreign key constraint from a field of one collation to a field of another collation can't be done. You will get error message 1757.
Either change the collation of one of the tables or create a work around with a new column that is used instead with the correct collation or create surrogate key columns with integers used for referencing.
If nothing else works and you really really need to fix this type of constraint and performance is not an issue, add a check constraints and/or triggers that will check the referential integrity of data put into the tables. These rules will have to cast all values in one table to the collation of the other in order to compare values so it will be slow and it will be really tricky for you to get use of indexes, proceed with caution.
For example you could have an insert trigger on the referencing table that check if a record with the inserted string exists in the referenced table. Then you would also have to add an update and delete trigger for the referenced table so that it doesn't fall out of range of values that are referenced by records in the referencing table or which cascades updates/deletes. Basically you replicate what foreign keys are and it gets really slow and scales horribly.
Short answer: don't do it, let the tables stay untied or fix the collation of one of them.
Sweet, I think the solution is very elegant. I'm writing it as an answer purely as it's the full alternative that closest resembles the required solution. But I'm going to mark your answer as the answer as it's the one that correctly answers my original question.
Right, so first what I did, was I got permission from the vendor who's trigger requires the foreign key, to create a new column in their table as a persisted computed column in the collation of the other vendors table:
DECLARE #Collation nvarchar(100)
DECLARE #SQL nvarchar(1000)
SET #Collation = ( SELECT collation_name FROM sys.columns WHERE OBJECT_ID IN ( SELECT OBJECT_ID FROM sys.objects WHERE type = 'U' AND name = 'Vendor1Table' ) AND name = 'Vendor1Column' )
SET #SQL = 'ALTER TABLE [Vendor2Table] ADD [Vendor2ComputedColumn] AS [Vendor2Column] COLLATE ' + #Collation + ' PERSISTED'
EXECUTE( #SQL )
GO
Next up, I added a candidate key to the computed column:
ALTER TABLE [Vendor2Table] ADD CONSTRAINT [CCUNQ1_x] UNIQUE NONCLUSTERED
(
[Vendor2ComputedColumn] ASC
)
WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON)
ON [PRIMARY]
GO
Then, I simply created the foreign key to the computed column:
ALTER TABLE [dbo].[Vendor1Table] WITH CHECK ADD CONSTRAINT [CCFOK01_x] FOREIGN KEY ( [Vendor1Column] )
REFERENCES [dbo].[Vendor2Table] ( [Vendor2ComputedColumn] )
GO
ALTER TABLE [dbo].[Vendor1Table] CHECK CONSTRAINT [CCFOK01_x]
GO
and finally, the original SQL Script passes with flying colours:
SELECT COLUMN_NAME FROM INFORMATION_SCHEMA.CONSTRAINT_COLUMN_USAGE
WHERE CONSTRAINT_NAME =
(
SELECT name FROM sys.foreign_keys
WHERE parent_object_id = OBJECT_ID( 'Vendor1Table' )
AND referenced_object_id = OBJECT_ID( 'Vendor2Table' )
)
Hopefully this small walkthrough helps some other soul some day :)
Thanks for the assist David, appreciate it!
During development of a PostgreSQL database, I made foreign keys in every table. The problem is that I forgot to select ON DELETE CASCADE option.
I need to set that options on all tables. It seems I can only drop and recreate every constraint manually.
Does anybody know any shortcuts?
Update the constraints directly in pg_constraint. First get the oid's you need:
SELECT oid,* FROM pg_constraint WHERE contype = 'f';
And then update these constraints:
UPDATE pg_constraint SET confupdtype = 'c' WHERE oid = 'fk oid';
More information can be found in the manual. Please do this work within a transaction and do some tests within the transaction as well. Just to be sure.
You could use something like liquibase (http://www.liquibase.org/) which gives you essentially versioned DB schema's, and then it would be really simple to add constraints.
Of course this is serious overkill for this particular problem, but it does attempt to solve the issues on why you had this problem in the first place.
Is it possible to change the constraint name in Postgres?
I have a PK added with:
ALTER TABLE contractor_contractor ADD CONSTRAINT commerce_contractor_pkey PRIMARY KEY(id);
And I want to to have different name for it, to be consistent with the rest of the system.
Shall I delete the existing PK constraint and create a new one? Or is there a 'soft' way to
manage it?
Thanks!
To rename an existing constraint in PostgreSQL 9.2 or newer, you can use ALTER TABLE:
ALTER TABLE name RENAME CONSTRAINT constraint_name TO new_constraint_name;
For the primary key, you should be able to just:
ALTER INDEX commerce_contractor_pkey RENAME TO whatever_new_name
That won't work for other types of constraints though. The best option there is to drop the old one and create a new one. Be sure to do it inside a transaction, so the system isn't live without it during rebuild. (And if you can't do it in a transaction, be sure to create the new one first, before dropping the old one)
We found that primary keys often lag behind the main table name. This script helped us identify and fix the ones with issues.
select
table_name,
constraint_name ,
'ALTER TABLE ' || table_name || ' RENAME CONSTRAINT ' || constraint_name || ' TO ' || left(table_name, 58) || '_pkey;'
from information_schema.table_constraints tc
where constraint_type = 'PRIMARY KEY'
and constraint_name <> left(table_name, 58) || '_pkey';
This finds all the tables where the primary key name is no longer the "default" pattern (<tablename>_pkey) and creates a rename script for each.
The 58 character limit above in code above is to account for the maximum size of constraint names (63bytes).
Obviously sense check what is returned prior to running it. Hope that is helpful for others.