How to find all foreign key constraints that are broken in Postgresql - postgresql

Question

We recently discovered an issue with our database where a foreign key constraint was not working correctly. Basically the primary table did not have any primary ids that matched the foreign key in the child table. When we dropped the foreign key constraint and tried to recreate it, it then threw an error that the foreign key constraint could not be created because there were foreign keys with no matching id in the parent table. Once those were cleaned up, it allowed us to recreate the foreign key.
Of course we are wondering how this happened to begin with. I worked with Oracle for 15 years and never saw a foreign key failing in this way. But our concern right now is how many other foreign keys are not working correctly. This is a problem because we have some BEFORE DELETE triggers that fail silently when the calling function returns a null because of a foreign_key_violation (this is how we discovered the issue to begin with).
EXCEPTION
WHEN foreign_key_violation
THEN RETURN NULL;
What we want to do is get all of the foreign keys in the database (probably a few thousand), loop over them all, and check every single one against it's parent table to see if any are "broken".
Basically:
Select all foreign keys using Postgres system tables.
Loop over all of them and do something like:
select count(parent_id) from child_table
where foreign_key_id not in (
select parent_id as foreign_key_id
from parent_table
)
);
For all the ones that are not 0, drop the foreign key constraint, fix the orphaned data, and recreate the foreign key constraint.
Does this sound reasonable? Has anyone done something like this before? What is the best way to get the foreign key constraints from Postgres?
Edit:
What we realized is that if we dropped and recreated the foreign keys, it would tell us which ones were problematic.
SELECT 'Alter table ' || conrelid::regclass || ' drop constraint ' || conname || '; alter table ' || conrelid::regclass || ' add constraint ' || conname || ' ' || pg_get_constraintdef(oid) || ';'
FROM pg_constraint
WHERE contype = 'f'
AND connamespace = 'public'::regnamespace
ORDER BY conrelid::regclass::text, contype DESC;
Next, run all the commands. If there are any foreign key violations, they will show up when the alter table add constraint command tries to run. Make a note of which ones had a problem and keep running the following commands until you have a list of all issues.
The following is example of sql commands to fix the issues.
delete from child_table where id_child_table in (
select id_child_table from child_table
where id_parent_id not in (
select id_parent_id
from parent_table
)
);
alter table child_table drop constraint child_table_fk1;
alter table child_table add constraint child_table_fk1 FOREIGN KEY (id_parent_table) REFERENCES parent_table(id_parent_table) ON UPDATE CASCADE ON DELETE CASCADE DEFERRABLE;

Answer 1

Using a basic data sample :
create table test1 (id1 serial, seq1 int, constraint pk1 primary key (id1, seq1));
create table test2 (id2 int, seq2 int, descr2 text, constraint pk2 primary key (id2, seq2), constraint fk1 foreign key (id2, seq2) references test1 (id1, seq1) ON DELETE RESTRICT ON UPDATE RESTRICT) ;
insert into test1 (seq1) values (1), (2), (3), (4), (5);
insert into test2 (id2, seq2, descr2) values (1,1,'A'), (2,2,'B'), (3,3,'C'), (4,4,'D'), (5,5,'E');
Considering the query to get the list of foreign keys (source internet) :
SELECT conrelid::regclass AS table_name,
conname AS foreign_key,
pg_get_constraintdef(oid)
FROM pg_constraint
WHERE contype = 'f'
AND connamespace = 'public'::regnamespace
ORDER BY conrelid::regclass::text, contype DESC;
Result :
table_name
foreign_key
pg_get_constraintdef
test2
fk1
FOREIGN KEY (id2, seq2) REFERENCES test1(id1, seq1) ON UPDATE RESTRICT ON DELETE RESTRICT
With some changes to get a computable resulting list :
SELECT conrelid::regclass AS ReferencingTable
, a.ReferencingKey
, b.ReferencedTable
, b.ReferencedKey
FROM pg_constraint AS pg
CROSS JOIN LATERAL
( SELECT '(' || string_agg(conrelid::regclass || '.' || fkey, ',' ORDER BY fkey.ORDINALITY) || ')' AS ReferencingKey
FROM pg_get_constraintdef(pg.oid) AS fk
CROSS JOIN LATERAL string_to_table(translate(regexp_substr(fk, '\([^\)]*\)', 1, 1), '() ', ''), ',') WITH ORDINALITY AS fkey
) AS a
CROSS JOIN LATERAL
( SELECT '(' || string_agg(ReferencedTable || '.' || rkey, ',' ORDER BY rkey.ORDINALITY) || ')' AS ReferencedKey
, ReferencedTable
FROM pg_get_constraintdef(pg.oid) AS fk
CROSS JOIN LATERAL split_part((regexp_match(fk, 'REFERENCES\s\w+'))[1], ' ', 2) AS ReferencedTable
CROSS JOIN LATERAL string_to_table(translate(regexp_substr(fk, '\([^\)]*\)', 1, 2), '() ', ''), ',') WITH ORDINALITY AS rkey
GROUP BY ReferencedTable
) AS b
WHERE contype = 'f'
AND connamespace = 'public' ::regnamespace
Result :
referencingtable
referencingkey
referencedtable
referencedkey
test2
(test2.id2,test2.seq2)
test1
(test1.id1,test1.seq1)
Creating a plpgsql function with schema_name as input data and with a dynamic query :
CREATE OR REPLACE FUNCTION test(IN schema_name text, OUT Referencing_Table text, OUT Referencing_Key text, OUT Referenced_Table text)
RETURNS setof record LANGUAGE plpgsql AS
$$
DECLARE
_row record ;
BEGIN
FOR _row IN
( SELECT conrelid::regclass AS ReferencingTable
, a.ReferencingKey
, b.ReferencedTable
, b.ReferencedKey
FROM pg_constraint AS pg
CROSS JOIN LATERAL
( SELECT '(' || string_agg(conrelid::regclass || '.' || fkey, ',' ORDER BY fkey.ORDINALITY) || ')' AS ReferencingKey
FROM pg_get_constraintdef(pg.oid) AS fk
CROSS JOIN LATERAL string_to_table(translate(regexp_substr(fk, '\([^\)]*\)', 1, 1), '() ', ''), ',') WITH ORDINALITY AS fkey
) AS a
CROSS JOIN LATERAL
( SELECT '(' || string_agg(ReferencedTable || '.' || rkey, ',' ORDER BY rkey.ORDINALITY) || ')' AS ReferencedKey
, ReferencedTable
FROM pg_get_constraintdef(pg.oid) AS fk
CROSS JOIN LATERAL split_part((regexp_match(fk, 'REFERENCES\s\w+'))[1], ' ', 2) AS ReferencedTable
CROSS JOIN LATERAL string_to_table(translate(regexp_substr(fk, '\([^\)]*\)', 1, 2), '() ', ''), ',') WITH ORDINALITY AS rkey
GROUP BY ReferencedTable
) AS b
WHERE contype = 'f'
AND connamespace = schema_name ::regnamespace )
LOOP
RETURN QUERY EXECUTE FORMAT (
'SELECT %L :: text AS Referencing_Table, %s :: text AS Referencing_Key, %L :: text AS Referenced_Table
FROM %I LEFT JOIN %I ON %s = %s
WHERE %s IS NULL'
, _row.ReferencingTable
, _row.ReferencingKey
, _row.ReferencedTable
, _row.ReferencingTable
, _row.ReferencedTable
, _row.ReferencedKey
, _row.ReferencingKey
, _row.ReferencedKey
) ;
END LOOP ;
END ;
$$ ;
SELECT * FROM test('public') should return the list of foreign keys of any table Referencing_Table in schema schema_name with no correspondance in table Referenced_Table
but not able to test the result in dbfiddle because not able to break the foreign key (superuser privilege)