We recently discovered an issue with our database where a foreign key constraint was not working correctly. Basically the primary table did not have any primary ids that matched the foreign key in the child table. When we dropped the foreign key constraint and tried to recreate it, it then threw an error that the foreign key constraint could not be created because there were foreign keys with no matching id in the parent table. Once those were cleaned up, it allowed us to recreate the foreign key.
Of course we are wondering how this happened to begin with. I worked with Oracle for 15 years and never saw a foreign key failing in this way. But our concern right now is how many other foreign keys are not working correctly. This is a problem because we have some BEFORE DELETE triggers that fail silently when the calling function returns a null because of a foreign_key_violation (this is how we discovered the issue to begin with).
EXCEPTION
WHEN foreign_key_violation
THEN RETURN NULL;
What we want to do is get all of the foreign keys in the database (probably a few thousand), loop over them all, and check every single one against it's parent table to see if any are "broken".
Basically:
Select all foreign keys using Postgres system tables.
Loop over all of them and do something like:
select count(parent_id) from child_table
where foreign_key_id not in (
select parent_id as foreign_key_id
from parent_table
)
);
For all the ones that are not 0, drop the foreign key constraint, fix the orphaned data, and recreate the foreign key constraint.
Does this sound reasonable? Has anyone done something like this before? What is the best way to get the foreign key constraints from Postgres?
Edit:
What we realized is that if we dropped and recreated the foreign keys, it would tell us which ones were problematic.
SELECT 'Alter table ' || conrelid::regclass || ' drop constraint ' || conname || '; alter table ' || conrelid::regclass || ' add constraint ' || conname || ' ' || pg_get_constraintdef(oid) || ';'
FROM pg_constraint
WHERE contype = 'f'
AND connamespace = 'public'::regnamespace
ORDER BY conrelid::regclass::text, contype DESC;
Next, run all the commands. If there are any foreign key violations, they will show up when the alter table add constraint command tries to run. Make a note of which ones had a problem and keep running the following commands until you have a list of all issues.
The following is example of sql commands to fix the issues.
delete from child_table where id_child_table in (
select id_child_table from child_table
where id_parent_id not in (
select id_parent_id
from parent_table
)
);
alter table child_table drop constraint child_table_fk1;
alter table child_table add constraint child_table_fk1 FOREIGN KEY (id_parent_table) REFERENCES parent_table(id_parent_table) ON UPDATE CASCADE ON DELETE CASCADE DEFERRABLE;
Using a basic data sample :
create table test1 (id1 serial, seq1 int, constraint pk1 primary key (id1, seq1));
create table test2 (id2 int, seq2 int, descr2 text, constraint pk2 primary key (id2, seq2), constraint fk1 foreign key (id2, seq2) references test1 (id1, seq1) ON DELETE RESTRICT ON UPDATE RESTRICT) ;
insert into test1 (seq1) values (1), (2), (3), (4), (5);
insert into test2 (id2, seq2, descr2) values (1,1,'A'), (2,2,'B'), (3,3,'C'), (4,4,'D'), (5,5,'E');
Considering the query to get the list of foreign keys (source internet) :
SELECT conrelid::regclass AS table_name,
conname AS foreign_key,
pg_get_constraintdef(oid)
FROM pg_constraint
WHERE contype = 'f'
AND connamespace = 'public'::regnamespace
ORDER BY conrelid::regclass::text, contype DESC;
Result :
table_name
foreign_key
pg_get_constraintdef
test2
fk1
FOREIGN KEY (id2, seq2) REFERENCES test1(id1, seq1) ON UPDATE RESTRICT ON DELETE RESTRICT
With some changes to get a computable resulting list :
SELECT conrelid::regclass AS ReferencingTable
, a.ReferencingKey
, b.ReferencedTable
, b.ReferencedKey
FROM pg_constraint AS pg
CROSS JOIN LATERAL
( SELECT '(' || string_agg(conrelid::regclass || '.' || fkey, ',' ORDER BY fkey.ORDINALITY) || ')' AS ReferencingKey
FROM pg_get_constraintdef(pg.oid) AS fk
CROSS JOIN LATERAL string_to_table(translate(regexp_substr(fk, '\([^\)]*\)', 1, 1), '() ', ''), ',') WITH ORDINALITY AS fkey
) AS a
CROSS JOIN LATERAL
( SELECT '(' || string_agg(ReferencedTable || '.' || rkey, ',' ORDER BY rkey.ORDINALITY) || ')' AS ReferencedKey
, ReferencedTable
FROM pg_get_constraintdef(pg.oid) AS fk
CROSS JOIN LATERAL split_part((regexp_match(fk, 'REFERENCES\s\w+'))[1], ' ', 2) AS ReferencedTable
CROSS JOIN LATERAL string_to_table(translate(regexp_substr(fk, '\([^\)]*\)', 1, 2), '() ', ''), ',') WITH ORDINALITY AS rkey
GROUP BY ReferencedTable
) AS b
WHERE contype = 'f'
AND connamespace = 'public' ::regnamespace
Result :
referencingtable
referencingkey
referencedtable
referencedkey
test2
(test2.id2,test2.seq2)
test1
(test1.id1,test1.seq1)
Creating a plpgsql function with schema_name as input data and with a dynamic query :
CREATE OR REPLACE FUNCTION test(IN schema_name text, OUT Referencing_Table text, OUT Referencing_Key text, OUT Referenced_Table text)
RETURNS setof record LANGUAGE plpgsql AS
$$
DECLARE
_row record ;
BEGIN
FOR _row IN
( SELECT conrelid::regclass AS ReferencingTable
, a.ReferencingKey
, b.ReferencedTable
, b.ReferencedKey
FROM pg_constraint AS pg
CROSS JOIN LATERAL
( SELECT '(' || string_agg(conrelid::regclass || '.' || fkey, ',' ORDER BY fkey.ORDINALITY) || ')' AS ReferencingKey
FROM pg_get_constraintdef(pg.oid) AS fk
CROSS JOIN LATERAL string_to_table(translate(regexp_substr(fk, '\([^\)]*\)', 1, 1), '() ', ''), ',') WITH ORDINALITY AS fkey
) AS a
CROSS JOIN LATERAL
( SELECT '(' || string_agg(ReferencedTable || '.' || rkey, ',' ORDER BY rkey.ORDINALITY) || ')' AS ReferencedKey
, ReferencedTable
FROM pg_get_constraintdef(pg.oid) AS fk
CROSS JOIN LATERAL split_part((regexp_match(fk, 'REFERENCES\s\w+'))[1], ' ', 2) AS ReferencedTable
CROSS JOIN LATERAL string_to_table(translate(regexp_substr(fk, '\([^\)]*\)', 1, 2), '() ', ''), ',') WITH ORDINALITY AS rkey
GROUP BY ReferencedTable
) AS b
WHERE contype = 'f'
AND connamespace = schema_name ::regnamespace )
LOOP
RETURN QUERY EXECUTE FORMAT (
'SELECT %L :: text AS Referencing_Table, %s :: text AS Referencing_Key, %L :: text AS Referenced_Table
FROM %I LEFT JOIN %I ON %s = %s
WHERE %s IS NULL'
, _row.ReferencingTable
, _row.ReferencingKey
, _row.ReferencedTable
, _row.ReferencingTable
, _row.ReferencedTable
, _row.ReferencedKey
, _row.ReferencingKey
, _row.ReferencedKey
) ;
END LOOP ;
END ;
$$ ;
SELECT * FROM test('public') should return the list of foreign keys of any table Referencing_Table in schema schema_name with no correspondance in table Referenced_Table
but not able to test the result in dbfiddle because not able to break the foreign key (superuser privilege)
How to list all constraints of all tables in PostgreSQL?
I need to search the unique constraint created a long time ago, the database has so many (600) tables. It's hard to find. Is there any query to list all constraints of all tables
You must refer to pg_constraint.
If you need to list just the unique constraint filter on contype, like this:
select * from pg_catalog.pg_constraint pc
where contype = 'u'
Documentation here: https://www.postgresql.org/docs/current/catalog-pg-constraint.html
The below query will List all the constraints of a table in the PostgreSQL database.
You can try below:-
SELECT con.*
FROM pg_catalog.pg_constraint con
INNER JOIN pg_catalog.pg_class rel ON rel.oid = con.conrelid
INNER JOIN pg_catalog.pg_namespace nsp ON nsp.oid = connamespace
WHERE nsp.nspname = '<schema name>'
AND rel.relname = '<table name>';
Replace with the name of your table & with the name of your schema.
It can be viewed in PSQL as using below:-
d+ <SCHEMA_NAME.TABLE_NAME>
I am using the PostgreSQL. I want to write a query that returns all the column names having foreign key constraint and also the name of the table these columns they refer to.
As far as I can see, the information_schema views don't give you the column names, so you'll have to use the catalog:
SELECT c.conrelid::regclass AS source_table,
a.attname AS column_name,
k.n AS position,
c.confrelid::regclass AS referenced_table
FROM pg_constraint AS c
CROSS JOIN LATERAL unnest(c.conkey) WITH ORDINALITY AS k(attnum, n)
JOIN pg_attribute AS a
ON k.attnum = a.attnum AND c.conrelid = a.attrelid
WHERE c.contype = 'f'
ORDER BY c.conrelid::regclass::text, k.n;
To get the data for only a specific table, add the following to the WHERE condition:
AND c.conrelid = 'mytable'::regclass
I have to drop all constraints in my Postgres database to update datatypes used by PKs and FKs. Before that I want to make sure that I can rebuild those constraints after the update. At first I execute the following code to get a table with three columns table_from, conname and pg_get_constraintdef, containing all constraint information.
select conrelid::regclass AS table_from, conname, pg_get_constraintdef(c.oid)
from pg_constraint c
join pg_namespace n ON n.oid = c.connamespace
where contype in ('f', 'p','c','u') order by contype
table_from contains the table name, e.g. foo. conname the name of the constraint, e.g. fk_foo_2_bar and pg_get_constraintdef contains the constraint definition, e.g.
FOREIGN KEY (foo_id) REFERENCES bar(bar_id) DEFERRABLE INITIALLY DEFERRED
How can I generate all the queries for creating my constraints? I don't want to do it by hand, because there are 1000+ constraints active.
Most of the work was actually done already...
select 'alter table ' || conrelid::regclass || ' add constraint ' || conname || ' ' || pg_get_constraintdef(c.oid)
from pg_constraint c
join pg_namespace n ON n.oid = c.connamespace
where contype in ('f', 'p','c','u') order by contype
I can find all of the foreign keys belonging to a certain table pretty well using the information_schema
But I can't figure out how I can find the foreign keys from OTHER tables which reference a certain table.
All I want to know is which rows from which tables in my database are referencing the primary key of one of my tables.
Is this what you're looking for?
SELECT * FROM pg_constraint WHERE confrelid=<oid of destination table>
Or if you just want to see them interactively, they shown in the output of \d <table name> in psql.
Let's make a few tables we can use for testing.
create table test (
n integer primary key
);
-- There might be more than one schema.
create schema scratch;
create table scratch.a (
test_n integer not null references test (n),
incept_date date not null default current_date,
primary key (test_n, incept_date)
);
create table b (
test_n integer not null references test (n),
incept_date date not null default current_date,
primary key (test_n, incept_date)
);
-- The same table name can exist in different schemas.
create table scratch.b (
test_n integer not null references test (n),
incept_date date not null default current_date,
primary key (test_n, incept_date)
);
I prefer to use information_schema views for this kind of stuff, because what I learn is portable to other database management systems.
I usually leave concatenation up to application programs, but I think it's
easier to understand the output here if I concatenate columns and give them aliases. The careful programmer will use the "full name" in all the joins--catalog (database), schema, and name.
select distinct
KCU2.table_catalog || '.' || KCU2.table_schema || '.' || KCU2.table_name referenced_table,
RC.constraint_catalog || '.' || RC.constraint_schema || '.' || RC.constraint_name full_constraint_name,
KCU1.table_catalog || '.' || KCU1.table_schema || '.' || KCU1.table_name referencing_table
from information_schema.referential_constraints RC
inner join information_schema.key_column_usage KCU1 on
RC.constraint_catalog = KCU1.constraint_catalog and
RC.constraint_schema = KCU1.constraint_schema and
RC.constraint_name = KCU1.constraint_name
inner join information_schema.key_column_usage KCU2 on
RC.unique_constraint_catalog = KCU2.constraint_catalog and
RC.unique_constraint_schema = KCU2.constraint_schema and
RC.unique_constraint_name = KCU2.constraint_name
where
KCU2.table_catalog = 'sandbox' and
KCU2.table_schema = 'public' and
KCU2.table_name = 'test'
order by referenced_table, referencing_table
;
referenced_table full_constraint_name referencing_table
--
sandbox.public.test sandbox.public.b_test_n_fkey sandbox.public.b
sandbox.public.test sandbox.scratch.a_test_n_fkey sandbox.scratch.a
sandbox.public.test sandbox.scratch.b_test_n_fkey sandbox.scratch.b
I think that will get you started. A foreign key doesn't have to reference a primary key; it can reference any candidate key. This query tells you which tables have a foreign key to our test table, sandbox.public.test, which seems to be what you're looking for.