I have a table with approx 5 million rows which has a fk constraint referencing the primary key of another table (also approx 5 million rows).
I need to delete about 75000 rows from both tables. I know that if I try doing this with the fk constraint enabled it's going to take an unacceptable amount of time.
Coming from an Oracle background my first thought was to disable the constraint, do the delete & then reenable the constraint. PostGres appears to let me disable constraint triggers if I am a super user (I'm not, but I am logging in as the user that owns/created the objects) but that doesn't seem to be quite what I want.
The other option is to drop the constraint and then reinstate it. I'm worried that rebuilding the constraint is going to take ages given the size of my tables.
Any thoughts?
edit: after Billy's encouragement I've tried doing the delete without changing any constraints and it takes in excess of 10 minutes. However, I have discovered that the table from which I'm trying to delete has a self referential foreign key ... duplicated (& non indexed).
Final update - I dropped the self referential foreign key, did my delete and added it back in. Billy's right all round but unfortunately I can't accept his comment as the answer!
Per previous comments, it should be a problem. That said, there is a command that may be what you're looking to - it'll set the constraints to deferred so they're checked on COMMIT, not on every delete. If you're doing just one big DELETE of all the rows, it won't make a difference, but if you're doing it in pieces, it will.
SET CONSTRAINTS ALL DEFERRED
is what you are looking for in that case. Note that constraints must be marked as DEFERRABLE before they can be deferred. For example:
ALTER TABLE table_name
ADD CONSTRAINT constraint_uk UNIQUE(column_1, column_2)
DEFERRABLE INITIALLY IMMEDIATE;
The constraint can then be deferred in a transaction or function as follows:
CREATE OR REPLACE FUNCTION f() RETURNS void AS
$BODY$
BEGIN
SET CONSTRAINTS ALL DEFERRED;
-- Code that temporarily violates the constraint...
-- UPDATE table_name ...
END;
$BODY$
LANGUAGE plpgsql VOLATILE
COST 100;
What worked for me was to disable one by one the TRIGGERS of those tables that are gonna be involved in the DELETE operation.
ALTER TABLE reference DISABLE TRIGGER ALL;
DELETE FROM reference WHERE refered_id > 1;
ALTER TABLE reference ENABLE TRIGGER ALL;
Solution is working in version 9.3.16. In my case time went from 45 minutes to 14 seconds executing DELETE operations.
As stated in the comments section by #amphetamachine, you will need to have admin privileges to the tables to perform this task.
If you try DISABLE TRIGGER ALL and get an error like permission denied: "RI_ConstraintTrigger_a_16428" is a system trigger (I got this on Amazon RDS), try this:
set session_replication_role to replica;
If this succeeds, all triggers that underlie table constraints will be disabled. Now it's up to you to make sure your changes leave the DB in a consistent state!
Then when you are done, reenable triggers & constraints for your session with:
set session_replication_role to default;
(This answer assumes your intent is to delete all of the rows of these tables, not just a selection.)
I also had to do this, but as part of a test suite. I found the answer, suggested elsewhere on SO. Use TRUNCATE TABLE as follows:
TRUNCATE TABLE <list-of-table-names> [RESTART IDENTITY] [CASCADE];
The following quickly deletes all rows from tables table1, table2, and table3, provided that there are no references to rows of these tables from tables not listed:
TRUNCATE TABLE table1, table2, table3;
As long as references are between the tables listed, PostgreSQL will delete all the rows without concern for referential integrity. If a table other than those listed references a row of one of these tables, the query will fail.
However, you can qualify the query so that it also truncates all tables with references to the listed tables (although I have not tried this):
TRUNCATE TABLE table1, table2, table3 CASCADE;
By default, the sequences of these tables do not restart numbering. New rows will continue with the next number of the sequence. To restart sequence numbering:
TRUNCATE TABLE table1, table2, table3 RESTART IDENTITY;
My PostgreSQL is 9.6.8.
set session_replication_role to replica;
work for me but I need permission.
I login psql with super user.
sudo -u postgres psql
Then connect to my database
\c myDB
And run:
set session_replication_role to replica;
Now I can delete from table with constraint.
Disable all table constraints
ALTER TABLE TableName NOCHECK CONSTRAINT ConstraintName
-- Enable all table constraints
ALTER TABLE TableName CHECK CONSTRAINT ConstraintName
Related
I am writing an idempotent schema change script for a Postgres 12 database. However I noticed that if I include the IF NOT EXISTS in an ADD COLUMN statement then even if the column already exists it is adding duplicate Indexes for the uniqueness constraint which already exists. Simple example:
-- set up base table
CREATE TABLE IF NOT EXISTS test_table
(id SERIAL PRIMARY KEY
);
-- statement intended to be idempotent
ALTER TABLE test_table
ADD COLUMN IF NOT EXISTS name varchar(50) UNIQUE;
Running this script creates a new index test_table_name_key[n] each time it is run. I can't find anything in the Postgres documentation and don't understand why this is allowed to happen? If I break it into two parts eg:
ALTER TABLE test_table
ADD COLUMN IF NOT EXISTS name varchar(50);
ALTER TABLE
ADD CONSTRAINT test_table_name_key UNIQUE (name);
Then the transaction fails because Postgres rejects the creation of a constraint which already exists (which I can then catch in a DO EXCEPTION block). As far as I can tell this is because doing it by this approach I am forced to give the constraint a name. This constrasts with the ALTER COLUMN SET NOT NULL which can be run multiple times without error or side effects as far as I can tell.
Question: why does it add a duplicate unique constraint and are there any problems with having multiple identical indexes on a table column? (I think this is a subtle 'error' and only spotted it by chance so am concerned it may arise in a production situation)
You can create multiple unique constraints on the same column as long as they have different names, simply because there is nothing in the PostgreSQL code that forbids that. Each unique constraint will create a unique index with the same name, because that is how unique constraints are implemented.
This can be a valid use case: for example, if the index is bloated, you could create a new constraint and then drop the old one.
But normally, it is useless and does harm, because each index will make data modifications on the table slower.
I wanted to remove all foreign key constraints from a schema. I was successful in dropping constraints from most of the tables but in few of them drop foreign key constraint query is getting stuck.
ALTER TABLE table_name DROP CONSTRAINT fkey_name;
I tried truncate cascade but it was also getting stuck. I deleted all rows from both the tables manually. Still getting stuck.
Edits: By getting stuck I mean query continues running for long time without any error message even though tables are empty.
Check for any dead locks using
SELECT * FROM pg_stat_activity;
If any then kill and run below sql,and then drop using
SELECT pg_terminate_backend(pid);
If Not solved check for any virtual transaction
SELECT database, gid FROM pg_prepared_xacts;
Rollback using
ROLLBACK PREPARED 'gid';
Postgres on Linux
When I do the following command.
DELETE FROM some_table;
It also deletes data from another table. How do I found out how?
There is probably a foreign key constraint that points to your table and is defined with ON DELETE CASCADE.
Alternatively, there may be a trigger on the table that deletes the rows.
In psql, use \d some_table to see all such foreign keys and triggers.
I'm trying to refresh some data that's referred to by other data - I want to truncate and reload the ms_automobile table, but the rm_automobile table has a foreign key to it.
It looks like the 'DISABLE TRIGGER' statements are working (run as postgres, a superuser):
mobilesurvey=# ALTER TABLE ms_automobile DISABLE TRIGGER ALL;
ALTER TABLE
mobilesurvey=# ALTER TABLE rm_automobile DISABLE TRIGGER ALL;
ALTER TABLE
But I can't then truncate the ms_automobile table:
mobilesurvey=# TRUNCATE TABLE ms_automobile;
ERROR: cannot truncate a table referenced in a foreign key constraint
DETAIL: Table "rm_automobile" references "ms_automobile".
HINT: Truncate table "rm_automobile" at the same time, or use TRUNCATE ... CASCADE.
Again, I do not want to lose the rm_automobile data; after the TRUNCATE I'm planning on doing a pg_restore that includes the missing ms_automobile data.
If possible, I'd like to disable instead of dropping the constraints - there are more of them, and maintaining disable/enable seems a lot less error-prone than maintaining drop/add.
So, how can I actually disable the foreign keys here?
Disabling triggers works as you expect on DELETE (and not on TRUNCATE).
DELETE FROM ms_automobile;
TRUNCATE is implemented in the specific way different from INSERT/UPDATE/DELETE. It doesn't use triggers but checks referential integrity once before its execution.
I have run into problems when restoring a PostgreSQL database schema in another server. More precisely, some of the tables don't seem to have the same foreign key constraints associated with them that they used to in the original database. For example, the ON DELETE CASCADE clause seems to have completely evaporated from all of the constraint definitions.
That's probably because the dumping procedure didn't backup the ON DELETE CASCADE clauses in your table definitions.
Firstly you should delete the foreign key constraints on your tables and then go on to altering them:
Something like the following:
ALTER TABLE ONLY *your_table* DROP CONSTRAINT your_constraint;
After that, recreate the constraints with something like:
ALTER TABLE ONLY your_table ADD CONSTRAINT your_constraint (...ON DELETE CASCADE, etc..);