As a database developer, I experienced this notice when I tried to make a data-only dump a PostgreSQL(10.1) database 'tlesson'.
Notice =>
pg_dump: NOTICE: there are circular foreign-key constraints on this table:
pg_dump: members
Dump command =>
$ pg_dump -U postgres -d translesson -a
A 'tlesson' table 'members' constraint =>
ALTER TABLE ONLY members
ADD CONSTRAINT friend_fk FOREIGN KEY (friend_id) REFERENCES members(member_id);
That is, 'friend_id' column refers own table's primary key as the foreign-key.
Should I drop the 'friend_fk' constraint to remove the notice I'm having?
If you always drop the entire database then this isn't a problem, because the generated SQL (or pg_restore) will enable (create) foreign keys only after all the data was loaded, so there is no problem in that case.
However if you only dump a single table without the FKs then, importing is only going to work if you manually drop the FK before restoring, then re-create it afterwards.
The reason is that it's nearly impossible to generate INSERT statements in the correct order if you have circular references
Related
Precondition:
I am using Liquibase with SQL scripts in my app. I started testing with Oracle DB, but now I need to switch to PostgreSQL DB.
Problem:
When I added constraints, I didn't add the names of the constraints.
Liquibase changelog contains a script for dropping unique and primary key constraints:
alter table SCENARIO drop primary key/
alter table SCENARIO drop unique (OWNER_ID)/
This syntax doesn't sync with PostgreSQL
Could you give some advice, on how to resolve this problem?
Screenshots:
Using Postgres 10.6
The issue:
Some data in my tables violates the foreign key constraints (not sure how). The constraints are ON DELETE CASCADE ON UPDATE CASCADE
On a pg_dump of the database, those foreign keys are dropped (due to being in an invalid state?)
A pg_restore is done into a blank database, which no longer has the foreign keys
The new database has all its primary keys updated to valid keys not used in a second database. Tables which had invalid data do not have their foreign keys updated, due to the now missing constraint.
A pg_dump of the new database is done, then the database is deleted
On a pg_restore into a second database which has the foreign key constraints, the data gets imported in an invalid state, and corrupts the new database.
What I want to do is this: Every few hours (or once a day, depending of how long the query would take), is to verify that all data in all the tables which have foreign keys are valid.
I have read about ALTER TABLE ... VALIDATE CONSTRAINT ... but this wouldn't fix my issue, as the data is not currently marked as NOT VALID. I know could do statements like:
DELETE FROM a WHERE a.b_id NOT IN ( SELECT b.id )
However, I have 144 tables with foreign keys, so this would be rather tedious. I would also maybe not want to immediately delete the data, but log the issue and inform user about a correction which will happen.
Of course, I'd like to know how the original corruption occurred, and prevent that; however at the moment I'm just trying to prevent it from spreading.
Example table:
CREATE TABLE dependencies (
...
from_task int references tasks(id) ON DELETE CASCADE ON UPDATE CASCADE NOT NULL,
to_task int references tasks(id) ON DELETE CASCADE ON UPDATE CASCADE NOT NULL,
...
);
Dependencies will end up with values for to_task and from_task which do not exist in the tasks table (see image)
Note:
Have tried EXPLAIN ANALYZE nothing odd
pg_tablespace, has just two records. pg_default and pg_global
relforcerowsecurity, relispartition are both 'false' on both tables
Arguments to pg_dump (from c++ call) arguments << "--file=" + fileName << "--username=" + connection.userName() << databaseName << "--format=c"
This is either an index (or table) corruption problem, or the constraint has been created invalid to defer the validity check till later.
pg_dump will never silently "drop" a constraint — perhaps there was an error while restoring the dump that you didn't notice.
The proper fix is to clean up the data that violate the constraint and re-create it.
If it is a data corruption problem, check your hardware.
There is no need to regularly check for data corruption, PostgreSQL is not in the habit of corrupting data by itself.
The best test would be to take a pg_dump regularly and see if restoring the dump causes any errors.
Postgres on Linux
When I do the following command.
DELETE FROM some_table;
It also deletes data from another table. How do I found out how?
There is probably a foreign key constraint that points to your table and is defined with ON DELETE CASCADE.
Alternatively, there may be a trigger on the table that deletes the rows.
In psql, use \d some_table to see all such foreign keys and triggers.
I searched for this problem. But my postgres user has enough grant and I do not think I have misspelling error. However I am newbie.
I have this error message:
21:38:03 set search_path='public'
21:38:03 ALTER TABLE public.tbl_user DROP CONSTRAINT "fk-user-access-user-id"
21:38:03 ERROR: constraint "fk-user-access-user-id" of relation "tbl_user" does not exist
I use the PhpStorm. I just open the database view, expanded the tbl_user table, right click and select "drop". And I got this error in the console.
So the above SQL command generated by the PhpStorm.
Then I tried with these commands manually on Ubuntu:
ALTER TABLE tbl_user DROP CONSTRAINT "fk-user-access-user-id"
ALTER TABLE "tbl_user" DROP CONSTRAINT "fk-user-access-user-id"
But I get the same error.
In the PhpStorm I see this definition:
"fk-user-access-user-id" FOREIGN KEY (access_id) REFERENCES tbl_access (id)
The tbl_access table exists with the primary id key.
I do not understand this error message, because the "fk-user-access-user-id" foreign key is at the tbl_user and so for me the 'relation "tbl_user" does not exist' strange. I do not understand.
I tried to find similar problem on StackOverflow, but I gave up after 20x question reading.
By the way, the postgres code was generated by the Yii framework.
$this->addColumn('{{%user}}', 'access_id', $this->integer()->notNull()->defaultValue(1)->after('status'));
$this->addForeignKey('fk-user-access-user-id', '{{%user}}', 'access_id', '{{%access}}', 'id');
first row mean add access_id column to the user table.
second row: create foreign key with 'fk-user...' name on tbl_user table's access_id column references to tbl_access table's id column.
So I used this PHP code to generate this SQL commands. I prefer this way because for me the migration files are very useful.
Most likely the definition and actual implementation in your underlying DB has changed from what the app has recorded. Depending on what the history is, either a change in the app for that foreign key relationship was not migrated to persist the change at the database level, or someone has executed some operation directly at the DB level to remove the relationship. You will need to sync up the app layer to the DB at this point I would think.
I have run into problems when restoring a PostgreSQL database schema in another server. More precisely, some of the tables don't seem to have the same foreign key constraints associated with them that they used to in the original database. For example, the ON DELETE CASCADE clause seems to have completely evaporated from all of the constraint definitions.
That's probably because the dumping procedure didn't backup the ON DELETE CASCADE clauses in your table definitions.
Firstly you should delete the foreign key constraints on your tables and then go on to altering them:
Something like the following:
ALTER TABLE ONLY *your_table* DROP CONSTRAINT your_constraint;
After that, recreate the constraints with something like:
ALTER TABLE ONLY your_table ADD CONSTRAINT your_constraint (...ON DELETE CASCADE, etc..);