PostgreSQL declarative partition - unique constraint on partitioned table must include all partitioning columns [duplicate] - postgresql

This question already has an answer here:
ERROR: unique constraint on partitioned table must include all partitioning columns
(1 answer)
Closed last month.
I'm trying to create a partitioned table which refers to itself, creating a doubly-linked list.
CREATE TABLE test2 (
id serial NOT NULL,
category integer NOT NULL,
time timestamp(6) NOT NULL,
prev_event integer,
next_event integer
) PARTITION BY HASH (category);
Once I add primary key I get the following error.
alter table test2 add primary key (id);
ERROR: unique constraint on partitioned table must include all partitioning columns
DETAIL: PRIMARY KEY constraint on table "test2" lacks column "category" which is part of the partition key.
Why does the unique constrain require all partitioned columns to be included?
EDIT: Now I understand why this is needed: https://www.postgresql.org/docs/current/ddl-partitioning.html#DDL-PARTITIONING-DECLARATIVE-LIMITATIONS
Once I add PK with both columns it works.
alter table test2 add primary key (id, category);
But then adding the FK to itself doesn't work.
alter table test2 add foreign key (prev_event) references test2 (id) on update cascade on delete cascade;
ERROR: there is no unique constraint matching given keys for referenced table "test2"
Since PK is not just id but id-category I can't create FK pointing to id.
Is there any way to deal with this or am I missing something?
I would like to avoid using inheritance partitioning if possible.
EDIT2: It seems this is a known problem. https://www.reddit.com/r/PostgreSQL/comments/di5mbr/postgresql_12_foreign_keys_and_partitioned_tables/f3tsoop/

Seems that there is no straightforward solution. PostgreSQL simply doesn't support this as of v14. One solution is to use triggers to enforce 'foreign key' behavior. Other is to use multi-column foreign keys. Both are far from optimal.

Related

Citus: How can I add self referencing table in distributed tables list

I'm trying to run create_distributed_table for tables which i need to shard and almost all of the tables have self relation ( parent child )
but when I run SELECT create_distributed_table('table-name','id');
it throws error cannot create foreign key constraint
simple steps to reproduce
CREATE TABLE TEST (
ID TEXT NOT NULL,
NAME CHARACTER VARYING(255) NOT NULL,
PARENT_ID TEXT
);
ALTER TABLE TEST ADD CONSTRAINT TEST_PK PRIMARY KEY (ID);
ALTER TABLE TEST ADD CONSTRAINT TEST_PARENT_FK FOREIGN KEY (PARENT_ID) REFERENCES TEST (ID);
ERROR
citus=> SELECT create_distributed_table('test','id');
ERROR: cannot create foreign key constraint
DETAIL: Foreign keys are supported in two cases, either in between two colocated tables including partition column in the same ordinal in the both tables or from distributed to reference tables
For the time being, it is not possible to shard a table on PostgreSQL without dropping the self referencing foreign key constraints, or altering them to include a separate and new distribution column.
Citus places records into shards based on the hash values of the distribution column values. It is most likely the case that the hashes of parent and child id values are different and hence the records should be stored in different shards, and possibly on different worker nodes. PostgreSQL does not have a mechanism to create foreign key constraints that reference records on different PostgreSQL clusters.
Consider adding a new column tenant_id and adding this column to the primary key and foreign key constraints.
CREATE TABLE TEST (
tenant_id INT NOT NULL,
id TEXT NOT NULL,
name CHARACTER VARYING(255) NOT NULL,
parent_id TEXT NOT NULL,
FOREIGN KEY (tenant_id, parent_id) REFERENCES test(tenant_id, id),
PRIMARY KEY (tenant_id, id)
);
SELECT create_distributed_table('test','tenant_id');
Note that parent and child should always be in the same tenant for this to work.

CREATE TABLE LIKE with different primary key for partitioning

I've got an existing table of dogs which I would like to partition by list using the colour column:
CREATE TABLE dogs (
id int PRIMARY KEY GENERATED ALWAYS AS IDENTITY,
colour text,
name text
)
;
Because it's not possible to partition an existing table, I'm going to make a new empty partitioned table then copy the data across.
CREATE TABLE
trade_capture._customer_invoice
(
LIKE
trade_capture.customer_invoice
INCLUDING ALL
)
PARTITION BY LIST (bill_month)
;
This reports:
ERROR: insufficient columns in PRIMARY KEY constraint definition
DETAIL: PRIMARY KEY constraint on table "_dogs" lacks column "colour" which is part of the partition key.
I know I can ignore the primary key like this, but it seems bad to have a table with no primary key!
CREATE TABLE
_dogs
(
LIKE
dogs
INCLUDING ALL
EXCLUDING INDEXES
)
PARTITION BY LIST (colour)
;
What's the best way to proceed so I have a partitioned table which still has a primary key?
Add the partitioning key to the primary key:
ALTER TABLE trade_capture._customer_invoice
ADD PRIMARY KEY (id, bill_month);
There is no other option to have a unique constraint on a partitioned table.

UNIQUE constraint on table "subscriber_historization" lacks column "processing_date" which is part of the partition key

I want to use constraint so i can use upsert. Because i don't want duplicate entry on customer_identifier_value.
on conflict (customer_identifier_value) do nothing
[42P10] ERROR: there is no unique or exclusion constraint matching the ON CONFLICT specification
When i create the constraint
alter table subscriber_historization
add constraint customer_identifier_value_unique unique (customer_identifier_value);
[0A000] ERROR: insufficient columns in UNIQUE constraint definition
Detail: UNIQUE constraint on table "subscriber_historization" lacks column "processing_date" which is part of the partition key.
Here is the DDL.
-- auto-generated definition
create table subscriber_historization
(
customer_identifier_value text not null,
product_value text,
contract_date_end date,
processing_date date not null,
constraint subscriber_historization_pk
primary key (processing_date, customer_identifier_value)
)
partition by RANGE (processing_date);
If i use
ON CONFLICT ON CONSTRAINT subscriber_historization_pk DO NOTHING
The row will be inserted if process_date is different. Then there will be duplicate entry on customer_identifier_value.
How to use upsert then?
Thanks for your help.
You cannot prevent that with partitioned tables, because all unique indexes must contain the partitioning key.
Your only way out is to use SERIALIZABLE transaction isolation throughout and verify the constraint with a trigger. This will be a performance hit, however.
This is a limitation of partitioning.

Dropping Unique Constraint - PostgreSQL

TL;DR
I am seeking clarity on this: does a FOREIGN KEY require a UNIQUE CONSTRAINT on the other side, specifically, in Postgres and, generally, in relational database systems?
Perhaps, I can test this, but I'll ask, if the UNIQUE CONSTRAINT is required by the FOREIGN KEY what would happen if I don't create it? Will the Database create one or will it throw an error?
How I got there
I had earlier on created a table with a column username on which I imposed a unique constraint. I then created another table with a column bearer_name having a FOREIGN KEY referencing the previous table's column username; the one which had a UNIQUE CONSTRAINT.
Now, I want to drop the UNIQUE CONSTRAINT on the username column from the database because I have later on created a UNIQUE INDEX on the same column and intuitively I feel that they serve the same purpose, or don't they? But the database is complaining that the UNIQUE INDEX has some dependent objects and so it can't be dropped unless I provide CASCADE as an option in order to drop even the dependent object. It's identifying the FOREIGN KEY on bearer_name column in the second table as the dependent object.
And is it possible for the FOREIGN KEY to be a point to the UNIQUE INDEX instead of the UNIQUE CONSTRAINT?
I am seeking clarity on this: does a FOREIGN KEY require a UNIQUE CONSTRAINT on the other side
No it does not require only UNIQUE CONSTRAINT. It could be PRIMARY KEY or UNIQUE INDEX.
Perhaps, I can test this, but I'll ask, if the UNIQUE CONSTRAINT is required by the FOREIGN KEY what would happen if I don't create it? Will the Database create one or will it throw an error?
CREATE TABLE tab_a(a_id INT, b_id INT);
CREATE TABLE tab_b(b_id INT);
ALTER TABLE tab_a ADD CONSTRAINT fk_tab_a_tab_b FOREIGN KEY (b_id)
REFERENCES tab_b(b_id);
ERROR: there is no unique constraint matching given keys
for referenced table "tab_b"
DBFiddle Demo
And is it possible for the FOREIGN KEY to be a point to the UNIQUE INDEX instead of the UNIQUE CONSTRAINT?
Yes, it is possible.
CREATE UNIQUE INDEX tab_b_i ON tab_b(b_id);
DBFiddle Demo2

Why does this foreign key using inheritance not work? [duplicate]

This question already has answers here:
PostgreSQL foreign key not existing, issue of inheritance?
(2 answers)
Closed 8 years ago.
create table abstract_addresses (
address_id int primary key
);
create table phone_numbers (
phone_number text not null unique
) inherits (abstract_addresses) ;
create table contacts (
name text primary key,
address_id int not null references abstract_addresses(address_id)
);
insert into phone_numbers values (1, '18005551212'); --works
select * from abstract_addresses;
address_id
1
select * from phone_numbers;
address_id phone_number
1 18005551212
insert into contacts values ('Neil', 1); --error
I get this error message:
ERROR: insert or update on table "contacts" violates foreign key constraint "contacts_address_id_fkey"
SQL state: 23503
Detail: Key (address_id)=(1) is not present in table "abstract_addresses".
Just a bad use-case for postgresql table inheritance?
Per the caveats in the docs:
A serious limitation of the inheritance feature is that indexes (including unique constraints) and foreign key constraints only apply to single tables, not to their inheritance children. This is true on both the referencing and referenced sides of a foreign key constraint.
http://www.postgresql.org/docs/current/static/ddl-inherit.html
To do what you want:
Create a table with only an id — like you did.
Don't use inheritance. Really don't. It's useful to partition log tables; not for what you're doing.
Make phone number ids default to nextval('abstract_addresses_address_id_seq'), or whatever the sequence name is.
Add a foreign key in phone_numbers referencing abstract_addresses (address_id). Make it deferrable, initially deferred.
Add an after insert trigger on phone_numbers that inserts a new row in abstract_addresses when needed.
If appropriate, add an after delete trigger on phone_numbers that cascade deletes abstract_addresses — make sure it occurs after the delete, else affected rows will report incorrect values when you delete from phone_numbers.
That way, you'll have an abstract_address for use in occasional tables that need such a thing, while still being able to have a hard reference to phone_numbers where the latter is what you actually want.
One caveat to be aware of: it doesn't play well with ORMs.