I want to use constraint so i can use upsert. Because i don't want duplicate entry on customer_identifier_value.
on conflict (customer_identifier_value) do nothing
[42P10] ERROR: there is no unique or exclusion constraint matching the ON CONFLICT specification
When i create the constraint
alter table subscriber_historization
add constraint customer_identifier_value_unique unique (customer_identifier_value);
[0A000] ERROR: insufficient columns in UNIQUE constraint definition
Detail: UNIQUE constraint on table "subscriber_historization" lacks column "processing_date" which is part of the partition key.
Here is the DDL.
-- auto-generated definition
create table subscriber_historization
(
customer_identifier_value text not null,
product_value text,
contract_date_end date,
processing_date date not null,
constraint subscriber_historization_pk
primary key (processing_date, customer_identifier_value)
)
partition by RANGE (processing_date);
If i use
ON CONFLICT ON CONSTRAINT subscriber_historization_pk DO NOTHING
The row will be inserted if process_date is different. Then there will be duplicate entry on customer_identifier_value.
How to use upsert then?
Thanks for your help.
You cannot prevent that with partitioned tables, because all unique indexes must contain the partitioning key.
Your only way out is to use SERIALIZABLE transaction isolation throughout and verify the constraint with a trigger. This will be a performance hit, however.
This is a limitation of partitioning.
Related
I am recreating an existing table as a partitioned table in PostgreSQL 11.
After some research, I am approaching it using the following procedure so this can be done online while writes are still happening on the table:
add a check constraint on the existing table, first as not valid and then validating
drop the existing primary key
rename the existing table
create the partitioned table under the prior table name
attach the existing table as a partition to the new partitioned table
My expectation was that the last step would be relatively fast, but I don't really have a number for this. In my testing, it's taking about 30s. I wonder if my expectations are incorrect or if I'm doing something wrong with the constraint or anything else.
Here's a simplified version of the DDL.
First, the inserted_at column is declared like this:
inserted_at timestamp without time zone not null
I want to have an index on the ID even after I drop the PK for existing queries and writes, so I create an index:
create unique index concurrently my_events_temp_id_index on my_events (id);
The check constraint is created in one transaction:
alter table my_events add constraint my_events_2022_07_events_check
check (inserted_at >= '2018-01-01' and inserted_at < '2022-08-01')
not valid;
In the next transaction, it's validated (and the validation is successful):
alter table my_events validate constraint my_events_2022_07_events_check;
Then before creating the partitioned table, I drop the primary key of the existing table:
alter table my_events drop constraint my_events_pkey cascade;
Finally, in its own transaction, the partitioned table is created:
alter table my_events rename to my_events_2022_07;
create table my_events (
id uuid not null,
... other columns,
inserted_at timestamp without time zone not null,
primary key (id, inserted_at)
) partition by range (inserted_at);
alter table my_events attach partition my_events_2022_07
for values from ('2018-01-01') to ('2022-08-01');
That last transaction blocks inserts and takes about 30s for the 12M rows in my test database.
Edit
I wanted to add that in response to the attach I see this:
INFO: partition constraint for table "my_events_2022_07" is implied by existing constraints
That makes me think I'm doing this right.
The problem is not the check constraint, it is the primary key.
If you make the original unique index include both columns:
create unique index concurrently my_events_temp_id_index on my_events (id,inserted_at);
And if you make the new table have a unique index rather than a primary key on those two columns, then the attach is nearly instantaneous.
These seem to me like unneeded restrictions in PostgreSQL, both that the unique index on one column can't be used to imply uniqueness on the both columns, and that the unique index on both columns cannot be used to imply the primary key (nor even a unique constraint--but only a unique index).
This question already has an answer here:
ERROR: unique constraint on partitioned table must include all partitioning columns
(1 answer)
Closed last month.
I'm trying to create a partitioned table which refers to itself, creating a doubly-linked list.
CREATE TABLE test2 (
id serial NOT NULL,
category integer NOT NULL,
time timestamp(6) NOT NULL,
prev_event integer,
next_event integer
) PARTITION BY HASH (category);
Once I add primary key I get the following error.
alter table test2 add primary key (id);
ERROR: unique constraint on partitioned table must include all partitioning columns
DETAIL: PRIMARY KEY constraint on table "test2" lacks column "category" which is part of the partition key.
Why does the unique constrain require all partitioned columns to be included?
EDIT: Now I understand why this is needed: https://www.postgresql.org/docs/current/ddl-partitioning.html#DDL-PARTITIONING-DECLARATIVE-LIMITATIONS
Once I add PK with both columns it works.
alter table test2 add primary key (id, category);
But then adding the FK to itself doesn't work.
alter table test2 add foreign key (prev_event) references test2 (id) on update cascade on delete cascade;
ERROR: there is no unique constraint matching given keys for referenced table "test2"
Since PK is not just id but id-category I can't create FK pointing to id.
Is there any way to deal with this or am I missing something?
I would like to avoid using inheritance partitioning if possible.
EDIT2: It seems this is a known problem. https://www.reddit.com/r/PostgreSQL/comments/di5mbr/postgresql_12_foreign_keys_and_partitioned_tables/f3tsoop/
Seems that there is no straightforward solution. PostgreSQL simply doesn't support this as of v14. One solution is to use triggers to enforce 'foreign key' behavior. Other is to use multi-column foreign keys. Both are far from optimal.
I am trying to create a table in PostgreSQL to contain lots of data and for that reason I want to use timescales hypertable as in the example below.
CREATE TABLE "datapoints" (
"tstz" timestamptz NOT NULL,
"id" bigserial UNIQUE NOT NULL,
"entity_id" bigint NOT NULL,
"value" real NOT NULL,
PRIMARY KEY ("id", "tstz", "entity_id")
);
SELECT create_hypertable('datapoints','tstz');
However, this throws an error - shown below. As far as I have figured out the error arise since the unique constraint isn't allowed in hypertables, but I really need the uniqueness. So does anyone have an idea on how to solve it or work around it?
ERROR: cannot create a unique index without the column "tstz" (used in partitioning)
SQL state: TS103
There is no way to avoid that.
TimescaleDB uses PostgreSQL partitioning, and it is not possible to have a primary key or unique constraint on a partitioned table that does not contain the partitioning key.
The reason behind that is that an index on a partitioned table consists of individual indexes on the partitions (these are the partitions of the partitioned index). Now the only way to guarantee uniqueness for such a partitioned index is to have the uniqueness implicit in the definition, which is only the case if the partitioning key is part of the index.
So you either have to sacrifice the uniqueness constraint on id (which is pretty much given if you use a sequence) or you have to do without partitioning.
TL;DR
I am seeking clarity on this: does a FOREIGN KEY require a UNIQUE CONSTRAINT on the other side, specifically, in Postgres and, generally, in relational database systems?
Perhaps, I can test this, but I'll ask, if the UNIQUE CONSTRAINT is required by the FOREIGN KEY what would happen if I don't create it? Will the Database create one or will it throw an error?
How I got there
I had earlier on created a table with a column username on which I imposed a unique constraint. I then created another table with a column bearer_name having a FOREIGN KEY referencing the previous table's column username; the one which had a UNIQUE CONSTRAINT.
Now, I want to drop the UNIQUE CONSTRAINT on the username column from the database because I have later on created a UNIQUE INDEX on the same column and intuitively I feel that they serve the same purpose, or don't they? But the database is complaining that the UNIQUE INDEX has some dependent objects and so it can't be dropped unless I provide CASCADE as an option in order to drop even the dependent object. It's identifying the FOREIGN KEY on bearer_name column in the second table as the dependent object.
And is it possible for the FOREIGN KEY to be a point to the UNIQUE INDEX instead of the UNIQUE CONSTRAINT?
I am seeking clarity on this: does a FOREIGN KEY require a UNIQUE CONSTRAINT on the other side
No it does not require only UNIQUE CONSTRAINT. It could be PRIMARY KEY or UNIQUE INDEX.
Perhaps, I can test this, but I'll ask, if the UNIQUE CONSTRAINT is required by the FOREIGN KEY what would happen if I don't create it? Will the Database create one or will it throw an error?
CREATE TABLE tab_a(a_id INT, b_id INT);
CREATE TABLE tab_b(b_id INT);
ALTER TABLE tab_a ADD CONSTRAINT fk_tab_a_tab_b FOREIGN KEY (b_id)
REFERENCES tab_b(b_id);
ERROR: there is no unique constraint matching given keys
for referenced table "tab_b"
DBFiddle Demo
And is it possible for the FOREIGN KEY to be a point to the UNIQUE INDEX instead of the UNIQUE CONSTRAINT?
Yes, it is possible.
CREATE UNIQUE INDEX tab_b_i ON tab_b(b_id);
DBFiddle Demo2
Can I have two tables with different constrains that are names the same - that is:
Table A with constraint C
Table B with constraint C
The constrains are different
Thanks!
I couldn't find any suitable reference in the documentation but it would seem that the uniqueness of the constraint name depends on the type of constraint.
unique and primary key constraint names must be unique and may not be reused in multiple tables, but check and foreign key constraint names can be used in multiple tables (although that might be a bad idea for obvious reasons).
I would guess that the reason is that unique and primary key constraints create indexes and therefore needs to be uniquely named.
I didn't find where is it mentionned in postgres documentation. Here's a confirmation that constraint name but be different in the database.
CREATE TABLE type_position(
type_position INTEGER CONSTRAINT pk_type_position PRIMARY KEY,
description VARCHAR( 64 )
);
CREATE TABLE type_position2(
type_position INTEGER CONSTRAINT pk_type_position PRIMARY KEY,
description VARCHAR( 64 )
);
And I got the message
ERROR: relation "pk_type_position" already exists