I'm trying to do an upsert to a table that has partial unique indexes
create table test (
p text not null,
q text,
r text,
txt text,
unique(p,q,r)
);
create unique index test_p_idx on test(p) where q is null and r is null;
create unique index test_pq_idx on test(p, q) where r IS NULL;
create unique index test_pr_idx on test(p, r) where q is NULL;
In plain terms, p is not null and only one of q or r can be null.
Duplicate inserts throw constraint violations as expected
insert into test(p,q,r,txt) values ('p',null,null,'a'); -- violates test_p_idx
insert into test(p,q,r,txt) values ('p','q',null,'b'); -- violates test_pq_idx
insert into test(p,q,r,txt) values ('p',null, 'r','c'); -- violates test_pr_idx
However, when I'm trying to use the unique constraint for an upsert
insert into test as u (p,q,r,txt) values ('p',null,'r','d')
on conflict (p, q, r) do update
set txt = excluded.txt
it still throws the constraint violation
ERROR: duplicate key value violates unique constraint "test_pr_idx"
DETAIL: Key (p, r)=(p, r) already exists.
But I'd expect the on conflict clause to catch it and do the update.
What am I doing wrong? Should I be using an index_predicate?
index_predicate
Used to allow inference of partial unique indexes. Any
indexes that satisfy the predicate (which need not actually be partial
indexes) can be inferred. Follows CREATE INDEX format.
https://www.postgresql.org/docs/9.5/static/sql-insert.html
I don't think it's possible to use multiple partial indexes as a conflict target. You should try to achieve the desired behaviour using a single index. The only way I can see is to use a unique index on expressions:
drop table if exists test;
create table test (
p text not null,
q text,
r text,
txt text
);
create unique index test_unique_idx on test (p, coalesce(q, ''), coalesce(r, ''));
Now all three tests (executed twice) violate the same index:
insert into test(p,q,r,txt) values ('p',null,null,'a'); -- violates test_unique_idx
insert into test(p,q,r,txt) values ('p','q',null,'b'); -- violates test_unique_idx
insert into test(p,q,r,txt) values ('p',null, 'r','c'); -- violates test_unique_idx
In the insert command you should pass the expressions used in the index definition:
insert into test as u (p,q,r,txt)
values ('p',null,'r','d')
on conflict (p, coalesce(q, ''), coalesce(r, '')) do update
set txt = excluded.txt;
Related
Given:
Table A with multiple rows and attributes: (A_attr1 (key) , A_attr2).
Table B with attributes (B_attr1 (key) , A_attr1 (foreign key), B_attr2).
How do I insert some values in the table B only if the foreign key exists?
In Postgres, we can use the Where Exists to implement your use case.
Here is an example of using it.
Insert Into Table_B Select 'Value 1', 'Foreign Value', 'Value 2' Where Exists
(Select 1 From Table_A Where A_attr1 = 'Foreign Value');
This will insert only if the "Foreign Value" is present in Table_A.
Hope this helps
First, we need to consider the fact that the condition (existence of foreign key in table A) is fundamental, in fact, if we try to add values in Table_B with an A_attr1 that it doesn't exist in Table_A we get an error of this type:
ERROR: the INSERT or the UPDATE on the TABLE table_B violates the foreign key constraint
"_A_attr1_"
DETAIL: the key (A_attr1)=(wrong_value) it's not present in the table "Table_A"
This is a possible solution:
INSERT INTO Table_B(B_attr1, A_attr1, B_attr2)
SELECT x.*
FROM (VALUES
(something,something_else, something_else2),
(something_else3,something_else4, something_else5),
...
(something_else20, something_else21,something_else22)
) x(B_attr1, A_attr1, B_attr2)
WHERE EXISTS(
SELECT FROM TABLE_A y
WHERE (y.A_attr1)=(x.A_attr1)
FOR SHARE);
The result is the addition in B of all the tuples that are acceptable (that is the one with the existing foreign keys).
This post is an extension of the following question:
PostgreSQL insert if foreign key exists
The solution is based on the comments on this post:
https://dba.stackexchange.com/questions/252875/how-to-make-on-conflict-work-for-compound-foreign-key-columns/252925#252925
I have a database table with a code column that uses a lowercase index to prevent code values that only differ in case (e.g. 'XYZ' = 'xYZ' = 'xyz'). The typical way in Postgresql is to create a function based index, like this: CREATE UNIQUE INDEX mytable_lower_code_idx ON mytable (lower(code)).
Now I have a case where I need upsert behaviour on that column:
-- first insert
INSERT INTO mytable (code) VALUES ('abcd');
-- second insert, with upsert behaviour
INSERT INTO mytable (code) VALUES ('Abcd')
ON CONFLICT (code) DO UPDATE
SET code='Abcd';
For the second insert I get a unique key violation: ERROR: duplicate key value violates unique constraint "mytable_lower_code_idx"
(I also tried to use ON CONFLICT ON CONSTRAINT mytable_lower_code_idx but Postgresql tells me that this constraint does not exist so maybe it doesn't treat the index as a constraint.)
My final question: Is there any way to make INSERT ... ON CONFLICT work together with indexes on expressions? Or must I introduce a physical indexed lowercase column to accomplish the task?
Use ON CONFLICT (lower(code)) DO UPDATE:
CREATE TABLE mytable (
code text
);
CREATE UNIQUE INDEX mytable_lower_code_idx ON mytable (lower(code));
INSERT INTO mytable VALUES ('abcd');
INSERT INTO mytable (code) VALUES ('Abcd')
ON CONFLICT (lower(code)) DO UPDATE
SET code='Abcd';
SELECT * FROM mytable;
yields
| code |
|------|
| Abcd |
Note that ON CONFLICT syntax
allows for the conflict target to be an index_expression (my emphasis):
ON CONFLICT conflict_target
where conflict_target can be one of:
( { index_column_name | ( index_expression ) } [ COLLATE collation ] [ opclass ] [, ...] ) [ WHERE index_predicate ]
ON CONSTRAINT constraint_name
and index_expression:
Similar to index_column_name, but used to infer expressions on
table_name columns appearing within index definitions (not simple
columns). Follows CREATE INDEX format.
Try to add your index as follow:
CREATE UNIQUE INDEX CONCURRENTLY mytable_lower_code_idx
ON mytable (lower(code));
ALTER TABLE mytable
ADD CONSTRAINT unique_mytab_code
UNIQUE USING INDEX mytable_lower_code_idx ;
and then:
INSERT INTO mytable (code) VALUES ('abcd');
-- second insert, with upsert behaviour
INSERT INTO mytable (code) VALUES ('Abcd')
ON CONFLICT ON CONSTRAINT unique_mytab_code DO UPDATE
SET code='Abcd';
I have a PostgeresDB with the following constraint:
CONSTRAINT "Car_Data_3PM_pkey" PRIMARY KEY ("F_ID", "Date"),
CONSTRAINT "Car_Data_3PM_F_ID_fkey" FOREIGN KEY ("F_ID")
REFERENCES "Bike_Data" ("F_ID") MATCH SIMPLE
ON UPDATE NO ACTION ON DELETE NO ACTION
When I try to insert multiple values using:
INSERT INTO "Car_Data_3PM" ("F_ID","Date","Price_Type","O","H","L","LT","EQ","V","NAD") VALUES (38,'2016-10-02 08:19:40.056679','x',0,0,0,112.145,0,0,112.145),(14,'2016-10-02 08:19:40.056679','x',0,0,0,5476,0,0,5476),(13,'2016-10-02
I get this error:
ERROR: insert or update on table "Car_Data_3PM" violates foreign key
constraint "Car_Data_3PM_F_ID_fkey" SQL state: 23503 Detail: Key
(F_ID)=(38) is not present in table "Bike_Data".
NO ROWS are inserted.
How can I make Postgres ONLY miss out the rows where the constraint is an issue? i.e Insert most of them?
You can't make Postgres ignore the values, but you can rewrite your statement to not insert those rows:
INSERT INTO "Car_Data_3PM" ("F_ID","Date","Price_Type","O","H","L","LT","EQ","V","NAD")
select *
from (
VALUES
(38,'2016-10-02 08:19:40.056679','x',0,0,0,112.145,0,0,112.145),
(14,'2016-10-02 08:19:40.056679','x',0,0,0,5476,0,0,5476),
... -- all other rows
) as x (id, date, price_type, o, h, l, lt, eq, v nad)
where exists (select 1
from "Bike_Data" bd
where bd."F_ID" = x .id)
One way is to write a trigger that filters out the bad values, like this:
CREATE FUNCTION car_insert_filter() RETURNS trigger
LANGUAGE plpgsql AS
$$BEGIN
IF EXISTS(SELECT 1 FROM "Bike_Data" WHERE "F_ID" = NEW."F_ID")
THEN
RETURN NEW;
ELSE
RAISE NOTICE 'Skipping row with "F_ID"=% and "Date"=%',
NEW."F_ID", NEW."Date";
RETURN NULL;
END IF;
END;$$;
CREATE TRIGGER car_insert_filter
BEFORE INSERT ON "Car_Data_3PM" FOR EACH ROW
EXECUTE PROCEDURE car_insert_filter();
I need to enforce uniqueness on a column but only when other column is true. For example:
create temporary table test(id serial primary key, property character varying(50), value boolean);
insert into test(property, value) values ('a', false);
insert into test(property, value) values ('a', true);
insert into test(property, value) values ('a', false);
And I enforce the uniqueness with a conditional index:
create unique index on test(property) where value = true;
So far so good, the problem arises when I try to change the row that has the value set to true. It works if I do:
update test set value = new_value from (select id, id=3 as new_value from test where property = 'a')new_test where test.id = new_test.id
But it doesn't when I do:
update test set value = new_value from (select id, id=1 as new_value from test where property = 'a')new_test where test.id = new_test.id
And I get:
ERROR: duplicate key value violates unique constraint "test_property_idx"
DETAIL: Key (property)=(a) already exists.
********** Error **********
ERROR: duplicate key value violates unique constraint "test_property_idx"
SQL state: 23505
Detail: Key (property)=(a) already exists.
Basically it works if the row with value true has a primary key with a bigger value than the current row which is truthy. Any idea on how to circumvent it?
Of course I could do:
update test set value = false where property='a';
update test set value = true where property = 'a' and id = 1;
However, I'm running these queries from node and it is preferable to run only one query.
I'm using Postgres 9.5
Your problem is that UPDATE statements cannot have an ORDER BY clause in SQL (it can have in some RDBMS, but not in PostgreSQL).
The usual solution to this is to make the constraint deferrable. But you use a partial unique index & indexes cannot be declared as deferrable.
Use an exclusion constraint instead: they are the generalization of unique constraints & can be partial too.
ALTER TABLE test
ADD EXCLUDE (property WITH =) WHERE (value = true)
DEFERRABLE INITIALLY DEFERRED;
I have the following scenario in a Postgres 9.3 database:
Tables B and C reference Table A.
Table C has an optional field that references table B.
I would like to ensure that for each row of table C that references table B, c.b.a = c.a. That is, if C has a reference to B, both rows should point at the same row in table A.
I could refactor table C so that if c.b is specified, c.a is null but that would make queries joining tables A and C awkward.
I might also be able to make table B's primary key include its reference to table A and then make table C's foreign key to table B include table C's reference to table A but I think this adjustment would be too awkward to justify the benefit.
I think this can be done with a trigger that runs before insert/update on table C and rejects operations that violate the specified constraint.
Is there a better way to enforce data integrity in this situation?
There is a very simple, bullet-proof solution. Works for Postgres 9.3 - when the original question was asked. Works for the current Postgres 13 - when the question in the bounty was added:
Would like information on if this is possible to achieve without database triggers
FOREIGN KEY constraints can span multiple columns. Just include the ID of table A in the FK constraint from table C to table B. This enforces that linked rows in B and C always point to the same row in A. Like:
CREATE TABLE a (
a_id int PRIMARY KEY
);
CREATE TABLE b (
b_id int PRIMARY KEY
, a_id int NOT NULL REFERENCES a
, UNIQUE (a_id, b_id) -- redundant, but required for FK
);
CREATE TABLE c (
c_id int PRIMARY KEY
, a_id int NOT NULL REFERENCES a
, b_id int
, CONSTRAINT fk_simple_and_safe_solution
FOREIGN KEY (a_id, b_id) REFERENCES b(a_id, b_id) -- THIS !
);
Minimal sample data:
INSERT INTO a(a_id) VALUES
(1)
, (2);
INSERT INTO b(b_id, a_id) VALUES
(1, 1)
, (2, 2);
INSERT INTO c(c_id, a_id, b_id) VALUES
(1, 1, NULL) -- allowed
, (2, 2, 2); -- allowed
Disallowed as requested:
INSERT INTO c(c_id, a_id, b_id) VALUES (3,2,1);
ERROR: insert or update on table "c" violates foreign key constraint "fk_simple_and_safe_solution"
DETAIL: Key (a_id, b_id)=(2, 1) is not present in table "b".
db<>fiddle here
The default MATCH SIMPLE behavior of FK constraints works like this (quoting the manual):
MATCH SIMPLE allows any of the foreign key columns to be null; if any of them are null, the row is not required to have a match in the referenced table.
So NULL values in c(b_id) are still allowed (as requested: "optional field"). The FK constraint is "disabled" for this special case.
We need the logically redundant UNIQUE constraint on b(a_id, b_id) to allow the FK reference to it. But by making it out to be on (a_id, b_id) instead of (b_id, a_id), it is also useful in its own right, providing a useful index on b(a_id) to support the other FK constraint, among other things. See:
Is a composite index also good for queries on the first field?
(An additional index on c(a_id) is typically useful accordingly.)
Further reading:
Differences between MATCH FULL, MATCH SIMPLE, and MATCH PARTIAL?
Enforcing constraints “two tables away”
I ended up creating a trigger as follows:
create function "check C.A = C.B.A"()
returns trigger
as $$
begin
if NEW.b is not null then
if NEW.a != (select a from B where id = NEW.b) then
raise exception 'a != b.a';
end if;
end if;
return NEW;
end;
$$
language plpgsql;
create trigger "ensure C.A = C.B.A"
before insert or update on C
for each row
execute procedure "check C.A = C.B.A"();
Would like information on if this is possible to achieve without database triggers
Yes, it is possible. The mechanism is called ASSERTION and it is defined in SQL-92 Standard(though it is not implemented by any major RDBMS).
In short it allows to create multiple-row constraints or multi-table check constraints.
As for PostgreSQL it could be emulated by using view with WITH CHECK OPTION and performing operation on view instead of base table.
WITH CHECK OPTION
This option controls the behavior of automatically updatable views. When this option is specified, INSERT and UPDATE commands on the view will be checked to ensure that new rows satisfy the view-defining condition (that is, the new rows are checked to ensure that they are visible through the view). If they are not, the update will be rejected.
Example:
CREATE TABLE a(id INT PRIMARY KEY, cola VARCHAR(10));
CREATE TABLE b(id INT PRIMARY KEY, colb VARCHAR(10), a_id INT REFERENCES a(id) NOT NULL);
CREATE TABLE c(id INT PRIMARY KEY, colc VARCHAR(10),
a_id INT REFERENCES a(id) NOT NULL,
b_id INT REFERENCES b(id));
Sample inserts:
INSERT INTO a(id, cola) VALUES (1, 'A');
INSERT INTO a(id, cola) VALUES (2, 'A2');
INSERT INTO b(id, colb, a_id) VALUES (12, 'B', 1);
INSERT INTO c(id, colc, a_id) VALUES (15, 'C', 2);
Violating the condition(connecting C with B different a_id on both tables)
UPDATE c SET b_id = 12 WHERE id = 15;;
-- no issues whatsover
Creating view:
CREATE VIEW view_c
AS
SELECT *
FROM c
WHERE NOT EXISTS(SELECT 1
FROM b
WHERE c.b_id = b.id
AND c.a_id != b.a_id) -- here is the clue, we want a_id to be the same
WITH CHECK OPTION ;
Trying update second time(error):
UPDATE view_c SET b_id = 12 WHERE id = 15;
--ERROR: new row violates check option for view "view_c"
--DETAIL: Failing row contains (15, C, 2, 12).
Trying brand new inserts with incorrect data(also errors)
INSERT INTO b(id, colb, a_id) VALUES (20, 'B2', 2);
INSERT INTO view_c(id, colc, a_id, b_id) VALUES (30, 'C2', 1, 20);
--ERROR: new row violates check option for view "view_c"
--DETAIL: Failing row contains (30, C2, 1, 20)
db<>fiddle demo