I have a simple table in PostgreSQL that has three columns:
id serial primary key
key varchar
value varchar
I have already seen this question here on SO: Insert, on duplicate update in PostgreSQL? but I'm wondering just how to get the id if it exists, instead of updating. If the standard practice is to always either "insert" or "update if exists", why is that? Is the cost of doing a SELECT (LIMIT 1) greater than doing an UPDATE?
I have the following code
INSERT INTO tag
("key", "value")
SELECT 'key1', 'value1'
WHERE
NOT EXISTS (
SELECT id,"key","value" FROM tag WHERE key = 'key1' AND value = 'value1'
);
which works in the sense that it doesn't insert if exists, but I'd like to get the id. Is there a "RETURNING id" clause or something similar that I could tap in there?
Yes there is returning
INSERT INTO tag ("key", "value")
SELECT 'key1', 'value1'
WHERE NOT EXISTS (
SELECT id, "key", "value"
FROM node_tag
WHERE key = 'key1' AND value = 'value1'
)
returning id, "key", "value"
To return the row if it already exists
with s as (
select id, "key", "value"
from tag
where key = 'key1' and value = 'value1'
), i as (
insert into tag ("key", "value")
select 'key1', 'value1'
where not exists (select 1 from s)
returning id, "key", "value"
)
select id, "key", "value"
from i
union all
select id, "key", "value"
from s
If the row does not exist it will return the inserted one else the existing one.
BTW, if the pair "key"/"value" makes it unique then it is the primary key, and there is no need for an id column. Unless one or both of the "key"/"value" pair can be null.
with vals as (
select 'key5' as key, 'value2' as value
)
insert into Test1 (key, value)
select v.key, v.value
from vals as v
where not exists (select * from Test1 as t where t.key = v.key and t.value = v.value)
returning id
sql fiddle demo
And you can store value returned to variables in form of ... RETURNING field1, field2,... INTO var1, var2,...
RETURNING will normally return a query which would return Error 'query has no destination for result data' if you call it in plpgsql without using its returned result set.
Related
I need to analyze survey data (stored in records) where a question can have a choice of options. My goal is to identify the answers given that were NOT within the range of allowed options for this question. However, my query returns everything (I suspect a subquery) and I don't know how to fix it.
Schema
The records stores its data in the data JSONB column. There, the keys are question UIDs, e.g. uid00000006 has the answer option1. option1 is a choice to select.
(Not all questions need to have a dropdown, so some other value is fine such as 42.)
{"uid00000006": {"value": "option1"}, "uid00000008": {"value": 42}}
A question optionally has a reference to a optionset (the dropdown) which has a range of optionvalues (the values of the dropdown) , e.g. option1, option2, option3 etc.
create table record
(
recordid bigint not null primary key,
uid varchar(11) unique,
data jsonb default '{}'::jsonb not null
);
create table question
(
questionid bigint not null primary key,
uid varchar(11) not null unique,
optionsetid bigint
);
create table optionset
(
optionsetid bigint not null primary key,
uid varchar(11) not null unique
);
create table optionvalue
(
optionvalueid bigint not null primary key,
uid varchar(11) not null unique,
code varchar(230) not null,
optionsetid bigint
);
-- create optionset
INSERT INTO optionset (optionsetid, uid) VALUES (1, 'uid00000001');
-- insert optionvalues into optionset
INSERT INTO optionvalue (optionvalueid, uid, code, optionsetid) VALUES (100, 'uid00000002', 'option1', 1);
INSERT INTO optionvalue (optionvalueid, uid, code, optionsetid) VALUES (101, 'uid00000003', 'option2', 1);
INSERT INTO optionvalue (optionvalueid, uid, code, optionsetid) VALUES (102, 'uid00000004', 'option3', 1);
INSERT INTO optionvalue (optionvalueid, uid, code, optionsetid) VALUES (103, 'uid00000005', 'option4', 1);
-- insert questions
INSERT INTO question (questionid, uid, optionsetid) VALUES (1001, 'uid00000006', 1);
INSERT INTO question (questionid, uid, optionsetid) VALUES (1002, 'uid00000007', 1);
INSERT INTO question (questionid, uid, optionsetid) VALUES (1003, 'uid00000008', NULL);
-- insert records
INSERT INTO record (recordid, uid, data) VALUES (10001, 'uid00000009', '{"uid00000006": {"value": "option1"}, "uid00000008": {"value": 42}}'::jsonb);
INSERT INTO record (recordid, uid, data) VALUES (10002, 'uid00000010', '{"uid00000006": {"value": "option2"}}'::jsonb);
INSERT INTO record (recordid, uid, data) VALUES (10003, 'uid00000011', '{"uid00000006": {"value": "UNMAPPED"}}'::jsonb);
My query
My drafted query is:
SELECT r.uid AS record_uid,
key AS question_uid,
os.uid AS optionset_uid,
value ->> 'value' AS value
FROM record r, JSONB_EACH(r.data)
JOIN question q ON q.uid = key
JOIN optionset os ON q.optionsetid = os.optionsetid
WHERE q.optionsetid IS NOT NULL
AND value::varchar NOT IN (SELECT DISTINCT code FROM optionvalue WHERE optionsetid = q.optionsetid)
;
DBFiddle
Problem
The query above returns all records instead only one. In reference to the sample data, the expected result would be to return only the record where the value is UNMAPPED (meaning it is the record where an answer was given that is not "valid").
You should change value::varchar NOT IN to value ->> 'value' NOT IN
SELECT
r.uid AS record_uid,
key AS question_uid,
os.uid AS optionset_uid,
value ->> 'value' AS value
FROM
record r, jsonb_each(r.data)
JOIN question q ON q.uid = key
JOIN optionset os ON q.optionsetid = os.optionsetid
WHERE
q.optionsetid IS NOT NULL
AND value ->> 'value' NOT IN (SELECT DISTINCT code FROM optionvalue WHERE optionsetid = q.optionsetid);
I have two tables, one containing all the hot columns and one the static ones. The static table has an unique constraint. When the conflict on the unique constraint triggers only the hot columns in the other table should be updated using the the id from the static table.
For better clarity some code:
CREATE TABLE tag (
id bigserial PRIMARY KEY
, key text
, value text
-- UNIQUE (key, value) -- ?
);
CREATE TABLE tag_hotcolumns (
id bigserial PRIMARY KEY
, hot text
, stuff text
);
with s as (
select id, "key", "value"
from tag
where key = 'key1' and value = 'value1'
), i as (
insert into tag ("key", "value")
select 'key1', 'value1'
where not exists (select 1 from s)
returning id
)
select id
from i
union all
select id
from s
The second block works fine, but I can't get the returned id into the insert statement for the tag_hotcolumns...
I tried:
insert into tag_attributes (with s as (
select id, "key", "value"
from tag
where key = 'key1' and value = 'value1'
), i as (
insert into tag ("key", "value")
select 'key1', 'value1'
where not exists (select 1 from s)
returning id
)
select id, 'hot1', 'stuff1'
from i
union all
select id
from s);
And that gives me "WITH clause containing a data-modifying statement must be at the top level
LINE 5: ), i as ("
Any help would be greatly apreciated :)
dwhitemv from stackexchange helped me solve this. The solution you can find here:
https://dbfiddle.uk/?rdbms=postgres_13&fiddle=f72cae495e6eed579d904a5c7b48f05b
Table definition:
CREATE TABLE public."FeatureToggles"
(
"Id" integer NOT NULL GENERATED BY DEFAULT AS IDENTITY ( INCREMENT 1 START 1 MINVALUE 1 MAXVALUE 2147483647 CACHE 1 ),
"IsDeleted" boolean NOT NULL,
"IsImported" boolean NOT NULL,
"TextProp" character varying(35),
CONSTRAINT "PK_FeatureToggles" PRIMARY KEY ("Id")
)
CREATE TABLE public."Additions"
(
"Id" integer NOT NULL GENERATED BY DEFAULT AS IDENTITY ( INCREMENT 1 START 1 MINVALUE 1 MAXVALUE 2147483647 CACHE 1 ),
"FeatureToggleId" int NOT NULL,
"IsDeleted" boolean NOT NULL,
"Url" character varying(35) NULL,
CONSTRAINT "PK_FeatureToggles" PRIMARY KEY ("Id")
CONSTRAINT "FK_Additions_FeatureToggles_FeatureToggleId" FOREIGN KEY ("FeatureToggleId")
REFERENCES public."FeatureToggles" ("Id") MATCH SIMPLE
ON UPDATE NO ACTION
ON DELETE CASCADE,
)
Insert one record into table:
INSERT INTO public."FeatureToggles" ("IsDeleted", "TextProp", "IsImported") VALUES(false, 'X', true);
Function:
CREATE OR REPLACE FUNCTION testfunctionname(jsonparam json)
RETURNS void AS
$BODY$
INSERT INTO "FeatureToggles" ("Id", "IsDeleted", "IsImported", "TextProp")
SELECT (COALESCE(SELECT "Id" FROM "FeatureToggles" WHERE "TextProp" = (prop->>'TextProp')::character varying(35)), 0),
(prop->>'IsDeleted')::boolean,
true,
(prop->>'TextProp')::character varying(35)
json_array_elements(jsonparam) prop
ON CONFLICT ("Id") DO
UPDATE SET
"IsDeleted" = EXCLUDED."IsDeleted"
INSERT INTO "Additions" ("FeatureToggleId", "IsDeleted", "Url")
SELECT (SELECT "Id" FROM "FeatureToggles" WHERE "TextProp" = (prop->>'TextProp')::character varying(35)),
(prop->>'IsDeleted')::boolean,
(prop->>'Additions')::character varying(35)
json_array_elements(jsonparam) prop
DELETE FROM "FeatureToggles" WHERE "IsImported" = true AND "TextProp" IS NOT IN (SELECT DISTINCT (prop->>'TextProp')::character varying(35)szi
json_array_elements(jsonparam) prop)
$BODY$
LANGUAGE sql
Sample JSON:
[
{
"IsDeleted": true,
"TextProp": "X",
"Additions":
[
"Test1",
"Test2"
]
},
{
"IsDeleted": false,
"TextProp": "Y",
"Additions":
[
"Test3",
"Test4"
]
}
]
Calling the function with this JSON param should update the one and only row in the FeatureToggles table to IsDeleted true and insert a new row into the FeatureToggles table with Id equals to 2, IsDeleted false and TextProp is Y. Also it should insert all Additions given in the JSON param into the corresponding table and with the correct foreign keys.
I ran into problems with populating the Id properties from the existing table and also inserting Additions into the other table.
It would be a great if the function would delete any rows in the FeatureToggle and the corresponding Additions table too if it does exists in table already, IsImported property is true, but is not in the JSON param.
Example if we change the insert script to:
INSERT INTO public."FeatureToggles" ("IsDeleted", "TextProp", "IsImported") VALUES(false, 'X', true);
INSERT INTO public."FeatureToggles" ("IsDeleted", "TextProp", "IsImported") VALUES(false, 'X222', true);
After calling the function with the same JSON param, the row with X222 should be deleted because it is marked as imported, but has no matching item (matched by TextProp property) within the new param list.
Any help would be much appreciated as this function needs to handle tens of thousands of records as parameter on each call.
You have several errors in your function (and your DDL)
Most importantly, json_array_elements() is a set returning function, so you need a FROM clause in order to generate multiple rows.
You also need to terminate each SQL statement in the function with ; and IS NOT IN is invalid - you need NOT IN
So the function should be something like this:
CREATE OR REPLACE FUNCTION testfunctionname(jsonparam json)
RETURNS void AS
$BODY$
INSERT INTO "FeatureToggles" ("Id", "IsDeleted", "IsImported", "TextProp")
SELECT coalesce(ft."Id", 0),
(prop->>'IsDeleted')::boolean,
true,
prop->>'TextProp'
FROM json_array_elements(jsonparam) prop
LEFT JOIN "FeatureToggles" ft on ft."TextProp" = (prop->>'TextProp')
ON CONFLICT ("Id") DO
UPDATE SET
"IsDeleted" = EXCLUDED."IsDeleted";
INSERT INTO "Additions" ("FeatureToggleId", "IsDeleted", "Url")
SELECT coalesce(ft."Id", 0),
(prop->>'IsDeleted')::boolean,
prop->>'Additions'
FROM json_array_elements(jsonparam) prop
JOIN "FeatureToggles" ft on ft."TextProp" = (prop->>'TextProp');
DELETE FROM "FeatureToggles"
WHERE "IsImported" = true
AND "TextProp" NOT IN (SELECT DISTINCT prop->>'TextProp' szi
FROM json_array_elements(jsonparam) prop);
$BODY$
LANGUAGE sql;
Note that ->> returns a text value, so there is no need to cast the result of those expression if the target column is text or varchar.
I also changed the scalar sub-queries to JOINs. The first insert is equivalent to an outer join - although I think that is wrong (but that's what your current code tries to do). Because if the join doesn't return anything, the INSERT will try to create a row with "Id" = 0 - bypassing the sequence generation. Using on conflict() with an auto-generated ID rarely makes sense. Maybe you want a unique index on TextProp?
I would probably implement that as a procedure rather than a function though.
Online example
I am trying to insert (clone) some records in a table and need to get source ids and ids that got generated. This simplified example demonstrates my issue. After new records are created, referencing their ids in a SELECT produces no results even though records do get created and subsequent SELECT on the table shows them. It feels like insert and select are happening in different transaction scopes.
CREATE TABLE tbl_value(
id int4 NOT NULL GENERATED ALWAYS AS identity PRIMARY KEY,
some_id INTEGER NOT NULL,
value VARCHAR NOT NULL
);
INSERT INTO tbl_value(some_id, value) VALUES(1000, 'value 1'), (1000, 'value 2'), (1000, 'value 3');
with
outer_input as
(
select id, some_id, value from tbl_value where id in (1,2)
),
inner_insert as
(
INSERT INTO tbl_value(some_id, value)
select 2000, value from outer_input
returning id
)
select * from tbl_value v inner join inner_insert i on v.id = i.id;
I have a simple table in PostgreSQL that has three columns:
id serial primary key
key varchar
value varchar
I have already seen this question here on SO: Insert, on duplicate update in PostgreSQL? but I'm wondering just how to get the id if it exists, instead of updating. If the standard practice is to always either "insert" or "update if exists", why is that? Is the cost of doing a SELECT (LIMIT 1) greater than doing an UPDATE?
I have the following code
INSERT INTO tag
("key", "value")
SELECT 'key1', 'value1'
WHERE
NOT EXISTS (
SELECT id,"key","value" FROM tag WHERE key = 'key1' AND value = 'value1'
);
which works in the sense that it doesn't insert if exists, but I'd like to get the id. Is there a "RETURNING id" clause or something similar that I could tap in there?
Yes there is returning
INSERT INTO tag ("key", "value")
SELECT 'key1', 'value1'
WHERE NOT EXISTS (
SELECT id, "key", "value"
FROM node_tag
WHERE key = 'key1' AND value = 'value1'
)
returning id, "key", "value"
To return the row if it already exists
with s as (
select id, "key", "value"
from tag
where key = 'key1' and value = 'value1'
), i as (
insert into tag ("key", "value")
select 'key1', 'value1'
where not exists (select 1 from s)
returning id, "key", "value"
)
select id, "key", "value"
from i
union all
select id, "key", "value"
from s
If the row does not exist it will return the inserted one else the existing one.
BTW, if the pair "key"/"value" makes it unique then it is the primary key, and there is no need for an id column. Unless one or both of the "key"/"value" pair can be null.
with vals as (
select 'key5' as key, 'value2' as value
)
insert into Test1 (key, value)
select v.key, v.value
from vals as v
where not exists (select * from Test1 as t where t.key = v.key and t.value = v.value)
returning id
sql fiddle demo
And you can store value returned to variables in form of ... RETURNING field1, field2,... INTO var1, var2,...
RETURNING will normally return a query which would return Error 'query has no destination for result data' if you call it in plpgsql without using its returned result set.