For a bulk insert, I have a foreign reference value. As part of the insert, I'd like to return both the reference and corresponding id's of the newly created records in order to create a mapping record in another system.
Using "RETURNING" works fine for the target table. Other than creating a dummy column in the target table, is there anyway to achieve what I'm trying to do?
Definitely do not want to do row-by-row processing.
NOTE: Currently using version 10.7
In my sample code, I tried "returning id, source.ref", but obviously this isn't supported.
create table test( id serial primary key, name varchar);
insert into test( name)
select source.name
from ( values('refa', 'name a'), ('refb', 'name b'), ('refc', 'name c') ) source(ref, name)
returning id --, source.ref
Use CTEs:
WITH a AS (
INSERT ...
RETURNING ...
), b AS (
INSERT ...
RETURNING ...
)
SELECT ...
FROM a JOIN b ON ...
Reference back to the source, if it is unique. Try something like this:
WITH q AS ( INSERT INTO test (name)
SELECT source.name
FROM ( VALUES ('refa', 'name a'), ('refb', 'name b'), ('refc', 'name c')
) AS source (ref, name)
RETURNING * )
SELECT q.id, source.ref
FROM q
JOIN ( VALUES ('refa', 'name a'), ('refb', 'name b'), ('refc', 'name c')
) AS source (ref, name) ON q.name = source.name
But if you want to add this mapping to another table, you might consider to re-structure your queries, to something like this:
INSERT INTO mapping ( ref, id )
SELECT source.ref, ( INSERT INTO test (name) VALUES ( source.name ) RETURNING id )
FROM ( VALUES ('refa', 'name a'), ('refb', 'name b'), ('refc', 'name c')
) AS source (ref, name) )
Related
I have two tables, one containing all the hot columns and one the static ones. The static table has an unique constraint. When the conflict on the unique constraint triggers only the hot columns in the other table should be updated using the the id from the static table.
For better clarity some code:
CREATE TABLE tag (
id bigserial PRIMARY KEY
, key text
, value text
-- UNIQUE (key, value) -- ?
);
CREATE TABLE tag_hotcolumns (
id bigserial PRIMARY KEY
, hot text
, stuff text
);
with s as (
select id, "key", "value"
from tag
where key = 'key1' and value = 'value1'
), i as (
insert into tag ("key", "value")
select 'key1', 'value1'
where not exists (select 1 from s)
returning id
)
select id
from i
union all
select id
from s
The second block works fine, but I can't get the returned id into the insert statement for the tag_hotcolumns...
I tried:
insert into tag_attributes (with s as (
select id, "key", "value"
from tag
where key = 'key1' and value = 'value1'
), i as (
insert into tag ("key", "value")
select 'key1', 'value1'
where not exists (select 1 from s)
returning id
)
select id, 'hot1', 'stuff1'
from i
union all
select id
from s);
And that gives me "WITH clause containing a data-modifying statement must be at the top level
LINE 5: ), i as ("
Any help would be greatly apreciated :)
dwhitemv from stackexchange helped me solve this. The solution you can find here:
https://dbfiddle.uk/?rdbms=postgres_13&fiddle=f72cae495e6eed579d904a5c7b48f05b
I am trying to insert (clone) some records in a table and need to get source ids and ids that got generated. This simplified example demonstrates my issue. After new records are created, referencing their ids in a SELECT produces no results even though records do get created and subsequent SELECT on the table shows them. It feels like insert and select are happening in different transaction scopes.
CREATE TABLE tbl_value(
id int4 NOT NULL GENERATED ALWAYS AS identity PRIMARY KEY,
some_id INTEGER NOT NULL,
value VARCHAR NOT NULL
);
INSERT INTO tbl_value(some_id, value) VALUES(1000, 'value 1'), (1000, 'value 2'), (1000, 'value 3');
with
outer_input as
(
select id, some_id, value from tbl_value where id in (1,2)
),
inner_insert as
(
INSERT INTO tbl_value(some_id, value)
select 2000, value from outer_input
returning id
)
select * from tbl_value v inner join inner_insert i on v.id = i.id;
I have three tables: clients, items and client_has_item.
CREATE TABLE clients (id_c serial primary key, name text);
CREATE TABLE items (id_i serial primary key, name text);
CREATE TABLE client_has_item
(id_c int references clients (id_c),
id_i int references items (id_i));
INSERT INTO clients (name) values ('client a'), ('client b');
INSERT INTO items (name) values ('item a'), ('item a');
Note that a client can have several items, an item can be owned by several clients and names are not unique.
If I want to insert some items for one client (1, 'client a') it is easy:
WITH inserted_items as (
INSERT INTO items (name) values ('item a'), ('item b')
RETURNING id_i
)
INSERT INTO client_has_item (id_c, id_i) select 1, id_i from inserted_items
My question is how to do this for several clients. Ideally I would do:
WITH inserted_items as (
INSERT INTO items (name) SELECT new_items.item_name from (values (1, 'item a'), (2, 'item b')) new_items (id_client, item_name)
RETURNING items.id_i, new_items.id_client
)
INSERT INTO client_has_item (id_c, id_i) select id_client, id_i from inserted_items
But I get the following error:
RETURNING items.id_i, new_items.id_client
********** Error **********
ERROR: missing FROM-clause entry for table "new_items"
What would be the best alternative? Of course I could do a loop, but I'd like to avoid that.
I'm trying to construct an object for use from my postgres backend. The tables in question look something like this:
We have some Things that essentially act as rows for a matrix where the columns are Field_Columns. Field_Values are filled cells.
Create Table Platform_User (
serial id PRIMARY KEY
)
Create Table Things (
serial id PRIMARY KEY,
INTEGER user_id REFERENCES Platform_User(id)
)
Create Table Field_Columns (
serial id PRIMARY KEY,
TEXT name,
)
Create Table Field_Values (
INTEGER field_column_id REFERENCES Field_Columns(id),
INTEGER thing_id REFERENCES Things(id)
TEXT content,
PRIMARY_KEY(field_column_id, thing_id)
)
This would be simple if I were trying to load just the Field_Values for a single Thing as JSON, which would look like this:
SELECT JSONB_OBJECT(
ARRAY(
SELECT name
FROM Field_Columns
ORDER BY Field_Columns.id
),
ARRAY(
SELECT Field_Values.content
FROM Fields_Columns
LEFT JOIN Field_Values ON Field_Values.field_column_id = Field_Columns.id
AND Field_Values.thing_id = Things.id
ORDER BY Field_Columns.id)
)
)
FROM Things
WHERE Thing.id = $1
however, I'd like to construct the JSON object to look like this when returned. I want to get an object of all the Fields:Field_Values objects for the Things that a user owns
{
14:
{
'first field':'asdf',
'other field':''
}
25:
{
'first field':'qwer',
'other field':'dfgdsfg'
}
43:
{
'first field':'',
'other field':''
}
}
My efforts to construct this query look like this, but I'm running into the problem where the JSONB object function doesn't want to construct an object where the value of the field is an object itself
SELECT (
JSONB_OBJECT(
ARRAY(SELECT Things.id::TEXT
FROM Things
WHERE Things.user_id = $2
ORDER BY Things.id
),
ARRAY(SELECT JSONB_OBJECT(
ARRAY(
SELECT name
FROM Field_Columns
ORDER BY Field_Columns.id),
ARRAY(
SELECT Field_Values.content
FROM Field_Columns
LEFT JOIN Field_Values ON Field_Values.field_column_Id = Field_Columns.id
AND Field_Values.thing_id = Things.id
ORDER BY Field_Columns.id)
)
FROM Things
WHERE Things.user_id = $2
ORDER BY Things.id
)
)
) AS thing_fields
The specific error I get is function jsonb_object(text[], jsonb[]) does not exist. Is there a way to do this that doesn't involve copious text conversions and nonsense like that? Or will I just need to abandon trying to sort my data in the query and do it in my code instead.
Your DDL scripts are syntactically incorrect so I created these for you:
create table platform_users (
id int8 PRIMARY KEY
);
create table things (
id int8 PRIMARY KEY,
user_id int8 REFERENCES platform_users(id)
);
create table field_columns (
id int8 PRIMARY KEY,
name text
);
create table field_values (
field_column_id int8 REFERENCES field_columns(id),
thing_id int8 REFERENCES things(id),
content text,
PRIMARY KEY(field_column_id, thing_id)
);
I also created some scripts to populate the db:
insert into platform_users(id) values (1);
insert into platform_users(id) values (2);
insert into platform_users(id) values (3);
insert into platform_users(id) values (4);
insert into platform_users(id) values (5);
insert into things(id, user_id) values(1, 1);
insert into things(id, user_id) values(2, 1);
insert into things(id, user_id) values(3, 2);
insert into things(id, user_id) values(4, 2);
insert into field_columns(id, name) values(1, 'col1');
insert into field_columns(id, name) values(2, 'col2');
insert into field_values(field_column_id, thing_id, content) values(1, 1, 'thing1 val1');
insert into field_values(field_column_id, thing_id, content) values(2, 1, 'thing1 val2');
insert into field_values(field_column_id, thing_id, content) values(1, 2, 'thing2 val1');
insert into field_values(field_column_id, thing_id, content) values(2, 2, 'thing2 val2');
Please include such scripts next time when you ask for help, and make sure that your scripts are correct. This will reduce the work needed to answer your question.
You can get your jsonb value by aggregating the key value pairs with jsonb_object_agg
select
t.id,
jsonb_object_agg(fc.name, fv.content)
from
things t inner join
field_values fv on fv.thing_id = t.id inner join
field_columns fc on fv.field_column_id = fc.id
group by 1
The results looking like this:
thing_id;jsonb_value
1;"{"col1": "thing1 val1", "col2": "thing1 val2"}"
2;"{"col1": "thing2 val1", "col2": "thing2 val2"}"
I encountered a lot of thread about this, the solutions suggested all tend to go the same way, but it is very inconvenient in my case.
Most of the time something like this is suggested.
DECLARE #Actors TABLE ( [Id] INT , [Name] VARCHAR(20) , [MovieId] INT);
DECLARE #Movie TABLE ( [Id] INT, [Name] VARCHAR(20), [FranchiseId] INT );
INSERT INTO #Actors
( Id, Name, MovieId )
VALUES ( 1, 'Sean Connery', 1 ),
( 2, 'Gert Fröbe', 1 ),
( 3, 'Honor Blackman', 1 ),
( 4, 'Daniel Craig', 2 ),
( 5, 'Judi Dench', 2 ),
( 2, 'Harrison Ford', 3 )
INSERT INTO #Movie
( Id, Name, FranchiseId )
VALUES ( 1, 'Goldfinger', 1 ),
( 2, 'Skyfall', 1 ),
( 3, 'Return of the Jedi', 2 )
SELECT m.Name ,
STUFF(( SELECT ',' + a_c.Name
FROM #Actors a_c
WHERE a_c.MovieId = m.Id
FOR
XML PATH('')
), 1, 1, '')
FROM #Actors a
JOIN #Movie m ON a.MovieId = m.Id
GROUP BY m.Id ,
m.Name
The Problem is (how shall I explain?), one does not really access the grouped Items (as Count(), Max(), Min(), ...), one does rebuild the joining pattern of the "outer query" and force in the WHERE statement, that the corresponding values are the same as those in the GROUP BY statement (in the outer query).
If you do not understand what I'm trying to say, I extended the Example above, by one additional table and you will see, that I will also have to extend the "Inner Query"
DECLARE #Actors TABLE ( [Id] INT , [Name] VARCHAR(20) , [MovieId] INT);
DECLARE #Movie TABLE ( [Id] INT, [Name] VARCHAR(20), [FranchiseId] INT );
DECLARE #Franchise TABLE ( [Id] INT , [Name] VARCHAR(20));
INSERT INTO #Actors
( Id, Name, MovieId )
VALUES ( 1, 'Sean Connery', 1 ),
( 2, 'Gert Fröbe', 1 ),
( 3, 'Honor Blackman', 1 ),
( 4, 'Daniel Craig', 2 ),
( 5, 'Judi Dench', 2 ),
( 2, 'Harrison Ford', 3 )
INSERT INTO #Movie
( Id, Name, FranchiseId )
VALUES ( 1, 'Goldfinger', 1 ),
( 2, 'Skyfall', 1 ),
( 3, 'Return of the Jedi', 2 )
INSERT INTO #Franchise
( Id, Name )
VALUES ( 1, 'James Bond' ),
( 2, 'Star Wars' )
SELECT f.Name ,
STUFF(( SELECT ',' + a_c.Name
FROM #Actors a_c
JOIN #Movie m_c ON a_c.MovieId = m_c.Id
WHERE m_c.FranchiseId = f.Id
FOR
XML PATH('')
), 1, 1, '')
FROM #Actors a
JOIN #Movie m ON a.MovieId = m.Id
JOIN #Franchise f ON m.FranchiseId = m.Id
GROUP BY f.Id ,
f.Name
And now, going somewhat further, imagine a huge query, very complicated, several grouping values over many tables. Performance is an issue. I don't want to rebuild the whole joining pattern in the "inner query".
So is there any other way? A way that does not kill performance and you do not have to duplicate the joining pattern?
Contrary to what I said in this comment, you need no GROUP BY clause, nor a WHERE clause, at all!
You simply need the outer SELECT to "iterate" over all franchises (or whatever you want to group by). Then in the inner SELECT, you need some JOINs to get to the franchise key column. Instead of a WHERE clause to filter by the outer franchise's key, simply use the outer franchise key directly in the INNER JOIN:
SELECT f.Name AS FranchiseName,
COALESCE(STUFF((SELECT DISTINCT ', ' + a.Name
FROM #Actor a
JOIN #Movie m ON a.MovieId = m.Id
WHERE m.FranchiseId = f.Id
ORDER BY ', ' + a.Name -- this is optional
FOR XML PATH('')), 1, 1, ''), '') AS ActorNames
FROM #Franchise f
Source of information: "High Performance T-SQL Using Window Functions" by Itzik Ben-Gak. Because SQL Server unfortunately does not have an aggregate/window function for concatenating values, the book's author recommends something like the above as the next best solution.
P.S.: I've removed my previous solution that substituted an additional JOIN for a WHERE clause; I am now fairly certain that a WHERE clause is likely to perform better. Nevertheless, I left some evidence of my previous solution (i.e. the striked-through text) because of that reference to a comment I made earlier.