Postgres Dynamic Query - postgresql

I have scenario were I have a master table which stores db table name and column name, I need to build dynamic query based on that.
CREATE TABLE MasterTable
(
Id int primary key,
caption varchar(100),
dbcolumnname varchar(100),
dbtablename varchar(100)
);
CREATE TABLE Engineers
(
Id int primary key,
Name varchar(100),
Salary BigInt
);
CREATE TABLE Executives
(
Id int primary key,
Name varchar(100),
Salary BigInt
);
CREATE TABLE Manager
(
Id int primary key,
Name varchar(100),
Salary BigInt
);
INSERT INTO Manager(Id, Name, Salary)
VALUES(1, 'Manager 1', 6000000);
INSERT INTO Executives(Id, Name, Salary)
VALUES(1, 'Executive 1', 6000000);
INSERT INTO Engineers(Id, Name, Salary)
VALUES(1, 'Engineer 1', 6000000);
INSERT INTO MasterTable(Id, caption, dbcolumnname, dbtablename)
VALUES (1, 'Name', 'name', 'Engineers');
INSERT INTO MasterTable(Id, caption, dbcolumnname, dbtablename)
VALUES (2, 'Name', 'name', 'Manager');
INSERT INTO MasterTable(Id, caption, dbcolumnname, dbtablename)
VALUES (3, 'Name', 'name', 'Executives');
INSERT INTO MasterTable(Id, caption, dbcolumnname, dbtablename)
VALUES (4, 'Salary', 'Salary', 'Engineers');
INSERT INTO MasterTable(Id, caption, dbcolumnname, dbtablename)
VALUES (5, 'Salary', 'Salary', 'Manager');
INSERT INTO MasterTable(Id, caption, dbcolumnname, dbtablename)
VALUES (6, 'Salary', 'Salary', 'Executives');
I want to build a stored procedure which accepts caption and Id and give result back based on dbcolumnname and dbtablename. For example if I pass Salary,Name as caption and Id as 1, stored procedure should be query of dbcolumn and dbtable, something like below.
Select Id as ID, name as Value from Engineers
UNION
Select Id as ID, name as Value from Manager
UNION
Select Id as ID, name as Value from Executives
UNION
Select Id as ID, Salary as Value from Executives
UNION
Select Id as ID, Salary as Value from Engineers
UNION
Select Id as ID, Salary as Value from Manager
I have heard of dynamic sql, can that be used here?
Fiddle
EDIT :: I got one dynamic query which builds union statement to get the output, however problem is i am not able to escape double quotes. Below is the query and Error
Query :
DO
$BODY$
BEGIN
EXECUTE string_agg(
format('SELECT %I FROM %I', dbcolumnname, dbtablename),
' UNION ')
FROM MasterTable;
END;
$BODY$;
Error:
ERROR: relation "Engineers" does not exist
LINE 1: SELECT name FROM "Engineers" UNION SELECT name FROM "Manager...

I'd like to suggest an alternative way of achieving what you want. That is, using PostgreSQL inheritance mechanism.
For instance:
CREATE TABLE ParentTable (
Id int,
Name varchar(100),
Salary BigInt
);
ALTER TABLE Engineers INHERIT ParentTable;
ALTER TABLE Executives INHERIT ParentTable;
ALTER TABLE Manager INHERIT ParentTable;
SELECT Id, Salary AS value FROM ParentTable
UNION
SELECT Id, Name AS value FROM ParentTable;
Now if you want to use MasterTable in order to restrict the set of tables used, you can do it as follows:
SELECT Id, Name AS value
FROM ParentTable
INNER JOIN pg_class ON parenttable.tableoid = pg_class.oid
INNER JOIN MasterTable ON LOWER(dbtablename) = LOWER(relname)
UNION
SELECT Id, Salary AS value
FROM ParentTable
INNER JOIN pg_class ON parenttable.tableoid = pg_class.oid
INNER JOIN MasterTable ON LOWER(dbtablename) = LOWER(relname)
However, you can not arbitrarily restrict the set of columns to retrieve from one table to another with this technique.

Table names and column names are case insensitive in SQL, unless they are quoted in double quotes. Postgres does this by folding unquoted identifiers to lower case.
So, your DDL:
CREATE TABLE MasterTable
(
Id int primary key,
caption varchar(100),
dbcolumnname varchar(100),
dbtablename varchar(100)
);
Will be interpreted by Postgres as
CREATE TABLE mastertable
(
id int primary key,
caption varchar(100),
dbcolumnname varchar(100),
dbtablename varchar(100)
);
You can avoid case folding by quoting the names:
CREATE TABLE "MasterTable"
(
"Id" int primary key,
caption varchar(100),
dbcolumnname varchar(100),
dbtablename varchar(100)
);
The %I format-specifier (internally uses quote_ident()) adds quotes to its argument (when needed)
, so the query asks for "MasterTable" when only mastertable is present in the schema.
But, it is easyer to just avoid MixedCase identifiers,

Related

Passing UUID of one table to another table as a Foreign key value in PostgreSQL

I have table Employee in Postgres:
drop table if exists employee;
create table employee (
id uuid default uuid_generate_v4 () primary key,
first_name varchar not null,
last_name varchar not null
);
And another table salary :
drop table if exists salary;
create table salary (
check_id uuid default uuid_generate_v4 () primary key,
salary int not null,
employee_id uuid references employee (id)
);
employee_id is the foreign key to id in the Employee table, but I don't understand how to insert a value inside employee_id since UUID is unique.
I am inserting values into Employee table:
insert into employee (first_name, last_name, email, code) values ( 'jonh', 'smith', 'jonh#example.com', '1');
And then if I try insert values into salary table:
insert into salary (salary ) values ('1000');
Then select command will return employee_id value empty.
But if I make it default uuid_generate_v4 (), then result is: Key (employee_id)=(c4ccd745-02ba-4a0e-8586-32e3c6a2b84a) is not present in table "employee".
I understand that because employee_id is a foreign key it should match with uuid in employee, but since uuid is mostly unique, how can I make it work?
You have to use the uuid that was inserted into the employee table. You can do this with a CTE in a single statement:
WITH new_employee AS (
INSERT INTO employee (first_name, last_name, email, code)
VALUES ('jonh', 'smith', 'jonh#example.com', '1')
RETURNING id
)
INSERT INTO salary (salary, employee_id)
SELECT 1000, id
FROM new_employee;

Copying records in a table with self referencing ids

I have a table with records which can reference another row in the same table so there is a parent-child relationship between rows in the same table.
What I am trying to achieve is to create the same data for another user so that they can see and manage their own version of this structure through the web ui where these rows are displayed as a tree.
Problem is when I bulk insert this data by only changing user_id, I lose the relation between rows because the parent_id values will be invalid for these new records and they should be updated as well with the newly generated ids.
Here is what I tried: (did not work)
Iterate over main_table
copy-paste the static values after each
do another insert on a temp table for holding old and new ids
update old parent_ids with new ids after loop ends
My attempt at doing such thing(last step is not included here)
create or replace function test_x()
returns void as
$BODY$
declare
r RECORD;
userId int8;
rowPK int8;
begin
userId := (select 1)
create table if not exists id_map (old_id int8, new_id int8);
create table if not exists temp_table as select * from main_table;
for r in select * from temp_table
loop
rowPK := insert into main_table(id, user_id, code, description, parent_id)
values(nextval('hibernate_sequence'), userId, r.code, r.description, r.parent_id) returning id;
insert into id_map (old_id, new_id) values (r.id, rowPK);
end loop;
end
$BODY$
language plpgsql;
My PostgreSQL version is 9.6.14.
DDL below for testing.
create table main_table(
id bigserial not null,
user_id int8 not null,
code varchar(3) not null,
description varchar(100) not null,
parent_id int8 null,
constraint mycompkey unique (user_id, code, parent_id),
constraint mypk primary key (id),
constraint myfk foreign key (parent_id) references main_table(id)
);
insert into main_table (id, user_id, code, description, parent_id)
values(0, 0, '01', 'Root row', null);
insert into main_table (id, user_id, code, description, parent_id)
values(1, 0, '001', 'Child row 1', 0);
insert into main_table (id, user_id, code, description, parent_id)
values(2, 0, '002', 'Child row 2', 0);
insert into main_table (id, user_id, code, description, parent_id)
values(3, 0, '002', 'Grand child row 1', 2);
How to write a procedure to accomplish this?
Thanks in advance.
It appears your task is coping all data for a given user to another while maintaining the hierarchical relationship within the new rows. The following accomplishes that.
It begins creating a new copy of the existing rows with the new user_id, including the old row parent_id. That will be user in the next (update) step.
The CTE logically begins with the new rows which have parent_id and joins to the old parent row. From here it joins to the old parent row to the new parent row using the code and description. At that point we have the new id along with the new parent is. At that point just update with those values. Actually for the update the CTE need only select those two columns, but I've left the intermediate columns so you trace through if you wish.
create or replace function copy_user_data_to_user(
source_user_id bigint
, target_user_id bigint
)
returns void
language plpgsql
as $$
begin
insert into main_table ( user_id,code, description, parent_id )
select target_user_id, code, description, parent_id
from main_table
where user_id = source_user_id ;
with n_list as
(select mt.id, mt.code, mt.description, mt.parent_id
, mtp.id p_id,mtp.code p_code,mtp.description p_des
, mtc.id c_id, mtc.code c_code, mtc.description c_description
from main_table mt
join main_table mtp on mtp.id = mt.parent_id
join main_table mtc on ( mtc.user_id = target_user_id
and mtc.code = mtp.code
and mtc.description = mtp.description
)
where mt.parent_id is not null
and mt.user_id = target_user_id
)
update main_table mt
set parent_id = n_list.c_id
from n_list
where mt.id = n_list.id;
return;
end ;
$$;
-- test
select * from copy_user_data_to_user(0,1);
select * from main_table;
CREATE TABLE 'table name you want to create' SELECT * FROM myset
but new table and myset column name should be equal and you can also
use inplace of * to column name but column name exist in new table
othwerwise getting errors

Using an id returned from an insert in a with statement in postgresql

Say that you have the following table structure, that you like wikipedia have the identity and state of a page stored in different tables:
create table endUsers (
uuid UUID primary key,
created timestamptz default now()
);
create table endUserRevisions (
id bigserial primary key,
endUser UUID not null references endUsers,
modified timestamptz default now(),
modifiedBy UUID not null references portalUsers,
name text not null,
company text not null,
email text not null
);
alter table endUsers add column
latestRevision bigint not null references endUserRevisions;
And that you then want to insert a completely new user into this database like:
with lastID as (
insert into endUserRevisions (endUser, name, company, email)
values ('08e7882c-7596-43d1-b4cc-69f855210d72', 'a', 'b', 'c') returning id)
insert into endUsers (uuid, latestRevision)
values ('08e7882c-7596-43d1-b4cc-69f855210d72', lastID);
-- or
with revision as (
insert into endUserRevisions (endUser, name, company, email)
values ('08e7882c-7596-43d1-b4cc-69f855210d72', 'a', 'b', 'c') returning *)
insert into endUsers (uuid, latestRevision)
values ('08e7882c-7596-43d1-b4cc-69f855210d72', revision.id);
Both these variants fail with either
column "lastid" does not exist
or
missing FROM-clause entry for table "last"
The reason why the fail is because each subquery is accessable to the surrounding context as a table, not as a plain value. In other words it must be accessed using a select statement like:
with revision as (
insert into endUserRevisions (endUser, name, company, email)
values ('08e7882c-7596-43d1-b4cc-79f855210d76', 'a', 'b', 'c') returning id)
insert into endUsers (uuid, latestRevision)
values ('08e7882c-7596-43d1-b4cc-79f855210d76', (select id from revision));
-- or
with revision as (
insert into endUserRevisions (endUser, name, company, email)
values ('08e7882c-7596-43d1-b4cc-79f855210d74', 'a', 'b', 'c') returning id)
insert into endUsers (uuid, latestRevision)
select '08e7882c-7596-43d1-b4cc-79f855210d74', revision.id from revision;

How to insert a single row in the parent table and then multiple rows in the child table in single SQL in PostgreSQL?

Please, find below my schema:
CREATE TABLE reps (
id SERIAL PRIMARY KEY,
rep TEXT NOT NULL UNIQUE
);
CREATE TABLE terms (
id SERIAL PRIMARY KEY,
terms TEXT NOT NULL UNIQUE
);
CREATE TABLE shipVia (
id SERIAL PRIMARY KEY,
ship_via TEXT NOT NULL UNIQUE
);
CREATE TABLE invoices (
id SERIAL PRIMARY KEY,
customer TEXT NOT NULL CONSTRAINT customerNotEmpty CHECK(customer <> ''),
term_id INT REFERENCES terms,
rep_id INT NOT NULL REFERENCES reps,
ship_via_id INT REFERENCES shipVia,
...
item_count INT NOT NULL,
modified TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP,
created TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP,
version INT NOT NULL DEFAULT 0
);
CREATE TABLE invoiceItems (
id SERIAL PRIMARY KEY,
invoice_id INT NOT NULL REFERENCES invoices ON DELETE CASCADE,
name TEXT NOT NULL CONSTRAINT nameNotEmpty CHECK(name <> ''),
description TEXT,
qty INT NOT NULL CONSTRAINT validQty CHECK (qty > 0),
price DOUBLE PRECISION NOT NULL
);
I am trying to insert an invoice along with its invoice items in one SQL using writable CTE. I am currently stuck with the following SQL statement:
WITH new_invoice AS (
INSERT INTO invoices (id, customer, term_id, ship_via_id, rep_id, ..., item_count)
SELECT $1, $2, t.id, s.id, r.id, ..., $26
FROM reps r
JOIN terms t ON t.terms = $3
JOIN shipVia s ON s.ship_via = $4
WHERE r.rep = $5
RETURNING id
) INSERT INTO invoiceItems (invoice_id, name, qty, price, description) VALUES
(new_invoice.id,$27,$28,$29,$30)
,(new_invoice.id,$31,$32,$33,$34)
,(new_invoice.id,$35,$36,$37,$38);
Of course, this SQL is wrong, here is what PostgreSQL 9.2 has to say about it:
ERROR: missing FROM-clause entry for table "new_invoice"
LINE 13: (new_invoice.id,$27,$28,$29,$30)
^
********** Error **********
ERROR: missing FROM-clause entry for table "new_invoice"
SQL state: 42P01
Character: 704
Is it possible at all?
EDIT 1
I am trying the following version:
PREPARE insert_invoice_3 AS WITH
new_invoice AS (
INSERT INTO invoices (id, customer, term_id, ship_via_id, rep_id, ..., item_count)
SELECT $1, $2, t.id, s.id, r.id, ..., $26
FROM reps r
JOIN terms t ON t.terms = $3
JOIN shipVia s ON s.ship_via = $4
WHERE r.rep = $5
RETURNING id
),
v (name, qty, price, description) AS (
VALUES ($27,$28,$29,$30)
,($31,$32,$33,$34)
,($35,$36,$37,$38)
)
INSERT INTO invoiceItems (invoice_id, name, qty, price, description)
SELECT new_invoice.id, v.name, v.qty, v.price, v.description
FROM v, new_invoice;
And here is what I get in return:
ERROR: column "qty" is of type integer but expression is of type text
LINE 19: SELECT new_invoice.id, v.name, v.qty, v.price, v.descriptio...
^
HINT: You will need to rewrite or cast the expression.
********** Error **********
ERROR: column "qty" is of type integer but expression is of type text
SQL state: 42804
Hint: You will need to rewrite or cast the expression.
Character: 899
I guess v (name, qty, price, description) is not enough, the data types must be specified as well. However, v (name, qty INT, price, description) does not work - syntax error.
EDIT 2
Next, I have just tried the second version:
PREPARE insert_invoice_3 AS WITH
new_invoice AS (
INSERT INTO invoices (id, customer, term_id, ship_via_id, rep_id, ..., item_count)
SELECT $1, $2, t.id, s.id, r.id, ..., $26
FROM reps r
JOIN terms t ON t.terms = $3
JOIN shipVia s ON s.ship_via = $4
WHERE r.rep = $5
RETURNING id
)
INSERT INTO invoiceItems (invoice_id, name, qty, price, description)
(
SELECT i.id, $27, $28, $29, $30 FROM new_invoice i
UNION ALL
SELECT i.id, $31, $32, $33, $34 FROM new_invoice i
UNION ALL
SELECT i.id, $35, $36, $37, $38 FROM new_invoice i
);
Here is what I get:
ERROR: column "qty" is of type integer but expression is of type text
LINE 15: SELECT i.id, $27, $28, $29, $30 FROM new_invoice i
^
HINT: You will need to rewrite or cast the expression.
********** Error **********
ERROR: column "qty" is of type integer but expression is of type text
SQL state: 42804
Hint: You will need to rewrite or cast the expression.
Character: 759
Seems like the same error. It is interesting that if I remove all the UNION ALL and leave just one SELECT statement - it works!
EDIT 3
Why do I have to cast the parameters? Is it possible to specify the type of columns in the CTE?
PostgreSQL has such an extended interpretation of the VALUES clause that it may be used as a subquery by itself.
So you may express your query in this form:
WITH new_invoice AS (
INSERT INTO ...
RETURNING id
),
v(a,b,c,d) AS (values
($27,$28,$29,$30),
($31,$32,$33,$34),
...
)
INSERT INTO invoiceItems (invoice_id, name, qty, price, description)
SELECT new_invoice.id, a,b,c,d FROM v, new_invoice;
That assumes you want to insert the cartesian product of new_invoice and the values, which mostly makes sense if new_invoice is actually a single-row value.
WITH new_invoice AS (
INSERT INTO invoices ...
RETURNING id
)
INSERT INTO invoiceItems (invoice_id, name, qty, price, description)
VALUES ((select id from new_invoice), $27 , $28, $29, $30),
((select id from new_invoice), $31 , $32, $33, $34),
((select id from new_invoice), $35 , $36, $37, $38);
Instead of insert ... values ...., use insert ... select ...:
) INSERT INTO invoiceItems (invoice_id, name, qty, price, description)
SELECT new_invoice.id,$27,$28,$29,$30 FROM new_invoice
UNION ALL
...

Copy content in TSQL

I need to copy content from one table to itself and related tables... Let me schematize the problem. Let's say I have two tables:
Order
OrderID : int
CustomerID : int
OrderName : nvarchar(32)
OrderItem
OrderItemID : int
OrderID : int
Quantity : int
With the PK being autoincremental.
Let's say I want to duplicate the content of one customer to another. How do I do that efficiently?
The problem are the PKs. I would need to map the values of OrderIDs from the original set of data to the copy in order to create proper references in OrderItem. If I just select-Insert, I won't be able to create that map.
Suggestions?
For duplicating one parent and many children with identities as the keys, I think the OUTPUT clause can make things pretty clean (SqlFiddle here):
-- Make a duplicate of parent 1, including children
-- Setup some test data
create table Parents (
ID int not null primary key identity
, Col1 varchar(10) not null
, Col2 varchar(10) not null
)
insert into Parents (Col1, Col2) select 'A', 'B'
insert into Parents (Col1, Col2) select 'C', 'D'
insert into Parents (Col1, Col2) select 'E', 'F'
create table Children (
ID int not null primary key identity
, ParentID int not null references Parents (ID)
, Col1 varchar(10) not null
, Col2 varchar(10) not null
)
insert into Children (ParentID, Col1, Col2) select 1, 'g', 'h'
insert into Children (ParentID, Col1, Col2) select 1, 'i', 'j'
insert into Children (ParentID, Col1, Col2) select 2, 'k', 'l'
insert into Children (ParentID, Col1, Col2) select 3, 'm', 'n'
-- Get one parent to copy
declare #oldID int = 1
-- Create a place to store new ParentID
declare #newID table (
ID int not null primary key
)
-- Create new parent
insert into Parents (Col1, Col2)
output inserted.ID into #newID -- Capturing the new ParentID
select Col1, Col2
from Parents
where ID = #oldID -- Only one parent
-- Create new children using the new ParentID
insert into Children (ParentID, Col1, Col2)
select n.ID, c.Col1, c.Col2
from Children c
cross join #newID n
where c.ParentID = #oldID -- Only one parent
-- Show some output
select * from Parents
select * from Children
Do you have to have the primary keys from table A as primaries in Table B? If not you can do a select statement with an insert into. Primary Key's are usually int's that start from an ever increasing seed (identity). Going around this and declaring an insert of this same data problematically has the disadvantage of someone thinking this is a distinct key set on this table and not a 'relationship' or foreign key value.
You can Select Primary Key's for inserts into other tables, just not themselves.... UNLESS you set the 'identity insert on' hint. Do not do this unless you know what this does as you can create more problems than it's worth if you don't understand the ramifications.
I would just do the ole:
insert into TableB
select *
from TableA
where (criteria)
Simple example (This assumes SQL Server 2008 or higher). My bad I did not see you did not list TSQL framework. Not sure if this will run on Oracle or MySql.
declare #Order Table ( OrderID int identity primary key, person varchar(8));
insert into #Order values ('Brett'),('John'),('Peter');
declare #OrderItem Table (orderItemID int identity primary key, OrderID int, OrderInfo varchar(16));
insert into #OrderItem
select
OrderID -- I can insert a primary key just fine
, person + 'Stuff'
from #Order
select *
from #Order
Select *
from #OrderItem
Add an extra helper column to Order called OldOrderID
Copy all the Order's from the #OldCustomerID to the #NewCustomerID
Copy all of the OrderItems using the OldOrderID column to help make the relation
Remove the extra helper column from Order
ALTER TABLE Order ADD OldOrderID INT NULL
INSERT INTO Order (CustomerID, OrderName, OldOrderID)
SELECT #NewCustomerID, OrderName, OrderID
FROM Order
WHERE CustomerID = #OldCustomerID
INSERT INTO OrderItem (OrderID, Quantity)
SELECT o.OrderID, i.Quantity
FROM Order o INNER JOIN OrderItem i ON o.OldOrderID = i.OrderID
WHERE o.CustomerID = #NewCustomerID
UPDATE Order SET OldOrderID = null WHERE OldOrderID IS NOT NULL
ALTER TABLE Order DROP COLUMN OldOrderID
IF the OrderName is unique per customer, you could simply do:
INSERT INTO [Order] ([CustomerID], [OrderName])
SELECT
2 AS [CustomerID],
[OrderName]
FROM [Order]
WHERE [CustomerID] = 1
INSERT INTO [OrderItem] ([OrderID], [Quantity])
SELECT
[o2].[OrderID],
[oi1].[Quantity]
FROM [OrderItem] [oi1]
INNER JOIN [Order] [o1] ON [oi1].[OrderID] = [o1].[OrderID]
INNER JOIN [Order] [o2] ON [o1].[OrderName] = [o2].[OrderName]
WHERE [o1].[CustomerID] = 1 AND [o2].[CustomerID] = 2
Otherwise, you will have to use a temporary table or alter the existing Order table like #LastCoder suggested.