Postgres subquery in FROM clause - postgresql

In Postgres, I'm using a subquery in a FROM clause, but don't really know how to get the result I want. Say I have 2 tables, tableNameRegister(idNum integer, tableName text) and mytable(comment text, orderVal integer).
The tableNameRegister would look something like this:
idnum | tablename
-------+------------
1 | mytable
2 | othertable
And mytable would look something like this:
comment | orderval
-----------+-------
comment 1 | 1
comment 2 | 2
What I want to to is take the tableName value from tableNameRegister, and select from that tableName value, but all in one query. Essentially, something like this:
tabName = 'SELECT tableName FROM tableNameRegister WHERE idNum = 1;'
'SELECT * FROM ${tabName} WHERE orderVal = 2'
Ideally this would return the row containing comment 2. So I tried doing it in a subquery:
'SELECT * FROM (SELECT tableName FROM tableNameRegister WHERE idNum = 1) AS tabname WHERE orderVal = 2;'
But found out but this doesn't work the way I thought it would. Essentially, it returns a sort of subtable based off the results of the subquery, and not the actual value of mytable that I want. I was wondering if there is a way to do what I intended, all in one query/using subqueries? Or is this something I would have to do programmatically and seperate into two queries?

This type of thing is best done through some sort of scripting or otherwise programmatically. With something like plpgsql we have more flexibility, but even with this, the tricky thing will be if all the tables do not share the same structure or at least common column names. If they did or could be abstracted in some way something like the following could get you started:
CREATE OR REPLACE FUNCTION tablequery(id_num int)
RETURNS TABLE (comment varchar, orderval int)
AS $$
DECLARE
tableName varchar(50);
BEGIN
SELECT table_name FROM tableNameRegister WHERE idnum = id_num LIMIT 1 into tableName;
RETURN QUERY EXECUTE format('SELECT * FROM %I WHERE orderVal = 2;', tableName);
END;
$$ LANGUAGE plpgsql;
select * FROM tableQuery(1);

Related

how to iterate in all schemas and find count from all tables present in all schemas with same table name for every 5mins?

imagine there are 5 schemas in my database and in every schema there is a common name table (ex:- table1) after every 5mins records get inserted in table1, how I can iterate in all schemas n calculate the count of table1[i have to automate the process so i am going to write the code in function and call that function after every 5mins using crontab].
Basically 2 options: Hard code schema.table and union the results. So something like:
create or replace function count_rows_in_each_table1()
returns table (schema_name text, number_or_rows integer)
language sql
as $$
select 'schema1', count(*) from schema1.table1 union all
select 'schema2', count(*) from schema2.table1 union all
select 'schema3', count(*) from schema3.table1 union all
...
select 'scheman', count(*) from scheman.table1;
$$;
The alternative being building the query dynamically from information_scheme.
create or replace function count_rows_in_each_table1()
returns table (schema_name text, number_of_rows bigint)
language plpgsql
as $$
declare
c_rows_count cursor is
select table_schema::text
from information_schema.tables
where table_name = 'table1';
l_tbl record;
l_sql_statement text = '';
l_connector text = '';
l_base_select text = 'select ''%s'', count(*) from %I.table1';
begin
for l_tbl in c_rows_count
loop
l_sql_statement = l_sql_statement ||
l_connector ||
format (l_base_select, l_tbl.table_schema, l_tbl.table_schema);
l_connector = ' union all ';
end loop;
raise notice E'Running Query: \n%', l_sql_statement;
return query execute l_sql_statement;
end;
$$;
Which is better. With few schema and few schema add/drop, opt for the first. It is direct and easily shows what you are doing. If you add/drop schema often then opt for the second. If you have many schema, but seldom add/drop them then modify the second to generate the first, save and schedule execution of the generated query.
NOTE: Not tested

Postgres - How to use variables to call a function and returns a table/rowset from DBeaver/pgAdmin

I'm new to Postgres. I tried to run the following in pgAdmin/DBeaver but getting "ERROR: query has no destination for result data" error
do $$
declare customerid integer := 151;
begin
SELECT * FROM get_orders(customerid);
end $$
My guts tell me that it is something simple. What do I need to change so it will display the results in DBeaver or pgAdmin?
I don't want something like this:
SELECT * FROM get_orders(151);
I do want to use something like a variable to separate from the actual select statement.
Thanks.
script segments to prepare the table/function
------------------
CREATE TABLE orders
(
id integer,
customerid INTEGER,
description varchar(100)
)
------------------
INSERT INTO Orders VALUES
(1,101, 'Test Order 1'),
(2,151, 'Random Order')
------------------
CREATE OR REPLACE FUNCTION get_orders (p_customerid int)
RETURNS TABLE (
id integer,
customerid INTEGER,
description varchar(100)
)
AS $$
BEGIN
RETURN QUERY SELECT
*
FROM
orders ord
WHERE
ord.customerid = p_customerid;
END; $$
LANGUAGE 'plpgsql';
SQL is different from other programming languages:
It has no concept of execution order, so the idea of first setting a variable and then using it is alien to SQL.
SQL has no concept of “variables” in the first place.
In SQL, a “program” is a single statement.
So, SQL is lacking when it comes to variables and procedural programming. The remedy is to use a function in a procedural language like PL/pgSQL. Then you can use variables, and you can return the output as function result.
The DO statement has no way of returning results.
with the help of this: How to declare a variable in a PostgreSQL query.
I guess DO statement can't return results. Here are alternatives:
DO $$
declare customerid integer := 151;
BEGIN
CREATE TEMP TABLE tmp_orders ON COMMIT DROP AS
SELECT * FROM get_orders(customerid);
END $$;
SELECT * FROM tmp_orders;
-----------------------------
PREPARE temp(int) AS
SELECT * FROM get_orders($1);
EXECUTE temp(151)
-----------------------------
WITH
custid AS (VALUES (151))
SELECT * FROM get_orders((table custid))
-- DBeaver Only
#set custid = 151
SELECT * FROM get_orders(${custid});

Dyanmic Table name in Postgres query

Is there any way to replace the table name in a query with value stored in another table ? This is in postgres sql
Eg
Meta_table
col1 | col 2
Table 1 | val1
Table 2 | val2
My requirement
select *
from (select col1 from meta_table where col2 = val2)
Probably the most flexible and efficient way is dynamically creating a temporary view using function:
create or replace function f_prepare(tname text, vname text) returns text language plpgsql as $$
begin
execute format(
'create or replace temporary view %I as select * from %I',
vname, tname);
return vname;
end $$;
Then you can use the created view in usual way, for example;
select f_prepare('pg_class', 'v_class');
select * from v_class where relname = 'pg_database'; -- Index on the source table will be used here
and using your code:
select f_prepare((select col1 from meta_table where col2 = 'val2'), 'v');
select * from v;
And as any other temporary objects, created views will not conflict with other sessions and will be dropped on disconnect.
If you want to change the table name of a table, then you can just update the relname column in table pg_class.
But for this, you need admin access to the Postgresql.
The query goes like:-
update pg_class set relname='new_table_name' where relname='old_table_name';
So to do this in single line, You can do like this:
update pg_class set relname=(select col1 from meta_table where col2 = val2) where relname='old_table_name';
You can use Do statement with cursor:
Try This:
DO $$
DECLARE
_query text;
_cursor CONSTANT refcursor := '_cursor';
BEGIN
_query := 'SELECT * FROM '|| (select col1 from meta_table where col2 = 'val1');
OPEN _cursor FOR EXECUTE _query;
END
$$;
FETCH ALL FROM _cursor;

Postgresql 9.1 select from all schemas

I have a Postgresql 9.1 database with couple hundred schemas. All have same structure, just different data. I need to perform a select on a table and get data from each schema. Unfortunately I haven't found a decent way to do it.
I tried setting the search path to schema_1,schema_2, etc and then perform a select on the table but it only selects data from the first schema.
The only way I managed to do it so far is by generating a big query like:
select * from schema_1.table
union
select * from schema_2.table
union
(...another 100 lines....)
Is there any other way to do this in a more reasonable fashion? If this is not possible, can I at least find out which of the schemas has records in that table without performing this select?
Different schemas mean different tables, so if you have to stick to this structure, it'll mean unions, one way or the other. That can be pretty expensive. If you're after partitioning through the convenience of search paths, it might make sense to reverse your schema:
Store a big table in the public schema, and then provision views in each of the individual schemas.
Check out this sqlfiddle that demonstrates my concept:
http://sqlfiddle.com/#!12/a326d/1
Also pasted inline for posterity, in case sqlfiddle is inaccessible:
Schema:
CREATE SCHEMA customer_1;
CREATE SCHEMA customer_2;
CREATE TABLE accounts(id serial, name text, value numeric, customer_id int);
CREATE INDEX ON accounts (customer_id);
CREATE VIEW customer_1.accounts AS SELECT id, name, value FROM public.accounts WHERE customer_id = 1;
CREATE VIEW customer_2.accounts AS SELECT id, name, value FROM public.accounts WHERE customer_id = 2;
INSERT INTO accounts(name, value, customer_id) VALUES('foo', 100, 1);
INSERT INTO accounts(name, value, customer_id) VALUES('bar', 100, 1);
INSERT INTO accounts(name, value, customer_id) VALUES('biz', 150, 2);
INSERT INTO accounts(name, value, customer_id) VALUES('baz', 75, 2);
Queries:
SELECT SUM(value) FROM public.accounts;
SET search_path TO 'customer_1';
SELECT * FROM accounts;
SET search_path TO 'customer_2';
SELECT * FROM accounts;
Results:
425
1 foo 100
2 bar 100
3 biz 150
4 baz 75
If you have to know some about data in tables, you have to do SELECT. There is no any other way. Schema is just logical addressing - for your case is important, so you use lot of tables, and you have to do massive UNION.
search_path works as expected. It has no meaning - return data from mentioned schemes, but it specify a order for searching not fully qualified table. Searching ends on first table, that has requested name.
Attention: massive unions can require lot of memory.
you can use a dynamic SQL and stored procedures with temp table:
postgres=# DO $$
declare r record;
begin
drop table if exists result;
create temp table result as select * from x.a limit 0; -- first table;
for r in select table_schema, table_name
from information_schema.tables
where table_name = 'a'
loop
raise notice '%', r;
execute format('insert into result select * from %I.%I',
r.table_schema,
r.table_name);
end loop;
end; $$;
result:
NOTICE: (y,a)
NOTICE: (x,a)
DO
postgres=# select * from result;
a
----
1
2
3
4
5
..
Here's one approach. You will need to pre-feed it all the schema names you are targeting. You could change this to just loop through all the schemas as Pavel shows if you know you want every schema. In my example I have three schemas that I care about each containing a table called bar. The logic will run a select on each schema's bar table and insert the value into a result table. At the end you have a table with all the data from all the tables. You could change this to update, delete, or do DDL. I chose to keep it simple and just collect the data from each table in each schema.
--START SETUP AKA Run This Section Once
create table schema3.bar(bar_id SERIAL PRIMARY KEY,
bar_name VARCHAR(50) NOT NULL);
insert into schema1.bar(bar_name) select 'One';
insert into schema2.bar(bar_name) select 'Two';
insert into schema3.bar(bar_name) select 'Three';
--END SETUP
DO $$
declare r record;
DECLARE l_id INTEGER = 1;
DECLARE l_schema_name TEXT;
begin
drop table if exists public.result;
create table public.result (bar_id INTEGER, bar_name TEXT);
drop table if exists public.schemas;
create table public.schemas (id serial PRIMARY KEY, schema_name text NOT NULL);
INSERT INTO public.schemas(schema_name)
VALUES ('schema1'),('schema2'),('schema3');
for r in select *
from public.schemas
loop
raise notice '%', r;
SELECT schema_name into l_schema_name
FROM public.schemas
WHERE id = l_id;
raise notice '%', l_schema_name;
EXECUTE 'set search_path TO ' || l_schema_name;
EXECUTE 'INSERT into public.result(bar_id, bar_name) select bar_id, bar_name from ' || l_schema_name || '.bar';
l_id = l_id + 1;
end loop;
end; $$;
--DEBUG
select * from schema1.bar;
select * from schema2.bar;
select * from schema3.bar;
select * from public.result;
select * from public.schemas;
--CLEANUP
--DROP TABLE public.result;
--DROP TABLE public.schemas;

PostgreSQL equivalent of Oracle "bulk collect"

In PostgreSQL exists some ways to make a statement using bulk collect into like in Oracle?
Example in Oracle:
create or replace procedure prc_tst_bulk_test is
type typ_person is table of tb_person%rowtype;
v_tb_person typ_person;
begin
select *
bulk collect into v_tb_person
from tb_person;
-- make a selection in v_tb_person, for instance
select name, count(*) from v_tb_person where age > 50
union
select name, count(*) from v_tb_person where gender = 1
end;
In PostgreSQL 10 you can use array_agg:
declare
v_ids int[];
begin
select array_agg(id) INTO v_ids
from mytable1
where host = p_host;
--use v_ids...
end;
You'll have array and it can be used to make select from it using unnest:
select * from unnest(v_ids) where ...
There is no such syntax in PostgreSQL, nor a close functional equivalent.
You can create a temporary table in your PL/PgSQL code and use that for the desired purpose. Temp tables in PL/PgSQL are a little bit annoying because the names are global within the session, but they work correctly in PostgreSQL 8.4 and up.
A better alternative for when you're doing all the work within a single SQL statement is to use a common table expression (CTE, or WITH query). This won't be suitable for all situations.
The example above would be much better solved by a simple RETURN QUERY in PL/PgSQL, but I presume your real examples are more complex.
Assuming that tb_person is some kind of expensive-to-generate view that you don't just want to scan in each branch of the union, you could do something like:
CREATE OR REPLACE FUNCTION prc_tst_bulk()
RETURNS TABLE (name text, rowcount integer) AS
$$
BEGIN
RETURN QUERY
WITH v_tb_person AS (SELECT * FROM tb_person)
select name, count(*) from v_tb_person where age > 50
union
select name, count(*) from v_tb_person where gender = 1;
END;
$$ LANGUAGE plpgsql;
This particular case can be further simplified into a plain SQL function:
CREATE OR REPLACE FUNCTION prc_tst_bulk()
RETURNS TABLE (name text, rowcount integer) AS
$$
WITH v_tb_person AS (SELECT * FROM tb_person)
select name, count(*) from v_tb_person where age > 50
union
select name, count(*) from v_tb_person where gender = 1;
$$ LANGUAGE sql;
You can use a PostgreSQL arrays too - it is similar to Oracle's collections:
postgres=# create table _foo(a int, b int);
CREATE TABLE
postgres=# insert into _foo values(10,20);
INSERT 0 1
postgres=# create or replace function multiply()
returns setof _foo as $$
/*
* two tricks are here
* table name can be used as type name
* table name can be used as fictive column that packs all fields
*/
declare a _foo[] = (select array(select _foo from _foo));
begin
return query select * from unnest(a)
union
all select * from unnest(a);
end;
$$ language plpgsql;
CREATE FUNCTION
postgres=# select * from multiply();
a | b
----+----
10 | 20
10 | 20
(2 rows)
But in your case Craig Ringer's proposal is perfect and should be preferable.
-- Fetch the next 5 rows in the cursor_01:
FETCH FORWARD 5 FROM cursor_01;
PostgreSQL 10+ works.
https://www.postgresql.org/docs/10/sql-fetch.html