I've written a procedure in Firebird (Dialect 3), which returns me something like this:
column1 | column2 | column3 | column4 | ...
----------|-------------|-----------|------------|--------
1 | 55 | 2.5 | 100€ | ...
The specific column names don't really matter. I access it like this
SELECT * FROM MY_PROCEDURE(:START_DATE, :END_DATE);
It only return one row so I guess I could also access it with EXECUTE_PROCEDURE.
Now what I want is to transpose the columns and the rows in the return
row | result
----------|---------
column1 | 1
column2 | 55
column3 | 2.0
column4 | 100€
... | ...
What I initially did is somethink like this:
select 'column1' AS row, column1 AS result
FROM MY_PROCEDURE(:START_DATE, :END_DATE)
union all
select 'column2' AS row, column2 AS result
FROM MY_PROCEDURE(:START_DATE, :END_DATE)
union all
...
Basically one query for each column. It worked. However, eventually I ran into this problem:
Dynamic SQL Error
Too many Contexts of Relation/Procedure/Views. Maxium allowed is 255.
So I need to restructure my script. As you can see, my SQL knowledge is pretty mediocre, and I simply don't know how to fetch each column as a row in a single select.
Would anyone be able to help? Thanks in advance.
Firebird by itself as no unpivot or other built-in support for transposing columns.
The 'best' solution, and probably the most performing solution would be to rewrite MY_PROCEDURE (or write an alternative version) to output the rows transposed.
For example, assuming your stored procedure does something like this:
set term #;
create procedure test_1
returns (id integer, column1 double precision, column2 double precision, column3 double precision)
as
begin
for
select id, column1, column2, column3
from sometable
into :id, :column1, :column2, :column3 do
begin
suspend;
end
end#
set term ;#
You can then rewrite this by manually transposing the values into separate suspends:
set term #;
create procedure test_2
returns (id integer, columnname varchar(100), columnvalue double precision)
as
declare column1 double precision;
declare column2 double precision;
declare column3 double precision;
begin
for
select id, column1, column2, column3
from sometable
into :id, :column1, :column2, :column3 do
begin
columnname = 'column1';
columnvalue = column1;
suspend;
columnname = 'column2';
columnvalue = column2;
suspend;
columnname = 'column3';
columnvalue = column3;
suspend;
end
end#
set term ;#
This will output something like
id columnname columnvalue
1 column1 1.0
1 column2 1.5
1 column3 5.0
2 ...etc
This solution does require that all output (columnvalue) has the same type. Otherwise you will need to cast to a common data type.
Alternatively, you could chain the first procedure into the second procedure by using for select * from test_1 into .... This maybe more or less efficient depending on the internals of your stored procedure:
set term #;
create procedure test_3
returns (id integer, columnname varchar(100), columnvalue double precision)
as
declare column1 double precision;
declare column2 double precision;
declare column3 double precision;
begin
for
select id, column1, column2, column3 from test_1
into :id, :column1, :column2, :column3 do
begin
columnname = 'column1';
columnvalue = column1;
suspend;
columnname = 'column2';
columnvalue = column2;
suspend;
columnname = 'column3';
columnvalue = column3;
suspend;
end
end#
set term ;#
This last option is probably best if you need both variants of the output, as this means you will only have single place for the logic of that stored procedure.
For ad-hoc querying, you can also replace the stored procedure with an execute block with the same code.
Related
My problem: I need to define foreign table dynamically and set different where conditions every time. I am doing this in function, but I am getting error which doesn't make sense to me during creation of the foreign table(via oracle_fdw).
Creation of foreign table that works:
CREATE FOREIGN TABLE MYFOREIGNTABLE
(
column1 int,
column2 text
)
SERVER fwdb
OPTIONS (table $$(
select
column1,
column2
from
table1
where
column3 = 5
and column4 = 'a'
)$$);
Now if I try to split the string for putting there my variables (instead of variable I left there number so anybody can try it), it stop working and I am getting error
[Code: 0, SQL State: 42601] ERROR: syntax error at or near "||"
CREATE FOREIGN TABLE MYFOREIGNTABLE
(
column1 int,
column2 text
)
SERVER fwdb
OPTIONS (table $$(
select
column1,
column2
from
table1
where
column3 = $$ || 5 || $$
and column4 = 'a'
)$$);
Just for sure I tried my string in select to make sure I didn't do any syntax mistake and it works no problem
select $$(
select
column1,
column2
from
table1
where
column3 = $$ || 5 || $$
and column4 = 'a'
)$$
I tried few other things like using concat() or putting my whole string into variable OPTIONS (table myvariable); But neither worked. What is the correct syntax here?
PostgreSQL 11.10 on x86_64-pc-linux-gnu, compiled by gcc (GCC) 4.8.5
20150623 (Red Hat 4.8.5-39), 64-bit
You have to use a string literal as value for a FDW option, expressions like the string concatenation you are trying to use are not allowed.
You will have to construct the complete statement with dynamic SQL, for example
DO
$$DECLARE
var integer := 5;
BEGIN
EXECUTE
format(
E'CREATE FOREIGN TABLE MYFOREIGNTABLE (\n'
' column1 int,\n'
' column2 text\n'
') SERVER fwdb OPTIONS (\n'
' table ''(SELECT column1,\n'
' column2\n'
' FROM table1\n'
' WHERE column3 = %s\n'
' AND column4 = ''''a'''')'')',
var
);
END;$$;
For string variables you have to get the quoting right by using quote_literal(quote_literal(var)).
I am trying to write sub-queries so that I search all tables for a column named id and since there are multiple tables with id column, I want to add the condition, so that id = 3119093.
My attempt was:
Select *
from information_schema.tables
where id = '3119093' and id IN (
Select table_name
from information_schema.columns
where column_name = 'id' );
This didn't work so I tried:
Select *
from information_schema.tables
where table_name IN (
Select table_name
from information_schema.columns
where column_name = 'id' and 'id' IN (
Select * from table_name where 'id' = 3119093));
This isn't the right way either. Any help would be appreciated. Thanks!
A harder attempt is:
CREATE OR REPLACE FUNCTION search_columns(
needle text,
haystack_tables name[] default '{}',
haystack_schema name[] default '{public}'
)
RETURNS table(schemaname text, tablename text, columnname text, rowctid text)
AS $$
begin
FOR schemaname,tablename,columnname IN
SELECT c.table_schema,c.table_name,c.column_name
FROM information_schema.columns c
JOIN information_schema.tables t ON
(t.table_name=c.table_name AND t.table_schema=c.table_schema)
WHERE (c.table_name=ANY(haystack_tables) OR haystack_tables='{}')
AND c.table_schema=ANY(haystack_schema)
AND t.table_type='BASE TABLE'
--AND c.column_name = "id"
LOOP
EXECUTE format('SELECT ctid FROM %I.%I WHERE cast(%I as text) like %L',
schemaname,
tablename,
columnname,
needle
) INTO rowctid;
IF rowctid is not null THEN
RETURN NEXT;
END IF;
END LOOP;
END;
$$ language plpgsql;
select * from search_columns('%3119093%'::varchar,'{}'::name[]) ;
The only problem is this code displays the table name and column name. I have to then manually enter
Select * from table_name where id = 3119093
where I got the table name from the code above.
I want to automatically implement returning rows from a table but I don't know how to get the table name automatically.
I took the time to make it work for you.
For starters, some information on what is going on inside the code.
Explanation
function takes two input arguments: column name and column value
it requires a created type that it will be returning a set of
first loop identifies tables that have a column name specified as the input argument
then it forms a query which aggregates all rows that match the input condition inside every table taken from step 3 with comparison based on ILIKE - as per your example
function goes into the second loop only if there is at least one row in currently visited table that matches specified condition (then the array is not null)
second loop unnests the array of rows that match the condition and for every element it puts it in the function output with RETURN NEXT rec clause
Notes
Searching with LIKE is inefficient - I suggest adding another input argument "column type" and restrict it in the lookup by adding a join to pg_catalog.pg_type table.
The second loop is there so that if more than 1 row is found for a particular table, then every row gets returned.
If you are looking for something else, like you need key-value pairs, not just the values, then you need to extend the function. You could for example build json format from rows.
Now, to the code.
Test case
CREATE TABLE tbl1 (col1 int, id int); -- does contain values
CREATE TABLE tbl2 (col1 int, col2 int); -- doesn't contain column "id"
CREATE TABLE tbl3 (id int, col5 int); -- doesn't contain values
INSERT INTO tbl1 (col1, id)
VALUES (1, 5), (1, 33), (1, 25);
Table stores data:
postgres=# select * From tbl1;
col1 | id
------+----
1 | 5
1 | 33
1 | 25
(3 rows)
Creating type
CREATE TYPE sometype AS ( schemaname text, tablename text, colname text, entirerow text );
Function code
CREATE OR REPLACE FUNCTION search_tables_for_column (
v_column_name text
, v_column_value text
)
RETURNS SETOF sometype
LANGUAGE plpgsql
STABLE
AS
$$
DECLARE
rec sometype%rowtype;
v_row_array text[];
rec2 record;
arr_el text;
BEGIN
FOR rec IN
SELECT
nam.nspname AS schemaname
, cls.relname AS tablename
, att.attname AS colname
, null::text AS entirerow
FROM
pg_attribute att
JOIN pg_class cls ON att.attrelid = cls.oid
JOIN pg_namespace nam ON cls.relnamespace = nam.oid
WHERE
cls.relkind = 'r'
AND att.attname = v_column_name
LOOP
EXECUTE format('SELECT ARRAY_AGG(row(tablename.*)::text) FROM %I.%I AS tablename WHERE %I::text ILIKE %s',
rec.schemaname, rec.tablename, rec.colname, quote_literal(concat('%',v_column_value,'%'))) INTO v_row_array;
IF v_row_array is not null THEN
FOR rec2 IN
SELECT unnest(v_row_array) AS one_row
LOOP
rec.entirerow := rec2.one_row;
RETURN NEXT rec;
END LOOP;
END IF;
END LOOP;
END
$$;
Exemplary call & output
postgres=# select * from search_tables_for_column('id','5');
schemaname | tablename | colname | entirerow
------------+-----------+---------+-----------
public | tbl1 | id | (1,5)
public | tbl1 | id | (1,25)
(2 rows)
When I select my data I can do a simple join to resolve the values of some of the columns but not all. Most of my columns have one value of data, like 10997, but other columns have multiple values of data, like 10997, 10889, 10123. I have created a function to resolve the three separate values in to the text I need but I am having trouble trying to figure out how to use it.
I have a basic join like this:
SELECT COLUMN1, COLUMN2, COLUMN3
FROM TABLE1 A
JOIN TABLE2 B ON A.ID = B.ID
The result of this will look like this:
Column1 Column2 Column3
1 11272, 11273, 11274, 11277 7712
The function I created uses declared variables and a table variable.
What I'd like to be able to do is something like this:
SELECT COLUMN1, dbo.MyFunction(COLUMN2), COLUMN3
FROM TABLE1 A
JOIN TABLE2 B ON A.ID = B.ID
Resulting in this:
Column1 Column2 Column3
1 Radio, Flyer, Internet, Bar 7712
The problem is that my function uses variables and I can't find where to put it in SQL (table valued function, stored proc, etc) so I can use it how I'd like. Each area has it's own limitation.
EDIT: Here is the code I created for my function, currently it's in a multi-statement table-valued function
DECLARE #looper INT, #res VARCHAR(100)
DECLARE #outList VARCHAR(300)
SET #outList = ''
DECLARE #tmpa TABLE (Item INT)
INSERT INTO #tmpa
SELECT Item
FROM fn_Split (', ', #input)
SELECT #looper = MIN(Item)
FROM #tmpa
WHILE #looper IS NOT NULL
BEGIN
SELECT #res = NAME
FROM FSM_CustomFormSelectOptions CFSO
WHERE ID = #looper
--print #res
SET #outList = #outList + #res + ', '
SELECT #looper = MIN(Item) FROM #tmpa WHERE Item > #looper
END
SET #outList = LEFT(#outList, LEN(#outList)-1)
INSERT INTO #Answers
VALUES(#outList)
RETURN
Sample call is:
select * from fn_GetAnswerText( '11273, 11274, 11275')
You are not returning anything
RETURN #outList
Hello what is the easiest way to duplicate a DB record over the same table?
My problem is that the table where I am doing this has many column, like 100+, and I don't like how the solution looks like. Here is what I do (this is inside plpqsql function):
...
1. duplicate record
INSERT INTO history
(SELECT NEXTVAL('history_id_seq'), col_1, col_2, ... , col_100)
FROM history
WHERE history_id = 1234
ORDER BY datetime DESC
LIMIT 1)
RETURNING
history_id INTO new_history_id;
2. update some columns
UPDATE history
SET
col_5 = 'test_5',
col_23 = 'test_23',
datetime = CURRENT_TIMESTAMP
WHERE history_id = new_history_id;
Here are the problems I am attempting to solve
Listing all these 100+ columns looks lame
When new column is added eventually the function should be updated too
On separate DB instances the column order might differ, which would cause the function fail
I am not sure if I can list them once more (solving issue 3) like insert into <table> (<columns_list>) values (<query>) but then the query looks even uglier.
I would like to achieve something like 'insert into ', but this seems impossible the unique primary key constraint will raise a duplication error.
Any suggestions?
Thanks in advance for you time.
This isn't pretty or particularly optimized but there are a couple of ways to go about this. Ideally, you might want to do this all in an UPDATE trigger though you could implement a duplication function something like this:
-- create source table
CREATE TABLE history (history_id serial not null primary key, col_2 int, col_3 int, col_4 int, datetime timestamptz default now());
-- add some data
INSERT INTO history (col_2, col_3, col_4)
SELECT g, g * 10, g * 100 FROM generate_series(1, 100) AS g;
-- function to duplicate record
CREATE OR REPLACE FUNCTION fn_history_duplicate(p_history_id integer) RETURNS SETOF history AS
$BODY$
DECLARE
cols text;
insert_statement text;
BEGIN
-- build list of columns
SELECT array_to_string(array_agg(column_name::name), ',') INTO cols
FROM information_schema.columns
WHERE (table_schema, table_name) = ('public', 'history')
AND column_name <> 'history_id';
-- build insert statement
insert_statement := 'INSERT INTO history (' || cols || ') SELECT ' || cols || ' FROM history WHERE history_id = $1 RETURNING *';
-- execute statement
RETURN QUERY EXECUTE insert_statement USING p_history_id;
RETURN;
END;
$BODY$
LANGUAGE 'plpgsql';
-- test
SELECT * FROM fn_history_duplicate(1);
history_id | col_2 | col_3 | col_4 | datetime
------------+-------+-------+-------+-------------------------------
101 | 1 | 10 | 100 | 2013-04-15 14:56:11.131507+00
(1 row)
As I noted in my original comment, you might also take a look at the colnames extension as an alternative to querying the information schema.
You don't need the update anyway, you can supply the constant values directly in the SELECT statement:
INSERT INTO history
SELECT NEXTVAL('history_id_seq'),
col_1,
col_2,
col_3,
col_4,
'test_5',
...
'test_23',
...,
col_100
FROM history
WHERE history_sid = 1234
ORDER BY datetime DESC
LIMIT 1
RETURNING history_sid INTO new_history_sid;
I have a table with n columns and I need to create a view which contains the frequencies of every unique value in every column. n is unknown since, I need to apply the solution on numerous tables with different number of columns.
For example i have table:
column1 column2 column3
value1 value2 value3
value2 value2 value1
value1 value2 value2
The view should be something like this:
columnname value frequency
column1 value1 2
column1 value2 1
column2 value2 3
...
Since I have very little experience with sql any help would be extremely appreciated.
Many thanks in advance!
Thus far I have come up with this but am sort of stonewalled now.
CREATE or REPLACE FUNCTION create_view () RETURNS setof record AS $$
DECLARE
col RECORD;
BEGIN
for col in execute 'select column_name from information_schema.columns
where table_name = ''table123''' LOOP
???
END LOOP;
return;
END;
$$
LANGUAGE 'plpgsql';
Simple SQL:
SELECT 'col1' AS col, col1 AS val, count(*) AS ct FROM tbl GROUP BY col1
UNION ALL
SELECT 'col2', col2, count(*) FROM tbl GROUP BY col2
UNION ALL
SELECT 'col3', col3, count(*) FROM tbl GROUP BY col3
PL/pqSQL function executing dynamic SQL:
CREATE OR REPLACE FUNCTION f_demo(_schema text, _tbl text)
RETURNS TABLE(col text, val text, ct bigint) AS
$xx$
DECLARE
_fld text;
_sql text := '';
BEGIN
FOR _fld IN
SELECT a.attname -- use quote_ident to safeguard against SQLi
FROM pg_catalog.pg_attribute a
WHERE a.attrelid = (COALESCE(_schema || '.', '') || _tbl)::regclass
AND a.attnum > 0
AND NOT a.attisdropped
-- AND a.attname ~~ '%col%' -- if you want to pick specific columns
LOOP
RETURN QUERY EXECUTE
'SELECT $1, ' || quote_ident(_fld) || '::text, count(*)
FROM ' || COALESCE(quote_ident(_schema) || '.', '') || quote_ident(_tbl) || '
GROUP BY 2'
USING _fld;
END LOOP;
END;
$xx$
LANGUAGE 'plpgsql';
Call:
SELECT * FROM f_demo('public', 'mytable');
Or, if you want to use the schema provided by search_path:
SELECT * FROM f_demo(NULL, 'mytable');
Major points
Works for any table and any number of columns with values of any type.
Values are cast to text to simplify my example. Could be done with a polymorphic type, too.
See this related answer for info on plpgsql techniques and links: https://stackoverflow.com/q/8146245/939860