I am trying to create the query as a string and execute that in PostgreSQL 10.
As far as I know, I can use the EXECUTE command to execute my query from a defined string.
Unfortunately, I have got an error: SQL Error [42601]: ERROR: syntax error at or near "execute"
Below is my code:
drop table if exists delinquent;
create table delinquent
(
report_date date
,account_id text
)
;
INSERT INTO delinquent VALUES('2019-07-23', 'a1234');
INSERT INTO delinquent VALUES('2019-07-23', 'b5679');
--------------
drop table if exists output1;
create temp table output1
(
report_date date
,account_id text
)
;
--------------
do $$
declare table_name text := 'delinquent';
begin
truncate table output1;
insert into output1
execute concat('select * from ',table_name);
end; $$;
select * from output1;
Anybody has an idea on what is wrong and what to do about it?
Many thanks,
You need to run the complete INSERT statement as dynamic SQL. And to build dynamic SQL, using format() is highly recommended to properly deal with identifiers and literals:
do $$
declare
table_name text := 'delinquent';
some_value text := 'a1234';
begin
truncate table output1;
execute format('insert into output1 select * from %I where some_column = %L',
table_name, some_value);
end; $$;
select *
from output1;
Is it possible to iterate over a table's records and make a left join with them in a stored procedure?
Something like this:
FOR r IN SELECT tablename FROM tablewithtablenames ORDER BY tablename ASC
LOOP
INSERT INTO temp_Results
SELECT
temp_ids.Key as Key,
loggedvalue.pk_timestamp,
FROM
(temp_idS AS temp_ids
LEFT JOIN
quote_ident(r.tablename) AS loggedvalue
ON temp_ids.Key = loggedvalue.pk_fk_id);
END LOOP;
Unfortunately i get the following error message when i want to execute the stored procedure. (Function creation was successful.)
Error message:
ERROR: column loggedvalue.pk_fk_id does not exist LINE 29:
ON temp_ids.Key = "loggedvalue...
I have the feeling that i convert the record in a wrong way maybe because when i manually replaced the quote_ident(r.tablename) to the name of the table that i know the r contains it was fine, also i traced out the r.tablename in the loop and it was correct also.
As a_horse_with_no_name pointed out i should have use dynamic sql because in plpgsql you can not use a variable as a table name so i eliminated the loop and i used a union all:
CREATE OR REPLACE FUNCTION getaffectedtables(
OUT tableNames TEXT)
as $$
BEGIN
SELECT TRIM(TRAILING ' UNION ALL ' FROM string_agg('','SELECT * FROM "' || "tablename" || '" UNION ALL '))
INTO tableNames
FROM exampleTable;
END;$$
LANGUAGE plpgsql;
Then i used dynamic execute:
DECLARE
affectednames TEXT;
BEGIN
affectednames := getaffectedtables();
EXECUTE '
SELECT
temp_ids.Key as Key,
loggedvalue.pk_timestamp,
FROM
(temp_idS AS temp_ids
LEFT JOIN
('|| affectednames ||') AS loggedvalue
ON temp_ids.Key = loggedvalue.pk_fk_id);';
I am trying to create crosstab queries in PostgreSQL such that it automatically generates the crosstab columns instead of hardcoding it. I have written a function that dynamically generates the column list that I need for my crosstab query. The idea is to substitute the result of this function in the crosstab query using dynamic sql.
I know how to do this easily in SQL Server, but my limited knowledge of PostgreSQL is hindering my progress here. I was thinking of storing the result of function that generates the dynamic list of columns into a variable and use that to dynamically build the sql query. It would be great if someone could guide me regarding the same.
-- Table which has be pivoted
CREATE TABLE test_db
(
kernel_id int,
key int,
value int
);
INSERT INTO test_db VALUES
(1,1,99),
(1,2,78),
(2,1,66),
(3,1,44),
(3,2,55),
(3,3,89);
-- This function dynamically returns the list of columns for crosstab
CREATE FUNCTION test() RETURNS TEXT AS '
DECLARE
key_id int;
text_op TEXT = '' kernel_id int, '';
BEGIN
FOR key_id IN SELECT DISTINCT key FROM test_db ORDER BY key LOOP
text_op := text_op || key_id || '' int , '' ;
END LOOP;
text_op := text_op || '' DUMMY text'';
RETURN text_op;
END;
' LANGUAGE 'plpgsql';
-- This query works. I just need to convert the static list
-- of crosstab columns to be generated dynamically.
SELECT * FROM
crosstab
(
'SELECT kernel_id, key, value FROM test_db ORDER BY 1,2',
'SELECT DISTINCT key FROM test_db ORDER BY 1'
)
AS x (kernel_id int, key1 int, key2 int, key3 int); -- How can I replace ..
-- .. this static list with a dynamically generated list of columns ?
You can use the provided C function crosstab_hash for this.
The manual is not very clear in this respect. It's mentioned at the end of the chapter on crosstab() with two parameters:
You can create predefined functions to avoid having to write out the
result column names and types in each query. See the examples in the
previous section. The underlying C function for this form of crosstab
is named crosstab_hash.
For your example:
CREATE OR REPLACE FUNCTION f_cross_test_db(text, text)
RETURNS TABLE (kernel_id int, key1 int, key2 int, key3 int)
AS '$libdir/tablefunc','crosstab_hash' LANGUAGE C STABLE STRICT;
Call:
SELECT * FROM f_cross_test_db(
'SELECT kernel_id, key, value FROM test_db ORDER BY 1,2'
,'SELECT DISTINCT key FROM test_db ORDER BY 1');
Note that you need to create a distinct crosstab_hash function for every crosstab function with a different return type.
Related:
PostgreSQL row to columns
Your function to generate the column list is rather convoluted, the result is incorrect (int missing after kernel_id), it can be replaced with this SQL query:
SELECT 'kernel_id int, '
|| string_agg(DISTINCT key::text, ' int, ' ORDER BY key::text)
|| ' int, DUMMY text'
FROM test_db;
And it cannot be used dynamically anyway.
#erwin-brandstetter: The return type of the function isn't an issue if you're always returning a JSON type with the converted results.
Here is the function I came up with:
CREATE OR REPLACE FUNCTION report.test(
i_start_date TIMESTAMPTZ,
i_end_date TIMESTAMPTZ,
i_interval INT
) RETURNS TABLE (
tab JSON
) AS $ab$
DECLARE
_key_id TEXT;
_text_op TEXT = '';
_ret JSON;
BEGIN
-- SELECT DISTINCT for query results
FOR _key_id IN
SELECT DISTINCT at_name
FROM report.company_data_date cd
JOIN report.company_data_amount cda ON cd.id = cda.company_data_date_id
JOIN report.amount_types at ON cda.amount_type_id = at.id
WHERE date_start BETWEEN i_start_date AND i_end_date
AND interval_type_id = i_interval
LOOP
-- build function_call with datatype of column
IF char_length(_text_op) > 1 THEN
_text_op := _text_op || ', ' || _key_id || ' NUMERIC(20,2)';
ELSE
_text_op := _text_op || _key_id || ' NUMERIC(20,2)';
END IF;
END LOOP;
-- build query with parameter filters
RETURN QUERY
EXECUTE '
SELECT array_to_json(array_agg(row_to_json(t)))
FROM (
SELECT * FROM crosstab(''SELECT date_start, at.at_name, cda.amount ct
FROM report.company_data_date cd
JOIN report.company_data_amount cda ON cd.id = cda.company_data_date_id
JOIN report.amount_types at ON cda.amount_type_id = at.id
WHERE date_start between $$' || i_start_date::TEXT || '$$ AND $$' || i_end_date::TEXT || '$$
AND interval_type_id = ' || i_interval::TEXT || ' ORDER BY date_start'')
AS ct (date_start timestamptz, ' || _text_op || ')
) t;';
END;
$ab$ LANGUAGE 'plpgsql';
So, when you run it, you get the dynamic results in JSON, and you don't need to know how many values were pivoted:
select * from report.test(now()- '1 week'::interval, now(), 1);
tab
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
[{"date_start":"2015-07-27T08:40:01.277556-04:00","burn_rate":0.00,"monthly_revenue":5800.00,"cash_balance":0.00},{"date_start":"2015-07-27T08:50:02.458868-04:00","burn_rate":34000.00,"monthly_revenue":15800.00,"cash_balance":24000.00}]
(1 row)
Edit: If you have mixed datatypes in your crosstab, you can add logic to look it up for each column with something like this:
SELECT a.attname as column_name, format_type(a.atttypid, a.atttypmod) AS data_type
FROM pg_attribute a
JOIN pg_class b ON (a.attrelid = b.relfilenode)
JOIN pg_catalog.pg_namespace n ON n.oid = b.relnamespace
WHERE n.nspname = $$schema_name$$ AND b.relname = $$table_name$$ and a.attstattarget = -1;"
I realise this is an older post but struggled for a little while on the same issue.
My Problem Statement:
I had a table with muliple values in a field and wanted to create a crosstab query with 40+ column headings per row.
My Solution was to create a function which looped through the table column to grab values that I wanted to use as column headings within the crosstab query.
Within this function I could then Create the crosstab query. In my use case I added this crosstab result into a separate table.
E.g.
CREATE OR REPLACE FUNCTION field_values_ct ()
RETURNS VOID AS $$
DECLARE rec RECORD;
DECLARE str text;
BEGIN
str := '"Issue ID" text,';
-- looping to get column heading string
FOR rec IN SELECT DISTINCT field_name
FROM issue_fields
ORDER BY field_name
LOOP
str := str || '"' || rec.field_name || '" text' ||',';
END LOOP;
str:= substring(str, 0, length(str));
EXECUTE 'CREATE EXTENSION IF NOT EXISTS tablefunc;
DROP TABLE IF EXISTS temp_issue_fields;
CREATE TABLE temp_issue_fields AS
SELECT *
FROM crosstab(''select issue_id, field_name, field_value from issue_fields order by 1'',
''SELECT DISTINCT field_name FROM issue_fields ORDER BY 1'')
AS final_result ('|| str ||')';
END;
$$ LANGUAGE plpgsql;
The approach described here worked well for me.
Instead of retrieving the pivot table directly. The easier approach is to let the function generate a SQL query string. Dynamically execute the resulting SQL query string on demand.
I have the following function:
In which I am updating one database table by joining other database table by using the dblink().
I have installed:
create extension dblink;
The more details as shown below:
CREATE OR REPLACE FUNCTION Fun_test
(
Table_Name varchar
)
RETURNS void AS
$BODY$
DECLARE
dynamic_statement varchar;
BEGIN
perform dblink_connect('port=5234 dbname=testdb user=postgres password=****');
dynamic_statement := 'With CTE AS
(
Select HNumber,JoiningDate,Name,Address
From '|| Table_Name ||'c
)
, Test_A
AS
(
Select Row_Number() over ( Partition by PNumber order by Date1 Desc,Date2 Desc) AS roNum,
Name,PNumber,Date1,Address
From dblink(
''Select distinct PNumber,
(
case when fname is null then '' else fname end || '' ||
case when lname is null then '' else lname end
) as FullName,
Address,
Date1,Date2
From testdb_Table
inner join CTE on CTE.HNumber = PNumber''
) Num
)
Update CTE
Set
Name = Test_A.FullName
,SubAddress_A = Test_A.Address
,Date1 = Test_A.Date1
from CTE
left outer join Test_A on
CTE.HNumber= Test_A.PNumber
where roNum =1';
RAISE INFO '%',dynamic_statement;
EXECUTE dynamic_statement;
perform dblink_disconnect();
END;
$BODY$
LANGUAGE PLPGSQL;
Calling Function:
select fun_test('test1');
Getting an error:
ERROR: a column definition list is required for functions returning "record"
LINE 11: From dblink
^
You have to tell PostgreSQL what the columns the dblink query will return are.
See the manual for dblink for details.
This is the same as for any function returning a runtime-determined record type. You can't query it without telling PostgreSQL what the column layout of the results will be.
You use a column specifier list, e.g.
SELECT * FROM my_function_returning_record() f(col1 text, col2 integer);
If you are on a current PostgreSQL version you may want to look at postgres_fdw as an alternative to dblink.
Suppose I create a table in Postgresql with a comment on a column:
create table t1 (
c1 varchar(10)
);
comment on column t1.c1 is 'foo';
Some time later, I decide to add another column:
alter table t1 add column c2 varchar(20);
I want to look up the comment contents of the first column, and associate with the new column:
select comment_text from (what?) where table_name = 't1' and column_name = 'c1'
The (what?) is going to be a system table, but after having looked around in pgAdmin and searching on the web I haven't learnt its name.
Ideally I'd like to be able to:
comment on column t1.c1 is (select ...);
but I have a feeling that's stretching things a bit far. Thanks for any ideas.
Update: based on the suggestions I received here, I wound up writing a program to automate the task of transferring comments, as part of a larger process of changing the datatype of a Postgresql column. You can read about that on my blog.
The next thing to know is how to obtain the table oid. I think that using this as part of comment on will not work, as you suspect.
postgres=# create table comtest1 (id int, val varchar);
CREATE TABLE
postgres=# insert into comtest1 values (1,'a');
INSERT 0 1
postgres=# select distinct tableoid from comtest1;
tableoid
----------
32792
(1 row)
postgres=# comment on column comtest1.id is 'Identifier Number One';
COMMENT
postgres=# select col_description(32792,1);
col_description
-----------------------
Identifier Number One
(1 row)
Anyhow, I whipped up a quick plpgsql function to copy comments from one table/column pair to another. You have to createlang plpgsql on the database and use it like this:
Copy the comment on the first column of table comtest1 to the id
column of the table comtest2. Yes, it should be improved but
that's left as work for the reader.
postgres=# select copy_comment('comtest1',1,'comtest2','id');
copy_comment
--------------
1
(1 row)
CREATE OR REPLACE FUNCTION copy_comment(varchar,int,varchar,varchar) RETURNS int AS $PROC$
DECLARE
src_tbl ALIAS FOR $1;
src_col ALIAS FOR $2;
dst_tbl ALIAS FOR $3;
dst_col ALIAS FOR $4;
row RECORD;
oid INT;
comment VARCHAR;
BEGIN
FOR row IN EXECUTE 'SELECT DISTINCT tableoid FROM ' || quote_ident(src_tbl) LOOP
oid := row.tableoid;
END LOOP;
FOR row IN EXECUTE 'SELECT col_description(' || quote_literal(oid) || ',' || quote_literal(src_col) || ')' LOOP
comment := row.col_description;
END LOOP;
EXECUTE 'COMMENT ON COLUMN ' || quote_ident(dst_tbl) || '.' || quote_ident(dst_col) || ' IS ' || quote_literal(comment);
RETURN 1;
END;
$PROC$ LANGUAGE plpgsql;
You can retrieve comments on columns using the system function col_description(table_oid, column_number). See this page for further details.