i need help in my plpgsql, must return a temporary table that has dynamic columns, how can I do this?
as the name of the columns may vary, I do not know how to finish this procedure
Sorry, google translator :D
CREATE OR REPLACE FUNCTION getreport(reportid INTEGER, userId VARCHAR)
RETURNS SETOF RECORD AS
$$
DECLARE
recordResultadoFinal RECORD;
recordResultadoNomeEspecificos RECORD;
varGetSqlRelatorio VARCHAR;
varAreaId queryreports.f_area%TYPE;
varClientId queryreports.f_client%TYPE;
varTableNameTemp VARCHAR := 'temp'||userId;
varSqlAlterTable VARCHAR := '';
varSqlUpdateTemp VARCHAR := '';
varNomeColunaSpecificData VARCHAR := '';
BEGIN
SELECT f_sql,f_area,f_client INTO varGetSqlRelatorio,varAreaId,varClientId FROM queryreports WHERE f_id = reportid;
EXECUTE 'DROP TABLE IF EXISTS '||varTableNameTemp;
EXECUTE 'CREATE TEMP TABLE '||varTableNameTemp||' AS '||varGetSqlRelatorio;
EXECUTE 'CREATE INDEX processid_idx ON '||varTableNameTemp||' USING btree (processid)';
FOR recordResultadoNomeEspecificos IN EXECUTE '
SELECT DISTINCT cs.f_id as idcoluna, cs.f_name as nomecoluna, cs.f_type as tipodado
FROM clientspecifics cs
INNER JOIN clientspecificdatas csd ON (cs.f_id = csd.f_clientspecific AND csd.f_process IN (SELECT processid FROM '||varTableNameTemp||'))
ORDER BY 2
'
LOOP
varSqlAlterTable := varSqlAlterTable||' ALTER TABLE '||varTableNameTemp||' ADD COLUMN specific_'||recordResultadoNomeEspecificos.idcoluna||' varchar;';
IF (recordResultadoNomeEspecificos.tipodado = 1) THEN varNomeColunaSpecificData := 'f_text';
ELSIF (recordResultadoNomeEspecificos.tipodado = 2) THEN varNomeColunaSpecificData := 'f_name';
ELSIF (recordResultadoNomeEspecificos.tipodado = 3) THEN varNomeColunaSpecificData := 'f_date';
ELSIF (recordResultadoNomeEspecificos.tipodado = 4) THEN varNomeColunaSpecificData := 'f_value';
ELSIF (recordResultadoNomeEspecificos.tipodado = 5) THEN varNomeColunaSpecificData := 'f_text';
END IF;
varSqlUpdateTemp := varSqlUpdateTemp||' UPDATE '||varTableNameTemp||' SET specific_'||recordResultadoNomeEspecificos.idcoluna||' = csd.'||varNomeColunaSpecificData||'
FROM clientspecificdatas csd
WHERE csd.f_process = processid
AND csd.f_clientspecific = '||recordResultadoNomeEspecificos.idcoluna||';';
END LOOP;
EXECUTE varSqlAlterTable;
EXECUTE varSqlUpdateTemp;
RETURN QUERY EXECUTE 'SELECT * FROM '||varTableNameTemp;
END;
$$ LANGUAGE 'plpgsql';
You have a few options:
You can return a refcursor and then fetch from that. This only works in transactions.
You can return an xml document and then process that in your app
You can return JSON or HSTORE.
You can return setof record and specify column lists in the function declaration but this is ugly and brittle.
The problem is that PostgreSQL needs to know return types when planning a query. this means you can't have things like jagged rows or dynamic numbers of rows. These have to be wrapped in something the planner can accommodate.
Related
I am trying to write function in postgresql, that creates temp_table with columns table_name text, table_rec jsonb and fill it through for loop with table names from my table containing names of tables and records in json. I have the for loop in string and I want to execute it. But it doesnt work.
I have variable rec record, sql_query text and tab_name text and I want to do this:
CREATE OR REPLACE FUNCTION public.test51(
)
RETURNS TABLE(tabel_name text, record_json jsonb)
LANGUAGE 'plpgsql'
COST 100
VOLATILE
ROWS 1000
AS $BODY$
declare
rec record;
tabel_name text;
tabel_names text[];
counter integer := 1;
sql_query text;
limit_for_sending integer;
rec_count integer;
begin
select into tabel_names array(select "TABLE_NAME" from public."TABLES");
create temp table temp_tab(tab_nam text, recik jsonb);
while array_length(tabel_names, 1) >= counter loop
tabel_name := '"' || tabel_names[counter] || '"';
select into limit_for_sending "TABLE_LIMIT_FOR_SENDING_DATA" from public."TABLES" where "TABLE_NAME" = tabel_name;
sql_query := 'select count(*) from public.' || tabel_name;
execute sql_query into rec_count;
if (rec_count >= limit_for_sending and limit_for_sending is not null) then
sql_query := 'for rec in select * from public.' || tabel_name || '
loop
insert into temp_tab
select ' || tabel_name || ', to_jsonb(rec);
end loop';
execute sql_query;
end if;
counter := counter + 1;
end loop;
return query
select * from temp_tabik;
drop table temp_tabik;
end;
$BODY$;
Thank you for response.
It seems you have some table that contains the information for which tables you want to return all rows as JSONB. And that meta-table also contains a column that sets a threshold under which the rows should not be returned.
You don't need the temp table or an array to store the table names. You can iterate through the query on the TABLES table and run the dynamic SQL directly in that loop.
return query in PL/pgSQL doesn't terminate the function, it just appends the result of the query to the result of the function.
Dynamic SQL is best created using the format() function because it is easier to read and using the %I placeholder will properly deal with quoted identifiers (which is really important as you are using those dreaded upper case table names)
As far as I can tell, your function can be simplified to:
CREATE OR REPLACE FUNCTION public.test51()
RETURNS TABLE(tabel_name text, record_json jsonb)
LANGUAGE plpgsql
AS
$BODY$
declare
rec record;
sql_query text;
rec_count bigint;
begin
for rec in
select "TABLE_NAME" as table_name, "TABLE_LIMIT_FOR_SENDING_DATA" as rec_limit
from public."TABLES"
loop
if rec.rec_limit is not null then
execute format('select count(*) from %I', rec.table_name)
into rec_count;
end if;
if (rec.rec_limit is not null and rec_count >= rec.rec_limit) then
sql_query := format('select %L, to_jsonb(t) from %I as t', rec.table_name, rec.table_name);
return query execute sql_query;
end if;
end loop;
end;
$BODY$;
Some notes
the language name is an identifier and should not be enclosed in single quotes. This syntax is deprecated and might be removed in a future version so don't get used to it.
you should really avoid those dreaded quoted identifiers. They are much more trouble than they are worth it. See the Postgres wiki for details.
I have some function on PostgreSQL 9.6 returning a cursor (refcursor):
CREATE OR REPLACE FUNCTION public.test_returning_cursor()
RETURNS refcursor
IMMUTABLE
LANGUAGE plpgsql
AS $$
DECLARE
_ref refcursor = 'test_returning_cursor_ref1';
BEGIN
OPEN _ref FOR
SELECT 'a' :: text AS col1
UNION
SELECT 'b'
UNION
SELECT 'c';
RETURN _ref;
END
$$;
I need to write another function in which a temp table is created and all data from this refcursor are inserted to it. But INSERT INTO ... FETCH ALL FROM ... seems to be impossible. Such function can't be compiled:
CREATE OR REPLACE FUNCTION public.test_insert_from_cursor()
RETURNS table(col1 text)
IMMUTABLE
LANGUAGE plpgsql
AS $$
BEGIN
CREATE TEMP TABLE _temptable (
col1 text
) ON COMMIT DROP;
INSERT INTO _temptable (col1)
FETCH ALL FROM "test_returning_cursor_ref1";
RETURN QUERY
SELECT col1
FROM _temptable;
END
$$;
I know that I can use:
FOR _rec IN
FETCH ALL FROM "test_returning_cursor_ref1"
LOOP
INSERT INTO ...
END LOOP;
But is there better way?
Unfortunately, INSERT and SELECT don't have access to cursors as a whole.
To avoid expensive single-row INSERT, you could have intermediary functions with RETURNS TABLE and return the cursor as table with RETURN QUERY. See:
Return a query from a function?
CREATE OR REPLACE FUNCTION f_cursor1_to_tbl()
RETURNS TABLE (col1 text) AS
$func$
BEGIN
-- MOVE BACKWARD ALL FROM test_returning_cursor_ref1; -- optional, see below
RETURN QUERY
FETCH ALL FROM test_returning_cursor_ref1;
END
$func$ LANGUAGE plpgsql; -- not IMMUTABLE
Then create the temporary table(s) directly like:
CREATE TEMP TABLE t1 ON COMMIT DROP
AS SELECT * FROM f_cursor1_to_tbl();
See:
Creating temporary tables in SQL
Still not very elegant, but much faster than single-row INSERT.
Note: Since the source is a cursor only the first call succeeds. Executing the function a second time would return an empty set. You would need a cursor with the SCROLL option and move to the start for repeated calls.
This function does INSERT INTO from refcursor. It is universal for all the tables. The only requirement is that all columns of table corresponds to columns of refcursor by types and order (not necessary by names).
to_json() does the trick to convert any primitive data types to string with double-quotes "", which are later replaced with ''.
CREATE OR REPLACE FUNCTION public.insert_into_from_refcursor(_table_name text, _ref refcursor)
RETURNS void
LANGUAGE plpgsql
AS $$
DECLARE
_sql text;
_sql_val text = '';
_row record;
_hasvalues boolean = FALSE;
BEGIN
LOOP --for each row
FETCH _ref INTO _row;
EXIT WHEN NOT found; --there are no rows more
_hasvalues = TRUE;
SELECT _sql_val || '
(' ||
STRING_AGG(val.value :: text, ',') ||
'),'
INTO _sql_val
FROM JSON_EACH(TO_JSON(_row)) val;
END LOOP;
_sql_val = REPLACE(_sql_val, '"', '''');
_sql_val = TRIM(TRAILING ',' FROM _sql_val);
_sql = '
INSERT INTO ' || _table_name || '
VALUES ' || _sql_val;
--RAISE NOTICE 'insert_into_from_refcursor(): SQL is: %', _sql;
IF _hasvalues THEN --to avoid error when trying to insert 0 values
EXECUTE (_sql);
END IF;
END;
$$;
Usage:
CREATE TABLE public.table1 (...);
PERFORM my_func_opening_refcursor();
PERFORM public.insert_into_from_refcursor('public.table1', 'name_of_refcursor_portal'::refcursor);
where my_func_opening_refcursor() contains
DECLARE
_ref refcursor = 'name_of_refcursor_portal';
OPEN _ref FOR
SELECT ...;
I am creating trigger in PostgresSQL. On update I would like to compare all of the values in a Hstore column and update changes in my mirror table. I managed to get names of my columns in variable k but I am not able to get values using it from NEW and OLD.
CREATE OR REPLACE FUNCTION function_replication() RETURNS TRIGGER AS
$BODY$
DECLARE
k text;
BEGIN
FOR k IN SELECT key FROM EACH(hstore(NEW)) LOOP
IF NEW.k != OLD.k THEN
EXECUTE 'UPDATE ' || TG_TABLE_NAME || '_2' || 'SET ' || k || '=' || new.k || ' WHERE ID=$1.ID;' USING OLD;
END IF;
END LOOP;
RETURN NEW;
END;
$BODY$
language plpgsql;
You should operate on hstore representations of the records new and old. Also, use the format() function for better control and readibility.
create or replace function function_replication()
returns trigger as
$body$
declare
newh hstore = hstore(new);
oldh hstore = hstore(old);
key text;
begin
foreach key in array akeys(newh) loop
if newh->key != oldh->key then
execute format(
'update %s_2 set %s = %L where id = %s',
tg_table_name, key, newh->key, oldh->'id');
end if;
end loop;
return new;
end;
$body$
language plpgsql;
Another version - with minimalistic numbers of updates - in partially functional design (where it is possible).
This trigger should be AFTER trigger, to be ensured correct behave.
CREATE OR REPLACE FUNCTION function_replication()
RETURNS trigger AS $$
DECLARE
newh hstore;
oldh hstore;
update_vec text[];
pair text[];
BEGIN
IF new IS DISTINCT FROM old THEN
IF new.id <> old.id THEN
RAISE EXCEPTION 'id should be immutable';
END IF;
newh := hstore(new); oldh := hstore(old); update_vec := '{}';
FOREACH pair SLICE 1 IN ARRAY hstore_to_matrix(newh - oldh)
LOOP
update_vec := update_vec || format('%I = %L', pair[1], pair[2]);
END LOOP;
EXECUTE
format('UPDATE %I SET %s WHERE id = $1',
tg_table_name || '_2',
array_to_string(update_vec, ', '))
USING old.id;
END IF;
RETURN NEW; -- the value is not important in AFTER trg
END;
$$ LANGUAGE plpgsql;
CREATE TABLE foo(id int PRIMARY KEY, a int, b int);
CREATE TABLE foo_2(LIKE foo INCLUDING ALL);
CREATE TRIGGER xxx AFTER UPDATE ON foo
FOR EACH ROW EXECUTE PROCEDURE function_replication();
INSERT INTO foo VALUES(1, NULL, NULL);
INSERT INTO foo VALUES(2, 1,1);
INSERT INTO foo_2 VALUES(1, NULL, NULL);
INSERT INTO foo_2 VALUES(2, 1,1);
UPDATE foo SET a = 20, b = 30 WHERE id = 1;
UPDATE foo SET a = NULL WHERE id = 1;
This code is little bit more complex, but all what should be escaped is escaped and reduce number of executed UPDATE commands. UPDATE is full SQL command and the overhead of full SQL commands should be significantly higher than code that reduce number of full SQL commands.
I'm having trouble referencing record variable type columns dynamically. I found loads of tricks online, but with regards to triggers mostly and I really hope the answer isn't "it can't be done"... I've got a very specific and simple need, see my example code below;
First I have an array containing a list of column names called "lCols". I loop through a record variable to traverse my data, replacing values in a paragraph which exactly match my column names.
DECLARE lTotalRec RECORD;
DECLARE lSQL text;
DECLARE lCols varchar[];
p_paragraph:= 'I am [names] and my surname is [surname]';
lSQL :=
'select
p.names,
p.surname
from
person p
';
FOR lTotalRec IN
execute lSQL
LOOP
-- Loop through the already created array of columns to replace the values in the paragraph
FOREACH lVal IN ARRAY lCols
LOOP
p_paragraph := replace(p_paragraph,'[' || lVal || ']',lTotalRec.lVal); -- This is where my problem is, because lVal is not a column of lTotalRec directly like this
END LOOP;
RETURN NEXT;
END LOOP;
My return value is the paragraph amended for each record in "lTotalRec"
You could convert your record to a json value using the row_to_json() function. Once in this format, you can extract columns by name, using the -> and ->> operators.
In Postgres 9.4 and up, you can also make use of the more efficient jsonb type.
DECLARE lJsonRec jsonb;
...
FOR lTotalRec IN
execute lSQL
LOOP
lJsonRec := row_to_json(lTotalRec)::jsonb;
FOREACH lVal IN ARRAY lCols
LOOP
p_paragraph := replace(p_paragraph, '[' || lVal || ']', lJsonRec->>lVal);
END LOOP;
RETURN NEXT;
END LOOP;
See the documentation for more details.
You can convert a row to JSON using row_to_json(), and then retrieve the column names using json_object_keys().
Here's an example:
drop table if exists TestTable;
create table TestTable (col1 text, col2 text);
insert into TestTable values ('a1', 'b1'), ('a2', 'b2');
do $$declare
sql text;
rec jsonb;
col text;
val text;
begin
sql := 'select row_to_json(row) from (select * from TestTable) row';
for rec in execute sql loop
for col in select * from jsonb_object_keys(rec) loop
val := rec->>col;
raise notice 'col=% val=%', col, val;
end loop;
end loop;
end$$;
This prints:
NOTICE: col=col1 val=a1
NOTICE: col=col2 val=b1
NOTICE: col=col1 val=a2
NOTICE: col=col2 val=b2
DO
For example, in MS-SQL, you can open up a query window and run the following:
DECLARE #List AS VARCHAR(8)
SELECT #List = 'foobar'
SELECT *
FROM dbo.PubLists
WHERE Name = #List
How is this done in PostgreSQL? Can it be done?
Complete answer is located in the official PostgreSQL documentation.
You can use new PG9.0 anonymous code block feature (http://www.postgresql.org/docs/9.1/static/sql-do.html )
DO $$
DECLARE v_List TEXT;
BEGIN
v_List := 'foobar' ;
SELECT *
FROM dbo.PubLists
WHERE Name = v_List;
-- ...
END $$;
Also you can get the last insert id:
DO $$
DECLARE lastid bigint;
BEGIN
INSERT INTO test (name) VALUES ('Test Name')
RETURNING id INTO lastid;
SELECT * FROM test WHERE id = lastid;
END $$;
DO $$
DECLARE
a integer := 10;
b integer := 20;
c integer;
BEGIN
c := a + b;
RAISE NOTICE'Value of c: %', c;
END $$;
You can use:
\set list '''foobar'''
SELECT * FROM dbo.PubLists WHERE name = :list;
That will do
Here's an example of using a variable in plpgsql:
create table test (id int);
insert into test values (1);
insert into test values (2);
insert into test values (3);
create function test_fn() returns int as $$
declare val int := 2;
begin
return (SELECT id FROM test WHERE id = val);
end;
$$ LANGUAGE plpgsql;
SELECT * FROM test_fn();
test_fn
---------
2
Have a look at the plpgsql docs for more information.
I've came across some other documents which they use \set to declare scripting variable but the value is seems to be like constant value and I'm finding for way that can be acts like a variable not a constant variable.
Ex:
\set Comm 150
select sal, sal+:Comm from emp
Here sal is the value that is present in the table 'emp' and comm is the constant value.
Building on #nad2000's answer and #Pavel's answer here, this is where I ended up for my Flyway migration scripts. Handling for scenarios where the database schema was manually modified.
DO $$
BEGIN
IF NOT EXISTS(
SELECT TRUE FROM pg_attribute
WHERE attrelid = (
SELECT c.oid
FROM pg_class c
JOIN pg_namespace n ON n.oid = c.relnamespace
WHERE
n.nspname = CURRENT_SCHEMA()
AND c.relname = 'device_ip_lookups'
)
AND attname = 'active_date'
AND NOT attisdropped
AND attnum > 0
)
THEN
RAISE NOTICE 'ADDING COLUMN';
ALTER TABLE device_ip_lookups
ADD COLUMN active_date TIMESTAMP;
ELSE
RAISE NOTICE 'SKIPPING, COLUMN ALREADY EXISTS';
END IF;
END $$;
For use variables in for example alter table:
DO $$
DECLARE name_pk VARCHAR(200);
BEGIN
select constraint_name
from information_schema.table_constraints
where table_schema = 'schema_name'
and table_name = 'table_name'
and constraint_type = 'PRIMARY KEY' INTO name_pk;
IF (name_pk := '') THEN
EXECUTE 'ALTER TABLE schema_name.table_name DROP CONSTRAINT ' || name_pk;
Postgresql does not have bare variables, you could use a temporary table.
variables are only available in code blocks or as a user-interface feature.
If you need a bare variable you could use a temporary table:
CREATE TEMP TABLE list AS VALUES ('foobar');
SELECT dbo.PubLists.*
FROM dbo.PubLists,list
WHERE Name = list.column1;
I had to do something like this
CREATE OR REPLACE FUNCTION MYFUNC()
RETURNS VOID AS $$
DO
$do$
BEGIN
DECLARE
myvar int;
...
END
$do$
$$ LANGUAGE SQL;
You can also simply make a constant query that you use in the actual query:
WITH vars as (SELECT 'foobar' AS list)
SELECT *
FROM dbo.PubLists, vars
WHERE Name = vars.list
Given the popularity, and somewhat incomplete answers I'll provide two solutions.
A do block that won't return rows. You can return rows with a transaction cursor, but it's a bit messy.
A function (that returns rows)
Below I'll use an over-baked example of updating the tweet on the bottom right "blurb" with "hello world".
id (serial)
pub_id (text)
tweet (text)
1
abc
hello world
2
def
blurb
A simple do block
do $$
declare
src_pub_id text;
dst_pub_id text;
src_id int;
dest_id int;
src_tweet text;
begin
src_pub_id := 'abc';
dst_pub_id := 'def';
-- query result into a temp variable
src_id := (select id from tweets where pub_id = src_pub_id);
-- query result into a temp variable (another way)
select tweet into src_tweet from tweets where id = src_id;
dest_id := (select id from tweets where pub_id = dst_pub_id);
update tweets set tweet=src_tweet where id = dest_id;
end $$ language plpgsql; -- need the language to avoid ERROR 42P13
A function
create or replace function sync_tweets(
src_pub_id text, -- function arguments
dst_pub_id text
) returns setof tweets as -- i.e. rows. int, text work too
$$
declare
src_id int; -- temp function variables (not args)
dest_id int;
src_tweet text;
begin
-- query result into a temp variable
src_id := (select id from tweets where pub_id = src_pub_id);
-- query result into a temp variable (another way)
select tweet into src_tweet from tweets where id = src_id;
dest_id := (select id from tweets where pub_id = dst_pub_id);
update tweets set tweet=src_tweet where id = dest_id;
return query -- i.e. rows, return 0 with return int above works too
select * from tweets where pub_id in (src_pub_id, dst_pub_id);
end
$$ language plpgsql; -- need the language to avoid ERROR 42P13
-- Run it!
select * from sync_tweets('abc', 'def');
-- Optional drop if you don't want the db to keep your function
drop function if exists sync_tweets(text, text);
/*
Outputs
__________________________________________________
| id (serial) | pub_id (text) | tweet (text) |
|---------------|-----------------|----------------|
| 1 | abc | hello world |
| 2 | def | blurb |
--------------------------------------------------
*/