"INSERT INTO ... FETCH ALL FROM ..." can't be compiled - postgresql

I have some function on PostgreSQL 9.6 returning a cursor (refcursor):
CREATE OR REPLACE FUNCTION public.test_returning_cursor()
RETURNS refcursor
IMMUTABLE
LANGUAGE plpgsql
AS $$
DECLARE
_ref refcursor = 'test_returning_cursor_ref1';
BEGIN
OPEN _ref FOR
SELECT 'a' :: text AS col1
UNION
SELECT 'b'
UNION
SELECT 'c';
RETURN _ref;
END
$$;
I need to write another function in which a temp table is created and all data from this refcursor are inserted to it. But INSERT INTO ... FETCH ALL FROM ... seems to be impossible. Such function can't be compiled:
CREATE OR REPLACE FUNCTION public.test_insert_from_cursor()
RETURNS table(col1 text)
IMMUTABLE
LANGUAGE plpgsql
AS $$
BEGIN
CREATE TEMP TABLE _temptable (
col1 text
) ON COMMIT DROP;
INSERT INTO _temptable (col1)
FETCH ALL FROM "test_returning_cursor_ref1";
RETURN QUERY
SELECT col1
FROM _temptable;
END
$$;
I know that I can use:
FOR _rec IN
FETCH ALL FROM "test_returning_cursor_ref1"
LOOP
INSERT INTO ...
END LOOP;
But is there better way?

Unfortunately, INSERT and SELECT don't have access to cursors as a whole.
To avoid expensive single-row INSERT, you could have intermediary functions with RETURNS TABLE and return the cursor as table with RETURN QUERY. See:
Return a query from a function?
CREATE OR REPLACE FUNCTION f_cursor1_to_tbl()
RETURNS TABLE (col1 text) AS
$func$
BEGIN
-- MOVE BACKWARD ALL FROM test_returning_cursor_ref1; -- optional, see below
RETURN QUERY
FETCH ALL FROM test_returning_cursor_ref1;
END
$func$ LANGUAGE plpgsql; -- not IMMUTABLE
Then create the temporary table(s) directly like:
CREATE TEMP TABLE t1 ON COMMIT DROP
AS SELECT * FROM f_cursor1_to_tbl();
See:
Creating temporary tables in SQL
Still not very elegant, but much faster than single-row INSERT.
Note: Since the source is a cursor only the first call succeeds. Executing the function a second time would return an empty set. You would need a cursor with the SCROLL option and move to the start for repeated calls.

This function does INSERT INTO from refcursor. It is universal for all the tables. The only requirement is that all columns of table corresponds to columns of refcursor by types and order (not necessary by names).
to_json() does the trick to convert any primitive data types to string with double-quotes "", which are later replaced with ''.
CREATE OR REPLACE FUNCTION public.insert_into_from_refcursor(_table_name text, _ref refcursor)
RETURNS void
LANGUAGE plpgsql
AS $$
DECLARE
_sql text;
_sql_val text = '';
_row record;
_hasvalues boolean = FALSE;
BEGIN
LOOP --for each row
FETCH _ref INTO _row;
EXIT WHEN NOT found; --there are no rows more
_hasvalues = TRUE;
SELECT _sql_val || '
(' ||
STRING_AGG(val.value :: text, ',') ||
'),'
INTO _sql_val
FROM JSON_EACH(TO_JSON(_row)) val;
END LOOP;
_sql_val = REPLACE(_sql_val, '"', '''');
_sql_val = TRIM(TRAILING ',' FROM _sql_val);
_sql = '
INSERT INTO ' || _table_name || '
VALUES ' || _sql_val;
--RAISE NOTICE 'insert_into_from_refcursor(): SQL is: %', _sql;
IF _hasvalues THEN --to avoid error when trying to insert 0 values
EXECUTE (_sql);
END IF;
END;
$$;
Usage:
CREATE TABLE public.table1 (...);
PERFORM my_func_opening_refcursor();
PERFORM public.insert_into_from_refcursor('public.table1', 'name_of_refcursor_portal'::refcursor);
where my_func_opening_refcursor() contains
DECLARE
_ref refcursor = 'name_of_refcursor_portal';
OPEN _ref FOR
SELECT ...;

Related

postgresql for loop script in text form can not be executed

I am trying to write function in postgresql, that creates temp_table with columns table_name text, table_rec jsonb and fill it through for loop with table names from my table containing names of tables and records in json. I have the for loop in string and I want to execute it. But it doesnt work.
I have variable rec record, sql_query text and tab_name text and I want to do this:
CREATE OR REPLACE FUNCTION public.test51(
)
RETURNS TABLE(tabel_name text, record_json jsonb)
LANGUAGE 'plpgsql'
COST 100
VOLATILE
ROWS 1000
AS $BODY$
declare
rec record;
tabel_name text;
tabel_names text[];
counter integer := 1;
sql_query text;
limit_for_sending integer;
rec_count integer;
begin
select into tabel_names array(select "TABLE_NAME" from public."TABLES");
create temp table temp_tab(tab_nam text, recik jsonb);
while array_length(tabel_names, 1) >= counter loop
tabel_name := '"' || tabel_names[counter] || '"';
select into limit_for_sending "TABLE_LIMIT_FOR_SENDING_DATA" from public."TABLES" where "TABLE_NAME" = tabel_name;
sql_query := 'select count(*) from public.' || tabel_name;
execute sql_query into rec_count;
if (rec_count >= limit_for_sending and limit_for_sending is not null) then
sql_query := 'for rec in select * from public.' || tabel_name || '
loop
insert into temp_tab
select ' || tabel_name || ', to_jsonb(rec);
end loop';
execute sql_query;
end if;
counter := counter + 1;
end loop;
return query
select * from temp_tabik;
drop table temp_tabik;
end;
$BODY$;
Thank you for response.
It seems you have some table that contains the information for which tables you want to return all rows as JSONB. And that meta-table also contains a column that sets a threshold under which the rows should not be returned.
You don't need the temp table or an array to store the table names. You can iterate through the query on the TABLES table and run the dynamic SQL directly in that loop.
return query in PL/pgSQL doesn't terminate the function, it just appends the result of the query to the result of the function.
Dynamic SQL is best created using the format() function because it is easier to read and using the %I placeholder will properly deal with quoted identifiers (which is really important as you are using those dreaded upper case table names)
As far as I can tell, your function can be simplified to:
CREATE OR REPLACE FUNCTION public.test51()
RETURNS TABLE(tabel_name text, record_json jsonb)
LANGUAGE plpgsql
AS
$BODY$
declare
rec record;
sql_query text;
rec_count bigint;
begin
for rec in
select "TABLE_NAME" as table_name, "TABLE_LIMIT_FOR_SENDING_DATA" as rec_limit
from public."TABLES"
loop
if rec.rec_limit is not null then
execute format('select count(*) from %I', rec.table_name)
into rec_count;
end if;
if (rec.rec_limit is not null and rec_count >= rec.rec_limit) then
sql_query := format('select %L, to_jsonb(t) from %I as t', rec.table_name, rec.table_name);
return query execute sql_query;
end if;
end loop;
end;
$BODY$;
Some notes
the language name is an identifier and should not be enclosed in single quotes. This syntax is deprecated and might be removed in a future version so don't get used to it.
you should really avoid those dreaded quoted identifiers. They are much more trouble than they are worth it. See the Postgres wiki for details.

Execute decode function stored in bytea column

I have a bytea column in a table that contains a function decode(). What I have done to get the actual data is as follows:
select filename, convert_from(data,'UTF-8') from attachments limit 20; //this returns me decode function
select decode(E'...','hex'); // I am executing the above returned function
The above is fine as long as I have to select one row. But now my requirement is to get more than one result. How can I get the result in single query? I have tried using pl/pgsql
CREATE OR REPLACE FUNCTION get_data(integer, _type anyelement, OUT _result anyelement)
AS
$x$
BEGIN
EXECUTE
'SELECT ' || (select convert_from(data,'UTF-8') as data from attachments limit $1)
INTO _result;
END;
$x$
LANGUAGE plpgsql;
But this works only for single row and single column. What I want is a single query to fetch 2 columns without using pl/pgsql if possible. I am using this query from my Java based web app.
Thanks!
You need procedural code for this, since there is no provision for dynamic statements in SQL.
The following function converts all attachments:
CREATE FUNCTION getemall(
IN v_type anyelement,
OUT v_result anyelement
) RETURNS SETOF anyelement
LANGUAGE plpgsql AS
$$DECLARE
v_stmt text;
BEGIN
FOR v_stmt IN
SELECT convert_from(data,'UTF-8')
FROM attachments
LOOP
EXECUTE v_stmt INTO v_result;
RETURN NEXT;
END LOOP;
END;$$;
This is how I have written the function with few changes
CREATE OR REPLACE FUNCTION getmeall(tName text, fNameCol text, dataCol text,fSize
numeric)
RETURNS TABLE(bdata bytea, fname text) LANGUAGE plpgsql AS
$$DECLARE
v_stmt text;
v_name text;
BEGIN
FOR v_stmt,v_name IN
EXECUTE format('SELECT encode(%s, ''escape''), %s FROM %s
WHERE $1 IS NOT NULL AND $2 IS NOT NULL LIMIT $3'
, dataCol, fNameCol, tName)
USING dataCol, fNameCol, fSize
LOOP
fname:=v_name;
IF strpos(v_stmt,'decode') = 1 THEN
EXECUTE 'SELECT ' || v_stmt INTO bdata;
ELSE
bdata:=v_stmt;
END IF;
RETURN NEXT;
END LOOP;
END;$$;
And finally calling it this way.
select * from getmeall('attachments', '"filename"', '"data"',2)

How to pass NEW.* to EXECUTE in trigger function

I have a simple mission is inserting huge MD5 values into tables (partitioned table), and have created a trigger and also a trigger function to instead of INSERT operation. And in function I checked the first two characters of NEW.md5 to determine which table should be inserted.
DECLARE
tb text;
BEGIN
IF TG_OP = 'INSERT' THEN
tb = 'samples_' || left(NEW.md5, 2);
EXECUTE(format('INSERT INTO %s VALUES (%s);', tb, NEW.*)); <- WRONG
END IF;
RETURN NULL;
END;
The question is how to concat the NEW.* into the SQL statement?
Best with the USING clause of EXECUTE:
CREATE FUNCTION foo ()
RETURNS trigger AS
$func$
BEGIN
IF TG_OP = 'INSERT' THEN
EXECUTE format('INSERT INTO %s SELECT $1.*'
, 'samples_' || left(NEW.md5, 2);
USING NEW;
END IF;
RETURN NULL;
END
$func$ LANGUAGE plpgsql;
And EXECUTE does not require parentheses.
And you are aware that identifiers are folded to lower case unless quoted where necessary (%I instead of %s in format()).
More details:
INSERT with dynamic table name in trigger function
How to dynamically use TG_TABLE_NAME in PostgreSQL 8.2?

Select a column default value into a variable in Pl/PgSQL

I'm trying to implement a generic trigger procedure to enable a sort a versioning scheme on tables. Tables all have version and current fields. On updates, in some situations based on a condition, i want to create a new version of a row instead of updating the old one. I'm having trouble getting the default value for the primary key field (always id).
Here's what i've done:
CREATE FUNCTION version_trigger() RETURNS trigger AS $$
DECLARE
id_default text;
id_value text;
BEGIN
IF version_condition() THEN
old.current = false;
-- I can read the default value
SELECT column_default INTO id_default
FROM information_schema.columns
WHERE table_name = TG_TABLE_NAME AND column_name = 'id';
-- THIS DOESN'T WORK!
EXECUTE 'SELECT $1' INTO id_value USING id_default;
new.id = id_value;
EXECUTE 'INSERT INTO ' || quote_ident(TG_TABLE_NAME) || ' SELECT ($1).*' USING new;
RETURN old;
END IF;
-- regular UPDATE
RETURN new;
END
$$ LANGUAGE plpgsql;
I'm just missing the step where i read the default value for the id (it's just a nextval() call). Can anyone help out on this?
Thanks in advance!
You cannot use a placeholder for expressions.
If DEFAULT doesn't have any reference to record data, then you can use EXECUTE statement, but little bit different
postgres=# DO $$
DECLARE x text; y text;
BEGIN
x := (SELECT column_default
FROM information_schema.columns
WHERE table_name = 'omega' AND column_name = 'a');
EXECUTE 'SELECT ' || x INTO y;
RAISE NOTICE '%', y;
END;
$$;
NOTICE: 2
DO

Postgres how to evaluate expression from query to variables in the function

I would like to be able to get values of function variables whose names are queried from a table
Edited to show querying a table instead of query from static values:
create table __test__
(
_col text
);
insert into __test__
(_col)
values('_a');
create or replace function __test()
returns void
language 'plpgsql' as
$$
declare
_r record;
_a int;
_b int;
_sql text;
begin
_a = 1;
_b = 0;
for _r in select _col as _nam from __test__ a loop
-- query returns one row valued "_a"
_sql = 'select ' || _r._nam ;
execute _sql into _b;
end loop;
raise info 'value of _b %', _b;
end;
$$;
select __test()
when function executes so that _b = 1. Is it possible?
same error ...
ERROR: column "_a" does not exist
LINE 1: select _a
^
QUERY: select _a
CONTEXT: PL/pgSQL function "__test" line 15 at EXECUTE statement
You could create a temporary table, insert your variable names and values in it, and then execute a select against that. Just clean up after. I have used approaches like that before. It works ok. It does have extra overhead though.
Edit: adding an example
CREATE FUNCTION switch (in_var text) RETURNS text
LANGUAGE PLPGSQL VOLATILE AS $$
declare t_test text;
switch_vals text[];
BEGIN
CREATE TEMPORARY TABLE switch_values (var text, value text);
EXECUTE $e$ INSERT INTO switch_values VALUES
('a', '1'), ('b', '2'), ('c', '3') $e$;
EXECUTE $e$ SELECT value FROM switch_values WHERE var = $e$ || quote_literal(in_var)
INTO t_test;
DROP TABLE switch_values;
RETURN t_test;
END; $$;
postgres=# select switch('a');
switch
--------
1
(1 row)
Let's try to reframe the question: what you're after would be the equivalent of Perl eval()
function, with its ability to execute a dynamically generated piece of code for which "any outer lexical variables are visible to it". In your example, the variable would be _a, but as you can see from the error message, it can't be interpolated by a dynamic SQL statement. The reason is that the SQL interpreter has no visibility on the current pl/pgsql variables, or even the knowledge that such variables exist. They are confined to pl/pgsql.
What would be needed here is a context-aware dynamically-generated pl/pgsql statement, but this language does not have this feature. It's doubtful that a trick could be found to achieve the result without this feature. For all its ability to interface nicely with SQL, other than that it's a fairly static language.
On the other hand, this would be no problem for pl/perl.