Is it possible to use unbound cursors to fully edit and replace a row in a table?
I'm using unbound cursors since the table is dynamically specified with a parameter, but I can't use the "UPDATE table SET column = value WHERE" syntax since the columns are unspecified.
CREATE OR REPLACE FUNCTION trim_table(in_table TEXT) AS $$
DECLARE
ref REFCURSOR;
current_row RECORD;
BEGIN
OPEN ref FOR EXECUTE 'SELECT * FROM '|| quote_ident(in_table);
LOOP
FETCH ref INTO current_row;
EXIT WHEN NOT FOUND;
current_row = my_row_function(current_row);
/*How can I replace my row here?*/
END LOOP;
CLOSE ref;
END
$$ LANGUAGE plpgsql;
All the example and answers I found show only how to update a single field and not the full record.
I think this code can help you in some ways :
select
string_agg('UPDATE '||table_schema||'.'||table_name||chr(13)||' SET '||column_name||' = TRIM('||column_name||')', '; '||chr(13)) into query
from information_schema.columns
where data_type in ('varchar', 'text')
and table_schema = 'your_schema'
and table_name = 'your_table_name';
execute query;
Put it in your procedure, modify it to your convenience, and you will no longer need this loop.
Related
I have a sample stored procedure where in I have to use a table for multiple operations. I want to declare the table name as a constant and then re-use it wherever required. Below is the sample code which i wrote:
CREATE OR REPLACE FUNCTION get_data()
RETURNS void AS
$func$
DECLARE
table_name_a CONSTANT TEXT = asp.monitoring_bookmark_original;
cursor_file CURSOR FOR
select distinct filename,systemuid from table_name_a;
cursor_data CURSOR FOR
select * from table_name_a where filename = v_filename and systemuid=v_systemuid order by mindatetime, maxdatetime;
BEGIN
--open the file cursor
//logic goes here
END;
$func$
LANGUAGE plpgsql;
When I try to run this procedure I am getting error:
ERROR: missing FROM-clause entry for table "asp"
LINE 1: SELECT asp.monitoring_bookmark_original
What is wrong in this code? How do I correct this?
Well you can use dynamic SQL, but realize dynamic SQL often adds way more complexity. Good when really needed but should be avoided when possible. The following shows what would be needed for what you want to do. Is not having to type the table name for each SQL statement worth the additional trouble?
create or replace function get_data()
returns void as
$func$
declare
table_name_a constant text = 'asp.monitoring_bookmark_original';
file_cursor text = 'select distinct filename,systemuid from %i';
file_ref refcursor;
file_rec record;
data_cursor text =$stmt$select * from %i where filename = '%s' and systemuid= '%s' order by mindatetime, maxdatetime$stmt$;
data_ref refcursor;
data_rec record;
begin
--open the file cursor
open file_ref for execute format(file_cursor,table_name_a);
loop
fetch next from file_ref into file_rec;
exit when not found;
-- and extending from what the second query inplies
open data_ref for execute format(data_cursor,table_name_a,file_rec.filename,file_rec.systemid);
loop
fetch next from data_ref into data_rec;
exit when not found;
--//logic goes here
end loop;
end loop ;
end;
$func$
language plpgsql;
I have some function on PostgreSQL 9.6 returning a cursor (refcursor):
CREATE OR REPLACE FUNCTION public.test_returning_cursor()
RETURNS refcursor
IMMUTABLE
LANGUAGE plpgsql
AS $$
DECLARE
_ref refcursor = 'test_returning_cursor_ref1';
BEGIN
OPEN _ref FOR
SELECT 'a' :: text AS col1
UNION
SELECT 'b'
UNION
SELECT 'c';
RETURN _ref;
END
$$;
I need to write another function in which a temp table is created and all data from this refcursor are inserted to it. But INSERT INTO ... FETCH ALL FROM ... seems to be impossible. Such function can't be compiled:
CREATE OR REPLACE FUNCTION public.test_insert_from_cursor()
RETURNS table(col1 text)
IMMUTABLE
LANGUAGE plpgsql
AS $$
BEGIN
CREATE TEMP TABLE _temptable (
col1 text
) ON COMMIT DROP;
INSERT INTO _temptable (col1)
FETCH ALL FROM "test_returning_cursor_ref1";
RETURN QUERY
SELECT col1
FROM _temptable;
END
$$;
I know that I can use:
FOR _rec IN
FETCH ALL FROM "test_returning_cursor_ref1"
LOOP
INSERT INTO ...
END LOOP;
But is there better way?
Unfortunately, INSERT and SELECT don't have access to cursors as a whole.
To avoid expensive single-row INSERT, you could have intermediary functions with RETURNS TABLE and return the cursor as table with RETURN QUERY. See:
Return a query from a function?
CREATE OR REPLACE FUNCTION f_cursor1_to_tbl()
RETURNS TABLE (col1 text) AS
$func$
BEGIN
-- MOVE BACKWARD ALL FROM test_returning_cursor_ref1; -- optional, see below
RETURN QUERY
FETCH ALL FROM test_returning_cursor_ref1;
END
$func$ LANGUAGE plpgsql; -- not IMMUTABLE
Then create the temporary table(s) directly like:
CREATE TEMP TABLE t1 ON COMMIT DROP
AS SELECT * FROM f_cursor1_to_tbl();
See:
Creating temporary tables in SQL
Still not very elegant, but much faster than single-row INSERT.
Note: Since the source is a cursor only the first call succeeds. Executing the function a second time would return an empty set. You would need a cursor with the SCROLL option and move to the start for repeated calls.
This function does INSERT INTO from refcursor. It is universal for all the tables. The only requirement is that all columns of table corresponds to columns of refcursor by types and order (not necessary by names).
to_json() does the trick to convert any primitive data types to string with double-quotes "", which are later replaced with ''.
CREATE OR REPLACE FUNCTION public.insert_into_from_refcursor(_table_name text, _ref refcursor)
RETURNS void
LANGUAGE plpgsql
AS $$
DECLARE
_sql text;
_sql_val text = '';
_row record;
_hasvalues boolean = FALSE;
BEGIN
LOOP --for each row
FETCH _ref INTO _row;
EXIT WHEN NOT found; --there are no rows more
_hasvalues = TRUE;
SELECT _sql_val || '
(' ||
STRING_AGG(val.value :: text, ',') ||
'),'
INTO _sql_val
FROM JSON_EACH(TO_JSON(_row)) val;
END LOOP;
_sql_val = REPLACE(_sql_val, '"', '''');
_sql_val = TRIM(TRAILING ',' FROM _sql_val);
_sql = '
INSERT INTO ' || _table_name || '
VALUES ' || _sql_val;
--RAISE NOTICE 'insert_into_from_refcursor(): SQL is: %', _sql;
IF _hasvalues THEN --to avoid error when trying to insert 0 values
EXECUTE (_sql);
END IF;
END;
$$;
Usage:
CREATE TABLE public.table1 (...);
PERFORM my_func_opening_refcursor();
PERFORM public.insert_into_from_refcursor('public.table1', 'name_of_refcursor_portal'::refcursor);
where my_func_opening_refcursor() contains
DECLARE
_ref refcursor = 'name_of_refcursor_portal';
OPEN _ref FOR
SELECT ...;
I have an insert trigger function in which NEW.schema_name references a schema. I want to dynamically copy the tables found inside that schema ('foobaz','barbaz') as 'foo' and 'bar'. I then can perform queries without dynamic sql.
How can I create a function or simply copy/paste the same block of code to achive that.
EDIT :
I cannot get that dynamic query to work.
The part inside the WITH statement is working.
Not the bottom 'execute' part. I do not know if it is a syntax problem, or bad cast or whatever constraint there is in pgsql that makes it not working.
WITH info_schema_subset_table as (SELECT table_schema, table_name,
array_to_string((regexp_split_to_array(table_name,'_'))[4:array_length(regexp_split_to_array(table_name,'_'),1)-1] as new_table
FROM information_schema.tables
where table_schema = "schema_searched"
ORDER BY new_table ASC)
EXECUTE 'CREATE TABLE $2 as (SELECT * FROM $1)'
USING info_schema_subset_table.table_schema || '.' ||info_schema_subset_table.table_name,info_schema_subset_table.new_table;
EDIT 2
... Broken code removed...
In the code below, in which I'm unsure if the syntax is right, I get the following from the trigger
Provider errors:
PostGIS error while adding features: ERREUR: l'opérateur n'existe pas : record ~~ unknown
LINE 1: SELECT old_table LIKE '%ens%'
^
HINT: Aucun opérateur ne correspond au nom donné et aux types d'arguments.
Vous devez ajouter des conversions explicites de type.
QUERY: SELECT old_table LIKE '%ens%'
CONTEXT: fonction PL/pgsql validation_sio.afi_validation_sio(), ligne 18 à CASE
EDIT 3 :
CREATE OR REPLACE FUNCTION foo.foo()
RETURNS TRIGGER AS
$BODY$
DECLARE
old_table record;
new_table record;
dynamic_query text;
BEGIN
IF TG_OP = 'INSERT'
THEN
FOR old_table IN SELECT table_schema|| '.' ||table_name
FROM information_schema.tables
where table_schema = NEW.nom_schema
LOOP
CASE
WHEN
old_table LIKE '%ens%' THEN
new_table := concat('SIT_',array_to_string((regexp_split_to_array(info_schema.old_table,'_'))[4:array_length(regexp_split_to_array(info_schema.old_table,'_'),1)-1],'_'));
ELSE
new_table := concat('SID_',array_to_string((regexp_split_to_array(info_schema.old_table,'_'))[4:array_length(regexp_split_to_array(info_schema.old_table,'_'),1)-1],'_'));
END CASE;
dynamic_query := format('SELECT * FROM' || old_table ||);
EXECUTE dynamic_query
INTO new_table;
END LOOP;
RETURN NEW;
END IF;
END;
$BODY$
LANGUAGE plpgsql VOLATILE;
CREATE TRIGGER foo
AFTER INSERT ON validation.validationfoo
FOR EACH ROW EXECUTE PROCEDURE foo.foo();
I've reformatted your trigger function a bit and changed a few things, see if this works.
CREATE OR REPLACE FUNCTION foo.foo()
RETURNS TRIGGER AS
$BODY$
DECLARE
old_table record;
new_table record;
dynamic_query text;
BEGIN
IF TG_OP = 'INSERT' THEN
FOR old_table IN
SELECT table_schema || '.' || table_name AS old_table_name
FROM information_schema.tables
WHERE table_schema = NEW.nom_schema
LOOP
new_table := concat(CASE WHEN old_table.old_table_name LIKE '%ens%' THEN 'SIT_' ELSE 'SID_' END,array_to_string((regexp_split_to_array(info_schema.old_table,'_'))[4:array_length(regexp_split_to_array(info_schema.old_table,'_'),1)-1],'_'));
dynamic_query := 'CREATE TABLE ' || new_table || ' AS SELECT * FROM ' || old_table.old_table_name;
EXECUTE dynamic_query;
END LOOP;
RETURN NEW;
END IF;
END;
$BODY$
LANGUAGE plpgsql VOLATILE;
So the main things:
old_table is a record, so your comparison of it to a string with LIKE was failing. You need to use the field name. So I gave your field a name, and used that field name in the LIKE comparison.
Changed the new_table assignment to put the CASE statement only on the one item that changes, to make the difference more obvious and the code more concise. Mind you, I don't know if the rest of that line is actually valid, I just left it as is.
Changed the creation dynamic_query. As I said in the comment, the format function was being used incorrectly, so I just went with standard string concatenation instead.
Changed dynamic_query's SQL to what I think you actually want it to do. You want it to copy the content of the table to a new table, right? So that will do it.
You cannot have EXECUTE inside an SQL statement, it is a PL/pgSQL statement.
Loop through the tables and issue one EXECUTE for each.
Mind that you cannot have a schema or table name as a parameter with USING, because these names need to be known at parse time.
Use the format function to construct your dynamic statement so you can avoid SQL injection by users who maliciously create tables with weird names.
I am adjusting some PL/pgSQL code so my refcursor can take the table name as parameter. Therefore I changed the following line:
declare
pointCurs CURSOR FOR SELECT * from tableName for update;
with this one:
OPEN pointCurs FOR execute 'SELECT * FROM ' || quote_ident(tableName) for update;
I adjusted the loop, and voilà, the loop went through. Now at some point in the loop I needed to update the record (pointed by the cursor) and I got stuck. How should I properly adjust the following line of code?
UPDATE tableName set tp_id = pos where current of pointCurs;
I fixed the quotes for the tableName and pos and added the EXECUTE clause at the beginning, but I get the error on the where current of pointCurs.
Questions:
How can I update the record?
The function was working properly for tables from the public schema and failed for tables from other schemas (e.g., trace.myname).
Any comments are highly appreciated..
Answer for (i)
1. Explicit (unbound) cursor
EXECUTE is not a "clause", but a PL/pgSQL command to execute SQL strings. Cursors are not visible inside the command. You need to pass values to it.
Hence, you cannot use the special syntax WHERE CURRENT OFcursor. I use the system column ctid instead to determine the row without knowing the name of a unique column. Note that ctid is only guaranteed to be stable within the same transaction.
CREATE OR REPLACE FUNCTION f_curs1(_tbl text)
RETURNS void AS
$func$
DECLARE
_curs refcursor;
rec record;
BEGIN
OPEN _curs FOR EXECUTE 'SELECT * FROM ' || quote_ident(_tbl) FOR UPDATE;
LOOP
FETCH NEXT FROM _curs INTO rec;
EXIT WHEN rec IS NULL;
RAISE NOTICE '%', rec.tbl_id;
EXECUTE format('UPDATE %I SET tbl_id = tbl_id + 10 WHERE ctid = $1', _tbl)
USING rec.ctid;
END LOOP;
END
$func$ LANGUAGE plpgsql;
Why format() with %I?
There is also a variant of the FOR statement to loop through cursors, but it only works for bound cursors. We have to use an unbound cursor here.
2. Implicit cursor in FOR loop
There is normally no need for explicit cursors in plpgsql. Use the implicit cursor of a FOR loop instead:
CREATE OR REPLACE FUNCTION f_curs2(_tbl text)
RETURNS void AS
$func$
DECLARE
_ctid tid;
BEGIN
FOR _ctid IN EXECUTE 'SELECT ctid FROM ' || quote_ident(_tbl) FOR UPDATE
LOOP
EXECUTE format('UPDATE %I SET tbl_id = tbl_id + 100 WHERE ctid = $1', _tbl)
USING _ctid;
END LOOP;
END
$func$ LANGUAGE plpgsql;
3. Set based approach
Or better, yet (if possible!): Rethink your problem in terms of set-based operations and execute a single (dynamic) SQL command:
-- Set-base dynamic SQL
CREATE OR REPLACE FUNCTION f_nocurs(_tbl text)
RETURNS void AS
$func$
BEGIN
EXECUTE format('UPDATE %I SET tbl_id = tbl_id + 1000', _tbl);
-- add WHERE clause as needed
END
$func$ LANGUAGE plpgsql;
SQL Fiddle demonstrating all 3 variants.
Answer for (ii)
A schema-qualified table name like trace.myname actually consists of two identifiers. You have to
either pass and escape them separately,
or go with the more elegant approach of using a regclass type:
CREATE OR REPLACE FUNCTION f_nocurs(_tbl regclass)
RETURNS void AS
$func$
BEGIN
EXECUTE format('UPDATE %s SET tbl_id = tbl_id + 1000', _tbl);
END
$func$ LANGUAGE plpgsql;
I switched from %I to %s, because the regclass parameter is automatically properly escaped when (automatically) converted to text.
More details in this related answer:
Table name as a PostgreSQL function parameter
I'm trying to create a function to get a field value from multiple tables in my database. I made script like this:
CREATE OR REPLACE FUNCTION get_all_changes() RETURNS SETOF RECORD AS
$$
DECLARE
tblname VARCHAR;
tblrow RECORD;
row RECORD;
BEGIN
FOR tblrow IN SELECT tablename FROM pg_catalog.pg_tables WHERE schemaname='public' LOOP /*FOREACH tblname IN ARRAY $1 LOOP*/
RAISE NOTICE 'r: %', tblrow.tablename;
FOR row IN SELECT MAX("lastUpdate") FROM tblrow.tablename LOOP
RETURN NEXT row;
END LOOP;
END LOOP;
END
$$
LANGUAGE 'plpgsql' ;
SELECT get_all_changes();
But it is not working, everytime it shows this error
tblrow.tablename" not defined in line "FOR row IN SELECT MAX("lastUpdate") FROM tblrow.tablename LOOP"
Your inner FOR loop must use the FOR...EXECUTE syntax as shown in the manual:
FOR target IN EXECUTE text_expression [ USING expression [, ... ] ] LOOP
statements
END LOOP [ label ];
In your case something along this line:
FOR row IN EXECUTE 'SELECT MAX("lastUpdate") FROM ' || quote_ident(tblrow.tablename) LOOP
RETURN NEXT row;
END LOOP
The reason for this is explained in the manual somewhere else:
Oftentimes you will want to generate dynamic commands inside your PL/pgSQL functions, that is, commands that will involve different tables or different data types each time they are executed. PL/pgSQL's normal attempts to cache plans for commands (as discussed in Section 39.10.2) will not work in such scenarios. To handle this sort of problem, the EXECUTE statement is provided[...]
Answer to your new question (mislabeled as answer):
This can be much simpler. You do not need to create a table just do define a record type.
If at all, you would better create a type with CREATE TYPE, but that's only efficient if you need the type in multiple places. For just a single function, you can use RETURNS TABLE instead :
CREATE OR REPLACE FUNCTION get_all_changes(text[])
RETURNS TABLE (tablename text
,"lastUpdate" timestamp with time zone
,nums integer) AS
$func$
DECLARE
tblname text;
BEGIN
FOREACH tblname IN ARRAY $1 LOOP
RETURN QUERY EXECUTE format(
$f$SELECT '%I', MAX("lastUpdate"), COUNT(*)::int FROM %1$I
$f$, tblname)
END LOOP;
END
$func$ LANGUAGE plpgsql;
A couple more points:
Use RETURN QUERY EXECUTE instead of the nested loop. Much simpler and faster.
Column aliases would only serve as documentation, those names are discarded in favor of the names declared in the RETURNS clause (directly or indirectly).
Use format() with %I to replace the concatenation with quote_ident() and %1$I to refer to the same parameter another time.
count() usually returns type bigint. Cast the integer, since you defined the column in the return type as such: count(*)::int.
Thanks,
I finally made my script like:
CREATE TABLE IF NOT EXISTS __rsdb_changes (tablename text,"lastUpdate" timestamp with time zone, nums bigint);
CREATE OR REPLACE FUNCTION get_all_changes(varchar[]) RETURNS SETOF __rsdb_changes AS /*TABLE (tablename varchar(40),"lastUpdate" timestamp with time zone, nums integer)*/
$$
DECLARE
tblname VARCHAR;
tblrow RECORD;
row RECORD;
BEGIN
FOREACH tblname IN ARRAY $1 LOOP
/*RAISE NOTICE 'r: %', tblrow.tablename;*/
FOR row IN EXECUTE 'SELECT CONCAT('''|| quote_ident(tblname) ||''') AS tablename, MAX("lastUpdate") AS "lastUpdate",COUNT(*) AS nums FROM ' || quote_ident(tblname) LOOP
/*RAISE NOTICE 'row.tablename: %',row.tablename;*/
/*RAISE NOTICE 'row.lastUpdate: %',row."lastUpdate";*/
/*RAISE NOTICE 'row.nums: %',row.nums;*/
RETURN NEXT row;
END LOOP;
END LOOP;
RETURN;
END
$$
LANGUAGE 'plpgsql' ;
Well, it works. But it seems I can only create a table to define the return structure instead of just RETURNS SETOF RECORD. Am I right?
Thanks again.