Passing table names in an array - postgresql

I need to do the same deletion or purge operation (based on several conditions) on a set of tables. For that I am trying to pass the table names in an array to a function. I am not sure if I am doing it right. Or is there a better way?
I am pasting just a sample example this is not the real function I have written but the basic is same as below:
CREATE OR REPLACE FUNCTION test (tablename text[]) RETURNS int AS
$func$
BEGIN
execute 'delete * from '||tablename;
RETURN 1;
END
$func$ LANGUAGE plpgsql;
But when I call the function I get an error:
select test( {'rajeev1'} );
ERROR: syntax error at or near "{"
LINE 10: select test( {'rajeev1'} );
^
********** Error **********
ERROR: syntax error at or near "{"
SQL state: 42601
Character: 179

Array syntax
'{rajeev1, rajeev2}' or ARRAY['rajeev1', 'rajeev2']. Read the manual.
TRUNCATE
Since you are deleting all rows from the tables, consider TRUNCATE instead. Per documentation:
Tip: TRUNCATE is a PostgreSQL extension that provides a faster
mechanism to remove all rows from a table.
Be sure to study the details. If TRUNCATE works for you, the whole operation becomes very simple, since the command accepts multiple tables:
TRUNCATE rajeev1, rajeev2, rajeev3, ..
Dynamic DELETE
Else you need dynamic SQL like you already tried. The scary missing detail: you are completely open to SQL injection and catastrophic syntax errors. Use format() with %I (not %s to sanitize identifiers like table names. Or, better yet in this particular case, use an array of regclass as parameter instead:
CREATE OR REPLACE FUNCTION f_del_all(_tbls regclass)
RETURNS void AS
$func$
DECLARE
_tbl regclass;
BEGIN
FOREACH _tbl IN ARRAY _tbls LOOP
EXECUTE format('DELETE * FROM %s', _tbl);
END LOOP;
END
$func$ LANGUAGE plpgsql;
Call:
SELECT f_del_all('{rajeev1,rajeev2,rajeev3}');
Explanation here:
Table name as a PostgreSQL function parameter

You used wrong syntax for text array constant in the function call. But even if it was right, your function is not correct.
If your function has text array as argument you should loop over the array to execute query for each element.
CREATE OR REPLACE FUNCTION test (tablenames text[]) RETURNS int AS
$func$
DECLARE
tablename text;
BEGIN
FOREACH tablename IN ARRAY tablenames LOOP
EXECUTE FORMAT('delete * from %s', tablename);
END LOOP;
RETURN 1;
END
$func$ LANGUAGE plpgsql;
You can then call the function for several tables at once, not only for one.
SELECT test( '{rajeev1, rajeev2}' );
If you do not need this feature, simply change the argument type to text.
CREATE OR REPLACE FUNCTION test (tablename text) RETURNS int AS
$func$
BEGIN
EXECUTE format('delete * from %s', tablename);
RETURN 1;
END
$func$ LANGUAGE plpgsql;
SELECT test('rajeev1');
I recommend using the format function.
If you want to execute a function (say purge_this_one_table(tablename)) on a group of tables identified by similar names you can use this construction:
create or replace function purge_all_these_tables(mask text)
returns void language plpgsql
as $$
declare
tabname text;
begin
for tabname in
select relname
from pg_class
where relkind = 'r' and relname like mask
loop
execute format(
'purge_this_one_table(%s)',
tabname);
end loop;
end $$;
select purge_all_these_tables('agg_weekly_%');

It should be:
select test('{rajeev1}');

Related

Update record of a cursor where the table name is a parameter

I am adjusting some PL/pgSQL code so my refcursor can take the table name as parameter. Therefore I changed the following line:
declare
pointCurs CURSOR FOR SELECT * from tableName for update;
with this one:
OPEN pointCurs FOR execute 'SELECT * FROM ' || quote_ident(tableName) for update;
I adjusted the loop, and voilĂ , the loop went through. Now at some point in the loop I needed to update the record (pointed by the cursor) and I got stuck. How should I properly adjust the following line of code?
UPDATE tableName set tp_id = pos where current of pointCurs;
I fixed the quotes for the tableName and pos and added the EXECUTE clause at the beginning, but I get the error on the where current of pointCurs.
Questions:
How can I update the record?
The function was working properly for tables from the public schema and failed for tables from other schemas (e.g., trace.myname).
Any comments are highly appreciated..
Answer for (i)
1. Explicit (unbound) cursor
EXECUTE is not a "clause", but a PL/pgSQL command to execute SQL strings. Cursors are not visible inside the command. You need to pass values to it.
Hence, you cannot use the special syntax WHERE CURRENT OFcursor. I use the system column ctid instead to determine the row without knowing the name of a unique column. Note that ctid is only guaranteed to be stable within the same transaction.
CREATE OR REPLACE FUNCTION f_curs1(_tbl text)
RETURNS void AS
$func$
DECLARE
_curs refcursor;
rec record;
BEGIN
OPEN _curs FOR EXECUTE 'SELECT * FROM ' || quote_ident(_tbl) FOR UPDATE;
LOOP
FETCH NEXT FROM _curs INTO rec;
EXIT WHEN rec IS NULL;
RAISE NOTICE '%', rec.tbl_id;
EXECUTE format('UPDATE %I SET tbl_id = tbl_id + 10 WHERE ctid = $1', _tbl)
USING rec.ctid;
END LOOP;
END
$func$ LANGUAGE plpgsql;
Why format() with %I?
There is also a variant of the FOR statement to loop through cursors, but it only works for bound cursors. We have to use an unbound cursor here.
2. Implicit cursor in FOR loop
There is normally no need for explicit cursors in plpgsql. Use the implicit cursor of a FOR loop instead:
CREATE OR REPLACE FUNCTION f_curs2(_tbl text)
RETURNS void AS
$func$
DECLARE
_ctid tid;
BEGIN
FOR _ctid IN EXECUTE 'SELECT ctid FROM ' || quote_ident(_tbl) FOR UPDATE
LOOP
EXECUTE format('UPDATE %I SET tbl_id = tbl_id + 100 WHERE ctid = $1', _tbl)
USING _ctid;
END LOOP;
END
$func$ LANGUAGE plpgsql;
3. Set based approach
Or better, yet (if possible!): Rethink your problem in terms of set-based operations and execute a single (dynamic) SQL command:
-- Set-base dynamic SQL
CREATE OR REPLACE FUNCTION f_nocurs(_tbl text)
RETURNS void AS
$func$
BEGIN
EXECUTE format('UPDATE %I SET tbl_id = tbl_id + 1000', _tbl);
-- add WHERE clause as needed
END
$func$ LANGUAGE plpgsql;
SQL Fiddle demonstrating all 3 variants.
Answer for (ii)
A schema-qualified table name like trace.myname actually consists of two identifiers. You have to
either pass and escape them separately,
or go with the more elegant approach of using a regclass type:
CREATE OR REPLACE FUNCTION f_nocurs(_tbl regclass)
RETURNS void AS
$func$
BEGIN
EXECUTE format('UPDATE %s SET tbl_id = tbl_id + 1000', _tbl);
END
$func$ LANGUAGE plpgsql;
I switched from %I to %s, because the regclass parameter is automatically properly escaped when (automatically) converted to text.
More details in this related answer:
Table name as a PostgreSQL function parameter

Create a function to get column from multiple tables in PostgreSQL

I'm trying to create a function to get a field value from multiple tables in my database. I made script like this:
CREATE OR REPLACE FUNCTION get_all_changes() RETURNS SETOF RECORD AS
$$
DECLARE
tblname VARCHAR;
tblrow RECORD;
row RECORD;
BEGIN
FOR tblrow IN SELECT tablename FROM pg_catalog.pg_tables WHERE schemaname='public' LOOP /*FOREACH tblname IN ARRAY $1 LOOP*/
RAISE NOTICE 'r: %', tblrow.tablename;
FOR row IN SELECT MAX("lastUpdate") FROM tblrow.tablename LOOP
RETURN NEXT row;
END LOOP;
END LOOP;
END
$$
LANGUAGE 'plpgsql' ;
SELECT get_all_changes();
But it is not working, everytime it shows this error
tblrow.tablename" not defined in line "FOR row IN SELECT MAX("lastUpdate") FROM tblrow.tablename LOOP"
Your inner FOR loop must use the FOR...EXECUTE syntax as shown in the manual:
FOR target IN EXECUTE text_expression [ USING expression [, ... ] ] LOOP
statements
END LOOP [ label ];
In your case something along this line:
FOR row IN EXECUTE 'SELECT MAX("lastUpdate") FROM ' || quote_ident(tblrow.tablename) LOOP
RETURN NEXT row;
END LOOP
The reason for this is explained in the manual somewhere else:
Oftentimes you will want to generate dynamic commands inside your PL/pgSQL functions, that is, commands that will involve different tables or different data types each time they are executed. PL/pgSQL's normal attempts to cache plans for commands (as discussed in Section 39.10.2) will not work in such scenarios. To handle this sort of problem, the EXECUTE statement is provided[...]
Answer to your new question (mislabeled as answer):
This can be much simpler. You do not need to create a table just do define a record type.
If at all, you would better create a type with CREATE TYPE, but that's only efficient if you need the type in multiple places. For just a single function, you can use RETURNS TABLE instead :
CREATE OR REPLACE FUNCTION get_all_changes(text[])
RETURNS TABLE (tablename text
,"lastUpdate" timestamp with time zone
,nums integer) AS
$func$
DECLARE
tblname text;
BEGIN
FOREACH tblname IN ARRAY $1 LOOP
RETURN QUERY EXECUTE format(
$f$SELECT '%I', MAX("lastUpdate"), COUNT(*)::int FROM %1$I
$f$, tblname)
END LOOP;
END
$func$ LANGUAGE plpgsql;
A couple more points:
Use RETURN QUERY EXECUTE instead of the nested loop. Much simpler and faster.
Column aliases would only serve as documentation, those names are discarded in favor of the names declared in the RETURNS clause (directly or indirectly).
Use format() with %I to replace the concatenation with quote_ident() and %1$I to refer to the same parameter another time.
count() usually returns type bigint. Cast the integer, since you defined the column in the return type as such: count(*)::int.
Thanks,
I finally made my script like:
CREATE TABLE IF NOT EXISTS __rsdb_changes (tablename text,"lastUpdate" timestamp with time zone, nums bigint);
CREATE OR REPLACE FUNCTION get_all_changes(varchar[]) RETURNS SETOF __rsdb_changes AS /*TABLE (tablename varchar(40),"lastUpdate" timestamp with time zone, nums integer)*/
$$
DECLARE
tblname VARCHAR;
tblrow RECORD;
row RECORD;
BEGIN
FOREACH tblname IN ARRAY $1 LOOP
/*RAISE NOTICE 'r: %', tblrow.tablename;*/
FOR row IN EXECUTE 'SELECT CONCAT('''|| quote_ident(tblname) ||''') AS tablename, MAX("lastUpdate") AS "lastUpdate",COUNT(*) AS nums FROM ' || quote_ident(tblname) LOOP
/*RAISE NOTICE 'row.tablename: %',row.tablename;*/
/*RAISE NOTICE 'row.lastUpdate: %',row."lastUpdate";*/
/*RAISE NOTICE 'row.nums: %',row.nums;*/
RETURN NEXT row;
END LOOP;
END LOOP;
RETURN;
END
$$
LANGUAGE 'plpgsql' ;
Well, it works. But it seems I can only create a table to define the return structure instead of just RETURNS SETOF RECORD. Am I right?
Thanks again.

How to use EXECUTE FORMAT ... USING in postgres function

CREATE OR REPLACE FUNCTION dummytest_insert_trigger()
RETURNS trigger AS
$BODY$
DECLARE
v_partition_name VARCHAR(32);
BEGIN
IF NEW.datetime IS NOT NULL THEN
v_partition_name := 'dummyTest';
EXECUTE format('INSERT INTO %I VALUES ($1,$2)',v_partition_name)using NEW.id,NEW.datetime;
END IF;
RETURN NULL;
END;
$BODY$
LANGUAGE plpgsql VOLATILE
COST 100;
ALTER FUNCTION dummytest_insert_trigger()
OWNER TO postgres;
I'm trying to insert using
insert into dummyTest values(1,'2013-01-01 00:00:00+05:30');
But it's showing error as
ERROR: function format(unknown) does not exist
SQL state: 42883
Hint: No function matches the given name and argument types. You might need to add explicit type casts.
Context: PL/pgSQL function "dummytest_insert_trigger" line 8 at EXECUTE statement
I'm unable get the error.
Your function could look like this in Postgres 9.0 or later:
CREATE OR REPLACE FUNCTION dummytest_insert_trigger()
RETURNS trigger AS
$func$
DECLARE
v_partition_name text := quote_ident('dummyTest'); -- assign at declaration
BEGIN
IF NEW.datetime IS NOT NULL THEN
EXECUTE
'INSERT INTO ' || v_partition_name || ' VALUES ($1,$2)'
USING NEW.id, NEW.datetime;
END IF;
RETURN NULL; -- You sure about this?
END
$func$ LANGUAGE plpgsql;
About RETURN NULL:
To ignore result in BEFORE TRIGGER of PostgreSQL?
I would advice not to use mixed case identifiers. With format( .. %I ..) or quote_ident(), you'd get a table named "dummyTest", which you'll have to double quote for the rest of its existence. Related:
Are PostgreSQL column names case-sensitive?
Use lower case instead:
quote_ident('dummytest')
There is really no point in using dynamic SQL with EXECUTE as long as you have a static table name. But that's probably just the simplified example?
You need explicit cast to text:
EXECUTE format('INSERT INTO %I VALUES ($1,$2)'::text ,v_partition_name) using NEW.id,NEW.datetime;

Postgres how to evaluate expression from query to variables in the function

I would like to be able to get values of function variables whose names are queried from a table
Edited to show querying a table instead of query from static values:
create table __test__
(
_col text
);
insert into __test__
(_col)
values('_a');
create or replace function __test()
returns void
language 'plpgsql' as
$$
declare
_r record;
_a int;
_b int;
_sql text;
begin
_a = 1;
_b = 0;
for _r in select _col as _nam from __test__ a loop
-- query returns one row valued "_a"
_sql = 'select ' || _r._nam ;
execute _sql into _b;
end loop;
raise info 'value of _b %', _b;
end;
$$;
select __test()
when function executes so that _b = 1. Is it possible?
same error ...
ERROR: column "_a" does not exist
LINE 1: select _a
^
QUERY: select _a
CONTEXT: PL/pgSQL function "__test" line 15 at EXECUTE statement
You could create a temporary table, insert your variable names and values in it, and then execute a select against that. Just clean up after. I have used approaches like that before. It works ok. It does have extra overhead though.
Edit: adding an example
CREATE FUNCTION switch (in_var text) RETURNS text
LANGUAGE PLPGSQL VOLATILE AS $$
declare t_test text;
switch_vals text[];
BEGIN
CREATE TEMPORARY TABLE switch_values (var text, value text);
EXECUTE $e$ INSERT INTO switch_values VALUES
('a', '1'), ('b', '2'), ('c', '3') $e$;
EXECUTE $e$ SELECT value FROM switch_values WHERE var = $e$ || quote_literal(in_var)
INTO t_test;
DROP TABLE switch_values;
RETURN t_test;
END; $$;
postgres=# select switch('a');
switch
--------
1
(1 row)
Let's try to reframe the question: what you're after would be the equivalent of Perl eval()
function, with its ability to execute a dynamically generated piece of code for which "any outer lexical variables are visible to it". In your example, the variable would be _a, but as you can see from the error message, it can't be interpolated by a dynamic SQL statement. The reason is that the SQL interpreter has no visibility on the current pl/pgsql variables, or even the knowledge that such variables exist. They are confined to pl/pgsql.
What would be needed here is a context-aware dynamically-generated pl/pgsql statement, but this language does not have this feature. It's doubtful that a trick could be found to achieve the result without this feature. For all its ability to interface nicely with SQL, other than that it's a fairly static language.
On the other hand, this would be no problem for pl/perl.

I want to have my pl/pgsql script output to the screen

I have the following script that I want output to the screen from.
CREATE OR REPLACE FUNCTION randomnametest() RETURNS integer AS $$
DECLARE
rec RECORD;
BEGIN
FOR rec IN SELECT * FROM my_table LOOP
SELECT levenshtein('mystring',lower('rec.Name')) ORDER BY levenshtein;
END LOOP;
RETURN 1;
END;
$$ LANGUAGE plpgsql;
I want to get the output of the levenshein() function in a table along with the rec.Name. How would I do that? Also, it is giving me an error about the line where I call levenshtein(), saying that I should use perform instead.
Assuming that you want to insert the function's return value and the rec.name into a different table. Here is what you can do (create the table new_tab first)-
SELECT levenshtein('mystring',lower(rec.Name)) AS L_val;
INSERT INTO new_tab (L_val, rec.name);
The usage above is demonstrated below.
I guess, you can use RAISE INFO 'This is %', rec.name; to view the values.
CREATE OR REPLACE FUNCTION randomnametest() RETURNS integer AS $$
DECLARE
rec RECORD;
BEGIN
FOR rec IN SELECT * FROM my_table LOOP
SELECT levenshtein('mystring',lower(rec.Name))
AS L_val;
RAISE INFO '% - %', L_val, rec.name;
END LOOP;
RETURN 1;
END;
$$ LANGUAGE plpgsql;
Note- the FROM clause is optional in case you select from a function in a select like netxval(sequence_name) and don't have any actual table to select from i.e. like SELECT nextval(sequence_name) AS next_value;, in Oracle terms it would be SELECT sequence_name.nextval FROM dual; or SELECT function() FROM dual;. There is no dual in postgreSQL.
I also think that the ORDER BY is not necessary since my assumption would be that your function levenshtein() will most likely return only one value at any point of time, and hence wouldn't have enough data to ORDER.
If you want the output from a plpgsql function like the title says:
CREATE OR REPLACE FUNCTION randomnametest(_mystring text)
RETURNS TABLE (l_dist int, name text) AS
$BODY$
BEGIN
RETURN QUERY
SELECT levenshtein(_mystring, lower(t.name)), t.name
FROM my_table t
ORDER BY 1;
END;
$$ LANGUAGE plpgsql;
Declare the table with RETURNS TABLE.
Use RETURN QUERY to return records from the function.
Avoid naming conflicts between column names and OUT parameters (from the RETURNS TABLE clause) by table-qualifying column names in queries. OUT parameters are visible everywhere in the function body.
I made the string to compare to a parameter to the function to make this more useful.
There are other ways, but this is the most effective for the task. You need PostgreSQL 8.4 or later.
For a one-time use I would consider to just use a plain query (= function body without the RETURN QUERY above).