I am tinkering with this snippet of SQL I reappropriated from another SO question:
CREATE OR REPLACE FUNCTION db_to_csv(path TEXT) RETURNS void AS $$
DECLARE
target RECORD;
statement TEXT;
BEGIN
FOR target IN
SELECT DISTINCT table_name
FROM information_schema.columns
WHERE table_schema='public'
AND position('_' in table_name) <> 1
ORDER BY 1
LOOP
statement := 'COPY '
|| target
|| ' TO '''
|| path
|| '/'
|| target
|| '.csv'
||''' DELIMITER '';'' CSV HEADER';
EXECUTE statement;
END LOOP;
return;
end;
$$ LANGUAGE plpgsql;
It very nearly almost works, but the statement it tries to execute looks like:
COPY (Bok_F_Acc) TO '/Users/jgt/dir/(Bok_F_Acc).csv' DELIMIT...
…Whereas I would like the statement to look like:
COPY Bok_F_Acc TO '/Users/jgt/dir/Bok_F_Acc.csv' DELIMIT...
How can I strip those parentheses?
SELECT QUOTE_IDENT(TRIM(BOTH '()' FROM target));
Note that this would remove any number of leading/trailing parentheses.
To remove at most one of each, you could use :
SELECT QUOTE_IDENT(REGEXP_REPLACE(target, E'\^\\(|\\)\$', '', 'g'));
Related
I am a newbie wrt functions and I am struggling with using the name of a table in the function body. I get an error "SQL Error [42703]: ERROR: column "tname" does not exist" when I call the function using
select "JsonToView"('data_import.import_360xero_report');
My code is below
create or replace
function data_import."JsonToView"(tname text) returns numeric
language plpgsql
as $function$
begin
do
$$
declare
l_keys text;
begin
drop view if exists v_json_view cascade;
select
string_agg(distinct format('import_data ->> %L as %I', jkey, jkey), ', ')
into
l_keys
from
import_360xero_report,
json_object_keys(import_data) as t(jkey);
execute 'create view v_json_view as select ' || l_keys || ' from ' || tname;
end;
$$;
return 0;
end $function$ ;
I have modified the code and the second create view query works with the table name but the first one does not.
Below if my modified code
create or replace
function data_import."JsonToView"(tname text) returns numeric
language plpgsql
as $function$
declare
l_keys text;
begin
drop view if exists v_json_view cascade;
execute $a$select
string_agg(distinct format('import_data ->> %L as %I', jkey, jkey), ', ')
into
l_keys
from $a$ ||
tname || $b$,
json_object_keys(import_data) as t(jkey)$b$;
execute 'create view v_json_view as select ' || l_keys || ' from ' || tname;
return 0;
end $function$ ;
The error I am getting is
SQL Error [0A000]: ERROR: EXECUTE of SELECT ... INTO is not implemented
Hint: You might want to use EXECUTE ... INTO or EXECUTE CREATE TABLE ... AS instead.
Where: PL/pgSQL function "JsonToView"(text) line 10 at EXECUTE
The problem is the superfluous nested DO statement.
The variable tname exists only in the scope of the function, not in the nested DO statement. DO is an SQL statement, not a PL/pgSQL statement, and there are no variables in SQL. Also, DO does not allow parameters.
Get rid of the DO and you will be fine.
I am trying to write a plpgsql procedure to perform spatial tiling of a postGIS table. I can perform the operation successfully using the following procedure in which the table names are hardcoded. The procedure loops through the tiles in tile_table and for each tile clips the area_table and inserts it into split_table.
CREATE OR REPLACE PROCEDURE splitbytile()
AS $$
DECLARE
tile RECORD;
BEGIN
FOR tile IN
SELECT tid, geom FROM test_tiles ORDER BY tid
LOOP
INSERT INTO split_table (id, areaname, ttid, geom)
SELECT id, areaname, tile.tid,
CASE WHEN st_within(base.geom, tile.geom) THEN st_multi(base.geom)
ELSE st_multi(st_intersection(base.geom, tile.geom)) END as geom
FROM area_table as base
WHERE st_intersects(base.geom, tile.geom);
COMMIT;
END LOOP;
END;
$$ LANGUAGE 'plpgsql';
Having tested this successfully, now I need to convert it to a dynamic procedure where I can provide the table names as parameters. I tried the following partial conversion, using format() for inside of loop:
CREATE OR REPLACE PROCEDURE splitbytile(in_table text, grid_table text, split_table text)
AS $$
DECLARE
tile RECORD;
BEGIN
FOR tile IN
EXECUTE format('SELECT tid, geom FROM %I ORDER BY tid', grid_table)
LOOP
EXECUTE
FORMAT(
'INSERT INTO %1$I (id, areaname, ttid, geom)
SELECT id, areaname, tile.tid,
CASE WHEN st_within(base.geom, tile.geom) THEN st_multi(base.geom)
ELSE st_multi(st_intersection(base.geom, tile.geom)) END as geom
FROM %2$I as base
WHERE st_intersects(base.geom, tile.geom)', split_table, in_table
);
COMMIT;
END LOOP;
END;
$$ LANGUAGE 'plpgsql';
But it throws an error
missing FROM-clause entry for table "tile"
So, how can I convert the procedure to a dynamic one? More specifically, how can I use the record data type (tile) returned by the for loop inside the loop? Note that it works when format is not used.
You can use EXECUTE ... USING to supply parameters to a dynamic query:
EXECUTE
format(
'SELECT r FROM %I WHERE c = $1.val',
table_name
)
INTO result_var
USING record_var;
The first argument to USING will be used for $1, the second for $2 and so on.
See the documentation for details.
Personally I use somehow different way to create dynamic functions. By concatination and execute function. You can also do like this.
CREATE OR REPLACE FUNCTION splitbytile()
RETURNS void AS $$
declare
result1 text;
table_name text := 'test_tiles';
msi text := '+7 9912 231';
msi text := 'Hello world';
code text := 'code_name';
_operator_id integer := 2;
begin
query1 := 'SELECT msisdn from ' || table_name || ' where msisdn = ''' || msi::text ||''';';
query2 := 'INSERT INTO ' || table_name || '(msisdn,usage,body,pr_code,status,sent_date,code_type,operator_id)
VALUES( ''' || msi::text || ''',' || true || ',''' || _body::text || ''',''' || code::text || ''',' || false || ',''' || time_now || ''',' || kod_type || ',' || _operator_id ||');';
execute query1 into result1;
execute query2;
END;
$function$
You just make your query as text then anywhere you want you can execute it. Maybe by checking result1 value inside If statement or smth like that.
So my problem is simple. I have a schema prod with many tables, and another one log with the exact same tables and structure (primary keys change that's it).
When I do UPDATE or DELETE in the schema prod, I want to record old data in the log schema.
I have the following function called after a update or delete:
CREATE FUNCTION prod.log_data() RETURNS trigger
LANGUAGE plpgsql AS $$
DECLARE
v RECORD;
column_names text;
value_names text;
BEGIN
-- get column names of current table and store the list in a text var
column_names = '';
value_names = '';
FOR v IN SELECT * FROM information_schema.columns WHERE table_name = quote_ident(TG_TABLE_NAME) AND table_schema = quote_ident(TG_TABLE_SCHEMA) LOOP
column_names = column_names || ',' || v.column_name;
value_names = value_names || ',$1.' || v.column_name;
END LOOP;
-- remove first char ','
column_names = substring( column_names FROM 2);
value_names = substring( value_names FROM 2);
-- execute the insert into log schema
EXECUTE 'INSERT INTO log.' || TG_TABLE_NAME || ' ( ' || column_names || ' ) VALUES ( ' || value_names || ' )' USING OLD;
RETURN NULL; -- no need to return, it is executed after update
END;$$;
The annoying part is that I have to get column names from information_schema for each row.
I would rather use this:
EXECUTE 'INSERT INTO log.' || TG_TABLE_NAME || ' SELECT ' || OLD;
But some values can be NULL so this will execute:
INSERT INTO log.user SELECT 2,,,"2015-10-28 13:52:44.785947"
instead of
INSERT INTO log.user SELECT 2,NULL,NULL,"2015-10-28 13:52:44.785947"
Any idea to convert ",," to ",NULL,"?
Thanks
-Quentin
First of all I must say that in my opinion using PostgreSQL system tables (like information_schema) is the proper way for such a usecase. Especially that you must write it once: you create the function prod.log_data() and your done. Moreover it may be dangerous to use OLD in that context (just like *) as always because of not specified elements order.
But,
to answer your exact question the only way I know is to do some operations on OLD. Just observe that you cast OLD to text by doing concatenation ... ' SELECT ' || OLD. The default casting create that ugly double-commas. So, next you can play with that text. In the end I propose:
DECLARE
tmp TEXT
...
BEGIN
...
/*to make OLD -> text like (2,,3,4,,)*/
SELECT '' || OLD INTO tmp; /*step 1*/
/*take care of commas at the begining and end: '(,' ',)'*/
tmp := replace(replace(tmp, '(,', '(NULL,'), ',)', ',NULL)'); /*step 2*/
/* replace rest of commas to commas with NULL between them */
SELECT array_to_string(string_to_array(tmp, ',', ''), ',', 'NULL') INTO tmp; /*step 3*/
/* Now we can do EXECUTE*/
EXECUTE 'INSERT INTO log.' || TG_TABLE_NAME || ' SELECT ' || tmp;
Of course you can do steps 1-3 in one big step
SELECT array_to_string(string_to_array(replace(replace('' || NEW, '(,', '(NULL,'), ',)', ',NULL)'), ',', ''), ',', 'NULL') INTO tmp;
In my opinion this approach isn't any better from using information_schema, but it's your call.
Can any one of you tell me how to approach this:
CREATE OR REPLACE FUNCTION name()
RETURNS ????? AS
$func$
BEGIN
SELECT * FROM tbl_a a;
SELECT * FROM tbl_b b;
END
$func$ LANGUAGE plpgsql;
Both tables have different structures.
You can use cursors but I can hardly imagine why you need such a function.
CREATE OR REPLACE FUNCTION my_multiselect(refcursor, refcursor) RETURNS VOID AS
$func$
BEGIN
OPEN $1 FOR SELECT * FROM information_schema.routines;
OPEN $2 FOR SELECT * FROM information_schema.sequences;
END
$func$ LANGUAGE plpgsql;
BEGIN;
SELECT my_multiselect('first_cursor_to_routines', 'second_cursor_to_sequences');
FETCH ALL IN first_cursor_to_routines;
FETCH ALL IN second_cursor_to_sequences;
COMMIT;
I'm not really sure what you're doing with this, but it sounds like you just want to return a union of these distinct result sets. You can do this with a dynamic query. I'm using Postgres 9.4.
CREATE OR REPLACE FUNCTION make_query(IN p_tables text[])
RETURNS void AS
$BODY$
DECLARE
v_qry text;
v_cols text;
v_types text;
v_as text;
BEGIN
EXECUTE format('
WITH sub AS (
SELECT
table_name,
column_name,
data_type
FROM
information_schema.columns
WHERE
table_name = ANY(%L)
ORDER BY
table_name,
ordinal_position)
,sub2 AS(
SELECT
DISTINCT ON (column_name, data_type)
column_name || '' '' || data_type AS def
FROM
sub
)
SELECT
string_agg(def, '','')
FROM
sub2;
',
p_tables
) INTO v_types;
v_qry := '
CREATE OR REPLACE FUNCTION name()
RETURNS TABLE(' || v_types || ') AS
$func$';
FOR i IN 1..array_upper(p_tables, 1)
LOOP
v_as := 'tbl' || i;
EXECUTE format('
WITH sub AS (
SELECT
table_name,
column_name,
data_type
FROM
information_schema.columns
WHERE
table_name = ANY(%L)
ORDER BY
table_name,
ordinal_position)
,sub2 AS(
SELECT
DISTINCT ON (column_name, data_type)
CASE WHEN table_name = ''%I''
THEN %L || ''.'' || column_name
ELSE ''NULL::'' || data_type
END AS cols
FROM
sub
)
SELECT
string_agg(cols, '','')
FROM
sub2;
',
p_tables,
p_tables[i],
v_as
) INTO v_cols;
IF i > 1 THEN
v_qry := v_qry || '
UNION ALL';
END IF;
v_qry := v_qry || '
SELECT ' || v_cols || ' FROM ' || p_tables[i] || ' AS ' || v_as;
IF i = array_upper(p_tables, 1) THEN
v_qry := v_qry || ';';
END IF;
END LOOP;
v_qry := v_qry || '
$func$ LANGUAGE sql;
';
EXECUTE v_qry;
END;
$BODY$
LANGUAGE plpgsql VOLATILE;
Sorry it looks ugly here, but this formatting helps the final product look nicer. If you're shy about executing a dynamic query like this off the bat, just replace EXECUTE v_qry; with RAISE INFO 'v_qry: %', v_qry; and it will simply print the dynamic query out in a message without executing it, so you can review what it will do once executed.
Then execute make_query() with a list of tables you want to display like this:
SELECT make_query(ARRAY['tbl_a', 'tbl_b']);
The result is that you will now have a function called name() which you can call in order to see the results of both tables at the same time, with all the union details already sorted out:
SELECT * FROM name();
I have a database with multiple identical schemas. There is a number of tables all named 'tran_...' in each schema. I want to loop through all 'tran_' tables in all schemas and pull out records that fall within a specific date range. This is the code I have so far:
CREATE OR REPLACE FUNCTION public."configChanges"(starttime timestamp, endtime timestamp)
RETURNS SETOF character varying AS
$BODY$DECLARE
tbl_row RECORD;
tbl_name VARCHAR(50);
tran_row RECORD;
out_record VARCHAR(200);
BEGIN
FOR tbl_row IN
SELECT * FROM pg_tables WHERE schemaname LIKE 'ivr%' AND tablename LIKE 'tran_%'
LOOP
tbl_name := tbl_row.schemaname || '.' || tbl_row.tablename;
FOR tran_row IN
SELECT * FROM tbl_name
WHERE ch_edit_date >= starttime AND ch_edit_date <= endtime
LOOP
out_record := tbl_name || ' ' || tran_row.ch_field_name;
RETURN NEXT out_record;
END LOOP;
END LOOP;
RETURN;
END;
$BODY$
LANGUAGE plpgsql;
When I attempt to run this, I get:
ERROR: relation "tbl_name" does not exist
LINE 1: SELECT * FROM tbl_name WHERE ch_edit_date >= starttime AND c...
#Pavel already provided a fix for your basic error.
However, since your tbl_name is actually schema-qualified (two separate identifiers in : schema.table), it cannot be escaped as a whole with %I in format(). You have to escape each identifier individually.
Aside from that, I suggest a different approach. The outer loop is necessary, but the inner loop can be replaced with a simpler and more efficient set-based approach:
CREATE OR REPLACE FUNCTION public.config_changes(_start timestamp, _end timestamp)
RETURNS SETOF text AS
$func$
DECLARE
_tbl text;
BEGIN
FOR _tbl IN
SELECT quote_ident(schemaname) || '.' || quote_ident(tablename)
FROM pg_tables
WHERE schemaname LIKE 'ivr%'
AND tablename LIKE 'tran_%'
LOOP
RETURN QUERY EXECUTE format (
$$
SELECT %1$L || ' ' || ch_field_name
FROM %1$s
WHERE ch_edit_date BETWEEN $1 AND $2
$$, _tbl
)
USING _start, _end;
END LOOP;
RETURN;
END
$func$ LANGUAGE plpgsql;
You have to use dynamic SQL to parametrize identifiers (or code), like #Pavel already told you. With RETURN QUERY EXECUTE you can return the result of a dynamic query directly. Examples:
Return SETOF rows from PostgreSQL function
Refactor a PL/pgSQL function to return the output of various SELECT queries
Remember that identifiers have to be treated as unsafe user input in dynamic SQL and must always be sanitized to avoid syntax errors and SQL injection:
Table name as a PostgreSQL function parameter
Note how I escape table and schema separately:
quote_ident(schemaname) || '.' || quote_ident(tablename)
Consequently I just use %s to insert the already escaped table name in the later query. And %L to escape it a string literal for output.
I like to prepend parameter and variable names with _ to avoid naming conflicts with column names. No other special meaning.
There is a slight difference compared to your original function. This one returns an escaped identifier (double-quoted only where necessary) as table name, e.g.:
"WeIRD name"
instead of
WeIRD name
Much simpler yet
If possible, use inheritance to obviate the need for above function altogether. Complete example:
Select (retrieve) all records from multiple schemas using Postgres
You cannot use a plpgsql variable as SQL table name or SQL column name. In this case you have to use dynamic SQL:
FOR tran_row IN
EXECUTE format('SELECT * FROM %I
WHERE ch_edit_date >= starttime AND ch_edit_date <= endtime', tbl_name)
LOOP
out_record := tbl_name || ' ' || tran_row.ch_field_name;
RETURN NEXT out_record;
END LOOP;