Data fetching from a View in Postgresql Using Comma Separated Input - postgresql

Created a view , my input will be in a comma separated , input contains only the column names of the view .
My input is '"Name","Description"' like this .
I tried like this
Select unnest(string_to_array('"Name","Description"',','))
From my_test_view
Unfortunately i got result like this
"Name"
"Description"
"Name"
"Description"
this only selecting my column names . I need json output with column values . my input is dynamic

Effectively, what you are trying to do is to dynamically use identifiers on a view for selecting data. You need to write a PL/pgSQL function to do that.
If you trust the input data (i.e. you are certain that the input only contains comma-separated column names) then the solution is very straightforward:
CREATE FUNCTION my_test_view_dynamic(columns text) RETURNS SET OF my_test_view AS $$
BEGIN
RETURN QUERY EXECUTE format('SELECT %s FROM my_test_view', columns);
END;
$$ LANGUAGE plpgsql STABLE STRICT;
If you are not so sure about the sanity of the input data, check it first:
CREATE FUNCTION my_test_view_dynamic(columns text) RETURNS SET OF my_test_view AS $$
DECLARE
c text;
safe_names text[] := '{}'::text[];
BEGIN
FOREACH c IN ARRAY regexp_split_to_array(columns, ',') LOOP
safe_names := safe_names || quote_identifier(c);
END LOOP;
RETURN QUERY EXECUTE format('SELECT %s FROM my_test_view', concat_ws(safe_names, ','));
END;
$$ LANGUAGE plpgsql STABLE STRICT;
Then call like:
SELECT * FROM my_text_view_dynamic('"Name","Description"');

Related

postgres plpgsql how to properly convert function to use FORMAT in DECLARE

I am writing a function in POSTGRES v13.3 that when passed an array of column names returns an array of JSONB objects each with the distinct values of one of the columns. I have an existing script that I wish to refactor using FORMAT in the declaration portion of the function.
The existing and working function looks like below. It is passed an array of columns and a dbase name. The a loop presents each column name to an EXECUTE statement that uses JSONB_AGG on the distinct values in the column, creates a JSONB object, and appends that to an array. The array is returned on completion. This is the function:
CREATE OR REPLACE FUNCTION foo1(text[], text)
RETURNS text[] as $$
declare
col text;
interim jsonb;
temp jsonb;
y jsonb[];
begin
foreach col in array $1
loop
execute
'select jsonb_agg(distinct '|| col ||') from ' || $2 into interim;
temp := jsonb_build_object(col, interim);
y := array_append(y,temp);
end loop;
return y;
end;
$$ LANGUAGE plpgsql;
I have refactored the function to the following. The script is now in the DECLARE portion of the function.
CREATE OR REPLACE FUNCTION foo2(_cols text[], _db text)
RETURNS jsonb[]
LANGUAGE plpgsql as
$func$
DECLARE
_script text := format(
'select jsonb_agg( distinct $1) from %1$I', _db
);
col text;
interim jsonb;
temp jsonb;
y jsonb[];
BEGIN
foreach col in array _cols
loop
EXECUTE _script USING col INTO interim;
temp := jsonb_build_object(col, interim);
y := array_append(y,temp);
end loop;
return y;
END
$func$;
Unfortunately the two functions give different results on a toy data set (see bottom):
Original: {"{\"id\": [1, 2, 3]}","{\"val\": [1, 2]}"}
Refactored: {{"id": ["id"]},{"val": ["val"]}}
Here is a db<>fiddle of the preceding.
The challenge is in the EXECUTE. In the first instance the col argument is treated as a column identifier. In the refactored function it seems to be treated as just a text string. I think my approach is consistent with the docs and tutorials (example), and the answer from this forum here and the links included therein. I have tried playing around with combinations of ", ', and || but those were unsuccessful and don't make sense in a format statement.
Where should I be looking for the error in my use of FORMAT?
NOTE 1: From the docs I have so possibly the jsonagg() and distinct are what's preventing the behaviour I want:
Another restriction on parameter symbols is that they only work in SELECT, INSERT, UPDATE, and DELETE commands. In other statement types (generically called utility statements), you must insert values textually even if they are just data values.
TOY DATA SET:
drop table if exists example;
create temporary table example(id int, str text, val integer);
insert into example values
(1, 'a', 1),
(2, 'a', 2),
(3, 'b', 2);
https://www.postgresql.org/docs/14/plpgsql-statements.html#PLPGSQL-STATEMENTS-SQL-ONEROW
The command string can use parameter values, which are referenced in
the command as $1, $2, etc. These symbols refer to values supplied in
the USING clause.
What you want is paramterize sql identifier(column name).
You cannot do that. Access column using variable instead of explicit column name
Which means that select jsonb_agg( distinct $1) from %1$I In here "$1" must be %I type. USING Expression in the manual (EXECUTE command-string [ INTO [STRICT] target ] [ USING expression [, ... ] ];) will pass the literal value to it. But it's valid because select distinct 'hello world' from validtable is valid.
select jsonb_agg( distinct $1) from %1$I In here $1 must be same type as %1$I namely-> sql identifier.
--
Based on the following debug code, then you can solve your problem:
CREATE OR REPLACE FUNCTION foo2(_cols text[], _db text)
RETURNS void
LANGUAGE plpgsql as
$func$
DECLARE
col text;
interim jsonb;temp jsonb; y jsonb[];
BEGIN
foreach col in array _cols
loop
EXECUTE format( 'select jsonb_agg( distinct ( %1I ) ) from %2I', col,_db) INTO interim;
raise info 'interim: %', interim;
temp := jsonb_build_object(col, interim);
raise info 'temp: %', temp;
y := array_append(y,temp);
raise info 'y: %',y;
end loop;
END
$func$;

Postgres split and convert string value to composite array

I have a Type with below structure.
create type t_attr as (id character varying(50), data character varying(100));
I get text/varchar value through a user-defined function. The value looks like below.
txt := 'id1#data1' , 'id2#data2';
(In the above example, there are 2 values separated by comma, but the count will vary).
I'm trying to store each set into the t_attr array using the below code. Since each set will contain a # as separator I use it to split the id and data values. For testing purpose I have used only one set below.
DO
$$
DECLARE
attr_array t_attr[];
txt text := 'id1#data1';
BEGIN
attr_array[1] := regexp_split_to_array(txt, '#');
END;
$$
LANGUAGE plpgsql;
But the above code throws error saying 'malformed record literal' and 'missing left paranthesis'.
Can someone please help on how to store this data into array? Thanks.
It is because you are trying to assign an array value to the record. Try:
do $$
declare
attr_array t_attr[];
txt text := 'id1#data1';
begin
attr_array[1] := (split_part(txt, '#', 1), split_part(txt, '#', 2));
raise info '%', attr_array;
end $$;
Output:
INFO: {"(id1,data1)"}
DO
However if you really need to split values using arrays:
do $$
declare
a text[];
begin
....
a := regexp_split_to_array(txt, '#');
attr_array[1] := (a[1], a[2]);
end $$;

How to get the value of a dynamically generated field name in PL/pgSQL

Sample code trimmed down the the bare essentials to demonstrate question:
CREATE OR REPLACE FUNCTION mytest4() RETURNS TEXT AS $$
DECLARE
wc_row wc_files%ROWTYPE;
fieldName TEXT;
BEGIN
SELECT * INTO wc_row FROM wc_files WHERE "fileNumber" = 17117;
-- RETURN wc_row."fileTitle"; -- This works. I get the contents of the field.
fieldName := 'fileTitle';
-- RETURN format('wc_row.%I',fieldName); -- This returns 'wc_row."fileTitle"'
-- but I need the value of it instead.
RETURN EXECUTE format('wc_row.%I',fieldName); -- This gives a syntax error.
END;
$$ LANGUAGE plpgsql;
How can I get the value of a dynamically generated field name in this situation?
Use a trick with the function to_json(), which for a composite type returns a json object with column names as keys:
create or replace function mytest4()
returns text as $$
declare
wc_row wc_files;
fieldname text;
begin
select * into wc_row from wc_files where "filenumber" = 17117;
fieldname := 'filetitle';
return to_json(wc_row)->>fieldname;
end;
$$ language plpgsql;
You don't need tricks. EXECUTE does what you need, you were on the right track already. But RETURN EXECUTE ... is not legal syntax.
CREATE OR REPLACE FUNCTION mytest4(OUT my_col text) AS
$func$
DECLARE
field_name text := 'fileTitle';
BEGIN
EXECUTE format('SELECT %I FROM wc_files WHERE "fileNumber" = 17117', field_name)
INTO my_col; -- data type coerced to text automatically.
END
$func$ LANGUAGE plpgsql;
Since you only want to return a scalar value use EXECUTE .. INTO ... - optionally you can assign to the OUT parameter directly.
RETURN QUERY EXECUTE .. is for returning a set of values.
Use format() to conveniently escape identifiers and avoid SQL injection. Provide identifiers names case sensitive! filetitle is not the same as fileTitle in this context.
Are PostgreSQL column names case-sensitive?
Use an OUT parameter to simplify your code.

Postgres function - using table mapping to update value

I have a general question. I have a function that creates a file. However, within that function presently I am hard coding the file name pattern based on argument inputs. Now I have come to a point where I need to have more than one file name pattern. I devised a way of using another table as the file name map that the function can simply call if the user inputs that file name pattern id. Here is my example to help better illustrate my point:
Here is the table creation and data insertion for referential purposes:
create table some_schema.file_mapper(
mapper_id integer not null,
file_name_template varchar);
insert into some_schema.file_mapper (mapper_id, file_name_template)
values
(1, '||v_first_name||''_''||v_last_name||')
(2, '||v_last_name||''_''||v_first_name||')
(3, '||v_last_name||''_''||v_last_name||');
Now the function itself
create or replace function some_schema.some_function(integer)
returns varchar as
$body$
Declare
v_filename_config_id alias for $1;
v_filename varchar;
v_first_name varchar;
v_last_name varchar;
cmd text;
Begin
v_first_name :='Joe';
v_last_name :='Shmoe';
cmd := 'select file_name_template
from some_schema.file_mapper
where mapper_id = '||v_filename_config||'';
execute cmd into v_filename;
raise notice 'checking "%"',v_filename;
return 'done';
end;
$body$
LANGUAGE plpgsql;
Now that I have this. I want to be able to mix and match file name patterns. So for instance I wanted to use mapper_id 3, I would expect a returned file of "Shmoe_Shmoe.csv" if I execute the script:
select from some_schema.some_function(2)
The Problem is whenever I get it to read the "v_filename" variable it will not evaluate and return the values from the function's variables. Originally, I believed it to be a quoting issue(and it probably still is). After messing with the quoting I have gotten about as far the error below:
ERROR: syntax error at or near "_"
LINE 4: ...s/dir/someplace/||v_last_name||'_'||v_firs...
^
QUERY: copy(
select *
from some_schema.some_table)
to '/dir/someplace/||v_last_name||'_'||v_first_name||.csv/;
DELIMITER,
csv HEADER;
CONTEXT: PL/pgSQL function some_schema.some_function(integer) line 27 at EXECUTE statement
As you can tell it is pretty much telling me it is a quoting issue. Is there a way I can get the function to properly evaluate the variable and return the proper file name? Let me know if I am not clear and need to elaborate.
This more or less illustrates the usage of format(). (Slightly reduced wrt the original question):
CREATE TABLE file_mapper
( mapper_id INTEGER NOT NULL PRIMARY KEY
, file_name_template TEXT
);
INSERT INTO file_mapper(mapper_id, file_name_template) VALUES (1,'one'), (2, 'two');
CREATE OR REPLACE FUNCTION dump_the_shit(INTEGER)
RETURNS VARCHAR AS
$body$
DECLARE
v_filename_config_id alias for $1;
v_filename VARCHAR;
name_cmd TEXT;
copy_cmd TEXT;
BEGIN
name_cmd := format( e'select file_name_template
from file_mapper
where mapper_id = %L;', v_filename_config_id );
EXECUTE name_cmd into v_filename;
RAISE NOTICE 'V_filename := %', v_filename;
copy_cmd := format( e'copy(
select *
from %I)
to \'/tmp/%s.csv\'
csv HEADER;' , 'file_mapper' , v_filename);
EXECUTE copy_cmd;
RETURN copy_cmd;
END;
$body$
LANGUAGE plpgsql;
SELECT dump_the_shit(1);
format function description
summary:
use %I for identifiers (tablenames and column names) [ FROM %I ]
use %L for literals such as query constants [ WHERE the_date = %L ]
use %s for ordinary strings [ to \'/tmp/%s.csv\' ]

Create a function to get column from multiple tables in PostgreSQL

I'm trying to create a function to get a field value from multiple tables in my database. I made script like this:
CREATE OR REPLACE FUNCTION get_all_changes() RETURNS SETOF RECORD AS
$$
DECLARE
tblname VARCHAR;
tblrow RECORD;
row RECORD;
BEGIN
FOR tblrow IN SELECT tablename FROM pg_catalog.pg_tables WHERE schemaname='public' LOOP /*FOREACH tblname IN ARRAY $1 LOOP*/
RAISE NOTICE 'r: %', tblrow.tablename;
FOR row IN SELECT MAX("lastUpdate") FROM tblrow.tablename LOOP
RETURN NEXT row;
END LOOP;
END LOOP;
END
$$
LANGUAGE 'plpgsql' ;
SELECT get_all_changes();
But it is not working, everytime it shows this error
tblrow.tablename" not defined in line "FOR row IN SELECT MAX("lastUpdate") FROM tblrow.tablename LOOP"
Your inner FOR loop must use the FOR...EXECUTE syntax as shown in the manual:
FOR target IN EXECUTE text_expression [ USING expression [, ... ] ] LOOP
statements
END LOOP [ label ];
In your case something along this line:
FOR row IN EXECUTE 'SELECT MAX("lastUpdate") FROM ' || quote_ident(tblrow.tablename) LOOP
RETURN NEXT row;
END LOOP
The reason for this is explained in the manual somewhere else:
Oftentimes you will want to generate dynamic commands inside your PL/pgSQL functions, that is, commands that will involve different tables or different data types each time they are executed. PL/pgSQL's normal attempts to cache plans for commands (as discussed in Section 39.10.2) will not work in such scenarios. To handle this sort of problem, the EXECUTE statement is provided[...]
Answer to your new question (mislabeled as answer):
This can be much simpler. You do not need to create a table just do define a record type.
If at all, you would better create a type with CREATE TYPE, but that's only efficient if you need the type in multiple places. For just a single function, you can use RETURNS TABLE instead :
CREATE OR REPLACE FUNCTION get_all_changes(text[])
RETURNS TABLE (tablename text
,"lastUpdate" timestamp with time zone
,nums integer) AS
$func$
DECLARE
tblname text;
BEGIN
FOREACH tblname IN ARRAY $1 LOOP
RETURN QUERY EXECUTE format(
$f$SELECT '%I', MAX("lastUpdate"), COUNT(*)::int FROM %1$I
$f$, tblname)
END LOOP;
END
$func$ LANGUAGE plpgsql;
A couple more points:
Use RETURN QUERY EXECUTE instead of the nested loop. Much simpler and faster.
Column aliases would only serve as documentation, those names are discarded in favor of the names declared in the RETURNS clause (directly or indirectly).
Use format() with %I to replace the concatenation with quote_ident() and %1$I to refer to the same parameter another time.
count() usually returns type bigint. Cast the integer, since you defined the column in the return type as such: count(*)::int.
Thanks,
I finally made my script like:
CREATE TABLE IF NOT EXISTS __rsdb_changes (tablename text,"lastUpdate" timestamp with time zone, nums bigint);
CREATE OR REPLACE FUNCTION get_all_changes(varchar[]) RETURNS SETOF __rsdb_changes AS /*TABLE (tablename varchar(40),"lastUpdate" timestamp with time zone, nums integer)*/
$$
DECLARE
tblname VARCHAR;
tblrow RECORD;
row RECORD;
BEGIN
FOREACH tblname IN ARRAY $1 LOOP
/*RAISE NOTICE 'r: %', tblrow.tablename;*/
FOR row IN EXECUTE 'SELECT CONCAT('''|| quote_ident(tblname) ||''') AS tablename, MAX("lastUpdate") AS "lastUpdate",COUNT(*) AS nums FROM ' || quote_ident(tblname) LOOP
/*RAISE NOTICE 'row.tablename: %',row.tablename;*/
/*RAISE NOTICE 'row.lastUpdate: %',row."lastUpdate";*/
/*RAISE NOTICE 'row.nums: %',row.nums;*/
RETURN NEXT row;
END LOOP;
END LOOP;
RETURN;
END
$$
LANGUAGE 'plpgsql' ;
Well, it works. But it seems I can only create a table to define the return structure instead of just RETURNS SETOF RECORD. Am I right?
Thanks again.